I have an SDK for my web service that is distributed as a Python library via PyPI. My library uses requests for communicating with the backend using typical REST-like requests.
I would like my library to be compatible with applications that are hosted on Google App Engine (GAE). According to the GAE documentation on HTTP requests:
To use requests, you'll need to install both requests and
requests-toolbelt using the vendoring instructions.
Once installed, use the requests_toolbelt.adapters.appengine module to
configure requests to use URLFetch:
So I follow the example given there and have this in my library's main module:
if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/'):
import requests_toolbelt.adapters.appengine
requests_toolbelt.adapters.appengine.monkeypatch()
This seems to do the trick when a client application using my library is actually running on an App Engine instance.
However, when the client application is run locally using the development web server (dev_appserver.py), os.getenv('SERVER_SOFTWARE') returns "Development/2.0" and so the monkeypatch is not executed. I subsequently get these errors when trying to issue requests:
ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))
How can I detect that the host application for my library is running either in Google App Engine itself, or inside the development web server? Checking for "Development/2.0" doesn't seem like it would be discriminating enough.
Or, is there a better general pattern to follow when shipping a shared Python library that needs support for "typical" network requests?
Digging through the Google Cloud SDK, it seems that Google's own method for determining if we are running in either the production or development (dev_appserver.py) Google App Engine environment is indeed to look for either of those values for SERVER_SOFTWARE. From apitools/base/py/util.py
def DetectGae():
"""Determine whether or not we're running on GAE.
This is based on:
https://developers.google.com/appengine/docs/python/#The_Environment
Returns:
True iff we're running on GAE.
"""
server_software = os.environ.get('SERVER_SOFTWARE', '')
return (server_software.startswith('Development/') or
server_software.startswith('Google App Engine/'))
Related
I have currently 5 different services in Google App Engine all of them are in FastAPI Python standard environment. When a service gets called they call an authorization service and continue if permissions are valid. Im using firewall rule to disable all incoming requests but allow my computer. When using the firewall rule I cannot call the other service because it return Access Forbidden. I then found something about the requests in Python in GAE that you have to use Google's URLfetch to make calls to other services. But when I use the monkeypatch() function from requests_toolbelt.adapters.appengine I recieve an error in App Engine
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/requests_toolbelt/adapters/appengine.py", line 121, in __init__
self.appengine_manager = gaecontrib.AppEngineManager(
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/urllib3/contrib/appengine.py", line 107, in __init__
raise AppEnginePlatformError(
urllib3.contrib.appengine.AppEnginePlatformError: URLFetch is not available in this environment.
The main reason to restrict the API's is that nobody is able to read the docs from the services.
As mentioned in this docThe Python 3 runtime doesn't need an intermediary service to handle outbound requests. If you want to migrate away from using URL Fetch APIs but still need similar functionality, you should migrate those requests to use a standard Python library such as the requests library.
As discussed in this thread seems you are also facing the similar issue
The presence of requests_toolbelt dependency in the project was the
problem: it somehow forced the requests library to use urllib3, which
requires URLFetch to be present, otherwise it raises an
AppEnginePlatformError. As suggested in the app engine docs
monkey-patching Requests with requests_toolbelt forces the former to
use URLFetch, which is no longer supported by GAE in a Python 3
runtime.
You may resolve this by removing requests_toolbelt from
requirements.txt file
You can also have a look at this stackoverflow thread
I have a Django application deployed on Google App Engine standard environment. I am interested in server side rendering of my JS frontend. Can I use node.js alongside Django on the same GAE? Maybe as microservice?
What you can do is to deploy each of your app as a separate service in App Engine and they will work independently as a microservice. To do so, make sure to set a service name for each of the app.yaml file of your apps:
service: service-name
Afterwards, you can communicate between your services through an HTTP invocation, such as a user request or a RESTful API call. Code in one service can't directly call code in another service.
Refer to this link for additional information about communicating between your services.
I have come across articles that talk about integrating python and Node but I personally haven't done it or seen it done on GAE.
If I were to take a stab, I think you would be looking at something like
Have the python app as a service (say it's available on python_service.myapp.appspot.com
Have your Node.js as your default service available on myapp.appspot.com
Your Nodejs will have a route and when this route is invoked, you make an http request to the python service, wait for a response and then your Nodejs app returns that response.
Our App, https://nocommandline.com is an Electron App (combo of Node.js & Vue.js) If you purchase a license and try to validate it, we make a call server side and our server side is Python based. It's not exactly the same thing you're looking at (since our App is not web-based) but this gives you an idea of what I was trying to describe.
I'm getting my feet wet with GCP and GAE, also nodejs and python and networking (I know).
[+] What I have:
Basically I have some nodejs code that takes in some input and is supposed to then send that input to some python code that will do more stuff to it. My first idea was to deploy the nodejs code via GAE, then host the python code in a python server, then make post requests from the nodejs front-end to the python server backend.
[+] What I would like to be able to do:
just deploy both my nodejs code and my python code in the same project and instance of GAE so that the nodejs is the frontend that people see but so that the python server is also running in the same environment and can just communicate with the nodejs without sending anything online.
[+] What I have read
https://www.netguru.co/blog/use-node-js-backend
Google App Engine - Front and Backend Web Development
and countless other google searches for this type of setup but to no avail.
If anyone can point me in the right direction I would really appreciate it.
You can't have both python and nodejs running in the same instance, but they can run as separate services, each with their own instance(s) inside the same GAE app/project. See Service isolation and maybe Deploying different languages services to the same Application [Google App Engine]
Using post requests can work pretty well, but will likely take some effort to ensure no outside access.
Since you intend to use as frontend the nodejs service you're limited to using only the flexible environment for it, which limits the inter-service communication options - you can't use push queues (properly supported only in the standard environment) which IMHO would be a better/more secure solution than post requests.
Another secure communication option would be for the nodejs service to place the data into the datastore and have the python service pick it up from there - the datastore is shared by all instances/versions/services inside the same GAE app. Also more loosely coupled IMHO - each service can function (at least for a while) without the other being alive (not possible if using the post requests).
Maybe of interest: How to tell if a Google App Engine documentation page applies to the standard or the flexible environment
UPDATE:
Node.JS is currently available in the standard environment as well, so you can use those features, see:
Now, you can deploy your Node.js app to App Engine standard environment
Google App Engine Node.js Standard Environment Documentation
I'm trying to setup a flask app on google app engine that will be something of a frontend management console for google container engine. Google has put out working APIs to spin up a container cluster, but it does not look like they have put out (python) APIs to administer kubernetes. That is, everything needed implement services, pods, RCs etc. seems to be setup to run through bash scripting. This is not compatible with the restrictions of google's app engine.
Is there a commonly accepted solution/package for this? Would it make more sense to abandon appengine in favor for a managed VM (not ideal)?
Thanks
As I mentioned in Submit jobs using API Client Library for Python?, the Kubernetes API uses a standard swagger specification, so it should be possible to generate a python client library. There is also pykube if you want to experiment with a existing client library.
I have some entities added on datastore - made a little web app (in Python) to read and write from the datastore through endpoints. I am able to use the webapp through endpoints from javascript.
I want to access the web app from an installed application on PC. Is there a way to access endpoints from installed applications written in Python? How?
Is there any other way to access the datastore from PC installed applications written in Python?
That's one of the beauties of AppEngine's endpoints. You should use the Python Client Library for Google's APIs to communicate with your endpoints
pip install --upgrade google-api-python-client
Then you'll construct a resource object to communicate with your api using the apiclient.discovery.build function. For eg:
from apiclient.discovery import build
api_root = 'https://<APP_ID>.appspot.com/_ah/api'
api = 'api_name'
version = 'api_version'
discovery_url = '%s/discovery/v1/apis/%s/%s/rest' % (api_root, api, version)
service = build(api, version, discoveryServiceUrl=discovery_url)
You can then perform operations at service.<endpoint_method> etc
A more complete example with authentication can be found here.
EDIT:
Or as #Zig recommends, directly use the Google Cloud API
pip install googledatastore
Trivial example