I'm trying to keep my Google Cloud project centered around App Engine (GAE) Standard Environment by running a UI in GAE Python 2.7 and then deploying an OpenAPI to Cloud Endpoints which also uses Python 2.7. I want my API to be able to check Google Cloud Storage (GCS) for a file, and then if it doesn't exist, fetch an Entity from Datastore, process some data with Text-
to-Speech, and then save resulting mp3 to GCS.
Under Python 2.7, I need to use the old (2015) cloudstorage client library, google-endpoints, google-endpoints-api-management and google-cloud-texttospeech modules, all in my ./lib folder.
With these modules installed, I can run the "echo" demo project without problem. This indicates that there are no dependency version conflicts. However, when trying to use texttospeech, i get this error:
ImportError: No module named grpc._cython.cygrpc
To fix this, I add the following to the libraries section of app.yaml:
- name: grpcio
version: latest
I then get this error:
from six.moves import http_client
ImportError: No module named moves
I try to fix it similarly, but the error persists
- name: six
version: latest
Other solutions didn't help. six 1.12.0 is installed in ./lib, and 1.11.0 comes with the latest gcloud cloud SDK (233.0.0).
How can I use text-to-speech with Cloud Endpoints? Even if i get this working, can I expect it to work with GCS and Datastore? So far, using Flask-RESTful under GAE seems easier.
These client libraries do not work with the first-generation Python 2.7 runtime, and there are no plans to support them.
You will need to use the new second-generation Python 3.7 runtime instead, which will let you install arbitrary dependencies. See "Python 3 Runtime Environment" for more details.
I think the answer is that Google Cloud Client Libraries aren't yet compatible with Google App Engine. The problem I had here persisted when trying to use Text To Speech in a GAE app even though it worked fine when running little tests locally outside of a GAE app.
This issue can be tracked here:
https://github.com/googleapis/google-cloud-python/issues/1893
Related
I have created a new project on GCP. I am trying to select/add data from firestore (already in native mode) from Python flask app. From local env. I am able to connect to firestore. But once I hosted my app on App engine, my API is not able to connect with Firestore. It's throwing the below error:
TypeError: with_scopes_if_required() got an unexpected keyword argument 'default_scopes'
_create_composite_credentials (/layers/google.python.pip/pip/lib/python3.7/site-packages/google/api_core/grpc_helpers.py:238)
I am not sure what I am missing here. I appreciate any support.
Thanks
I had the same issue, and I just fixed it today.
As you can see in the changelog https://googleapis.dev/python/google-api-core/latest/changelog.html recently they have updated the requirements for google-auth lib.
You need to check your requirements.txt.
In my case, we had two libs pinned that we weren't using directly.
google-auth==1.21.1
google-auth-oauthlib==0.4.1
I just removed them from the requirements.txt so that dependencies versions are handled automatically, and it solved the issue.
So I guess it depends on your case, on how you are managing dependencies, but you might want to play with google-auth versions.
I have a Flask app running in Google Cloud App Engine. I want the user to be able to call MATLAB functions on their local instance - if they have MATLAB installed locally and the correct license, of course.
Running locally the app works well using matlab.engine, however, when deployed to google cloud platform it fails during build. Looking in the logs:
ModuleNotFoundError: No module named 'matlabengineforpython3_7
So I suspect it is because the server cannot import the required dlls etc. for the python matlab engine package to work.
Is there a way to pass the required files to google app engine? Is this approach even possible?
My users will always have a local copy of MATLAB, so I am trying to find a solution that avoids needing to pay for the MATLAB server license.
I don't believe that any solution is possible that would avoid a Matlab server license. Your server cannot access installed Matlab on the computers of your users.
To install non-Python software with App Engine you need to use a custom runtime with App Engine Flexible. Check the GAE docs for more details.
I am running a webapp2 sever in Google Cloud SDK and making use of the Naked module to run node.js scripts from python2 in a Vagrant box.
While starting the server which has this code snippet
from Naked.toolshed.shell import muterun_js
def get_signed_hash(username):
response_from_js = muterun_js('./views/api/generateSignedTransaction.js',
nonce)
signed_hash = response_from_js.stdout
print('0x' + signed_hash)
I am getting this error message
ImportError: No module named Naked.toolshed.shell
I am making use of the requests library as well and that is not causing any problems. Tried to debug the problem as to maybe the location of the packages, they are both in the same place:
>>> import requests
>>> print(requests.__file__)
/home/vagrant/.local/lib/python2.7/site-packages/requests/__init__.pyc
>>> import Naked
>>> print(Naked.__file__)
/home/vagrant/.local/lib/python2.7/site-packages/Naked/__init__.pyc
Infact I ran the script in the same location as a standalone file calling get_signed_hash('username') and it runs successfully. Is there any other place I need to check?
For the first generation standard environment apps dependencies need to be installed inside the app (or vendored-in). Having them in the system site-packages (where the regular, standalone apps are loading libraries from) is not enough. Even if you somehow convince the local development server to run the app, it won't work when uploaded to GAE as GAE doesn't have access to your local system.
Typically you'd install your library (and all its dependencies) inside a lib directory inside your app dir, so they could be uploaded to GAE together with your app. See Copying a third-party library.
Alternatively you could try to just symlink the Naked directory into your lib dir, which could work if its all other dependencies are already satisfied in a similar manner. See related How do I access a vendored library from a module in Python Google App Engine?
I am successfully able to get BQ data from one project to another from the advice in this answer. However this only works when deployed on my development/staging instance and not my local development server on Google App Engine.
My findings are that it works in production because you include:
libraries:
- name: pycrypto
version: "latest"
in app.yaml. However these libraries are not accessible from the dev server. I have tried installing everything locally (Pycrypto, oauth2client, openSSL) after digging through some docs and tracing the error but still cannot get it to work. I have tried installing through pip and manually doing the build/install from the raw files to no avail. Any advice on getting these queries to work on the local django server? Working on Ubuntu if that matters, perhaps it's looking in the wrong spot for the libraries?
If its just the libs that are mising follow this answer https://stackoverflow.com/a/11405769/3877822 to insatll pycrypto to the root of your project
As #Udi suggests in the comment below, the following command also
installs pycrypto and can be used in virtualenv as well:
easy_install
http://www.voidspace.org.uk/downloads/pycrypto26/pycrypto-2.6.win32-py2.7.exe
Notice to choose the relevant link for your setup from this list
I am trying to access Xero (Accounting Software) from my Google App Engine system. (Python 2.7).
The Xero example uses M2Crypto but this uses .c files which don't seem to work on App Engine. I also downloaded pycrypto from Google but this has the same problem.
At this stage I would just like my App Engine program to have a button for the user to log in to Xero. Any pointers for me?
Note: Some of these packages appear to need a visual studio or a c compiler, which I don't have.
An update. It appears that only a 'private' application needs pycrypto or m2crypto, neither of which I can install. A public application does not need these, the example I am following has both. I am continuing to work through this.
Thanks in advance
David (Windows Vista, Python 2.7, Current Google App Engine SDK)
I was able to access xero using pyzero, however there were a couple of gotchas, the most significant being the need to upgrade the version of urllib3 that comes packaged as part of the requests library.
I've created a simple project that demonstrates it in use.
https://github.com/hamish/gae_xero
The libraries that I needed to install were:
https://codeload.github.com/freakboy3742/pyxero/zip/master
http://labix.org/download/python-dateutil/python-dateutil-1.5.tar.gz
https://codeload.github.com/kennethreitz/requests/zip/master
https://codeload.github.com/requests/requests-oauthlib/zip/master
https://pypi.python.org/packages/source/o/oauthlib/oauthlib-0.6.1.tar.gz
https://pypi.python.org/packages/source/u/urllib3/urllib3-1.7.1.tar.gz (instaled into the requests/packages/ directory)
Additionally the pycrypto library must be installed and enabled:
sudo pip install pycrypto
[excerpt from app.yaml]
- name: pycrypto
version: latest
Your best bet will be to access the Xero API Endpoints via the AppEngine URL Fetch Service. You'll probably have to satisfy Xero API Authentication along the way.