I have inherited an app-engine project.
I try to deploy from my dev machine:
appcfg.py --oauth2 update PATH
I have installed app-engine SDK but I cannot find appcfg.py
I have googled and it seems like appcfg.py is no longer in use?
How should I deploy otherwise?
Yes, you are correct appcfg.py is no longer used to deploy.
Instead you should use the gcloud cli instead:
gcloud app deploy --project project-id --version version-number
See the reference documentation here:
https://cloud.google.com/sdk/gcloud/reference/app/deploy
Also, make sure you have an updated app.yaml in your project.
In addition to the app-engine SDK for JAVA
I had to install app-engine SDK for python
https://cloud.google.com/appengine/docs/standard/php/tools/uploadinganapp
Related
I'm in the process of upgrading a python2.7 gae std app to python3.7.
Everything is great except my indices just wont work.
I have a simple looking index.yaml file:
indexes:
- kind: Response
ancestor: yes
properties:
- name: __key__
direction: desc
And when I run certain commands then I get this
google.api_core.exceptions.FailedPrecondition: 400 no matching index found. recommended index is:
- kind: Response
ancestor: yes
properties:
- name: __key__
direction: desc
The command I'm using to run dev_appserver is: dev_appserver.py --application=my_project_id app.yaml.
My index.yaml file lives in the same directory as my app.yaml file.
Nothing else is running. The application itself is a Flask api, the error is coming up when I curl one of the endpoints.
What I've tried
After poking around docs and SO for a bit it seems like I might need to run the datastore emulator locally. So made sure that my gcloud components were up to date then:
gcloud beta emulators datastore env-init
# which gave me:
export DATASTORE_DATASET=firestore-datastore-280307
export DATASTORE_EMULATOR_HOST=::1:8608
export DATASTORE_EMULATOR_HOST_PATH=::1:8608/datastore
export DATASTORE_HOST=http://::1:8608
export DATASTORE_PROJECT_ID=firestore-datastore-280307
# then
gcloud beta emulators datastore start --project=my_project_id
# which gave me
stuff...
[datastore] API endpoint: http://::1:8679
[datastore] If you are using a library that supports the DATASTORE_EMULATOR_HOST environment variable, run:
[datastore]
[datastore] export DATASTORE_EMULATOR_HOST=::1:8679
[datastore]
[datastore] Dev App Server is now running.
So by combining these outputs it looks like my environment should be like:
export DATASTORE_DATASET=my_project_id
export DATASTORE_EMULATOR_HOST=::1:8679
export DATASTORE_EMULATOR_HOST_PATH=::1:8679/datastore
export DATASTORE_HOST=http://::1:8679
export DATASTORE_PROJECT_ID=my_project_id
Cool. So I leave the emulator running and try to get my dev_appserver to connect to it:
export DATASTORE_DATASET=my_project_id
export DATASTORE_EMULATOR_HOST=::1:8679
export DATASTORE_EMULATOR_HOST_PATH=::1:8679/datastore
export DATASTORE_HOST=http://::1:8679
export DATASTORE_PROJECT_ID=my_project_id
dev_appserver.py --application=my_project_id app.yaml
It starts up but when I curl my endpoint I get the same index error.
So then I kill dev_appserver and try it like this:
# same env vars as before
dev_appserver.py --support_datastore_emulator=true --application=my_project_id app.yaml
Then I get a new error:
RuntimeError: Cannot use the Cloud Datastore Emulator because the packaged grpcio is incompatible to this system. Please install grpcio using pip
I pip installed grpcio in a python2.7 env to get over that error. Now it looks like everything is running. But I'm still getting the missing index error.
And another strange thing: If I go to http://localhost:8000 and try navigating to anythin to do with datastores then I get errors like:
ConnectionError: Cannot connect to Cloud Datastore Emulator on ::1:{THE_PORT}
Which is very weird.
I'm considering going back to 2.7.
Don't go back to 2.7! If you have this problem in production, go to your developer console and check your indexes: https://console.cloud.google.com/.....
See if they are still building. It does take some time for the indexes to build.
If this only happens in development:
dev_appserver is buggy in Windows. I couldn't tell if you in on Windows or not. I had problems using dev_appserver in a virtual environment even on Mac when porting an app to Python 3.7.
You state you are using Flask. Try using the Flask server in development instead of dev_appserver. That is what worked for me. There is good documentation on this. You will start it with something like:
cd /Users/myname/venv37
source ./bin/activate
export FLASK_APP=/Users/myname/path_to_app
FLASK_ENV=development flask run --port 5000
edit:
ndb is not compatible with python 3.7. They have developed a new service Google Cloud NDB which makes the old ndb data usable: https://cloud.google.com/appengine/docs/standard/python3/migrating-to-cloud-ndb
New apps should use Cloud Datastore or Firestore. But legacy ndb apps can migrate to Google Cloud NDB.
From within VSCode, using the Azure web service extension, I can deploy a Hello-World flask app successfully to my Azure web app (linux, python 3.6 stack).
When I try to deploy my real flask app (with dependencies listed in a requirements.txt) file, and I specify "pip install --upgrade -r requirements.txt" as the web app "startup command" in the Azure portal, browsing the web app page gives an "Application Error".
The log for my web app shows the startup code failing when trying to build "psutil" due to the error unable to execute 'gcc': No such file or directory.
When the above failed, I also tried using "wheels" as part of my deployment as described here (https://blogs.msdn.microsoft.com/azureossds/2015/06/29/install-native-python-modules-on-azure-web-apps-api-apps/), but that didn't seem to change anything (same 'missing gcc' error).
Is this the correct way to deploy a flask app with python library dependencies from VSCode? I would expect that the deployment process would automatically process the requirements.txt (without having to specify the startup command) and I would expect it to be able to install the python libraries without error.
Due to you were using Azure WebApp for Linux based on Docker Container, the MSDN blog Install Python Modules on Azure App Services for Azure WebApp for Windows is not useful for your issue.
So for your issue of missing gcc error, per my experience, the solution to fix it is to configure a custom Linux container which had been pre-installed the gcc tool-chains and then to deploy your app into it from VS Code.
As references, please try to follow the documents below.
Configure a custom Linux container for Azure App Service . To install the gcc toolchains in a custom docker image via change the Docker file like install openssh as the section Enable SSH said.
Deploy to Azure using Docker from VS Code with Azure App Service extension.
The last few days I tried to create an App Engine Application based on DJango using PyCharm Professional 2016.3.2..
But everytime I try to create a new App Engine Application using Django as an third-party library PyCharm throws an Error and tells 'Create App Engine Application - Incorrect or corrupted App Engine SKD: cannot finde DJango installation'.
I already tried to install everything again, including Python, PyCharm, Django and the SDK, but it still doesn't work.
I would really appreciate any kind of help, because this problem drives me crazy.
You need to install extras.
gcloud components install app-engine-python-extras
And then try to create django (app engine) project again.
The legacy Appengine SDK ships with some Django versions included in its lib folder.
$ find 1.9.40/ -type d -name [Dd]jango*
1.9.40/google_appengine/lib/jinja2-2.6/examples/rwbench/django
1.9.40/google_appengine/lib/jinja2-2.6/ext/django2jinja
1.9.40/google_appengine/lib/django-1.4
1.9.40/google_appengine/lib/django-1.4/django
1.9.40/google_appengine/lib/PyAMF-0.6.1/pyamf/tests/adapters/django_app
1.9.40/google_appengine/lib/django-1.2
1.9.40/google_appengine/lib/django-1.2/django
1.9.40/google_appengine/lib/PyAMF-0.7.2/pyamf/adapters/tests/django_app
1.9.40/google_appengine/lib/django-0.96
1.9.40/google_appengine/lib/django-0.96/django
1.9.40/google_appengine/lib/django-1.5
1.9.40/google_appengine/lib/django-1.5/django
1.9.40/google_appengine/lib/django-1.3
1.9.40/google_appengine/lib/django-1.3/django
1.9.40/google_appengine/lib/django-1.9
1.9.40/google_appengine/lib/django-1.9/django
1.9.40/google_appengine/google/appengine/_internal/django
1.9.40/google_appengine/google/appengine/ext/django
1.9.40/google_appengine/google/appengine/ext/builtins/django_wsgi
1.9.40/google_appengine/google/storage/speckle/python/django
However the Google Cloud SDK does not include any Django versions in lib, only with Django utilities for its own internal use.
$ gcloud --version
Google Cloud SDK 138.0.0
app-engine-python 1.9.49
...
$ find google-cloud-sdk/ -type d -name [Dd]jango*
google-cloud-sdk/lib/third_party/oauth2client/contrib/django_util
google-cloud-sdk/lib/googlecloudsdk/third_party/appengine/ext/builtins/django_wsgi
google-cloud-sdk/platform/google_appengine/lib/jinja2-2.6/examples/rwbench/django
google-cloud-sdk/platform/google_appengine/lib/jinja2-2.6/ext/django2jinja
google-cloud-sdk/platform/google_appengine/lib/PyAMF-0.6.1/pyamf/tests/adapters/django_app
google-cloud-sdk/platform/google_appengine/lib/PyAMF-0.7.2/pyamf/adapters/tests/django_app
google-cloud-sdk/platform/google_appengine/google/appengine/_internal/django
google-cloud-sdk/platform/google_appengine/google/appengine/ext/django
google-cloud-sdk/platform/google_appengine/google/appengine/ext/builtins/django_wsgi
google-cloud-sdk/platform/google_appengine/google/storage/speckle/python/django
google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/contrib/django_util
google-cloud-sdk/.install/.backup/lib/googlecloudsdk/third_party/appengine/ext/builtins/django_wsgi
google-cloud-sdk/.install/.backup/platform/google_appengine/lib/jinja2-2.6/examples/rwbench/django
google-cloud-sdk/.install/.backup/platform/google_appengine/lib/jinja2-2.6/ext/django2jinja
google-cloud-sdk/.install/.backup/platform/google_appengine/lib/PyAMF-0.6.1/pyamf/tests/adapters/django_app
google-cloud-sdk/.install/.backup/platform/google_appengine/lib/PyAMF-0.7.2/pyamf/adapters/tests/django_app
google-cloud-sdk/.install/.backup/platform/google_appengine/google/appengine/_internal/django
google-cloud-sdk/.install/.backup/platform/google_appengine/google/appengine/ext/django
google-cloud-sdk/.install/.backup/platform/google_appengine/google/appengine/ext/builtins/django_wsgi
google-cloud-sdk/.install/.backup/platform/google_appengine/google/storage/speckle/python/django
google-cloud-sdk/.install/.backup/platform/gsutil/third_party/oauth2client/oauth2client/contrib/django_util
The solution is to install Django according to Google's instructions for installing third party packages, that is:
Install django in your third party libs folder:pip install django -t lib
Add from google.appengine.ext import vendor;vendor.add('lib') in your appengine_config.py module
As #snakecharmerb observed the django packages are missing in the app-engine-python component (1.9.49) of the cloud SDK (138.0.0).
They are, however, present in the standalone GAE SDK (also 1.9.49):
/usr/local # diff google_appengine_1.9.49/lib google-cloud-sdk-138.0.0/platform/google_appengine/lib | grep -vi common
Only in google_appengine_1.9.49/lib: django-0.96
Only in google_appengine_1.9.49/lib: django-1.2
Only in google_appengine_1.9.49/lib: django-1.3
Only in google_appengine_1.9.49/lib: django-1.4
Only in google_appengine_1.9.49/lib: django-1.5
Only in google_appengine_1.9.49/lib: django-1.9
Only in google_appengine_1.9.49/lib: graphy
So an alternative to vendoring django into your app might be to switch to this SDK instead of the cloud SDK. See What is the relationship between Google's App Engine SDK and Cloud SDK?, including comments.
Note that it's not the recommended way these days, tho.
You could also install both and try to copy/symlink the missing packages into the cloud SDK. YMMV.
To download the GAE SDK in the Download the SDK for App Engine page click on the Optionally, you can also download the original App Engine SDK for Python. "link" - it's actually a control for the expandable download section below:
I'm trying to deploy an app engine Managed VM python/flask application using the gcloud command.
When developing locally I can pass environment variables to my flask server. I do this so that I can read a config file per environment. For example:
gunicorn -b :8080 --env CONFIG_FILE=config_dev.py main:app
Now, i'd like to deploy my application and use config_production.py as my config file. It seems I'd need to pass the environment variable CONFIG_FILE=config_dev.py to gcloud preview app deploy to set it. I checked the help pages but didnt see a way to set this.
Previously when I was developing on app engine sandbox I could do:
appcfg.py -E ENVIRONMENT:dev update app.yaml
Is there an equivalent argument for gcloud?
I've heard that other platforms support auto-deployment of their code to production when they push changes to their Git repository.
Can I set up something similar to this for AppEngine? How?
I'm using Python2.7 on Windows, and bitbucket as a repository.
Thanks!
Since app engine deploy is just a python script, why can't you just write a shell script that calls 'git push' followed by 'python appcfg.py deploy'?
Any bitbucket hooks that will send from bitbucket->appengine after you upload to app is probably a bad idea since it will require storing your app engine login credentials on github.
Other projects might have your app server pull from github/bitbucket. You can do this if your app engine site just serves static websites using http://drydrop.binaryage.com/, but you can't update actual running code this way.
Recently, App Engine added Push to Deploy features: https://developers.google.com/appengine/docs/push-to-deploy
It only has built in support for GitHub, but it might still be possible to configure BitBucket to work with it.