I'm working on deploying my first dash app via heroku. I already did the example deployment given here: https://dash.plotly.com/deployment
which was quick and easy. Now I'm running through the same process with my own dash app, which is much more involved than the example app in the link, but which runs fine locally.
In step 2 given in the link, you initialize an empty git repo and run a new virtualenv, okay. Then you have to reinstall all the app's dependencies into the new virtual env.
This is my question, what are my app dependencies? Are those the modules I install at the top of my apps .py file? such as pandas, numpy, sklearn, etc?
Yes, that's correct. They must be defined in the requirements.txt file in the root of your project.
You can create the file manually or running this command
pip freeze > requirements.txt
The file looks something like this
pandas==1.2.0
numpy==1.19.4
Related
I made a website in node.js and I have used child process to run python scripts and use their results in my website. But when I host my application on heroku, heroku is unable to run the python scripts.
I have noticed that heroku was able to run python scripts which only had inbuilt python packages like sys, json but it was failing for scripts which were using external packages like requests, beautifulsoup.
Attempt-1
I made a requirements.txt file inside the project folder and pushed my code to heroku again. But it was still not working. I noticed that heroku uses the requirements.txt file only for python based web applications but not for node.js applications.
Attempt-2
I made a python virtual env inside the project folder and imported the required python packages into the venv. I then removed gitignore from the venv and pushed the whole venv folder to heroku. It still didn't work.
Please let me know if anyone came across a way to handle this.
You can use a Procfile to specify a command to install the python dependencies then start the server.
Procfile:
web: pip install -r requirements.txt && npm start
Place Procfile in your project's root directory and deploy to heroku.
Solution:
Have a requirements.txt file in your project folder.
After you create a heroku app using "heroku create", heroku will identify your app as python based and will not download any of the node dependencies.
Now, go to your heroku profile and go to your app's settings. There is an option named "Add Buildpack" in there. Click on that and add node.js as one of the buildpacks.
Go back to your project and push it to heroku. This time heroku will identify your app as both python and node.js based and will download all the required files.
In a django project I have imported firebase_admin and it worked perfectly on local host, but when I used Heroku to host my web app it gave error no module named 'firbase-admin'. I cross-checked my requirements files and it has firbase-admin. I don't know what's that I did wrong as I made sure that requirements used in virtual environment is same to that in Heroku requirements files.
make sure firebase-admin==4.3.0 is in your requirements.txt file and try not to generate dependencies manually because you can missout on some libraries preferably use pip freeze > requirements.txt
This is my first experience deploying a Flask web app to Azure.
I followed this tutorial.
The default demo app they have works fine for me.
Afterwards, I pushed my Flask app via git. The log shows deployment was successful. However, when I browse the hosted app via link provided in "Application Properties", I get a 500 error as follows:
The page cannot be displayed because an internal server error has
occurred.
Most likely causes: IIS received the request; however, an internal
error occurred during the processing of the request. The root cause of
this error depends on which module handles the request and what was
happening in the worker process when this error occurred. IIS was not
able to access the web.config file for the Web site or application.
This can occur if the NTFS permissions are set incorrectly. IIS was
not able to process configuration for the Web site or application. The
authenticated user does not have permission to use this DLL. The
request is mapped to a managed handler but the .NET Extensibility
Feature is not installed.
The only off-base thing I can see by browsing the wwwroot via KUDU is that none of the packages I have installed in my local virtual environment are installed on Azure despite the existence of the "requirements.txt" file in wwwroot.
My understanding is that Azure would pip install any non existent package that it finds in the requirements.txt upon GIT successful push. But it doesn't seem to be happening for me.
Am I doing something wrong and the missing packages is just a symptom or could it be the cause the issue?
Notes:
My Flask app works fine locally (linux) and on a 3rd party VPS
I redeployed several times starting from scratch to no avail (I use local GIT method)
I cloned the Azure Flask demo app locally, changed just the app folder and pushed back to Azure, yet no success.
Azure is set to Python 2.7 same as my virtual env locally
As suggested in the tutorial linked above, I deleted the "env" folder and redeployed to trick Azure to reinstall the virtual env. It did but with its own default packages not the one in my requirements.txt.
My requirements.txt has the following:
bcrypt==3.1.0 cffi==1.7.0 click==6.6 Flask==0.11.1 Flask-Bcrypt==0.7.1
Flask-Login==0.3.2 Flask-SQLAlchemy==2.1 Flask-WTF==0.12
itsdangerous==0.24 Jinja2==2.8 MarkupSafe==0.23 pycparser==2.14
PyMySQL==0.7.7 python-http-client==1.2.3 six==1.10.0 smtpapi==0.3.1
SQLAlchemy==1.0.14 Werkzeug==0.11.10 WTForms==2.1
As Azure Web Apps will run a deploy.cmd script as the deployment task to control which commands or tasks will be run during the deployment.
You can use the command of Azure-CLI azure site deploymentscript --python to get the python applications' deployment task script.
And you can find the following script in this deploy.cmd sciprt:
IF NOT EXIST "%DEPLOYMENT_TARGET%\requirements.txt" goto postPython
IF EXIST "%DEPLOYMENT_TARGET%\.skipPythonDeployment" goto postPython
echo Detected requirements.txt. You can skip Python specific steps with a .skipPythonDeployment file.
So the .skipPythonDeployment will skip all the following steps in deployment task, including creating virtual environment.
You can try to remove .skipPythonDeployment from your application, and try again.
Additionally, please refer to https://github.com/projectkudu/kudu/wiki/Custom-Deployment-Script for more info.
Expanding on this question I am trying to deploy Django on OpenShift but I'm having some problems understanding OpenShift.
I have managed to get as far as setting up a quick app with the git repo https://github.com/openshift/django-example but have the following questions:
Can I develop locally after git cloning to my local? (virtualenv, adding packages)
Packages, what's the deal? local, remote, adding, sycing, virtualenv, git, ...
I came across this line in Nate Aune's PaaS Bakeoff (slide 42) for setup.py and it looks quite useful:
install_requires=open('%s/project.txt' % \
os.environ.get('OPENSHIFT_REPO_DIR', PROJECT_ROOT)).readlines(),
(because I know I can pip freeze > requirments.txt in my virtualenv)
... Is %s/project.txt in wsgi or tthe directory below wsgi? Do I have to set PROJECT_ROOT with some funky os stuff?
EDIT
Basically:
Is it best to ssh into your OpenShift application (lest say you have a dev one) and work there or work off a local copy?
How do you install python packages after you have ssh'ed in to your OpenShift app? (virtualenv)
If you ssh'ed into your OpenShift app do you have to do anything after: creating a project, creating an app (manage.py startapp ...), changing code in your django app
If local is the best option:
How do I use the example locally?
Do I need to setup a virtualenv to work locally?
How do I make sure the python packages django needs are on OpenShift?
How do I add python packages to my OpenShift version (I'm presuming git doesn't do that)
I would suggest developing in a local copy.
In my experience there is no reason to ssh in, unless if you
need to perform one-of operation (migrating databases for example)
The requirements for the openshift app are specified in the setup.py file.
install_requires=['Django>=1.6.0', 'redis>=2.0']
Locally you can of course work in a virtualenv
I was facing a similar situation and found the answer provided by Luis Masuelli to the question posted here very informative. Hope it might help you.
I'm following this guide and trying to develop a Flask app to run on the Google App Engine. I followed the guide to the letter but when I launch the dev app server from the Launcher and go to http://localhost:8080/, I get a HTTP 500 error.
I check the logs and it says No module named flask. Then I check the interactive console in the admin console by running import flask and I get the same error message. I can import flask in any other python file without error.
Is there a way to fix this?
Working a bit with GAE and Flask I have realized this:
Running directly with Python
To run the app with python directly (python app.py) you need have the dependents packages installed in your environment using the command: pip install flask
Running with dev_appserver.py
To run the app with the dev_appserver.py provided by GAE SDK you need have all dependent packages inside your project, as: Flask, jinja2... Look in my another answer a example how to configure this packages : https://stackoverflow.com/a/14248647/1050818
UPDATED
Running Python, Virtualenv, Flask and GAE on Windows
Install Python
Install Python http://www.python.org/ftp/python/2.7.2/python-2.7.2.msi
Click in Windows Start button and search by "Edit the system environment" and open
Go to the tab Advanced and click on button "Environment Variables…"
When the Environment Variables window opens, choose Path from the System variables list and click Edit…
Add this ;C:\Python27;C:\Python27\Scripts at the end of the value and save
Install setuptools MS Windows installer (Necessary to install PIP on Windows)
Choose the correct installer for you in this page http://pypi.python.org/pypi/setuptools#files( I used this one: http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11.win32-py2.7.exe#md5=57e1e64f6b7c7f1d2eddfc9746bbaf20)
Download the installar and Install that
Install PIP
Download PIP http://pypi.python.org/pypi/pip#downloads
Extract it to any folder
From that directory, type python setup.py install
Install Virtualenv
Execute pip install virtualenv
Execute this mkdir c:\virtualenvs to create a folder to the Virtual Envs
Execute this cd c:\virtualenvs to access that folder
Execute virtualenv flaskdemo to create a virtualenv for you project
Active the virtualenv c:\virtualenvs\flaskdemo\scripts\activate
Install Google App Engine SDK
Install the SDK https://developers.google.com/appengine/downloads
Create the project
Create a directory for your project
Create the main of your application https://github.com/maxcnunes/flaskgaedemo/blob/master/main.py
Create the configuration of your appliction for Google App Engine https://github.com/maxcnunes/flaskgaedemo/blob/master/app.yaml
Create a file to let GAE initialize your application https://github.com/maxcnunes/flaskgaedemo/blob/master/initialize_gae.py
(Look a example of the code here: https://github.com/maxcnunes/flaskgaedemo )
Install Flask to run Locally
Execute pip install flask
Install Flask to run on the GAE
Download Flask https://github.com/mitsuhiko/flask/archive/0.9.zip and extract the folder flask inside your project
Download Werkzeug https://github.com/mitsuhiko/werkzeug/archive/0.8.3.zip and extract the folder werkzeug inside your project
Download Jinja2 https://github.com/mitsuhiko/jinja2/archive/2.6.zip and extract the folder jinja2 inside your project
Download Simple Json https://github.com/simplejson/simplejson/archive/v3.0.5.zip and extract the folder simplejson inside your project
Running the application with GAE SDK
Open Google App Engine Launcher
Add a new application
Run the application
Click in Browse button to open your application on browser
Finally click on Deploy button to deploy your application
Usually, templates come with a requirements.txt. If not, add your dependencies there and then run pip install -t lib -r requirements.txt to force the libraries to be saved in the lib folder.
Make sure you've added lib to appengine_config.py with vendor.add('lib') if it's not already there.
I was also facing the same issue and after spending 1 day on it have found out my silly mistake actually while refactoring my flask app I have changed
appengine_config.py to some other name.
Ideally appengine_config.py should look like below if you are having all your dependencies in lib folder only
from google.appengine.ext import vendor
#Add any libraries installed in the "lib" folder.
vendor.add('lib')
And because it was not able to find and execute appengine_config.py so lib folder was not registered as a dependency folder. To check you can try printing something in appengine_config.py to check if it's being executed on server startup.
tldr: use appengine_config.py and copy your virtualenv to a folder called lib, then make SURE you are running the app via dev_appserver.py
(the below is via bash in ubuntu)
SO after a long battle, I find that virtual env and gcloud dont play nice -
I copied everything from my virtual env dir
.../.virtualenvs/nimble/local/lib/python2.7/site-packages
into
[projectdir]/lib
and my appengine_config.py finally worked locally like it does in the cloud, but I absolutely HAVE to run
dev_appserver.py [my proj dir here]
or the google.appengine module wont load. did not know I should be using dev server. I feel very dumb.
for reference, heres the appengine_config.py
"""`appengine_config` gets loaded when starting a new application instance."""
print 'running app config yaya!'
from google.appengine.ext import vendor
vendor.add('lib')
print 'I am the line after adding lib, it should have worked'
import os
print os.getcwd()
Do you have Extra Libraries component for Python installed? It can be installed with
gcloud components install app-engine-python-extras
After installing this extra library you should be able to use built-in flask library without a problem. For more information, refer to this page
Source