I have a few questions that I am hoping someone can answer here. I realize I have knowledge gap and I am hoping someone here can fill it or at least point me towards the right direction.
I had built a Flask API that I could run inside a Docker Container on Ubuntu OS. To create a Swagger API Documentation, I added to my Flask application:
from flask_swagger_ui import get_swaggerui_blueprint
Rather than moving forward, I stopped the Docker and ran it again and as expected, It resulted in "502 Gateway error". Upon Checking Docker logs here is what I noticed:
ModuleNotFoundError: No module named 'flask_swagger_ui'
I understood this problem as I needed to add this dependency inside my docker. I added the following code to my
"requirements.txt" file: flask_swagger_ui==3.36.0.
However this did not change anything and I still received the same error. I tried adding the module directly into the Dockerfile I created through this code but that did not work either:
RUN pip install flask_swagger_ui
Doing one or both of these things results in 502 Gateway error. I have tried to add several other modules to my application Like Pandas and Sklearn but they result in the same error. I can NOT make any changes to my application without getting Bad Gateway.
To figure out what is wrong, I removed all the changes I had made and got it back to where the only dependency is Flask 2.0.1 in my "requiements.txt". After testing that the Docker is running fine, In a state of confusion, I removed "start.sh", "Dockerfile" and "requirements.txt" and put them in trash bin and restarted Docker.
To my bewilderment, The Docker was working fine! with all the files mentioned above in the trash bin! Since at this point now I was completely lost, I added all the dependencies back again to see if it works this time and got stuck with the same 502 Error and the Logs showed the same issue.
My question then is:
How do I add a dependency to my application?
How on Earth is my Docker container still running after 3 required files in trash bin? I stopped and restarted Docker again, As far as I know, It should not have worked.
Removing "Flask 2.0.1" from the requirements.txt file while it was in my directory and restarting my docker did not affect my Docker application at all and it was running fine. What is happening?
Here is my Dockerfile:
FROM tiangolo/uwsgi-nginx-flask:python3.8-alpine3.7
RUN apk --update add bash nano
RUN pip install pandas flask_swagger_ui
ENV STATIC_URL /static
ENV STATIC_PATH /home/faraz/python_docker/TestApp/app/static
COPY ./requirements.txt /home/faraz/python_docker/TestApp/requirements.txt
RUN pip install -r requirements.txt
and here is my requirements.txt:
Flask==2.0.1
flask_swagger_ui==3.36.0
pandas==1.2.4
Finally, my "start.sh":
#!/bin/bash
app="docker.test"
docker build -t ${app} .
docker run -d -p 56733:80 \
--name=${app} \
-v $PWD:/app ${app}
I had posted another question when I was building my API and the link below will take you to it:Method Not Allowed The method is not allowed for the requested URL. 405 Error
As per suggested by one of the comments: Here is the Github Repository:
https://github.com/frazali32/Docker-Flask-API
I found the answer to my issue. As I had suspected, It was nothing but Knowledge gap. The concept is explained brilliantly in the forum below:
Rebuild Docker container on file changes
Basically,Deleting or changing something locally does nothing to the container unless you are mounting it as a volume or you do not rebuild. In my case, I deleted the Docker Image created in my first attempt, Built another Image and restarted the Docker. It works now!
Related
I am attempting to set-up a docker container with php-apache and python. This is primarily for a php web application. For part of the functionality I wrote a python script that is utilizing a python library which fulfills functionality that I couldn't find with php. Otherwise I'd have just tried to stick to php for everything. I run the python script with php's shell_exec command. Everything works in my local development environment; however, when I attempt to push to production problems arise. Anyways, I have been trying for hours (tons of research on the topic) and I cannot figure out how to get Python installed on the same Docker container as php-apache. Here is an example of a Dockerfile I've been using:
FROM python:3.7
RUN apt-get update
RUN apt-get install python3.7
COPY requirements.txt ./
RUN pip3 install -r requirements.txt
FROM php:7.4.13-apache
RUN docker-php-ext-install mysqli pdo pdo_mysql
With this set-up I get "sh: 1: python: not found".
If I remove the last two lines (php-apache) the container keeps restarting continuously (though python is installed in this case). I've tried many other examples of dockerfiles for python and combined with php-apache none have worked.
I ended up going with the suggestion from #DavidMaze and setting up two separate containers, one for php:apache, and one for python. For the python container I built a simple flask application with endpoints that, when a GET request is made, run a specific python function. I used PHP curl to communicate with this flask api from the php:apache container.
I am deploying a web app on AWS's elastic beanstalk and am coming across the same error:
[StageApplication]. Stop running the command. Error: chown /var/app/staging/venv/bin/python: no such file or directory.
I see in my environment configuration the property: PYTHONPATH : /var/app/venv/staging-LQM1lest/bin
My app runs perfectly fine locally with the command 'python applicaiton.py'.
Any advice on how I can fix this?
Make sure you aren't pushing your local venv to your EB referenced Git Repo. I was trying to make updates from another machine and accidentally pushed it to master which caused this exact error. I had to use the following to remove it even after adding to gitignore file. Give that a shot and/or double check.
git rm -r --cached venv
git commit -m 'Remove the now ignored directory venv'
git push origin master
I discovered that eb deploy was deploying my local venv directory, even though it was in .gitignore. I confirmed this by going to "Application versions" in the sidebar and downloading the deployed .zip file under the "source" column. Sure enough, my venv was in there.
I was running eb deploy from a subdirectory in my git repo. As I intended, eb was only deploying this subdirectory. However, for some reason, it was ignoring the .gitignore file I had in this subdirectory. I added a .ebignore file and put my venv in there, and it fixed the problem. Note that when using .ebignore, eb no longer deploys what's commited in git, but rather what's in the working directory. More here.
I'm trying to update a Docker image on my Heroku app.
I just used the following commands from this documentation article: https://devcenter.heroku.com/articles/container-registry-and-runtime
heroku login
heroku container:login
heroku container:push web
heroku container:release web
heroku open
This set of commands worked perfectly the first time, except that I got an error about not being able to map to the $PORT variable I figured out how to fix the problem (I'm using Flask), so I updated my api.py (I named it api.py instead of app.py), and now I want to push the updates to the Heroku app. But running through those commands, it seems to use the cache (I've had this problem on my local machine before; I have to use the --no-cache option when I buid). I don't really know when it's being built on Heroku.
Anyways, at the end of the release command, it says: The process type web was not updated, because it is already running the specified docker image.
How do I get it to update?
Just for the people out there looking at this, it turned out it was a simple mistake in my Dockerfile.
Basically, I was using git clone to get my app files. Because of this, I had no ADD commands anywhere, so all the results were being cached by docker.
All I had to do was run ADD instead of git clone to import my project files, and now everything runs perfectly well :)
Recently I'm learning Flask by the book "Flask Web Development". When I completed the code and deployed it to Heroku, the following error happened:
ImportError: /app/.heroku/python/lib/python3.6/
sitepackages/psycopg2/.libs/libresolv-2-c4c53def.5.so:
symbol __res_maybe_init version GLIBC_PRIVATE not defined
in file libc.so.6 with link time reference
However, this works fine locally. I have searched relevant questions about psycopg2, and I have adjusted the version of psycopg2 but the same error still happens. Please, how can I solve the problem?
I got the same problem. I solved it by forcing heroku to clean up the python virtual environment, and reinstall the requirements.txt file using psycopg2>=2.7,<2.8 --no-binary psycopg2. I must admit it felt a little like magic, but for what its worth here are the steps I took:
Using Git Bash for Windows, I logged in to heroku with heroku login.
Next, I navigated to the root folder of the git/project folder I was working on.
There I made sure that my requirements.txt was updated by running venv/Scripts/pip freeze > requirements.txt. You will probably have to find your own python/virtual environment path.
Next I changed the line/entry for psycopg2 to psycopg2>=2.7,<2.8 --no-binary psycopg2.
Then saved all the content of the requirements.txt file to my desktop, making sure the one in my project folder was empty. Not sure if this step was necessary. But that's what I did, so I'm just gonna mention it here.
Now in the Git Bash terminal I ran heroku apps:info -a blogglistene to find the version of the my heroku stack which for me was heroku-18.
Next I look online for the supported versions of python for heroku-18. It said the default was python-3.6.6 (which I then assumed is the one I was running), and some others.
I then picked python-2.7.15 arbitrarily from the list, and added that text to a runtime.txt file which I placed in the root of my project folder.
Next I committed and pushed this to my heroko git repo, and it was deployed. This will temporarily break the build. From what I understand, the magic behind this is that heroku now fully destroys the python interpreted and environment, so no lingering cached files or anything. Clean slate.
Then I reverted the content of runtime.txt to the default of python-3.6.6 and added back the content of the requirements.txt file, making sure the changes I made are there.
Finally, I committed and pushed this, and my application was back up and running.
I had the exact same problem when I was following "Flask Web Development". I spent a day plus trying to solve it and eventually succeeded. It's not as complicated as André C. Andersen's method.
Make sure you are on master branch. Not other branches.
The book is second edition of Flask Web Development by Miguel Grinberg. Chapter 17. Deployment. When you work all the way till Deploying with git push, after git push heroku master to upload the application to the heroku remote. The application is now deployed and running, but it is not going to work correctly because the deploy command that initializes the database tables has not been executed yet. You need to use
heroku run flask deploy
to create database on heroku. That's where the problem appears.
This is because at this point your git repo HEAD is at 17c. It's not the master branch yet. If you commit any change now, it won't change heroku environment. Meaning when you do
git branch
You will see:
(HEAD detached from 17c)
master
But this is not what you want. Not head attached to 17c. You want to see only master, which is achieved with
git checkout master
Then git branch will show only master branch. Therefore you can make adjustments on the project and heroku will update. Doing this because you will eventually git push to master branch. I think you can git push to other branches in order for heroku to work, but I don't know how to do that.
Change psycopg2==2.7.3 to the newest version, e.g. psycopg2==2.8.4. Another workaround is to change it to psycopg2-binary==2.8.3.
psycopg2 2.7.3 will give you GLIBC_PRIVATE error. (You can check it by running heroku run bash, then starting python and importing psycopg2. If heroku dependency is 2.7.3, then you will see the same error.) The official website notes that version 2.7.3.1 drops libresolv which causes the issue, therefore using this version or later should do the trick.
Now, change your requirements/heroku.txt (heroku environment dependency) line psycopg2==2.7.3 to psycopg2--2.8.4 or psycopg2-binary==2.8.3. (For some reason binary version of psycopg2 will work without issue. Don't know why, but I saw A LOT of discussion on stackover.) Then,
git add .
git commit -m "notes"
As usual.
git push heroku master
Push to heroku master branch. If you do not do step 1, then git will show Everything up-to-date. Therefore, it won't make changes because you are committing not to master. Now, you can see git is downloading the appropriate package.
You can double check if the packages are correct by opening heroku bash then pip list. You will see psycopg2 (and psycopg2-binary if you chose the workaround). Now, when you start Python, you can import psycopg2 with no problem.
Finally. heroku run flask deploy. Voila!
I was stuck on step 1, since whatever change I made with the project, heroku just didn't update.
This is my first experience deploying a Flask web app to Azure.
I followed this tutorial.
The default demo app they have works fine for me.
Afterwards, I pushed my Flask app via git. The log shows deployment was successful. However, when I browse the hosted app via link provided in "Application Properties", I get a 500 error as follows:
The page cannot be displayed because an internal server error has
occurred.
Most likely causes: IIS received the request; however, an internal
error occurred during the processing of the request. The root cause of
this error depends on which module handles the request and what was
happening in the worker process when this error occurred. IIS was not
able to access the web.config file for the Web site or application.
This can occur if the NTFS permissions are set incorrectly. IIS was
not able to process configuration for the Web site or application. The
authenticated user does not have permission to use this DLL. The
request is mapped to a managed handler but the .NET Extensibility
Feature is not installed.
The only off-base thing I can see by browsing the wwwroot via KUDU is that none of the packages I have installed in my local virtual environment are installed on Azure despite the existence of the "requirements.txt" file in wwwroot.
My understanding is that Azure would pip install any non existent package that it finds in the requirements.txt upon GIT successful push. But it doesn't seem to be happening for me.
Am I doing something wrong and the missing packages is just a symptom or could it be the cause the issue?
Notes:
My Flask app works fine locally (linux) and on a 3rd party VPS
I redeployed several times starting from scratch to no avail (I use local GIT method)
I cloned the Azure Flask demo app locally, changed just the app folder and pushed back to Azure, yet no success.
Azure is set to Python 2.7 same as my virtual env locally
As suggested in the tutorial linked above, I deleted the "env" folder and redeployed to trick Azure to reinstall the virtual env. It did but with its own default packages not the one in my requirements.txt.
My requirements.txt has the following:
bcrypt==3.1.0 cffi==1.7.0 click==6.6 Flask==0.11.1 Flask-Bcrypt==0.7.1
Flask-Login==0.3.2 Flask-SQLAlchemy==2.1 Flask-WTF==0.12
itsdangerous==0.24 Jinja2==2.8 MarkupSafe==0.23 pycparser==2.14
PyMySQL==0.7.7 python-http-client==1.2.3 six==1.10.0 smtpapi==0.3.1
SQLAlchemy==1.0.14 Werkzeug==0.11.10 WTForms==2.1
As Azure Web Apps will run a deploy.cmd script as the deployment task to control which commands or tasks will be run during the deployment.
You can use the command of Azure-CLI azure site deploymentscript --python to get the python applications' deployment task script.
And you can find the following script in this deploy.cmd sciprt:
IF NOT EXIST "%DEPLOYMENT_TARGET%\requirements.txt" goto postPython
IF EXIST "%DEPLOYMENT_TARGET%\.skipPythonDeployment" goto postPython
echo Detected requirements.txt. You can skip Python specific steps with a .skipPythonDeployment file.
So the .skipPythonDeployment will skip all the following steps in deployment task, including creating virtual environment.
You can try to remove .skipPythonDeployment from your application, and try again.
Additionally, please refer to https://github.com/projectkudu/kudu/wiki/Custom-Deployment-Script for more info.