I am deploying a web app on AWS's elastic beanstalk and am coming across the same error:
[StageApplication]. Stop running the command. Error: chown /var/app/staging/venv/bin/python: no such file or directory.
I see in my environment configuration the property: PYTHONPATH : /var/app/venv/staging-LQM1lest/bin
My app runs perfectly fine locally with the command 'python applicaiton.py'.
Any advice on how I can fix this?
Make sure you aren't pushing your local venv to your EB referenced Git Repo. I was trying to make updates from another machine and accidentally pushed it to master which caused this exact error. I had to use the following to remove it even after adding to gitignore file. Give that a shot and/or double check.
git rm -r --cached venv
git commit -m 'Remove the now ignored directory venv'
git push origin master
I discovered that eb deploy was deploying my local venv directory, even though it was in .gitignore. I confirmed this by going to "Application versions" in the sidebar and downloading the deployed .zip file under the "source" column. Sure enough, my venv was in there.
I was running eb deploy from a subdirectory in my git repo. As I intended, eb was only deploying this subdirectory. However, for some reason, it was ignoring the .gitignore file I had in this subdirectory. I added a .ebignore file and put my venv in there, and it fixed the problem. Note that when using .ebignore, eb no longer deploys what's commited in git, but rather what's in the working directory. More here.
Related
I have an old project from 2 years ago that I would like to put onto another Heroku (so it is accessible if the first one stops working). Originally I build this project on Linux, and now I'm on Windows.
I cloned the Github project, added the .env file (for API calls), and tried to push it to the new Heroku page. It didn't work at first so I installed Nodejs and Python buildpacks. I also renamed master to main.
I've been getting errors & trying to work through them, but I don't understand this last one:
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
remote: Waiting for release... failed.
After Googling for answers, this is what I've tried sofar:
pip install virtualenv
> Requirement already satisfied:
virtualenv --python C:\Python27\python.exe venv
> 'virtualenv' is not recognized as an internal or external command, operable program or batch file.
python virtualenv --python C:\Python27\python.exe venv
> python: can't open file 'C:\Users\USER\Desktop\PROJECT from PROJECT\virtualenv': [Errno 2] No such file or directory
Any idea what I need to do? I need to create a PYTHONPATH environment variable? Or I need to activate the project in a virtual environment? Or something else?
I was able to copy my old Heroku app onto a different Heroku account:
1: Clone the app from Heroku onto my local machine:
heroku git:clone -a myapp
(Heroku doesn't recommend this because it is not a storage repository. But since I knew my app was working, I wanted the exact working code.)
2: Logout of my first Heroku account and login to my second one. Then change my login credentials in the command line:
heroku login
3: Remove the original Heroku branch:
git remote rm heroku
4: Add the new Heroku branch:
heroku git:remote -a example-app
5: My original app uses React and Python, so I had to make sure both buildpacks were installed in the settings:
heroku/nodejs
heroku/python
6: Try to push to Heroku; this failed for me:
git push heroku main
7: My first error was I didn't specify which version of Python the app needs so Heroku was automatically using the latest one. My app needs old 3.7.1 and Heroku was giving it new 3.10.4. To fix I created a runtime.txt file on the root directory:
python-3.7.13
(Heroku doesn't support 3.7.1 anymore. So I tried 3.7.13, which is the oldest one they do support.)
8: My second error was this:
" ImportError: cannot import name 'FieldDoesNotExist' from 'django.db.models.fields' (/app/.heroku/python/lib/python3.7/site-packages/django/db/models/fields/init.py)"
This is my mistake since I didn't clearly specify which version of Django to use.
9: In the requirements.txt file I changed it to this:
Django==3.0.6
10: And then I was able to push the code to Heroku, the app works just like the original:
git push heroku main
I have a few questions that I am hoping someone can answer here. I realize I have knowledge gap and I am hoping someone here can fill it or at least point me towards the right direction.
I had built a Flask API that I could run inside a Docker Container on Ubuntu OS. To create a Swagger API Documentation, I added to my Flask application:
from flask_swagger_ui import get_swaggerui_blueprint
Rather than moving forward, I stopped the Docker and ran it again and as expected, It resulted in "502 Gateway error". Upon Checking Docker logs here is what I noticed:
ModuleNotFoundError: No module named 'flask_swagger_ui'
I understood this problem as I needed to add this dependency inside my docker. I added the following code to my
"requirements.txt" file: flask_swagger_ui==3.36.0.
However this did not change anything and I still received the same error. I tried adding the module directly into the Dockerfile I created through this code but that did not work either:
RUN pip install flask_swagger_ui
Doing one or both of these things results in 502 Gateway error. I have tried to add several other modules to my application Like Pandas and Sklearn but they result in the same error. I can NOT make any changes to my application without getting Bad Gateway.
To figure out what is wrong, I removed all the changes I had made and got it back to where the only dependency is Flask 2.0.1 in my "requiements.txt". After testing that the Docker is running fine, In a state of confusion, I removed "start.sh", "Dockerfile" and "requirements.txt" and put them in trash bin and restarted Docker.
To my bewilderment, The Docker was working fine! with all the files mentioned above in the trash bin! Since at this point now I was completely lost, I added all the dependencies back again to see if it works this time and got stuck with the same 502 Error and the Logs showed the same issue.
My question then is:
How do I add a dependency to my application?
How on Earth is my Docker container still running after 3 required files in trash bin? I stopped and restarted Docker again, As far as I know, It should not have worked.
Removing "Flask 2.0.1" from the requirements.txt file while it was in my directory and restarting my docker did not affect my Docker application at all and it was running fine. What is happening?
Here is my Dockerfile:
FROM tiangolo/uwsgi-nginx-flask:python3.8-alpine3.7
RUN apk --update add bash nano
RUN pip install pandas flask_swagger_ui
ENV STATIC_URL /static
ENV STATIC_PATH /home/faraz/python_docker/TestApp/app/static
COPY ./requirements.txt /home/faraz/python_docker/TestApp/requirements.txt
RUN pip install -r requirements.txt
and here is my requirements.txt:
Flask==2.0.1
flask_swagger_ui==3.36.0
pandas==1.2.4
Finally, my "start.sh":
#!/bin/bash
app="docker.test"
docker build -t ${app} .
docker run -d -p 56733:80 \
--name=${app} \
-v $PWD:/app ${app}
I had posted another question when I was building my API and the link below will take you to it:Method Not Allowed The method is not allowed for the requested URL. 405 Error
As per suggested by one of the comments: Here is the Github Repository:
https://github.com/frazali32/Docker-Flask-API
I found the answer to my issue. As I had suspected, It was nothing but Knowledge gap. The concept is explained brilliantly in the forum below:
Rebuild Docker container on file changes
Basically,Deleting or changing something locally does nothing to the container unless you are mounting it as a volume or you do not rebuild. In my case, I deleted the Docker Image created in my first attempt, Built another Image and restarted the Docker. It works now!
I made a website in node.js and I have used child process to run python scripts and use their results in my website. But when I host my application on heroku, heroku is unable to run the python scripts.
I have noticed that heroku was able to run python scripts which only had inbuilt python packages like sys, json but it was failing for scripts which were using external packages like requests, beautifulsoup.
Attempt-1
I made a requirements.txt file inside the project folder and pushed my code to heroku again. But it was still not working. I noticed that heroku uses the requirements.txt file only for python based web applications but not for node.js applications.
Attempt-2
I made a python virtual env inside the project folder and imported the required python packages into the venv. I then removed gitignore from the venv and pushed the whole venv folder to heroku. It still didn't work.
Please let me know if anyone came across a way to handle this.
You can use a Procfile to specify a command to install the python dependencies then start the server.
Procfile:
web: pip install -r requirements.txt && npm start
Place Procfile in your project's root directory and deploy to heroku.
Solution:
Have a requirements.txt file in your project folder.
After you create a heroku app using "heroku create", heroku will identify your app as python based and will not download any of the node dependencies.
Now, go to your heroku profile and go to your app's settings. There is an option named "Add Buildpack" in there. Click on that and add node.js as one of the buildpacks.
Go back to your project and push it to heroku. This time heroku will identify your app as both python and node.js based and will download all the required files.
Recently I'm learning Flask by the book "Flask Web Development". When I completed the code and deployed it to Heroku, the following error happened:
ImportError: /app/.heroku/python/lib/python3.6/
sitepackages/psycopg2/.libs/libresolv-2-c4c53def.5.so:
symbol __res_maybe_init version GLIBC_PRIVATE not defined
in file libc.so.6 with link time reference
However, this works fine locally. I have searched relevant questions about psycopg2, and I have adjusted the version of psycopg2 but the same error still happens. Please, how can I solve the problem?
I got the same problem. I solved it by forcing heroku to clean up the python virtual environment, and reinstall the requirements.txt file using psycopg2>=2.7,<2.8 --no-binary psycopg2. I must admit it felt a little like magic, but for what its worth here are the steps I took:
Using Git Bash for Windows, I logged in to heroku with heroku login.
Next, I navigated to the root folder of the git/project folder I was working on.
There I made sure that my requirements.txt was updated by running venv/Scripts/pip freeze > requirements.txt. You will probably have to find your own python/virtual environment path.
Next I changed the line/entry for psycopg2 to psycopg2>=2.7,<2.8 --no-binary psycopg2.
Then saved all the content of the requirements.txt file to my desktop, making sure the one in my project folder was empty. Not sure if this step was necessary. But that's what I did, so I'm just gonna mention it here.
Now in the Git Bash terminal I ran heroku apps:info -a blogglistene to find the version of the my heroku stack which for me was heroku-18.
Next I look online for the supported versions of python for heroku-18. It said the default was python-3.6.6 (which I then assumed is the one I was running), and some others.
I then picked python-2.7.15 arbitrarily from the list, and added that text to a runtime.txt file which I placed in the root of my project folder.
Next I committed and pushed this to my heroko git repo, and it was deployed. This will temporarily break the build. From what I understand, the magic behind this is that heroku now fully destroys the python interpreted and environment, so no lingering cached files or anything. Clean slate.
Then I reverted the content of runtime.txt to the default of python-3.6.6 and added back the content of the requirements.txt file, making sure the changes I made are there.
Finally, I committed and pushed this, and my application was back up and running.
I had the exact same problem when I was following "Flask Web Development". I spent a day plus trying to solve it and eventually succeeded. It's not as complicated as André C. Andersen's method.
Make sure you are on master branch. Not other branches.
The book is second edition of Flask Web Development by Miguel Grinberg. Chapter 17. Deployment. When you work all the way till Deploying with git push, after git push heroku master to upload the application to the heroku remote. The application is now deployed and running, but it is not going to work correctly because the deploy command that initializes the database tables has not been executed yet. You need to use
heroku run flask deploy
to create database on heroku. That's where the problem appears.
This is because at this point your git repo HEAD is at 17c. It's not the master branch yet. If you commit any change now, it won't change heroku environment. Meaning when you do
git branch
You will see:
(HEAD detached from 17c)
master
But this is not what you want. Not head attached to 17c. You want to see only master, which is achieved with
git checkout master
Then git branch will show only master branch. Therefore you can make adjustments on the project and heroku will update. Doing this because you will eventually git push to master branch. I think you can git push to other branches in order for heroku to work, but I don't know how to do that.
Change psycopg2==2.7.3 to the newest version, e.g. psycopg2==2.8.4. Another workaround is to change it to psycopg2-binary==2.8.3.
psycopg2 2.7.3 will give you GLIBC_PRIVATE error. (You can check it by running heroku run bash, then starting python and importing psycopg2. If heroku dependency is 2.7.3, then you will see the same error.) The official website notes that version 2.7.3.1 drops libresolv which causes the issue, therefore using this version or later should do the trick.
Now, change your requirements/heroku.txt (heroku environment dependency) line psycopg2==2.7.3 to psycopg2--2.8.4 or psycopg2-binary==2.8.3. (For some reason binary version of psycopg2 will work without issue. Don't know why, but I saw A LOT of discussion on stackover.) Then,
git add .
git commit -m "notes"
As usual.
git push heroku master
Push to heroku master branch. If you do not do step 1, then git will show Everything up-to-date. Therefore, it won't make changes because you are committing not to master. Now, you can see git is downloading the appropriate package.
You can double check if the packages are correct by opening heroku bash then pip list. You will see psycopg2 (and psycopg2-binary if you chose the workaround). Now, when you start Python, you can import psycopg2 with no problem.
Finally. heroku run flask deploy. Voila!
I was stuck on step 1, since whatever change I made with the project, heroku just didn't update.
I have a working django app running on mý localhost. It is running inside a virtual environment.
No I want to deploy the same project into a Google Compute Engine.
For that I have a question.
After I set up the production server including starting the virutal environment with vritualenv env do I need to clone in the project code from git including the env directory or only the source code including manage.py?
The process is described differently and so it is a bit confusing.
Main problem is the clarity of deploying the django app to production and the virtual environment setup using git for the code transfer.
Thank you for flow explanation.
My local structure is the following:
valuation <-- project directory w/ manage.py
valuation <-- project w/ settings.py
prophet <-- app
In my production server I have the following structure
opt/valuation <-- virtual environment
valuation <-- empty directory, [this][1] says I should clone code here
My question is what should I clone from my local project and what do keep out (mainly the manage.py, settings.py etc) so that the project will run.
Thanks.
no You don't need the clone the env folder, just make the requirements.text file , which will track all the plugins used in that project.You can update the requirements file using command
pip freeze > requirements.text
on the server just create the new env and install all the plugins using below command
pip install -r requirements.text