AWS - install python packages while deploying EC2 server - python

i am currently working on a project where I need to deploy python application on my EC2 instance. I cannot seem to figure out how to install packages on my EC2 instance during application installation. I have tried installation via "requirements.txt" file
mysql-connector-python==8.0.22
as well as via .ebextenstion/01_packages.config file that looks as following:
packages:
yum:
mysql-connector-python: '8.0.22'
requests: '2.24.0'
BUT every time is deploy new version of my application i get an error and updated list of packages is not installed.
Any assistance is highly appreciated.

I verified on EB Python 3.7 AL2 version 3.1.4 that the following requirements.txt works:
mysql-connector-python==8.0.22
requests==2.24.0
Thus if you have other issues, they are not caused by the requirements.txt on that platform.

Ended up creating setup.config file in my .ebextentions, where I configured command to install required packages.
commands:
install_requests:
command: "pip install requests"
install_wheel:
command: "pip install wheel"
install_mysql-connector-python:
command: "pip install mysql-connector-python"

Related

How to force install all dependencies in virtualenv?

I am trying to deploy my flask app on AWS Lambda by using Zappa. I carefully followed all the tutorials but I always get deployment error :
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
I entered zappa tail and I figured out that the dependencies are not installed even after I made pip freeze > requirements.txt . The problem is, there are a lot of dependencies and it really boring to install each of them manually.
Is there any magic command that will allow me to automatically install all dependencies in the virtualenv ?
Why don't you just run
pip install -r requirements.txt?

Azure app service cannot find installed modules

I am attempting to deploy a Django app to a Linux server via an Azure App Service. During the deployment via Azure Devops Pipelines, all requirements are installed from my requirements.txt file in the root directory of my project.
I have used the Kudu console to confirm the dependencies are installed to /antenv/lib/python3.7/site-packages on the server, however, the app crashes due to an error:
ModuleNotFoundError: No module named 'django'
I am beginning to think the virtual environment may be failing to actually start but do not know how to check or how to start it if this is the case.
Has anyone had a similar issue to this during their deployment? If so how did you resolve it? Any advise is much appreciated. Thank you!
Newest
Change target path ,--target="./.python_packages/lib/site-packages" .
- bash: |
python3.8 -m venv worker_venv
source worker_venv/bin/activate
pip3.8 install setuptools
pip3.8 install --target="./.python_packages/lib/site-packages" -r requirements.txt
displayName: 'Install Application Dependencies'
You need to install a new Python runtime at the path D:\home via Kudu site extensions.(Windows)
The problems is, azure app service use virtualenv by default, so the requirements.txt package is automaticly installed to python in the virtualenv... so I just edit the deploy.cmd to install requirements to python (extension)
For more details, you can refer Peter Pan's answer in below post.
Why is the azure app service django deploy keep failing?

AWS Lambda with Zappa fails on "import module 'handler': No module named 'werkzeug' "

After deploying my python application with Zappa and visiting the aws link, I can see the following error:
When I checked the logs, I found the source of the error:
I then decided to pip install -r requirements.txt to ensure that it's installed inside my virtual environment (which it is):
Requirement already satisfied: Werkzeug==0.12 in ./flaskapi/lib/python3.6/site-packages (from -r requirements.txt (line 41))
Something is going wrong when it's uploaded to AWS. I'm not sure if this is the core issue, but I did notice the package name in the logs is different than the one inside requirements.txt file. The package name in the logs doesn't start with a capitalized 'W', while the package name in the requirements.txt does. Other than that, I'm not sure what I'm doing wrong.
Any and all help is appreciated
I solved this issue by upgrading to python 3.7 . I would recommend starting a new virtualenv, configured to use python 3.7 .
if you do not have python3.7 on your system, you will need to install it. This site is the one I used. Works on AWS cloud9 too.
installing python 3.7
virtualenv env -p python3.7
source ./env/bin/activate
python --version
output should be "Python 3.7.X"
then continue setting up your app like normal.

How to pip install bcrypt on Azure webapp?

Disclosure : First time Azure experience
I am deploying a Flask app to Azure Webapp. All deployment steps are fine till I hit the bcrypt package installation and it fails.
After much research based on error log output, I found out that I might need to install bcrypt using wheelhouse (*.WHL)
I downloaded the below files from here
bcrypt-3.1.0-cp27-cp27m-win32.whl
bcrypt-3.1.0-cp27-cp27m-win_amd64.whl
and I copied them to D:\home\site\repository\wheelhouse
Then, I activated the virtualenv through KUDU and I run this command:
d:\home\site\wwwroot\env\scripts\pip install -r requirements.txt --find-links d:\home\site\repository\wheelhouse
I get no messages or any log output. When I run a pip freeze > tmp.txt I get a blank file.
But when I run d:\python27\scripts\pip install -r requirements.txt --find-links d:\home\site\repository\wheelhouse
It starts installing the packages until it gets to bcrypt and it errs out with this message:
Skipping bcrypt-3.1.0-cp27-cp27m-win32.whl because it is not compatible with
this Python
Which is a confusing message because the wheel is for Python 2.7
Since my Flask app works fine on both my Linux and Windows dev machines, I went ahead and created my own brypt wheel file in my Windows computer which runs the same Python version on Azure. I uploaded the new .whl, redid the steps above and I still get the same error message
Extra notes:
python -V on Azure console returns 2.7.8
python -V while virtualenv is activated also returns 2.7.8
Azure portal > Application Settings shows "Python version 2.7" and "Platform 32bits".
After deployment, all packages in requirements.txt are installed except bcrypt.
Visiting the web page gives a 500 error (which i expect due to missing lib)
I removed the virtualenv and GIT pushed the repo with --find-links wheelhouse at the top of requirements.txt as stated here.
However, I get a Unable to find vcvarsall.bat error. That's why I m trying to manually install via wheel
I deleted the whole virtualenv, uploaded wheel files for all required packages to \repository\wheelhouse and added --no-index to my pip install command. Everything gets installed except bcrypt.
I tried bcrypt==3.1.1, bcrypt==3.1.0 and just bcrypt without specifying the version and it doesnt make any difference.
I ran out of ideas. Anyone knows what's wrong? How do I go about installing bcrypt on Azure webapp?
I was finally able to get the Flask app working on Azure Webapps.
Unfortunately, I couldn't do it using my usual dev tools.
Solution:
I created a VirtualEnv in Visual Studio using my requirements.txt file
Moved my Flask code to Visual Studio
Click on Publish to Azure Webapps
It does what it does and once completed, you may still get a 500 error.
If that is the case, use KUDU and take a look at your web.config file and modify the WSGI_ALT_VIRTUALENV_HANDLER value to match your Flask app name.
This is the only way I was able to get bcrypt to install correctly for my Flask app to work. I hope this saves someone valuable time.
That's all folks.
You could have tried using easy_install to install dependencies that failed to do so by pip. What I did was :
Go into KUDU debug Console. This is listed under Development Tools
Try executing env\scripts\pip install -r requirements.txt
For every failed dependency (as bcrypt), try and install it through env\scripts\easy_install bcrypt==3.1.0
You could also try and automate this by editing the deploy.cmd file. Such that at any point that pip fails, easy_install is executed

Can't install Apns on my debian GCE instance

I have a GCE instance with Debian wheezy on it, I'm running my phython/Django code (my beckend) for an IOS application.
I'm trying to use a TSL push notification library but I'm failing to install it
sudo easy_install apns -
got an error: Setup script exited with error: file '/tmp/easy_install-sOkSjl/apns-2.0/apns-send' does not exist
sudo pip install apns -
got an Exception & error: [Errno 104] Connection reset by peer
sudo pip install pyapns -
didn't work either
if I copy it to my GCE instance how do I install & configure it, in Github documentation only states how to use easy_install.
When I was trying to install apns using pip, I also got the "apns-send does not exist" message. This was solved by downloading this script from their git repo and placing it in the build folder (in my case, build/apns).
For some reason, it seems it is not included in the version on PyPI.

Categories

Resources