Develop Reference

Failing to deploy a Django web app to AWS beanstalk

I am trying to deploy A django app to beanstalk and I get some errors related to python and requirements.txt and I can't figure out what to do, Any help is appreciated.
here are the errors I get: (the logs are in pastebin bellow)
ERROR
[Instance: i-0e7826c4558b1d21a] Command failed on instance. Return code: 1 Output: (TRUNCATED)...) File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2022-04-28 15:13:52 UTC+0000
ERROR
Your requirements.txt is invalid. Snapshot your logs for details.
logs:
https://pastebin.com/g1uZLqur
here is my requirements.txt
Django==2.2.6
django-environ==0.4.5
pytz==2019.3
sqlparse==0.3.0
django-paypal==1.0.0
https://github.com/darklow/django-suit/tarball/v2
git+https://github.com/nn3un/reportlab-mirror#egg=reportlab
boto3==1.4.4
django-storages==1.8
psycopg2
django-guardian
bokeh==1.4.0
django-ses==2.0.0
mock

How to install pythonocc-core on Amazon Linux 2 AMI?

I have a Django application which it's deployed to Elastic Beanstalk, Amazon Linux 2 AMI and I'm very new with AWS. I have to use pythonocc-core package in my application and I cannot install it via pip. If I SSH into an instance and install it manually, it won't be a great way because of autoscaling of EB environment. I added the .config file below for installing anaconda and my deployment was successful.
commands:
00_download_conda:
command: 'wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh'
test: test ! -d /anaconda
01_install_conda:
command: 'bash Anaconda3-2020.02-Linux-x86_64.sh -b -f -p /anaconda'
test: test ! -d /anaconda
02_create_home:
command: 'mkdir -p /home/wsgi'
Deployment failed when I added the following command lines to the continuation of this .config file.
commands:
03_conda_create:
command: conda create --name=whatever-name python=3.7
04_conda_activate:
command: source activate whatever-name
05_conda_install:
command: conda install -c dlr-sc pythonocc-core=7.4.0
But I've got an error such as below.
Traceback (most recent call last):
File "/opt/aws/bin/cfn-init", line 171, in <module>
worklog.build(metadata, configSets)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 129, in build
Contractor(metadata).build(configSets, self)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 530, in build
self.run_config(config, worklog)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 542, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 260, in build
changes['commands'] = CommandTool().apply(self._config.commands)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/command_tool.py", line 117, in apply
raise ToolError(u"Command %s failed" % name)
ToolError: Command 03_conda_create failed
How can I fix this issue?

A good way would be to use Commands in your .ebextensions folder.
For example, you could have a file .ebextensions/10_install_pythonocc.config:
commands:
10_conda_create:
command: conda create --name=whatever-name python=3.7
20_conda_activate:
command: source activate whatever-name
30_conda_install:
command: conda install -c dlr-sc pythonocc-core=7.4.0
Regarding anaconda itself, here are instructions that can be tried.

Elastic Beanstalk with python 3.4 still using python 2.7

I just spin up some environment using EB with python 3.4 and Django but it keeps failing, looks like the error occurs when installing using pip install -r requirements.txt this are the events from the web console:
Time Type Details
2017-10-06 20:22:39 UTC-0600 WARN Environment health has transitioned from Pending to Degraded. Command failed on all instances. Initialization completed 69 seconds ago and took 14 minutes.
2017-10-06 20:22:20 UTC-0600 ERROR Create environment operation is complete, but with errors. For more information, see troubleshooting documentation.
2017-10-06 20:21:17 UTC-0600 INFO Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2017-10-06 20:21:17 UTC-0600 ERROR [Instance: i-0b46caf0e3099458c] Command failed on instance. Return code: 1 Output: (TRUNCATED)...) File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2017-10-06 20:21:14 UTC-0600 ERROR Your requirements.txt is invalid. Snapshot your logs for details.
I followed this tutorial: django-elastic-beanstalk-django and deploying-a-django-app-and-postgresql-to-aws-elastic-beanstalk on both I'm stuck at the same step

I had an exact same problem. Here is the solution I found:
Go to your env's configuration
Click the conf button of Instances
Under Server section, check your Custom AMI ID
For now, leave the page and go to AWS EC2 Console page
Press Launch Instance
You will now be on Step 1: Choose an Amazon Machine Image (AMI)
Tada! You will see these ami ids that you can choose from:
Go back to no.3 and fill up the Custom AMI ID, refer from no.6
(FYI, I used amazon linux, but it is up to you to decide)
Click Apply and wait till the configuration is done
Re-follow the deploying django app
DONE!

Not able to deploy the django app on elastic beanstalk

When trying to deploy my django app using eb deploy I am getting the following error. It is totally unseful and does not let me knwo what to do. Please help.
I am following the tutorial here : http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-django.html
sh-3.2# eb deploy
Creating application version archive "app-ca0a-151123_205119".
Uploading learningsite/app-ca0a-151123_205119.zip to S3. This may take a while.
Upload Complete.
INFO: Environment update is starting.
INFO: Deploying new version to instance(s).
ERROR: Your requirements.txt is invalid. Snapshot your logs for details.
ERROR: [Instance: i-18ee63c1] Command failed on instance. Return code: 1 Output: (TRUNCATED)...)
File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1.
Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
ERROR: Unsuccessful command execution on instance id(s) 'i-18ee63c1'. Aborting the operation.
ERROR: Failed to deploy application.
ERROR: Failed to deploy application.
sh-3.2#

welcome to EB! You'll get plenty of hard to decrypt error messages.
The important line here is ERROR: Your requirements.txt is invalid. Snapshot your logs for details. There is something wrong with your requirements.txt file and EB is not able to install.
Locally if you run pip install -r requirements.txt you might be able to see the error.

Invalid requirements.txt on deploying django app to aws beanstalk

I have been deploying my app without any issues to the beanstalk environment until my latest commit.
Now I get this
Time Type Details
2013-09-01 10:19:12 UTC+0800 ERROR Failed to deploy application.
2013-09-01 10:19:11 UTC+0800 ERROR Responses from [i-50f40d06] were received, but the commands failed.
2013-09-01 10:19:11 UTC+0800 ERROR Your requirements.txt is invalid. Snapshot your logs for details.
2013-09-01 10:19:11 UTC+0800 ERROR [Instance: i-50f40d06 Module: AWSEBAutoScalingGroup ConfigSet: Hook-PreAppDeploy] Failed on instance with return code: 1 Output: Error occurred during build: Command hooks failed .
The snapshot logs say this:
2013-09-01 02:19:08,852 [INFO] (9941 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Output from script: New python executable in /opt/python/run/venv/bin/python2.6
Not overwriting existing python script /opt/python/run/venv/bin/python (you must use /opt/python/run/venv/bin/python2.6)
Installing distribute..................................................................................................................................................................................................done.
Installing pip................done.
2013-09-01 02:19:08,564 ERROR Error installing dependencies: Command '/opt/python/run/venv/bin/pip install --use-mirrors -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status -9
Traceback (most recent call last):
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 31, in main
install_dependencies()
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 25, in install_dependencies
shell=True)
File "/usr/lib/python2.6/subprocess.py", line 502, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install --use-mirrors -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status -9
2013-09-01 02:19:08,853 [ERROR] (9941 MainThread) [directoryHooksExecutor.py-34] [root directoryHooksExecutor error] Script failed with returncode 1
2
My requirements.txt is the same as before. I tried removing everything in the requirements.txt as well but I get the same error again which makes me think the problem was before requirements.txt was run.
I don't know the order in which things are deployed. ANyway, here is my config file:
packages:
yum:
libjpeg-devel: '6b'
zlib-devel: []
freetype-devel: []
container_commands:
01_install_mysqldb:
command: "pip install distribute==0.6.28; pip install mysql-python;"
02_syncdb:
command: "python manage.py syncdb --noinput"
leader_only: true
03_createadmin:
command: "python scripts/createadmin.py"
leader_only: true
04_collectstatic:
command: "python manage.py collectstatic --noinput"
05_migrate_wizards:
command: "python manage.py migrate wizards --noinput"
leader_only: true
06_migrate_facebook:
command: "python manage.py migrate facebook --noinput"
leader_only: true
07_migrate_socialaccount:
command: "python manage.py migrate socialaccount 0011 --noinput"
leader_only: true
08_migrate_missions:
command: "python manage.py migrate missions --noinput"
leader_only: true
09_migrate_mailchimp:
command: "python manage.py migrate mailchimp --noinput"
leader_only: true
10_migrate_actstream:
command: "python manage.py migrate actstream --noinput"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: ideatory/wsgi.py
- namespace: aws:elasticbeanstalk:container:python:staticfiles
option_name: /static/
value: static/
- option_name: DJANGO_SETTINGS_MODULE
value: ideatory.settings

I had the exact same problem. The only thing that worked for me was (automatically) rebuilding my entire environment. You can do that in the AWS console. Go to your environment and click Actions > Rebuild Environment. This will take some time, afterwards it will automatically relaunch your app without this error.
Update:
The problem keeps returning so now and then. I figured out that something is going wrong when pip compiles psycopg, which is needed for postgreSQL support. Haven't found a real solution yet.

I was able to make this work by adding postgresql93-devel and postgresql93 to my .ebextensions
as described in Customizing the Software on EC2 Instances Running Linux
This is working for 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
in .ebextensions/01_pre_deps.config:
packages:
yum:
gcc-c++: []
make: []
openssl-devel: []
git: []
python-devel: []
python27-devel: []
ruby-devel: []
postgresql93: []
postgresql93-devel: []
# nginx: []
rubygems:
bundler: '1.7.3'
compass-core: '1.1.0.alpha.3'
neat-compass: '1.6.0'

Have you made sure mysql-devel package is installed? It doesn't seem to be included in the yum section of your config file.

Although a different scenario than described by the OP, it may be of use to someone:
We got the same ERROR Your requirements.txt is invalid. Snapshot your logs for details. message while trying to deploy a minimal test app to a newly created t1.micro instance on Elastic Beanstalk.
The test app consisted of the basic AWS Python sample app (from here), to which we added the .ebextensions/software.config and requirements.txt from our production environment. The sole purpose was to test these two files in deployment.
Closer inspection of the logs uncovered a MemoryError during pip install. After upgrading to a t2.small instance, the deployment succeeded without issue.

For whatever reason AWS wouldn't install postgresql-devel packets from my .extensions/01_packets.config as yolk suggested. So instead I had to add
container_commands:
01_addPostgreSQLPacket:
command: 'yum install postgresql95-devel'
leader_only: true
to my 02_django.config. After That it worked just fine.