I have issues deploying my Django application to AWS in the free tier during the pip installation of my requirements.txt.
As I could see here the issue seems to be related to scipy:
AWS Elastic Beanstalk failed to install Python package using requirements.txt Git Pip
The question has been resolved as he could uppgrade to t2.medium but this requires a paid account.
Is there any tip for the free tier?
Could an option like '--no-cache-dir' make it work ? Is there a way to force it (as the pip install is automatically done)
Is there any other way to deploy a Django application on AWS ?
EDIT: It is actually because of psycopg2..
Collecting psycopg2==2.8.4 (from -r /opt/python/ondeck/app/requirements.txt (line 4))
Downloading https://files.pythonhosted.org/packages/84/d7/6a93c99b5ba4d4d22daa3928b983cec66df4536ca50b22ce5dcac65e4e71/psycopg2-2.8.4.tar.gz (377kB)
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
/usr/lib64/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'project_urls'
warnings.warn(msg)
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
As per #phd comment : pip install psycopg2-binary did the trick instead of pip install psycopg2
Although I like the simplicity of the accepted solution, the psycopg2-binary installation notes advise against using it for production:
The binary package is a practical choice for development and testing but in production it is advised to use the package built from sources.
I don't know if this really is a big issue, but, for the sake of completeness, I'll add an alternative:
The Error: pg_config executable not found. can also be fixed by installing postgresql-devel using yum.
On Amazon Linux 2 you could do this using .platform hooks. For example, in you app repository (next to your .ebextensions), add a .platform/hooks/prebuild/01_install_yum_packages.sh file with the following content:
#!/bin/bash
yum -y install postgresql-devel
As described in the docs, you have to set execution permissions on this file before committing it to you repo. E.g. chmod +x 01_install_yum_packages.sh.
Related
I'm developing my own conda package. I'm using a setup.py in the process of generating that package. When developing, it can be useful to conda install --no-deps package and ./setup.py develop in the repo. The install works, but setup.py errors out because it cannot satisfy a dependency. There is another package with the same name but to low version on pypi. However, the correct version of this dependency is already installed via conda. Why does it first try to install the package before checking if it is already installed? How can I make setup.py realize that nothing needs to be installed?
How is the package built?
I have this blt.bat:
"%PYTHON%" setup.py install --single-version-externally-managed --record=record.txt
if errorlevel 1 exit 1
and the dependencies listed in the meta.yaml. It falls to setup.py to define the package version (read from a .py file), define the entry points and the included non-Python files. It gets the dependencies by parsing the meta.yaml.
What is the error message?
This is the output of setup.py develop:
$ python setup.py develop
running develop
C:\ProgramData\Miniconda3\envs\my_env\lib\site-packages\setuptools\command\easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
C:\ProgramData\Miniconda3\envs\my_env\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running egg_info
writing my_package.egg-info\PKG-INFO
writing dependency_links to my_package.egg-info\dependency_links.txt
writing entry points to my_package.egg-info\entry_points.txt
writing requirements to my_package.egg-info\requires.txt
writing top-level names to my_package.egg-info\top_level.txt
reading manifest file 'my_package.egg-info\SOURCES.txt'
writing manifest file 'my_package.egg-info\SOURCES.txt'
running build_ext
Creating c:\programdata\miniconda3\envs\my_env\lib\site-packages\my_package.egg-link (link to .)
my_package 6.1.0 is already the active version in easy-install.pth
Installing entry_point0-script.py script to C:\ProgramData\Miniconda3\envs\my_env\Scripts
Installing entry_point0.exe script to C:\ProgramData\Miniconda3\envs\my_env\Scripts
Installing entry_point1-script.py script to C:\ProgramData\Miniconda3\envs\my_env\Scripts
Installing entry_point1.exe script to C:\ProgramData\Miniconda3\envs\my_env\Scripts
Installed c:\users\jpoppinga\my_git_repository
Processing dependencies for my_package==6.1.0
Searching for my_dep<13.1,>=13.0.1
Reading https://pypi.org/simple/my_dep/
C:\ProgramData\Miniconda3\envs\my_env\lib\site-packages\pkg_resources\__init__.py:123: PkgResourcesDeprecationWarning: is an invalid version and will not be supported in a future release
warnings.warn(
No local packages or working download links found for my_dep<13.1,>=13.0.1
error: Could not find suitable distribution for Requirement.parse('my_dep<13.1,>=13.0.1')
We want to exclude the Python source code from the package we create. But after configuring setup.py, I failed excluding the py files. I have been using python setup.py bdist_wheel as command. Is there any way to exclude to source code from Python package? Basically we do not want to expose the source codes.
The wheel plugin for setuptools that handles bdist_wheel does not have the --exclude_source_files which is only supported by bdist_egg. The egg packages are however deprecated and not supported by pip for example. What you can do however:
pip3 install wheel
python3 setup.py bdist_egg --exclude-source-files
wheel convert dist/mypackage-1.0-py3.6.egg
The wheel utility takes a source-stripped egg and converts it into whl package.
This came up for us putting Python libs on an embedded system with very minimal available memory.
It can be achieved with:
./setup.py bdist_egg --exclude_source_files
Works for both distutils and setuptools.
Well, we also encountered this issue (around 2 years ago) and didn't find a sensible automation process for it. So I wrote my own.
You're welcomed to it: setup.py template
Both of the following commands successfully install my package without error.
pip install git+https://path_to_repo/repo_name.git#v17.8.0
pip install git+https://path_to_repo/repo_name.git#v17.8.0#egg=repo_name
What is the difference?
I'm using pip 7.1.0 and 9.0.1
Working Out the Name and Version
For each candidate item, pip needs to know the project name and
version. For wheels (identified by the .whl file extension) this can
be obtained from the filename, as per the Wheel spec. For local
directories, or explicitly specified sdist files, the setup.py
egg_info command is used to determine the project metadata. For sdists
located via an index, the filename is parsed for the name and project
version (this is in theory slightly less reliable than using the
egg_info command, but avoids downloading and processing unnecessary
numbers of files).
Any URL may use the #egg=name syntax to explicitly state the project name.
First of all, I am running Python 2.7.5 on a Mac. I am trying to install a package and can't get it to work despite different approaches. The package is called python-fs-stack and is available here. https://pypi.python.org/pypi/python-fs-stack. I tried pip install python-fs-stack, sudo pip install python-fs-stack, easy_install python-fs-stack, sudo easy_install python-fs-stack and nothing worked. I then downloaded the package and got an error about a README.rst not being found. I commented out this line in the setup.py file and re-ran it. It got a lot farther, but then there was another error. Here is the output:
>>sudo python /Downloads/python-fs-stack-0.2/setup.py install
running install
running bdist_egg
running egg_info
writing python_fs_stack.egg-info/PKG-INFO
writing top-level names to python_fs_stack.egg-info/top_level.txt
writing dependency_links to python_fs_stack.egg-info/dependency_links.txt
warning: manifest_maker: standard file 'setup.py' not found
error: package directory 'familysearch' does not exist
I would really like to be able to get this package up and running, but I am at a loss. What should I try?
I'm trying to start a Django app on Heroku using Windows and I'm getting stuck on the following error when I try to pip install psycopg2:
Downloading/unpacking psycopg2
Downloading psycopg2-2.4.5.tar.gz (719Kb): 719Kb downloaded
Running setup.py egg_info for package psycopg2
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info\psycopg2.egg-info
writing pip-egg-info\psycopg2.egg-info\PKG-INFO
writing top-level names to pip-egg-info\psycopg2.egg-info\top_level.txt
writing dependency_links to pip-egg-info\psycopg2.egg-info\dependency_links.txt
writing manifest file 'pip-egg-info\psycopg2.egg-info\SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
I've googled the error and it seems you need libpq-dev python-dev as dependencies for postgres under Python. I also turned up a link that says you gt into trouble if you don't have the postgres bin folder in your Path so I installed Postgres manually and tried again. This time I get:
error: Unable to find vcvarsall.bat
I am still a python N00b so I am lost. Could someone point me in a general direction?
You can use Cygwin and install all the dependencies, but I went through that issue last week and then I realized it was far easier to use a Virtual Box with Ubuntu as my Guest OS. I still did the development on Windows but used Ubuntu just to communicate/push to heroku
I found the answer to my problem. I believe the problem is that pip is looking to compile the dependency from source. The solution is to find a link to pre-compiled version of the dependency. You can then pass this link to the easyinstall library installed in your virtualenv. Easyinstall will download and install the pre-compiled version of the dependency.