How to solve dependency conflicts in requirements.txt file in Django deployment - python

I am trying to deploy my Django app on GCP using google appengine.
First I deployed the app after testing on localhost by following this documentation by Google appengine.
deployed the app using gcloud app deploy
But there is some issue and the server is not running showing the error as 502 Bad Gateway
Then I checked the logs and then realized that I forgot to upload the requiremts.txt file.
the uploaded file and tried to deploy the app again.
But got an error as
ERROR: Cannot install -r requirements.txt (line 19), -r requirements.txt (line 21), -r requirements.txt (line 27) and grpcio==1.48.1 because these package versions have conflicting dependencies.
Here is some dependency conflict between modules Gcloud suggested a documentation https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts to solve this but I'm not getting it actually how to solve the conflict of modules in requirements.txt
Here is the requirements.txt file
APScheduler==3.6.3
asgiref==3.5.2
backports.zoneinfo==0.2.1
beautifulsoup4==4.11.1
cachetools==4.2.2
certifi==2022.6.15
charset-normalizer==2.1.1
dill==0.3.5.1
Django==4.0.6
django-environ==0.9.0
django-social-share==2.3.0
environ==1.0
google==3.0.0
google-api-core==2.10.0
google-auth==2.11.0
google-cloud-secret-manager==2.12.4
google-cloud-speech==2.15.1
googleapis-common-protos==1.56.4
grpc-google-iam-v1==0.12.4
grpcio==1.48.1
grpcio-status==1.48.1
idna==3.3
Pillow==9.2.0
proto-plus==1.22.1
protobuf==4.21.5
psycopg2==2.9.3
pulumi==3.39.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pytz==2022.2.1
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
requests==2.28.1
rsa==4.9
semver==2.13.0
six==1.16.0
soupsieve==2.3.2.post1
sqlparse==0.4.2
tornado==6.2
tzdata==2022.1
tzlocal==4.2
urllib3==1.26.12
And an error log
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build 7ac526d7-15d5-45f6-afe7-727b4216f79d status: FAILURE
.... This could take a while.
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of google to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of environ to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of django-social-share to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of django-environ to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of django to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of dill to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of charset-normalizer to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cachetools to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of backports-zoneinfo to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of asgiref to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of apscheduler to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 19), -r requirements.txt (line 21), -r requirements.txt (line 27) and grpcio==1.48.1 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested grpcio==1.48.1
grpc-google-iam-v1 0.12.4 depends on grpcio<2.0.0dev and >=1.0.0
grpcio-status 1.48.1 depends on grpcio>=1.48.1
pulumi 3.39.3 depends on grpcio==1.47
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
Please help me to solve this error
Thanks

I would use something like poetry to manage your dependencies while you work on your app. It will actually help you not to run into conflicts and solve most of them for you automatically.
Then when you are ready to deploy your app you can do:
poetry export -f requirements.txt --output requirements.txt
And poetry will generate a requirements.txt file that you can upload to GCP

Related

Pre-release version number in mixed python/rust project with maturin

I am working on a mixed Rust/Python project being build with maturin. I am having trouble figuring out how I can specify a pre-release version that can than be installed by pip.
The start of my Cargo.toml is
[package]
version = "0.0.1-test.1"
When I do maturin build --release, it creates my_project-0.0.1_test.1-cp39-cp39-macosx_10_7_x86_64.whl. Note that 0.0.1-test.1 in the version was changed to 0.0.1_test.1 in the name of the wheel file. (The dash became an underscore.)
If I publish this to my Nexus server and try to install it in a separate project, pip gives an error:
$ pip install my_project==0.0.1_test.1
Looking in indexes: ...
ERROR: Could not find a version that satisfies the requirement my_project==0.0.1_test.1 (from versions: 0.0.1-test.1, 0.0.1.dev1)
If I try it with a dash instead of an underscore:
pip install py_mod_query_builder==0.0.1-test.1
Looking in indexes: ...
ERROR: Exception:
<stack trace>
pip._vendor.packaging.version.InvalidVersion: Invalid version: '0.0.1-test.1'
If I change the version in Cargo.toml to 0.0.1_troy.1, then Cargo fails to parse the build file:
$ maturin build --release
🍹 Building a mixed python/rust project
💥 maturin failed
Caused by: Cargo metadata failed. Do you have cargo in your PATH?
Caused by: Error during execution of `cargo metadata`: error: failed to parse manifest at `/Users/me/my-project/Cargo.toml`
Caused by:
unexpected character '_' after patch version number for key `package.version`
I did have some success with pip install path/to/my_project*.whl. However, that is difficult to incorporate into building a docker image. The wheel is outside the project with the Dockerfile, so docker will not be able to find it without jumping through a bunch of hoops, while downloading it from the server during the docker build is simple, but fails due to the dash/underscore problem.
Additionally, while both projects are being developed, I would like the CI pipeline for the development branch to use the pre-release version. Getting access to the wheel file (without having pip download it) is nearly impossible in that situation.
It seems that Cargo insists on the format being #.#.#-string and pip insists on it being #.#.#_string. Is there a way to make both of them happy?
Versions:
Installing project
pip 22.1
python 3.9.12
Building project
pip 20.0.4
python 3.9.12
maturin 0.8.3
rustc 1.56.1 (59eed8a2a 2021-11-01)

can't find my packages from pycharm/heroku

I have deployed a package into https://pypi.org/project/core-lib/0.0.1.dev1/
But pycharm or heroku can't find my package with this error message
Can you help me, please?
Thank you
Pycharm
No matching package version found: 'core-lib==0.0.1.dev1' (required: ==0.0.1.dev1, installed: <nothing>, latest: <nothing>)
heroku deploy
-----> Installing requirements with pip
ERROR: Could not find a version that satisfies the requirement core-lib==0.0.0.8 (from -r /tmp/build_c6b601b1/requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for core-lib==0.0.0.8 (from -r /tmp/build_c6b601b1/requirements.txt (line 1))
! Push rejected, failed to compile Python app.
! Push failed
Installed your package from pypi using PyCharm, and I am able to find it
(randomtesting) C:\Users\User\PycharmProjects\randomtesting>pip list
Package Version
---------------------- -------------------
core-lib 0.0.1.dev1
Perhaps the file was released into pypi very recently?
And I checked the link again, now it has the latest version.
Maybe try this?
pip install core-lib
You've declared that your package must be used under Python >= 3.7. Please make sure you use Python with high enough version. For example, I tried to install the package with Python 2.7 and pip failed to find the project. With Python 3.7 it was installed successfully.

Pip: could not find a version. No matching distribution found

I'm trying to install Flask-ACL:
https://mikeboers.github.io/Flask-ACL
$ pip search acl | grep -i flask
Flask-ACL (0.0.1) - Access control lists for Flask.
flask-miracle-acl (0.2) - The fabric between the Flask framework and Miracle ACL
Flask-Sandbox (0.1.0)- ACL Route controls for Flask
Flask-SimpleACL (1.2)- Simple ACL extension
$ pip install Flask-ACL
Collecting Flask-ACL
Could not find a version that satisfies the requirement Flask-ACL (from versions: )
No matching distribution found for Flask-ACL
What's wrong here?
P.S. Pip was upgraded a few minutes ago.
UPDATE:
$ python --version
Python 2.7.3
I'm running it under virtualenv.
pip install -Iv Flask-ACL
Collecting Flask-ACL
1 location(s) to search for versions of Flask-ACL:
* https://pypi.python.org/simple/flask-acl/
Getting page https://pypi.python.org/simple/flask-acl/
Looking up "https://pypi.python.org/simple/flask-acl/" in the cache
Current age based on date: 507
Freshness lifetime from max-age: 600
Freshness lifetime from request max-age: 600
The response is "fresh", returning cached response
600 > 507
Analyzing links from page https://pypi.python.org/simple/flask-acl/
Could not find a version that satisfies the requirement Flask-ACL (from versions: )
Cleaning up...
No matching distribution found for Flask-ACL
As I can see, there is no such package on Pypi:
https://pypi.python.org/simple/flask-acl/
but this one exist:
https://pypi.python.org/pypi/Flask-ACL
What is wrong with my pip?
The developers of Flask-ACL made a mistake that they did not manage to upload the Flask-ACL library onto PyPi(where pip searches for modules). so you will have to install it using pip from their GitHub page.
You can do so like this:
pip install "git+https://github.com/mikeboers/Flask-ACL"
A more common reason for this type of error is probably that your environment does not meet the package's requirements.
For example, this occurs if your environment's Python version does not match the required Python version specified by the package:
(my_python36_env)> pip install package_that_requires_python37
ERROR: Could not find a version that satisfies the requirement package_that_requires_python37.
ERROR: No matching distribution found for package_that_requires_python37.
Pay attention if your file ~/.pip/pip.conf not contain any additional index-url values.
Then it can trying to look for specific package version in repository that not exists.
Some workaround solution is to remove pip.conf
I had flask installed in the anaconda but it had some issues when it import it using
from tensorflow.keras.applications.imagenet_utils import preprocess_input, decode_predictions
so instead of the above command, i changed it to
from tensorflow.python.keras.applications.imagenet_utils import preprocess_input, decode_predictions
and it worked fine

Pip installs old version of package [duplicate]

This question already has answers here:
Why is pip installing an old version of my package?
(15 answers)
Closed 2 years ago.
Pip installs old version of my package
$ pip install pywps
Collecting pywps
Downloading pywps-3.2.6.tar.gz (123kB)
...
If you go to https://pypi.python.org/pypi/pywps/ the version 3.2.6 is not even mentioned there.
It's only mentioned at https://pypi.python.org/simple/pywps/
Any idea, why 3.2.6 is prefered over 4.0.0?
Thanks
P.S. Older responses do not seem to apply to this case.
If you run the command with a verbose option (and an up to date pip), you can see what's happening.
pip install -v pywps
It gives the following output:
1 location(s) to search for versions of pywps:
* https://pypi.python.org/simple/pywps/
...
Skipping link https://pypi.python.org/packages/f9/93/5c2c4c95e53b6193bf239ecc49cb859fd77d181311145edd13ba4cd39e09/pywps-4.0.0-py3.5.egg#md5=338eb2e56a36abc684800961b7e4ee0a (from https://pypi.python.org/simple/pywps/); unsupported archive format: .egg
...
Found link https://pypi.python.org/packages/c8/e6/8b88bc134f714f73e296466ab6b5b5a5ad96c44d35dcbcf41ccf9b76a283/pywps-3.2.6.tar.gz#md5=32bbbefacce633baa9147c74e4416c98 (from https://pypi.python.org/simple/pywps/), version: 3.2.6
Using version 3.2.6 (newest of versions: 3.2.6)
"GET /packages/c8/e6/8b88bc134f714f73e296466ab6b5b5a5ad96c44d35dcbcf41ccf9b76a283/pywps-3.2.6.tar.gz HTTP/1.1" 200 123280
Downloading pywps-3.2.6.tar.gz (123kB)
The egg file is ignored because .egg files are not supported in pip. They were supported by easy_install. I think the best solution is to start producing wheel files, which I think for pywps can be universal. If you also want to keep supporting the easy_install command, you can upload the tar.gz file or keep producing eggs.
You can install version 4.0.0 via pip straight from github:
sudo pip install git+https://github.com/geopython/pywps.git#master#egg=pywps

What can I do to speed up deb pkg building with dh-virtualenv?

I am looking for techniques to speed up building a deb pkg from my python application:
Source: my-shiny-app
Section: python
Priority: extra
Maintainer: me <me_write_me#me.com>
Build-Depends: debhelper (>= 9), python, dh-virtualenv (>= 0.6-1)
Standards-Version: 3.9.5
Package: my-shiny-app
Architecture: amd64
Pre-Depends: dpkg (>= 1.16.1), python2.7, ${misc:Pre-Depends}
Depends: ${python:Depends}, ${misc:Depends}, libzmq1
Description: my-app
The most of the time takes building python libraries that my application depends on.
UPDATE:
I have found a ticket in dh-virtualenv github. They suggest to use pip7. Pip7 supports local caching of wheel packages (copy&paste from the ticket):
#!/usr/bin/make -f
%:
dh $# --with python-virtualenv
WHEEL_ROOT:= file:///var/wheelhouse
override_dh_virtualenv:
pip wheel -w=$(WHEEL_ROOT) -f $(WHEEL_ROOT) -r requirements.txt
dh_virtualenv \
--extra-pip-arg "--use-wheel" \
--extra-pip-arg "--find-links=$(WHEEL_ROOT)"
Problem is: the created deb package does not include python and libraries.
If building dependencies is the bottleneck, try upgrading to the latest version of pip. The later versions store pre-built packages in a local wheel cache making subsequent builds much faster.
If downloading packages takes long, you should copy your dependencies to a local folder on your build-server. Make sure you copy the packages to local folder on the file system. While pip supports access via --find-links=http://your-local-package-host/, collecting dependencies is still rather slow unless you set up https & proper caching headers for the package host.
If installing the app itself is taking long (hangs after "Processing /some/folder") then you're experiencing the slow copy pip installation issue. You could try to work around that by replacing pip install . with python setup.py sdist and pip install dist/my-shiny-app-1.0.tar.gz in dh-virtualenv.

Categories

Resources