We have a python/django based web application, many components of which are installed using pip. So I would like to ask if there is a way to save or download and save the particular python packages that we are having pip install (example: pip install django==1.5.1). We would like to have in the end a collection of the packages in the versions known to be working and with which the app was developed locally. Any and all advice will be appreciated.
If I understood your question right, you can pip freeze > requirements.txt, this command will add all the libraries you have used/"downloaded" for your app in the file requirements.txt(in case it exists the file be overwritten). This command allows you to later do pip install -r requirements.txt. However, be aware that your Django project must be running in a virtual environment, otherwise the install command will attempt to install all the python packages in your development machine.
The freeze command will allow you to have the current version of the app so upon installation will attempt to install that same version. Your requirements file will look something like:
Flask==0.8
Jinja2==2.6
Werkzeug==0.8.3
certifi==0.0.8
chardet==1.0.1
distribute==0.6.24
gunicorn==0.14.2
requests==0.11.1
Your packages are installed (if using virtualenv) at: ../<your project>/<your virtual env>/<lib>/<python version>/<site-packages>/
As for downloading you can use pip install --download command as #atupal suggested in his response, however think if this is really needed you can also fork those libraries on github to accomplish the same.
Here is a good source of information on how this works: http://www.pip-installer.org/en/latest/cookbook.html
Maybe what you want is:
Download the packages:
pip install --download /path/to/download/to packagename
OR
pip install --download=/path/to/packages/downloaded -r requirements.txt
install all of those libraries just downloaded:
pip install --no-index --find-links="/path/to/downloaded/dependencies" packagename
OR
pip install --no-index --find-links="/path/to/downloaded/packages" -r requirements.txt
Shamelessly stolen from this question
Create a requirements.txt file.
Put:
django==1.5.1
in the first line.
Then run pip install -r requirements.txt
Then you can complete that file...
Related
I've never done any Python so I'm not familiar with the package versions and dependencies system overall. I'm trying to run this repo https://github.com/Maaxion/homeassistant2influxdb
For this, I want to use Docker. So once I've cloned the repo, I've added this Dockerfile at the root and followed what was explained in the readme to the best I could:
FROM ubuntu:18.04
RUN apt update -y
RUN apt install python3 python3.7-dev python3-venv python3-pip git -y
WORKDIR /home
COPY . .
RUN git clone --depth=1 https://github.com/home-assistant/core.git home-assistant-core
RUN python3 -m venv .venv
RUN . .venv/bin/activate
RUN python3 -m pip install --upgrade --force pip
RUN pip3 install -r home-assistant-core/requirements.txt
RUN pip3 install -r requirements.txt
It goes fine until it tries to install with pip3 with that line: pip3 install -r home-assistant-core/requirements.txt and I get:
Collecting atomicwrites-homeassistant==1.4.1
Downloading atomicwrites_homeassistant-1.4.1-py2.py3-none-any.whl (7.1 kB)
ERROR: Cannot install awesomeversion==22.9.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested awesomeversion==22.9.0
The user requested (constraint) awesomeversion==22.9.0
To fix this you could try to:
loosen the range of package versions you've specified
remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
I'm really not sure how to solve this despite taking a look at the link above...
Is it something to do with pip3? Have I missed something in the Dockerfile? How can I solve that issue? I've been looking online but there doesn't seem to be silver bullet answer for this kind of issues.
Could anyone provide some guidance? Thanks!
Try not to specify a direct version, like awesomeversion>=22.4.0 in your requirements.txt file.
What has probably happened is that one of the other requirements is specified as coolRequirement >= somenumber, (or just coolRequirement).
This means that pip is grabbing the latest recommended release for your python version. One of these requirements probably conflicts with something that awesomeversion==22.9.0 requires. One possible solution would be to change awesomeversion==22.9.0 to awesomeversion>=22.9.0, but that may not work if newer versions of awesomeversion break something else.
The real solution would be to figure out what versions of the other requirements worked in the past, and lock those down the same way that awesomeversion is specified.
I downloaded some packages in my environment using pip command. And I want to have a copy of them to transfer them to another environment. I know that using:
pip freeze > requirements.txt
will generate requirements into a file, but since my second environment does not have access to internet i can not use:
pip install -r requirements.txt
to install that packages again.
Is there any way to copy installed packages? or somehow install packages in a specified directory in my first environment?
Thanks
You can use pip download followed by pip install --find-links to achieve what you want.Here is the steps involved
Get the requirements
pip freeze>requirements.txt
Download the packages to a folder
pip download -r requirements.txt -d path_to_the_folder
From the new environment
pip install -r requirements.txt --find-links=path_to_the_folder
I created a project in PyCharm that uses flask (among a few other modules) installed in a PyCharm-created virutal environment of Python 3.6.1, and the app itself works great in PyCharm. When I went to set up a requirements.txt file for other virtual requirements, however, I noticed that only virtualenv was listed in the output file. (To create the file, I went to "console" and did "pip freeze > requirements.txt".)
After testing some things out, I noticed that pip list only gave me three modules installed to begin with. I know this is not true because when I go into the settings of my interpreter, PyCharm says there are a lot of other modules installed, but it doesn't seem like they're actually installed.
How can I create a requirements file in PyCharm effectively? Do I just have to list all the modules I installed in my readme and have my users figure out how to install them all themselves?
Screenshots below:
Project Settings dialog:
Pip list output:
Use pipreqs
$ pip install pipreqs
$ pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt
This will export the packages used in your current project directory into requirements.txt
See pipreqs
Why not pip freeze?
pip freeze only saves the packages that are installed with pip
install in your environment.
pip freeze saves all packages in the
environment including those that you don't use in your current
project. (if you don't have virtualenv)
Sometimes you just need
to create requirements.txt for a new project without installing
modules.
It's certainly strange, the only thing I can think of is that it's a problem with virtualenv on Windows.
Anyways it wouldn't be best practice to create your requirements.txt from pip freeze, because you have more packages installed than the ones that your project requires.
E.g. lets say that your project only requires Flask:
$ pip install Flask
$ pip freeze
click==6.7
Flask==0.12.2
itsdangerous==0.24
Jinja2==2.9.6
MarkupSafe==1.0
Werkzeug==0.12.2
As you can see by installing Flask many more packages were installed, but you don't have to list those in your requirements.txt, as they are not your projects requirements, they are Flask's requirements.
Therefore you should construct your requirements.txt manually. What I usually do is pip install Flask; pip freeze |grep Flask and copy the line Flask==0.12.2 to my requirements.txt, doing this every time I install something with pip.
In order to make packages installed offline, I use the -d (or --download) option to pip install. For instance, pip install --download dependencies -r requirements.txt will download the packages for all required dependencies mentioned in requirements.txt to dependencies dir (but will not install them). Then I use pip install --no-index --find-links dependencies -r requirements.txt to install those downloaded packages without accessing the network.
Most of the time it works fine, but sometimes installation fails with error "Could not find a version that satisfies the requirement xyz". After doing pip install --user xyz --find-links dependencies manually (xyz IS present in the dependencies folder), installation fails with the same "Could not find a version that satisfies the requirement abc" error, but with different package 'abc'. It repeats several times until I manually resolve all failed dependencies.
How could I make run pip install --no-index --find-links dependencies -r requirements.txt without those weird dependency errors not finding packages that are already there?
Make sure of two things:
The pip version is the same in the offline server and in the online one.
To find out: pip -V
To update (if needed): pip install --upgrade pip
The python version is the same in both virtual enviroments or servers.
To find out: python (the header will have the version info)
In my case I was calling pip install --download outside the virtual environment (using default python version - 2.7) and then installing in a virtual environment with python 3 and the error I got was exactly the one you mentioned.
To install dependences, the appengine-python-flask-skeleton docs advise running this command:
pip install -r requirements.txt -t lib
That works simply enough.
Now say I want to add the Requests package.
Ideally I just add it to the requirements.txt file:
# This requirements file lists all third-party dependencies for this project.
#
# Run 'pip install -r requirements.txt -t lib/' to install these dependencies
# in `lib/` subdirectory.
#
# Note: The `lib` directory is added to `sys.path` by `appengine_config.py`.
Flask==0.10
requests
And then re-run the command:
pip install -r requirements.txt -t lib
However, as this Github issue for pip notes, pip is not idempotent with the -T option recommended by Google here. The existing flask packages will be re-added and this will lead to the following error when running the devapp
ImportError: cannot import name exceptions
How can I best work around this problem?
Like said, updating pip solves the issue for many, but for what it's worth I think you can get around all of this if the use of virtualenv is an option. Symlink /path/to/virtualenv's/sitepackages/ to lib/ and just always keep an up to date requirements.txt file. There are no duplication of packages this way and one won't have to manually install dependencies. See also https://stackoverflow.com/a/30447848/2295256
Upgrading to the latest version of pip solved my problem (that issue had been closed):
pip install -U pip
Otherwise, as noted in that thread, you can always just wipe out your lib directory and reinstall from scratch. One note of warning: if you manually added additional packages to the lib directory not tracked in requirements.txt, they would be lost and have to be re-installed manually.