I created a project in PyCharm that uses flask (among a few other modules) installed in a PyCharm-created virutal environment of Python 3.6.1, and the app itself works great in PyCharm. When I went to set up a requirements.txt file for other virtual requirements, however, I noticed that only virtualenv was listed in the output file. (To create the file, I went to "console" and did "pip freeze > requirements.txt".)
After testing some things out, I noticed that pip list only gave me three modules installed to begin with. I know this is not true because when I go into the settings of my interpreter, PyCharm says there are a lot of other modules installed, but it doesn't seem like they're actually installed.
How can I create a requirements file in PyCharm effectively? Do I just have to list all the modules I installed in my readme and have my users figure out how to install them all themselves?
Screenshots below:
Project Settings dialog:
Pip list output:
Use pipreqs
$ pip install pipreqs
$ pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt
This will export the packages used in your current project directory into requirements.txt
See pipreqs
Why not pip freeze?
pip freeze only saves the packages that are installed with pip
install in your environment.
pip freeze saves all packages in the
environment including those that you don't use in your current
project. (if you don't have virtualenv)
Sometimes you just need
to create requirements.txt for a new project without installing
modules.
It's certainly strange, the only thing I can think of is that it's a problem with virtualenv on Windows.
Anyways it wouldn't be best practice to create your requirements.txt from pip freeze, because you have more packages installed than the ones that your project requires.
E.g. lets say that your project only requires Flask:
$ pip install Flask
$ pip freeze
click==6.7
Flask==0.12.2
itsdangerous==0.24
Jinja2==2.9.6
MarkupSafe==1.0
Werkzeug==0.12.2
As you can see by installing Flask many more packages were installed, but you don't have to list those in your requirements.txt, as they are not your projects requirements, they are Flask's requirements.
Therefore you should construct your requirements.txt manually. What I usually do is pip install Flask; pip freeze |grep Flask and copy the line Flask==0.12.2 to my requirements.txt, doing this every time I install something with pip.
Related
I would like to create a virtual environment for my team. My team, works in different places and everyone has their own environment, it causes a lot of problems, everyone has a different version of the libraries (Python, RobotFramework).
I thought about:
creating one common environment, I used virtualenv.
Installing the prepared libraries (python and robotframework) with one command pip install ...,
Prepared libraries will be in the git repository so that everyone can modify them, change the library version.
I have the first and third parts done, but I have a problem with the second. How to create such a package of libraries to be able to install it with one pip install command.
Should I create an environment locally, install all the libraries in it, and send them to git? Or should I package the project via setuptool (to tar.gz)?
Unfortunately, I cannot find the answer to this question, it seems to me that none of the above solutions is optimal.
The easiest way of doing it would be creating a text file of all the libraries you are using in pip with the command.
pip freeze > requirements.txt
This will create a file listing all the packages with their versions that are being used. To install that ask every team member to place that requirement file in their projects and use
pip install -r requirements.txt
With pip, you could download your dependencies. These will be .tar.gz, .whl or .zip files. Note that this could be complicated if your team uses multiple OS.
Here is an example which will download the dependencies into the directory named "dependencies", you can push this to git along with the requirements file.
pip freeze > req.txt
pip download -r req.txt -d dependencies
When someone clones your repository, they can install the dependencies offline with the following command.
pip install --no-index --find-links=dependencies -r req.txt
When I create a virtualenv for a Python project, it get's "polluted" by packages that I install for my convenience (like iPython or other packages that my editor "VS Code" depends on, like "pylint").
But these packages are not relevant for my project. So if I do pip freeze > requirements.txt, I see that only a few packages are relevant for my project.
What is the best way to clean up?
Install those packages in a global context so that I can use them in every project I begin? or
Do a pip freeze > requirements.txt, then edit the requirements file and remove not needed packages?
What we do here:
First we have the project's requirement file - the one used for deployments. This is not built using pip freeze but manually edited so it only contains relevant packages.
Then we have the "dev" requirement file with packages that are only useful for development but are required to work on the project (linters, additionnal testing stuff etc).
And finally each is free to maintain his own personal additional requirements (editor-related packages etc).
Note that using virtualenvwrapper (which really helps for development installs) you define hooks that will install packages when you create a new virtual env.
Here is an alternative solution for preparing requirements.txt manually.
The project I mentioned above prepares a requirements.txt
for your project based on the imports you did in your project's Python files.
Assuming all of your Python files in myproject, doing these in your terminal:
$ pip install pipreqs
$ pipreqs myproject
will generate a requirements.txt file for you.
This way, you can just pip install -r requirements.txt in your virtual environment instead of pip freeze > requirement.txt since you will have only the packages which are related to your project.
I've been looking around for a package manager that can be used with python. I want to list project dependencies in a file.
For example ruby uses Gemfile where you can use bundle install.
How can I achieve this in Python?
The pip tool is becoming the standard in equivalent of Ruby's gems.
Like distribute, pip uses the PyPI package repository (by default) for resolving and downloading dependencies.
pip can install dependencies from a file listing project dependencies (called requirements.txt by convention):
pip install -r requirements.txt
You can "freeze" the current packages on the Python path using pip as well:
pip freeze > requirements.txt
When used in combination with the virtualenv package, you can reliably create project Python environments with a project's required dependencies.
Pipenv
(I know it's an old question, and it already has an answer but for anyone coming here looking for a different answer like me.)
I've found a very good equivalent for npm, It's called pipenv. It handles both virtualenv and pip requirements at the same time so it's more like npm.
Simple Use Case
pip install pipenv
then you can make a new virtualenv with third version of python, as well as making a pipfile that will be filled with your projects requirement and other stuff:
pipenv install --three
using your created virtualenv:
pipenv shell
installing a new python package:
pipenv install requests
running your .py file is like:
pipenv run python somefile.py
you can find it's doc here.
Python uses pip for a package manager. The pip install command has a -r <file> option to install packages from the specified requirements file.
Install command:
pip install -r requirements.txt
Example requirements.txt contents:
Foo >= 1.2
PickyThing <1.6,>1.9,!=1.9.6,<2.0a0,==2.4c1
SomethingWhoseVersionIDontCareAbout
See the Requirements Parsing section of the docs for a full description of the format: https://pip.pypa.io/en/stable/user_guide/#requirements-files
This is how I restrict pip's scope to the current project. It feels like the opposite if you're coming from NodeJS's npm or PHP's composer where you explicitly specify global installations with -g or --global.
If you don't already have virtualenv installed, then install it globally with:
pip install virtualenv
Each Python project should have its own virtualenv installation. It's easy to set one up, just cd to your project's root and:
python3 -m virtualenv env # creates env folder with everything you need
Activate virtualenv:
source env/bin/activate
Now, any interaction with pip is contained within your project.
Run pip install package_name==version for each of your dependencies. They are installed in ./env/lib/python3.x/site-packages/
When you want to save your project's dependencies to a file, run:
pip freeze > requirements.txt
You actually don't need -l or --local if you're in an activated project-specific virtualenv (which you should be).
Now, when you want to install your dependencies from requirements.txt, set up your virtualenv, and run:
pip install -r requirements.txt
That's all.
This is an old question but things are constantly evolving.
Further to the other answer about pipenv. There is also a python package manger called poetry.
There is a detailed comparison between pipenv and poerty here: Feature comparison between npm, pip, pipenv and poetry package managers. It also links the features to common npm features.
Here is a comparison of pipenv vs poetry vs pdm: https://dev.to/frostming/a-review-pipenv-vs-poetry-vs-pdm-39b4
The conclusion is that pdm is the winner.
But in my experience, poetry is easier than pdm to integrate with IDEs.
To install dependences, the appengine-python-flask-skeleton docs advise running this command:
pip install -r requirements.txt -t lib
That works simply enough.
Now say I want to add the Requests package.
Ideally I just add it to the requirements.txt file:
# This requirements file lists all third-party dependencies for this project.
#
# Run 'pip install -r requirements.txt -t lib/' to install these dependencies
# in `lib/` subdirectory.
#
# Note: The `lib` directory is added to `sys.path` by `appengine_config.py`.
Flask==0.10
requests
And then re-run the command:
pip install -r requirements.txt -t lib
However, as this Github issue for pip notes, pip is not idempotent with the -T option recommended by Google here. The existing flask packages will be re-added and this will lead to the following error when running the devapp
ImportError: cannot import name exceptions
How can I best work around this problem?
Like said, updating pip solves the issue for many, but for what it's worth I think you can get around all of this if the use of virtualenv is an option. Symlink /path/to/virtualenv's/sitepackages/ to lib/ and just always keep an up to date requirements.txt file. There are no duplication of packages this way and one won't have to manually install dependencies. See also https://stackoverflow.com/a/30447848/2295256
Upgrading to the latest version of pip solved my problem (that issue had been closed):
pip install -U pip
Otherwise, as noted in that thread, you can always just wipe out your lib directory and reinstall from scratch. One note of warning: if you manually added additional packages to the lib directory not tracked in requirements.txt, they would be lost and have to be re-installed manually.
We have a python/django based web application, many components of which are installed using pip. So I would like to ask if there is a way to save or download and save the particular python packages that we are having pip install (example: pip install django==1.5.1). We would like to have in the end a collection of the packages in the versions known to be working and with which the app was developed locally. Any and all advice will be appreciated.
If I understood your question right, you can pip freeze > requirements.txt, this command will add all the libraries you have used/"downloaded" for your app in the file requirements.txt(in case it exists the file be overwritten). This command allows you to later do pip install -r requirements.txt. However, be aware that your Django project must be running in a virtual environment, otherwise the install command will attempt to install all the python packages in your development machine.
The freeze command will allow you to have the current version of the app so upon installation will attempt to install that same version. Your requirements file will look something like:
Flask==0.8
Jinja2==2.6
Werkzeug==0.8.3
certifi==0.0.8
chardet==1.0.1
distribute==0.6.24
gunicorn==0.14.2
requests==0.11.1
Your packages are installed (if using virtualenv) at: ../<your project>/<your virtual env>/<lib>/<python version>/<site-packages>/
As for downloading you can use pip install --download command as #atupal suggested in his response, however think if this is really needed you can also fork those libraries on github to accomplish the same.
Here is a good source of information on how this works: http://www.pip-installer.org/en/latest/cookbook.html
Maybe what you want is:
Download the packages:
pip install --download /path/to/download/to packagename
OR
pip install --download=/path/to/packages/downloaded -r requirements.txt
install all of those libraries just downloaded:
pip install --no-index --find-links="/path/to/downloaded/dependencies" packagename
OR
pip install --no-index --find-links="/path/to/downloaded/packages" -r requirements.txt
Shamelessly stolen from this question
Create a requirements.txt file.
Put:
django==1.5.1
in the first line.
Then run pip install -r requirements.txt
Then you can complete that file...