While developing my app I didn't use an environment. Now I want to use one and export all dependencies of my app in an environment.yml / requirements.txt file that afterwards I can use it to build a docker image.
The issue is, if I create an environment and then export it with:
conda env export > environment.yml
I get no dependencies in that file.
Or if I use:
pip freeze --local > requirements.txt
I see all system modules that have nothing to do with my project.
I would imagine conda or pip has something that would just go through all my files in the directory I am and place all imports and their dependencies inside the environment.yml/requirements.txt file.
I can't find the command to do that..
You can use virtualenv to isolate your pip environment of your application from rest of your system. Use:
virtualenv <your_project_path>/venv
This will create a virtual environment of your app. Then use;
source venv/bin/activate
This will isolate your pip environment. Reinstall all your dependencies and run pip freeze you will see only project related dependencies.
pip freeze by default fetches all installed pip modules over the system. If you use virtualenv and then install your dependencies, your pip modules will reside in your application folder.
edit
I would recommend a good IDE based on your comments such as PyCharm. You can follow tutorial here for setting up venv and handling all your dependencies. Once done, you can run pip freeze for your requirements.txt
Related
Pip Freeze shows all the libraries I have on my computer not just the libraries in the virtual environment.
I am trying to create a requirements.txt file for my virtual environment. I'm using an anaconda distribution. I am creating a flask app. I have navigated to my project folder created the virtual environment added flask and then when I make the command pip freeze it clearly shows items that are not in my virtual environment like xlwings, pandas and stuff I use that has nothing to do with flask.
Any way I can create a requirements file from my virtual environment.
I can clearly see my virtual environment is active with (venv) to the left.
Edit: I created a short video showing I get the same list of libraries whether I'm in my virtual environment or not. Also I'm showing the site-packages in my virtual environment and showing that these libraries aren't there I'm specifically pointing out xlwings.
https://youtu.be/xEFZ3dSaqoY
So i'm not sure why it was happening, but I deleted the virtual environment and re-created it (I had a previous requirements.txt that was correct). Then I ran pip freeze again and it all worked. Not sure what happened, but it works for me now.
I had this same problem and it happened because i change the name of my venv folder.
To solve this i used this command in my terminal [yourVenv]\Scripts\python -m pip freeze.
If you don't want to do it like this everytime, try creating a new enviorment like this.
[yourVenv]\Scripts\python -m pip freeze > requirements.txt
python -m venv [yourVenv]
[yourVenv]\Scripts\activate
python -m pip install --upgrade pip
pip install -r requirements.txt
You need to activate your pip environment in the console that you are trying to run pip freeze in. That way it uses the environment's pip and not your global pip.
So in your console, navigate to your virtual environment folder. From there go to the "Scripts" folder. Then enter the word "activate" into the console.
You should then see next to the console cursor the name of your virtual environment. At that point you can use the pip that's inside of your virtual environment and all the normal pip commands will point to it.
In my case i was using vscode IDE so when i created .venv its recommended use worksapce, so i choosed yes thats why its took all from my system' lib.
for this you can delete old on .venv and create new one, this time don't choose plugin's recommendations.
for create veirtual enviroment in window/linx
python -m venv .venv or python3 -m venv .venv
to activate .venv win use (.venv\scripts\activate)
and for linux use (source .venv/bin/activate)
all command like pip freeze, pip list, pip freeze > requirements.txt will work
I am coming from NodeJS and learning Python and was wondering how to properly install the packages in requirements.txt file locally in the project.
For node, this is done by managing and installing the packages in package.json via npm install. However, the convention for Python project seems to be to add packages to a directory called lib. When I do pip install -r requirements.txt I think this does a global install on my computer, similar to nodes npm install -g global install. How can I install the dependencies of my requirements.txt file in a folder called lib?
use this command
pip install -r requirements.txt -t <path-to-the-lib-directory>
If you're looking to install dependencies in special (non-standard) local folder for a specific purpose (e.g. AWS Lambda), see this question: install python package at current directory.
For normal workflows the following is the way to install dependencies locally (instead of globally, equivalent to npm i instead of npm i -g in Node):
The recommended way to do this is by using a virtual environment. You can install virtualenv via pip with
pip install virtualenv
Then create a virtual environment in your project directory:
python3 -m venv env # previously: `virtualenv env`
Which will create a directory called env (you can call it anything you like though) which will mirror your global python installation. Inside env/ there will be a directory called lib which will contain Python and will store your dependencies.
Then activate the environment with:
source env/bin/activate
Then install your dependencies with pip and they will be installed in the virtual environment env/:
pip install -r requirements.txt
Then any time you return to the project, run source env/bin/activate again so that the dependencies can be found.
When you deploy your program, if the deployed environment is a physical server, or a virtual machine, you can follow the same process on the production machine. If the deployment environment is one of a few serverless environments (e.g. GCP App Engine), supplying a requirements.txt file will be sufficient. For some other serverless environments (e.g. AWS Lambda) the dependencies will need to be included in the root directory of the project. In that case, you should use pip install -r requirements.txt -t ./.
I would suggest getting the Anaconda navigator.
You can download it here: https://www.anaconda.com
Anaconda allows you to create virtual environments through a graphical interface. You can download any pip package that is available through Anaconda.
Then all you have to do after you have created and added onto your environment is to got to your designated python editor (I mainly use Pycharm) and setting the path to the virtual environment’s interpreter when you select or change the interpreter for your project.
Hope this helps.
I have a .env file that I used to create a virtual environment.
I want to install the same packages (as specified in the .env file), but this time
I dont want it to be a virtual environment. How can I do this?
Many thanks.
Note:
A .env file can be used by miniconda or anaconda to create virtual environments like so:
conda create --name optimus --file alpha.env
then you can run
source activate optimus
to activate your virtualEnv. But how can I do something similar to:
pip install -r myFile.env
to install all packages specified in myFile.env, but not in a virtual environment.
Here is my alpha.env file:
cairo=1.12.2=2
dateutil=2.1=py27_2
freetype=2.4.10=0
numpy=1.6.2=py27.4
If you have access to your original virtual env or can create a new one as you describe I think a very simple solution would be to simply create a requirements.txt file. This is simply a file that names all the packages that you have installed in a specific python environment.
The file can be created by simply running
pip freeze > requirements.txt
This will create a file in your working directory called requirements.txt. You have to have your virtual environment active. To then install the packages in your new environment you just have to enter it (deactivate your virtualenv or what ever you need to do) and run
pip install -r requirements.txt
I have never seen a .env file, but by the sound of it it could actually be exactly the same as a requirements.txt file but under a different name. See if the output of
pip freeze
looks the same (as far as format etc) as what you have in your .env file. In that case you could also just run the command you would in your virtual env ie.
pip install -r myFile.env
EDIT:
After seeing your .env file output I am convinced that you can simply run the
pip install -r myFile.env
command.
I do not have root previlege on a linux server so I want to creat a virtual python according to creating a "virtual" python.
After I run virtual-python.py, I do have python in ~/bin/python:
Then, according to setuptools PyPI page, I download ez_setup.py and run ~/bin/python ez_setup.py. Error occurs:
What should I do?
Looking at the linked website, it looks outdated. You use pip, not easy_install.
For installing development packages, I always take the following rules in account:
The system package manager is responsible for system-wide packages, so never use sudo pip. This doesn't just match the question, but this is always a good idea.
The package manager packages are probably outdated. You'll want an up-to-date version for development tools.
I recommend the following way to install local development tools.
$ # Install pip and setuptools on a user level
$ curl https://bootstrap.pypa.io/get-pip.py | python - --user
$ # Add the executables to your path. Add this to your `.bashrc` or `.profile` as well
$ export PATH=$PATH/$HOME/.local/bin
At this point pip should be accessible from the command line and usable without sudo. Use this to install virtualenv, the most widely used tools to set up virtual environments.
$ pip install virtualenv --user
Now simply use virtualenv to set up an environment to run your application in:
$ virtualenv myapp
Now activate the virtual environment and do whatever you would like to do with it. Note that after activating the virtual environment, pip refers to pip installed inside of the virtualenv, not the one installed on a user level.
$ source myapp/bin/activate
(myapp)$ pip install -r requirements.txt # This is just an example
You'll want to create a new virtual environment for each application you run on the server, so the dependencies can't conflict.
Tutorials online are telling me to put venv in my .gitignore file. Why wouldn't I want to push my virtual environment so that I or other developers could easily pull the project to their locals and conveniently have all dependencies?
On top of what Othman said, virtualenvs are simply not portable. Trying to move it will break it, and it's easier to create a new environment than to fix it. So, even on deployment platforms that do use virtual environments, checking them in to git is not going to work.
virtualenv is a tool to create isolated Python environments.
Heroku gives you one environment and you may install your packages using requirements.txt which is required by Heroku for Django applications.
If you want to share these dependencies with other developers use another remote to github. and push your requirements.txt
then tell your developers to install the packages using this file.
Example
requirements.txt
Django==1.3
Fabric==1.2.0
Jinja2==2.5.5
PyYAML==3.09
To install these packages in one time use:
pip install -r /path/to/requirements.txt
Moreover, when you run the application in your local machine then the virtual environment files might change, which will make push useless things to your repo.
Note: if you want to know which packages installed in your virtual environment then use pip freeze
If you want to export the packages to requirements.txt then run
pip freeze > requirements.txt