After moving over to Fedora (from Windows), I realized that it came with both installations of Python 2.7.5 and Python 3.6.6.
As I familiarized myself with using Python, I learned of the great utilities of virtual environments and how organized they keep everything.
However, my current dilemma is for which Python version should I do pip(2 or 3) install virtualenv virtualenvwrapper.
From my research, I understand that the virtualenvwrapper provides the ability to create a virtual environment using a specified version of Python: mkvirtualenv -p /usr/bin/python(2 or 3) {name}.
Therefore, should I only install virtualenv and virtualenvwrapper on one of the Python versions and use the aforementioned feature? Or should I install virtualenv and virtualenvwrapper on both versions of Python.
Would there be any conflicts?
Edit
More importantly, assuming that I have virtualenv and virtualenvwrapper installed for both Python 2.7.5 and Python 3.6.6, which version's command is called when I run any of the following: workon, mkvirtualenv, rmvirtualenv, etc.?
Would there be any conflicts?
Not until you mistakenly run the default system python command with a script that's using the opposite version as compared to the more specific python2 or python3 commands.
The virtualenvs do not conflict, and must be activated to be used. You can also of course have as many virtualenv's as you wish.
To avoid any problems setting up an environment, its suggested to run python2 -m virtualenv for example, rather than simply virtualenv command itself
For the commands listed at the bottom of the question, it depends on how your PATH is configured. Personally, I use pyenv rather than virtualenv directly, which injects itself into the OS PATH variable
Related
I need python3.6 for tensorflow installation, so I downloaded python3.6.12.tar. And I found that I should pip install tarfile. However, in this case it is an older version of python. FYI, In my computer(laptop) I installed python3.9.
My question is: can I pip install python.tar inside a virtualenv?
This is not how virtual environments work. I suggest you to do a little bit more research on virtual environments in Python.
Virtual Environments and Packages
Basically you need to install the necessary python version onto your machine. Then go ahead and use that specific python (which is version 3.6 in your case), to create a virtual environment with the command
~ /usr/bin/<path-to-python3.6> -m venv venv
This command will create a folder called venv. Now you need to source the activation script inside this folder to activate your environment.
Handy note: if you are dealing with different versions of python, a more robust way of handling such situations is via using a tool called pyenv.
Sorry if I sound a bit foolish.
I'm confused about this
what's the difference between the two
virtualenv myvenv
and
-m venv myvenv
The first one works well for me in creating virtual environments while the other does not.
I CD into my development directory and use "virtualenv myvenv" and it creates the virtual environment. But if I use "-m venv myvenv" it just gives errors. Please help me understand
venv is a package shipped directly with python 3. So you don't need to pip install anything.
virtualenv instead is an independent library available at https://virtualenv.pypa.io/en/stable/ and can be install with pip.
They solve the same problem and work in a very similar manner.
If you use python3 I suggest to avoid an "extra" dependency and just stick with venv.
Your error is probably because you use Python2/pip2.
I think the virtualenv docs explain this the best:
venv is a subset of virtualenv integrated into the standard library since Python 3.3. The subset meaning that only part of virtualenvs functionality is in venv:
venv can be slower since it does not have the "app-data seed method". See more about virtualenv Seeders in the docs.
venv can only be updated by upgrading the Python version, while virtualenv is updated using pip.
venv is not extendable
virtualenv has richer programmatic API (describe virtual environments without creating them). See the venv API here.
venv cannot automatically discover arbitrarily installed python versions, while virtualenv does. This means, that with venv you have to specify the full path of the python executable, if you want to use some other python version than the first one in the PATH. With virtualenv, you can just give the version number. See python discovery in the virtualenv documentation.
To me the differences are quite subtle and the only practical difference has been that venv is included in the standard library (since 3.3). I have been using python -m venv venv for a long time and have never needed an alternative.
I'm developing a python project on my desktop (OS X) and it's running well. Now I need to run it on a computing cluster that is running Linux and I have no root access. On my Home in the computing cluster I installed Anaconda locally. Then when I run my project there I got so many strange bugs and errors. THe code runs fine on my desktop. So I thought probably the problem is that probably Anaconda has newr modules than what I have and that's probably causing errors. (backward compatibility issues)
Is there any way to use the same python modules versions I have in my desktop on the computing cluster?
I wonder if there is a command that produces all the packages and modules I have on my desktop with their versions and then import this file into some tool in that linux machine and then I get a cloned version of my setup. Is there something like that?
Use pyenv to control the python version
pyenv is designed for installing python into the home folder so you don't need root access (but even with root access it's a great idea), it lets you use a specified version of python within your home folder hierarchy rather than the OS version. You should also install pyenv virtualenv so you can use a virtual environment for your project (not strictly essential, but virtual environments are a great idea and you should always use one, with pyenv they're practically effortless).
One of the neat things about pyenv is the pyenv local command which specifies which version of python (or which virtualenv) should be used for a folder (and subfolders), once you've used pyenv local in your project folder to set the python version any time you use the python command it'll use the version set by pyenv local. It's not needed if you only install one version of python, and don't use a virtualenv (in that case you can use pyenv global to set the version for the user). The neatest thing about pyenv local/global setup is it works great with both scripts and manually invoking python, it is simply set-and-forget unlike other python environment managers which require activation.
In brief, once you've set up pyenv you control exactly what version of python will run, and as it is installed into the home folder the OS has no power to affect it.
Having installed pyenv and pyenv virtualenv, you would then use it to install the same version of python as is used on your development machine, the commands you'd run would be something like this:
pyenv install 3.4.2
pyenv virtualenv 3.4.2 my_project_env
cd my_project
pyenv local my_project_env
Install modules into python environment
To get a list of python module version use pip freeze, you can do this on your development machine:
pip freeze > requirements.txt
Now copy requirements.txt to your deployment machines (with pyenv already setup using pyenv local or pyenv global) and run:
pip install -r requirements.txt
Which will install the same modules into the python environment.
Duplicating a pyenv
While it's kind of dodgy, once you've done this, you can even copy the entire installation (i.e. at least the .bashrc file and .pyenv folder) onto other machines, and if the machines have the same OS and the home folders have the same name, the transplanted environment should work fine. It might be more responsible to use a setup script but I have copied pyenv environments without anything terrible happening.
If you have pip installed you could try this pip freeze.
If you want to get list of modules from python shell:
help('modules')
I have installed Python 3.4.0 and created virtual environment with python -m venv myenv. How can I change Python version in my virtual environment? Documentation says:
Each virtual environment has its own Python binary (allowing creation
of environments with various Python versions) and can have its own
independent set of installed Python packages in its site directories.
UPDATE
Please, note that I ask about venv from standard library, not about virtualenv.
Let me provide some links.
This is PEP 405. http://legacy.python.org/dev/peps/pep-0405/
Python venv. http://docs.python.org/3.4/library/venv.html
Virtualenv. http://www.virtualenv.org/en/latest/
I don't see something like a --python flag in venv.
Are venv and virtualenv absolutely similar?
Is venv is so unpopular and no one uses it so that virtualenv remains the standard?
On Linux/Mac you can easily install multiple versions of Python next to the main one and you can use the venv package from the standard library to create virtual environments from each version >= 3.3.
Create venv
$ python3.3 -m venv myvenv_foo # Create a python3.4 venv named 'myvenv_foo'
$ python3.4 -m venv myvenv_bar # Create a python3.4 venv named 'myvenv_bar'
$ python3.5 -m venv myvenv_baz # Create a python3.5 venv named 'myvenv_baz'
# etc...
Activate venv
source myvenv_foo/bin/activate # Activates venv 'myvenv_foo'
Deactivate venv
deactivate
Notice: python vs pythonX.X
If you have multiple Python versions installed, you can access each one by adding the version num to the command e.g. python3.5, python3.6, etc. But keep in mind that when you activate a venv, you bind it to the clean/versionless python command, for as long as it's activated. E.g:
$ python -V # Use the *clean* 'python' command to show the main version of the OS.
Python 2.7.6
$ python3.5 -m venv myvenv_foo # Create a new venv from 'python3.5'.
$ source myvenv_foo/bin/activate # Activate venv.
$ python -V # The *clean* 'python' command is now bound to your activated venv.
Python 3.5.2
$ deactivate # Deactivate venv.
$ python -V # Now the *clean* command is bound back to the main version.
Python 2.7.6
Note
I suggest using Pipenv to create/handle virutal environments over the venv package.
From the offical docs:
Managing multiple virtual environments directly can become tedious, so the dependency management tutorial introduces a higher level tool, Pipenv, that automatically manages a separate virtual environment for each project and application that you work on.
This is a very good question as there are several python modules / libraries (built-in & third party) with similar names and purposes. Can completely sympathise with OP's confusion.
There are really two different behaviours / responsibilities:
1). The ability to switch between different versions of (System) Python Interpreter eg. 2.7.10 or 3.5.0 etc
2). The ability to create virtual environments (which is just a local folder containing all the plumbing (binaries and libs) for a particular version of python. Can sort of think of this as a frozen local instance of a particular python version. Essentially it is a self-contained, light-weight python installation.
A module like pyvenv provides 2) above. It will allow you to create a virtual environment that is set at the version of Python that was used to create it.
$ python --version
Python 3.5.0
$ pyvenv myenv # myenv is now a local environment using Python 3.5.0
For further infoormation on pyvenv, see library/venv
A module like pyenv (the names are confusing, right? Notice, pyenv, and not pyvenv) on the other hand, controls which VERSION of python your system is basically running. This provides 1) above. So, when not running a particular virtual env via pyvenv etc, this is the "global" version in use. In fact, it is slightly more convoluted than that (as you can also setup local configuration etc), but essentially that is enough for this discussion.
For further information on pyenv see github.com/yyuu/pyenv
Suppose I want to run Python versions 2.7.10 and 3.5.0, then I would use pyenv to install these two versions (here, I chose as globals), and can view this using:
$ pyenv versions
system
* 2.7.10 (set by ~/.pyenv/version)
* 3.5.0 (set by ~/.pyenv/version)
$ python --version
Python 3.5.0
$ which python
~/.pyenv/shims/python
$ python2.7 --version
Python 2.7.10
Yes, there are several prominant alternatives to each of the above referenced modules / libs. Heated discussions on Reddit / SOF etc detailing and arguing which is best. Many of them do very similar things...
It's simply impossible. To create a Python venv of a specific Python version, we need this specific version.
Obviously, a Python interpreter doesn't "include" all the previous versions with their behavior. Python 3.4.1 cannot contain Python 2.7.8 executable anywhere inside.
As another answer said, you need the specific version already there to create virtual env for it. So if you already have it somewhere in your system, you can do it. For example, OSX comes with Python2.7, and so to create a 2.7 virtual environment to avoid messing with the system one, do:
$ virtualenv -p /usr/local/opt/python#2/bin/python2.7 venv
Basically:
$ virtualenv -p <path/to/existing/python> <path/to/new/virtualenv/>
As of 2021, you can't use multiple Python versions with the standard library venv. It will use just the installed Python.
For alternatives you need other tools. You can use:
Conda distribution that has a very robust virtual environment tool. You will be able to easily use multiple Python versions. You'll need to take some care if your environments must be used in *nix and Windows (tip: use conda env export --from-history).
the pyenv module that tries to bring other systems more robust package handling to Python. I don't have experience with it, but the package description is sound. It looks like they are doing the right thing.
Virtualenvwrapper is a user-friendly shell around Python's virtualenv.
Python 3.3 ships with pyvenv built into the standard library, which aims to supercede virtualenv.
But if I install Virtualenvwrapper on Python3.3, it still installs virtualenv, leading me to believe it doesn't use 'pyvenv' under the covers.
Presumably this doesn't really matter - if I wish to use virtualenvwrapper on Python3.3 I should happily let it use virtualenv instead of pyvenv, and will (for the moment) suffer no ill effects?
Sorry this answer is a bit delayed. pyvenv does not aim to supersede virtualenv, in fact virtualenv in Python 3 depends on the standard library venv module.
The pyvenv command creates an absolutely minimal virtual environment into which other packages can be installed.
The Python 3 version of virtualenv actually subclasses the standard library's implementation and provides hooks to automatically install setuptools and pip into the environment which pyvenv doesn't do on it's own.
As far as I know virtualenvwrapper depends on virtualenv only because the mkvirtualenv or mkproject commands allow you to specify packages to be installed into the new environment, this only works because virtualenv will have already installed setuptools and pip.
So to answer your question I believe you should be able to use virtualenvwrapper on environments created by pyvenv as long as you follow virtualenvwrapper's conventions for where to put things and you either manually install setuptools and pip into the environment or don't use any package management features of virtualenvwrapper.