This question already has answers here:
How to recreate a virtual env in python
(2 answers)
Closed 3 years ago.
I have two computers (running Ubuntu) that have access to one server. The shared home directory of the two computers is on the server.
On computer (A) python 3.5 is installed, on computer (B) python 3.7 is installed. I created a virtualenv from computer (B) on the shared home directory (using python 3.7).
Now although it is possible to activate that virtualenv from computer (A) it does not use the "virtualenv-python 3.7" but the system's python 3.5. So technically the virtualenv is activated but effectively it is not.
Note that the VIRTUAL_ENV path is set correctly.
I thought that virtualenv-folder is a fully enclosed environment, not even needing any python installed on the system. So why is it not working?
A virtualenv doesn't have a full Python installation. Instead, it links to an installation present on the system (FYI that link is in <env>/lib/orig-prefix.txt). The env's directory tree has some stubs and special logic but it uses the bulk from that installation.
So if you run activate on a system that doesn't have the same Python at the same path as the one virtualenv was created for, that script won't work correctly. It might happen to hook to something else present at the same path but this is not a supported scenario so all bets are off.
If you need a "fully enclosed environment", you may want to take a look at pyenv which does exactly that -- installs a full Python under your home dir. (Or you can just install Python from source to somewhere under your home dir -- but pyenv makes it easy to switch to that installation and back.)
The short answer is because on both computers A and B, the path to your python is likely just an alias. You can verify this by following the path on computer B and using the OS's GUI you can see it is a symlink to the actual python installation sitting somewhere else on your machine (probably usr/bin/). For example, on one machine where multiple installations of Python is found, I see that my virtual environment (.virtualenvs/revconnecion/include/python3.6) is an alias to the original /anaconda3/include/python3.6.
Solution:
You can create your virtual environment by specifying the python version itself when creating your environment:
python3.7 -m venv sharedvenv
or:
virtualenv -p python3.7
You can also manually change the symlinks / alias but the above method works better. Using the specific version of Python itself to create the virtual environment leaves no ambiguity and explicit is better than implicit.
Run which python on both Computer A and B to verify that it's pointing to the right version of Python.
Related
I'm teaching a beginners python class, the environment is Anaconda, VS Code and git (plus a few extras from a requirements.txt).
For the windows students this runs perfectly, however the mac students have an existing python (2.7) to contend with.
The windows students (i.e. they have a windows computer), their environment when they debug matches their console environment. However the mac students seem to be locked to their 2.7 environment.
I've tried aliasing, as suggested here and here
alias python2='python'
alias python='python3'
alias pip2='pip'
alias pip='pip3'
I've modified the .bash_profile file
echo 'export PATH="/Users/$USER/anaconda3/bin:$PATH"' >>.bash_profile
Both of these seem to work perfectly to modify their Terminal environments, when launched externally to VS Code. Neither seem to do anything to the environment launched from [cmd]+[`].
I've also tried conda activate base in the terminal, which seems to have no effect on a python --version or a which python
They can run things using python 3, but that means that they need to remember that they are different to the other 2/3 of the students. It's othering for them, and more work for me!
The students are doing fine, launching things from their external terminal, but it would streamline things greatly if the environments could be as consistent as possible across the OSs.
Whilst they are complete beginners, they can run a shell script. They currently have one that installs pip requirements and vs code extensions.
Is there a configuration that will keep the terminal in line with the debug env?
In my opinion the best practice is to create Python virtual environments (personally I love using conda environments, especially on Mac where you stuck with unremovable old Python version). Then VSCode will automatically (after installing very powerful Python extension) find all your virtual environments. This way you will teach your students a good practice of handling Python zoo a.k.a. package incompatibilities. Terminal environments settings will be consistent with VSCode, without being dependent on unneeded any more aliases. Obviously, virtual environments are OS independent, so you will be more consistent and remove unnecessary confusion between different students.
The additional bonus of the virtenvs is that you can create one exactly according to your requirements.txt and switch from one to another with a single click (in terminal it takes two commands: deactivate -> activate).
You can read more about how to handle Python virtual environments on VSCode site
Given the aliases are run just once and are not persistent in .bash_profile, python targets the default interpreter rather than the expected conda python3 interpreter.
Try to symlink conda's python3 executable to capture the python namespace
ln -sf /Users/$USER/anaconda3/bin/python3 /Users/$USER/anaconda3/bin/python
This will create or update the symlink. Use the same approach for pip and pip3.
Python in vscode let's you select which interpreter will be used to run the scripts.
It is in settings under "python.pythonPath", just set it to point to the interpreter of choice.
It can be set on a project basis as well (which is how you ensure that a project that has a virtual environment will execute using that interpreter and packages), you just select Workspace in the settings pane and add the desired python interpreter there.
I'm attempting to find a set of steps necessary to make a virtual environment of python 3.6 on windows relocatable.
1st I created a virtual environment on virtualenv 15.1.0 with the following command:
virtualenv
--always-copy
-a "path\to\project\dir"
-r "path\to\requirements.txt"
venv_name
After this, I run the following command to use the built in 'make paths relative ' functionality of virtualenv:
virtualenv --relocatable venv_name
Part of my requirements.txt is pypiwin32 library which, at least when installed via pip, wont work until the:
python Scripts/pywin32_postinstall.py -install
script is run (See here for details).
At this point, if I search the venv directory for clues of hardcoding, I see them in scripts\activate.bat, which I can make relative by changing this:
set "VIRTUAL_ENV=C:\path\to\venv"
into this:
pushd %~dp0..
set VIRTUAL_ENV=%CD%
popd
There are some other other places where I had to make slight adjustments to make them relative (I used the search in folder feature of sublime with my username as the search parameter - it brought up all the path\to\username\then\some\more style lines in the directory.
There are 2 hardcoded paths which are not so simple:
1. "path\to\venv\Lib\orig-prefix.txt"
I understand that orig-prefix.txt is a record of which is the source python installation on which the venv was based and so cannot really be relative but may need to be left blank if moving the venv to another machine (it's absence may crash the python launcher but its emptiness is fine.)
2. "path\to\venv\Lib\site-packages\virtualenv_path_extensions.pth"
This is trickier. As it is a hard-coded path which is then added to sys.path as a location to look for modules, when I move the venv to another machine where this path doesn't exist, the module load will fail.
Is there a way I can add relative paths to the configuration files such as virtualenv_path_extensions.pth
Normally environments are tied to a specific path. That means that you cannot move an environment around or copy it to another computer. You can fix up an environment to make it relocatable with the command:
$ virtualenv --relocatable ENV
This will make some of the files created by setuptools use relative paths, and will change all the scripts to use activate_this.py instead of using the location of the Python interpreter to select the environment.
Note: scripts which have been made relocatable will only work if the virtualenv is activated, specifically the python executable from the virtualenv must be the first one on the system PATH. Also note that the activate scripts are not currently made relocatable by virtualenv --relocatable.
Note: you must run this after you’ve installed any packages into the environment. If you make an environment relocatable, then install a new package, you must run virtualenv --relocatable again.
Also, this does not make your packages cross-platform. You can move the directory around, but it can only be used on other similar computers. Some known environmental differences that can cause incompatibilities: a different version of Python, when one platform uses UCS2 for its internal unicode representation and another uses UCS4 (a compile-time option), obvious platform changes like Windows vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C libraries on the system, if those C libraries are located somewhere different (either different versions, or a different filesystem layout).
I have a Python app running on windows that has imports for the following packages:
requests
json
psycopg2
I copy the entire project (I used Pycharms to write the app and import the packages) to a new machine and expected it would work. The new machine is also windows and I'm trying to run my script from the command line (i.e. no Pycharm on the new machine).
Instead, I get an error saying "ModuleNotFoundError: No module named 'requests'"
If I look at the project, I have the directories:
venv
Lib
site-packages
requests
What am I missing/doing wrong?
You have a couple of options here but first the problem. You are exporting your code base to a new machine without the required Modules installed on that machine and/or within your Python project's environment. After you have python installed on your new machine, you need to be sure to point your PyCharm Project to the proper environment.
File > Default Preferences > Project Interpreter
The window that appears on the right will contain a drop down menu labeled Project Interpreter. If you click on the drop down, it should reveal a list of the available Python environments on your machine.
Based on your description of your site-packages directory I would assume you do not have your interpreter pointed the proper environment on your new machine. With that said, you would be better served creating a new virtual python environment on your machine and installing each relevant dependency within that environment.
Take a look at this post here for your first best option on re-creating your old python environment on your new machine.
EDIT: I apologize for not reading the question more thoroughly before answering the questions. If this is running on a Windows machine you will need to double check the environment path python is using. It is very easy to install python at a different PATH than the command line environment is checking on a Windows box. If for example your PATH is pointing to a different version of Python and PIP is installing packages somewhere else this issue can occur. Double check your System PATH for python and which version the command line is running.
On the new machine you must source venv/bin/activate so your path environment variables are set properly. In particular, which python should say venv/bin/python rather than /usr/bin/python. Also, take care to conda env update or pip install -r requirements.txt so you'll have suitable venv libraries on the new machine.
I have a full python installation with files in /usr/local/, but also have one that I compiled from source sitting in ~/python_dist. If I look at sys.path on each interpreter I see that they indeed import from different libraries.
Currently I can run $ PYTHONPATH=~/other_py_libs ~/python_dist/bin/python to invoke the custom interpreter with some other modules available in the path. However, I don't want permanently change the global PYTHONPATH variable.
How can I permanently change the python path for only one specific python install?
The easiest way to do this is to use a virtualenv (manage with virtualenvwrapper). With virtual environments you can set up different, isolated python environments (kind of like little python playgrounds). Switching between them (with the help of virtualenvwrapper) is as easy as typing workon envname. You don't have to worry about switching the PYTHONPATH around, and you can direct scripts to use a specific environment simply by running them with the python install in that environment, e.g. using #! /home/myname/.virtualenvs/envname/bin python.
Just reformatted my Mac to Yosemite and installed Python 2 & 3 using Homebrew. I've also setup some aliases in my bash_profile file which allows me to activate which version of Python I want to use. So if I type p3, it'll launch the python3 shell.
I'm wondering if it's possible to activate a specific version of Python without it starting the Python shell? So if I bring any files into the Terminal for instance, it'll use the version which I've activated?
Cheers!
Apologies if this has been answered elsewhere, I had a good search but I couldn't find anything.
Rather than writing your own scripts to manage different Python versions, I would suggest using a highly-used manager that has been tested in and out by the community: pyenv. With pyenv you can:
Easily install several different Python versions from the command line with no issues of them fighting (pyenv install 3.4.2)
Create virtual environments from any one of those versions if you want to compartmentalize the packages that are available (pyenv virtualenv 3.4.2 mypy3projectvenv), and
Set specific environments or versions to be active either
globally (pyenv global [version-or-venv]),
locally in and below folders you configure (pyenv local [ver-or-venv]), usually useful for projects that you have at a specific version/virtualenv, and
local to the shell until closed (pyenv shell [ver-or-venv]) (this is perhaps most-similar to your putative p3 command.
After setting the Python you want to use, all Python-related calls are redirected to their appropriate target (e.g. python, pip, easy_install, ipython*, django-admin*). Don't execute the Python scripts with any special command, just call them normally (or prefix a standard #!/usr/bin/env python shebang)
*If installed in that version/virtualenv
If you use the pyenv-installer script:
curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash
Then add the couple lines it tells you to your ~/.bash_profile script (and either source it or restart bash)...you'll be up and running in seconds. The trick is usually installing all the Python build dependencies with brew (sqlite, OpenSSH, zlib...), but after that then you're golden.