I've created a virtualenv for my project on Linux and installed the packages I need just fine. When setting up my project in Pycharm, I can select the interpreter to use from venv/bin and Pycharm picks up all the packages in that virtualenv.
I also like to develop on a Windows machine, and I'd like to be able to use the same virtualenv I created on Linux when I clone it from git. However the interpreter created by Linux is a script that runs python3; this means I'd have to create a new Windows style virtualenv.
So my question is, is there a way I can create a venv that can be used cross platform?
A virtualenv is specific to the platform and location it was created on. You should not check it into version control. Each environment you develop in will have its own virtualenv. You make your package installable with setup.py, and/or specify requirements in requirements.txt in order to recreate the virtualenv.
Related
TL;DR: can't configure a Python Interpreter on PyCharm (Windows) using an existing Poetry environment in WSL. When trying to set the Poetry environment path under Add Python Interpreter > Poetry Environment > Existing Environment, the needed Python executable simply does not show. What am I doing wrong?
====================================================
Details:
I'm using PyCharm Pro 2021.3 on Windows 11, with Python running on WSL2 (Ubuntu 20.04). I am trying to add a python interpreter for an existing Poetry environment I created on WSL2, and it just does not seem to work.
PyCharm's current support of Poetry is via adopting this plugin. From what I could gather from the plugin's official documentation, in order to define an interpreter with an existing Poetry environment, I go to Python Interpreter > Add > Poetry Environment, choose Existing environment, and put in the path to that specific environment:
In order to find the path to that environment, I run "poetry env info", which gives a path in the following pattern:
\\wsl$\Ubuntu-20.04\home\$USER$\.cache\pypoetry\virtualenvs\my-pretty-project-<some-hash>-py3.8\
When running which python in the environment, I see the python executable is at:
\\wsl$\Ubuntu-20.04\home\$USER$\.cache\pypoetry\virtualenvs\my-pretty-project-<some-hash>-py3.8\bin\python
However - when I browse to that location in PyCharm, the Python file simple does not show. The bin directory appears as empty - as also described in this question. However - and similarly to what described in said question - if I try to redefine the default interpreter path for WSL to be the path to that Poetry environment, the Python executable is there and kicking:
(The solution described in the aforementioned question, sadly, does not work for my problem, as I am already using the patch to the Poetry environment).
What can I do to make this work?
Let me get this straight: You want PyCharm for Windows to execute Python binaries in WSL?
That cannot happen.
Binaries in WSL are "ELF" binaries which Windows cannot execute (outside WSL). If the virtualenv was created by poetry from within WSL, it will contain ELF Python binaries. And that is why PyCharm for Windows won't ever pick it up. Because ultimately PyCharm for Windows relies on Windows to execute the binaries, as long as you don't choose the WSL option.
Explicitly selecting the "WSL" option indicates to PyCharm that for this particular virtualenv you want PyCharm to invoke the binaries using WSL.
The solution is either to re-create the virtualenv in Windows, or just use the "WSL" option but you have to manage the poetry manually via WSL shell.
Can I have two separate installations of the same Python version on the same machine with separately tracked libraries and so on? I mean without using virtual machines or Docker.
The machine is Macbook Air M1.
You can but it is not recommended. The recommended way is to create a virtual environment see: https://docs.python.org/3/library/venv.html.
If you don't want a virtual environment you can just download different python versions from the python website. I would recommend to not add the installation folders to your PATH variable yet.
Go to the python installation folders (see this question for more info about the installation directory: python location on mac osx) and change the name of the python executable (python.exe) to something like python3x.exe where x is the python version. Also do this for pip in the Scripts folder (If there is more than one pip executable you can delete those). Now add the installation folders to your path variable.
Now you can run your python version of choice from the terminal with:
python3x <your python program>
If you want to install packages for that version of python run:
pip3x <packages you want to install for that python version>
I have python 3.8 installed on my pc, but i need 3.7 for a specific task. When i try to set up a virtual environment via
virtualenv -p "my/path/to/python37.exe"
it calls the installer, so i am to install py37 manually, but then it ends up with an error:
Error 0x80070666: Cannot install a product when a newer version is installed
Is there a proper way to implement such a thing?
Thanks!
Have you tried anaconda or miniconda (the lighter version of anaconda)? Having more python versions in different environments and switching between them is quite easy.
I haven't installed Anaconda. If you want to have multiple different environments with pure Python you can do it like this:
1) Install the python versions that you want with the exe installer, ie Python 2.x.x, Python 3.7.x, Python 3.8.x etc, maybe inside a common folder like C:\Python
2) Then edit the System Variables path and pinpoint to the folder for the version you want to create a virtual environment. You need two entries here, one to python folder (for the python.exe) and one to the Scripts folder (for the pip.exe)
3) Open command prompt and hit python. You ll see that it's showing the apporpriate version. Install the virtual environment with "python -m venv name_of_env_you_want"
4) If you want to create a virtual environment with a different version change the paths
I am working on a django project on two separate systems, Debian Jessie and Mac El Capitan. The project is hosted on github where both systems will pull from or push to.
However, I noticed that on my Debian, when I run python --version, it gives me Python 3.4.2 but on my Mac, it gives me Python 2.7.10 despite being in the same virtual environment. Moreover, when I run django-admin --version on my Debian, it gives me 1.10 while on my Mac, 1.8.3.
This happens even when I freshly clone the projects from github and run the commands.
Why is it that the virtual environment does not keep the same version of python and django?
Now you understand that virtual environments can't be transferred easily from machine to machine. It's common to use the
pip freeze
command and store its output in a file called requirements.txt. Then anyone else can rebuild your envirnment on their machine by running
pip install -r requirements.txt
When you create a new virtual environment you can say which Python interpreter you want to use with the -p or --python switch, which should be followed by the path to the correct executable.
I'd personally recommend against modifying the system Python in any way, because system maintenance routines often rely on its integrity. It's relatively simple to install new copies for Python 2 and 3 somewhere like /usr/local/bin (Mac users often use brew for this) and have virtual environments that rely on different Python interpreters.
Thanks to #Oliver and #Daniel's comments that lead me to the answer why it did not work.
I started the virtual environment on my Debian with python 3. virtualenv made the virtual environment but it was specifically for Debian.
When I used it for mac, since it could not run the python executable in the virtual environment (since it is only compatible with Debian), hence, it used my Mac's system python, which is Python 2.7.10.
In summary, as virtualenv uses the python executable on the system, when the python executable is run on another system, it will not work.
I'm developing a python project on my desktop (OS X) and it's running well. Now I need to run it on a computing cluster that is running Linux and I have no root access. On my Home in the computing cluster I installed Anaconda locally. Then when I run my project there I got so many strange bugs and errors. THe code runs fine on my desktop. So I thought probably the problem is that probably Anaconda has newr modules than what I have and that's probably causing errors. (backward compatibility issues)
Is there any way to use the same python modules versions I have in my desktop on the computing cluster?
I wonder if there is a command that produces all the packages and modules I have on my desktop with their versions and then import this file into some tool in that linux machine and then I get a cloned version of my setup. Is there something like that?
Use pyenv to control the python version
pyenv is designed for installing python into the home folder so you don't need root access (but even with root access it's a great idea), it lets you use a specified version of python within your home folder hierarchy rather than the OS version. You should also install pyenv virtualenv so you can use a virtual environment for your project (not strictly essential, but virtual environments are a great idea and you should always use one, with pyenv they're practically effortless).
One of the neat things about pyenv is the pyenv local command which specifies which version of python (or which virtualenv) should be used for a folder (and subfolders), once you've used pyenv local in your project folder to set the python version any time you use the python command it'll use the version set by pyenv local. It's not needed if you only install one version of python, and don't use a virtualenv (in that case you can use pyenv global to set the version for the user). The neatest thing about pyenv local/global setup is it works great with both scripts and manually invoking python, it is simply set-and-forget unlike other python environment managers which require activation.
In brief, once you've set up pyenv you control exactly what version of python will run, and as it is installed into the home folder the OS has no power to affect it.
Having installed pyenv and pyenv virtualenv, you would then use it to install the same version of python as is used on your development machine, the commands you'd run would be something like this:
pyenv install 3.4.2
pyenv virtualenv 3.4.2 my_project_env
cd my_project
pyenv local my_project_env
Install modules into python environment
To get a list of python module version use pip freeze, you can do this on your development machine:
pip freeze > requirements.txt
Now copy requirements.txt to your deployment machines (with pyenv already setup using pyenv local or pyenv global) and run:
pip install -r requirements.txt
Which will install the same modules into the python environment.
Duplicating a pyenv
While it's kind of dodgy, once you've done this, you can even copy the entire installation (i.e. at least the .bashrc file and .pyenv folder) onto other machines, and if the machines have the same OS and the home folders have the same name, the transplanted environment should work fine. It might be more responsible to use a setup script but I have copied pyenv environments without anything terrible happening.
If you have pip installed you could try this pip freeze.
If you want to get list of modules from python shell:
help('modules')