How does one create virtual environment that runs on it's own executable?
I used anaconda to create new Venv (to experiment with installing new modules) by cloning some old Venv
conda create --prefix ./new_env --clone C:\full_path_to_old_env\old_env
But new Venv is still using the "python.exe" from old Venv, as can be seen by calling
import sys print(sys.executable)
Because of that, when I install new modules, using for example
pip install selenium
inside the new Venv, it gets installed in the folder of new Venv (as expected) and is unreachable for old executable.
I saw similar question addressed here Changing Python Executable by changing where to install new modules, but that defeats the idea of independent environments.
The only way to for Venv to run it's own copy of python I found is using
sys.executable = r'C:\full_path_to_new_venv\python.exe'
at the beginning of the notebook, but this seems more like forced patch, rather than solution.
This also gets overridden with every initiation of new Venv, so all notebooks running in new Venv would need it
I found what was wrong: I was using Jupyter Notebook to run python virtual environments. But I did not install ipython kernel to new VEnv
Apparently, in such case Jupyter doesn't throw an error when a kernel exists somewhere else and just runs the kernel it found.
Hence, the fix is to install ipython kernel to new VEnv:
$ipython kernel install --name .venv --user
Related
i'm having a weird problem...
I can install packages using the built-in package manager in pycharm. But for some reason everytime i use "pip install (xx)" it is installing the packages in a conda env somewhere on my mac...
How can i solve this ?
I've tried the following:
close --> reopen pycharm //
deactivate and activate the venv //
Checked project intepreter is the right one (which it is...)
You're inside the virtual environment venv, while being inside the Conda base environment (note the (venv) and (base) to the left of your prompt). Conda is likely overriding your venv's pip.
My bet as to why this is happening is that, during installation, you set Conda to autostart its base environment whenever a new terminal is open (be it inside PyCharm or not).
You can try to either:
exit Conda (with conda deactivate) and try pip install again (check to see that you're still inside the venv virtual environment).
install the packages directly from PyCharm's GUI - note the small + sign on the bottom-left of the package list. This won't solve the issue related to your terminal, but will function as a workaround for now.
Note that these aren't guaranteed to work, because you may have additional configurations on your system (either installed directly by you, or indirectly by Conda when you installed it).
I apologize in advance for my poor vocabulary - I do not know much about environments, paths, and things of the sort. I am on macOS Catalina.
I created a program using Spyder from Anaconda. My program uses packages like pandas and numpy which are built into Anaconda. I used to be able to run the program in Terminal with the command: $ python3 app.py.
However, I recently downloaded Python 3.9.1 from https://www.python.org/downloads/mac-osx/. Ever since then, I have been unable to run the program in Terminal because of missing package errors like
import numpy as np
ModuleNotFoundError: No module named 'numpy'
I went to my applications folder, right clicked on Python 3.9.1, and clicked Move To Trash. This did not solve my problem. I reinstalled Anaconda (I did not uninstall it - just simply installed again), but that also did not solve my problem. I am able to run the program in Spyder (from Anaconda), but I wish to run it in Terminal again.
I believe the solution is simple, but I am not sure what to do. I have tried searching and reading but I am not familiar with the terminology. I think I just need to reset the environment, but I am not sure.
Anaconda is used for creating closed enviorments, so you don't need to thrash your computer with global packages.
Imagine you have 2 different projects. Project A works only on python 3.2 and Project B works on 3.8.
That's where anaconda comes in.Managing enviorments with conda
conda create -n PROJECTA python=3.2
conda create -n PROJECTB python=3.8
Now activate env you wish to work with. For macOS
source activate PROJECTA
Now you should see (PROJECTA) instead of (base)
Now inside this PROJECTA you can install modules you require like
pip install numpy
and when executing .py file
move to dir with your app.py file and
python3 app.py
this will be opened in conda enviorment you created and activated, using modules you installed in this env.
You can still edit py file and execute it through shell, but it will throw errors if you try to run it from IDE without linking project to respected conda enviorment.
If you are using PyCharm Configure a Conda vir env in PyCharm
You must create separate environments for every projects or it will get messed up.
conda create -n name_of_environment python=3.6
You must not delete the python folder into trash rather uninstall it Python 3.9.1 and remove its path from the directory.
In short if you run python from terminal and it is not supporting your Installed anaconda packages . You should use anaconda prompt instead of CMD.
Because your anaconda is not added to path rather then it picks up the python 3.9 you have installed from Python 3.9.1 from https://www.python.org/downloads/mac-osx/. ( This is the python with separate environment then anaconda so it wont detect anaconda packages ).
When you run the cmd and enter python it runs the python that you downloaded and installed rather then Anaconda
At the time of installing anaconda it gives option to add conda variables to path you can select those at installing stage / or add manually
In a project where I have to run some Jupyter notebooks, I created a virtual environment using pipenv and installed some packages (note that I used the --site-packages flag).
Although now I am now able to run the notebooks with pipenv run papermill ..., I cannot run them from Jupyter using pipenv run or pipenv shell because of some ModuleNotFoundError exceptions.
In particular, the modules that are note found in the second case are the ones installed in the virtual environment only and not inherited from global-sites.
Indeed, if I check the sys.path I can see the difference in the two cases: in the second there is no ~/.local/share/virtualenvs/... entry.
Why am I having this issue and how can it be solved? (If possible, I would prefer not to pollute my ~/.local/share/jupyter/kernels with other kernels from virtualenvs).
As was suggested here, you also need to make sure that the kernel is also under the venv:
python -c "import IPython"
python -m ipykernel install --user --name=my-virtualenv-name
and then switch the kernel named "my-virtualenv-name" in the jupyter user interface
On my latest clean install, I elected to run Jupyter in it's own virtual environment.
I just made a new venv, installed Jupyter and it's dependencies into it, and then ran jupyter lab from the activated venv. Everything worked fine, and it wasn't until later when I was reading up on some other stuff I noticed the recommended practice is usually to install a new IPython kernel into the environment and use that.
See here:https://medium.com/#eleroy/jupyter-notebook-in-a-virtual-environment-virtualenv-8f3c3448247
And here: https://anbasile.github.io/programming/2017/06/25/jupyter-venv/
My install seems to work fine, and this virtual environment is the only place where I have Jupyter/IPython installed. What is the difference between the way I'm running and the methods suggested in these articles?
Edited to address comments:
Method from links:
Create venv
Install dependencies into venv
Install new IPython kernel into venv
Fire up Jupyter and select newly created IPython kernel for notebooks.
My method:
Create venv
Install Jupyter into venv
Install dependencies into venv
Fire up Jupyter from inside activated venv. Only a single kernel is available.
This Real Python site had a decent explanation of the "why virtual" question. Here's a salient quote:
At its core, the main purpose of Python virtual environments is to
create an isolated environment for Python projects. This means that
each project can have its own dependencies, regardless of what
dependencies every other project has.
--EDIT--
(Since the OP added the steps, side-by-side, the answer is being updated to provided a side-by-side comparison.)
Because Basile created the virtual environment first, you would be able to create different dependency stacks for a new Python project or new Jupyter notebook.
(Speculation here, since I have not gone through the OP's steps:) Since the OP's method placed the Jupyter notebook inside the virtual environment, you'll be limited to that single set of dependencies.
I attempt to create a virtual environment (and store my Python scripts) at directory /Users/MyName/Desktop/TestFolder. Specifically, I'd like to use numpy 1.8.0 in the virtual environment.
My default project interpreter in PyCharm is ~/anaconda/bin/python. How do I do this in PyCharm?
Use virtualenv to create isolated envinronment:
python3 -m venv ~/VirtualPython
~/VirtualPython/bin/pip3 install numpy # and whatever You want to have as dependencies
Make PyCharm be using that virtualenv: https://www.jetbrains.com/help/pycharm/2017.1/creating-virtual-environment.html
Use ~/VirtualPython/bin/python3 as interpreter
More about virtualenv: https://docs.python.org/3/library/venv.html
Feel no fear using Your new venv. Really, there is always a fear like "should I do some magick before virtualenv scripts is really virtual"? Answer is "No". When You run ~/VirtualPython/bin/python3 my_script.py it's always will be executed by virtualenv