I have a bash script that I'm trying to execute anytime an hg update occurs. The goal of this bash script is to switch to the correct virtualenv. For the sake of simplicity, this script is called .test - shown as follows:
#!/bin/bash
echo 'testing hg update hook'
source ~/.virtualenvs/myvirtualenv/bin/activate
Whenever I call this script from my shell using source .test everythying works properly; I can see the results of echo and my shell changes to reflect the activated virtualenv
However,when I do an hg update, the virtualenv is not being activated. The script is firing as I can see the echo result; however, my shell is not updated to reflect the activated virtualenv. Below is the hook setup in my .hg/hgrc file is below. Any ideas why my virtualenv isn't being activated in this hook?
[hooks]
# Update to the correct virtualenv when switching branches (hg update branchname)
update = source .test
UPDATE 1: Per this answer, I don't believe the hg update hook is firing in my current shell; which is why the virtualenv activates when I run the script manually but fails from the hook
Your problem is that when you invoke a shell script, any changes to the environment variables do not get exported to the calling shell (hence why you need to call source activate from the surrounding shell).
The good news is that you don't strictly need to call activate in order to access a virtual environment. What activate will do is:
Add the virtualenv's bin directory to $PATH.
Set the VIRTUAL_ENV environment variable.
Modify your prompt.
None of this is necessary in order to use the virtualenv, and you can execute the python binary in the virtualenv without ever using the script; the prompt is likely not relevant for your use case, you can add the directory (or just the python executable) to your path by symlinking it, and you need the VIRTUAL_ENV environment variable only for software that for some reason needs to be aware of the virtualenv it's running in. If necessary, you can figure it out from sys.executable. For example:
import sys, os
def find_venv():
python = sys.executable
for i in xrange(10):
if not os.path.islink(python):
break
python = os.path.realpath(python)
return os.path.dirname(os.path.dirname(python))
if not os.environ.has_key("VIRTUAL_ENV"):
os.environ["VIRTUAL_ENV"] = find_venv()
Related
I'm using direnv to source my virtualenv when I change into the directory.
/project
.envrc
/env <--- my virtualenv
.envrc
source env/bin/activate
When I change directory into /project I get the output:
direnv: loading .envrc
direnv: export +VIRTUAL_ENV -PS2 ~PATH
It prepends the env directory to my PATH environment variable so when I run which python and which pip both point to python and pip that's in my env directory
=> which python
/USER/project/env/bin/python
=> which pip
/USER/project/env/bin/pip
However it doesn't seem to run source env/bin/activate as I expect it to. I expect it to activate my virtualenv by adding the virtualenv name (env) to my CLI prompt and give access to the deactivate command, neither of that happens. Is there something I'm misunderstanding about how direnv and virtualenv work? I'm new to python so I'm not sure if there are existing tools to do something like this.
I think it's important to understand how direnv works to form a proper mental model first; direnv doesn't load the .envrc directly in the current shell. Instead, it starts a new bash shell, executes the .envrc in there, records the changes in environment and exports the diff bash into the current shell.
What is happening here is that:
virtualenv is using $PS1 to set the prompt. This is a local variable and thus not re-exported. direnv also filters PS1 because it causes segfaults on the old macOS bash when it's unset.
The deactivate() function is not exported from the bash sub-shell as it's not an environment variable.
In practice the activation worked as you noticed. python is in the right path and running pip or easy_install is going to install things in the virtualenv. deactivation is not necessary as direnv will automatically unload the environment when cd-ing out of the directory.
To restore the custom prompt, there is more info available on the wiki: https://github.com/direnv/direnv/wiki/Python#restoring-the-ps1
There is a "hidden" feature to do what you want in direnv. You have to take a look at the toolbox that is loaded by direnv for you to use in the .envrc files. You can use the layout command with python (layout python3) to activate a virtualenv on entering the dir, and deactivating it when exiting the directory. It will even take care of creating the virtualenv the first time.
Also take a look at source_up that keep loading .envrc files higher in the file system. I start all my projects by creating a .envrc file with the following:
layout python3
source_up
This will create, activate and deactivate a python virtualenv automatically, and keep on reading variables from higher-level .envrc files. Environement variables for the current project only will go in the local .envrc.
I have a command that only runs correctly inside a Python virtual environment I've configured (as intended). I know that I can run the command as
$ cmd args
once I've activated the venv. But (due to the constraints of the tool I'm using) I need to activate run (and deactivate?) in one line: something equivalent to running
$ activate_somehow cmd args
outside the command line.
Is there a way to do this?
You can generally run something in a virtual environment simply by using a fully qualified path to the script. For example, if I have:
virtualenv .venv
Then I can install something into that virtual environment without activating it by running:
.venv/bin/pip install foo
This should be true for anything installed using standard Python mechanisms.
After looking into the generated bin/activate script, it seems like the only thing relevant to python is the VIRTUAL_ENV variable, so this should be enough to get going:
$ env VIRTUAL_ENV=path/to/venv python ...
Note that the python executable in the bin directory of target environment is just a symlink to globally installed interpreter, which does nothing other that setting process executable path. Assuming the program does not make use of it, utilizing the main binary itself seems harmless. In case you have installed a package which in turn installs some executables, just specify the absolute path:
$ env VIRTUAL_ENV=path/to/venv path/to/venv/bin/executable
You can create a simple wrapper script which runs activate, executes your command, and then deactivates simply by exiting the script in which your environment was activated.
#!/bin/sh
. ${venv-./env}/bin/activate
"$#"
This lets you set the environment variable venv to the path of the environment you want to use, or else uses ./env if it is unset. Perhaps a better design would be to pass the env as the first parameter:
#!/bin/sh
. "$1"/bin/activate
shift
"$#"
Either way, save this somewhere in your PATH ($HOME/bin is a common choice for your private scripts) and give it executable permission.
I found venv-run which should do what you ask:
pip install venv-run
venv-run cmd args
Larsk's answer is probably cleaner, but this is another possible way.
Assuming you use UNIX and your user is user and you have a virtual environment in home (any) directory, ie /home/user/venv, you can make a script like:
#!/bin/sh
export VIRTUAL_ENV=/home/user/venv
export PATH=/home/user/venv/bin:$PATH
python3 "$#"
We can make this script executable (eg call it venv-python3 and do chmod +x venv-python3) and call it as such, or put it some place discoverable in PATH - let's say alongside python. Assuming you have sudo rights:
sudo cp venv-python3 /usr/bin/venv-python3
Then we can call that instead of the python callable. Since the variables are set within the script, explicit call on deactivate is not necessary at exit.
Example:
user#machine ~ % venv-python3 --help
This works for at least for virtualenv version 20.0.17 but if adopted, you should be keeping an eye on what variables bin/activate sets, if this ever changes.
Yes, you can execute the python file using a virtual environment in a single line of command on windows.
venv\Scripts\activate&&python fall_detector.py
I installed pgadmin4 in my home directory in a virtual environment called "pgadmin4".
I use fish shell and it runs perfectly fine with:
~/pgadmin4/bin/python3 ~/pgadmin4/lib/python3.10/site-packages/pgadmin4/pgAdmin4.py
Just in case this helps somebody.
I am running a program using virtualenv. But the multiprocessing.Process spawned here uses system python by default. How do I force it to use virtualenv python.
import os
from multiprocessing import Process
def function1():
# do_something_here
p = Process(func2(), args=(param,))
p.start()
return something
def func2(param):
os.system("which python")
Here it prints "/usr/bin/python". But I need it to use virtualenv python instead.
With sudo venv/bin/python, you effectively activated virtualenv by using python executable in virtualenv directly.
multiprocessing.Process spawn child process with fork(), without exec(), it uses exactly the same python executable as the parent process.
You could confirm python executable in-use by:
>>> import sys
>>> print(sys.executable)
/Users/georgexsh/workspace/tmp/venv/bin/python
>>> print(sys.exec_prefix)
/Users/georgexsh/workspace/tmp/venv/bin/..
Do not use which to determine running python executable path. which, as a Bash command, searches each element of $PATH, for a directory containing an executable file named "python", as you use virtualenv's python directly, not by run its shell activate script first, $PATH not get patched with virtualenv's, as a result, shell command which python would output path of the system python executable.
In fact, at python layer, $PATH is irrelevant, patching $PATH is for the convenience at the Bash layer, to invoke python executable in the virtualenv path by just typing "python", rather than typing full path, What matters most is which python executable is invoked, but not how it is get invoked.
Your problem is here (copied your comment):
#georgexsh I was running it using sudo. By default if you use sudo
then it will use system python. So, I have used "sudo venv/bin/python
main.py" to run the program. Even though I am using venv's python here
it returns "/usr/bin/python" for "os.system('which python')". I don't
understand this behaviour
Basically, what you explain here is something where your virtualenv is not active.
When you activate a virtualenv (. venv/bin/activate), the activation script will change your environment so that your PYTHONPATH is correct and Python executable is searched (and found) first in the virtual env directory. This is what virtualenv does.
By just executing the Python binary from virtualenv directories, your environment is not set for the virtual environment, so any subsequent calls to Python use your default path - as virtualenv is not there to override it.
When you execute sudo, a new process/shell is created and it does not inherit your virtual environment. You might be able to use sudo -E to pass environment but it depends on your sudo. The bulletproof version that should work in every environment is to execute a shell that first activates virtualenv and then executes your script. Something like this:
sudo -- bash -c ". /home/test/mytest/bin/activate; which python"
This executes a bash shell as root, then activates the virtual environment and finally tells you which python it uses. Just modify the above command with your virtual environment path and it might even work.
If your system is shared, just keep in mind that this is a horrible thing to allow your regular users do from security perspective. If you create a passwordless sudo for your regular users to do this, it would give them root access with little tweaking. If it is your own system and the requirement is the knowledge of root password anyway, it does not matter.
I am using Django1.10, when I process python manage.py migrate.
I get:
ImportError: no module named gravatar.
Before that, I have installed python2.7 and run it in virtual environment, as well as django_gravatar.
What's the problem?
You have to activate the virtual environment first. See this section of the docs:
In a newly created virtualenv there will also be a activate shell script. For Windows systems, activation scripts are provided for the Command Prompt and Powershell.
On Posix systems, this resides in /ENV/bin/, so you can run:
$ source bin/activate
For some shells (e.g. the original Bourne Shell) you may need to use the . command, when source does not exist. There are also separate activate files for some other shells, like csh and fish. bin/activate should work for bash/zsh/dash.
This will change your $PATH so its first entry is the virtualenv’s bin/ directory. (You have to use source because it changes your shell environment in-place.) This is all it does; it’s purely a convenience. If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or /path/to/ENV/bin/python-script.py) there’s no need for activation.
The activate script will also modify your shell prompt to indicate which environment is currently active. To disable this behaviour, see VIRTUAL_ENV_DISABLE_PROMPT.
To undo these changes to your path (and prompt), just run:
$ deactivate
On Windows, the equivalent activate script is in the Scripts folder:
\path\to\env\Scripts\activate
I use Linux (Ubuntu 14). I need to save a variable to my virtualenv, so I write on the bash shell with the virtualenv activated:
$ export SECRET_KEY='asdgladfkga....'
and then I can do (using python):
import os
key = os.environ["SECRET_KEY"]
Everything works fine until I restart the computer, then, the variable is erased. How do I truly save it? can I do this without installing virtualenvwrapper?
If you want the SECRET_KEY variable to be defined only when you activate your virtualenv, then you could modify the preactivate script in your virtualenv bin directory by adding
export SECRET_KEY='asdgladfkga....'
This isn't really a virtualenv issue. virtualenv is meant to sandbox your Python environment. Running the activate script, which you source, does change some environment variables at the OS level, but those are not saved/sandboxed, and are not connected to the virtualenv itself other than in the current instance of the shell in which you ran activate.
When you reboot, your .bashrc will load your bash environment back up based on the contents of that file.
If you wanted to keep a "secret key" -- offhand, I'm not sure I get why you want to do this; what are you trying to accomplish? -- you could have a wrapper script that:
Set the key in the current shell via EXPORT
echos the key to a file in your home directory with the same EXPORT command
Then, you could modify your .bashrc to source that file if it exists, so it would get loaded on startup, and therefore survive reboots.
When the virtualenv wrapper was run again, the process would repeat (and not change anything functionally if the key didn't change; you could optimize it to not bother setting the variable again nor write out the file if the key is the same).