When I'm trying to create a new Python 3 virtual environment by using mkvirtualenv (virtualenvwrapper command) and os.system like this
import os
os.system('mkvirtualenv foo')
nothing happens.
os.system("mate-terminal -e 'workon foo'")
doesn't work either.
The point is to quickly create a new virtual env and work on it later for each project (it's an automation script). virtualenvwrapper is the most convenient option.
The mkvirtualenv and workon commands are shell functions, not executables in your PATH[0].
To make them available in the shell you execute them in, you need to source the virtualenvwrapper.sh shell script defining them. You might be better off calling virtualenv /path/to/foo directly.
How to activate that virtualenv is another story, though, and will depend on the context you want to use it in. If you activate it in a subprocess, each process using it will have to be run in or under that child.
Hth,
dtk
PS In addition, you might look into the subprocess module (or even the third-party sh) for calling external programs. Happy coding :)
[0]: See $ which workon in a terminal vs $ which bash
The following codes in the bash shell script
env_name="<your env name>"
echo "Create virtual environment"
source `which virtualenvwrapper.sh`
mkvirtualenv $env_name -p python<$version>
source $HOME/.virtualenvs/$env_name/bin/activate
workon $env_name
then run bash script (for example: test.sh) from terminal source test.sh
Related
I'm trying to set up a job file to run on a linux cluster using
$ qsub network.job
I have loaded the module needed to execute the script, however a also need to activate a virtual environment that comes with the module.
#$ -S /bin/bash
#$ -cwd
#$ -j y
#$ -o output/
echo "Running job script"
module load python/python3
./network.py
echo "Finished job script"
Including
$source activate machinelearning
does not activate the virtualenv
What do I need to add to the file to activate the virtualenv machine learning that comes when loading the module python/python3?
source /PATH/WHERE/YOUR/VIRTUAL/ENVIRONMENT/EXISTS/bin/activate
for example in the subdirectory venv of your homedir
source ~/venv/machinelearning/bin/activate
Judging by your comment response, the source command may not be the problem. The typical approach in python virtual environments is to call source on the activate script inside the virtual environment. For more information on source, see this superuser post.
In this case, since the command source activate machinelearning works for you in an interactive session, it's likely that there's a script called activate in your home directory (or whichever directory you were working from) that takes machinelearning as an argument, and activates the appropriate virtual environment.
Here's my best stab at solving your immediate problem:
Since you are able to activate the virtual environment in an interactive session, log in to an interactive session and activate the virtual environment. Once you have done so, type which python3 in your terminal, and it should give you the full path to the virtual environment you're working in. Take that full path and replace the last python3 with activate, and that is the script you should source. For example:
$>which python3
/home/cmf05/environments/machinelearning/bin/python3
Then place this in your script between module load python/python3 and ./network.py:
source /home/cmf05/environments/machinelearning/bin/activate
As an aside: module load isn't something that's built in to any systems I've ever worked on, but it looks like you're submitting a job to a cluster that's managed by slurm or similar. It seems like this is loading all of the specific files you need for your compute job. I would recommend tagging the question with your specific cluster computing environment and putting that in the title for more focused help, in case this doesn't solve your issue.
So I have a python script that generates an animation - and it requires libraries that I have in a conda environment. I need to run this script as soon as my computer turns on, so I've written a short bash script that I added to "startup applications". This bash script runs on startup, and reads like this:
#!/bin/bash
conda activate myenv
cd ~/scripts
python generate.py
When I run this in terminal myself, it's fine, but whenever I turn on the computer, the python part of the script doesn't execute, and when I check errors i find:
conda: command not found
and then i also see the python script failed to run because it's missing libraries (from the conda environment not activating)
I have tried adding lines to the bash script replacing "conda activate" with "source activate", I have tried adding echo ". /home/<user>/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc to the bash script, replacing "conda" with /home/barrat/anaconda3/bin/conda, and even adding whoami to the bash script that runs at startup to make sure that i haven't magically become root by chance... none of this has worked. I would really appreciate any help. it's 3 AM and i'm a bit desperate.
You might have solved the issue yet, but for future viewers, this worked for me:
if [ -f "/path/to/anaconda3/etc/profile.d/conda.sh" ]; then
. "/path/to/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate myenv
fi
Add this instead of conda activate myenv.
Well as you are trying to activate an environment to start your scripts, it may also be possible for you to make a startup script itself to do the desired task by using subprocess module from python.
Try making a demo.py script like :
import os
import system
import subprocess
import x
subprocess.run(["command name", "value"]) #for all scripts you want to execute
and then you can put this python script to run at startup.
You can start quite a few amount of operations without noticable speed changes to your system and always can monitor it easily by starting the processes one after the other using time.sleep() in between two calls.
a colleague of mine has written a python script that I need to use, which is called within a shell script. It produces plots with matplotlib. However, when I try to run his script, it fails in matplotlib commands with "ImportError: No module named PyQt4". The python script is called within the shell script with a syntax like
./script.py
script.py begins with a line to specify the python exec to use from within his miniconda environment, like
#!/user/miniconda/envs/py27/bin/python
I think the problem is that the code uses the default PyQt on my system when I run this command. I tried running script.py with the python exec in his environment, but this gives the same error. This also occurs if I try to run the script on his computer when logged into my account. Is there a way that I can run this script as if I were my colleague within my account?
Have your colleague generate a yaml file with his environment dependencies, then create a copy of his environment on your computer to run the script.
# your coworker runs:
conda env export -n [name of his environment] > environ.yml
Once you get yaml file, you can run
conda env create -f environ.yml
to copy the environment. From there, activate it and run the script
# on Windows
activate [environment name]
python ./script.py
# on *nix
source activate [environment name]
python ./script.py
I have a command that only runs correctly inside a Python virtual environment I've configured (as intended). I know that I can run the command as
$ cmd args
once I've activated the venv. But (due to the constraints of the tool I'm using) I need to activate run (and deactivate?) in one line: something equivalent to running
$ activate_somehow cmd args
outside the command line.
Is there a way to do this?
You can generally run something in a virtual environment simply by using a fully qualified path to the script. For example, if I have:
virtualenv .venv
Then I can install something into that virtual environment without activating it by running:
.venv/bin/pip install foo
This should be true for anything installed using standard Python mechanisms.
After looking into the generated bin/activate script, it seems like the only thing relevant to python is the VIRTUAL_ENV variable, so this should be enough to get going:
$ env VIRTUAL_ENV=path/to/venv python ...
Note that the python executable in the bin directory of target environment is just a symlink to globally installed interpreter, which does nothing other that setting process executable path. Assuming the program does not make use of it, utilizing the main binary itself seems harmless. In case you have installed a package which in turn installs some executables, just specify the absolute path:
$ env VIRTUAL_ENV=path/to/venv path/to/venv/bin/executable
You can create a simple wrapper script which runs activate, executes your command, and then deactivates simply by exiting the script in which your environment was activated.
#!/bin/sh
. ${venv-./env}/bin/activate
"$#"
This lets you set the environment variable venv to the path of the environment you want to use, or else uses ./env if it is unset. Perhaps a better design would be to pass the env as the first parameter:
#!/bin/sh
. "$1"/bin/activate
shift
"$#"
Either way, save this somewhere in your PATH ($HOME/bin is a common choice for your private scripts) and give it executable permission.
I found venv-run which should do what you ask:
pip install venv-run
venv-run cmd args
Larsk's answer is probably cleaner, but this is another possible way.
Assuming you use UNIX and your user is user and you have a virtual environment in home (any) directory, ie /home/user/venv, you can make a script like:
#!/bin/sh
export VIRTUAL_ENV=/home/user/venv
export PATH=/home/user/venv/bin:$PATH
python3 "$#"
We can make this script executable (eg call it venv-python3 and do chmod +x venv-python3) and call it as such, or put it some place discoverable in PATH - let's say alongside python. Assuming you have sudo rights:
sudo cp venv-python3 /usr/bin/venv-python3
Then we can call that instead of the python callable. Since the variables are set within the script, explicit call on deactivate is not necessary at exit.
Example:
user#machine ~ % venv-python3 --help
This works for at least for virtualenv version 20.0.17 but if adopted, you should be keeping an eye on what variables bin/activate sets, if this ever changes.
Yes, you can execute the python file using a virtual environment in a single line of command on windows.
venv\Scripts\activate&&python fall_detector.py
I installed pgadmin4 in my home directory in a virtual environment called "pgadmin4".
I use fish shell and it runs perfectly fine with:
~/pgadmin4/bin/python3 ~/pgadmin4/lib/python3.10/site-packages/pgadmin4/pgAdmin4.py
Just in case this helps somebody.
I have virtualenvwrapper and virtualenv installed on an Ubuntu 11.10 machine. I currently use a shell script to workon a virtualenv, and then run a Python application. Something like this:
workon $1
export PYTHONPATH=/media/work/application-$2/src
cd $PYTHONPATH/core
python startapp.py -option1 $3 -option2 $4 --loglevel=INFO
I'd like to switch to Fabric rather than bash (with an eventual goal of making this invocation cross-platform). Can I use Fabric, which is itself a Python application, to make these kinds of calls on the local system? (workon, export, python)
Yup, you can even look at the section in the FAQ related to this very question.
-- Update --
With newer versions of Fabric you can now use the prefix() context manager see here