a colleague of mine has written a python script that I need to use, which is called within a shell script. It produces plots with matplotlib. However, when I try to run his script, it fails in matplotlib commands with "ImportError: No module named PyQt4". The python script is called within the shell script with a syntax like
./script.py
script.py begins with a line to specify the python exec to use from within his miniconda environment, like
#!/user/miniconda/envs/py27/bin/python
I think the problem is that the code uses the default PyQt on my system when I run this command. I tried running script.py with the python exec in his environment, but this gives the same error. This also occurs if I try to run the script on his computer when logged into my account. Is there a way that I can run this script as if I were my colleague within my account?
Have your colleague generate a yaml file with his environment dependencies, then create a copy of his environment on your computer to run the script.
# your coworker runs:
conda env export -n [name of his environment] > environ.yml
Once you get yaml file, you can run
conda env create -f environ.yml
to copy the environment. From there, activate it and run the script
# on Windows
activate [environment name]
python ./script.py
# on *nix
source activate [environment name]
python ./script.py
Related
I understand that this question may have been repeated many times before - I am new to Nifi and unfortunately none of the previous answers worked for me.
I am trying to run a simple python script using the ExecuteStreamCommand processor (in Windows 10). However, I cannot call the python script directly from Nifi as it requires the conda environment to be activated (it gives me an import error for python packages- the same happens when the script is called from both nifi and command prompt).
I have since written a basic batch script to activate the conda environment and run the python script (see below).
activate.bat
CALL conda activate base
python sample.py
CALL conda deactivate
This works well and produces the output I need when run from the command line. However, calling the bat file from Nifi does not seem to do anything. It does not generate an error (simply pass the incoming flowfile to output with the working directory added to each line of the incoming csv file).
The config in Nifi:
Command Arguments: C:\Temp\activate.bat
Command Path: C:\Windows\System32\cmd.exe
Any help would be greatly appreciated.
This worked for me!
cmd "/c activate [my_env] && python my_script.py && deactivate"
Source: Conda environments and .BAT files
Still not sure what the underlying issue was ....
So I have a python script that generates an animation - and it requires libraries that I have in a conda environment. I need to run this script as soon as my computer turns on, so I've written a short bash script that I added to "startup applications". This bash script runs on startup, and reads like this:
#!/bin/bash
conda activate myenv
cd ~/scripts
python generate.py
When I run this in terminal myself, it's fine, but whenever I turn on the computer, the python part of the script doesn't execute, and when I check errors i find:
conda: command not found
and then i also see the python script failed to run because it's missing libraries (from the conda environment not activating)
I have tried adding lines to the bash script replacing "conda activate" with "source activate", I have tried adding echo ". /home/<user>/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc to the bash script, replacing "conda" with /home/barrat/anaconda3/bin/conda, and even adding whoami to the bash script that runs at startup to make sure that i haven't magically become root by chance... none of this has worked. I would really appreciate any help. it's 3 AM and i'm a bit desperate.
You might have solved the issue yet, but for future viewers, this worked for me:
if [ -f "/path/to/anaconda3/etc/profile.d/conda.sh" ]; then
. "/path/to/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate myenv
fi
Add this instead of conda activate myenv.
Well as you are trying to activate an environment to start your scripts, it may also be possible for you to make a startup script itself to do the desired task by using subprocess module from python.
Try making a demo.py script like :
import os
import system
import subprocess
import x
subprocess.run(["command name", "value"]) #for all scripts you want to execute
and then you can put this python script to run at startup.
You can start quite a few amount of operations without noticable speed changes to your system and always can monitor it easily by starting the processes one after the other using time.sleep() in between two calls.
I have a command that only runs correctly inside a Python virtual environment I've configured (as intended). I know that I can run the command as
$ cmd args
once I've activated the venv. But (due to the constraints of the tool I'm using) I need to activate run (and deactivate?) in one line: something equivalent to running
$ activate_somehow cmd args
outside the command line.
Is there a way to do this?
You can generally run something in a virtual environment simply by using a fully qualified path to the script. For example, if I have:
virtualenv .venv
Then I can install something into that virtual environment without activating it by running:
.venv/bin/pip install foo
This should be true for anything installed using standard Python mechanisms.
After looking into the generated bin/activate script, it seems like the only thing relevant to python is the VIRTUAL_ENV variable, so this should be enough to get going:
$ env VIRTUAL_ENV=path/to/venv python ...
Note that the python executable in the bin directory of target environment is just a symlink to globally installed interpreter, which does nothing other that setting process executable path. Assuming the program does not make use of it, utilizing the main binary itself seems harmless. In case you have installed a package which in turn installs some executables, just specify the absolute path:
$ env VIRTUAL_ENV=path/to/venv path/to/venv/bin/executable
You can create a simple wrapper script which runs activate, executes your command, and then deactivates simply by exiting the script in which your environment was activated.
#!/bin/sh
. ${venv-./env}/bin/activate
"$#"
This lets you set the environment variable venv to the path of the environment you want to use, or else uses ./env if it is unset. Perhaps a better design would be to pass the env as the first parameter:
#!/bin/sh
. "$1"/bin/activate
shift
"$#"
Either way, save this somewhere in your PATH ($HOME/bin is a common choice for your private scripts) and give it executable permission.
I found venv-run which should do what you ask:
pip install venv-run
venv-run cmd args
Larsk's answer is probably cleaner, but this is another possible way.
Assuming you use UNIX and your user is user and you have a virtual environment in home (any) directory, ie /home/user/venv, you can make a script like:
#!/bin/sh
export VIRTUAL_ENV=/home/user/venv
export PATH=/home/user/venv/bin:$PATH
python3 "$#"
We can make this script executable (eg call it venv-python3 and do chmod +x venv-python3) and call it as such, or put it some place discoverable in PATH - let's say alongside python. Assuming you have sudo rights:
sudo cp venv-python3 /usr/bin/venv-python3
Then we can call that instead of the python callable. Since the variables are set within the script, explicit call on deactivate is not necessary at exit.
Example:
user#machine ~ % venv-python3 --help
This works for at least for virtualenv version 20.0.17 but if adopted, you should be keeping an eye on what variables bin/activate sets, if this ever changes.
Yes, you can execute the python file using a virtual environment in a single line of command on windows.
venv\Scripts\activate&&python fall_detector.py
I installed pgadmin4 in my home directory in a virtual environment called "pgadmin4".
I use fish shell and it runs perfectly fine with:
~/pgadmin4/bin/python3 ~/pgadmin4/lib/python3.10/site-packages/pgadmin4/pgAdmin4.py
Just in case this helps somebody.
I'd like to use Windows Task Scheduler to run a python script within a virtual environment. I'd like the Scheduler to run a .bat file that will
activate the virtualenv
run the script
These steps work together from the command line, and they work individually in a .bat, but I can't seem to get them to work together from the .bat. It seems the virtualenv is not fully activated when I try to execute the python script and confused as to why.
My .bat looks like this:
call workon venv
cd path/to/Python/proj
python -m script.py
I've tried adding timeouts immediately after the call to workon and tried moving the workon to seperate .bat called from my first file, but the other lines still execute before the virtualenv is activated. Any help is greatly appreciated!
You do not need to activate the virtual environment while running in .bat. All you need to do is to run the python.exe file in your virtual environment.
{path to virtual environment directory}/Scripts/python.exe path/to/your/file.py
In Windows Task Scheduler you can specify the path in which the command prompt will open. So all you need to do is when adding the action, use path to your python in the field Program/script, the name of the file to be run in Add arguments field, and the path to your file.py in Start in field.
P.S if you are reading or writing files in your python file, note that your path will be relative to the one you specify in your start in field in the Action window
You can use an ampersand & operator in a oneliner batch file.
call workon venv & cd path/to/Python/proj & python -m script.py
It will run each command after the other.
You can also double up the ampersand to make it a conditional operator. &&:
call workon venv && cd path/to/Python/proj && python -m script.py
Here the command will only run, if the previous command completed successfully, in other words ERRORLEVEL = 0
Just type
call .\venv\Scripts\activate.bat
in the .bat file and any command afterwards will see the venv activated
for the record call in a cmd pauses the execution of the current script, executes the called one and then resumes.
Create .bat file
write virtual environment activate script location and python file location as below use '&' operator to run two commands.
as below:
"E:\Call Allocation Engine\Development\development_env\Scripts\"activate & python run.py
https://i.stack.imgur.com/31Gkh.png
finally place this file in desired folder and run using cmd.
E:\Call Allocation Engine\Development\Optimisation\Scheduling>file_name.bat
this script will activate virtual environment and run your python code in that environment.
Another way to do this is to make a shortcut of the batch file and then change the "Start in" field.
After that remember to use the full paths in your batch file since it will be running from a difference location.
Edit activate.bat and place this line at the bottom:
python yourscript.py
Schedule the activate.bat itself and it will automatically run your script after the virtual environment activated.
I am setting up calls to python (Anaconda distribution) via BAT files and the windows task scheduler.
I've now used environments for the first time and was trying to set a .bat file up as below:
activate [my_env]
python my_script.py
deactivate
Unfortunately it appears that the second command does not get executed.
Use the 'call' command when activating/deactivating the environment.
call activate [my_env]
python my_script.py
call conda deactivate
See https://github.com/conda/conda/issues/794
Are you sure you need a batch file? I think this should work.
cmd "/c activate [my_env] && python my_script.py && deactivate"
When I made a simple file containing
print("Hello")
Which I called myprint.py and ran
cmd "/c activate anaconda33 && python myprint.py && deactivate"
This worked for me. You could also put this in a one line batch file.
All activate does is put the environment in the front of the PATH. You can just call the absolute path to the python in the environment you want, like C:\Anaconda\python my-script.py.