I have written my first python script using Spyder as the IDE within Anaconda. Yay! The script pulls data from google finance and emails it to me. When run inside Spyder it works fine.
Now I want to schedule that script to run at a specific time during the day. So, after researching, I have tried to set up the job to run in Cron with the following syntax:
15 12 * * * users/paul/desktop/pythonscript.py
I thought this would run the script at 12:15 but nothing happens.
I tried experimenting by opening the script in IDLE and running it or running it from a terminal but I can't get any of those to work because it tells me that none of the modules are imported.
So...can anybody tell me how to schedule a python script that was written in Spyder to run at a specific time?
Thanks!
You will need to specify the PATH variable within Cron and make sure python3 is in it. (It looks as from your comments you are using python3, make sure you know if you are using 2 or 3, just typing python will usually default to python 2) You can make normal edits like this with:
crontab -e
Then add the full path to python before your call to your job. EDIT: This path needs to be the path to your anaconda environment python (to avoid compatibility issues between other versions of python on your system).
PATH=path/to/anaconda/env/bin #you need to look this up
15 12 * * * python3 users/paul/desktop/pythonscript.py
See How to get CRON to call in the correct PATHs
If you don't include the folder that contains the anaconda environment python3 in your PATH, it will not run exactly like it does in spyder. If you would like to know where anaconda version of python is type this in bash:
conda info --envs
conda env list #or you could try this
If your command includes the call to your anaconda environment python, then you do not need the shebang in
pythonscript.py. If you don't want to include the call to python in your command in crontab, then include the shebang in your python script on the first line.
These may be useful:
run a crontab job using an annaconda env (see the second answer there)
https://conda.io/docs/user-guide/tasks/manage-environments.html (a guide for managing conda environments)
First check whether your script contains the python shebang or not.
#!/usr/local/bin/python
above will the the path for your python enviornment and this will run just fine or change your cron syntax, and tell it to call python to run your file and
use your complete path for your file such as /home/users/.....
15 12 * * * python users/paul/desktop/pythonscript.py
Goal - I created python scripts(Scipt X) using Spyder(conda base),I want to schedule it in cron.
Solution - I create one more script(script Y),written below code
prehand:-
switch to conda env by typing below command in terminal
conda activate //Switch to conda env
which python //check the python version in my case(~anaconda3/bin/python)
Code below in script Y:
import os,import subprocess
subprocess.call("~anaconda3/bin/python <FULL SCRIPT X PATH>")
I call this (script Y) in crontab.
Crontab entry
min hour date month week ~anaconda3/bin/python <full path to Script Y>
you are now ready to schedule spyder scripts in crontab.
Related
I understand that this question may have been repeated many times before - I am new to Nifi and unfortunately none of the previous answers worked for me.
I am trying to run a simple python script using the ExecuteStreamCommand processor (in Windows 10). However, I cannot call the python script directly from Nifi as it requires the conda environment to be activated (it gives me an import error for python packages- the same happens when the script is called from both nifi and command prompt).
I have since written a basic batch script to activate the conda environment and run the python script (see below).
activate.bat
CALL conda activate base
python sample.py
CALL conda deactivate
This works well and produces the output I need when run from the command line. However, calling the bat file from Nifi does not seem to do anything. It does not generate an error (simply pass the incoming flowfile to output with the working directory added to each line of the incoming csv file).
The config in Nifi:
Command Arguments: C:\Temp\activate.bat
Command Path: C:\Windows\System32\cmd.exe
Any help would be greatly appreciated.
This worked for me!
cmd "/c activate [my_env] && python my_script.py && deactivate"
Source: Conda environments and .BAT files
Still not sure what the underlying issue was ....
I was trying to run a Python script via Mac's Automator and the command is very straight forward:
"cd /Users/myname/Desktop/project && python3 myprojectapp.py".
However, every time I tried to run it, Automator raised an error such as ModuleNotFoundError. This was however, impossible since I had all libraries (e.g. Pandas) installed and running the command in the Terminal as written above worked flawlessly.
Now, I've read somewhere for a similar problem to just include:
"export PATH=/usr/local/bin:$PATH" before the command and it worked. Now, before I go on with my life, I would like to understand what exactly this extra line does and how it affects Automator to the point of making the script work.
Thank you in advance!
That command basically modifies the environment variable PATH and puts the directory /usr/local/bin before everything that is currently in PATH. However, that command is temporary, and the environment variable PATH is restored when the session closes.
What could be happening is the python you're running in terminal and the python Automator is running are different./usr/local/bin probably contains the same python version as you are using in terminal. Take a look at ~/.bash_profile to see if something similar to export PATH=/usr/local/bin:$PATH is in there.
Another way to check is to type which python in both and see if it points to the same python. You probably have yet another python somewhere in the list of directories in your PATH variable.
It's common to use virtual python environments to keep track of which python is running and to experiment with python without messing with system python. Examples of these include: Anaconda and virtualenv.
So I have a python script that generates an animation - and it requires libraries that I have in a conda environment. I need to run this script as soon as my computer turns on, so I've written a short bash script that I added to "startup applications". This bash script runs on startup, and reads like this:
#!/bin/bash
conda activate myenv
cd ~/scripts
python generate.py
When I run this in terminal myself, it's fine, but whenever I turn on the computer, the python part of the script doesn't execute, and when I check errors i find:
conda: command not found
and then i also see the python script failed to run because it's missing libraries (from the conda environment not activating)
I have tried adding lines to the bash script replacing "conda activate" with "source activate", I have tried adding echo ". /home/<user>/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc to the bash script, replacing "conda" with /home/barrat/anaconda3/bin/conda, and even adding whoami to the bash script that runs at startup to make sure that i haven't magically become root by chance... none of this has worked. I would really appreciate any help. it's 3 AM and i'm a bit desperate.
You might have solved the issue yet, but for future viewers, this worked for me:
if [ -f "/path/to/anaconda3/etc/profile.d/conda.sh" ]; then
. "/path/to/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate myenv
fi
Add this instead of conda activate myenv.
Well as you are trying to activate an environment to start your scripts, it may also be possible for you to make a startup script itself to do the desired task by using subprocess module from python.
Try making a demo.py script like :
import os
import system
import subprocess
import x
subprocess.run(["command name", "value"]) #for all scripts you want to execute
and then you can put this python script to run at startup.
You can start quite a few amount of operations without noticable speed changes to your system and always can monitor it easily by starting the processes one after the other using time.sleep() in between two calls.
a colleague of mine has written a python script that I need to use, which is called within a shell script. It produces plots with matplotlib. However, when I try to run his script, it fails in matplotlib commands with "ImportError: No module named PyQt4". The python script is called within the shell script with a syntax like
./script.py
script.py begins with a line to specify the python exec to use from within his miniconda environment, like
#!/user/miniconda/envs/py27/bin/python
I think the problem is that the code uses the default PyQt on my system when I run this command. I tried running script.py with the python exec in his environment, but this gives the same error. This also occurs if I try to run the script on his computer when logged into my account. Is there a way that I can run this script as if I were my colleague within my account?
Have your colleague generate a yaml file with his environment dependencies, then create a copy of his environment on your computer to run the script.
# your coworker runs:
conda env export -n [name of his environment] > environ.yml
Once you get yaml file, you can run
conda env create -f environ.yml
to copy the environment. From there, activate it and run the script
# on Windows
activate [environment name]
python ./script.py
# on *nix
source activate [environment name]
python ./script.py
I want to be able to run a python script at the command line using Enthought Canopy, but I don't want to specify the full path to the script.
As I see it, there are two options.
Option 1: Make the python script an executable, add #!/usr/bin/env python to the top of the script, and put the directory containing the script on my $PATH. Now I can execute the script like this:
$ run.py
Option 2: As suggested by Andrew Clark in another SO post, just put the directory containing the script on my $PYTHONPATH. Then I can execute the script like this:
$ python -m run.py
The -m causes python to search the $PYTHONPATH.
I prefer Option 2, and it works fine with the system python on my mac (v2.7.2), but I cannot get it to work with Enthought Canopy. I can load Canopy python and import modules in the same directory as run.py, so I know that I have the path correct. I just cannot execute the script from the command line. Is this a bug or am I doing something wrong?
BTW, it's probably a typo, but just to make sure you should be using the module name, not the file name, with the -m option. For example, python -m run
If that is not the problem then make sure that the python that is used in your option 2 is the python located in your Canopy User virtual environment. You can use the which command to verify that. For example:
$ which python
/Users/YourUserId/Library/Enthought/Canopy_64bit/User/bin/python
If that is not what you get then you can either add that bin folder to the beginning of your PATH environment variable, or you can activate that virtual environment like this:
source /Users/YourUserId/Library/Enthought/Canopy_64bit/User/bin/activate