First of all, I have tried every method suggested on this site and others but I still can't get it to work.
My python script is located in my home folder. It imports modules like requests, time, and other third party modules. It works fine with command line.
But it doesn't work with crontab. I think the problem is that when it's run from cron, import doesn't work and the script fails.
crontab:
PATH=/sbin:/bin:/usr/sbin:/usr/bin
SHELL=/bin/bash
PYTHONPATH=/home/chanzerre:/usr/lib64/python35.zip:/usr/lib64/python3.5:/usr/lib64/python3.5/plat-linux:/usr/lib64/python3.5/lib-dynload:/usr/lib64/python3.5/site-packages:/usr/lib/python3.5/site-packages
* * * * * /home/chanzerre/script.py
My Python script's structure:
#! /usr/bin/python3.5
import requests as req
import time
from pprint import pprint
# third party imports here
#code here
Can anyone help?
Would give more details if needed.
P.S.
Please, don't mark it as dupe, because I have tried all methods suggested in the similar questions asked by others and for the love of my life, it still doesn't work.
Have you tried making your script.py an executable file? Just typing the path to a Python file won't run the script. You need to add a shebang to your file:
#!/usr/bin/python
and then make it executable:
$ mv /home/chanzerre/script.py /home/chanzerre/script
$ chmod +x /home/chanzerre/script
then your crontab becomes
* * * * * /home/chanzerre/script
Try that and see if it runs your file.
Have you set all the environment variables that your script needs in order to run? If not, it won't work.For example,if you are running that script behind a proxy server, and the proxy variable is not set, it won't work.
Set your proxy variable in crontab or in the python script itself and it will work.
Related
Solaris environment. This script runs fine when I run directly but crontab does not do anything in it
Code:
19 * * * * python3 usr/src/utils/CE/ce_common_utils/ce_env_utils.py
I'm not getting the difference between crontabs environment and bash's. Anything obvious to check/correct?
Crontab starts in the given user's home directory. Is usr/src/... in that directory or do you need to give the full path to that script?
Figured it out. Needed to provide PYTHONPATH value to the script
I'm building a basic Twitter scraper with Python that I want to run off of my RaspPi 4b on an hourly basis. The script is written and works perfectly when called from the terminal using
python scraper.py
Now, I want to automate it to run without my own physical prompt. I did chmod with the script, then opened the crontab, and using the editor added this line (I understand that it's for every minute, I just want to see it work):
* * * * * /usr/bin/python home/pi/Desktop/twitter_scraper/scraper.py
However, nothing executes on its own. I'm not quite sure why this is because I specified the directories of both the Python program and the interpreter. Do I need to add anything else besides that line to the cron file? The Python script does access other files located in the same directory, but I did not think that would matter much. Do I need to restart my Pi for it to take effect?
When it come to python is better to run the code in to the directory where it is located. And in such case the cron will be something like:
* * * * * cd /home/pi/Desktop/twitter_scraper; /usr/bin/python scraper.py
you do not need anything more just do in this way
* * * * * python /home/pi/Desktop/twitter_scraper/scraper.py
if it doesn't work you can check your system log. and see the errors for debugging.
I need to run a cron job on a python script to generate basemap plots.
The script by itself runs ok manually.
A simple print("Hello") at the start of the program with the rest commented out also runs ok on cron with
*/10 * * * * /usr/bin/python3 ~/PythonFiles/TestScript.py > /dev/null 2>&1 >>log.txt
I made the file an executable using chmod +x and added a shebang (#!/home/usr/anaconda3/bin/python) at the start of the program. I can monitor activity in the log file via a printed message at the start of the program too
When I come to run the "normal" program which includes modules (urllib.request, datetime, matplotlib, basemap, pygrib, numpy, ...), the script then stops outputting anything to log.txt
So I suspect it is to do with modules and possibly their locations. I checked and they seem to have been installed in various places (.../pkgs, .../conda-meta, .../site-packages, etc...)
First of all, is what I suspect correct?
Secondly, how do I fix it so that cron knows where to find all the libraries to run the job?
Many thanks!
I suspected it was to do with module location paths. After trawling through websites and tweaking inputs to cron, the following works!
SHELL=/bin/sh
HOME=/home/stephane
PYTHONPATH=/home/stephane/anaconda3/bin/python
PATH=/home/stephane/anaconda3/lib/python3.6/site-packages
*/2 * * * * /home/stephane/anaconda3/bin/python ~/PythonFiles/TestScript.py >/dev/null 2>&1 >> log.txt
Note: matplotlib seems to need "import matplotlib as mpl; mpl.use('Agg')" to run off cron.
Thanks to all!
I am trying to set up a cron job which is executing a python file of mine. The python file is using some manually installed modules. The cron job now throws an error, as it 'cannot find' the specified module (yes, I tested it: if executed manually the script does work & have access to the module).
I did now recieve the cryptic info (from the hoster’s support) to 'try adding the -m flag to the command, followed by the path to the module that it cannot find.' Unfortunatelly I do not quite understand this advice.
Assuming that my cron job command (via Cpanel) would out of the box be:
0 * * * * python /home/public_html/cgi-bin/cronrun.py
which works if the python script does not rely on external modules.
So my questions are:
Is the -m flag appropriate?
If so, how do I use it?
And what do I do, if there is more than just one additional module that the script needs?
Thank you very much in advance!
Your cron job won't likely be running with the same environment that you have. To see this, first run env > [somepath_that_you_can_reach]. Then set up a cron to do the same thing in a shell script using a different path. Compare the two. You will need your PYTHONPATH to be the same for the cron job to work. If that is the problem, then in your python script:
import sys
sys.path.append('[the path part that you need for it to work]')
before your import statements.
I know cron is capricious and I am trying to figure out how to deal with it on linux.
I have the following test_cron.py executable Python script for testing cron :
#!/usr/bin/env python
import os
os.makedirs('test_cron_dir')
f = open('test_cron_dir/test_file','w')
f.write('stuff')
f.close()
I added two lines to my crontab to run the script in two different folders :
* * * * * python /home/me/test_cron.py
* * * * * python /home/me/some_folder/test_cron.py
The problem is : cron runs the test_cron.py script located in /home/me/ but does not run the one located in /home/me/some_folder/.
I have changed the paths in the script to absolute paths but it does not change anything to the situation. Also, I have tried to use the root crontab and it does not change anything.
Can anyone please shine the light of knowledge and experience on me ? Thanks a lot.
cron is running crontab(5) entries from the home directory of the user.
You need to change appropriately the directory i.e. to call the chdir(2) syscall (either thru the cd shell builtin, or inside your python script using os.chdir).
You should query the current directory (using getcwd(3), or the pwd command, or the os.getcwd in Python) in your script.
Also check your PATH if running commands.