I know cron is capricious and I am trying to figure out how to deal with it on linux.
I have the following test_cron.py executable Python script for testing cron :
#!/usr/bin/env python
import os
os.makedirs('test_cron_dir')
f = open('test_cron_dir/test_file','w')
f.write('stuff')
f.close()
I added two lines to my crontab to run the script in two different folders :
* * * * * python /home/me/test_cron.py
* * * * * python /home/me/some_folder/test_cron.py
The problem is : cron runs the test_cron.py script located in /home/me/ but does not run the one located in /home/me/some_folder/.
I have changed the paths in the script to absolute paths but it does not change anything to the situation. Also, I have tried to use the root crontab and it does not change anything.
Can anyone please shine the light of knowledge and experience on me ? Thanks a lot.
cron is running crontab(5) entries from the home directory of the user.
You need to change appropriately the directory i.e. to call the chdir(2) syscall (either thru the cd shell builtin, or inside your python script using os.chdir).
You should query the current directory (using getcwd(3), or the pwd command, or the os.getcwd in Python) in your script.
Also check your PATH if running commands.
Related
Solaris environment. This script runs fine when I run directly but crontab does not do anything in it
Code:
19 * * * * python3 usr/src/utils/CE/ce_common_utils/ce_env_utils.py
I'm not getting the difference between crontabs environment and bash's. Anything obvious to check/correct?
Crontab starts in the given user's home directory. Is usr/src/... in that directory or do you need to give the full path to that script?
Figured it out. Needed to provide PYTHONPATH value to the script
I'm building a basic Twitter scraper with Python that I want to run off of my RaspPi 4b on an hourly basis. The script is written and works perfectly when called from the terminal using
python scraper.py
Now, I want to automate it to run without my own physical prompt. I did chmod with the script, then opened the crontab, and using the editor added this line (I understand that it's for every minute, I just want to see it work):
* * * * * /usr/bin/python home/pi/Desktop/twitter_scraper/scraper.py
However, nothing executes on its own. I'm not quite sure why this is because I specified the directories of both the Python program and the interpreter. Do I need to add anything else besides that line to the cron file? The Python script does access other files located in the same directory, but I did not think that would matter much. Do I need to restart my Pi for it to take effect?
When it come to python is better to run the code in to the directory where it is located. And in such case the cron will be something like:
* * * * * cd /home/pi/Desktop/twitter_scraper; /usr/bin/python scraper.py
you do not need anything more just do in this way
* * * * * python /home/pi/Desktop/twitter_scraper/scraper.py
if it doesn't work you can check your system log. and see the errors for debugging.
I am attempting to run a python 3 script every 1 minute using cron on a raspberrypi 3, for testing, where eventually it will just be run once a day.
To start, I made a new cron job using: sudo crontab -e, and typed in the following code for a once a minute job:
*/1 * * * * /home/pi/folder/file.py
Then I saved and closed and waited. My python script emails me text when executed, so I should have seen an email come in. It runs fine (and emails me) when I execute it manually outside of cron.
So, what am I doing wrong with cron for it not to run? And do I need to make the python file executable or something with chmod?
Possible duplicate of Execute python Script on Crontab
EDIT:
Adding comment here since the comment box mangled my formatting.
In your example above it looks like you are just trying to "run" the file. You need to call the python executable, and pass it an argument that points to your file.
From the StackOverflow comment mentioned above look at this crontab entry:
*/2 * * * * /usr/bin/python /home/souza/Documets/Listener/listener.py
Take a look at the first part of the command /usr/bin/python this is pointing to the python executable not just to the .py file you want to run.
I'm rather green in python development but learning nevertheless.
I've written some python codes but they are mostly one-off use and to be run at command line. I still have no idea how to automate and schedule my codes. Lets say I have:
written a python script that pulls some CSV data from API and saves it in /tmp.
written another script to ingest the csv data and transform it into XML per line.
Each time I want to do this, I find myself doing:
$ python getdata.py
$ python converttoxml.py
In shell, I think one can write a wrapper script and cron it. Right? If so, how is this done in the world of python? Bear in mind we have to include all python libraries/ modules used, too.
P.S. developing in Linux environment using PyCharm.
Login to cron using crontab -e
Scroll to bottom at add a line following the following format:
m h dom mon dow command which is minutes, hours, day of month, day of week, command
So if you want to run your command on the hour every hour you would have
* /1 * * * * python path/to/file/getdata.py
whereas if you want it to run at 12 only then you would have
* 12 * * * * python path/to/file/getdata.py
Python files could be treated as executables. You just need to give them executing permissions with chmod +x my_file.py and tell the bash which interpreter should be used to parse the file by adding #!/usr/bin/python as the 1st line of the code file.
Once you've done the two things, you could just run your file using ./my_file.py and the python script should be executed.
From this point on, it's just like any ordinary program and could be used in cron/systemd/whatever other use you need. Once you have a working Python script, it should be treated no differently than any other executable on your system - in this regard, there's no "Python World".
As far as extra modules are concerned - this shouldn't be an issue as Python will still use the same way to resolve library location. It might be necessary to update $PYTHON_PATH to add a library path - but if the library is installed correctly it shouldn't be an issue while for libraries contained within your local directory (only in the case Python fails to find them) you could add:
sys.path.apped(os.path.dirname(__file__)) to the beginning of your script.
First of all, I have tried every method suggested on this site and others but I still can't get it to work.
My python script is located in my home folder. It imports modules like requests, time, and other third party modules. It works fine with command line.
But it doesn't work with crontab. I think the problem is that when it's run from cron, import doesn't work and the script fails.
crontab:
PATH=/sbin:/bin:/usr/sbin:/usr/bin
SHELL=/bin/bash
PYTHONPATH=/home/chanzerre:/usr/lib64/python35.zip:/usr/lib64/python3.5:/usr/lib64/python3.5/plat-linux:/usr/lib64/python3.5/lib-dynload:/usr/lib64/python3.5/site-packages:/usr/lib/python3.5/site-packages
* * * * * /home/chanzerre/script.py
My Python script's structure:
#! /usr/bin/python3.5
import requests as req
import time
from pprint import pprint
# third party imports here
#code here
Can anyone help?
Would give more details if needed.
P.S.
Please, don't mark it as dupe, because I have tried all methods suggested in the similar questions asked by others and for the love of my life, it still doesn't work.
Have you tried making your script.py an executable file? Just typing the path to a Python file won't run the script. You need to add a shebang to your file:
#!/usr/bin/python
and then make it executable:
$ mv /home/chanzerre/script.py /home/chanzerre/script
$ chmod +x /home/chanzerre/script
then your crontab becomes
* * * * * /home/chanzerre/script
Try that and see if it runs your file.
Have you set all the environment variables that your script needs in order to run? If not, it won't work.For example,if you are running that script behind a proxy server, and the proxy variable is not set, it won't work.
Set your proxy variable in crontab or in the python script itself and it will work.