I need to run a cron job on a python script to generate basemap plots.
The script by itself runs ok manually.
A simple print("Hello") at the start of the program with the rest commented out also runs ok on cron with
*/10 * * * * /usr/bin/python3 ~/PythonFiles/TestScript.py > /dev/null 2>&1 >>log.txt
I made the file an executable using chmod +x and added a shebang (#!/home/usr/anaconda3/bin/python) at the start of the program. I can monitor activity in the log file via a printed message at the start of the program too
When I come to run the "normal" program which includes modules (urllib.request, datetime, matplotlib, basemap, pygrib, numpy, ...), the script then stops outputting anything to log.txt
So I suspect it is to do with modules and possibly their locations. I checked and they seem to have been installed in various places (.../pkgs, .../conda-meta, .../site-packages, etc...)
First of all, is what I suspect correct?
Secondly, how do I fix it so that cron knows where to find all the libraries to run the job?
Many thanks!
I suspected it was to do with module location paths. After trawling through websites and tweaking inputs to cron, the following works!
SHELL=/bin/sh
HOME=/home/stephane
PYTHONPATH=/home/stephane/anaconda3/bin/python
PATH=/home/stephane/anaconda3/lib/python3.6/site-packages
*/2 * * * * /home/stephane/anaconda3/bin/python ~/PythonFiles/TestScript.py >/dev/null 2>&1 >> log.txt
Note: matplotlib seems to need "import matplotlib as mpl; mpl.use('Agg')" to run off cron.
Thanks to all!
Related
I'm building a basic Twitter scraper with Python that I want to run off of my RaspPi 4b on an hourly basis. The script is written and works perfectly when called from the terminal using
python scraper.py
Now, I want to automate it to run without my own physical prompt. I did chmod with the script, then opened the crontab, and using the editor added this line (I understand that it's for every minute, I just want to see it work):
* * * * * /usr/bin/python home/pi/Desktop/twitter_scraper/scraper.py
However, nothing executes on its own. I'm not quite sure why this is because I specified the directories of both the Python program and the interpreter. Do I need to add anything else besides that line to the cron file? The Python script does access other files located in the same directory, but I did not think that would matter much. Do I need to restart my Pi for it to take effect?
When it come to python is better to run the code in to the directory where it is located. And in such case the cron will be something like:
* * * * * cd /home/pi/Desktop/twitter_scraper; /usr/bin/python scraper.py
you do not need anything more just do in this way
* * * * * python /home/pi/Desktop/twitter_scraper/scraper.py
if it doesn't work you can check your system log. and see the errors for debugging.
Recently I set up a twitter retweet bot, but after few minutes it's going down.
I used this python bot and I'm using "nohup python mrbot.py &" to keep it running in the background after closing the terminal. Even if I use it as "python mrbot.py", it stops working after around 30 minutes. So could someone please tell me how to automatically restart it every 30 minutes?
I'm a very beginner at this. *Remember I'm running it in background using "nohup". I don't know if this helps.
Crontab would be the ideal way to do it. Please search for crontab and how to enable and schedule a crontab job.
For your example a crontab entry would look like
30 * * * * /path/to/your/python/script.py
Since you are running ubuntu, you should have python in your path and have a shebang in the python script. A shebang is the first line in your python script with path to the python executable, it is in following format
#!/usr/bin/python
You are on Ubuntu, so you can use crontab command. Open terminal and type:
crontab -e
This will prompt you with (probably) an empty document. Then write the following line:
*/30 * * * * /usr/bin/python script.py
This will make sure your script runs every 30 minutes.
You can read more about crontab and what it does and how it works here
I've been trying to automatically execute the script facebook-online-friend-tracker, which opens chrome, logs into facebook and writes the number of online friends onto a .csv file (https://github.com/bhamodi/facebook-online-friend-tracker).
I wrapped it into a script that I've called facebooktracker.
When I execute ./facebooktracker manually from the terminal, everything works fine. But since I want to collect some statistics, I have set up a cron job to work every 10 minutes.
By using: crontab -e, I've set:
*/10 * * * * /home/enrico/facebooktracker
and it does not work, meaning that it doesn't write on the .csv file (syslog shows that the command has been executed though).
I have tried to use Cron to execute a simple script that writes "Hello world" on a file and it works fine, I've tried to use Cron to open a GUI application and it works fine.
Therefore it looks like the script works fine, cron works fine, but they are not willing to work together.
Things that I have tried (yet to no avail) are:
*/10 * * * * env DISPLAY =:0 /home/enrico/facebooktracker
*/10 * * * * env DISPLAY =:0 /home/enrico/facebooktracker > /dev/null 2>&1
Directly use the script facebook-online-friend-tracker without wrapping, by setting in cron:
*/10 * * * * /home/enrico/anaconda2/bin/facebook-online-friend-tracker --user "username" --password "password" --path "path"
Add echo "Hello world" at the end of the facebooktracker script, setting the output to a .log file ( >> facebooktrackerlog.log) and it does write "Hello world", but still doesn't write the number of online facebook friends on the .csv file
I've run out of ideas. Anyone has a clue? I'd really appreciate it. Thanks!
When you run it manually it's running as you with any .profile etc loaded. When cron runs it your profile hasn't been loaded. Try loading your profile as the first part of the cron job.
I'm the author of the facebook-online-friend-tracker script. I see that you had some trouble setting up a cron job to execute the script every 10 minutes. You'll be happy to know that as of v2.0.0, I've implemented scheduling as part of the script. You no longer have to set up cron jobs or task schedulers. Simply run the script once and follow the prompts. To upgrade, simple run:
pip install facebook-online-friend-tracker --upgrade
The issue is that when I execute the Python script normally from the terminal it is working fine but when the same file is being executed from the cron, there is no update at the server end.
File permissions have been set to 755. Earlier getting an error "No MTA installed, discarding output"; to solve that I use >/dev/null 2>&1 at the end of my cron job. After that I get no error but still the issue remains the same. Also I have mentioned the environment on top of my python script.
configuration of cron is as follows:
* * * * * sudo python3 /home/pi/json_working/json_to_server_update.py >/dev/null 2>&1
the problem is solved now. I am using user crontab and i solved the issue by using os.path.isfile(os.path.join("path", "file name")) rather than os.path.isfile("path of file"). The latter one is actually a path and not a file so the output was always false and the sync was not made due to that. Now everything is working fine.
I am writing a script in python3 for Ubuntu that should be executed all X Minutes and should automatic start after logging in. Therefore I want to create a daemon (is it the right solution for that?) but I haven't found any modules / examples for python3, just for python 2.X. Do you know something what I can work with?
Thank you,
I would simply make the script, and have it somewhere, and then add a line to the crontab of the user who you want to run the script. This may be the root.
sudo crontab -e
To start the editor of the crontab
X * * * * /usr/bin/python /path/to/the/script
This way the script will be executed every X minutes. No need to daemonize, no need to make your own timer in the script.
Suppose for python script name is monitor. use following steps:
copy monitor script in /usr/local/bin/ (not necessary)
Also add a copy in /etc/init.d/
Then execute following command to make it executable
sudo -S chmod "a+x" "/etc/init.d/monitor"
At last run update.rc command
sudo -S update-rc.d "monitor" "defaults" "98"
this will execute you monitor whenever you login for all tty.