I am trying to run a few python scripts through Crontab in Centos docker container, but nothing I have tried is working.
Firstly I installed cron:
yum install vixie-cron
Then I ran it as a service:
/etc/init.d/crond start
(I also ran /sbin/service crond start because some answers to related questions suggested so)
ps aux | grep cron shows:
root 16917 0.0 0.0 23288 1252 ? Ss 18:53 0:00 crond
root 16929 0.0 0.0 9720 836 pts/0 S+ 18:55 0:00 grep cron
crontab -l looks like:
0 17 1 * * /root/proj/env/bin/python /root/proj/files/frontend/file1.py > /var/log/cron.log
0 9 4 * * /root/proj/env/bin/python /root/proj/files/frontend/file2.py > /var/log/cron.log
0 17 15 * * /root/proj/env/bin/python /root/proj/files/frontend/file3.py > /var/log/cron.log
0 9 18 * * /root/proj/env/bin/python /root/proj/files/frontend/file4.py > /var/log/cron.log
0 14 * * * /root/proj/env/bin/python /root/proj/files/frontend/file5.py > /var/log/cron.log
0 8 * * * /root/proj/env/bin/python /root/proj/files/frontend/file6.py > /var/log/cron.log
* * * * * echo 'Check!!' > /var/log/cron.log
All python scripts and the cron.log file have permission 777
The last entry in crontab is just to check if anything is getting written to the log file... but nothing is being written there..
Any idea how to solve this?
PS: I looked through a ton of related QnAs but none of them helped.
Some answers had suggested writing to /etc/crontab.. so I even made the entry: * * * * * root echo 'Blah' > var/log/cron.log there.. but to no effect :(
After a day wasted in suboptimal Googling and experimentation, the answer to this conundrum was commenting out the following line in the file /etc/pam.d/crond (and then running service crond restart):
session required pam_loginuid.so
This is because of some Docker-Centos security issue. I don't have further details on why this works. (Check this for details.)
Also, troubleshooting this made me realize a few gotchas which might be useful:
Check that you have crond running as a service (if not, use
/etc/init.d/crond start)
Each line in your crontab should be followed be a newline character
Your scripts and log file should have suitable write/execute
permissions
crontab file shouldn't have weird Windows environment characters
(like ^M)
In some rare cases, crond might have different time zone than your
system
Also check /etc/cron.allow and /etc/cron.deny files to verify who
can add/edit cron jobs
May your day be invested in more fruitful pursuits, stranger.
I recommend run cron under host and run command inside the container
example:
0 17 1 * * /usr/bin/docker exec container_name /root/proj/env/bin/python /root/proj/files/frontend/file1.py > /var/log/cron.log
Related
I am trying to run python script via a cron job, but have had no luck the last two days, and I am running out of hairs to pull out.
Some info:
-->I have approximately 20 hours experience using Linux, so I might have missed something very basic.
-->Linux server (Ubuntu) on Linode.com
-->Script runs in terminal
-->Script has the permissions "0644"
-->#!/usr/bin/env python3.7
is added to the beginning of the script
-->The script belongs to the user "adamsavage" and I have tried to add it to both this users cron file and the cron file belonging to root using crontab -e and sudo crontab -e respectively
-->The cron files looks like this, and has a newline at the end:
* * * * * /usr/bin/python3 /home/adamsavage/python-scripts/send_new_sessions.py >> /home/adamasavage/log.txt 2>&1
-->sudo grep CRON /var/log/syslog returns this:
Apr 2 15:25:01 noeluddig CRON[7728]: (adamsavage) CMD (/home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7729]: (root) CMD (/home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7730]: (adamsavage) CMD (echo 'Yo' >> /home/adamsavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7731]: (root) CMD (echo 'Yo' >> /home/adamsavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7732]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
-->I have also tried the following:
* * * * * cd /home/adamsavage/python_scripts/ && /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
and
* * * * * cd /home/adamsavage/python_scripts/ && /usr/bin/python /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
-The script is supposed to send me an sms with some info, which again works when it is ran from terminal.
-I should also mention that just having echo "message" >> /home/adamsavage/ouput.txt does actually run and prints "message" to that file.
What am I missing? Suffice to say, help will be greatly appreciated!:)
You should run your python script with python command:
* * * * * path/to/python /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
For example:
* * * * * /usr/bin/python3 /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
Try to add 2>&1 at the end of your cron line:
* * * * * path/to/python /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt 2>&1
I have no idea why this is, but what made it work was to remove the absolute path to the log file. If anyone can explain this, that would be great!
I am running a Linux instance on ec2 I need a cron job to run every minute
When I use crontab in the terminal ( i am logged in using the terminal btw ) it does not seem to be working and when I use the cron.d option it does not seem to be working either
cron.d file * * * * * ec2-user python3 /home/ec2-user/Programs/file.py
this does not seem to work and I'm out of ideas if somebody could walk me threw the process of creating a cron on ec2 that would be really helpful
To check to see if the cron daemon is running, search the running processes with the ps command. The cron daemon's command will show up in the output as crond.
$ ps -ef | grep crond
root 2560 1 0 07:37 ? 00:00:00 crond
oracle 2953 2714 0 07:53 pts/0 00:00:00 grep crond
The entry in this output for grep crond can be ignored but the other entry for crond can be seen running as root. This shows that the cron daemon is running.
This is my crontab.
SHELL=/bin/bash
PATH=/home1/<user>/.pyenv/shims:/home1/<user>/.pyenv/bin:/usr/lib64/qt-3.3/bin:/usr/nhnkrb5/bin:/usr/bin:/bin:/usr/X11R6/bin:/home1/<user>/.local/bin:/home1/<user>/bin
HOME=/home1/<user>
# Do something
0 1 * * * /home1/<user>/.pyenv/shims/python /home1/<user>/folder/myscript.py >> /home1/<user>/folder/$(date "+%Y.%m.%d-%H.%M.%S").log 2>&1
It runs perfectly from the terminal no matter where I execute it from. I have tried every answer on this page, and my cron doesn't return any errors.
https://askubuntu.com/questions/23009/why-crontab-scripts-are-not-working
I have also checked that my $PATH the cron runs from is identical to the one in my bash. Checked using * * * * * env > /tmp/env.txt
Please tell me what am I doing wrong.
It is better to add the python header at line 1 to your main script. For example if your main script file is app.py then it's content can be following:
!#/home1/<user>/.pyenv/shims/python
# Here goes your Python Script.
and then
$ chmod +x app.py
Now you can add
# Do something
0 1 * * * /home1/<user>/folder/myscript.py >> /home1/<user>/folder/$(date "+%Y.%m.%d-%H.%M.%S").log 2>&1
I ran into the same problem, script working when I am running manually but didn't work with crontab. This method worked.
I am looking to use crontab in Bash to run a python script. What I have below does not work.
SHELL=/bin/bash
11 22 * * * username /usr/lib/python2.7 /mnt/c/Users/Eric/Documents/Feedparser/crontab.py
Nor did this:
SHELL=/bin/bash
PATH=/usr/lib/python2.7
5 22 * * * username python /mnt/c/Users/Eric/Documents/Feedparser/crontab.py
You can try logging the output and error of the execution of the command in crontab using :
11 22 * * * username /usr/lib/python2.7 /mnt/c/Users/Eric/Documents/Feedparser/crontab.py > /tmp/crontab.log 2>&1
This may give you the idea about what the problem is.
you need to remove the username in your cron command. Otherwise, it will try to run it as an executable with arguments python /mnt/c/Users/Eric/Documents/Feedparser/crontab.py
just put :
5 22 * * * python /mnt/c/Users/Eric/Documents/Feedparser/crontab.py
You may wish to consider making your python file executable and calling it directly. Your crontab would then say:
5 22 * * * ./mnt/c/Users/Eric/Documents/Feedparser/crontab.py
or
5 22 * * * cd /mnt/c/Users/Eric/Documents/Feedparser && ./crontab.py
To do this, make your file executable:
chmod +x /mnt/c/Users/Eric/Documents/Feedparser/crontab.py
And add a shebang to the first line of your python file:
#!/usr/bin/env python
I am trying to get a Python script to run hourly using crontab however I cannot seem to get it to work.
The Python program runs fine and completes if I manually run it from terminal.
$ python /home/pi/Documents/Project/Base_Prog.py
My crontab is setup like so:
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow command
#reboot /usr/bin/python /home/pi/Documents/Project/Base_Prog.py
0 * * * * /usr/bin/python /home/pi/Documents/Project/Base_Prog.py
0 * * * * /usr/bin/python /home/pi/Documents/SimpleCronTest.py
Let me clarify though, I followed suggestions on this similar post. I tried the simple test script that creates an output file and that worked. However under the same crontab with the same settings, my program doesn't complete.
The interesting point is that using TOP when the script is due to start, the Python window briefly pops up before disappearing again. So I assume cron is at least sort of working.
I have added the:
#!/usr/bin/env
Python line to the top of the Base_Prog.py file.
My Python program is an API scraper that finds its inputs from one file and writes the results to another, all files in the project directory have full write permissions using chmod 777.
I am at a loss as to what is causing this.
UPDATE
The output log for both the simple test and my programme in the syslog is:
Apr 2 14:29:01 raspberrypi CRON[1455]: (pi) CMD (python /home/pi/Documents/Project/Base_Prog.py)
Apr 2 14:29:01 raspberrypi CRON[1456]: (pi) CMD (python /home/pi/Documents/CronTest.py)
I think full solution is:
First add a SHEBANG line on top of your python script:
#!/usr/bin/env python
Make your script executable with chmod +x
If your script work with stdout or if you need to know what's wrong:
0 * * * * /usr/bin/python /home/pi/Documents/Project/Base_Prog.py >> tmp.log
And just to clarify:
0 * * * * -means the cron will run always when the minutes are 0 (hourly)
You need to see the output of the cron execution for any clues, not just running it from terminal. Perhaps there are some permission/ownership errors or something along those lines. From the cron execution, you can send output to a file to view:
0 * * * * python /home/pi/Documents/Project/Base_Prog.py 1> /dev/null 2> /home/pi/Documents/Project/Base_Prog.err
and you can view /home/pi/Documents/Project/Base_Prog.err after for clues. Or you can also have it emailed to you:
0 * * * * python /home/pi/Documents/Project/Base_Prog.py 1> /dev/null 2>&1 | mail -s "Base Program Output" you#company.com