Develop Reference

How to get crontab job depencies working and avoid command not found

My cron job needs to run a shell script which calls python commands which depends on python and a pip package. How can I get my script to work in cron?
My shell script is simply
#!/bin/bash
TOKEN=XXXX-XXXX-XXXX
bdate=$(date -d "$date -30 days" +"%Y%m%d")
# slack-cleaner was installed with pip install slack-cleaner
slack-cleaner --token $TOKEN --message --channel ci-builds --botname circle-ci --before $bdate
My crontab is
50 3 * * * bash /home/myuser/bin/slack_cleaner.sh
I get the errors in the mail log
tail -f /var/mail/myuser # outputs
X-Cron-Env: <HOME=/home/myuser>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=myuser>
Message-Id: <20210108034301.5F00F44531#someplace>
Date: Fri, 8 Jan 2021 03:43:01 +0000 (UTC)
/home/myuser/bin/slack_cleaner.sh: line 5: slack-cleaner: command not found

Load the user's PATH variable in crontab -e
copy the output of echo $PATH and then run crontab -e
add these 2 lines before your cron job(s)
SHELL=/bin/bash
PATH=<paste what you copied in step 2 here>

Cron job wont run python script - even after two days of googling

I am trying to run python script via a cron job, but have had no luck the last two days, and I am running out of hairs to pull out.
Some info:
-->I have approximately 20 hours experience using Linux, so I might have missed something very basic.
-->Linux server (Ubuntu) on Linode.com
-->Script runs in terminal
-->Script has the permissions "0644"
-->#!/usr/bin/env python3.7
is added to the beginning of the script
-->The script belongs to the user "adamsavage" and I have tried to add it to both this users cron file and the cron file belonging to root using crontab -e and sudo crontab -e respectively
-->The cron files looks like this, and has a newline at the end:
* * * * * /usr/bin/python3 /home/adamsavage/python-scripts/send_new_sessions.py >> /home/adamasavage/log.txt 2>&1
-->sudo grep CRON /var/log/syslog returns this:
Apr 2 15:25:01 noeluddig CRON[7728]: (adamsavage) CMD (/home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7729]: (root) CMD (/home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7730]: (adamsavage) CMD (echo 'Yo' >> /home/adamsavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7731]: (root) CMD (echo 'Yo' >> /home/adamsavage/log.txt)
Apr 2 15:25:01 noeluddig CRON[7732]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
-->I have also tried the following:
* * * * * cd /home/adamsavage/python_scripts/ && /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
and
* * * * * cd /home/adamsavage/python_scripts/ && /usr/bin/python /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
-The script is supposed to send me an sms with some info, which again works when it is ran from terminal.
-I should also mention that just having echo "message" >> /home/adamsavage/ouput.txt does actually run and prints "message" to that file.
What am I missing? Suffice to say, help will be greatly appreciated!:)

You should run your python script with python command:
* * * * * path/to/python /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt
For example:
* * * * * /usr/bin/python3 /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt

Try to add 2>&1 at the end of your cron line:
* * * * * path/to/python /home/adamsavage/python_scripts/send_new_sessions.py >> /home/adamasavage/log.txt 2>&1

I have no idea why this is, but what made it work was to remove the absolute path to the log file. If anyone can explain this, that would be great!

Cronjobs run differently (correctly) when command typed at console

I have encountered a strange problem with crontab: I have correct entries in crontab, have set my path correctly, and yet still when the jobs are run through crontab, they run differently (e.g. log files are not produced, and data is not written to file). However, when I type the EXACT commands at the command line, the scripts work as intended.
This is not likely a permissions problem, as I am running the cronjobs as myself.
Here are the relevant details:
Crontab entries:
0,30 8-18 * * 1-5 python /home/rascal/work/skunkworks/scripts/python/downloaders/make_whoopy.py
0,30 7-18 * * 1-5 /home/rascal/work/skunkworks/scripts/python/scrapers/funny_quotes.sh
0,30 7-20 * * 1-5 /home/rascal/work/skunkworks/scripts/python/scrapers/even_funnier_quotes.sh
/var/log/syslog entries (filtered for CRON)
Nov 1 09:00:01 BIGBERTHA CRON[8998]: (rascal) CMD (/home/rascal/work/skunkworks/scripts/python/scrapers/funny_quotes.sh)
Nov 1 09:00:01 BIGBERTHA CRON[8999]: (rascal) CMD (/home/rascal/work/skunkworks/scripts/python/scrapers/even_funnier_quotes.sh)
Nov 1 09:00:01 BIGBERTHA CRON[9004]: (rascal) CMD (python /home/rascal/work/skunkworks/scripts/python/downloaders/make_whoopy.py)
Nov 1 09:09:01 BIGBERTHA CRON[9306]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && /usr/lib/php/sessionclean)
Nov 1 09:09:01 BIGBERTHA CRON[9313]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Nov 1 09:17:01 BIGBERTHA CRON[9495]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Environment (on Server machine running cron jobs)
rascal#BIGBERTHA:~/work/skunkworks/scripts/python/scrapers$ which python
/usr/bin/python
rascal#BIGBERTHA:~/work/skunkworks/scripts/python/scrapers$ which bash
/bin/bash
rascal#BIGBERTHA:~$ which scrapy
/usr/local/bin/scrapy
funny_quotes.sh
#!/bin/bash
. /home/rascal/.profile
cd /home/rascal/work/skunkworks/scripts/python/scrapers
scrapy crawl funny_quotes
Running in command prompt ...
rascal#BIGBERTHA:~$ /home/rascal/work/skunkworks/scripts/python/scrapers/funny_quotes.sh
2017-11-01 09:57:05 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: scrapers)
2017-11-01 09:57:05 [scrapy.utils.log] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'scrapers.spiders', 'SPIDER_MODULES': ['scrapers.spiders'], 'BO
T_NAME': 'scrapers'}
2017-11-01 09:57:05 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.corestats.CoreStats']
...
Why are jobs being run differently through cron?

It turns out that this was related to an environment issue. Cron reported to syslog that it had run the job, yet no files/log files were created.
Eventually, the way I solved this was to add a MAILTO=username to my crontab. After I had done this, when the job run and failed, I saw the error message:
scrapy: command not found
I then resolved this by adding the following statment to my bash script:
PATH=$PATH:/usr/local/bin/

1) According to my knowledge, don't we do bash funny_quotes.sh instead of just funny_quotes.sh
2) Change your cron commands from
python /home/rascal/work/skunkworks/scripts/python/downloaders/make_whoopy.py
to
cd /home/rascal/work/skunkworks/scripts/python/downloaders/ && python make_whoopy.py
and same this one
/home/rascal/work/skunkworks/scripts/python/scrapers/funny_quotes.sh
to
cd /home/rascal/work/skunkworks/scripts/python/scrapers/ && bash funny_quotes.sh
3) Check your cron logs
grep CRON /var/log/syslog
I am sure you are getting error something like scrapy - command not found or something similar.
To fix it, do this
Enter and copy the output of echo $PATH from shell.
And then open crontab -e
At the very top of file, write PATH=YOUR_COPIED_CONTENTS
And that should work.

You can create a custom log for your cron.
0,30 8-18 * * 1-5 python /home/.../make_whoopy.py /home/make_whoopy.py.log 2>&1
This will send stderr and stdout to make_whoopy.py.log
If you want to see the output in screen (dirty solution)
0,30 8-18 * * 1-5 python /home/.../make_whoopy.py > /dev/pts/1
Of course this will work when you are logged and have to change /dev/pts/1 accordingly to your tty
Another alternative is send to mail
0,30 8-18 * * 1-5 python /home/.../make_whoopy.py > /dev/null 2>&1 | mail -s "cron output" you#example.org
And the cherry of the cake is send the output to syslog
0,30 8-18 * * 1-5 python /home/.../make_whoopy.py | /usr/bin/logger -t make_whoopy
The -t make_whoopy will create a tag named make_whoopy on syslog.

Variables in Crontab Amazon Linux

Can't get crontabs with variables working on Amazon Linux. It's weird cause those variables do work in the command line of the system, but won't work inside the cronjob. I know the variables are the problem, cause when I remove the part from --argstill >, the cronjob will be executed.
Worked fine on Ubuntu:
0 7 * * * /usr/bin/python /path/to/cronjob.py --method all --args $(date -d "1 day ago" +%Y%m%d),$(date +%Y%m%d) > /dev/null 2>&1
Also tried:
47 16 * * * python /path/to/cronjob.py --method all --args `date -d "1 day ago" +%Y%m%d`,`date +%Y%m%d` > /dev/null 2>&1

#tedder42 solution worked for me, I only needed to add the backslashes before the %:
47 16 * * * python /path/to/cronjob.py --method all --args `date -d "1 day ago" +\%Y\%m\%d`,`date +\%Y\%m\%d` > /dev/null 2>&1

Run Python Script Using Cygwin Crontab on Windows

I succeed in setting up Cygwin Crontab on Windows. I need to run Python script every 5 minutes.
*/5 * * * * run-one C:\Python27\Scripts\myScript.py > C:\Python27\Scripts\myScript.log 2>&1
Above didn't work in crotab. Also the log file cygstart cron.log is empty. How I'm suppose to fix and run the script?
P.S. Script is running fine using idle editor.

Run your script directly in the Cygwin command line:
C:\Python27\Scripts\myScript.py > C:\Python27\Scripts\myScript.log 2>&1
If it does not work, you may need to specify python.exe directly and use forward slash ("/") rather than backslash ("\"):
C:/Python27/python.exe C:/Python27/Scripts/myScript.py > C:/Python27/Scripts/myScript.log 2>&1
If it works, then please make sure cron is running. If it is running, it should give similar output as below:
$ cygcheck.exe -c | grep cron
cron 4.1-61 OK
cron-debuginfo 4.1-61 OK
$ cygrunsrv -Q cron
Service : cron
Display name : Cron daemon
Current State : Running
Controls Accepted : Stop
Command : /usr/sbin/cron -n
$ ps -lef | grep cron
SYSTEM 4852 4680 ? 15:16:50 /usr/sbin/cron
If it is not running, run
$ cygrunsrv --start cron
If there is an error, you may need to reinstall cron.
$ cyglsa-config
Then
$ cron-config
And follow the instructions.