I've been trying to figure out how to best call a script with cronjobs and am unable to figure it out. Either I go with a custom command where I use the following in .ebextension/"some config file":
container_commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/mycron && chmod 644 /etc/cron.d/mycron"
leader_only: true
some_cron_job.txt:
* * * * * root source /opt/python/run/venv/bin/activate && source /opt/python/current/env && /usr/bin/python /opt/python/current/app/manage.py cron_command >> /var/log/myjob.log 2>&1
This works when i run the command locally but after having uploaded it to eb I get the following error:
File "/opt/python/current/app/manage.py", line 18
) from exc
^ SyntaxError: invalid syntax
Or I could call the script directly:
* * * * * root source /opt/python/run/venv/bin/activate && source /opt/python/current/env && /usr/bin/python /opt/python/current/app/api/cron.py >> /var/log/myjob.log 2>&1
But am then getting import errors when trying to import a function from a another file in the same directory:
ImportError: attempted relative import with no known parent package
I'm quite lost and would appreciate any help.
I managed to find a working solution where I instead used:
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
0/10 * * * * root source /opt/python/current/env && /opt/python/run/venv/bin/python3 /opt/python/current/app/manage.py cron_command >> /var/log/newjob.log 2>&1
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/mycron.bak"
I think the problem came about because of some python version problems in the virtual environment.
Related
Greetings fellow programmers,
I am currently using django-background-tasks(https://django-background-tasks.readthedocs.io/en/latest/)
to run some background tasks on AWS elasticbeanstalk. I initially use this command in the main .config container commands but I get timeout error in deployment because this management command will never finish(it continues to run on).
Now, I am trying using the approach suggested for running cron job on elasticbeanstalk(https://aws.amazon.com/premiumsupport/knowledge-center/cron-job-elastic-beanstalk/). Please take a look at my code, it is not running the command. what is wrong pls? I just need the command python manage.py process_tasks to keep running on. This is working properly on my local machine since i can easily open another terminal to fire up the python manage.py process_tasks command
files:
"/etc/cron.d/process_tasks_cron":
mode: "000644"
owner: root
group: root
content: |
* * * * * root /usr/local/bin/99_process_tasks.sh
"/usr/local/bin/99_process_tasks.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
date > /tmp/date
# Your actual script content
source /var/app/venv/*/bin/activate
python manage.py process_tasks
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
this a working version of the code. thanks and hope it helps someone out there.
files:
"/etc/cron.d/cron_process":
mode: "000644"
owner: root
group: root
content: |
* * * * * root /usr/local/bin/task_process.sh
"/usr/local/bin/task_process.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
exec &>> /tmp/cron_capture_log.txt
date > /tmp/date
source /var/app/venv/staging-LQM1lest/bin/activate
cd /var/app/current
python manage.py process_tasks
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/cron_process.bak"
I'm working on implementing some cronjobs with my application running on elastic beanstalk but am unsure of how to proceed. My current cron-linux.config file in the .ebextension folder looks like:
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
* * * * * root /usr/bin/python opt/python/current/app/api/cron.py > /dev/null
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
I've used eb ssh to make sure that the paths point to the correct location. The problem is that I'm not getting any error messages so it's quite hard to know where the problem is. Any help would be much appreciated!
You are supressing your outputs.
Try replacing the schedule line from:
* * * * * root /usr/bin/python opt/python/current/app/api/cron.py > /dev/null
To:
* * * * * root /usr/bin/python opt/python/current/app/api/cron.py > /home/<USER>/logs/backup.log 2>&1
You should be able to see the logs in /home/<USER>/logs/backup.log. Make sure that your script is outputing messages, success or error.
I followed the suggested solution for this question.
And came up with the following docker file
FROM ubuntu:16.04
ADD write_time.py /
USER root
RUN apt-get update && \
apt-get install -y python cron && \
chmod +x /write_time.py && \
(crontab -l 2>/dev/null; echo "* * * * * cd / && /usr/bin/python /write_time.py >> test.out") | crontab -
the write_time.py is
#!/usr/bin/env python
import datetime
time = datetime.datetime.now()
time = time.strftime("%Y-%m-%dT%H:%M:%S.%f")
print(time)
with open("time.txt", "a") as f:
f.write(time+"\n")
After i build with the below command and run it -
docker build . -t se
docker run -it se
I exec into the contaier, to check if either test.out or test.txt is created at / but i do not see either.(have waited for more than 2 mins)
Anything i am doing wrong here?
Thanks!
Solved it.
Docker CMD needs to be set to cron deamon.
I'm trying to run a python script through crontab, but it can't import any of the libraries that it needs when it is run this way. When I run the scripts outside of crontab, there is no issue, and I know that I have these libraries installed.
Do I need to specify a path to them or something?
Many thanks
crontab file:
SHELL=/bin/bash
MAILTO=jess.chambers#gmail.com
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local
*/1 * * * * cd ~/Downloads/guichets && python newRdvChecker.py -G1 -S0 >>/tmp/stdout.log 2>&1
*/1 * * * * cd ~/Downloads/guichets && python newRdvChecker.py -G2 -S20 >>/tmp/stdout.log 2>&1
*/1 * * * * cd ~/Downloads/guichets && python newRdvChecker.py -G3 -S40 >>/tmp/stdout.log 2>&1
Error log:
Traceback (most recent call last):
File "newRdvChecker.py", line 2, in <module>
import requests
I'm running this setup on my Linux Mint computer, if that makes a difference
Determine what exact python executable you use when running the script from CLI (with which python) and specify the full path to python in your crontab.
The code in crontab 0 * * * * cd /home/scrapy/foo/ && scrapy crawl foo >> /var/log/foo.log
It failed to run the crawl, as there was no log in my log file.
I tested using 0 * * * * cd /home/scrapy/foo/ && pwd >> /var/log/foo.log, it echoed '/home/scrapy/foo' in log.
I also tried PATH=/usr/local/bin and PATH=/usr/bin, but no success.
I'm able to run it manually by typing cd /home/scrapy/foo/ && scrapy crawl foo in command line.
Any thoughts? Thanks.
Problem solved. Rather than running the crawl as root, use crontab -u user -e to create a crontab for user, and run as user.