Develop Reference

Python crontab adding newline character automatically and disabling other cron job

I've written a python script to run certain cron jobs and using crontab for the same.
Following is the code snippet:
import os
import inspect
from crontab import CronTab
def add_cron_job(scripts_list,frequency):
my_cron = CronTab(user='simrat')
for script in scripts_list:
if not cron_exists(my_cron,script):
command = 'python {}'.format(script)
job = my_cron.new(command=command, comment=script)
job.minute.every(frequency)
my_cron.write()
def cron_exists(my_cron, script):
for job in my_cron:
if job.comment == script:
return True
return False
if __name__ == "__main__":
#Frequency of every 1 minute
test_script = ['test1.py', 'test2.py']
add_cron_job(test_script,1)
#Frequency of every 1 day
test_script2 = ['test3.py']
add_cron_job(test_script2,1440)
Following is the output of 'crontab -e' (notice additional spaces added)
* * * * * python test1.py # test1.py
* * * * * python test2.py # test2.py
*/1440 * * * * python test3.py # test3.py
When I re-run the python cron_job script, it somehow disables the last cron_job(test3.py) and following is the output of crontab file:
* * * * * python test1.py # test1.py
* * * * * python test2.py # test2.py
# DISABLED LINE
# */1440 * * * * python test3.py # test3.py
*/1440 * * * * python test3.py # test3.py
along with an error it throws on console:
No handlers could be found for logger "crontab"
So my question is 3 fold:
Why is additional space at the top of the command when running my_cron.write()?
Why does it disable the last cron job instead of ignoring it as it already exists (def cron_exits should have taken care of that)
What's the signifance of the error thrown?

I ran you code and got following error in second run:
'1440', not in 0-59 for Minutes
I changed 1440 to 14 and ran code multiple times. And found same code every time (without deleting)
* * * * * python test1.py # test1.py
* * * * * python test2.py # test2.py
*/14 * * * * python test3.py # test3.py
I have not read complete CronTab code but it is clear that they have put some validator during reading existing cron commands but they are not validating it during writing.
Also i called add_cron_job with different arguments and found each time my_cron.write() is called it is adding a new line. This is not a bug but a feature.
Finally
No handlers could be found for logger "crontab" is logging issue. Try this Crontab Logger issue

Crontab python django issue

The following works as expected:
python /usr/share/str8RED/manage.py getLiveResults
However, nothing happens when I use the following cronjob:
*/1 * * * * python /usr/share/str8RED/manage.py getLiveResults
Using the link below I have managed to create a error log:
http://matthewwittering.com/blog/django-tips/running-a-django-management-commands-with-crontab.html
This informs me that:
Traceback (most recent call last):
File "/usr/share/str8RED/manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named django.core.management
I can get cronjab working and running every minute with echo "Hello World". Any help would be appreciated, many thanks, Alan.
Contents of getLiveResults.py:
from django.core.management import BaseCommand
from straightred.models import StraightredTeam
from straightred.xmlsoccer import XmlSoccer
#The class must be named Command, and subclass BaseCommand
class Command(BaseCommand):
# Show this when the user types help
help = "My test command"
# A command must define handle()
def handle(self, *args, **options):
xmlsoccer = XmlSoccer(api_key='XYZ', use_demo=False)
teams = xmlsoccer.call_api(method='GetAllTeamsByLeagueAndSeason',
seasonDateString='1617',
league='English League Championship')
numberOfTeamsUpdated = 0
for team in teams:
if '{http://xmlsoccer.com/Team}Team_Id' in team.keys():
teamUpdate = StraightredTeam(teamid=team['{http://xmlsoccer.com/Team}Team_Id'],
teamname=team['{http://xmlsoccer.com/Team}Name'],
country=team['{http://xmlsoccer.com/Team}Country'],
stadium=team['{http://xmlsoccer.com/Team}Stadium'],
homepageurl=team['{http://xmlsoccer.com/Team}HomePageURL'],
wikilink=team['{http://xmlsoccer.com/Team}WIKILink'],
currentteam=1)
teamUpdate.save()
numberOfTeamsUpdated = numberOfTeamsUpdated + 1
self.stdout.write("Hello world!")

If you are using virtual env, then you need to activate the environment,
maybe something like:
*/1 * * * * /usr/share/str8RED/.env/bin/python /usr/share/str8RED/manage.py getLiveResults

Using environment dependent django settings inside fabric task

I'm trying to use the database configuration set on settings files to make a database dump using fabric.
There's more than one settings file, so I'd like to be able to do so based on the environment I choose.
by now, my task is like this
def dump_database():
with cd('~/project_folder'), prefix(WORKON_VIRTUALENV):
django.settings_module(env.settings)
from django.conf import settings
dbname = settings.DATABASES['default']['NAME']
dbuser = settings.DATABASES['default']['USER']
dbpassword = settings.DATABASES['default']['PASSWORD']
fname = '/tmp/{0}-backup-{1}.sql.gz'.format(
dbname,
time.strftime('%Y%m%d%H%M%S')
)
run('mysqldump -u %s -p=%s %s | gzip -9 /tmp/backup-%s.sql.gz' % (
dbuser,
dbpassword,
dbname,
fname))
But I'm getting an ImportError:
ImportError: Could not import settings 'project.settings.production'
I've tried to use shell_env() to set the DJANGO_SETTINGS_MODULE instead of django.settings_module(env.settings), with the same result.
I use a task to change the environment based on a environment dict:
def environment(name):
env.update(environments[name])
env.environment = name
This way, I want to be able to create a dump from multiple hosts like:
fab environment:live dump_database
fab environment:otherhost dump_database
Without having to reproduce database settings from all hosts on fabfile.

Importing your Django settings file in fabric is explained here.
http://fabric.readthedocs.org/en/1.3.3/api/contrib/django.html
Quoting from the above link:
from fabric.api import run
from fabric.contrib import django
django.settings_module('myproject.settings')
from django.conf import settings
def dump_production_database():
run('mysqldump -u %s -p=%s %s > /tmp/prod-db.sql' % (
settings.DATABASE_USER,
settings.DATABASE_PASSWORD,
settings.DATABASE_NAME
))

NOTE: I don't answer the question but offer a different solution
I had the same problem.. so I did custom .py script like that:
I created a file named dump_db.py (placed next to fabfile.py for example, that is on the remote machine)
import os
import sys
from datetime import datetime
from django.conf import settings
def dump_mysql():
os.environ.setdefault("DJANGO_SETTINGS_MODULE", SETTINGS_MODULE)
DB_NAME = settings.DATABASES['default']['NAME']
DB_USER = settings.DATABASES['default']['USER']
DB_PASSWORD = settings.DATABASES['default']['PASSWORD']
dump_file_name = '{time}_{db_name}.sql'.format(
time=datetime.now().strftime('%Y_%m_%d'),
db_name=DB_NAME,
)
os.system('mysqldump -u {db_user} -p{db_password} {db_name} > {to_file}'.format(
db_user=DB_USER,
db_password=DB_PASSWORD,
db_name=DB_NAME,
to_file=dump_file_name,
))
return dump_file_name
if __name__ == '__main__':
try:
SETTINGS_MODULE = sys.argv[1:].pop()
except IndexError:
SETTINGS_MODULE = 'project_name.settings'
print dump_mysql()
As you see sys.argv[1:].pop() tries to take optional argument (the setting module in this case).
So in my fabfile:
import os
from fabric.api import env, local, run, prefix, cd
.....
def dump():
current_dir = os.getcwd()
with prefix('source {}bin/activate'.format(env.venv)), cd('{}'.format(env.home)):
dumped_file = run('python dump_db.py {}'.format(env.environment)) # the optional argument given
file_path = os.path.join(env.home, dumped_file)
copy_to = os.path.join(current_dir, dumped_file)
scp(file_path, copy_to)
def scp(file_path, copy_to):
local('scp {}:{} {}'.format(env.host, file_path, copy_to))
where env.environment = 'project_name.settings.env_module'
And this is how I dump my DB and copy it back to me.
Hope it comes handy to someone! :)

Launching a .exe Upon a Timer's Due_Date

How Would i Go About Adding a File To launch when it see that's it's the due_date? I've try'd quite i few different method's from Google but i'm still having a hard time figuring it out. currently it's set to wait 36 hours after launching the .py file.
any help would be great and it'd get this monkey off my back for good!
import datetime
import croniter
import crontab
import time
c = croniter.croniter("0 9,10,11 * * TUE")
next_due_date = c.get_next(datetime.datetime)
while True:
now = datetime.datetime.now()
if now > next_due_date:
do_something(line.py)
time.sleep(60 * 60 * 36)
else:
time.sleep(60 * 60 * 2)

If it's a .exe you can just use os.system("myexecutable.exe") after launching the python
import datetime
import croniter
import crontab
import time
c = croniter.croniter("0 9,10,11 * * TUE")
next_due_date = c.get_next(datetime.datetime)
while True:
now = datetime.datetime.now()
if now > next_due_date:
do_something(line.py) # Edit: fixed tabbing; just in case it wasn't tabbed in
# your script
# Use os.system to run the exe
os.system("myexecutable.exe")
time.sleep(60 * 60 * 36)
else:
time.sleep(60) # Edit: I always find that it's better to have a smaller
# sleep time
You can also use the subprocess module so you can halt the script or track if the exe is still running instead.

Django standalone run in cron

I want to run automatic newsletter function in my crontab, but no matter what I try - I cannot make it work. What is the proper method for doing this ?
This is my crontab entry :
0 */2 * * * PYTHONPATH=/home/muntu/rails python2.6 /home/muntu/rails/project/newsletter.py
And the newsletter.py file, which is located in the top folder of my django project :
#! /usr/bin/env python
import sys
import os
os.environ["DJANGO_SETTINGS_MODULE"] = "project.settings"
from django.core.management import setup_environ
from project import settings
setup_environ(settings)
from django.template.loader import get_template, render_to_string
from django.template import Context
from django.core.mail import EmailMultiAlternatives
from django.utils.translation import ugettext_lazy as _
from django.core.mail import send_mail
from django.conf import settings
from project.utilsFD.models import *
from django.http import HttpResponse, HttpResponseRedirect, Http404
def main(argv=None):
if argv is None:
argv = sys.argv
template_html = 'static/newsletter.html'
template_text = 'static/newsletter.txt'
newsletters = Newsletter.objects.filter(sent=False)
adr = NewsletterEmails.objects.all()
for a in adr:
for n in newsletters:
to = a.email
from_email = settings.DEFAULT_FROM_EMAIL
subject = _(u"Newsletter - Method #1")
text_content = render_to_string(template_text, {"title": n.title,"text": n.text, 'date': n.data, 'email': to})
html_content = render_to_string(template_html, {"title": n.title,"text": n.text, 'date': n.data, 'email': to})
msg = EmailMultiAlternatives(subject, text_content, from_email, [to])
msg.attach_alternative(html_content, "text/html")
msg.content_subtype = "html"
msg.send()
n.sent = True
n.save()
if __name__ == '__main__':
main()
What am I doing wrong ? The function itself was working without any problems when run as django app, but when I was trying to run it from console, it gave me :
Traceback (most recent call last):
File "newsletter.py", line 7, in <module>
from project import settings
ImportError: No module named project
And it does not work from cron at all.

Try changing your cron entry to:
0 */2 * * * cd /home/muntu/rails && python2.6 /home/muntu/rails/project/newsletter.py
This will ensure that the "rails" directory is in python's path. If you want to set the PYTHONPATH, then create a shell script:
#!/bin/sh
export PYTHONPATH=/home/muntu/rails
python2.6 /home/muntu/rails/project/newsletter.py
and put the shell script in the cron entry.