I am trying to run a subprocess command to do a git pull.
The cwd of the Git repository is /home/ubuntu/Ingest.
The id_rsa that I'm using with Github is located at /home/ubuntu/.ssh/id_rsa.
How would I run a subprocess call to do the following?
import shlex, subprocess
subprocess.call(shlex.split('git pull origin master'), cwd='/home/ubuntu/Ingest')
The log looks like:
movies_ec2.py:43#__init__ [INFO] Version not up to date...Doing a git pull and exiting...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The script is running from cron and is not picking up on the id_rsa. (Note: I am not looking to use GitPython). What do I need to change in my cron job or script so that this will work? My cron job is currently:
# sudo crontab -e
*/1 * * * * STAGE=production /home/ubuntu/Ingest/ingest/movies_ec2.py > /home/ubuntu/test.log 2>&1
The following answer addresses this question quite well: How to specify in crontab by what user to run script?. However, in short, this can be accomplished by modifying the user in the crontab to be "ubuntu", which is the current user that is used to create all the files, etc.
$ sudo vim /etc/crontab
*/1 * * * * ubuntu /home/ubuntu/Ingest/ingest/movies_ec2.py > /home/ubuntu/test.log 2>&1
Related
I need to run a bash script periodically on a Jetson Nano (so, Ubuntu 18.04). The script should run system updates, pull some Python code from a repository, and run it as a specified user.
So, I created this script:
#! /bin/bash
## system updates
sudo apt update
sudo apt upgrade
## stop previous instances of the Python code
pkill python3
## move to python script folder
cd /home/user_name/projects/my_folder
## pull updates from repo
git stash
git pull
## create dummy folder to check bash script execution to this point
sudo -u user_name mkdir /home/user_name/projects/dummy_folder_00
## launch python script
sudo -u user_name /usr/bin/python3 python_script.py --arg01 --arg02
## create dummy folder to check bash script execution to this point
sudo -u user_name mkdir /home/user_name/projects/dummy_folder_01
I created a cron job running this script as root, by using
sudo crontab -e
and adding the entry
00 13 * * * /home/user_name/projects/my_folder/script.sh
Now, I can see that at the configured time, both the dummy folders are created, and they actually belong to user_name. However, the Python script isn't launched.
I tried creating the cron job as non root user (crontab -e), but at this point even if the Python script gets exectured, I guess I wouldn't be able to run apt update/upgrade.
How can I fix this?
Well, if the dummy folders did get created, that means the sudo statements work, so i'd say theres a 99%+ chance that python was infact started.
I'm guessing the problem is that you havent specified the path for the python file, and your working directory likely isn't what you're expecting it to be.
change:
sudo -u user_name /usr/bin/python3 python_script.py --arg01 --arg02
to something like
sudo -u user_name /usr/bin/python3 /path/to/your/python_script.py --arg01 --arg02
then test.
If that didn't solve the problem , then enable some logging, change the line to:
sudo -u user_name /usr/bin/python3 /path/to/your/python_script.py --arg01 --arg02 \
1> /home/user_name/projects/dummy_folder_00/log.txt 2>&1 ;
and test again, it should log STDOUT and STDERR to that file then.
I have a CRON job set like this :
0 0 * * * cd /home/path/to/script && sudo -u myuser ./thescript.sh
This script builds a docker image running a simple python app dumping on file a test validation report. When I run this script in a terminal everything works fine (the generated file goes in /home/myuser). Unfortunately, when I run the CRON job, the file is created, but empty. It must have something to do with the Root owning the CRON job, but I can't figure out how to get it done
Any clue ?
A sudo requires a TTY and cron doesn't run commands with a TTY. You need to run the cron for root user
This can be done using
sudo crontab -e
Then in cron don't use sudo
0 0 * * * cd /home/path/to/script && ./thescript.sh
I am using python django community ami from bitnami and i am not able to execute python script in cron via virtual env
Directly executing script in terminal works e.g.
/path/to/env/bin/python /path/to/script.py (works)
crontab (non sudo)
*/opt/bitnami/python/bin/python /path/to/script.py (works)
/path/to/env/bin/python /path/to/script.py >> /tmp/log.out 2>&1 (not working)*
"Cannot locate wrapped file"
One possible error is forgetting to enable the virtual environment.
I'd have expected to see a crontab line along the lines of the following (taken from django-cron) which includes a call to '.../bin/activate'.
> crontab -e */5 * * * * source /home/ubuntu/.bashrc && source /home/ubuntu/work/your-project/bin/activate && python /home/ubuntu/work/your-project/src/manage.py runcrons > /home/ubuntu/cronjob.log
I have a dockerfile which automates the building of an image.
I am using the docker cloud, connected to Digital Ocean as the server.
Within my dockerfile, I get the software I need, add the relevant GitHub repository containing the python scripts I wish to run. I then start the cron scheduler and add the script with appropriate times. For example:
The cron_files.txt file looks like this:
0 12 * * * /usr/bin/python /home/dir/run_check.py
0 15 * * * /usr/bin/python /home/dir/run_push.py
In my dockerfile, I do the following:
RUN service cron start
RUN service cron status
RUN crontab -u root cron_files.txt
In the log files, I can see that cron is succesfully started.
Edit, thanks to r0manarmy for this - How to run a cron job inside a docker container?
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/hello-cron
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/hello-cron
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD cron && tail -f /var/log/cron.log
How do I edit the above to create the crontab file from the cron_files.txt rather than the above example?
I've tried ADD crontab cron_files.txt /etc/cron.d/feeds
But this returns:
ADD crontab cron_files.txt /etc/cron.d/feeds
lstat crontab: no such file or directory
Ps. I am using FROM debian:jessie
You probably want to set cron as the CMD:
In order to do this, just use the crond command on, say, alpine linux.
Take a look at this example for ideas:
https://gist.github.com/mhubig/a01276e17496e9fd6648cf426d9ceeec
Normally I have a crontab like:
1 * * * * /home/praneeth/wru-pam/venv/bin/python3.4 /home/praneeth/wru-pam/pam_site/manage.py notify_about_changes
I have been raising serious issues with crontab execution at some times it raises an error that main.cf is missing and when I have have created a file main.cf
Next fatal error it raised what etc/mailname is missing.
To some extend I believe commands which are executing in terminal are not absolutely written in crontab. I have the commands below to execute my task
In the Linux Terminal I normally execute in this procedure:
$ source venv/bin/activate
$ cd pam_site
$ export DJANGO_SETTINGS_MODULE=project.settings.development
$ python manage.py notify_about_changes
Task: To represent the above commands in crontab which would enable me to avoid any postfix/sendmail error and execute notify_about_changes.py(which sends email notifications in a timely manner)
finally I got the problem resolved as below:-
1.I re-edited the crontab as
$sudo crontab -e
*/1 * * * * /home/praneeth/wru-pam/venv/bin/python3.4 /home/praneeth/wru-pam/pam_site/manage.py notify_about_changes --settings=project.settings.development
2.restarted the crontab as
$sudo service cron restart
For every one minute I have been getting email notifications.