I have followed a few posts on here trying to run either a python or shell script on my ec2 instance after every boot not just the first boot.
I have tried the:
[scripts-user, always] to /etc/cloud/cloud.cfg file
Added script to ./scripts/per-boot folder
and
adding script to /etc/rc.local
Yes the permissions were changed to 755 for /etc/rc.local
I am attempting to pipe the output of the file into a file located in the /home/ubuntu/ directory and the file does not contain anything after boot.
If I run the scripts (.sh or .py) manually they work.
Any suggestions or request for additional info to help?
So the current solution appears to be a method I wrote off in my initial question post as I may have not performed the setup exactly as outline in the link below...
This link -->
How do I make cloud-init startup scripts run every time my EC2 instance boots?
The link shows how to modify the /etc/cloud/cloud.cfg file to update scripts-user to [scripts-user, always]
Also that link says to add your *.sh file to /var/lib/cloud/scripts/per-boot directory.
Once you reboot your system your script should have executed and you can verify this in: sudo cat /var/log/cloud-init.log
if your script still fails to execute try to erase the instance state of your server with the following command: sudo rm -rf /var/lib/cloud/instance/*
--NOTE:--
It appears print commands from a python script do not pipe (>>) as expected but echo commands pipe easily
Fails to pipe
sudo python test.py >> log.txt
Pipes successfully
echo "HI" >> log.txt
Is this something along the lines that you want?
It copies the script to the instance, connects to the instance, and runs the script right away.
ec2 scp ~/path_to_script.py : instance_name -y && ec2 ssh instance_name -yc "python script_name.py" 1>/dev/null
I read that the use of rc.local is getting deprecated. One thing to try is a line in /etc/crontab like this:
#reboot full-path-of-script
If there's a specific user you want to run the script as, you can list it after #reboot.
Related
I have a python script. Script have selenium with Chrome and go to a website, take data and put in CSV file.
This is a very long work.
I put the script on the server. And run. All work.
But I need script work in the background.
chmod +x createdb.py
nohup python ./createdb.py &
And I see
(env)$ nohup ./createdb.py &
[1] 32257
(env)$ nohup: ignoring input and appending output to 'nohup.out'
Press Enter.
(env)$ nohup ./createdb.py &
[1] 32257
(env)$ nohup: ignoring input and appending output to 'nohup.out'
[1]+ Exit 1 nohup ./createdb.py
Then it runs and immediately writes errors to the file, that Chrome did not start or there was no click.
I want to remind you that if you start without nohup, then everything will work.
What am I doing wrong? How to run a script?
Thank you very much.
You could create a background daemon (service)
You taged Ubuntu 16.04 it means you got systemd, for more information on how to set it up, please visit this link
create a file called <my_service>.system
and put it there: /etc/systemd/system
you systemd unit could look like this:
[Unit]
Description=my service
After=graphical.target
[Service]
Type=simple
WorkingDirectory=/my_dir
ExecStart=python my_script.py
[Install]
WantedBy=multi-user.target
then all you have to do is, reload systemd manage and start your service:
sudo systemctl daemon-reload
sudo systemctl myservice start
You can use the screen command, it works perfectly.
Here is a very good link: https://www.rackaid.com/blog/linux-screen-tutorial-and-how-to/
You can use a simple command, from the env directory:
(env)$ python /path/to/createdb.py > logger.txt 2>&1 &
This will help for storing the program logs in a defined file called "logger.txt"
Using ubuntu's 16.04 crontab and #reboot to run python3 script. The script runs properly on reboot as I see the logged output. However, my script's os.system command is not running. It runs fine if ran outside of crontab. My scripts are all executable.
crontab -l output:
SHELL=/bin/bash
#reboot nohup /usr/bin/python3 -u /home/path/scheduler.py >> /path/log.out &
scheduler.py code:
#...(check if web server is running...if not restart)
os.system('nohup /usr/bin/python3 -u /path/webserver/main.py &')
print('this function ran')
When I logged the output of the os.system command , there was no output.
As a side note, I am running python schedule commands to check the general health of a webserver. crontab doesn't seem to be the right tool for this so I just use crontab to start my python scheduler on reboot.
I am using flask as the webserver, and would use gunicorn and systemctrl if I could get it to work... but it didn't so this is my workaround.
The point is that, the command called by os.system is not in default path.
For example, tcpdump is not in /usr/bin/.
So, you can solve the problem by adding the full path of the command.
I was facing the same issue when we try to run python script directly in crontab it just by passes the os.system() commands.
Make launcher.sh:
#!bin/bash
cd /home/pi/
sudo python example.py
Then, make your script executable:
chmod 755 launcher.sh
And at last, add your script to crontab:
crontab -e
and add this line at the end:
#reboot sh /home/pi/launcher.sh
(I set the program to run at each reboot)
I have a simple python script which I want to start a daemon-service in background in docker container
/sbin/start-stop-daemon --start --user root --make-pidfile --pidfile /var/lock/subsys/my-application.pid --exec 'python /opt/app/uc/monitor/bin/my-application.py'
when I execute this command in a shell I get
/sbin/start-stop-daemon: unable to stat //python /opt/app/uc/monitor/bin/my-application.py (No such file or directory)
However when execute just the below command in shell it works
python /opt/app/uc/monitor/bin/my-application.py
I'm sure the python is installed and all the links have been setup.
Thanks for the help
That error message implies that start-stop-daemon is looking for a file to open (the stat operation is a check before it opens the file) and treating your 'python ... ' argument as if it was a file.
See this example which confirms this. You may need to read the man page for start-stop-daemon, for your Ubuntu version, to check what a valid command would be for your setup.
Simplest solution is probably to create a shell script (say /opt/app/uc/monitor/bin/run-my-application.sh), and put this into it:
#!/bin/bash
python /opt/app/uc/monitor/bin/my-application.py
Be sure to do chmod +x on this file. If python is not found, use which python to find the path to python and use that in the script.
Now try:
/sbin/start-stop-daemon --start --user root --make-pidfile --pidfile /var/lock/subsys/my-application.pid --exec '/opt/app/uc/monitor/bin/run-my-application.sh'
I have a python script that creates a csv and saves it to:
file = example.csv
fileDone = os.path.abspath('/home/bw/temp/%s'%file1)
with open(fileDone, 'w+') as myFile:
a = csv.writer(myFile, delimiter=',')
a.writerow(['login report for %s from %s to %s:\n\n'%(name[0],start_date,end_date)])
dato = ['Username','Logins','Platform']
a.writerow(dato)
for vizL in vizList:
data =[str(vizL[0]),str(vizL[1]),"Viz"]
a.writerow(data)
for appL in appList:
data =[str(appL[0]),str(appL[1]),"Analytics"]
a.writerow(data)
then a bash script calls the python script:
#!/bin/bash
python scriptname.py
REPORT_MONTH=`/bin/date "+%d %B %Y"`
echo -e "Attached is the login reports for $REPORT_MONTH\n\n\n\n\n\nGenerated by: $0" | mutt -a /home/bw/temp/example.csv -c my#email.com -s "Daily Login Reports For $REPORT_MONTH"
Then cron runs the bash script daily.
When I run the bash script manually, everything works fine and the dates are correct. But when its run by cron, it doesn't seem to overwrite the csv file and just sends one that was already in the /temp folder. So I assume it is not regenerating the csv hence not running the python script.
This is my first time setting up a cron job to run a bash script that runs a python script. Any potential insight as to why this is happening would be greatly appreciated.
There are two things that I would try:
First, try setting the full path to the python script in the bash script, something like this:
#!/bin/bash
python /home/user/scripts/scriptname.py
REPORT_MONTH=`/bin/date "+%d %B %Y"`
echo -e "Attached is the login reports for $REPORT_MONTH\n\n\n\n\n\nGenerated by: $0" | mutt -a /home/bw/temp/example.csv -c my#email.com -s "Daily Login Reports For $REPORT_MONTH"
Second, if the previous didn't work, it might be a problem with the cron job not having permissions to overwrite the file. Which user is the cron running under? (it's usually root). Can you post the output of crontab -l? If you remove the file and let the cron create it, does it gets created?
First I would make sure your cron is using bash; by default it uses sh. You can include SHELL=/bin/bash to change it.
I have a short Python script that needs to run at startup - Ubuntu 13.10. I have tried everything I can think of but can't get it to run. The script:
#!/usr/bin/python
import time
with open("/home/username/Desktop/startup.txt", 'a') as f:
f.write(str(time.time()) + " It worked!")
(The actual script is a bit different, as I'm just using this for testing purposes, but you get the idea.)
I've tried all of the following, with no luck:
Put the command python startuptest.py in crontab, as #reboot
python /home/username/Documents/startuptest.py, both as the regular user and as sudo
Put the command python /home/username/Documents/startuptest.py in /etc/rc.local
Opened Ubuntu's Startup Applications and put the command there
Done all of the preceding, putting the command into a shell script
and calling that shell script instead
Nothing works. I get the feeling I'm missing something simple. Any ideas? (The script runs fine if I just run the command from a terminal.)
Instructions
Copy the python file to /bin:
sudo cp -i /path/to/your_script.py /bin
Add A New Cron Job:
sudo crontab -e
Scroll to the bottom and add the following line (after all the #'s):
#reboot python /bin/your_script.py &
The “&” at the end of the line means the command is run in the background and it won’t stop the system booting up.
Test it:
sudo reboot
Practical example:
Add this file to your Desktop: test_code.py (run it to check that it works for you)
from os.path import expanduser
import datetime
file = open(expanduser("~") + '/Desktop/HERE.txt', 'w')
file.write("It worked!\n" + str(datetime.datetime.now()))
file.close()
Run the following commands:
sudo cp -i ~/Desktop/test_code.py /bin
sudo crontab -e
Add the following line and save it:
#reboot python /bin/test_code.py &
Now reboot your computer and you should find a new file on your Desktop: HERE.txt
Put this in /etc/init (Use /etc/systemd in Ubuntu 15.x)
mystartupscript.conf
start on runlevel [2345]
stop on runlevel [!2345]
exec /path/to/script.py
By placing this conf file there you hook into ubuntu's upstart service that runs services on startup.
manual starting/stopping is done with
sudo service mystartupscript start
and
sudo service mystartupscript stop
If you are on Ubuntu you don't need to write any other code except your Python file's code , Here are the Steps :-
Open Dash (The First Icon In Sidebar).
Then type Startup Applications and open that app.
Here Click the Add Button on the right.
There fill in the details and in the command area browse for your Python File and click Ok.
Test it by Restarting System . Done . Enjoy !!
Create file ~/.config/autostart/MyScript.desktop
with
[Desktop Entry]
Encoding=UTF-8
Name=MyScript
Comment=MyScript
Icon=gnome-info
Exec=python /home/your_path/script.py
Terminal=false
Type=Application
Categories=
X-GNOME-Autostart-enabled=true
X-GNOME-Autostart-Delay=0
It helps me!
In similar situations, I've done well by putting something like the following into /etc/rc.local:
cd /path/to/my/script
./my_script.py &
cd -
echo `date +%Y-%b-%d_%H:%M:%S` > /tmp/ran_rc_local # check that rc.local ran
This has worked on multiple versions of Fedora and on Ubuntu 14.04 LTS, for both python and perl scripts.
nano /etc/rc.local
and edit in
python ~/path-to-script.py
worked for me