Develop Reference

How to run the Python program in the background in Ubuntu server

I have a python script. Script have selenium with Chrome and go to a website, take data and put in CSV file.
This is a very long work.
I put the script on the server. And run. All work.
But I need script work in the background.
chmod +x createdb.py
nohup python ./createdb.py &
And I see
(env)$ nohup ./createdb.py &
[1] 32257
(env)$ nohup: ignoring input and appending output to 'nohup.out'
Press Enter.
(env)$ nohup ./createdb.py &
[1] 32257
(env)$ nohup: ignoring input and appending output to 'nohup.out'
[1]+ Exit 1 nohup ./createdb.py
Then it runs and immediately writes errors to the file, that Chrome did not start or there was no click.
I want to remind you that if you start without nohup, then everything will work.
What am I doing wrong? How to run a script?
Thank you very much.

You could create a background daemon (service)
You taged Ubuntu 16.04 it means you got systemd, for more information on how to set it up, please visit this link
create a file called <my_service>.system
and put it there: /etc/systemd/system
you systemd unit could look like this:
[Unit]
Description=my service
After=graphical.target
[Service]
Type=simple
WorkingDirectory=/my_dir
ExecStart=python my_script.py
[Install]
WantedBy=multi-user.target
then all you have to do is, reload systemd manage and start your service:
sudo systemctl daemon-reload
sudo systemctl myservice start

You can use the screen command, it works perfectly.
Here is a very good link: https://www.rackaid.com/blog/linux-screen-tutorial-and-how-to/

You can use a simple command, from the env directory:
(env)$ python /path/to/createdb.py > logger.txt 2>&1 &
This will help for storing the program logs in a defined file called "logger.txt"

crontab won't run os.system python command

Using ubuntu's 16.04 crontab and #reboot to run python3 script. The script runs properly on reboot as I see the logged output. However, my script's os.system command is not running. It runs fine if ran outside of crontab. My scripts are all executable.
crontab -l output:
SHELL=/bin/bash
#reboot nohup /usr/bin/python3 -u /home/path/scheduler.py >> /path/log.out &
scheduler.py code:
#...(check if web server is running...if not restart)
os.system('nohup /usr/bin/python3 -u /path/webserver/main.py &')
print('this function ran')
When I logged the output of the os.system command , there was no output.
As a side note, I am running python schedule commands to check the general health of a webserver. crontab doesn't seem to be the right tool for this so I just use crontab to start my python scheduler on reboot.
I am using flask as the webserver, and would use gunicorn and systemctrl if I could get it to work... but it didn't so this is my workaround.

The point is that, the command called by os.system is not in default path.
For example, tcpdump is not in /usr/bin/.
So, you can solve the problem by adding the full path of the command.

I was facing the same issue when we try to run python script directly in crontab it just by passes the os.system() commands.
Make launcher.sh:
#!bin/bash
cd /home/pi/
sudo python example.py
Then, make your script executable:
chmod 755 launcher.sh
And at last, add your script to crontab:
crontab -e
and add this line at the end:
#reboot sh /home/pi/launcher.sh
(I set the program to run at each reboot)

ec2 run scripts every boot

I have followed a few posts on here trying to run either a python or shell script on my ec2 instance after every boot not just the first boot.
I have tried the:
[scripts-user, always] to /etc/cloud/cloud.cfg file
Added script to ./scripts/per-boot folder
and
adding script to /etc/rc.local
Yes the permissions were changed to 755 for /etc/rc.local
I am attempting to pipe the output of the file into a file located in the /home/ubuntu/ directory and the file does not contain anything after boot.
If I run the scripts (.sh or .py) manually they work.
Any suggestions or request for additional info to help?

So the current solution appears to be a method I wrote off in my initial question post as I may have not performed the setup exactly as outline in the link below...
This link -->
How do I make cloud-init startup scripts run every time my EC2 instance boots?
The link shows how to modify the /etc/cloud/cloud.cfg file to update scripts-user to [scripts-user, always]
Also that link says to add your *.sh file to /var/lib/cloud/scripts/per-boot directory.
Once you reboot your system your script should have executed and you can verify this in: sudo cat /var/log/cloud-init.log
if your script still fails to execute try to erase the instance state of your server with the following command: sudo rm -rf /var/lib/cloud/instance/*
--NOTE:--
It appears print commands from a python script do not pipe (>>) as expected but echo commands pipe easily
Fails to pipe
sudo python test.py >> log.txt
Pipes successfully
echo "HI" >> log.txt

Is this something along the lines that you want?
It copies the script to the instance, connects to the instance, and runs the script right away.
ec2 scp ~/path_to_script.py : instance_name -y && ec2 ssh instance_name -yc "python script_name.py" 1>/dev/null

I read that the use of rc.local is getting deprecated. One thing to try is a line in /etc/crontab like this:
#reboot full-path-of-script
If there's a specific user you want to run the script as, you can list it after #reboot.

Add CRON via shell to run Python

My cron doesn't seem to execute every 5 mins. Can anyone show me where I have gone wrong?
I made it executable using this command:
chmod +x /etc/utilities/poll.py
I can run it manually with this command:
cd /etc/utilities
python poll.py
When I run it like this I get an error:
root#li453-78:~# /etc/utilities/poll.py
-bash: /etc/utilities/poll.py: Permission denied
This is the command I use to add it to shell (via my automatic deployment script):
crontab -l | { cat; echo "*/5 * * * * /etc/utilities/poll.py"; } | crontab -
The start of my python file is like this:
#!/usr/bin/env python
So, could someone please enlighten me about how I should be adding the cron to my debian server via shell so that it executes?

Using the help from here, for whatever reason, even though I had the code correct to make the script executable, this line didn't seem to fire in my deployment script, meaning that all I had to do was run it afterwards to make it executable and then everything worked.
Lesson learnt: If you need to do this, the code above works

python - running ssh/rsync commands in a script called via crontab

I have a very strange issue that I can't seem to figure out.
When I execute a python script containing the following lines while inside a SSH terminal (putty), it works fine. But the moment I run the script via crontab or even nohup python myscript >/dev/null 2>&1& it doesn't seem to execute these commands.
subprocess.call('rsync -avr /path/to/folder/. --include "delta.*" --exclude "*" -e "ssh -o StrictHostKeyChecking=no -i /path/to/key.pem" ec2-user#'+server+':/path/to/folder/', shell=True)
local('ssh -t -o StrictHostKeyChecking=no -i /path/to/key.pem ec2-user#'+server+' "sudo /usr/bin/indexer -c /path/to/sphinx.conf --merge main delta --rotate"')
Basically all the above is doing is syncing a folder with new sphinx search engine updates to a remote server, then the second line runs a remote ssh command to force the search engine to rotate updates into production.
I do have fabric installed (hence the local command) but to avoid having to fab a second file I was hoping a single line of code could allow me to execute sudo commands on a remote server.
Can someone help me out?

I found the answer, for ssh commands in a script run in the background, you need to to have -t -t to force a pseudo terminal.
Reference:
Pseudo-terminal will not be allocated because stdin is not a terminal