I am trying to schedule running a Python module in cron. This question is somewhat similar to this but I think asks for a different use case.
Task
I have a Python script I run in the shell as a module like this:
python -m myscript
It prints a bunch of numbers (via print) and works when I run it from the shell.
Question
I am now trying to run this every minute using a cron job, like so:
*/1 * * * * python -m myscript
Question1: This does not print to the terminal as expected / I don't see any output. Why? (To test if it is running at all, I redirected output to a file, which creates an empty file every minute).
Question2: My thinking was that any command that works when I run it manually in the shell will also work the same way when started via cron. Is that mistaken? E.g do I still have to make the script executable and such?
Question3: Thinking about it, I was not quite sure where cron will direct e.g. a print / stdout command and could not find in the docs. Do I have to / Should I manually specify the output target if I want it to print to a new shell window?
Currently running this on elementary OS but I want to eventually migrate to a Raspberry Pi. Any help is much appreciated!
Related
I have an embedded system on which I run code live. Every time I want to run code, I start two scripts in two different terminals: "run1.sh" and "run2.sh". I can see the output of those scripts in my terminals (I wish to too).
Now I want to make a python script that starts those two scripts in two different terminals. I want to still see their output. Also I want to insert a password from the python script to the terminals, since the scripts run in sudo mode. I've played a lot with supbrocess and the PIPES but I've never achieved all of the above requirements simultaneously. How can these requirements be met?
I'm using Ubuntu btw (so I have gnome terminal)
Update : I was probably not clear in my question, but this has to be inside a python script. It is not for my convenience, it's part of an integration process. The code of the script will be part of a larger python program, so the whole point of the question is how do I do it in python.
Based on your new information added I've created an small python script which will launch two terminals and their output separately:
Main script:
mortiz#florida:~/Documents/projects/python/split_python_execution$ cat split_pythonstuff.py
#!/usr/bin/python3
import subprocess
subprocess.call(['gnome-terminal', '-x', 'python', '/home/mortiz/Documents/projects/python/split_python_execution/script1.py'])
subprocess.call(['gnome-terminal', '-x', 'python', '/home/mortiz/Documents/projects/python/split_python_execution/script2.py'])
Script 1:
mortiz#florida:~/Documents/projects/python/split_python_execution$ cat script1.py
#!/usr/bin/python3
while True :
print ('script 1')
Script 2:
mortiz#florida:~/Documents/projects/python/split_python_execution$ cat script2.py
#!/usr/bin/python3
while True:
print ('script 2')
From here I guess you can develop anything you want.
UPDATE: About sudo
Sudoers is a great way of controlling which things can be executed by specific users providing passwords or not.
If you add this line in /etc/sudoers there's not need for a password when you pass sudo to your command:
<YOUR_USER> ALL = NOPASSWD : /usr/bin/python <SCRIPT.py>
In your question as far as I understand you have the password stored inside the script. There's no need to do that and it's a bad practice. Sudoers would be a better way.
Anyway, if you want to do it in an insecure way then refer to this question and place it before the commands in the scripts provided in this answer.
The linked provided works:
echo -e "mypassword\n" | sudo -S python test.py
15
You only need to implement that on the previous code.
You could install Terminator and configure one profile per terminal to run any script you want.
I have a default template which will load 3 terminals and run 3 different commands / or scripts if you wanted to:
When I load that profile the first one will move me to my projects dir and list them. The next one will run df -h to see the space available and the lower my ip configuration.
This way would save you lots of programming and it's quite easy.
UPDATE: It will run any command, bash, zsh, python, etc.. available for your terminal. If the script is locally in your machine:
python <your_script_1> # first terminal profile
python <your_script_2> # second terminal profile
both would be executed "at the same time".
If your scripts are remote in the target machine, simply create a bash script using ssh to connect to the remote machine with a private key and then running the script, the result is the same in both scenarios.
EDIT: The best thing is setting colors and transparency for each terminal, so you can enjoy the penguin's selfie while you work.
I am attempting to run a python 3 script every 1 minute using cron on a raspberrypi 3, for testing, where eventually it will just be run once a day.
To start, I made a new cron job using: sudo crontab -e, and typed in the following code for a once a minute job:
*/1 * * * * /home/pi/folder/file.py
Then I saved and closed and waited. My python script emails me text when executed, so I should have seen an email come in. It runs fine (and emails me) when I execute it manually outside of cron.
So, what am I doing wrong with cron for it not to run? And do I need to make the python file executable or something with chmod?
Possible duplicate of Execute python Script on Crontab
EDIT:
Adding comment here since the comment box mangled my formatting.
In your example above it looks like you are just trying to "run" the file. You need to call the python executable, and pass it an argument that points to your file.
From the StackOverflow comment mentioned above look at this crontab entry:
*/2 * * * * /usr/bin/python /home/souza/Documets/Listener/listener.py
Take a look at the first part of the command /usr/bin/python this is pointing to the python executable not just to the .py file you want to run.
I'm rather green in python development but learning nevertheless.
I've written some python codes but they are mostly one-off use and to be run at command line. I still have no idea how to automate and schedule my codes. Lets say I have:
written a python script that pulls some CSV data from API and saves it in /tmp.
written another script to ingest the csv data and transform it into XML per line.
Each time I want to do this, I find myself doing:
$ python getdata.py
$ python converttoxml.py
In shell, I think one can write a wrapper script and cron it. Right? If so, how is this done in the world of python? Bear in mind we have to include all python libraries/ modules used, too.
P.S. developing in Linux environment using PyCharm.
Login to cron using crontab -e
Scroll to bottom at add a line following the following format:
m h dom mon dow command which is minutes, hours, day of month, day of week, command
So if you want to run your command on the hour every hour you would have
* /1 * * * * python path/to/file/getdata.py
whereas if you want it to run at 12 only then you would have
* 12 * * * * python path/to/file/getdata.py
Python files could be treated as executables. You just need to give them executing permissions with chmod +x my_file.py and tell the bash which interpreter should be used to parse the file by adding #!/usr/bin/python as the 1st line of the code file.
Once you've done the two things, you could just run your file using ./my_file.py and the python script should be executed.
From this point on, it's just like any ordinary program and could be used in cron/systemd/whatever other use you need. Once you have a working Python script, it should be treated no differently than any other executable on your system - in this regard, there's no "Python World".
As far as extra modules are concerned - this shouldn't be an issue as Python will still use the same way to resolve library location. It might be necessary to update $PYTHON_PATH to add a library path - but if the library is installed correctly it shouldn't be an issue while for libraries contained within your local directory (only in the case Python fails to find them) you could add:
sys.path.apped(os.path.dirname(__file__)) to the beginning of your script.
Recently I set up a twitter retweet bot, but after few minutes it's going down.
I used this python bot and I'm using "nohup python mrbot.py &" to keep it running in the background after closing the terminal. Even if I use it as "python mrbot.py", it stops working after around 30 minutes. So could someone please tell me how to automatically restart it every 30 minutes?
I'm a very beginner at this. *Remember I'm running it in background using "nohup". I don't know if this helps.
Crontab would be the ideal way to do it. Please search for crontab and how to enable and schedule a crontab job.
For your example a crontab entry would look like
30 * * * * /path/to/your/python/script.py
Since you are running ubuntu, you should have python in your path and have a shebang in the python script. A shebang is the first line in your python script with path to the python executable, it is in following format
#!/usr/bin/python
You are on Ubuntu, so you can use crontab command. Open terminal and type:
crontab -e
This will prompt you with (probably) an empty document. Then write the following line:
*/30 * * * * /usr/bin/python script.py
This will make sure your script runs every 30 minutes.
You can read more about crontab and what it does and how it works here
I would like to create a simple Python program that will concurrently execute 2 independent scripts. For now, the two scripts just print a sequence of numbers but my intention is to use this program to concurrently run a few Twitter streaming programs in the future.
I suspect I need to use subprocess.Popen but I cannot quite get my head around what arguments I should put in there. There was a similar question on StackOverflow but the code provided there (pasted below) doesn't print anything. I will appreciate your help.
My files are:
thread1.py
thread2.py
import subprocess
subprocess.Popen(['screen', './thread1.py']))
subprocess.Popen(['screen', './thread2.py'])
Use supervisord
supervisord is process control system just for the purpose of running multiple command line scripts.
It features:
multiple controlled processes
autorestarting failed runs
log stdout and stderr output
starting scripts in order (using priority)
command line utility to view latest log output, stop, start, restart the processes
This solution works only on *nix based systems, it is not available on Windows.
As wanderlust mentioned, why do you want to do it this way and not via linux command line?
Otherwise, the solution you post is doing what it is meant to, i.e, you are doing this at the command line:
screen ./thread1.py
screen ./thread2.py
This will open a screen session and run the program and output within this screen session, such that you will not see the output on your terminal directly. To trouble shoot your output, just execute the scripts without the screen call:
import subprocess
subprocess.Popen(['./thread1.py'])
subprocess.Popen(['./thread2.py'])
Content of thread1.py:
#!/usr/bin/env python
def countToTen():
for i in range(10):
print i
countToTen()
Content of thread2.py:
#!/usr/bin/env python
def countToHundreds():
for i in range(10):
print i*100
countToHundreds()
Then don't forget to do this on the command line:
chmod u+x thread*.py
You can also just open several Command Prompt windows to run several Python programs at once - just run one in each of them:
In each Command Prompt window, go to the correct directory (such as C:/Python27) and then type 'python YourCodeNo1.py' in one Command Prompt window, 'python YourCodeNo2.py' in the next one ect. .
I'm currently running 3 codes at one time in this way, without slowing any of them down.