I need to run multiple programs one after the other and they each run in a console window.
I want the console window to be visible, but a new window is created for each program. This is annoying because each window is opened in a new position from where the other is closed and steals focus when working in Eclipse.
This is the initial code I was using:
def runCommand( self, cmd, instream=None, outstream=None, errstream=None ):
proc = subprocess.Popen( cmd, stdin=instream, stdout=outstream, stderr=errstream )
while True:
retcode = proc.poll()
if retcode == None:
if mAbortBuild:
proc.terminate()
return False
else:
time.sleep(1)
else:
if retcode == 0:
return True
else:
return False
I switched to opening a command prompt using 'cmd' when calling subprocess.Popen and then calling proc.stdin.write( b'program.exe\r\n' ).
This seems to solve the one command window problem but now I can't tell when the first program is done and I can start the second. I want to stop and interrogate the log file from the first program before running the second program.
Any tips on how I can achieve this? Is there another option for running the programs in one window I haven't found yet?
Since you're using Windows, you could just create a batch file listing each program you want to run which will all execute in a single console window. Since it's a batch script you can do things like put conditional statements in it as shown in the example.
import os
import subprocess
import textwrap
# create a batch file with some commands in it
batch_filename = 'commands.bat'
with open(batch_filename, "wt") as batchfile:
batchfile.write(textwrap.dedent("""
python hello.py
if errorlevel 1 (
#echo non-zero exit code: %errorlevel% - terminating
exit
)
time /t
date /t
"""))
# execute the batch file as a separate process and echo its output
kwargs = dict(stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
universal_newlines=True)
with subprocess.Popen(batch_filename, **kwargs).stdout as output:
for line in output:
print line,
try: os.remove(batch_filename) # clean up
except os.error: pass
In section 17.5.3.1. Constants in the subprocess module documentation there's description of subprocess.CREATE_NEW_CONSOLE constant:
The new process has a new console, instead of inheriting its parent’s
console (the default).
As we see, by default, new process inherits its parent's console. The reason you observe multiple consoles being opened is the fact that you call your scripts from within Eclipse, which itself does not have console so each subprocess creates its own console as there's no console it could inherit. If someone would like to simulate this behavior it's enough to run Python script which creates subprocesses using pythonw.exe instead of python.exe. The difference between the two is that the former does not open a console whereas the latter does.
The solution is to have helper script — let's call it launcher — which, by default, creates console and runs your programs in subprocesses. This way each program inherits one and the same console from its parent — the launcher. To run programs sequentially we use Popen.wait() method.
--- script_run_from_eclipse.py ---
import subprocess
import sys
subprocess.Popen([sys.executable, 'helper.py'])
--- helper.py ---
import subprocess
programs = ['first_program.exe', 'second_program.exe']
for program in programs:
subprocess.Popen([program]).wait()
if input('Do you want to continue? (y/n): ').upper() == 'N':
break
Related
When running a secondary python script:
Is it possible to run a subprocess.Popen, or subprocess.call or even execfile in a new terminal? (as in simply a different terminal than the current terminal where the script is run).
Alternatively, if before running my program (main), I open two terminals first, can I then point the secondary script to the second terminal? (so somehow getting the ID of open terminals, and then using a specific one among them, to perform the subprocess).
An example, two subprocesses to be run, first.py should be called first, only then the second is called, second.py. Because the two scripts first.py and second.py are interdependent (as in first.py goes to wait mode, until second.py is run, then first.py resumes, and I don't know how to make this communication work between them in terms of subprocesses.)
import subprocess
command = ["python", "first.py"]
command2 = ["python", "second.py"]
n = 5
for i in range(n):
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
p2 = subprocess.Popen(command2, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
output = p.stdout.readline().strip()
print output
if output == 'stop':
print 'success'
p.terminate()
p2.terminate()
break
Framework (Ubuntu, python 2.7)
I guess you want something like
subprocess.call(['xterm','-e','python',script])
Good old xterm has almost no frills; on a Freedesktop system, maybe run xdg-terminal instead. On Debian, try x-terminal-emulator.
However, making your program require X11 is in most cases a mistake. A better solution is to run the subprocesses with output to a log file (or a socket, or whatever) and then separately run tail -f on those files (in a different terminal, or from a different server over ssh, or with output to a logger which supports rsyslog, or or or ...) which keeps your program simple and modular, free from "convenience" dependencies.
If you're using tmux, you can specify which target you want the command to run in:
tmux send -t foo.0 ls ENTER
So, if you've created a tmux session foo.0, you should be able to do:
my_command = 'ls'
tmux_cmd = ['tmux', 'send', '-t', 'foo.0', my_command]
p = subprocess.Popen(tmux_cmd)
You can specify the tty of the terminal window you wish the command to be carried out in:
ls > /dev/ttys004
However, I would recommend going for the tmux approach for greater control (see my other answer).
I have python script that takes command line arguments. The way I get the command line arguments is by reading a mongo database. I need to iterate over the mongo query and launch a different process for the single script with different command line arguments from the mongo query.
Key is, I need the launched processes to be:
separate processes share nothing
when killing the process, I need to be able to kill them all easily.
I think the command killall -9 script.py would work and satisfies the second constraint.
Edit 1
From the answer below, the launcher.py program looks like this
def main():
symbolPreDict = initializeGetMongoAllSymbols()
keys = sorted(symbolPreDict.keys())
for symbol in keys:
# Display key.
print(symbol)
command = ['python', 'mc.py', '-s', str(symbol)]
print command
subprocess.call(command)
if __name__ == '__main__':
main()
The problem is that mc.py has a call that blocks
receiver = multicast.MulticastUDPReceiver ("192.168.0.2", symbolMCIPAddrStr, symbolMCPort )
while True:
try:
b = MD()
data = receiver.read() # This blocks
...
except Exception, e:
print str(e)
When I run the launcher, it just executes one of the mc.py (there are at least 39). How do I modify the launcher program to say "run the launched script in background" so that the script returns to the launcher to launch more scripts?
Edit 2
The problem is solved by replacing subprocess.call(command) with subprocess.Popen(command)
One thing I noticed though, if I say ps ax | grep mc.py, the PID seem to be all different. I don't think I care since I can kill them all pretty easily with killall.
[Correction] kill them with pkill -f xxx.py
There are several options for launching scripts from a script. The easiest are probably to use the subprocess or os modules.
I have done this several times to launch things to separate nodes on a cluster. Using os it might look something like this:
import os
for i in range(len(operations)):
os.system("python myScript.py {:} {:} > out.log".format(arg1,arg2))
using killall you should have no problem terminating processes spawned this way.
Another option is to use subprocess which has got a wide range of features and is much more flexible than os.system. An example might look like:
import subprocess
for i in range(len(operations)):
command = ['python','myScript.py','arg1','arg2']
subprocess.call(command)
In both of these methods, the processes are independent and share nothing other than a parent PID.
I've got this simple python script that ought to fork new processes and then have each execute a command using os.execlp, but the execution only occurs once. I'm curious if there's a timing issue going on that is preventing the additional forks from executing:
import os
for n in range(5):
PID = os.fork()
if PID == 0: #the child...
print("This child's PID is: %s" % os.getpid())
os.execlp('open','-n','-a','Calculator')
# os.popen('open -n -a Calculator')
# os._exit(0)
else:
print("new child forked: %d" % PID)
For this, the "open -n -a Appname" command in OS X launches a new instance of the specified application, so the above code should replace the forked process with the "open" command, and this should run 5 times. However, it only runs once so only one instance of Calculator is opened. Despite this, the parent lists 5 child PIDs forked.
If I comment out the os.execlp line and uncomment the os.popen and os._exit lines following it, then this works properly and the child processes all run the "open" command; however, I am wondering why the approach of replacing the forked process using execlp (or execvp and other similar variants) is not working? Clearly the child process is running, as I can use piping to run the "open" command just fine.
This is with python 3.4.3.
The first argument, after the executable, is arg[0] which is by convention the name of the executable. This is useful, if you have symbolic links, which determine the behavior of the programm. In your case, you name the programm '-n' and the real arguments are only -a and Calculator. So you have to repeat 'open':
os.execlp('open', 'open', '-n', '-a', 'Calculator')
I am attempting to to launch a python script from within another python script, but in a minimized console, then return control to the original shell.
I am able to open the required script in a new shell below, but it's not minimized:
#!/usr/bin/env python
import os
import sys
import subprocess
pyTivoPath="c:\pyTivo\pyTivo.py"
print "Testing: Open New Console"
subprocess.Popen([sys.executable, pyTivoPath], creationflags = subprocess.CREATE_NEW_CONSOLE)
print
raw_input("Press Enter to continue...")
Further, I will need to be able to later remotely KILL this shell from the original script, so I suspect I'll need to be explicit in naming the new process. Correct?
Looking for pointers, please. Thanks!
Note: python27 is mandatory for this application. Eventually will also need to work on Mac and Linux.
Do you need to have the other console open? If you now the commands to be sent, then I'd recommend using Popen.communicate(input="Shell commands") and it will automate the process for you.
So you could write something along the lines of:
# Commands to pass into subprocess (each command is separated by a newline)
commands = (
"command1\n" +
"command2\n"
)
# Your process
py_process = subprocess.Popen(*yourprocess_here*, stdin=PIPE, shell=True)
# Feed process the needed input
py_process.communicate(input=commands)
# Terminate when finished
py_process.terminate()
The code above will execute the process you specify and even send commands but it won't open a new console.
This question already has answers here:
Run a program from python, and have it continue to run after the script is killed
(5 answers)
Closed 8 years ago.
I have recently come across some situations where I want to start a command completely independently, and in a different process then the script, equivalent typing it into a terminal, or more specifically writing the command into a .sh or .desktop and double clicking it. The request I have are:
I can close the python window without closing the application.
The python window does not display the stdout from the application (you don't want random text appearing in a CLI), nor do I need it!.
Things I have tried:
os.system waits.
subprocess.call waits.
subprocess.Popen starts a subprocess (duh) closing the parent process thus quits the application
And thats basically all you can find on the web!
if it really comes down to it I could launch a .sh (or .bat for windows), but that feels like a cheap hack and not the way to do this.
If you were to place an & after the command when called from os.system, would that not work? For example:
import os
os.system( "google-chrome & disown " )
import subprocess
import os
with open(os.devnull, 'w') as f:
proc = subprocess.Popen(command, shell=True, stdout=f, stderr=f)
Spawn a child process "the UNIX way" with fork and exec. Here's a simple example of a shell in Python.
import os
prompt = '$ '
while True:
cmds = input(prompt).split()
if len(cmds):
if (os.fork() == 0):
# Child process
os.execv(cmds[0], cmds)
else:
# Parent process
# (Hint: You don't have to wait, you can just exit here.)
os.wait()