I'm trying to restart a subprocess if it crashes, but somewhy this loop just doesn't work. I've been wondering if that's even possible?
def dont_stop(conv):
try:
subprocess.call(['python', 'main.py', str(conv)])
except:
dont_stop(conv)
if __name__ == '__main__':
proc = []
for conv in range(3,8):
p = multiprocessing.Process(name=f'p{conv}', target=dont_stop, args=(conv,))
p.start()
proc.append(p)
for p in proc:
p.join()
The subprocess.call function doesn't raise an exception if the program it is running exits in a non-standard way. All it does is return the "return code" from the process you told it to run. That's usually 0 for a process that exits normally, and some other value for a program that crashes (the specific meanings of non-zero values vary between programs and OSs).
Here's a simple solution that replaces your recursive code with a loop that checks the return value of the subprocess:
def dont_stop(conv):
retval = 1
while retval != 0: # a return value of zero indicates a normal exit
retval = subprocess.call(['python', 'main.py', str(conv)])
An alternative approach is to stop using subprocess.call and use subprocess.check_call instead. That function checks the return code and raises an exception if it's not zero. While often that's what we'd prefer, it's actually a bit uglier here.
def dont_stop(conv):
while True:
try:
subprocess.check_call(['python', 'main.py', str(conv)])
break
except subprocess.CalledProcessError:
# do logging here?
pass
Since the program you're running is also a Python program, you might consider importing it, rather than running it in a separate interpreter. That might let your dont_stop function directly interact with the main.py code, such as catching and logging exceptions. The details of that are much too dependent on the design of main.py and what it's supposed to be doing though, so I'm not going to show any suggested code for this approach.
Related
I am trying to display the final results.txt file via default program. I've tried with bare Popen() without run() and got the same effect. The target file is opening (for me it's the see mode) but after exiting it I receive:
Warning: program returned non-zero exit code #256
Is there any way to ignore it and prevent my program from displaying such warning? I don't care about it because it's the last thing the program does, so I don't want people to waste their time clicking Enter each time...
Code's below:
from subprocess import run, Popen
if filepath[len(filepath)-1] != '/':
try:
results = run(Popen(['start', 'results.txt'], shell=True), stdout=None, shell=True, check=False)
except TypeError:
pass
else:
try:
results = run(Popen(['open', 'results.txt']), stdout=None, check=False)
except TypeError:
pass
except FileNotFoundError:
try:
results = run(Popen(['see', 'results.txt']), stdout=None, check=False)
except TypeError:
pass
except FileNotFoundError:
pass
Your immediate error is that you are mixing subprocess.run with subprocess.Popen. The correct syntax is
y = subprocess.Popen(['command', 'argument'])
or
x = subprocess.run(['command', 'argument'])
but you are incorrectly combining them into, effectively
x = subprocess.run(subprocess.Popen(['command', 'argument']), shell=True)
where the shell=True is a separate bug in its own right (though it will weirdly work on Windows).
What happens is that Popen runs successfully, but then you try to run run on the result, which of course is not a valid command at all.
You want to prefer subprocess.run() over subprocess.Popen in this scenario; the latter is for hand-wiring your own low-level functionality in scenarios where run doesn't offer enough flexibility (such as when you require the subprocess to run in parallel with your Python program as an independent process).
Your approach seems vaguely flawed for Unix-like systems; you probably want to run xdg-open if it's available, otherwise the value of os.environ["PAGER"] if it's defined, else fall back to less, else try more. Some ancient systems also have a default pager called pg.
You will definitely want to add check=True to actually make sure your command fails properly if the command cannot be found, which is the diametrical opposite of what you appear to be asking. With this keyword parameter, Python checks whether the command worked, and will raise an exception if not. (In its absence, failures will be silently ignored, in general.) You should never catch every possible exception; instead, trap just the one you really know how to handle.
Okay, I've achieved my goal with a different approach. I didn't need to handle such exception, I did it without the subprocess module.
Question closed, here's the final code (it looks even better):
from os import system
from platform import system as sysname
if sysname() == 'Windows':
system('start results.txt')
elif sysname() == 'Linux':
system('see results.txt')
elif sysname() == 'Darwin':
system('open results.txt')
else:
pass
I'm launching a number of subprocesses with subprocess.Popen in Python.
I'd like to check whether one such process has completed. I've found two ways of checking the status of a subprocess, but both seem to force the process to complete.
One is using process.communicate() and printing the returncode, as explained here: checking status of process with subprocess.Popen in Python.
Another is simply calling process.wait() and checking that it returns 0.
Is there a way to check if a process is still running without waiting for it to complete if it is?
Ouestion: ... a way to check if a process is still running ...
You can do it for instance:
p = subprocess.Popen(...
"""
A None value indicates that the process hasn't terminated yet.
"""
poll = p.poll()
if poll is None:
# p.subprocess is alive
Python » 3.6.1 Documentation popen-objects
Tested with Python:3.4.2
Doing the
myProcessIsRunning = poll() is None
As suggested by the main answer, is the recommended way and the simplest way to check if a process running. (and it works in jython as well)
If you do not have the process instance in hand to check it.
Then use the operating system TaskList / Ps processes.
On windows, my command will look as follows:
filterByPid = "PID eq %s" % pid
pidStr = str(pid)
commandArguments = ['cmd', '/c', "tasklist", "/FI", filterByPid, "|", "findstr", pidStr ]
This is essentially doing the same thing as the following command line:
cmd /c "tasklist /FI "PID eq 55588" | findstr 55588"
And on linux, I do exactly the same using the:
pidStr = str(pid)
commandArguments = ['ps', '-p', pidStr ]
The ps command will already be returning error code 0 / 1 depending on whether the process is found. While on windows you need the find string command.
This is the same approach that is discussed on the following stack overflow thread:
Verify if a process is running using its PID in JAVA
NOTE:
If you use this approach, remember to wrap your command call in a try/except:
try:
foundRunningProcess = subprocess.check_output(argumentsArray, **kwargs)
return True
except Exception as err:
return False
Note, be careful if you are developing with VS Code and using pure Python and Jython.
On my environment, I was under the illusion that the poll() method did not work because a process that I suspected that must have ended was indeed running.
This process had launched Wildfly. And after I had asked for wildfly to stop, the shell was still waiting for user to "Press any key to continue . . .".
In order to finish off this process, in pure python the following code was working:
process.stdin.write(os.linesep)
On jython, I had to fix this code to look as follows:
print >>process.stdin, os.linesep
And with this difference the process did indeed finish.
And the jython.poll() started telling me that the process is indeed finished.
As suggested by the other answers None is the designed placeholder for the "return code" when no code has been returned yet by the subprocess.
The documentation for the returncode attribute backs this up (emphasis mine):
The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet.
A negative value -N indicates that the child was terminated by signal N (POSIX only).
An interesting place where this None value occurs is when using the timeout parameter for wait or communicate.
If the process does not terminate after timeout seconds, a TimeoutExpired exception will be raised.
If you catch that exception and check the returncode attribute it will indeed be None
import subprocess
with subprocess.Popen(['ping','127.0.0.1']) as p:
try:
p.wait(timeout=3)
except subprocess.TimeoutExpired:
assert p.returncode is None
If you look at the source for subprocess you can see the exception being raised.
https://github.com/python/cpython/blob/47be7d0108b4021ede111dbd15a095c725be46b7/Lib/subprocess.py#L1930-L1931
If you search that source for self.returncode is you'll find many uses where the library authors lean on that None return code design to infer if an app is running or not running. The returncode attribute is initialized to None and only ever changes in a few spots, the main flow in invocations to _handle_exitstatus to pass on the actual return code.
You could use subprocess.check_output to have a look at your output.
Try this code:
import subprocess
subprocess.check_output(['your command here'], shell=True, stderr=subprocess.STDOUT)
Hope this helped!
I am programming in python which involves me implementing a shell in Python in Linux. I am trying to run standard unix commands by using os.execvp(). I need to keep asking the user for commands so I have used an infinite while loop. However, the infinite while loop doesn't work. I have tried searching online but they're isn't much available for Python. Any help would be appreciated. Thanks
This is the code I have written so far:
import os
import shlex
def word_list(line):
"""Break the line into shell words."""
lexer = shlex.shlex(line, posix=True)
lexer.whitespace_split = False
lexer.wordchars += '#$+-,./?#^='
args = list(lexer)
return args
def main():
while(True):
line = input('psh>')
split_line = word_list(line)
if len(split_line) == 1:
print(os.execvp(split_line[0],[" "]))
else:
print(os.execvp(split_line[0],split_line))
if __name__ == "__main__":
main()
So when I run this and put in the input "ls" I get the output "HelloWorld.py" (which is correct) and "Process finished with exit code 0". However I don't get the output "psh>" which is waiting for the next command. No exceptions are thrown when I run this code.
Your code does not work because it uses os.execvp. os.execvp replaces the current process image completely with the executing program, your running process becomes the ls.
To execute a subprocess use the aptly named subprocess module.
In case of an ill-advised programming exercise then you need to:
# warning, never do this at home!
pid = os.fork()
if not pid:
os.execvp(cmdline) # in child
else:
os.wait(pid) # in parent
os.fork returns twice, giving the pid of child in parent process, zero in child process.
If you want it to run like a shell you are looking for os.fork() . Call this before you call os.execvp() and it will create a child process. os.fork() returns the process id. If it is 0 then you are in the child process and can call os.execvp(), otherwise continue with the code. This will keep the while loop running. You can have the original process either wait for it to complete os.wait(), or continue without waiting to the start of the while loop. The pseudo code on page 2 of this link should help https://www.cs.auckland.ac.nz/courses/compsci340s2c/assignments/A1/A1.pdf
Here i'm reading and comparing values from the two logs using 'for' loop.Problem is i'm not able to continue to next TC after sys.exit command. Let me know if required more clarification
f = open('/tmp/ftplog', 'r')
for line in f:
m = re.findall("5\d+", line)
#print m
fd = open('/tmp/tellog', 'r')
for line in fd:
n = re.findall(r"5\d+", line)
#print n
if m == n:
print "passed"
sys.exit()
####TC-02####
def tc02(ipadrr,login,password,ftpipaddr,ftplogin,ftppassword,ftpfilename):
try:
telconn2 = pexpect.spawn(ipadrr)
You can add hooks that will be executed on exit using atexit. http://docs.python.org/2/library/atexit.html?highlight=atexit#atexit
However, needing to do this in a simple script is usually a sign your logic is wrong. Do you really need to exit? Could you throw an exception instead? break? return? For example, try having the logic in a function, the function returning when it is done, and some code that calls it and does something with the returned result.
sys.exit actually throws a SystemExit exception, which you can catch, but you really shouldn't. Restructure your program so you don't have to.
try this
print "passed"
return ####instead of sys.exit use return
A way to do this if it's really needed is to call the same script but with different parameters in a subprocess just before you exit it like this:
import subprocess
p = subprocess.Popen("exec your_script.py -parameters parameter", stdout=subprocess.PIPE, shell=True)
Then you can add some checks for a specific parameter that you will provide and execute only this part of your code (e.g. the tc02() function that you need).
Just keep in mind that once you call the script as a subprocess, you won't be able to stop it from the console with Ctr+C, since this will kill the parent process, but not the child processes. In order to kill everything you need to call a method like this:
p.kill()
I'm writing an IRC bot in Python, due to the alpha nature of it, it will likely get unexpected errors and exit.
What's the techniques that I can use to make the program run again?
You can use sys.exit() to tell that the program exited abnormally (generally, 1 is returned in case of error).
Your Python script could look something like this:
import sys
def main():
# ...
if __name__ == '__main__':
try:
main()
except Exception as e:
print >> sys.stderr, e
sys.exit(1)
else:
sys.exit()
You could call again main() in case of error, but the program might not be in a state where it can work correctly again.
It may be safer to launch the program in a new process instead.
So you could write a script which invokes the Python script, gets its return value when it finishes, and relaunches it if the return value is different from 0 (which is what sys.exit() uses as return value by default).
This may look something like this:
import subprocess
command = 'thescript'
args = ['arg1', 'arg2']
while True:
ret_code = subprocess.call([command] + args)
if ret_code == 0:
break
You can create wrapper using subprocess(http://docs.python.org/library/subprocess.html) which will spawn your application as a child process and track it's execution.
The easiest way is to catch errors, and close the old and open a new instance of the program when you do catch em.
Note that it will not always work (in cases it stops working without throwing an error).