I am trying to display the final results.txt file via default program. I've tried with bare Popen() without run() and got the same effect. The target file is opening (for me it's the see mode) but after exiting it I receive:
Warning: program returned non-zero exit code #256
Is there any way to ignore it and prevent my program from displaying such warning? I don't care about it because it's the last thing the program does, so I don't want people to waste their time clicking Enter each time...
Code's below:
from subprocess import run, Popen
if filepath[len(filepath)-1] != '/':
try:
results = run(Popen(['start', 'results.txt'], shell=True), stdout=None, shell=True, check=False)
except TypeError:
pass
else:
try:
results = run(Popen(['open', 'results.txt']), stdout=None, check=False)
except TypeError:
pass
except FileNotFoundError:
try:
results = run(Popen(['see', 'results.txt']), stdout=None, check=False)
except TypeError:
pass
except FileNotFoundError:
pass
Your immediate error is that you are mixing subprocess.run with subprocess.Popen. The correct syntax is
y = subprocess.Popen(['command', 'argument'])
or
x = subprocess.run(['command', 'argument'])
but you are incorrectly combining them into, effectively
x = subprocess.run(subprocess.Popen(['command', 'argument']), shell=True)
where the shell=True is a separate bug in its own right (though it will weirdly work on Windows).
What happens is that Popen runs successfully, but then you try to run run on the result, which of course is not a valid command at all.
You want to prefer subprocess.run() over subprocess.Popen in this scenario; the latter is for hand-wiring your own low-level functionality in scenarios where run doesn't offer enough flexibility (such as when you require the subprocess to run in parallel with your Python program as an independent process).
Your approach seems vaguely flawed for Unix-like systems; you probably want to run xdg-open if it's available, otherwise the value of os.environ["PAGER"] if it's defined, else fall back to less, else try more. Some ancient systems also have a default pager called pg.
You will definitely want to add check=True to actually make sure your command fails properly if the command cannot be found, which is the diametrical opposite of what you appear to be asking. With this keyword parameter, Python checks whether the command worked, and will raise an exception if not. (In its absence, failures will be silently ignored, in general.) You should never catch every possible exception; instead, trap just the one you really know how to handle.
Okay, I've achieved my goal with a different approach. I didn't need to handle such exception, I did it without the subprocess module.
Question closed, here's the final code (it looks even better):
from os import system
from platform import system as sysname
if sysname() == 'Windows':
system('start results.txt')
elif sysname() == 'Linux':
system('see results.txt')
elif sysname() == 'Darwin':
system('open results.txt')
else:
pass
Related
I'm trying to restart a subprocess if it crashes, but somewhy this loop just doesn't work. I've been wondering if that's even possible?
def dont_stop(conv):
try:
subprocess.call(['python', 'main.py', str(conv)])
except:
dont_stop(conv)
if __name__ == '__main__':
proc = []
for conv in range(3,8):
p = multiprocessing.Process(name=f'p{conv}', target=dont_stop, args=(conv,))
p.start()
proc.append(p)
for p in proc:
p.join()
The subprocess.call function doesn't raise an exception if the program it is running exits in a non-standard way. All it does is return the "return code" from the process you told it to run. That's usually 0 for a process that exits normally, and some other value for a program that crashes (the specific meanings of non-zero values vary between programs and OSs).
Here's a simple solution that replaces your recursive code with a loop that checks the return value of the subprocess:
def dont_stop(conv):
retval = 1
while retval != 0: # a return value of zero indicates a normal exit
retval = subprocess.call(['python', 'main.py', str(conv)])
An alternative approach is to stop using subprocess.call and use subprocess.check_call instead. That function checks the return code and raises an exception if it's not zero. While often that's what we'd prefer, it's actually a bit uglier here.
def dont_stop(conv):
while True:
try:
subprocess.check_call(['python', 'main.py', str(conv)])
break
except subprocess.CalledProcessError:
# do logging here?
pass
Since the program you're running is also a Python program, you might consider importing it, rather than running it in a separate interpreter. That might let your dont_stop function directly interact with the main.py code, such as catching and logging exceptions. The details of that are much too dependent on the design of main.py and what it's supposed to be doing though, so I'm not going to show any suggested code for this approach.
I'm launching a number of subprocesses with subprocess.Popen in Python.
I'd like to check whether one such process has completed. I've found two ways of checking the status of a subprocess, but both seem to force the process to complete.
One is using process.communicate() and printing the returncode, as explained here: checking status of process with subprocess.Popen in Python.
Another is simply calling process.wait() and checking that it returns 0.
Is there a way to check if a process is still running without waiting for it to complete if it is?
Ouestion: ... a way to check if a process is still running ...
You can do it for instance:
p = subprocess.Popen(...
"""
A None value indicates that the process hasn't terminated yet.
"""
poll = p.poll()
if poll is None:
# p.subprocess is alive
Python » 3.6.1 Documentation popen-objects
Tested with Python:3.4.2
Doing the
myProcessIsRunning = poll() is None
As suggested by the main answer, is the recommended way and the simplest way to check if a process running. (and it works in jython as well)
If you do not have the process instance in hand to check it.
Then use the operating system TaskList / Ps processes.
On windows, my command will look as follows:
filterByPid = "PID eq %s" % pid
pidStr = str(pid)
commandArguments = ['cmd', '/c', "tasklist", "/FI", filterByPid, "|", "findstr", pidStr ]
This is essentially doing the same thing as the following command line:
cmd /c "tasklist /FI "PID eq 55588" | findstr 55588"
And on linux, I do exactly the same using the:
pidStr = str(pid)
commandArguments = ['ps', '-p', pidStr ]
The ps command will already be returning error code 0 / 1 depending on whether the process is found. While on windows you need the find string command.
This is the same approach that is discussed on the following stack overflow thread:
Verify if a process is running using its PID in JAVA
NOTE:
If you use this approach, remember to wrap your command call in a try/except:
try:
foundRunningProcess = subprocess.check_output(argumentsArray, **kwargs)
return True
except Exception as err:
return False
Note, be careful if you are developing with VS Code and using pure Python and Jython.
On my environment, I was under the illusion that the poll() method did not work because a process that I suspected that must have ended was indeed running.
This process had launched Wildfly. And after I had asked for wildfly to stop, the shell was still waiting for user to "Press any key to continue . . .".
In order to finish off this process, in pure python the following code was working:
process.stdin.write(os.linesep)
On jython, I had to fix this code to look as follows:
print >>process.stdin, os.linesep
And with this difference the process did indeed finish.
And the jython.poll() started telling me that the process is indeed finished.
As suggested by the other answers None is the designed placeholder for the "return code" when no code has been returned yet by the subprocess.
The documentation for the returncode attribute backs this up (emphasis mine):
The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet.
A negative value -N indicates that the child was terminated by signal N (POSIX only).
An interesting place where this None value occurs is when using the timeout parameter for wait or communicate.
If the process does not terminate after timeout seconds, a TimeoutExpired exception will be raised.
If you catch that exception and check the returncode attribute it will indeed be None
import subprocess
with subprocess.Popen(['ping','127.0.0.1']) as p:
try:
p.wait(timeout=3)
except subprocess.TimeoutExpired:
assert p.returncode is None
If you look at the source for subprocess you can see the exception being raised.
https://github.com/python/cpython/blob/47be7d0108b4021ede111dbd15a095c725be46b7/Lib/subprocess.py#L1930-L1931
If you search that source for self.returncode is you'll find many uses where the library authors lean on that None return code design to infer if an app is running or not running. The returncode attribute is initialized to None and only ever changes in a few spots, the main flow in invocations to _handle_exitstatus to pass on the actual return code.
You could use subprocess.check_output to have a look at your output.
Try this code:
import subprocess
subprocess.check_output(['your command here'], shell=True, stderr=subprocess.STDOUT)
Hope this helped!
Is there a difference between the two code snippets:
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
main()
Vs
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())
I see, most python executables in my ubuntu /usr/bin (or) /usr/local/bin directory use sys.exit. Doesn't the process stop, once the function returns.
Why do people wrap their executable functions inside sys.exit?
Note: This code is taken from openstack-nova python client and this question focusses only on python's sys.exit and not about openstack internals.
sys.exit() is there to pass the right exit code back to shell. If you want to differentiate the response in case of (for example) bad authentication, network issues, broken response, etc. exit codes are useful for that.
If you don't use specific sys.exit(value), you have two options only - success (exit code 0), or exception was thrown (exit code 1).
There are actually two ways to use sys.exit, as explained in the docs.
Your main can return 0 on success and 1 (or any other number from 2-127) on error; that number becomes your program's exit status. (The number 2 has a special meaning; it implies that the failure was because of invalid arguments. Some argument-parsing libraries will automatically sys.exit(2) if they can't parse the command line. The other numbers 3-127 all mean whatever you want them to.)
Or you can return None on success, and a string (or any object with a useful __str__ method) on failure. A None means exit status 0, anything else gets printed to stderr and gives exit status 1.
It used to be traditional to use the second form to signal failure by doing something like return "Failed to open file" from your main function, and the docs still mention doing that, but it's not very common anymore; it's just as easy, and more flexible, to output what you want and return the number you want.
If you just fall off the end of the script without a sys.exit, that's equivalent to sys.exit(0); if you exit through an exception, that's equivalent to passing the traceback to sys.exit—it prints the traceback to stderr and exits with status 1.
It seems that python supports many different commands to stop script execution.The choices I've found are: quit(), exit(), sys.exit(), os._exit()
Have I missed any?
What's the difference between them? When would you use each?
Let me give some information on them:
quit() simply raises the SystemExit exception.
Furthermore, if you print it, it will give a message:
>>> print (quit)
Use quit() or Ctrl-Z plus Return to exit
>>>
This functionality was included to help people who do not know Python. After all, one of the most likely things a newbie will try to exit Python is typing in quit.
Nevertheless, quit should not be used in production code. This is because it only works if the site module is loaded. Instead, this function should only be used in the interpreter.
exit() is an alias for quit (or vice-versa). They exist together simply to make Python more user-friendly.
Furthermore, it too gives a message when printed:
>>> print (exit)
Use exit() or Ctrl-Z plus Return to exit
>>>
However, like quit, exit is considered bad to use in production code and should be reserved for use in the interpreter. This is because it too relies on the site module.
sys.exit() also raises the SystemExit exception. This means that it is the same as quit and exit in that respect.
Unlike those two however, sys.exit is considered good to use in production code. This is because the sys module will always be there.
os._exit() exits the program without calling cleanup handlers, flushing stdio buffers, etc. Thus, it is not a standard way to exit and should only be used in special cases. The most common of these is in the child process(es) created by os.fork.
Note that, of the four methods given, only this one is unique in what it does.
Summed up, all four methods exit the program. However, the first two are considered bad to use in production code and the last is a non-standard, dirty way that is only used in special scenarios. So, if you want to exit a program normally, go with the third method: sys.exit.
Or, even better in my opinion, you can just do directly what sys.exit does behind the scenes and run:
raise SystemExit
This way, you do not need to import sys first.
However, this choice is simply one on style and is purely up to you.
The functions* quit(), exit(), and sys.exit() function in the same way: they raise the SystemExit exception. So there is no real difference, except that sys.exit() is always available but exit() and quit() are only available if the site module is imported (docs).
The os._exit() function is special, it exits immediately without calling any cleanup functions (it doesn't flush buffers, for example). This is designed for highly specialized use cases... basically, only in the child after an os.fork() call.
Conclusion
Use exit() or quit() in the REPL.
Use sys.exit() in scripts, or raise SystemExit() if you prefer.
Use os._exit() for child processes to exit after a call to os.fork().
All of these can be called without arguments, or you can specify the exit status, e.g., exit(1) or raise SystemExit(1) to exit with status 1. Note that portable programs are limited to exit status codes in the range 0-255, if you raise SystemExit(256) on many systems this will get truncated and your process will actually exit with status 0.
Footnotes
* Actually, quit() and exit() are callable instance objects, but I think it's okay to call them functions.
Different Means of Exiting
os._exit():
Exit the process without calling the cleanup handlers.
exit(0):
a clean exit without any errors / problems.
exit(1):
There was some issue / error / problem and that is why the program is exiting.
sys.exit():
When the system and python shuts down; it means less memory is being used after the program is run.
quit():
Closes the python file.
Summary
Basically they all do the same thing, however, it also depends on what you are doing it for.
I don't think you left anything out and I would recommend getting used to quit() or exit().
You would use sys.exit() and os._exit() mainly if you are using big files or are using python to control terminal.
Otherwise mainly use exit() or quit().
sys.exit is the canonical way to exit.
Internally sys.exit just raises SystemExit. However, calling sys.exitis more idiomatic than raising SystemExit directly.
os.exit is a low-level system call that exits directly without calling any cleanup handlers.
quit and exit exist only to provide an easy way out of the Python prompt. This is for new users or users who accidentally entered the Python prompt, and don't want to know the right syntax. They are likely to try typing exit or quit. While this will not exit the interpreter, it at least issues a message that tells them a way out:
>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit
>>> exit()
$
This is essentially just a hack that utilizes the fact that the interpreter prints the __repr__ of any expression that you enter at the prompt.
I am developing a wrapper around gdb using python. Basically, I just want to be able to detect a few setup annoyances up-front and be able to run a single command to invoke gdb, rather than a huge string I have to remember each time.
That said, there are two cases that I am using. The first, which works fine, is invoking gdb by creating a new process and attaching to it. Here's the code that I have for this one:
def spawnNewProcessInGDB():
global gObjDir, gGDBProcess;
from subprocess import Popen
from os.path import join
import subprocess
binLoc = join(gObjDir, 'dist');
binLoc = join(binLoc, 'bin');
binLoc = join(binLoc, 'mycommand')
profileDir = join(gObjDir, '..')
profileDir = join(profileDir, 'trash-profile')
try:
gGDBProcess = Popen(['gdb', '--args', binLoc, '-profile', profileDir], cwd=gObjDir)
gGDBProcess.wait()
except KeyboardInterrupt:
# Send a termination signal to the GDB process, if it's running
promptAndTerminate(gGDBProcess)
Now, if the user presses CTRL-C while this is running, it breaks (i.e. it forwards the CTRL-C to GDB). This is the behavior I want.
The second case is a bit more complicated. It might be the case that I already had this program running on my system and it crashed, but was caught. In this case, I want to be able to connect to it using gdb to get a stack trace (or perhaps I was already running it, and I simply now want to connect to the process that's already in memory).
As a convenience feature, I've created a mirror function, which will connect to a running process using gdb:
def connectToProcess(procNum):
global gObjDir, gGDBProcess
from subprocess import Popen
import subprocess
import signal
print("Connecting to mycommand process number " + str(procNum) + "...")
try:
gGDBProcess = Popen(['gdb', '-p', procNum], cwd=gObjDir)
gGDBProcess.wait()
except KeyboardInterrupt:
promptAndTerminate(gGDBProcess)
Again, this seems to work as expected. It starts gdb, I can set breakpoints, run the program, etc. The only catch is that it doesn't forward CTRL-C to gdb if I press it while the program is running. Instead, it jumps immediately to promptAndTerminate().
I'm wondering if anyone can see why this is happening - the two calls to subprocess.Popen() seem identical to me, albeit that one is running gdb in a different mode.
I have also tried replacing the call to subprocess.Popen() with the following:
gGDBProcess = Popen(['gdb', '-p', procNum], cwd=gObjDir, stdin=subprocess.PIPE)
but this leads to undesirable results as well, because it doesn't actually communicate anything to the child gdb process (e.g. if I type in c to start the program running again after it is broken upon connection from gdb, it doesn't do anything). Again, it terminates the running python process when I type CTRL-C.
Any help would be appreciated!