A job in Jenkins calls my python script, in which I want to call make to compile my UT codes, and raise sys.exit(1) if a compiling error like "make: *** [ ] Error 1" occurs.
I also need to print output in real time.
Here's my python script:
make_process = subprocess.Popen("make clean all;", shell=True, stdout=subprocess.PIPE, stderr=sys.stdout.fileno())
while True:
line = make_process.stdout.readline()
if not line:break
print line, #output to console in time
sys.stdout.flush()
But how do I capture the make error and let this python script fail?
Note:
make_process.wait() is 0 when make error happens.
this answer doesn't work for me:
Running shell command from Python and capturing the output
Update:
It turned out to be a makefile issue. See the comments on the accepted answer. But for this python question, pentadecagon gave the right answer.
You can check the return value of the make process by
make_process.poll()
This returns "None" if the process is still running, or the error code if it's finished. If you just want the output to end up on the console there is no need to do this manually
The output goes to the console anyway, and can do it like this:
make_process = subprocess.Popen("make clean all", stderr=subprocess.STDOUT)
if make_process.wait() != 0:
something_went_wrong();
Related
I'm trying to write a simple software to scan some bluetooth devices (beacon with advertising) and I have a problem with the instruction subprocess.Popen
p1 = subprocess.Popen(['timeout','10s','hcitool','lescan'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
p1.wait()
output, error = p1.communicate()
print("stdout: {}".format(output))
print("stderr: {}".format(error))
the output and error variables are empty!
If I remove the stdout=subprocess.PIPE,stderr=subprocess.PIPE from the Popen I can see the right result in the console, if I change the command with ['ls','-l'] it works fine and I see the result in the variables .
I've tryed with subprocess.run (with the timeout) and it is the same.
If I don't use the timeout obviously the command never ends.
I can't use pybluez and my python version is the 3.7
Can someone help me?
Solved using ['timeout','-s','INT','10s','hcitool','lescan'] as command instead of ['timeout','10s','hcitool','lescan'].
Maybe in the second case the process was not killed well and I didn't receive the output.
Thank you the same.
I am trying to retrive error(exit code importantly) thrown by any command line execution in python script itself. I am using subprocess for this. When I execute any wrong commands then it throws an error in terminal as usual, but then it stops executing python file and I can't get store error.
Look at the code. p_status is supposed to store exit code. But before printing it stops the script after throwing error in terminal.
process = subprocess.Popen([<command>], stdout = subprocess.PIPE)
output = process.communicate()
p_status = process.wait()
print(p_status)
I went through different solutions and tried all of them but couldn't get the required result.
Got this problem solved using the following code:
try:
subprocess.Popen([<command>], stdout = subprocess.PIPE)
except OSError as error:
print(error.errno) #for exit code
P.S - credits #karolch
I'm running an application from within my code, and it rewrites files which I need to read later on in the code. There is no output the goes directly into my program. I can't get my code to wait until the subprocess has finished, it just goes ahead and reads the unchanged files.
I've tried subprocess.Popen.wait(), subprocess.call(), and subprocess.check_call(), but none of them work for my problem. Does anyone have any idea how to make this work? Thanks.
Edit: Here is the relevant part of my code:
os.chdir('C:\Users\Jeremy\Documents\FORCAST\dusty')
t = subprocess.Popen('start dusty.exe', shell=True)
t.wait()
os.chdir('C:\Users\Jeremy\Documents\FORCAST')
Do you use the return object of subprocess.Popen()?
p = subprocess.Popen(command)
p.wait()
should work.
Are you sure that the command does not end instantly?
If you execute a program with
t = subprocess.Popen(prog, Shell=True)
Python won't thrown an error, regardless whether the program exists or not. If you try to start an non-existing program with Popen and Shell=False, you will get an error. My guess would be that your program either doesn't exist in the folder or doesn't execute. Try to execute in the Python IDLE environment with Shell=False and see if you get a new window.
I am trying to determine the best way to execute something in command line using python. I have accomplished this with subprocess.Popen() on individual files. However, I am trying to determine the best way to do this many time with numerous different files. I am not sure if I should create a batch file and then execute that in command, or if I am simply missing something in my code. Novice coder here so I apologize in advance. The script below returns a returncode of 1 when I use a loop, but a 0 when not in a loop. What is the best approach for the task at hand?
def check_output(command, console):
if console == True:
process = subprocess.Popen(command)
else:
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
output, error = process.communicate()
returncode = process.poll()
return returncode, output, error
for file in fileList.split(";"):
...code to create command string...
returncode, output, error = check_output(command, False)
if returncode != 0:
print("Process failed")
sys.exit()
EDIT: An example command string looks like this:
C:\Path\to\executable.exe -i C:\path\to\input.ext -o C:\path\to\output.ext
Try using the commands module (only available before python 3)
>>> import commands
>>> commands.getstatusoutput('ls /bin/ls')
(0, '/bin/ls')
Your code might look like this
import commands
def runCommand( command ):
ret,output = commands.getstatutoutput( command )
if ret != 0:
sys.stderr.writelines( "Error: "+output )
return ret
for file in fileList.split(';'):
commandStr = ""
# Create command string
if runCommand( commandStr ):
print("Command '%s' failed" % commandStr)
sys.exit(1)
You are not entirely clear about the problem you are trying to solve. If I had to guess why your command is failing in the loop, its probably the way you handle the console=False case.
If you are merely running commands one after another, then it is probably easiest to cast aside Python and stick your commands into a bash script. I assume you merely want to check errors and abort if one of the commands fails.
#!/bin/bash
function abortOnError(){
"$#"
if [ $? -ne 0 ]; then
echo "The command $1 failed with error code $?"
exit 1
fi
}
abortOnError ls /randomstringthatdoesnotexist
echo "Hello World" # This will never print, because we aborted
Update: OP updated his question with sample data that indicate he is on Windows.
You can get bash for Windows through cygwin or various other packages, but it may make more sense to use PowerShell if you are on Windows. Unfortunately, I do not have a Windows box, but there should be a similar mechanism for error checking. Here is a reference for PowerShell error handling.
You might consider using subprocess.call
from subprocess import call
for file_name in file_list:
call_args = 'command ' + file_name
call_args = call_args.split() # because call takes a list of strings
call(call_args)
It also will output 0 for success and 1 for failure.
What your code is trying to accomplish is to run a command on a file, and exit the script if there's an error. subprocess.check_output accomplishes this - if the subprocess exits with an error code it raises a Python error. Depending on whether you want to explicitly handle errors, your code would look something like this:
file in fileList.split(";"):
...code to create command string...
subprocess.check_output(command, shell=True)
Which will execute the command and print the shell error message if there is one, or
file in fileList.split(";"):
...code to create command string...
try:
subprocess.check_output(command,shell=True)
except subprocess.CalledProcessError:
...handle errors...
sys.exit(1)
Which will print the shell error code and exit, as in your script.
I have inherited some code which is periodically (randomly) failing due to an Input/Output error being raised during a call to print. I am trying to determine the cause of the exception being raised (or at least, better understand it) and how to handle it correctly.
When executing the following line of Python (in a 2.6.6 interpreter, running on CentOS 5.5):
print >> sys.stderr, 'Unable to do something: %s' % command
The exception is raised (traceback omitted):
IOError: [Errno 5] Input/output error
For context, this is generally what the larger function is trying to do at the time:
from subprocess import Popen, PIPE
import sys
def run_commands(commands):
for command in commands:
try:
out, err = Popen(command, shell=True, stdout=PIPE, stderr=PIPE).communicate()
print >> sys.stdout, out
if err:
raise Exception('ERROR -- an error occurred when executing this command: %s --- err: %s' % (command, err))
except:
print >> sys.stderr, 'Unable to do something: %s' % command
run_commands(["ls", "echo foo"])
The >> syntax is not particularly familiar to me, it's not something I use often, and I understand that it is perhaps the least preferred way of writing to stderr. However I don't believe the alternatives would fix the underlying problem.
From the documentation I have read, IOError 5 is often misused, and somewhat loosely defined, with different operating systems using it to cover different problems. The best I can see in my case is that the python process is no longer attached to the terminal/pty.
As best I can tell nothing is disconnecting the process from the stdout/stderr streams - the terminal is still open for example, and everything 'appears' to be fine. Could it be caused by the child process terminating in an unclean fashion? What else might be a cause of this problem - or what other steps could I introduce to debug it further?
In terms of handling the exception, I can obviously catch it, but I'm assuming this means I wont be able to print to stdout/stderr for the remainder of execution? Can I reattach to these streams somehow - perhaps by resetting sys.stdout to sys.__stdout__ etc? In this case not being able to write to stdout/stderr is not considered fatal but if it is an indication of something starting to go wrong I'd rather bail early.
I guess ultimately I'm at a bit of a loss as to where to start debugging this one...
I think it has to do with the terminal the process is attached to. I got this error when I run a python process in the background and closed the terminal in which I started it:
$ myprogram.py
Ctrl-Z
$ bg
$ exit
The problem was that I started a not daemonized process in a remote server and logged out (closing the terminal session). A solution was to start a screen/tmux session on the remote server and start the process within this session. Then detaching the session+log out keeps the terminal associated with the process. This works at least in the *nix world.
I had a very similar problem. I had a program that was launching several other programs using the subprocess module. Those subprocesses would then print output to the terminal. What I found was that when I closed the main program, it did not terminate the subprocesses automatically (as I had assumed), rather they kept running. So if I terminated both the main program and then the terminal it had been launched from*, the subprocesses no longer had a terminal attached to their stdout, and would throw an IOError. Hope this helps you.
*NB: it must be done in this order. If you just kill the terminal, (for some reason) that would kill both the main program and the subprocesses.
I just got this error because the directory where I was writing files to ran out of memory. Not sure if this is at all applicable to your situation.
I'm new here, so please forgive if I slip up a bit when it comes to the code detail.
Recently I was able to figure out what cause the I/O error of the print statement when the terminal associated with the run of the python script is closed.
It is because the string to be printed to stdout/stderr is too long. In this case, the "out" string is the culprit.
To fix this problem (without having to keep the terminal open while running the python script), simply read the "out" string line by line, and print line by line, until we reach the end of the "out" string. Something like:
while true:
ln=out.readline()
if not ln: break
print ln.strip("\n") # print without new line
The same problem occurs if you print the entire list of strings out to the screen. Simply print the list one item by one item.
Hope that helps!
The problem is you've closed the stdout pipe which python is attempting to write to when print() is called
This can be caused by running a script in the background using & and then closing the terminal session (ie. closing stdout)
$ python myscript.py &
$ exit
One solution is to set stdout to a file when running in the background
Example
$ python myscript.py > /var/log/myscript.log 2>&1 &
$ exit
No errors on print()
It could happen when your shell crashes while the print was trying to write the data into it.
For my case, I just restart the service, then this issue disappear. don't now why.
My issue was the same OSError Input/Output error, for Odoo.
After I restart the service, it disappeared.