Develop Reference

Python subprocess doesn't run until calling process finished

EDIT 1 - added more code
I'm not sure that proc.communicate was needed, it was one of the suggestions I found from some other stackoverflow code.(Sorry I was tired last night and didn't think too much before asking the question.)
I should add that I am not an experienced coder (mechanical engineer) as you can probably tell from my code
In my Gui I have a button to call a subprocess
The subprocess (screenshot-cmd.exe) creates a png of a cropped screen shot but it won't actually produce the file until there is an error or if the button click event is over.
This makes me think that the subprocess is not actually run until the event is finished
I want to call the process several times after a single button press and move the files that it produces after each one is produced
if I use proc.wait(), the process hangs indefinitely.
How do I stop this?
# function to take a single image called 'fileName' and place it in directory 'dir'
def takeImage(dir,fileName):
# calculate the view to capture to get the whole display window in.
clientRect = win32gui.GetClientRect(win32gui.GetForegroundWindow())
windowRect = win32gui.GetWindowRect(win32gui.GetForegroundWindow())
print(windowRect)
windowSize = [windowRect[2]-windowRect[0],windowRect[3]-windowRect[1]]
print(windowSize)
print(clientRect)
diffSize = [windowSize[0] -clientRect[2], windowSize[1] - clientRect[3]]
lrbBorder = diffSize[0]/2
topBorder = diffSize[1] - lrbBorder
print("sizeDiff = " + str(diffSize))
windowName = win32gui.GetWindowText(win32gui.GetForegroundWindow())
handleId = win32gui.GetForegroundWindow()
leftMar = designLabel.GetPosition()[0] + lrbBorder
topMar = designLabel.GetPosition()[1] + topBorder + designLabel.GetSize()[1]
rightMar = leftMar + scene.width
bottMar = topMar+scene.height
margins = [leftMar,topMar,rightMar,bottMar]
print(margins)
# now print the view.
#command_line = r"screenshot-cmd -wt '" + windowName + "' -rc " + str(margins[0]) + " " + str(margins[1]) + " " + str(margins[2]) + " " + str(margins[3]) + " -o " + fileName
command_line = r"screenshot-cmd -wt '" + windowName + "' -rc " + str(margins[0]) + " " + str(margins[1]) + " " + str(margins[2]) + " " + str(margins[3]) + " -o " + fileName
print(command_line)
args = shlex.split(command_line)
proc = subprocess.Popen(args)
proc.wait()
wx.Yield()
if not os.path.isdir(dir):
os.makedirs(dir)
newPath = os.path.join(dir,fileName)
if os.path.exists(newPath):
os.remove(newPath)
oldPath = os.path.join(os.getcwd(), fileName)
print("Old Path: " + oldPath)
print("Exists: " + str(os.path.exists(oldPath)))
shutil.move(oldPath,newPath)
return
#event called upon clicking 'takeTenImag' button
def takeTenImgE(evt):
global designNo
global workingDirectory
global numDesigns
fileNameRoot = "test_"
fileExtention = ".png"
# check there are at least 10 designs
if numDesigns > 9 and os.path.exists(workingDirectory):
# find directory path to put images in
dir = os.path.join(workingDirectory, "images")
# for each design
for x in range(10):
print("design =" + str(designNo))
fileName = fileNameRoot + str(designNo) + fileExtention
print("------------------")
print("for x = " + str(x) + " " + fileName)
# create image and save
print(dir)
takeImage(dir,fileName)
#move to next design
wx.PostEvent(forwardDesign, wx.CommandEvent(wx.wxEVT_COMMAND_BUTTON_CLICKED, forwardDesign.GetId()) )
wx.Yield()
print("design =" + str(designNo))
return
takeTenImg = wx.Button(p, label='Take Ten Images', pos=(rb + visScaleText.GetSize()[0]+10,takeImg.GetPosition()[1]+5 +takeImg.GetSize()[1]), size = (100,30))
takeTenImg.Bind(wx.EVT_BUTTON, takeTenImgE)
https://code.google.com/p/screenshot-cmd/

Barnaby, you may be over-complicating your subprocess use. Popen is typically used for when you need to communicate with the process during the time it is running. From the sound of it, you don't need to do that, so might want to use a higher level function. See the docs on subprocess' various invocations, and perhaps try using the call method. You'll need shell=True, as detailed in SO questions such as this one.

I've found that the error is in my calling of subprocess.
I was using:
command_line = r"screenshot-cmd -wt '" + windowName + ...."
args = shlex.split(command_line)
subprocess.call(args,shell=True)
changing this to:
command_line = r"screenshot-cmd -wt '" + windowName + ...."
subprocess.call(command_line,shell=True)
solves the hang.
What is peculiar is that both options work when not inside a wx button click event(i.e. a python script launched from the command line), but only the second works when inside the wx button click event.
If anyone could enlighten me why that would be most appreciated.
EDIT:
upon further investigation, the hang is caused by trying to specify the active window in screenshot-cmd.
To solve this I find the position of the window using
windowRect = win32gui.GetWindowRect(win32gui.GetForegroundWindow())
and then use screenshot-cmd without specifying a window.
This solves all issues although it is unclear why this causes problems

Execute Shell Script from python with arguments

I want to execute a shell script with 3 arguments from a python script. (as described here: Python: executing shell script with arguments(variable), but argument is not read in shell script)
Here is my code:
subprocess.call('/root/bin/xen-limit %s %s %s' % (str(dom),str(result),str('--nosave'),), shell=True)
variables dom and result are containing strings.
And here is the output:
/bin/sh: --nosave: not found
UPDATE:
That is the variable "result":
c1 = ['/bin/cat', '/etc/xen/%s.cfg' % (str(dom))]
p1 = subprocess.Popen(c1, stdout=subprocess.PIPE)
c2 = ['grep', 'limited']
p2 = subprocess.Popen(c2, stdin=p1.stdout, stdout=subprocess.PIPE)
c3 = ['cut', '-d=', '-f2']
p3 = subprocess.Popen(c3, stdin=p2.stdout, stdout=subprocess.PIPE)
c4 = ['tr', '-d', '\"']
p4 = subprocess.Popen(c4, stdin=p3.stdout, stdout=subprocess.PIPE)
result = p4.stdout.read()
After that, the variable result is containing a number with mbit (for example 16mbit)
And dom is a string like "myserver"

from subprocess import Popen, STDOUT, PIPE
print('Executing: /root/bin/xen-limit ' + str(dom) + ' ' + str(result) + ' --nosave')
handle = Popen('/root/bin/xen-limit ' + str(dom) + ' ' + str(result) + ' --nosave', shell=True, stdout=PIPE, stderr=STDOUT, stdin=PIPE)
print(handle.stdout.read())
If this doesn't work i honestly don't know what would.
This is the most basic but yet error describing way of opening a 3:d party application or script while still giving you the debug you need.

Why not you save --nosave to a variable and pass the variable in subprocess

It's simpler (and safer) to pass a list consisting of the command name and its arguments.
subprocess.call(['/root/bin/xen-limit]',
str(dom),
str(result),
str('--nosave')
])
str('--nosave') is a no-op, as '--nosave' is already a string. The same may be true for dom and result as well.

Python. Second step of subprocess.Popen truncates results of first

In the snipet of my python script below, I think that temp2 doesn't wait for temp to finish running, the output can be large, but is just text. This truncates the result ('out') from temp, it stops mid line. 'out' from temp works fine until temp 2 is added. I tried adding time.wait() as well as subprocess.Popen.wait(temp). These both allow temp to run to completion so that 'out' is not truncated but disrupt the chaining process so that there is no 'out2'. Any ideas?
temp = subprocess.Popen(call, stdout=subprocess.PIPE)
#time.wait(1)
#subprocess.Popen.wait(temp)
temp2 = subprocess.Popen(call2, stdin=temp.stdout, stdout=subprocess.PIPE)
out, err = temp.communicate()
out2, err2 = temp2.communicate()

According to the Python Docs communicate() can accept a stream to be sent as input. If you change stdin of temp2 to subprocess.PIPE and put out in communicate(), the data is properly piped.
#!/usr/bin/env python
import subprocess
import time
call = ["echo", "hello\nworld"]
call2 = ["grep", "w"]
temp = subprocess.Popen(call, stdout=subprocess.PIPE)
temp2 = subprocess.Popen(call2, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = temp.communicate()
out2, err2 = temp2.communicate(out)
print("Out: {0!r}, Err: {1!r}".format(out, err))
# Out: b'hello\nworld\n', Err: None
print("Out2: {0!r}, Err2: {1!r}".format(out2, err2))
# Out2: b'world\n', Err2: None

Following "Replacing shell pipeline" section from the docs:
temp = subprocess.Popen(call, stdout=subprocess.PIPE)
temp2 = subprocess.Popen(call2, stdin=temp.stdout, stdout=subprocess.PIPE)
temp.stdout.close()
out2 = temp2.communicate()[0]

Python code to send command through command line

final="cacls " + "E:/" + "\"" + list1[2] + " " + list1[3] + "\"" + " /p " + str
os.system(final)
I am trying to set permission to a folder Using Python but while running this command , User input needs to be provided too i.e
it asks ARE YOU SURE(Y/N) and the user needs to enter "Y" or "N"
Is there any way to use python to send the user input "Y" along with the above code?
pro = subprocess.Popen(final,shell=True, stdin=subprocess.PIPE)
pro.communicate(bytes("Y\r\n",'utf-8'))
I have added the following code . The program exits without setting the permission.
http://jimmyg.org/blog/2009/working-with-python-subprocess.html#writing-to-standard-input

Try using the subprocess module
import subprocess
cmd = ["cacls", "E:/" + list1[2], list1[3], "/p", str]
pro = subprocess.Popen(final, stdin=subprocess.PIPE)
pro.communicate("y\r\n")

As a smart programmer, use PBS
Then, the code is:
from pbs import type as echo# Isn't it echo for Windows? If not, use the correct one
script = Command("/path/to/cacls ")
print script(echo("Y"), ("E:/" + "\"" + list1[2] + " " + list1[3] + "\"" + " /p " + str).split())

Pass python script output to another programs stdin

I have an application that takes input, either from the terminal directly or I can use a pipe to pass the output of another program into the stdin of this one. What I am trying to do is use python to generate the output so it's formatted correctly and pass that to the stdin of this program all from the same script. Here is the code:
#!/usr/bin/python
import os
import subprocess
import plistlib
import sys
def appScan():
os.system("system_profiler -xml SPApplicationsDataType > apps.xml")
appList = plistlib.readPlist("apps.xml")
sys.stdout.write( "Mac_App_List\n"
"Delimiters=\"^\"\n"
"string50 string50\n"
"Name^Version\n")
appDict = appList[0]['_items']
for x in appDict:
if 'version' in x:
print x['_name'] + "^" + x['version'] + "^"
else:
print x['_name'] + "^" + "no version found" + "^"
proc = subprocess.Popen(["/opt/altiris/notification/inventory/lib/helpers/aex- sendcustominv","-t","-"], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
proc.communicate(input=appScan())
For some reason this subprocess I am calling doesn't like what is coming into stdin. However if I remove the subprocess items and just have the script print to stdout and then call the script from the terminal (python appScan.py | aex-sendcustominv), aex-sendcustominv is able to accept the input just fine. Is there any way to take a functions output in python and send it to the stdin of an subprocess?

The problem is that appScan() only prints to stdout; appScan() returns None, so proc.communicate(input=appScan()) is equivalent to proc.communicate(input=None). You need appScan to return a string.
Try this (not tested):
def appScan():
os.system("system_profiler -xml SPApplicationsDataType > apps.xml")
appList = plistlib.readPlist("apps.xml")
output_str = 'Delimiters="^"\nstring50 string50\nName^Version\n'
appDict = appList[0]['_items']
for x in appDict:
if 'version' in x:
output_str = output_str + x['_name'] + "^" + x['version'] + "^"
else:
output_str = output_str + x['_name'] + "^" + "no version found" + "^"
return output_str
proc = subprocess.Popen(["/opt/altiris/notification/inventory/lib/helpers/aex- sendcustominv","-t","-"], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
proc.communicate(input=appScan())