Develop Reference

Python output from Popen command from inside a thread into a GUI form in real time

Being new to Python I have looked around the site and found partial answers but nothing that helps make things clear. This is what I have. The main window through a button activates the thread which runs a command (wash -C -i monX) and a table needs to populate with the results in the GUI in real time. I found the code for the loop here Intercepting stdout of a subprocess while it is running.
the code is here https://github.com/theodhori-dhiamanti/wifern/blob/master
the code in question is:
class WashThread(QtCore.QThread):
def __init__(self, parent=None):
super(WashThread, self).__init__(parent)
def run(self):
global mon_iface
try:
if mon_iface != '':
device = mon_iface
cmd = ['wash', '-C', '-i', device]
wash_cmd = Popen(cmd, stdout=PIPE)
for line in iter(wash_cmd.stdout.readline, b''):
if line.strip() == '' or line.startswith('---'): continue
if line.startswith('Wash') or line.startswith('Copyright') or line.startswith('BSSID'): continue
print line
Split = line.split(' ')
wash_bssid = Split[0]
wash_essid = Split[58]
wash_power = Split[19]
wash_locked = Split[42]
else:
print('No Good')
except OSError:
pass
and the GUI part that calls this method and where the results need to be used is:
def wash(self):
row = 0
col = 0
self.wash_tableWidget.setColumnCount(4)
self.wash_tableWidget.setColumnWidth(1,150)
self.wash_tableWidget.setColumnWidth(4,30)
self.wash_tableWidget.setColumnWidth(3,70)
self.wash_tableWidget.setRowCount(10)
if self.start_wash_Button.text() == 'Start':
self.start_wash_Button.setText('Stop')
self.wash_thread.start()
row_item = QtGui.QTableWidgetItem(wash_bssid)
x = QtGui.QTableWidgetItem(wash_essid)
y = QtGui.QTableWidgetItem(wash_power)
z = QtGui.QTableWidgetItem(wash_locked)
self.wash_tableWidget.setItem(row, col, row_item)
self.wash_tableWidget.setItem(row, 1, x)
self.wash_tableWidget.setItem(row, 2, y)
self.wash_tableWidget.setItem(row, 3, z)
row += 1
else:
try:
os.kill(cmd.pid, SIGTERM)
print('Done')
except OSError:
pass
except UnboundLocalError:
pass
self.wash_thread = WashThread() # method initialized in main body
How can I transfer the output from the thread to the main portion?
After I than can assign the values appropriately to the table fields?
Any help is greatly appreciated.
* Note: out of the thread the method works as intended.

Pytonic way of passing values between process

I need a simple way to pass the stdout of a subprocess as a list to another function using multiprocess:
The first function that invokes subprocess:
def beginRecvTest():
command = ["receivetest","-f=/dev/pcan33"]
incoming = Popen(command, stdout = PIPE)
processing = iter(incoming.stdout.readline, "")
lines = list(processing)
return lines
The function that should receive lines:
def readByLine(lines):
i = 0
while (i < len(lines)):
system("clear")
if(lines[i][0].isdigit()):
line = lines[i].split()
dictAdd(line)
else:
next
print ; print "-" *80
for _i in mydict.keys():
printMsg(mydict, _i)
print "Keys: ", ; print mydict.keys()
print ; print "-" *80
sleep(0.3)
i += 1
and the main from my program:
if __name__ == "__main__":
dataStream = beginRecvTest()
p = Process(target=dataStream)
reader = Process(target=readByLine, args=(dataStream,))
p.start()
reader.start()
I've read up on using queues, but I don't think that's exactly what I need.
The subprocess called returns infinite data so some people have suggested using tempfile, but I am totally confused about how to do this.
At the moment the script only returns the first line read, and all efforts on looping the beginRecvTest() function have ended in compilation errors.

Python read console input using select on the same line

I need to use a non blocking way to read from the console. I successfully managed to do this using select
ready = select.select(read_list, [], [], timeout)[0]
read_list = [sys.stdin]
timeout = 0.1 # seconds
My problem now is that I need to provide a text line (eg UI) before the input and would like the input cursor to be on the same line. Before when I was not using select I could achieve this by doing:
buff = raw_input(' ENTER CODE: ------\b\b\b\b\b\b')
In this way the cursor would have been just after the comma (eg. on the first '-')
Now that I need to use stdin, the cursor always goes at the beginning of a new line. Even if I do:
print(' ENTER CODE: ------\b\b\b\b\b\b\r')
while read_list:
ready = select.select(read_list, [], [], timeout)[0]
if not ready:
idle_work()
else:
for file in ready:
line = file.readline()
if not line: # EOF, remove file from input list
read_list.remove(file)
elif line.rstrip(): # optional: skipping empty lines
#treat_input(line)
buff =line.upper()
ETC...
Any ideas ?

I resolved this appending a ' at the end of the print statement and then flushing the std:
print(' ENTER CODE: ------\b\b\b\b\b\b'),
sys.stdout.flush()

Get output from a program with the user input that program takes

I'm trying to capture a string from the output of a subprocess and when the subprocess asks for user input, include the user input in the string, but I can't get stdout to work.
I got the string output from stdout using a while loop, but I don't know how to terminate it after reading the string.
I tried using subprocess.check_output, but then I can't see the prompts for user input.
import subprocess
import sys
child = subprocess.Popen(["java","findTheAverage"], stdout = subprocess.PIPE, stdin = subprocess.PIPE )
string = u""
while True:
line = str(child.stdout.read(1))
if line != '':
string += line[2]
print(string)
else:
break
print(string)
for line in sys.stdin:
print(line)
child.stdin.write(bytes(line, 'utf-8'))
EDIT:
With help and code from Alfe post I now have a string getting created from the subprocess programs output, and the users input to that program, but its jumbled about.
The string appears to first get The first letter of the output, then the user input, then the rest of the output.
Example of string muddling:
U2
3ser! please enter a double:U
4ser! please enter another double: U
5ser! please enter one final double: Your numbers were:
a = 2.0
b = 3.0
c = 4.0
average = 3.0
Is meant to be:
User! please enter a double:2
User! please enter another double: 3
User! please enter one final double: 4
Your numbers were:
a = 2.0
b = 3.0
c = 4.0
average = 3.0
Using the code:
import subprocess
import sys
import signal
import select
def signal_handler(signum, frame):
raise Exception("Timed out!")
child = subprocess.Popen(["java","findTheAverage"], universal_newlines = True, stdout = subprocess.PIPE, stdin = subprocess.PIPE )
string = u""
stringbuf = ""
while True:
print(child.poll())
if child.poll() != None and not stringbuf:
break
signal.signal(signal.SIGALRM, signal_handler)
signal.alarm(1)
try:
r, w, e = select.select([ child.stdout, sys.stdin ], [], [])
if child.stdout in r:
stringbuf = child.stdout.read(1)
string += stringbuf
print(stringbuf)
except:
print(string)
print(stringbuf)
if sys.stdin in r:
typed = sys.stdin.read(1)
child.stdin.write(typed)
string += typed
FINAL EDIT:
Alright, I played around with it and got it working with this code:
import subprocess
import sys
import select
import fcntl
import os
# the string that we will return filled with tasty program output and user input #
string = ""
# the subprocess running the program #
child = subprocess.Popen(["java","findTheAverage"],bufsize = 0, universal_newlines = True, stdout = subprocess.PIPE, stdin = subprocess.PIPE )
# stuff to stop IO blocks in child.stdout and sys.stdin ## (I stole if from http://stackoverflow.com/a/8980466/2674170)
fcntl.fcntl(child.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)
# this here in the unlikely event that the program has #
# finished by the time the main loop is first running #
# because if that happened the loop would end without #
# having added the programs output to the string! #
progout = ""
typedbuf = "#"
### here we have the main loop, this friendly fellah is
### going to read from the program and user, and tell
### each other what needs to be known
while True:
## stop when the program finishes and there is no more output
if child.poll() != None and not progout:
break
# read from
typed = ""
while typedbuf:
try:
typedbuf = sys.stdin.read(1)
except:
break
typed += typedbuf
stringbuf = "#"
string += typed
child.stdin.write(typed)
progout = ""
progoutbuf = "#"
while progoutbuf:
try:
progoutbuf = child.stdout.read(1)
except:
typedbuf = "#"
break
progout += progoutbuf
if progout:
print(progout)
string += progout
# the final output string #
print( string)

You need select to read from more than one source at the same time (in your case stdin and the output of the child process).
import select
string = ''
while True:
r, w, e = select.select([ child.stdout, sys.stdin ], [], [])
if child.stdout in r:
string += child.stdout.read()
if sys.stdin in r:
typed = sys.stdin.read()
child.stdin.write(typed)
string += typed
You will still need to find a proper breaking condition to leave that loop. But you probably get the idea already.
I want to give a warning at this point: Processes writing into pipes typically buffer until the latest possible moment; you might not expect this because when testing the same program from the command line (in a terminal) typically only lines get buffered. This is due to performance considerations. When writing to a terminal, typically a user expects to see the output as soon as possible. When writing to a pipe, typically a reading process is happy to be given larger chunks in order to sleep longer before they arrive.

Get active window title in X

I'm trying to get the title of the active window. The application is a background task so if the user has Eclipse open the function returns "Eclipse - blabla", so it's not getting the window title of my own window. I'm developing this in Python 2.6 using PyQt4.
My current solution, borrowed and slightly modified from an old answer here at SO, looks like this:
def get_active_window_title():
title = ''
root_check = ''
root = Popen(['xprop', '-root'], stdout=PIPE)
if root.stdout != root_check:
root_check = root.stdout
for i in root.stdout:
if '_NET_ACTIVE_WINDOW(WINDOW):' in i:
id_ = i.split()[4]
id_w = Popen(['xprop', '-id', id_], stdout=PIPE)
for j in id_w.stdout:
if 'WM_ICON_NAME(STRING)' in j:
if title != j.split()[2]:
return j.split("= ")[1].strip(' \n\"')
It works for most windows, but not all. For example it can't find my kopete chat windows, or the name of the application i'm currently developing.
My next try looks like this:
def get_active_window_title(self):
screen = wnck.screen_get_default()
if screen == None:
return "Could not get screen"
window = screen.get_active_window()
if window == None:
return "Could not get window"
title = window.get_name()
return title;
But for some reason window is always None.
Does somebody have a better way of getting the current window title, or how to modify one of my ways, that works for all windows?
Edit:
In case anybody is wondering this is the way I found that seems to work for all windows.
def get_active_window_title(self):
root_check = ''
root = Popen(['xprop', '-root'], stdout=PIPE)
if root.stdout != root_check:
root_check = root.stdout
for i in root.stdout:
if '_NET_ACTIVE_WINDOW(WINDOW):' in i:
id_ = i.split()[4]
id_w = Popen(['xprop', '-id', id_], stdout=PIPE)
id_w.wait()
buff = []
for j in id_w.stdout:
buff.append(j)
for line in buff:
match = re.match("WM_NAME\((?P<type>.+)\) = (?P<name>.+)", line)
if match != None:
type = match.group("type")
if type == "STRING" or type == "COMPOUND_TEXT":
return match.group("name")
return "Active window not found"

xdotool can do that.
xdotool getactivewindow

I modified your solution slightly so it should run more efficiently (it passes parameters to xprop so only the data it needs is returned). Also, I'm not sure it's necessary to buffer the output of xprop so I took that out. It should also correct return "Active window not found" if for some reason it can't find the active window.
def get_active_window_title(self):
root = Popen(['xprop', '-root', '_NET_ACTIVE_WINDOW'], stdout=PIPE)
for line in root.stdout:
m = re.search('^_NET_ACTIVE_WINDOW.* ([\w]+)$', line)
if m != None:
id_ = m.group(1)
id_w = Popen(['xprop', '-id', id_, 'WM_NAME'], stdout=PIPE)
break
if id_w != None:
for line in id_w.stdout:
match = re.match("WM_NAME\(\w+\) = (?P<name>.+)$", line)
if match != None:
return match.group("name")
return "Active window not found"

You can get the active window title with xdotool:
$ xdotool getactivewindow getwindowname

This is too late to be useful but it does work and I have programs that use wnck.
The wnck example needs a call to screen.force_update() before wnck will work. Without that wnck does not have any information about the windows on the screen. That is:
def get_active_window_title(self):
screen = wnck.screen_get_default()
if screen is None:
return "Could not get screen"
screen.force_update()
window = screen.get_active_window()
if window is None:
return "Could not get window"
title = window.get_name()
return title

I see that the question is a bit dusty by now, also support for Python 2 is nearing the end of its scheduled lifetime, so I thought I'd mention a more Python 3'ic version.
In fact, it's probably better than just more 3-style - in Python 3 "Popen objects are supported as context managers via the with statement: on exit, standard file descriptors are closed, and the process is waited for".
Thus the below is probably more adequate and less resource hungry for newer Pythons:
(also, without withs, you might bump into 'too many open files' problems - which I of course found out about in the hard way :) - should you query for the window title frequently enough on Ubuntu ~16 .)
with Popen(['xprop', '-root', '_NET_ACTIVE_WINDOW'], stdout=PIPE) as root:
for line in root.stdout:
line = str(line, encoding="UTF-8")
m = re.search('^_NET_ACTIVE_WINDOW.* ([\w]+)$', line)
if m is not None:
id_ = m.group(1)
with Popen(['xprop', '-id', id_, 'WM_NAME'],
stdout=PIPE) as id_w:
for line in id_w.stdout:
line = str(line, encoding="UTF-8")
match = re.match("WM_NAME\(\w+\) = \"(?P<name>.+)\"$",
line)
if match is not None:
return match.group("name")
break
return "Active window not found"