Develop Reference

Queue empty for some reason, even with wrapper function to put arguments in

I have a project where I'm having a device collect information from a scale once a mixing process is done, and I'm having some trouble with a threading process. This code has the function run while a window is open telling the person using the device what is happening:
def recipe_info_gather():
fourth_win.destroy()
time=int(2.5*1000)+500*process.get_number_of_components()
print(time)
global recipe_window
recipe_window=Tk()
recipe_window.title("Getting Recipe Info")
rec_inter=window(recipe_window)
rec_inter.my_canvas.create_text(rec_inter.width/2,rec_inter.height/2, text="Please wait as the pi gets the recipe information from the scale",font=("Arial", 20), fill='white')
press_OK(ser)
t=threading.Thread(target=get_recipe_info, args=(ser,))
t.start()
recipe_window.after(time,finish_or_continue)
recipe_window.mainloop()
and this is the function that the thread is calling
def store_in_queue(func):
#this is solely for the get_recipe_info function for running while window is open
def wrapper(*args):
my_queue.put(func(*args))
return wrapper
#store_in_queue
def get_recipe_info(ser):
from get_name import process
r={}
r['name']=(get_stable_weight(ser)).decode("utf-8")
time.sleep(0.5)
r['total']=(get_stable_weight(ser)).decode("utf-8")[7:14]
time.sleep(0.5)
r['fact']=(get_stable_weight(ser)).decode("utf-8")
time.sleep(0.5)
r['recalcs']=(get_stable_weight(ser)).decode("utf-8")
time.sleep(0.5)
r['ignores']=(get_stable_weight(ser)).decode("utf-8")
time.sleep(0.5)
r['status']=(get_stable_weight(ser)).decode("utf-8")
time.sleep(0.5)
# for i in range(1,process.get_number_of_components()+1):
# r['wgt{}'.format(str(i))]=(get_stable_weight(ser))[6:12].decode("utf-8")
# if i!=process.get_number_of_components():
# time.sleep(0.5)
r['wgt1']=(get_stable_weight(ser))[6:12].decode("utf-8")
time.sleep(0.5)
r['wgt2']=(get_stable_weight(ser))[6:12].decode("utf-8")
time.sleep(0.5)
r['wgt3']=(get_stable_weight(ser))[6:12].decode("utf-8")
for i in range(1,4):
print(r['wgt{}'.format(str(i))])
return r
I also have a line my_queue=queue.Queue() to store the recipe information in, and I have another function that's supposed to extract the resuting dictionary from this variable after the recipe_info_gather() function is done executing with this line: recipe=my_queue.get_nowait(). I had this working before, but now all of a sudden, when I execute this line, I get an error saying the queue is empty even though I had the wrapper function to put arguments in it. It also looks like the get_recipe_info() function actually executes after the get_nowait() line for some reason, so it looks like the thread isn't running concurrently while that window is open.
I was wondering if anyone had intuition as to why this is happening and how to fix it?
Edit: I got a little bit further and found out that wrapper is being returned as None, but even then, I'm not sure why that's happening, so some input would be good.

Python two things at the same time with threading

I'm creating a small program, just for fun, that opens programs and do things like that.
I'm doing a Wikipedia search and the program will read it out and I want the text to be written out to the screen by MessageBoxW.
I want this two thing to happen at the same, because now it frist showing the message box and after I closed the window then it's reading up the text
def Mbox(title, text, style):
return ctypes.windll.user32.MessageBoxW(0, text, title, style)
def Mbox(title, text, style):
return ctypes.windll.user32.MessageBoxW(0, text, title, style)
def wikipediaSearch():
global user_input
user_input = user_input.replace("search", '')
result = wikipedia.summary(user_input, sentences=2)
raw_text = result
convert_text = unicodedata.normalize('NFKD', raw_text).encode('ascii', 'ignore')
convert_text = convert_text.decode('utf-8')
new_text = re.sub(r'\(.*\)', '', convert_text)
print(new_text)
Mbox(user_input, new_text, 0)
voice.say(new_text)
voice.runAndWait()

Create a helper class that runs pyttsx in a separate thread:
import threading
import pyttsx
class Say(object):
"""Function-like class to speak using pyttsx."""
_thread = None
def __init__(self, message):
if not isinstance(message, str):
raise ValueError("message is not a string")
if Say._thread is not None:
Say._thread.join()
Say._thread = None
Say._thread = threading.Thread(target=self._worker,
name="Say",
args=(message,))
Say._thread.start()
def _worker(self, message):
engine = pyttsx.init()
engine.say(message)
engine.runAndWait()
def WaitAllSaid():
if Say._thread is not None:
Say._thread.join()
Say._thread = None
Because pyttsx behaves like a singleton, and only one instance of pyttsx can speak in the same Python process at any time, I encapsulated the global variables into the Say class, and have the instance constructor wait for any existing utterances to complete, then start a new thread for pyttsx to speak.
Essentially, Say(message) waits for any utterances in progress to complete, then starts speaking the new voice, and returns. It does not wait for the message to be completely spoken before it returns; it returns immediately when the message begins.
WaitAllSaid() waits for any utterances in progress, then reaps the worker thread. If you want to modify the pyttsx engine or voice properties, call WaitAllSaid() first to make sure no utterances are in progress at that time. Otherwise poor pyttsx might get confused.
The last four lines of OP's wikipediaSearch function now becomes something like
print(new_text)
Say(new_text)
Mbox(user_input, new_text, 0)
WaitAllSaid()
If pyttsx is already speaking, then Say() blocks until all previous messages have been said. It returns immediately when the specified message starts to play.
The WaitAllSaid() just blocks until everything that has been said has been uttered. You can omit it from the wikipediaSearch() function, as long as you make sure that WaitAllSaid() is called before the Python program exits.
On the not-exactly conventional design: On Linux at least, pyttsx has issues if one tries to use the same pyttsx object for separate statements. Having the helper thread create the instance works much better. Testing on Linux, this pattern was the most robust one among global variables and various forms of singleton classes. I do not use Windows at all, so I cannot test on it, alas.

python and Cocoa: about statusbar script

I am creating a little script which check the number of mail in my gmail account and print them in the
status bar. The function gmail() returns the number of new emails. I have few questions, but first this is the code I wrote so far (clearly I am a novice):
class MyApplicationAppDelegate(NSObject):
var = 1
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
global ngmail
self.statusItem = NSStatusBar.systemStatusBar().statusItemWithLength_(NSVariableStatusItemLength)
while var == 1 :
ngmail2 = gmail();
if ngmail2 !=ngmail:
self.statusItem.setTitle_("loading")
self.statusItem.setTitle_(ngmail2)
ngmail = ngmail2
time.sleep(6)
1) Why do I need the line "self.statusItem.setTitle_("loading")" ? without that line it wouldn't update itself. I really do not know why.
2) it runs as it should, but whenever I get close to the number in the status bar, the spinning wheel appear.
I guess the reason is because I am using while, and instead I should be using something like nsrunloop or something like that. Can anyone advice on this?
3) If I put my mac to sleep and I wake it up, the script stops working. Any solution? maybe this is related to question 2) above.
Thanks!

All of your problems come from the fact that you're blocking the main thread.
In Cocoa, or almost any other GUI framework, the main thread runs a loop that waits for the next event, calls an event handler, and repeats until quit.
Your event handler, applicationDidFinishLaunching_, never returns. This means Cocoa can never handle the next event. Eventually, the OS will notice that you're not responding and put up the beachball.
With Cocoa, sometimes it sneaks in some other events each time you give it a chance, like on the setTitle_ calls, and there are some things the OS can fake even if you're not responding, like keeping the window redrawing, so it isn't always obvious that your app is not responsive. But that doesn't mean you don't need to solve the problem.
There are a number ways to do this, but is the easiest is probably to use a background thread. Then, applicationDidFinishLaunching_ can kick off the background thread and then return immediately, allowing the main thread to get back to its job handling events.
The only tricky bit is that code running on background threads can't make calls to UI objects. So, what do you do about that?
That's what performSelectorOnMainThread_withObject_waitUntilDone_ is for.
Here's an example:
class MyApplicationAppDelegate(NSObject):
var = 1
def background_work(self):
global ngmail
while var == 1 :
ngmail2 = gmail();
if ngmail2 !=ngmail:
self.statusItem.setTitle_("loading")
self.statusItem.performSelectorOnMainThread_withObject_waitUntilDone_('setTitle:', ngmail2, False)
time.sleep(6)
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
self.statusItem = NSStatusBar.systemStatusBar().statusItemWithLength_(NSVariableStatusItemLength)
self.background_worker = threading.Thread(target=self.background_work)
self.background_worker.start()
The only tricky bit is that you have to use the ObjC name for the selector (setTitle:), not the Python name (setTitle_).
However, your code has another subtle bug: var isn't actually synchronized, so it's possible for you to change its value in the main thread, without the background thread ever noticing.
On top of that, doing a sleep(6) means that it will take up to 6 seconds to quit your app, because the background thread won't get to the code that checks var until it finishes sleeping.
You can fix both of these by using a Condition.
class MyApplicationAppDelegate(NSObject):
var = 1
condition = threading.Condition()
def background_work(self):
global ngmail
with condition:
while var == 1:
ngmail2 = gmail();
if ngmail2 != ngmail:
self.statusItem.performSelectorOnMainThread_withObject_waitUntilDone_('setTitle:', ngmail2, False)
condition.wait(6)
#classmethod
def shutdown_background_threads(cls):
with condition:
var = 0
condition.notify_all()
(I assume you used a class attribute for var instead of an instance attribute on purpose, so I likewise made the condition a class attribute and the shutdown method a class method.)

How to interrupt a program while it's running?

I wrote a program that reads a text file and runs an .exe for every line in the text file. This results in opening a new command line window for every time i run the .exe. The windows do close on their own once the current task is finished, but the problem is as follows:
If i have 100 lines in the text file, this means that i call the .exe file 100 times. My problem with that is if i want to cancel the run after it already started, i have to click on the red "X" to close every window one after the another.
What i am trying to do is have some sort of a command interrupt the running program and either close all upcoming windows or just stop the for loop from running.
Is it possible to write into the console a command to interrupt the current running code?
Would it be better to use some sort of a key event listener? If so, are there any built-in key listeners in Python? I can't seem to find any. Does that mean that i have to install Pygame just so i can use a key event listener?
Maybe i should try to listen to the command line and detect an exit code on one of the windows that i manually close and that way end the for loop?

There are a few ways you could go about this. But pretty much you have one main issue - you need some sort of flag that can be switched such that the code will know it must stop. For instance, if the code is working in a while-loop, it should check at the start of this loop if the flag is valid, or if the flag is telling the loop to stop...
while flag:
# do code
There are a few ways to implement this flagging like operation for your needs. I will discuss the threading option. First, you need to understand how threading works, and then you need to mold your script such that instead of "running an executable" for each line of the text file, you would read the text file, and put all the lines into a queue, then you would have a few threads that read from that queue, and perform the desired action (like running an executable) but instead of running an external executable, you should mimick this with Python, this thread should be a daemon thread.. and it should have a main loop which checks if a flag that exists in the parent thread is turned on...
Below is an example:
from threading import Thread
from Queue import Queue
import sys
import time
class Performer():
def __init__(self):
self.active = False
self.queue = Queue()
def action(self, line):
pass # your code should be here
def operate(self, text_file, threads=5):
with open(text_file) as f:
for line in f:
self.queue.put(line)
self.active = True
thread_pool = []
for i in range(threads):
t = Thread(target=self.__thread, name=('worker-%d' % i))
t.daemon = True
t.start()
thread_pool.append(t)
while self.active:
try:
if self.queue.empty():
break
except KeyboardInterrupt:
self.active = False
sys.exit('user + keyboard = byebye')
else:
time.sleep(1)
def __thread(self):
while self.active:
if not self.queue.empty():
try:
self.action(self.queue.get())
except Exception:
pass # do something here

What is the use of 'self.update' in Roger Stuckey's wxPython Multiprocessing Example code

I was reading Roger Stuckey's wxPython Multiprocessing code to try to make a similar program myself. Full code can be found here.
The code runs fine without any modification. However, I found a parameter self.update been pass around between the GUI class MyFrame to the processing class TaskSErverMP. I have been searched throughout the entire code snippet and couldn't figure out what it is doing in the code -- it has never been initialized and used anyhow.
In the class MyFrame:
def OnStart(self, event):
...
self.taskserver.processTasks(self.update)
...
def OnStop(self, event):
...
self.taskserver.processStop(self.update)
...
def update(self, output):
"""
Get and print the results from one completed task.
"""
self.output_tc.AppendText('%s [%d] calculate(%d) = %.2f\n'...
...
# Give the user an opportunity to interact
wx.YieldIfNeeded()
In the class TaskServerMP:
def run(self):
...
self.processTasks(self.update)
...
def processTasks(self, resfunc=None):
...
def processStop(self, resfunc=None):
...
def update(self, output):
"""
Get and print the results from one completed task.
"""
sys.stdout.write('%s [%d] calculate(%d) = %.2f' % ....
So I thought that is a dependency injection practice but nothing more. I then removed it from the code and the strangest thing happened -- the program doesn't work anymore! I have the GUI displayed and I was able to start the processing. However, the GUI just hanged and then later Windows reported that the program is not responding. I have end up kill all the pythonw.exe processes manually from the Windows Task Manager.
Then I start to think if there is anything to do with the signature of the functions processTasks and processStop in the class TaskServerMP. But I really have no idea how I can associate the parameter self.update to the optional argument resfunc.
I don't think there is anything wrong in Roger's code. But it bothers me if I cannot twisted around the source to test out my understanding of the code.
I use Python 2.7 in Windows 7.

MyFrame.update is a method. You can see its definition on line 365.
So self.update is a bound method, meaning it can be called as if it were a regular function.
You can see that processTasks takes a resfunc parameter; then, at least 165, if it got a function or method as that resfunc parameter, it calls it.
The idea here is that processTasks leaves it up to the caller to decide how to print out progress updates as each task completes. One class might do it by writing them to stdout. A different class might instead update a GUI progress bar.
This is a pretty typical way to pass callbacks around in Python code.
So, why does the program hang if you take out the self.update? Well, look what's inside it, at line 372:
# Give the user an opportunity to interact
wx.YieldIfNeeded()
In wx, as in most GUI frameworks, the main thread is running an "event loop", something which has to process each event (a mouse move, a keypress, whatever) as it comes in, and then wait for the next one. You write your code as a bunch of event handlers—when someone clicks this button, run that function; etc. Your event handlers have to return quickly. Otherwise, the event loop doesn't get to pick up and dispatch the next event, so your GUI isn't responding. In wx, the Yield family of functions make life easier. As long as you Yield often enough, you don't have to return quickly. But you still have to do one or the other—either return early, or Yield—or the GUI will hang.
Here's a very simple example showing how to use bound methods:
class Foo(object):
def __init__(self, name):
self.name = name
def print_name(self):
print(self.name)
def give_me_a_printer_function(self):
return self.print_name
spam = Foo('Spam')
my_function1 = spam.print_name
my_function2 = spam.give_me_a_printer_function()
my_function1()
my_function2()
This will print Spam twice.
Functions and methods are first class values in Python—you can pass them around just like you can pass around numbers, strings, lists, and class instances. You can even print them out (although you'll get something ugly like <bound method Foo.print_name of <__main__.Foo object at 0x104629190>>).