Python Multithreaded Messenger Simulation. Stuck on timerThread update. What do? - python

I have a piece of code that simulates a system of messengers (think post office or courier service) delivering letters in a multithreaded way. I want to add a way to manage my messengers "in the field" to increase the efficiency of my system.
tl;dr: How do I update my tens-to-hundreds of timerthreads so they wait longer before calling their function?
Here's what the code I've written so far is supposed to do in steps.
Someone asks for a letter
We check to see if there are any available messengers. If none, we say "oops, sorry. can't help you with that"
If at least one is available, we send the messenger to deliver the letter (new timer thread with its wait param as the time it takes to get there and back)
When the messenger gets back, we put him in the back of the line of available messengers to wait for the next delivery
I do this by removing Messenger objects from a double ended queue, and then adding them back in after a timerthread is done waiting. This is because my Messengers are all unique and eventually I want to track how many deliveries each has had, how far they have traveled, and other stuff.
Here's a pseudoish-codesnippet of the larger program I wrote for this
numMessengers=5
messengerDeque=deque()
pOrder=0.0001
class Messenger:
def __init__(self):
for i in range(numMessengers):
messenger=Messenger()
messengerDeque.append(messenger)
def popDeque():
messenger=idleDeque.popleft()
print 'messenger #?, sent'
return messenger
def appendDeque(messenger):
print 'messenger #?, returned'
messengerDeque.append(messenger)
def randomDelivery():
if numpy.random.randint(0,10000)<=(pOrder*10000):
if len(messengerDeque)!=0:
messenger=popDeque()
tripTime=distance/speed*120
t=threading.Timer(tripTime,appendDeque,args=[messenger])
t.start()
else:
print "oops, sorry. can't help you with that"
The above works in my program.
What I would like to add is some way to 'reroute' my messengers with new orders.
Lets say you have to deliver a letter within an hour of when you get it. You have five messengers and five orders, so they're all busy. You then get a sixth order.
Messenger 2 will be back in 20 minutes, and order six will take 30 minutes to get to the delivery destination. So instead of saying "oops, we can't help you". We would say, ok, Messenger 2, when you get back, immediately go deliver letter six.
With the code I've written, I think this could be done by checking the active threads to see how long until they call their functions, pick the first one you see where that time + how long your new delivery takes is < 1 hr, cancel it, and start a new thread with the time left plus the new time to wait.
I just don't know how to do that.
How do you check how long is left in a timerthread and update it without making a huge mess of your threads?
I'm also open to other, smarter ways of doing what I described.
YAY PYTHON MULTITHREADING!!!!!
Thanks for the help

Using the class threading.Timer wont fulfill your needs. Although there is a "interval" member in Timer instances, once the Timer(thread) started running any changes in interval (time-out) are not considered.
Furthermore you need to know how much time is still left for the timer to be triggered, for which there isn't a method as far as I know.
Furthermore you probably also need a way to identify which Timer instance you need to update with the new timeout value, but this is up-to you.
You should implement your own Timer class, perhaps something along the lines of:
import threading
import time
class MyTimer(threading.Thread):
def __init__(self, timeout, event):
super(MyTimer, self).__init__()
self.to = timeout
self.evt = event
def setTimeout(self, v):
self.end = time.time() + v
def run(self):
self.start = time.time()
self.end = time.time() + self.to
while self.end > time.time():
time.sleep(0) # instead of thread.yield
self.evt()
def getRemaining(self):
return self.end - time.time()
def hi(): print "hi"
T=MyTimer(20,hi)
T.start()
for i in range(10):
time.sleep(1)
# isAlive gives you True if the thread is running
print T.getRemaining(), T.isAlive()
T.setTimeout(1)
for i in range(3):
time.sleep(1)
print T.getRemaining(), T.isAlive()

Related

Add time delay between every statement of python code

Is there an easy way to execute time delay (like time.sleep(3)) between every statement of Python code without having to explicitly write between every statement?
Like in the below Python Script which performs certain action on SAP GUI window. Sometimes, the script continues to the next statement before the previous statement is complete. So, I had to add a time delay between every statement so that it executes correctly. It is working with time delay, but I end up adding time.sleep(3) between every line. Just wondering if there is a better way?
import win32com.client
import time
sapgui = win32com.client.GetObject("SAPGUI").GetScriptingEngine
session = sapgui.FindById("ses[0]")
def add_record(employee_num, start_date, comp_code):
try:
time.sleep(3)
session.findById("wnd[0]/tbar[0]/okcd").text = "/npa40"
time.sleep(3)
session.findById("wnd[0]").sendVKey(0)
time.sleep(3)
session.findById("wnd[0]/usr/ctxtRP50G-PERNR").text = employee_num
time.sleep(3)
session.findById("wnd[0]").sendVKey(0)
time.sleep(3)
session.findById("wnd[0]/usr/ctxtRP50G-EINDA").text = start_date
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT/ctxtRP50G-WERKS[1,0]").text = comp_code
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT/ctxtRP50G-PERSG[2,0]").text = "1"
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT/ctxtRP50G-PERSK[3,0]").text = "U1"
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT").getAbsoluteRow(0).selected = True
time.sleep(3)
return "Pass"
except:
return "failed"
The right way to do what you asked for is almost certainly to use the debugger, pdb.
The right way to do what you want is probably something completely different: find some signal that tells you that the step is done, and wait for that signal. With problems like this, almost any time you pick will be way, way too long 99% of the time, but still too short 1% of the time. That signal may be joining a thread, or waiting on a (threading or multiprocessing) Condition, or getting from a queue, or awaiting a coroutine or future, or setting the sync flag on an AppleEvent, or… It really depends on what you're doing.
But if you really want to do this, you can use settrace:
def sleeper(frame, event, arg):
if event == 'line':
time.sleep(2)
return sleeper
sys.settrace(sleeper)
One small problem is that the notion of line used by the interpreter may well not be what you want. Briefly, a 'line' trace event is triggered whenever the ceval loop jumps to a different lnotab entry (see lnotab_notes.txt in the source to understand what that means—and you'll probably need at least a passing understanding of how bytecode is interpreted, at least from reading over the dis docs, to understand that). So, for example, a multiline expression is a single line; the line of a with statement may appear twice, etc.1
And there's probably an even bigger problem.
Sometimes, the script continues to next step before the previous step is fully complete.
I don't know what those steps are, but if you put the whole thread to sleep for 2 seconds, there's a good chance the step you're waiting for won't make any progress, because the thread is asleep. (For example, you're not looping through any async or GUI event loops, because you're doing nothing at all.) If so, then after 2 seconds, it'll still be just as incomplete as it was before, and you'll have wasted 2 seconds for nothing.
1. If your notion of "line" is closer to what's described in the reference docs on lexing and parsing Python, you could create an import hook that walks the AST and adds an expression statement with a Call to time.sleep(2) after each list element in each body with a module, definition, or compound statement (and then compiles and execs the result as usual).
Anything you want to happen in a program has to be explicitly stated - this is the nature of programming. This is like asking if you can print hello world without calling print("hello world").
I think the best advice to give you here is: don't think in terms of "lines", but think in term of functions.
use debugging mode and watch each and every line executing line by line.

Python: display dynamic information on user input

I would like to be able to receive command line input from user in a python script, and at the same time display to the user some dynamic information.
The user should be able to enter text, but this should not block the displaying of information.
My goal is to create a game, where I show users a countdown while they still have time to enter an answer.
Is this achievable?
Yeah. To create a countdown in the console, you could do something like this:
from time import sleep
for num in reversed(range(0,11)):
print(num)
sleep(1.0)
or
from time import sleep
time = 10
while time != 0:
print(time)
time = time - 1
sleep(1.0)
Either will countdown from 10 to 0 with a second in between each number. Since you might want the user to be able to enter answers as quickly or slowly as like before reaching 0... you might want to look into running two loops concurrently. this thread might be helpful for that. You'll want to figure out how to break out of both loops if the user gets the right answer (and have something come up that says they got the right answer) or if the user runs out of time.
Well sounded like an interesting thing to look into so I did, ran into a few problems pretty soon.
First, I was able to make a running counter but the problem is, since it is a loop, the next layer the counter loop will reset everything you've answered on the previous layer unless you've pressed enter to input answer(answer + enter , during that 1 second period).
if you are making reflex based thing that you only need to press single keys you might be able to succeed with my code by using module getch.
There were few modules that I could use for making the timer and program run at the same time, threading and multiprocessing(better results with threading).
It's basically a working thing if you just remove the counter function which adds the loop, but you won't see the timer.
Pygame might be a good thing to do this with, haven't used it myself but I'd presume so.
here is my code
import time
import threading
import os
timel = 5
def question():
q = input("",)
print(q)
if q == "2":
print("\nCorrect")
else:
exit("\nFalse, You lose!!")
def counter():
timel = 5
for i in range(0, timel):
print("How much is 1+1?", timel)
timel -= 1
time.sleep(1)
os.system("cls")
def timer(seconds):
time.sleep(seconds)
exit("\nTIMES UP, You lose!!")
def exit(msg):
print(msg)
os._exit(1)
def main():
thread = threading.Thread(target=timer, args=(int("{}".format(timel)),))
thread2 = threading.Thread(target=counter)
thread.start()
thread2.start()
question()
if __name__ == "__main__":
main()

Python 3 (Bot) script stops working

I'm trying to connect to a TeamSpeak server using the QueryServer to make a bot. I've taken advice from this thread, however I still need help.
This is The TeamSpeak API that I'm using.
Before the edits, this was the summary of what actually happened in my script (1 connection):
It connects.
It checks for channel ID (and it's own client ID)
It joins the channel and starts reading everything
If someone says an specific command, it executes the command and then it disconnects.
How can I make it so it doesn't disconnect? How can I make the script stay in a "waiting" state so it can keep reading after the command is executed?
I am using Python 3.4.1.
I tried learning Threading but either I'm dumb or it doesn't work the way I thought it would. There's another "bug", once waiting for events, if I don't trigger anything with a command, it disconnects after 60 seconds.
#Librerias
import ts3
import threading
import datetime
from random import choice, sample
# Data needed #
USER = "thisisafakename"
PASS = "something"
HOST = "111.111.111.111"
PORT = 10011
SID = 1
class BotPrincipal:
def __init__(self, manejador=False):
self.ts3conn = ts3.query.TS3Connection(HOST, PORT)
self.ts3conn.login(client_login_name=USER, client_login_password=PASS)
self.ts3conn.use(sid=SID)
channelToJoin = Bot.GettingChannelID("TestingBot")
try: #Login with a client that is ok
self.ts3conn.clientupdate(client_nickname="The Reader Bot")
self.MyData = self.GettingMyData()
self.MoveUserToChannel(ChannelToJoin, Bot.MyData["client_id"])
self.suscribirEvento("textchannel", ChannelToJoin)
self.ts3conn.on_event = self.manejadorDeEventos
self.ts3conn.recv_in_thread()
except ts3.query.TS3QueryError: #Name already exists, 2nd client connect with this info
self.ts3conn.clientupdate(client_nickname="The Writer Bot")
self.MyData = self.GettingMyData()
self.MoveUserToChannel(ChannelToJoin, Bot.MyData["client_id"])
def __del__(self):
self.ts3conn.close()
def GettingMyData(self):
respuesta = self.ts3conn.whoami()
return respuesta.parsed[0]
def GettingChannelID(self, nombre):
respuesta = self.ts3conn.channelfind(pattern=ts3.escape.TS3Escape.unescape(nombre))
return respuesta.parsed[0]["cid"]
def MoveUserToChannel(self, idCanal, idUsuario, passCanal=None):
self.ts3conn.clientmove(cid=idCanal, clid=idUsuario, cpw=passCanal)
def suscribirEvento(self, tipoEvento, idCanal):
self.ts3conn.servernotifyregister(event=tipoEvento, id_=idCanal)
def SendTextToChannel(self, idCanal, mensajito="Error"):
self.ts3conn.sendtextmessage(targetmode=2, target=idCanal, msg=mensajito) #This works
print("test") #PROBLEM HERE This doesn't work. Why? the line above did work
def manejadorDeEventos(sender, event):
message = event.parsed[0]['msg']
if "test" in message: #This works
Bot.SendTextToChannel(ChannelToJoin, "This is a test") #This works
if __name__ == "__main__":
Bot = BotPrincipal()
threadprincipal = threading.Thread(target=Bot.__init__)
threadprincipal.start()
Prior to using 2 bots, I tested to launch the SendTextToChannel when it connects and it works perfectly, allowing me to do anything that I want after it sends the text to the channel. The bug that made entire python code stop only happens if it's triggered by the manejadorDeEventos
Edit 1 - Experimenting with threading.
I messed it up big time with threading, getting to the result where 2 clients connect at same time. Somehow i think 1 of them is reading the events and the other one is answering. The script doesn't close itself anymore and that's a win, but having a clone connection doesn't looks good.
Edit 2 - Updated code and actual state of the problem.
I managed to make the double connection works more or less "fine", but it disconnects if nothing happens in the room for 60 seconds. Tried using Threading.timer but I'm unable to make it works. The entire question code has been updated for it.
I would like an answer that helps me to do both reading from the channel and answering to it without the need of connect a second bot for it (like it's actually doing...) And I would give extra points if the answer also helps me to understand an easy way to make a query to the server each 50 seconds so it doesn't disconnects.
From looking at the source, recv_in_thread doesn't create a thread that loops around receiving messages until quit time, it creates a thread that receives a single message and then exits:
def recv_in_thread(self):
"""
Calls :meth:`recv` in a thread. This is useful,
if you used ``servernotifyregister`` and you expect to receive events.
"""
thread = threading.Thread(target=self.recv, args=(True,))
thread.start()
return None
That implies that you have to repeatedly call recv_in_thread, not just call it once.
I'm not sure exactly where to do so from reading the docs, but presumably it's at the end of whatever callback gets triggered by a received event; I think that's your manejadorDeEventos method? (Or maybe it's something related to the servernotifyregister method? I'm not sure what servernotifyregister is for and what on_event is for…)
That manejadorDeEventos brings up two side points:
You've declared manejadorDeEventos wrong. Every method has to take self as its first parameter. When you pass a bound method, like self.manejadorDeEventos, that bound self object is going to be passed as the first argument, before any arguments that the caller passes. (There are exceptions to this for classmethods and staticmethods, but those don't apply here.) Also, within that method, you should almost certainly be accessing self, not a global variable Bot that happens to be the same object as self.
If manejadorDeEventos is actually the callback for recv_in_thread, you've got a race condition here: if the first message comes in before your main threads finishes the on_event assignment, the recv_on_thread won't be able to call your event handler. (This is exactly the kind of bug that often shows up one time in a million, making it a huge pain to debug when you discover it months after deploying or publishing your code.) So, reverse those two lines.
One last thing: a brief glimpse at this library's code is a bit worrisome. It doesn't look like it's written by someone who really knows what they're doing. The method I copied above only has 3 lines of code, but it includes a useless return None and a leaked Thread that can never be joined, not to mention that the whole design of making you call this method (and spawn a new thread) after each event received is weird, and even more so given that it's not really explained. If this is the standard client library for a service you have to use, then you really don't have much choice in the matter, but if it's not, I'd consider looking for a different library.

Accessing pexpect's internal "countdown" remainder?

I am wondering if it is possible to get a script to grab the "remaining time" in the "timeout counter" or whatever is used internally by pexpect:
index, match_obj, text = session.expect(["New Software Release is ready -- Version"], timeout=int(DOWNLOAD_TIMEOUT))
if match_obj:
logger.info('Download to primary card complete.')
logger.debug('Download took %d seconds', SOMETHINGGOESHERE)
else:
logger.critical('Download to primary card took too long.')
quit(session)
For the life of me, I can't seem to find any attempt to do this anywhere here or via Google :)
I assume I can do something more complicated where I keep track of my own timer, but I was hoping for a simpler solution, given that pexpect is already keeping track of a time value, in seconds.
pexpect source code shows that the remaining time is kept in a local variable therefore it is not accessible outside of the function after it returns.
It is not complicated to keep track of the remaining time yourself:
end_time = time.time() + timeout
index = child.expect([..], timeout=timeout)
remaining_time = end_time - time.time()
If you need to do it multiple times, you could subclass pexpect.spawn class and override expect() method to return the remaining time together with the index.

python and Cocoa: about statusbar script

I am creating a little script which check the number of mail in my gmail account and print them in the
status bar. The function gmail() returns the number of new emails. I have few questions, but first this is the code I wrote so far (clearly I am a novice):
class MyApplicationAppDelegate(NSObject):
var = 1
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
global ngmail
self.statusItem = NSStatusBar.systemStatusBar().statusItemWithLength_(NSVariableStatusItemLength)
while var == 1 :
ngmail2 = gmail();
if ngmail2 !=ngmail:
self.statusItem.setTitle_("loading")
self.statusItem.setTitle_(ngmail2)
ngmail = ngmail2
time.sleep(6)
1) Why do I need the line "self.statusItem.setTitle_("loading")" ? without that line it wouldn't update itself. I really do not know why.
2) it runs as it should, but whenever I get close to the number in the status bar, the spinning wheel appear.
I guess the reason is because I am using while, and instead I should be using something like nsrunloop or something like that. Can anyone advice on this?
3) If I put my mac to sleep and I wake it up, the script stops working. Any solution? maybe this is related to question 2) above.
Thanks!
All of your problems come from the fact that you're blocking the main thread.
In Cocoa, or almost any other GUI framework, the main thread runs a loop that waits for the next event, calls an event handler, and repeats until quit.
Your event handler, applicationDidFinishLaunching_, never returns. This means Cocoa can never handle the next event. Eventually, the OS will notice that you're not responding and put up the beachball.
With Cocoa, sometimes it sneaks in some other events each time you give it a chance, like on the setTitle_ calls, and there are some things the OS can fake even if you're not responding, like keeping the window redrawing, so it isn't always obvious that your app is not responsive. But that doesn't mean you don't need to solve the problem.
There are a number ways to do this, but is the easiest is probably to use a background thread. Then, applicationDidFinishLaunching_ can kick off the background thread and then return immediately, allowing the main thread to get back to its job handling events.
The only tricky bit is that code running on background threads can't make calls to UI objects. So, what do you do about that?
That's what performSelectorOnMainThread_withObject_waitUntilDone_ is for.
Here's an example:
class MyApplicationAppDelegate(NSObject):
var = 1
def background_work(self):
global ngmail
while var == 1 :
ngmail2 = gmail();
if ngmail2 !=ngmail:
self.statusItem.setTitle_("loading")
self.statusItem.performSelectorOnMainThread_withObject_waitUntilDone_('setTitle:', ngmail2, False)
time.sleep(6)
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
self.statusItem = NSStatusBar.systemStatusBar().statusItemWithLength_(NSVariableStatusItemLength)
self.background_worker = threading.Thread(target=self.background_work)
self.background_worker.start()
The only tricky bit is that you have to use the ObjC name for the selector (setTitle:), not the Python name (setTitle_).
However, your code has another subtle bug: var isn't actually synchronized, so it's possible for you to change its value in the main thread, without the background thread ever noticing.
On top of that, doing a sleep(6) means that it will take up to 6 seconds to quit your app, because the background thread won't get to the code that checks var until it finishes sleeping.
You can fix both of these by using a Condition.
class MyApplicationAppDelegate(NSObject):
var = 1
condition = threading.Condition()
def background_work(self):
global ngmail
with condition:
while var == 1:
ngmail2 = gmail();
if ngmail2 != ngmail:
self.statusItem.performSelectorOnMainThread_withObject_waitUntilDone_('setTitle:', ngmail2, False)
condition.wait(6)
#classmethod
def shutdown_background_threads(cls):
with condition:
var = 0
condition.notify_all()
(I assume you used a class attribute for var instead of an instance attribute on purpose, so I likewise made the condition a class attribute and the shutdown method a class method.)

Categories

Resources