Develop Reference

trio.Event(): Which is “better”: setting and initializing a new Event or checking if someone is waiting for it beforehand?

import trio
work_available = trio.Event()
async def get_work():
while True:
work = check_for_work()
if not work:
await work_available.wait()
else:
return work
def add_work_to_pile(...):
...
if work_available.statistics().tasks_waiting:
global work_available
work_available.set()
work_available = trio.Event()
In this Python-like code example I get work in bursts via add_work_to_pile(). The workers which get work via get_work() are slow. So most of the time add_work_to_pile() is called there will be no one waiting on work_available.
Which is better/cleaner/simpler/more pythonic/more trionic/more intended by the trio developers?
checking if someone is looking for the Event() via statistics().tasks_waiting, like in the example code, ...or...
unconditionally set() setting the Event() and creating a new one each time? (Most of them in vain.)
Furthermore... the API does not really seem to expect regular code to check if someone is waiting via this statistics() call...
I don’t mind spending a couple more lines to make things clearer. But that goes both ways: a couple CPU cycles more are fine for simpler code...

Creating a new Event is roughly the same cost as creating the _EventStatistics object within the statistics method. You'll need to profile your own code to pick out any small difference in performance. However, although it is safe and performant, the intent of statistics across trio's classes is for debug rather than core logic. Using/discarding many Event instances would be relatively more along the intent of the devs.
A more trionic pattern would be to load each work item into a buffered memory channel in place of your add_work_to_pile() method and then iterate on that in the task that awaits get_work. I feel the amount of code is comparable to your example:
import trio
send_chan, recv_chan = trio.open_memory_channel(float('inf'))
async def task_that_uses_work_items():
# # compare
# while True:
# work = await get_work()
# handle_work(work)
async for work in recv_chan:
handle_work(work)
def add_work_to_pile():
...
for work in new_work_set:
send_chan.send_nowait(work)
# maybe your work is coming in from a thread?
def add_work_from_thread():
...
for work in new_work_set:
trio_token.run_sync_soon(send_chan.send_nowait, work)
Furthermore, it's performant because the work items are efficiently rotated through a deque internally. This code would checkpoint for every work item so you may have to jump through some hoops if you want to avoid that.

I think you might want a trio.ParkingLot. It gives more control over parking (i.e. which is like Event.wait()) and unparking (which is like Event.set() except that it doesn't stop future parkers from waiting). But it doesn't have any notion of being set at all so you would need to store that information separately. If you work is naturally Truety when set (e.g. a non-empty list) then that might be easy anyway. Example:
available_work = []
available_work_pl = trio.ParkingLot()
async def get_work():
while not available_work:
await available_work_pl.park()
result = list(available_work)
available_work.clear()
return result
def add_work_to_pile():
available_work.append(foo)
available_work_pl.unpark()
Edit: Replaced "if" with "while" in get_work(). I think if has a race condition: if there are two parked tasks and then add_work_to_pile() gets called twice, then one get_work() would get both work items but the other would still be unparked and return an empty list. Using while instead will make it loop back around until more data is added.

IMHO you don't want an event in the first place. The combination of an array and something that tells the reader there's work in the array is already available as memory channels. They have the additional advantage that you can tell them how much work to accept before the sender stalls.
send_channel, recv_channel = trio.open_memory_channel(10)
get_work = recv_channel.receive
add_work_to_pile = send_channel.send
# both are async functions or use the _nowait() versions

how to call a callback function after certain amount of time

I'm using Twisted along with Txmongo lib.
In the following function, I want to invoke cancelTest() 5 secs later. But the code does not work. How can I make it work?
from twisted.internet import task
def diverge(self, d):
if d == 'Wait':
self.flag = 1
# self.timeInit = time.time()
clock = task.Clock()
for ip in self.ips:
if self.factory.dictQueue.get(ip) is not None:
self.factory.dictQueue[ip].append(self)
else:
self.factory.dictQueue[ip] = deque([self])
# self.factory.dictQueue[ip].append(self)
log.msg("-----------------the queue after wait")
log.msg(self.factory.dictQueue)
###############################HERE, this does not work
self.dtime = task.deferLater(clock, 5, self.printData)
#############################
self.dtime.addCallback(self.cancelTest)
self.dtime.addErrback(log.err)
else:
self.cancelTimeOut()
d.addCallback(self.dispatch)
d.addErrback(log.err)
def sendBackIP(self):
self.ips.pop(0)
log.msg("the IPs: %s" % self.ips)
d = self.factory.service.checkResource(self.ips)
d.addCallback(self.diverge) ###invoke above function
log.msg("the result from checkResource: ")
log.msg()

In general reactor.callLater() is the function you want. So if the function needs to be called 5 seconds later, your code would look like this:
from twisted.internet import reactor
reactor.callLater(5, cancelTest)
One thing that is strange is that your task.deferLater implementation should also work. However without seeing more of your code I don't think I can help you more other than stating that it's strange :)
References
https://twistedmatrix.com/documents/current/core/howto/defer.html#callbacks
http://twistedmatrix.com/documents/current/api/twisted.internet.base.ReactorBase.html#callLater

you're doing almost everything right; you just didn't get the Clock part correctly.
twisted.internet.task.Clock is a deterministic implementation of IReactorTime, which is mostly used in unit/integration testing for getting a deterministic output from your code; you shouldn't use that in production.
So, what should you use in production? reactor! In fact, all production reactor implementations implement the IReactorTime interface.
Just use the following import and function call:
from twisted.internet import reactor
# (omissis)
self.dtime = task.deferLater(reactor, 5, self.printData)
Just some sidenotes:
in your text above the snippet, you say that you want to invoke cancelTest after five seconds, but in the code you actually invoke printData; of course if printData just prints something, doesn't raise and returns an immediate value, this will cause the cancelTest function to be executed immediately after since it's a chained callcack; but if you want to actually be 100% sure, you should call cancelTest within deferLater, not printData.
Also, I don't understand if this is a kind of "timeout"; please be advised that such callback will be triggered in all situations, even if the tests take less than five seconds. If you need a cancelable task, you should use reactor.callLater directly; that will NOT return a deferred you can use, but will let you cancel the scheduled call.

Python 3 (Bot) script stops working

I'm trying to connect to a TeamSpeak server using the QueryServer to make a bot. I've taken advice from this thread, however I still need help.
This is The TeamSpeak API that I'm using.
Before the edits, this was the summary of what actually happened in my script (1 connection):
It connects.
It checks for channel ID (and it's own client ID)
It joins the channel and starts reading everything
If someone says an specific command, it executes the command and then it disconnects.
How can I make it so it doesn't disconnect? How can I make the script stay in a "waiting" state so it can keep reading after the command is executed?
I am using Python 3.4.1.
I tried learning Threading but either I'm dumb or it doesn't work the way I thought it would. There's another "bug", once waiting for events, if I don't trigger anything with a command, it disconnects after 60 seconds.
#Librerias
import ts3
import threading
import datetime
from random import choice, sample
# Data needed #
USER = "thisisafakename"
PASS = "something"
HOST = "111.111.111.111"
PORT = 10011
SID = 1
class BotPrincipal:
def __init__(self, manejador=False):
self.ts3conn = ts3.query.TS3Connection(HOST, PORT)
self.ts3conn.login(client_login_name=USER, client_login_password=PASS)
self.ts3conn.use(sid=SID)
channelToJoin = Bot.GettingChannelID("TestingBot")
try: #Login with a client that is ok
self.ts3conn.clientupdate(client_nickname="The Reader Bot")
self.MyData = self.GettingMyData()
self.MoveUserToChannel(ChannelToJoin, Bot.MyData["client_id"])
self.suscribirEvento("textchannel", ChannelToJoin)
self.ts3conn.on_event = self.manejadorDeEventos
self.ts3conn.recv_in_thread()
except ts3.query.TS3QueryError: #Name already exists, 2nd client connect with this info
self.ts3conn.clientupdate(client_nickname="The Writer Bot")
self.MyData = self.GettingMyData()
self.MoveUserToChannel(ChannelToJoin, Bot.MyData["client_id"])
def __del__(self):
self.ts3conn.close()
def GettingMyData(self):
respuesta = self.ts3conn.whoami()
return respuesta.parsed[0]
def GettingChannelID(self, nombre):
respuesta = self.ts3conn.channelfind(pattern=ts3.escape.TS3Escape.unescape(nombre))
return respuesta.parsed[0]["cid"]
def MoveUserToChannel(self, idCanal, idUsuario, passCanal=None):
self.ts3conn.clientmove(cid=idCanal, clid=idUsuario, cpw=passCanal)
def suscribirEvento(self, tipoEvento, idCanal):
self.ts3conn.servernotifyregister(event=tipoEvento, id_=idCanal)
def SendTextToChannel(self, idCanal, mensajito="Error"):
self.ts3conn.sendtextmessage(targetmode=2, target=idCanal, msg=mensajito) #This works
print("test") #PROBLEM HERE This doesn't work. Why? the line above did work
def manejadorDeEventos(sender, event):
message = event.parsed[0]['msg']
if "test" in message: #This works
Bot.SendTextToChannel(ChannelToJoin, "This is a test") #This works
if __name__ == "__main__":
Bot = BotPrincipal()
threadprincipal = threading.Thread(target=Bot.__init__)
threadprincipal.start()
Prior to using 2 bots, I tested to launch the SendTextToChannel when it connects and it works perfectly, allowing me to do anything that I want after it sends the text to the channel. The bug that made entire python code stop only happens if it's triggered by the manejadorDeEventos
Edit 1 - Experimenting with threading.
I messed it up big time with threading, getting to the result where 2 clients connect at same time. Somehow i think 1 of them is reading the events and the other one is answering. The script doesn't close itself anymore and that's a win, but having a clone connection doesn't looks good.
Edit 2 - Updated code and actual state of the problem.
I managed to make the double connection works more or less "fine", but it disconnects if nothing happens in the room for 60 seconds. Tried using Threading.timer but I'm unable to make it works. The entire question code has been updated for it.
I would like an answer that helps me to do both reading from the channel and answering to it without the need of connect a second bot for it (like it's actually doing...) And I would give extra points if the answer also helps me to understand an easy way to make a query to the server each 50 seconds so it doesn't disconnects.

From looking at the source, recv_in_thread doesn't create a thread that loops around receiving messages until quit time, it creates a thread that receives a single message and then exits:
def recv_in_thread(self):
"""
Calls :meth:`recv` in a thread. This is useful,
if you used ``servernotifyregister`` and you expect to receive events.
"""
thread = threading.Thread(target=self.recv, args=(True,))
thread.start()
return None
That implies that you have to repeatedly call recv_in_thread, not just call it once.
I'm not sure exactly where to do so from reading the docs, but presumably it's at the end of whatever callback gets triggered by a received event; I think that's your manejadorDeEventos method? (Or maybe it's something related to the servernotifyregister method? I'm not sure what servernotifyregister is for and what on_event is for…)
That manejadorDeEventos brings up two side points:
You've declared manejadorDeEventos wrong. Every method has to take self as its first parameter. When you pass a bound method, like self.manejadorDeEventos, that bound self object is going to be passed as the first argument, before any arguments that the caller passes. (There are exceptions to this for classmethods and staticmethods, but those don't apply here.) Also, within that method, you should almost certainly be accessing self, not a global variable Bot that happens to be the same object as self.
If manejadorDeEventos is actually the callback for recv_in_thread, you've got a race condition here: if the first message comes in before your main threads finishes the on_event assignment, the recv_on_thread won't be able to call your event handler. (This is exactly the kind of bug that often shows up one time in a million, making it a huge pain to debug when you discover it months after deploying or publishing your code.) So, reverse those two lines.
One last thing: a brief glimpse at this library's code is a bit worrisome. It doesn't look like it's written by someone who really knows what they're doing. The method I copied above only has 3 lines of code, but it includes a useless return None and a leaked Thread that can never be joined, not to mention that the whole design of making you call this method (and spawn a new thread) after each event received is weird, and even more so given that it's not really explained. If this is the standard client library for a service you have to use, then you really don't have much choice in the matter, but if it's not, I'd consider looking for a different library.

get called gobject finished state

In a python script i do a gobject call. I need to know, when its finished. are there any possible ways to check this?
Are there Functions or so on to check?
My code is:
gobject.idle_add(main.process)
class main:
def process():
<-- needs some time to finish -->
next.call.if.finished()
I want to start start another object, pending on the first to finish.
I looked through the gobject reference, but i didn't find something necessary.
Thanks

I am pretty sure you can do something like this, but in your case, as I understood is simpler and you do not need the result from process(), you just need to use something like
main.event.wait() // next.call.if.finished()
I already had to use the very same approach from that link, including the necessity of the result, which is a plus.

An alternative is to start the idle function with a list of the objects you want to process, so instead of waiting for one object to finish and then starting another one, you can let the idle function re-run itself:
def process():
# process object
if any_objects_left:
# set up the next object
return True
return False # remove the idle callback

Twisted non-blocking method - how to?

My code looks like this:
... # class Site(Resource)
def render_POST(self,request)
otherclass.doAssync(request.args)
print '1'
return "done" #that returns the HTTP response, always the same.
...
def doAssync(self,msg):
d = defer.Deferred()
reactor.callLater(0,self.doStuff,d,msg)
d.addCallback(self.sucess)
def doStuff(self,d,msg):
# do some stuff
time.sleep(2) #just for example
d.callback('ok')
def sucess(msg):
print msg
The output:
1
ok
So far, so good, but, the HTTP response (return 'done'), only happens after the delay (time.sleep(2)).
I can tell this, because the browser keeps 'loading' for 2 seconds.
What am I doing wrong?

What you are doing wrong is running a blocking call (time.sleep(2)), while Twisted expects you to only perform non-blocking operations. Things that don't wait. Because you have that time.sleep(2) in there, Twisted can't do anything else while that function is sleeping. So it can't send any data to the browser, either.
In the case of time.sleep(2), you would replace that with another reactor.callLater call. Assuming you actually meant for the time.sleep(2) call to be some other blocking operation, how to fix it depends on the operation. If you can do the operation in a non-blocking way, do that. For many such operations (like database interaction) Twisted already comes with non-blocking alternatives. If the thing you're doing has no non-blocking interface and Twisted doesn't have an alternative to it, you may have to run the code in a separate thread (using for example twisted.internet.threads.deferToThread), although that requires your code is actually thread-safe.