Develop Reference

Stopping cocotb forked coroutine

I have a coroutine that waiting for a signal rising :
#cocotb.coroutine
def wait_for_rise(self):
yield RisingEdge(self.dut.mysignal)
I'm launching it in my «main» test function like it :
mythread = cocotb.fork(wait_for_rise())
I want to stop it after a while even if no signal rise happen. I tryed to «kill» it:
mythread.kill()
But exception happen :
Send raised exception: 'RunningCoroutine' object has no attribute '_join'
File "/opt/cocotb/cocotb/decorators.py", line 121, in send
return self._coro.send(value)
File "/myproject.py", line 206, in i2c_read
wTXDRwthread.kill()
File "/opt/cocotb/cocotb/decorators.py", line 151, in kill
cocotb.scheduler.unschedule(self)
File "/opt/cocotb/cocotb/scheduler.py", line 453, in unschedule
if coro._join in self._trigger2coros:
Is there a solution to stop forked coroutine properly ?

This very much looks like it is the same problem as in https://github.com/potentialventures/cocotb/issues/650 - you can subscribe to the issue to be notified when its status changes.

Flask/http- tell the client that the request needs some more time to complete

I have a web service(REST) where one request might take up to 30 sec to return an answer (lots of calculation). There is a risk, that during the calculation, the client webbrowser aborts(?) the existing connection and retries. Here is the console-output of the server-side:
Exception happened during processing of request from ('127.0.0.1', 53209)
Traceback (most recent call last):
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 290, in _handle_request_noblock
self.process_request(request, client_address)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 318, in process_request
self.finish_request(request, client_address)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 331, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 654, in __init__
self.finish()
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 713, in finish
self.wfile.close()
File "C:\Users\tmx\Anaconda2\lib\socket.py", line 283, in close
self.flush()
File "C:\Users\tmx\Anaconda2\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
One option is what I thought of is to somehow notify the client that "I'm alivem but the request is still needs some more time", or to somehow set the timeout on server side. What are the possibilities?

It's difficult to run code in Flask after you've already returned some data. Your options are to either use something like a task queue (see Celery), or to yield your response in multiple parts.
Views in Flask can return strings, but they can also return iterables that contain strings. So you could return "abc", ["abc"], or a generator that will yield "abc". If you do your processing between yields, data will get sent to the client while the request is still running.
Take a look at the following example:
def generator_that_does_the_calculation():
sleep(1)
yield "I'm alive, but I need some time\n"
sleep(1)
yield "Still alive here\n"
sleep(1)
yield "Done\n"
#app.route('/calculate')
def calculate():
return Response(generator_that_does_the_calculation())

What can shut down a websocket connection?

I use websocket_server in order to provide a one way (server to client) websocket connection.
I have several threads on the server which query at given intervals (while True: ... time.sleep(60)) an API and then perform a server.send_message() call to update the client. All of this works fine.
From time to time, without any particular reason, I get a crash:
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Python35\lib\threading.py", line 914, in _bootstrap_inner
self.run()
File "C:\Python35\lib\threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "D:/Dropbox/dev/domotique/webserver.py", line 266, in calendar
server.send_message(client, json.dumps({"calendar": events}))
File "C:\Python35\lib\site-packages\websocket_server\websocket_server.py", line 71, in send_message
self._unicast_(client, msg)
File "C:\Python35\lib\site-packages\websocket_server\websocket_server.py", line 119, in _unicast_
to_client['handler'].send_message(msg)
File "C:\Python35\lib\site-packages\websocket_server\websocket_server.py", line 194, in send_message
self.send_text(message)
File "C:\Python35\lib\site-packages\websocket_server\websocket_server.py", line 240, in send_text
self.request.send(header + payload)
BrokenPipeError: [WinError 10058] A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call
There is no shutdown call in my code. What else can shut a websocket down?

The WebSocket client can ask the server to close the connection (or directly close it). From the library's code:
if not b1:
logger.info("Client closed connection.")
self.keep_alive = 0
return
if opcode == CLOSE_CONN:
logger.info("Client asked to close connection.")
self.keep_alive = 0
return
You could check self.keep_alive to know if the socket is still open.

Catching bottle server errors

I am trying to get my bottle server so that when one person in a game logs out, everyone can immediately see it. As I am using long polling, there is a request open with all the users.
The bit I am having trouble with is catching the exception that is thrown when the user leaves the page from the long polling that can no longer connect to the page. The error message is here.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/gevent/pywsgi.py", line 438, in handle_one_response
self.run_application()
File "/usr/lib/python2.7/dist-packages/gevent/pywsgi.py", line 425, in run_application
self.process_result()
File "/usr/lib/python2.7/dist-packages/gevent/pywsgi.py", line 416, in process_result
self.write(data)
File "/usr/lib/python2.7/dist-packages/gevent/pywsgi.py", line 373, in write
self.socket.sendall(msg)
File "/usr/lib/python2.7/dist-packages/gevent/socket.py", line 509, in sendall
data_sent += self.send(_get_memory(data, data_sent), flags)
File "/usr/lib/python2.7/dist-packages/gevent/socket.py", line 483, in send
return sock.send(data, flags)
error: [Errno 32] Broken pipe
<WSGIServer fileno=3 address=0.0.0.0:8080>: Failed to handle request:
request = GET /refreshlobby/1 HTTP/1.1 from ('127.0.0.1', 53331)
application = <bottle.Bottle object at 0x7f9c05672750>
127.0.0.1 - - [2013-07-07 10:59:30] "GET /refreshlobby/1 HTTP/1.1" 200 160 6.038377
The function to handle that page is this.
#route('/refreshlobby/<id>')
def refreshlobby(id):
while True:
yield lobby.refresh()
gevent.sleep(1)
I tried catching the exception within the function, and in a decorator which I put to wrap #route, neither of which worked. I tried making an #error(500) decorator, but that didn't trigger, either. It seems that this is to do with the internals of bottle.
Edit: I know now that I need to be catching socket.error, but I don't know whereabouts in my code

The WSGI runner
Look closely at the traceback: this in not happening in your function, but in the WSGI runner.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/gevent/pywsgi.py", line 438, in handle_one_response
self.run_application()
The way the WSGI runner works, in your case, is:
Receives a request
Gets a partial response from your code
Sends it to the client (this is where the exception is raised)
Repeats steps 2-3
You can't catch this exception
This error is not raised in your code.
It happens when you try to send a response to a client that closed the connection.
You'll therefore not be able to catch this error from within your code.
Alternate solutions
Unfortunately, it's not possible to tell from within the generator (your code) when it stops being consumed.
It's also not a good idea to rely on your generator being garbage collected.
You have a couple other solutions.
"Last seen"
Another way to know when an user disconnects would probably be to record a "last seen", after your yield statement.
You'll be able to identify clients that disconnected if their last seen is far in the past.
Other runner
Another, non-WSGI runner, will be more appropriate for a realtime application. You could give tornado a try.

Error with time-consuming REST service in Python

I'm writing the code to attend a service. I receive POST requests to a particular service and start processing. The whole process is rather simple; I loop trough items in the request and add each item to the database. The problem arise when I have to process a lot of items and the loop takes like three minutes to finish, then when I try to respond:
status = '200 OK'
headers = [('Content-type', 'application/json'),('Access-Control-Allow-Origin','*')]
start_response(status, headers)
return json.dumps(response)
I get this error:
Exception happened during processing of request from ('XXX.XXX.XXX.XXX', 49172)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/local/lib/python2.7/SocketServer.py", line 310, in process_request
self.finish_request(request, client_address)
File "/usr/local/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python2.7/SocketServer.py", line 640, in __init__
self.finish()
File "/usr/local/lib/python2.7/SocketServer.py", line 693, in finish
self.wfile.flush()
File "/usr/local/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
I don't know if this helps, but the POST request is a forwarded POST made from a browser to a different domain (that's why the a Access-Control-Allow-Origin) and all the accesses to the database are made using a single object that interacts with the database using SQLAlchemy (can be seen similar to a Java EE DAO pattern).
How do I avoid this error?

You maybe are violating the idea behind REST.
If the proccessing could take some time, the service may want to answer it with an 202 Accepted Response! For a full overview of http response codes follow this link: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
202 Accepted
The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place. There is no facility for re-sending a status code from an asynchronous operation such as this.
The 202 response is intentionally non-committal. Its purpose is to allow a server to accept a request for some other process (perhaps a batch-oriented process that is only run once per day) without requiring that the user agent's connection to the server persist until the process is completed. The entity returned with this response SHOULD include an indication of the request's current status and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled.

I could be wrong, but it just looks to me like the socket is timing out. You shouldn't leave the client hanging for a response for more than three minutes.
Instead, you should validate the data and send a message stating that it was received. If necessary, you can use something like AJAX to tell the client that the data was entered after it was received.