When running unittests I often get error: [Errno 32] Broken pipe. It seems to be a harmless error that happens during testing, but I haven't been able to prevent or otherwise suppress it.
Some things that I have tried include changing SIGPIPE handling to SIG_DFL and running the app with threaded=True. If I were to try/except I don't know which code to wrap since this is in the context of unittesting.
I don't care about catching/preventing the error as much as just suppressing its output while all the other tests finish running. What else should I try?
Edit:
Here is some example code that often, but does not always result in the error:
def test_register(self):
self.driver.get(self.get_server_url() + url_for(u'register'))
body = self.driver.find_element_by_id(u'body')
username_input = body.find_element_by_id(u'username')
username_input.send_keys(self.USER1_DISPLAY_USERNAME)
privacy_policy = body.find_element_by_id(u'privacy_policy')
privacy_policy.click()
#NOTE: not shown - more lines filling out form elements exactly like the above lines
self.driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
register_button = body.find_element_by_id(u'onclick-register')
register_button.click()
And here is an example of the error message:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 44668)
Traceback (most recent call last):
File "/usr/lib/python2.7/SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 657, in __init__
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 716, in finish
self.wfile.close()
File "/usr/lib/python2.7/socket.py", line 283, in close
self.flush()
File "/usr/lib/python2.7/socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
This seems to only happen with integration tests. I'm using Selenium and a LiveServerTestCase with a LIVESERVER_PORT port set to 8943. The mention of port 44668 in the error message looks suspicious.
My problem sounds exactly like Suppress print output in unittests but none of those solutions work.
I found a solution by redirecting stderr and stdout.
In create_app() add the following:
_stderr, _stdout = sys.stderr, sys.stdout
null = open(os.devnull,'wb')
sys.stdout = sys.stderr = null
Related
I have a web service(REST) where one request might take up to 30 sec to return an answer (lots of calculation). There is a risk, that during the calculation, the client webbrowser aborts(?) the existing connection and retries. Here is the console-output of the server-side:
Exception happened during processing of request from ('127.0.0.1', 53209)
Traceback (most recent call last):
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 290, in _handle_request_noblock
self.process_request(request, client_address)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 318, in process_request
self.finish_request(request, client_address)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 331, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 654, in __init__
self.finish()
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 713, in finish
self.wfile.close()
File "C:\Users\tmx\Anaconda2\lib\socket.py", line 283, in close
self.flush()
File "C:\Users\tmx\Anaconda2\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
One option is what I thought of is to somehow notify the client that "I'm alivem but the request is still needs some more time", or to somehow set the timeout on server side. What are the possibilities?
It's difficult to run code in Flask after you've already returned some data. Your options are to either use something like a task queue (see Celery), or to yield your response in multiple parts.
Views in Flask can return strings, but they can also return iterables that contain strings. So you could return "abc", ["abc"], or a generator that will yield "abc". If you do your processing between yields, data will get sent to the client while the request is still running.
Take a look at the following example:
def generator_that_does_the_calculation():
sleep(1)
yield "I'm alive, but I need some time\n"
sleep(1)
yield "Still alive here\n"
sleep(1)
yield "Done\n"
#app.route('/calculate')
def calculate():
return Response(generator_that_does_the_calculation())
There's been pretty much of discussion on this topic, and still the peculiarity of my question is that I am using beforeSend, and I blame the error on my possibly incorrect usage of this operator.
So I have this JS code:
DATA_LIST_REQUEST = $.ajax(
{
type:'GET',
url:"/ajax_refresh_order_suggestions/",
beforeSend : function()
{
if(DATA_LIST_REQUEST != null)
DATA_LIST_REQUEST.abort();
},
data:data,
success: function(response)
{
refresh_zeon_data_table(response, $table);
}
});
And the respective Django view which has nothing special in its code, so I don't post it to keep the question concise (will provide it on request).
The above code is called every time the user changes combobox. And sometimes - NOT ALWAYS - I get this unpleasant error: [Errno 10053] An established connection was aborted by the software in your host machine error which most interestingly does not lead to fatal error behaviour, I still receive an answer, but it takes me 8 or 9 seconds instead of otherwise instant response.
Am I doing something wrong in the above-mentioned JS code ?
Here's the full traceback
Exception happened during processing of request from ('127.0.0.1', 61839)
Traceback (most recent call last):
File "C:\Python27\lib\SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "C:\Python27\lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python27\lib\site-packages\django\core\servers\basehttp.py", line 99, in __init__
super(WSGIRequestHandler, self).__init__(*args, **kwargs)
File "C:\Python27\lib\SocketServer.py", line 657, in __init__
self.finish()
File "C:\Python27\lib\SocketServer.py", line 716, in finish
self.wfile.close()
File "C:\Python27\lib\socket.py", line 283, in close
self.flush()
File "C:\Python27\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
----------------------------------------
[31/Oct/2016 01:27:29] "GET /ajax_refresh_order_suggestions/?q=&legal_entity_own_id=5398&vendors=%5B%5D&merchandise=%5B%5D&positive_chosen_quantity_filter=true&zero_chosen_quantity_filter=true&suggested_equal_to_chosen=true&suggested_less_than_chosen=true&suggested_greater_than_chosen=true HTTP/1.1" 200 46
Can anyone help me with my python WebSockets server for some reason whenever I try to send JSON data there are many errors, here is a link to my code:
http://textuploader.com/v5kn
My Client hasn't changed since the last update and doesn't send anything after connection to the server so it should be fine.
Heres the error:
connection established ('127.0.0.1', 51777)
Handshaking...
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 51429)
Traceback (most recent call last):
File "C:\Python27\lib\SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "C:\Python27\lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python27\lib\SocketServer.py", line 655, in __init__
self.handle()
File "C:/Users/Heslop/Desktop/Projects/SnakeServer/server.py", line 64, in handle
self.read_next_message()
File "C:/Users/Heslop/Desktop/Projects/SnakeServer/server.py", line 68, in read_next_message
length = ord(self.rfile.read(2)[1]) & 127
IndexError: string index out of range
In fact you can try the third-lib for ServerSide-ClientSide msg(frame) encoding and decoding for you, so you can focus on app-building.
Here is my little wrapping for it: https://github.com/HelloLyfing/py-websocket-helper
Since I am the only one who ever used this wrapper, so if you got any problems, please let me know.
I've tried to simplify this as much as possible but I'm still getting an error. I have a simple http server (bottle) that upon receiving a post request executes a function which is supposed to quickly fork itself. The parent process simply returns a job ID and closes while the child process continues to process same data (which is a list of URLs). I've removed all the input and output functions and hard coded the data but my program is still crashing. The funny part is when I alter the program to run directly at the command line rather then start an http server and wait for bottle to execute it everything works fine!
#!/usr/bin/python
#This is a comment
import sys, time, bottle, os
from threading import Thread
from Queue import Queue
from bottle import route, run, request, abort
num_fetch_threads = 2
url_queue = Queue()
def fetchURLContent(i, q):
while True:
#print '%s: Looking for URLs in queue' % i
url = q.get()
#print 'URL found: %s' % url[0]
q.task_done()
time.sleep(1)
#route('/', method='POST') # or #route('/login', method='POST')
def main():
urls = ['http://www.yahoo.com', 'http://www.google.com']
newpid = os.fork()
if newpid == 0:
for i in range(num_fetch_threads):
worker = Thread(target=fetchURLContent, args=(i, url_queue))
worker.setDaemon(True)
worker.start()
print 'Queuing: ', url
for url in urls:
url_queue.put(url)
time.sleep(2)
print 'main thread waiting...'
url_queue.join()
print 'Done'
else:
print "Your job id is 5"
return
def webServer():
run(host='33.33.33.10', port=8080)
if __name__ == "__main__":
print 'Listening on 8080...'
webServer()
The error message I get is as follows:
Listening on 8080...
Bottle v0.11.3 server starting up (using WSGIRefServer())...
Listening on http://33.33.33.10:8080/
Hit Ctrl-C to quit.
33.33.33.1 - - [19/Oct/2012 21:21:24] "POST / HTTP/1.1" 200 0
Traceback (most recent call last):
File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/usr/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
self.finish_content()
File "/usr/lib/python2.7/wsgiref/handlers.py", line 246, in finish_content
self.send_headers()
9 url_queue = Queue()
File "/usr/lib/python2.7/wsgiref/handlers.py", line 268, in send_headers
self.send_preamble()
File "/usr/lib/python2.7/wsgiref/handlers.py", line 189, in send_preamble
self._write('HTTP/%s %s\r\n' % (self.http_version,self.status))
File "/usr/lib/python2.7/wsgiref/handlers.py", line 389, in _write
self.stdout.write(data)
File "/usr/lib/python2.7/socket.py", line 324, in write
self.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
Exception happened during processing of request from ('33.33.33.1', 57615)
Traceback (most recent call last):
File "/usr/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
self.wfile.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
Any ideas?
Your main() function terminates immediately without returning anything. Bottle writes an empty HTTP response to the socket and the web server closes the connection.
Your forked off process stays a bit longer in main(), but then terminates too and causes Bottle to write another empty response to the already closed socket. Thats the error you get (broken pipe).
Forking at that point cannot work. HTTP does not allow more than one response per request. You can either block until all work is done and then send a response, or send the response immediately and do the work in a differed thread.
I am using xmlrpc to contact a local server. On the client side, Sometimes the following socket timeout error and happens and its not a consistent error.
Why is it happening? What could be the reason for socket timeout?
<class 'socket.timeout'>: timed out
args = ('timed out',)
errno = None
filename = None
message = 'timed out'
strerror = None
Traceback on the server side is as follows
Exception happened during processing of request from ('127.0.0.1', 34855)
Traceback (most recent call last):
File "/usr/lib/python2.4/SocketServer.py", line 222, in handle_request
self.process_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 241, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.4/SocketServer.py", line 521, in __init__
self.handle()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 314, in handle
self.handle_one_request()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 308, in handle_one_request
method()
File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 441, in do_POST
self.send_response(200)
File "/usr/lib/python2.4/BaseHTTPServer.py", line 367, in send_response
self.send_header('Server', self.version_string())
File "/usr/lib/python2.4/BaseHTTPServer.py", line 373, in send_header
self.wfile.write("%s: %s\r\n" % (keyword, value))
File "/usr/lib/python2.4/socket.py", line 256, in write
self.flush()
File "/usr/lib/python2.4/socket.py", line 243, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
I killed the server and restarted it. Its working fine now.
What could be the reason?
My machine's RAM went full yesterday night by a process and came back to normal today morning.
Will this error be because of some swapping of processes?
Looks like the client socket it timing out waiting for the server to respond. Is it possible that your server might take a lot time to respond some times? Also, if the server is causing the machine to go into swap, that would slow it down making a timeout possible.
If I remember right, socket timeout is not set in xmlrpc in python. Are you doing socket.setdefaulttimeout somewhere in your code?
If it is expected that your server will take time once in a while, then you could set a higher timeout value using above.
HTH