Develop Reference

Python/Flask Unittest, suppress Broken Pipe Error 32

When running unittests I often get error: [Errno 32] Broken pipe. It seems to be a harmless error that happens during testing, but I haven't been able to prevent or otherwise suppress it.
Some things that I have tried include changing SIGPIPE handling to SIG_DFL and running the app with threaded=True. If I were to try/except I don't know which code to wrap since this is in the context of unittesting.
I don't care about catching/preventing the error as much as just suppressing its output while all the other tests finish running. What else should I try?
Edit:
Here is some example code that often, but does not always result in the error:
def test_register(self):
self.driver.get(self.get_server_url() + url_for(u'register'))
body = self.driver.find_element_by_id(u'body')
username_input = body.find_element_by_id(u'username')
username_input.send_keys(self.USER1_DISPLAY_USERNAME)
privacy_policy = body.find_element_by_id(u'privacy_policy')
privacy_policy.click()
#NOTE: not shown - more lines filling out form elements exactly like the above lines
self.driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
register_button = body.find_element_by_id(u'onclick-register')
register_button.click()
And here is an example of the error message:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 44668)
Traceback (most recent call last):
File "/usr/lib/python2.7/SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 657, in __init__
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 716, in finish
self.wfile.close()
File "/usr/lib/python2.7/socket.py", line 283, in close
self.flush()
File "/usr/lib/python2.7/socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
This seems to only happen with integration tests. I'm using Selenium and a LiveServerTestCase with a LIVESERVER_PORT port set to 8943. The mention of port 44668 in the error message looks suspicious.
My problem sounds exactly like Suppress print output in unittests but none of those solutions work.

I found a solution by redirecting stderr and stdout.
In create_app() add the following:
_stderr, _stdout = sys.stderr, sys.stdout
null = open(os.devnull,'wb')
sys.stdout = sys.stderr = null

Flask/http- tell the client that the request needs some more time to complete

I have a web service(REST) where one request might take up to 30 sec to return an answer (lots of calculation). There is a risk, that during the calculation, the client webbrowser aborts(?) the existing connection and retries. Here is the console-output of the server-side:
Exception happened during processing of request from ('127.0.0.1', 53209)
Traceback (most recent call last):
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 290, in _handle_request_noblock
self.process_request(request, client_address)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 318, in process_request
self.finish_request(request, client_address)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 331, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 654, in __init__
self.finish()
File "C:\Users\tmx\Anaconda2\lib\SocketServer.py", line 713, in finish
self.wfile.close()
File "C:\Users\tmx\Anaconda2\lib\socket.py", line 283, in close
self.flush()
File "C:\Users\tmx\Anaconda2\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
One option is what I thought of is to somehow notify the client that "I'm alivem but the request is still needs some more time", or to somehow set the timeout on server side. What are the possibilities?

It's difficult to run code in Flask after you've already returned some data. Your options are to either use something like a task queue (see Celery), or to yield your response in multiple parts.
Views in Flask can return strings, but they can also return iterables that contain strings. So you could return "abc", ["abc"], or a generator that will yield "abc". If you do your processing between yields, data will get sent to the client while the request is still running.
Take a look at the following example:
def generator_that_does_the_calculation():
sleep(1)
yield "I'm alive, but I need some time\n"
sleep(1)
yield "Still alive here\n"
sleep(1)
yield "Done\n"
#app.route('/calculate')
def calculate():
return Response(generator_that_does_the_calculation())

Receiving Errno 10053 (An established connection was aborted by the software in your host machine) on Ajax calls in Django

There's been pretty much of discussion on this topic, and still the peculiarity of my question is that I am using beforeSend, and I blame the error on my possibly incorrect usage of this operator.
So I have this JS code:
DATA_LIST_REQUEST = $.ajax(
{
type:'GET',
url:"/ajax_refresh_order_suggestions/",
beforeSend : function()
{
if(DATA_LIST_REQUEST != null)
DATA_LIST_REQUEST.abort();
},
data:data,
success: function(response)
{
refresh_zeon_data_table(response, $table);
}
});
And the respective Django view which has nothing special in its code, so I don't post it to keep the question concise (will provide it on request).
The above code is called every time the user changes combobox. And sometimes - NOT ALWAYS - I get this unpleasant error: [Errno 10053] An established connection was aborted by the software in your host machine error which most interestingly does not lead to fatal error behaviour, I still receive an answer, but it takes me 8 or 9 seconds instead of otherwise instant response.
Am I doing something wrong in the above-mentioned JS code ?
Here's the full traceback
Exception happened during processing of request from ('127.0.0.1', 61839)
Traceback (most recent call last):
File "C:\Python27\lib\SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "C:\Python27\lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python27\lib\site-packages\django\core\servers\basehttp.py", line 99, in __init__
super(WSGIRequestHandler, self).__init__(*args, **kwargs)
File "C:\Python27\lib\SocketServer.py", line 657, in __init__
self.finish()
File "C:\Python27\lib\SocketServer.py", line 716, in finish
self.wfile.close()
File "C:\Python27\lib\socket.py", line 283, in close
self.flush()
File "C:\Python27\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
----------------------------------------
[31/Oct/2016 01:27:29] "GET /ajax_refresh_order_suggestions/?q=&legal_entity_own_id=5398&vendors=%5B%5D&merchandise=%5B%5D&positive_chosen_quantity_filter=true&zero_chosen_quantity_filter=true&suggested_equal_to_chosen=true&suggested_less_than_chosen=true&suggested_greater_than_chosen=true HTTP/1.1" 200 46

Python ssl server reporting TLSV1_ALERT_UNKNOWN_CA

I have a Python SSL server that uses a self-signed certificate. I start my server like this:
httpd = BaseHTTPServer.HTTPServer(('', 443), MyHTTPHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, certfile='server.pem', server_side=True, cert_reqs=0)
httpd.serve_forever()
I get the following error when I connect using Firefox:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 51194)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 655, in __init__
self.handle()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 310, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 734, in recv
return self.read(buflen)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 621, in read
v = self._sslobj.read(len or 1024)
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1751)
----------------------------------------
I do not see this behavior with Chrome or another client. It only happens on the first connection (complaints about certificate) until I accept the certificate. The exception actually does not cause the program to quit.
Why am I getting an error on the server? How can I avoid the exception?

The TLv1 unknown CA alert is sent by some clients if they cannot verify the certificate of the server because it is signed by an unknown issuer CA. You can avoid this kind of exception if you use a certificate which is already trusted by the client or which can be validated against a root CA of the client (don't forget to include the chain certificates too).
If you cannot avoid this error (for instance because you are using a self-signed certificate) then you have to catch the exception and deal with it by closing the connection. You might need to do this by using handle_request to handle each request by itself and catch exceptions instead of using serve_forever.

I had the same error as you, even though I had a signed certificate from Sectigo. Turns out, you need a certificate chain rather than only your domain's certificate itself.
Source
While referring to this site, and the following video: https://www.youtube.com/watch?v=_YjX7rtiAsk
, I found that I need to create a new file called certificate-chain.pem, and manually/with the help of scripts concatenate (join) three certificates - domain certificate, CA certificate and the USERTrust certificate, one after the other.
Then, in the file you need, point to this certificate bundle/chain. This is a solution which I wept on for 7 hours.

python xmlrpc timeout error

I am using xmlrpc to contact a local server. On the client side, Sometimes the following socket timeout error and happens and its not a consistent error.
Why is it happening? What could be the reason for socket timeout?
<class 'socket.timeout'>: timed out
args = ('timed out',)
errno = None
filename = None
message = 'timed out'
strerror = None
Traceback on the server side is as follows
Exception happened during processing of request from ('127.0.0.1', 34855)
Traceback (most recent call last):
File "/usr/lib/python2.4/SocketServer.py", line 222, in handle_request
self.process_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 241, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.4/SocketServer.py", line 521, in __init__
self.handle()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 314, in handle
self.handle_one_request()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 308, in handle_one_request
method()
File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 441, in do_POST
self.send_response(200)
File "/usr/lib/python2.4/BaseHTTPServer.py", line 367, in send_response
self.send_header('Server', self.version_string())
File "/usr/lib/python2.4/BaseHTTPServer.py", line 373, in send_header
self.wfile.write("%s: %s\r\n" % (keyword, value))
File "/usr/lib/python2.4/socket.py", line 256, in write
self.flush()
File "/usr/lib/python2.4/socket.py", line 243, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
I killed the server and restarted it. Its working fine now.
What could be the reason?
My machine's RAM went full yesterday night by a process and came back to normal today morning.
Will this error be because of some swapping of processes?

Looks like the client socket it timing out waiting for the server to respond. Is it possible that your server might take a lot time to respond some times? Also, if the server is causing the machine to go into swap, that would slow it down making a timeout possible.
If I remember right, socket timeout is not set in xmlrpc in python. Are you doing socket.setdefaulttimeout somewhere in your code?
If it is expected that your server will take time once in a while, then you could set a higher timeout value using above.
HTH