Develop Reference

How do I forcibly disconnect all currently connected clients to my TCP or HTTP server during shutdown?

I have a fake HTTP server that I use as a fixture in my testing. At some point in the test, I want to stop the server regardless of any still open connections. Clients on these open connections should get a TCP FIN.
I am aware that usually production servers need to solve different problem, that of quiescing, sometimes called graceful shutdown. This is the opposite of what I want.
With a standalone process, it is usually possible to simply get the process to quit and the OS will take care of the rest. (Forcibly killing processes is easy, while forcibly killing threads is not.) My fake server is, however, running in a thread of the test process itself, so I don't have this option (and I don't want to externalize it if there is other way around).
I investigated this issue in Python, with the HTTPServer class, where I was not able to find any solution.
I also investigated this in Go, where I was able to find the concept of Contexts, which is close to what I need, but it works the other way around: a http server would propagate a Context that can be used to cancel e.g. a database lookup if a client disconnected.
Edit: looks like Go actually does what I need and has a separate graceful and nongraceful shutdown methods, with the nongraceful being net/http#Server.Close.
server = http.server.HTTPServer(...)
thread = threading.Thread(run=server.serve_forever)
thread.start()
# a client has connected ....
server.shutdown()
# at this point I want to have the server stopped,
# without waiting for the request handling to complete

I've implemented the Go solution in Python. When new client connects, I remember the client socket, and when I want to quit, I shutdown all remembered sockets.
It seems to work.
import socket
import http.server.HTTPServer
class MyHTTPServer(HTTPServer):
"""Adds a method to the HTTPServer to allow it to exit gracefully"""
def __init__(self, addr, handler_cls):
super().__init__(addr, handler_cls)
self._client_sockets: List[socket.socket] = []
self.server_killed = False
def get_request(self) -> Tuple[socket.socket, Any]:
"""Remember the client socket"""
sock, addr = super().get_request()
self._client_sockets.append(sock)
return sock, addr
def shutdown_request(self, request: socket.socket) -> None:
"""Forget the client socket"""
self._client_sockets.remove(request)
print(f"{self._client_sockets=}")
super().shutdown_request(request)
def force_disconnect_clients(self) -> None:
"""Shutdown the remembered sockets"""
for client in self._client_sockets:
client.shutdown(socket.SHUT_RDWR)
Usage
server = MyHTTPServer(server_addr, MyRequestHandler)
# in a new thread
while not server.server_killed:
self._server.handle_request()
# ... use the server (keep in mind it can have at most one client at a time) ...
# in the main program
server.server_killed = True
server.force_disconnect_clients()
server.server_close()

Establish Concurrent Websocket Connections to server Using Python Twisted Websocket Client

I'm trying to open concurrent websocket connections from client end(which has 60k ports limit per machine) using Python Twisted Authobhan Websocket Client.But I'm unable to open not more than 20k connections using below code:
from autobahn.twisted.websocket import WebSocketClientProtocol, WebSocketClientFactory
class WebSocketClient(WebSocketClientProtocol):
def _handshake_request(self):
pass
def onOpen(self):
self._handshake_request()
def onConnect(self, response):
pass
def onMessage(self,data):
pass
class WebSocketFactory(WebSocketClientFactory):
"""WebSocketClient Factory"""
protocol = WebSocketClient
if __name__ == '__main__':
factory = WebSocketFactory()
##### Note here. ######
for _ in range(num_connections):
reactor.connectTCP(ws_url, ws_port, factory)
#####
reactor.run()
I have used "reactor.connectTCP" in a loop, Does it the correct way to open the concurrent websocket connections using Twisted?
Let me know.

You need to perform fewer concurrent connection attempts. You may be able to sustain 20k or more connections but you won't be able to establish them all simultaneously.
Limit yourself to a few dozen or a hundred at a time. You may want to use twisted.internet.task.cooperate for this.

Stop processing Flask route if request aborted

I have a flask REST endpoint that does some cpu-intensive image processing and takes a few seconds to return. Often, this endpoint gets called, then aborted by the client. In these situations I would like to cancel processing. How can I do this in flask?
In node.js, I would do something like:
req.on('close', function(){
//some handler
});
I was expecting flask to have something similar, or a synchronous method (request.isClosed()) that I could check at certain points during my processing and return if it's closed, but I can't find one.
I thought about sending something to test that the connection is still open, and catching the exception if it fails, but it seems Flask buffers all outputs so the exception isn't thrown until the processing completes and tries to return the result:
An established connection was aborted by the software in your host machine
How can I cancel my processing half way through if the client aborts their request?

There is a potentially... hacky solution to your problem. Flask has the ability to stream content back to the user via a generator. The hacky part would be streaming blank data as a check to see if the connection is still open and then when your content is finished the generator could produce the actual image. Your generator could check to see if processing is done and return None or "" or whatever if it's not finished.
from flask import Response
#app.route('/image')
def generate_large_image():
def generate():
while True:
if not processing_finished():
yield ""
else:
yield get_image()
return Response(generate(), mimetype='image/jpeg')
I don't know what exception you'll get if the client closes the connection but I'm willing to bet its error: [Errno 32] Broken pipe

As far as I know you can't know if a connection was closed by the client during the execution because the server is not testing if the connection is open during the execution. I know that you can create your custom request_handler in your Flask application for detecting if after the request is processed the connection was "dropped".
For example:
from flask import Flask
from time import sleep
from werkzeug.serving import WSGIRequestHandler
app = Flask(__name__)
class CustomRequestHandler(WSGIRequestHandler):
def connection_dropped(self, error, environ=None):
print 'dropped, but it is called at the end of the execution :('
#app.route("/")
def hello():
for i in xrange(3):
print i
sleep(1)
return "Hello World!"
if __name__ == "__main__":
app.run(debug=True, request_handler=CustomRequestHandler)
Maybe you want to investigate a bit more and as your custom request_handler is created when a request comes you can create a thread in the __init__ that checks the status of the connection every second and when it detects that the connection is closed ( check this thread ) then stop the image processing. But I think this is a bit complicated :(.

I was just attempting to do this same thing in a project and I found that with my stack of uWSGI and nginx that when a streaming response was interrupted on the client's end that the following errors occurred
SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request
uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 404] during GET
IOError: write error
and I could just use a regular old try and except like below
try:
for chunk in iter(process.stdout.readline, ''):
yield chunk
process.wait()
except:
app.logger.debug('client disconnected, killing process')
process.terminate()
process.wait()
This gave me:
Instant streaming of data using Flask's generator functionality
No zombie processes on cancelled connection

How to know using BaseHTTPRequestHandler that client closed connection

I am writing http server that can serve big files to client.
While writing to wfile stream it is possible that client closes connection and my server gets socket error (Errno 10053).
Is it possible to stop writing when client closes connection?

You can add these methods to your BaseHTTPRequestHandler class so that you can know if the client closed the connection:
def handle(self):
"""Handles a request ignoring dropped connections."""
try:
return BaseHTTPRequestHandler.handle(self)
except (socket.error, socket.timeout) as e:
self.connection_dropped(e)
def connection_dropped(self, error, environ=None):
"""Called if the connection was closed by the client. By default
nothing happens.
"""
# add here the code you want to be executed if a connection
# was closed by the client
In the second method: connection_dropped, you can add some code that you want to be executed each time a socket error (e.g. client closed the connection) occures.

How to deliver instance of object to instance of SocketServer.BaseRequestHandler?

This is problem.
My primary work is : deliver "s" object to "handle" method in TestRequestHandler class.
My first step was : deliver "s" object through "point" method to TestServer class, but here im stuck. How to deliver "s" object to TestRequestHandler? Some suggestions?
import threading
import SocketServer
from socket import *
class TestRequestHandler(SocketServer.BaseRequestHandler):
def __init__(self, request, client_address, server):
SocketServer.BaseRequestHandler.__init__(self, request, client_address, server)
return
def setup(self):
return SocketServer.BaseRequestHandler.setup(self)
def handle(self):
data = self.request.recv(1024)
if (data):
self.request.send(data)
print data
def finish(self):
return SocketServer.BaseRequestHandler.finish(self)
class TestServer(SocketServer.TCPServer):
def __init__(self, server_address, handler_class=TestRequestHandler):
print "__init__"
SocketServer.TCPServer.__init__(self, server_address, handler_class)
return
def point(self,obj):
self.obj = obj
print "point"
def server_activate(self):
SocketServer.TCPServer.server_activate(self)
return
def serve_forever(self):
print "serve_forever"
while True:
self.handle_request()
return
def handle_request(self):
return SocketServer.TCPServer.handle_request(self)
if __name__ == '__main__':
s = socket(AF_INET, SOCK_STREAM)
address = ('localhost', 6666)
server = TestServer(address, TestRequestHandler)
server.point(s)
t = threading.Thread(target=server.serve_forever())
t.setDaemon(True)
t.start()

If I understand correctly, I think you perhaps are misunderstanding how the module works. You are already specifying an address of 'localhost:6666' for the server to bind on.
When you start the server via your call to serve_forever(), this is going to cause the server to start listening to a socket on localhost:6666.
According to the documentation, that socket is passed to your RequestHandler as the 'request' object. When data is received on the socket, your 'handle' method should be able to recv/send from/to that object using the documented socket API.
If you want a further abstraction, it looks like your RequestHandler can extend from StreamRequestHandler and read/write to the socket using file-like objects instead.
The point is, there is no need for you to create an additional socket and then try to force your server to use the new one instead. Part of the value of the SocketServer module is that it manages the lifecycle of the socket for you.
On the flip side, if you want to test your server from a client's perspective, then you would want to create a socket that you can read/write your client requests on. But you would never pass this socket to your server, per se. You would probably do this in a completely separate process and test your server via IPC over the socket.
Edit based on new information
To get server A to open a socket to server B when server A receives data one solution is to simply open a socket from inside your RequestHandler. That said, there are likely some other design concerns that you will need to address based on the requirements of your service.
For example, you may want to use a simple connection pool that say opens a few sockets to server B that server A can use like a resource. There may already be some libraries in Python that help with this.
Given your current design, your RequestHandler has access to the server as a member variable so you could do something like this:
class TestServer(SocketServer.TCPServer):
def point (self, socketB):
self.socketB = socketB # hold serverB socket
class TestRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
if (data):
self.request.send(data)
print data
self.server.socketB ... # Do whatever with the socketB
But like I said, it may be better for you to have some sort of connection pool or other object that manages your server B socket such that your server A handler can just acquire/release the socket as incoming requests are handled.
This way you can better deal with conditions where server B breaks the socket. Your current design wouldn't be able to handle broken sockets very easily. Just some thoughts...

If the value of s is set once, and not reinitialized - you could make it a class variable as opposed to an instance variable of TestServer, and then have the handler retrieve it via a class method of TestServer in the handler's constructor.
eg: TestServer._mySocket = s

Ok, my main task is this. Construction of the listening server (A-server - localhost, 6666) which during start will open "hard" connection to the different server (B-server - localhost, 7777).
When the customer send data to the A-server this (A-server) sends data (having that hard connection to the B-server) to B-server, the answer receives from the B-server to A-server and answer sends to the customer.
Then again : the customer sends data, A-server receives them, then sends to the B-server, the answer receives data from the B-server and A-server send data to the customer.
And so round and round. The connection to the B-server is closes just when the server A will stop.
All above is the test of making this.