What would be the best method to restrict access to my XMLRPC server by IP address? I see the class CGIScript in web/twcgi.py has a render method that is accessing the request... but I am not sure how to gain access to this request in my server. I saw an example where someone patched twcgi.py to set environment variables and then in the server access the environment variables... but I figure there has to be a better solution.
Thanks.
When a connection is established, a factory's buildProtocol is called to create a new protocol instance to handle that connection. buildProtocol is passed the address of the peer which established the connection and buildProtocol may return None to have the connection closed immediately.
So, for example, you can write a factory like this:
from twisted.internet.protocol import ServerFactory
class LocalOnlyFactory(ServerFactory):
def buildProtocol(self, addr):
if addr.host == "127.0.0.1":
return ServerFactory.buildProtocol(self, addr)
return None
And only local connections will be handled (but all connections will still be accepted initially since you must accept them to learn what the peer address is).
You can apply this to the factory you're using to serve XML-RPC resources. Just subclass that factory and add logic like this (or you can do a wrapper instead of a subclass).
iptables or some other platform firewall is also a good idea for some cases, though. With that approach, your process never even has to see the connection attempt.
Okay, another answer is to get the ip address from the transport, inside any protocol:
d = self.transport.getHost () ; print d.type, d.host, d.port
Then use the value to filter it in any way you want.
I'd use a firewall on windows, or iptables on linux.
Related
I have a python socket server that listens on a port, and accepts all incoming connections using:
(conn, address) = socket.accept()
However, I wish to accept connections only from certain ip address.
Currently, I close the connection if the address isn't registered, to accomplish this.
But is there a better way to do this, by directly rejecting connections from unregistered addresses, instead of accepting connections and then closing them?
It's not possible to indicate Connection refused to clients from some IP addresses, and to establish the connection to clients from other IP addresses. This is not a Python limitation, but a lower-level, BSD socket layer limitation. You can't do it even from C.
The closest behavior in general you can do in Python is closing the connection quickly after it has been accepted:
sock, addr = server_socket.accept()
if addr[0] != '12.34.56.78':
sock.close()
return
...
Then the client would see the connection being accepted, and very shortly after that the client would see EOF when reading from it, and it wouldn't be able to write to it.
However it's possible to limit by interface (i.e. network card) at bind time, by using one of:
server_socket.bind(('', 65432)) # Bind on any interface.
server_socket.bind(('127.0.0.1', 65432)) # Bind on loopback (localhost clients only).
server_socket.bind(('34.56.78.91', 65432))
So in the 127.0.0.1 version, telnet 127.0.0.1 65432 (as a client) would work, but telnet myhostname 65432 would yield Connection refused (and the server_socket.accept() call won't get this connection).
If you read the docs you can find the BaseServer.verify_request(request, client_address) which tells you this:
Must return a Boolean value; if the value is True, the request will be processed, and if it’s False, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns True.
Microsoft appears to support this functionality via the SO_CONDITIONAL_ACCEPT socket option
This appears to require usage of WSAAccept to accept connections
This constant does not appear in pythons socket module on my windows 8 machine. I don't think there is an option to use WSAAccept via python's builtin socket module.
If I understand correctly, this will allow your server to respond to SYN packets immediately with RST packets when configured to do so instead of finishing the handshake and exchanging FIN packets. Note that usage of this flag removes responsibility to handle connections from the operating system and places it on the application, so there is plenty of room for errors and performance hits to occur. If a performance boost was the goal, it might not be not worth pursuing
It is possible to do at the C level on windows. Pythons ctypes module allows interfacing with C code, so it is technically possible to do via a python interface. But it likely requires a non trivial amount of effort. If you are certain you require this feature, it may be less effort to find a C socket library that supports this out of the box, then you could make a ctypes wrapper for that.
Not sure if this is the right title for my problem, but here it goes:
I am currently implementing a Distributed Hash Table (DHT) with an API which can be contacted through TCP. It can serve multiple API calls like PUT, GET, Trace, while listening on multiple IP/Port combinations like this:
factory = protocol.ServerFactory()
factory.protocol = DHTServer
for ip in interfaces:
for port in ports:
reactor.listenTCP(int(port), factory, interface=ip)
print ("Listening to: "+ ip +" on Port: "+port)
reactor.run()
Now those "external" API calls are going to be executed by the underlying DHT implementation (Kademlia, Chord or Pastry). Those underlying DHT implementations are using different protocols to communicate with one another. Kademlia for example uses RPC through UDP.
The protocol for the TCP API (DHTServer in the Code above) has an internal DHT protocol like this:
self.protocol = Kademlia(8088, [("192.168.2.1", 8088)])
Now if a client makes two seperate API requests after one another i get this error message on the second request:
line 197, in _bindSocket
raise error.CannotListenError(self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on any:8088: [Errno 10
048] Normalerweise darf jede Socketadresse (Protokoll, Netzwerkadresse oder Ansc
hluss) nur jeweils einmal verwendet werden.
Which basically says that each socket address is only to be used once. I am not quite sure, but i guess it is because for each API request a new DHTServer protocol instance is created, which in turn also creates a new Kademlia instance and both are trying to listen on the same address. But why is this the case? Shouldn't the first DHTServer protocol instance be destroyed after the first request is served? What am i doing wrong? Is there a better way of doing this? I only recently started working with twisted, so please be patient.
Thanks a lot!
I don't know anything about twisted, but kademlia is a stateful network service, having to maintain its routing table and all that.
Consider sharing a single kademlia instance (and thus underlying UDP socket) across your requests.
My solution to this was to write my own Factory with the inner protocol already pre-defined. Thus i can access it from every instance and it stays the same.
I have a simple proxy server made using twisted
destination = portforward.ProxyFactory(dest_host, dest_port)
reactor.listenTCP(listen_port, destination)
reactor.run()
I would like to change the dest_port under certain conditions without having to restart the server.
I tried:
new_dest = portforward.ProxyFactory(dest_host, new_dest_port)
reactor.listenTCP(listen_port, new_dest)
Of course this produced an address already in use exception.
Is this possible to change the proxy destination during operation?
reactor.listenTCP returns an object which provides IListeningPort which has a stopListening method that stops the server on that port (note that it returns a Deferred and the server isn't actually stopped until the Deferred fires).
You can use this stopListening method before your second listenTCP call to free up the server port for use by the new, reconfigured server.
I have a twisted ReconnectingClientFactory and i can successfully connect to given ip and port couple with this factory. And it works well.
reactor.connectTCP(ip, port, myHandsomeReconnectingClientFactory)
In this situation, when the server is gone, myHandsomeReconnectingClientFactory tries to connect same ip and port (as expected).
My goal is, when the server which serves on given ip and port couple is gone, connecting to a backup server (which have different ip and port).
Any ideas/comments on how to achieve this goal will be appreciated.
Id try something like:
class myHandsomeReconnectingClientFactory(protocol.ReconnectingClientFactory):
def __init_(self, hosts):
# hosts should be a list of tuples (host, port)
self._hosts = hosts
def clientConnectionFailed(self, connector, reason):
if self.continueTrying:
self._try_next_host(connector)
def clientConnectionLost(self, connector, unused_reason):
if self.continueTrying:
self._try_next_host(connector)
def _try_next_host(self, connector):
# round robing of servers
to_try = self._hosts.pop(0)
self._hosts.append(to_try)
connector.host, connector.port = to_try
self.connector = connector
self.retry()
I haven't actually tested it, but at least it should give you a good starting point. Good uck.
ReconnectingClientFactory doesn't have this capability. You can build your own factory which implements this kind of reconnection logic, mostly by hooking into the clientConnectionFailed factory method. When this is called and the reason seems to you like that justifies switching servers (eg, twisted.internet.error.ConnectionRefused), pick the next address on your list and use the appropriate reactor.connectXYZ method to try connecting to it.
You could also try constructing this as an endpoint (which is the newer high-level connection setup API that is preferred by some), but handling reconnection with endpoints is not yet a well documented topic.
How can i access underlying socket from twisted.web.client.Agent? I need to enable TCP_NODELAY on this socket.
Unfortunately Agent doesn't make it as easy as it would be if you were working directly with a Protocol instance, but it's not impossible either.
The key lies here, in the class definition of Agent:
_protocol = HTTP11ClientProtocol
In order to get access to the transport you could override connectionMade on HTTP11ClientProtocol, as well as the Agent.
So you'd end up with something like:
from twisted.web import client
class MyHTTPClient(client.HTTP11ClientProtocol):
def connectionMade(self):
self.transport.setTcpNoDelay(True)
client.HTTP11ClientProtocol.connectionMade(self) # call the super-class's connectionMade
class MyAgent(client.Agent):
_protocol = MyHTTPClient
Now use MyAgent in lieu of Agent and you'll get TCP nodelay on the client.
** Note **, this isn't the only way to do this, but one way you can do so and continue to use Agent.request. Alternately, write your own agent which crafts the request and connects it to a Client and wires up your request, along with TCP nodelay, in a deferred chain.
** Note 2 ** In this case, it's fine to assume 'transport' has the setTcpNoDelay() method because it's a pretty reasonable assumption you'll be using TCP as the transport for an HTTP request. This may not be a smart idea all over twisted, though.