I want to open a TCP client socket in Python. Do I have to go through all the low-level BSD create-socket-handle / connect-socket stuff or is there a simpler one-line way?
Opening sockets in python is pretty simple. You really just need something like this:
import socket
sock = socket.socket()
sock.connect((address, port))
and then you can send() and recv() like any other socket
OK, this code worked
s = socket.socket()
s.connect((ip,port))
s.send("my request\r")
print s.recv(256)
s.close()
It was quite difficult to work that out from the Python socket module documentation. So I'll accept The.Anti.9's answer.
For developing portable network programs of any sort in Python, Twisted is quite useful. One of its benefits is providing a convenient layer above low-level socket APIs.
Related
I am new to the module socket and I am learning how to implement it. I currently have this basic server written.
import socket
s = socket.socket()
host = "Some IP"
port = 12345
s.bind((host, port))
print "Setting up server on", host + ":" + str(port)
s.listen(5)
while True:
c, addr = s.accept()
print 'Got connection from', addr
c.send('Thank you for connecting.')
print c.recv(1024)
c.close()
However, I realize this server can only deal with one client at a time. To further my training, I want to handle multiple clients at one time. So I made a research and came across the library SocketServer. I also heard the method of threading the regular socket module, so every new connection creates a new thread. The information I found was not enough. Please help me to understand the difference between these two methods I found and which one to use where.
Thanks in advance.
The socket library is very low-level, you really have to implement most things yourself.
SocketServer is more high-level. It uses the socket library internally, and provides an easy interface, that allows you deal with multiple clients at a time, and you don't have to worry about the whole low-level stuff involved in using sockets.
Look at the examples here and compare them to your own code. You'll probably see the differences.
https://docs.python.org/2/library/socketserver.html#examples
I am doing asynchronous networking programming with tornado, I've created a socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM))
and put it in tornado iostream
iostream = tornado.iostream.IOStream(sock)
I wonder if I could set the socket 'keepalive' after then? Is the setting still valid after the iostream creation. Thank you in advance.
I would say it's usually best to set any socket options you want before creating the IOStream, but in most cases it's fine to set it afterwards as well (as long as the underlying socket option can be set on a socket that is already connected). As of Tornado 4.0 the only option IOStream touches directly is TCP_NODELAY.
I am trying to find the easiest way to read from multiple (around 100) udp datagram sockets in python. I have looked at tornado, but tornado touts http/tcp rather than udp support.
Right now I have threads dedicated to each udp socket; however, this doesn't seem very efficient.
The SocketServer module has a built-in UDP server with options for threading and forking.
Another option is the use the select module which will allow you to focus only on the sockets where data is already available for reading.
I must confess I never used it, but maybe Twisted will suit your needs.
It supports lots of protocols, even serial connections.
I'd like to add some comments on the initial question even though it already has an accepted answer.
If you have multiple connections which need independent processing (or at least processing with not much synchronization), its okay to use one thread per connection and do blocking reads. Modern schedulers won't kill you for that one. It is a fairly efficient way of handling the connections. If you are concerned about memory footprint, you could reduce the stack size of the threads accordingly (does not apply to python).
The threads/processes will stay in a non-busy waiting state for most of the time (while waiting for new data) and not consume any CPU time.
If you do not want or can not use threads, the select call is definetly the way to go. This is also low-level and efficient waiting and as a bonus, gives you the list of sockets which triggered.
asyncoro supports asynchronous TCP and UDP sockets (among many other features). Unlike with other frameworks, programming with asyncoro is very similar to that of threads. A simple UDP client/server program to illustrate:
import socket, asyncoro
def server_proc(n, sock, coro=None):
for i in xrange(n):
msg, addr = yield sock.recvfrom(1024)
print('Received "%s" from %s:%s' % (msg, addr[0], addr[1]))
sock.close()
def client_proc(host, port, coro=None):
sock = asyncoro.AsynCoroSocket(socket.socket(socket.AF_INET, socket.SOCK_DGRAM))
msg = 'client socket: %s' % (sock.fileno())
yield sock.sendto(msg, (host, port))
sock.close()
if __name__ == '__main__':
sock = asyncoro.AsynCoroSocket(socket.socket(socket.AF_INET, socket.SOCK_DGRAM))
sock.bind(('127.0.0.1', 0))
host, port = sock.getsockname()
n = 100
server_coro = asyncoro.Coro(server_proc, n, sock)
for i in range(n):
asyncoro.Coro(client_proc, host, port)
asyncoro uses efficient polling mechanisms where possible. Only with Windows and UDP sockets it uses inefficient 'select' (but uses efficient Windows I/O Completion Ports for TCP if pywin32 is installed).
I think if you do insist on using tornado's ioloop and want to do UDP socket processing, you should use a UDP version of the tornado IOStream. I have done this with success in my own projects. It is a little bit of a misnomer to call it UDPStream (since it is not quite a stream), but the basic use should be very easy for you to integrate into your application.
See the code at: http://kyle.graehl.org/coding/2012/12/07/tornado-udpstream.html
I'm a beginner in python (2.6/2.7) who has been thrown in the deep end to create a network service to an existing python app.
I've got a UDP server up and running which works just great but I'm asking for help in making it slightly more bullet proof.
Here is the base code I have written, the usual standard boiler plate:
import sys
import socket
from threading import Thread
def handleInput():
sock = socket.socket( socket.AF_INET, socket.SOCK_DGRAM )
sock.bind( ("127.0.0.1",5005) )
# socket always binded?
while True:
data, addr = sock.recvfrom(512)
# is 'data' usable?
t=Thread(target=handleInput)
t.daemon=True
t.start()
# can thread die?
Firstly, I understand that the socket is in a sense always available, it doesn't have to listen, so there is no such thing as failing and you have to reconnect. Is this correct?
Secondly, the issue with returned data from recvfrom. Is it a simple 'if data : then' to check if its valid?
Thirdly, and finally, the thread. Can a thread bomb out? If it does whats the best way to restart it?
Any other tips would be welcome.
(Note: I cannot use external libraries, e.g. twisted etc.)
Some answers for your questions:
UDP sockets are connectionless, and as such "always available". Please also be aware that the UDP message length is limited to 65535 bytes IIRC.
Data is just a string, and its validity will depend on the used protocol. Your if data: ... will work as a test for receiving something, but the recvfrom() call will block anyway until it receives something so this seems superfluous.
I have no knowledge about python thread reliability in general.
I would also check the SocketServer module and the associated UDPServerclass - this might make for easier implementation. The SocketServer is part of python's standard library.
I've read that socketserver is easier to use, but for someone who is just learning about sockets, which would be quicker and more beginner-friendly, socket or socketserver? For a very basic client/server setup using stream sockets. (Python)
socket is the low-level interface which SocketServer (as well as other networking code) is based off of. I'd start out learning it, whether you plan to use it directly or not, just so that you know what you're working with.
Also, SocketServer is of no use if you're writing client code. :)