I'm trying to implement a timeout that terminates a python script when no connections are receiving for a defined time interval. So far I manage to implement the timeout using the following code:
import sys
import socket
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('192.168.43.112', 5001)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
while True:
try:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
# Receive the data in small chunks and retransmit it
while True:
data = connection.recv(16)
print >>sys.stderr, 'received "%s"' % data
if data:
print >>sys.stderr, 'Do stuff here'
else:
print >>sys.stderr, 'no more data from', client_address
sock.settimeout(5)
break
finally:
# Clean up the connection
connection.close()
except socket.timeout:
break
The code is working correctly in the sense that after establishing a connection and ending that very same connection, after 5 seconds the script terminates. However, if during the timeout window I try to make another connection I have the following error:
starting up on 192.168.43.112 port 5001
waiting for a connection
connection from ('192.168.43.1', 47550)
received "Data 0
"
Do stuff here
received ""
no more data from ('192.168.43.1', 47550)
waiting for a connection
connection from ('192.168.43.1', 39010)
---------------------------------------------------------------------------
error Traceback (most recent call last)
/Users/location/Desktop/sandbox/data_fetcher.py in <module>()
24 # Receive the data in small chunks and retransmit it
25 while True:
---> 26 data = connection.recv(16)
27 print >>sys.stderr, 'received "%s"' % data
28 if data:
error: [Errno 35] Resource temporarily unavailable
I'm not entirely sure how you want this all to work, and I find it a bit surprising that it happens this way right now (I didn't expect the timeout to have this effect), but based on the EAGAIN error (errno 35), what's happening is that the timeout on the main socket—which gets set only once you've had a first connection—is causing the second-accepted socket to be in non-blocking mode as well. This means that when you call connection.recv and there's no data immediately, you get that OSError raised.
I suspect some of this might vary a bit between OSes, but I was able to reproduces this on FreeBSD (you're probably running on Linux).
A minimal change that works around it—I don't think this is necessarily the best way to code this, but it does work—is to explicitly set the accepted socket to blocking:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
connection.setblocking(1)
With this, the code behaves much better (I added a small test framework that spins off your code as a separate process, then makes several connections with varying delays).
Related
I started learning networking with python can somebody help me out with this code as I am unable to connect more than 5 clients at a time. Can someone please suggest me a solution for this?
def main():
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', listening_port))
s.listen(5)
print "[*] Initializing Sockets ... Done"
print "[*] Sockets Binded Successfully ..."
print("[*] Server Started Successfully [%d]\n" % (listening_port))
except Exception, e:
print e
sys.exit(2)
while 1:
try:
conn, addr = s.accept()
data = conn.recv(buffer_size)
start_new_thread(conn_string, (conn, data, addr))
except KeyboardInterrupt:
s.close()
print "\n[*] Proxy Server Shutting Down ..."
sys.exit(1)
s.close()
def conn_string(conn, data, addr):
print conn
print addr
print data
As described in the python socket API:
socket.listen(backlog)
Listen for connections made to the socket. The
backlog argument specifies the maximum number of queued connections
and should be at least 0; the maximum value is system-dependent
(usually 5), the minimum value is forced to 0.
Increase the number from 5 to the number of simultaneous connections you wish to have to the server.
The accepted answer is incorrect. I'll try to explain you why.
listen takes an argument of queued connections. That means, how many new connections can be put into the queue. That's why even in the documentation it's specified 5, while obviously servers serves more than 5 concurrent clients usually.
Once a socket is created (accepted) it is being forwarded to another thread, which performs the actual work. This allows to the listening thread moves back to listening mode and wait for more clients.
The problem with your code is, your main thread which is the listening thread actually reads data from buffer, where it shouldn't. The recv() function blocks (read more about I/O stream blocking)
You need to move the reading process from the listening thread to the worker; and make sure to close() the socket when it's done. The code would look something like this:
def main():
try:
...
s.listen(5)
except Exception, e:
...
while 1:
try:
conn, addr = s.accept()
start_new_thread(conn_string, (conn, addr))
except KeyboardInterrupt:
s.close()
print "\n[*] Proxy Server Shutting Down ..."
sys.exit(1)
s.close()
def conn_string(conn, addr):
data = conn.recv(buffer_size)
print conn
print addr
print data
conn.close()
I have a socket server defined below and created a client as well. They are able to communicate each other without issues.
Now, multiple applications needs to connect to this socket server and will push the data. Different clients subscribed to this server will read the data from the server.
How can I have my applications connect to this server and start writing the data even without the client connecting to it.
import socket
import sys
import time
def socket_connection_1():
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 9999)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
while True:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection \n'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
# Receive the data in small chunks and retransmit it
while True:
print 'its true'
# data = connection.recv(16)
data = "This is a test message from python socket \n"
# print >>sys.stderr, 'received "%s"' % data
if data:
print 'data being sent'
print >>sys.stderr, 'sending data back to the client'
connection.sendall(data)
else:
print >>sys.stderr, 'no more data from', client_address
break
time.sleep(60)
finally:
# Clean up the connection
connection.close()
socket_connection_1()
Is this what you are asking?
You have a server application
The server application produces data
You have a client application
The client application can connect to the server application and retrieve the data
You want the server application to produce the data only once, even if multiple clients connect to the server and retrieve the same data
You want the server to be able to produce the data at a time that is independent of when any clients connect to the server
If yes, this is what you are asking, then you have to buffer the data that the server produces somewhere. That "somewhere" can be in a file or in memory.
As an optimization, you can produce and buffer the data only when the first client connects, to avoid wasting resources on producing the data when no client connects.
You will need some logic to decide when to delete ("purge") the buffered data, to free the storage capacity when no more clients will connect.
If your application can produce multiple types of data and your storage capacity is limited, you will need some logic to decide which data to purge to make room for new data that you want to produce.
Now we are getting close to the concept of a "cache".
I've been looking and dealing with this issue for a week. I have client code that causes select() to return a socket that has actually closed from external reasons throwing an error 9 BAD FILE DESCRIPTOR, however I tested the code from a different python file and CANNOT get it to error. Ive tried a million things. heres a snippet from the server:
NOTE: This will work for a few iterations and then suddenly break, it errors out in the message_queue as key error due to the file descriptor breaking even tho a message/no message has a key for that socket present.
#Create the socket to communicate with uWSGI applications
server_address = ('localhost', 10001)
server = create_server_socket(server_address)
#Sockets which we expect to read on from select()
input_sockets = [server]
#Sockets which we expect to write to from select()
output_sockets = []
#Message buffer dicitonary for outgoing messages
message_queue = {}
#Now wait for connections endlessly
while input_sockets:
print >> sys.stderr, "Waiting for the next event..."
readable, writable, exceptional = select.select(input_sockets, output_sockets, input_sockets)
#Handle input_sockets
for s in readable:
#Server socket is available for reading now
if s is server:
#Create a connection and address object when incoming request is recieved
connection, client_addr = s.accept()
print >> sys.stderr, "Connection recieved from %s!" % (client_addr,)
#Set client connection to non blocking as well
connection.setblocking(0)
#Add this socket to input sockets as it will read for client data
input_sockets.append(connection)
#Give connection a queue for sending messages to it
message_queue[connection] = Queue.Queue()
#A client has sent data so we can handle its request
else:
#Pull data from the client
data = ""
try:
while True:
message = s.recv(1024)
if not message:
break
data += message
except Exception as e:
print str(e)
if data:
#Readable client socket has data
print >> sys.stderr, 'Recieved "%s" from %s' % (data, s.getpeername())
message_queue[s].put(data)
#Add output channel now to send message
if s not in output_sockets:
output_sockets.append(s)
#There is no data to be read, socket must be closed
else:
print >> sys.stderr, 'Closing', client_addr,'after recieving no data.'
#Stop listening for input on the socket
if s in output_sockets:
output_sockets.remove(s)
input_sockets.remove(s)
#Close the connection
s.close()
del message_queue[s]
#Handle writable connections
for s in writable:
if s:
try:
next_message = message_queue[s].get_nowait()
except:
print >> sys.stderr, 'No data to send for', s.getpeername()
output_sockets.remove(s)
else:
try:
print >> sys.stderr, 'Sending "%s" to %s' % (next_message, s.getpeername())
s.sendall(next_message)
except:
print >> sys.stderr, 'No data to send for', s.getpeername()
output_sockets.remove(s)
#s.sendall('EOF:!##$:EOF')
#Now handle any exceptions
for s in exceptional:
print >> sys.stderr, 'Handling exception on ', s.getpeername()
input_sockets.remove(s)
if s in output_sockets:
output_sockets.remove(s)
s.close()
#Remove any messages
del message_queue[s]
client:
messages = [ 'This is the message. ',
'It will be sent ',
'in parts.',
]
server_address = ('localhost', 10001)
# Create a TCP/IP socket
socks = [ socket.socket(socket.AF_INET, socket.SOCK_STREAM),
socket.socket(socket.AF_INET, socket.SOCK_STREAM),
]
# Connect the socket to the port where the server is listening
print >>sys.stderr, 'connecting to %s port %s' % server_address
for s in socks:
s.connect(server_address)
for message in messages:
# Send messages on both sockets
for s in socks:
print >>sys.stderr, '%s: sending "%s"' % (s.getsockname(), message)
s.send(message)
# Read responses on both sockets
for s in socks:
data = s.recv(1024)
print >>sys.stderr, '%s: received "%s"' % (s.getsockname(), data)
if not data:
print >>sys.stderr, 'closing socket', s.getsockname()
s.close()
NOTE: This client side is only to test and start passing messages.
There is a race in your code when a socket is returned as both readable and writable and you close the socket because the read returned 0 bytes. In this case you remove the socket from input_sockets, output_sockets and message_queue but the closed socket is still in writable and it will thus try to write to it inside the same iteration of the select loop.
I have no idea if this is the race you'll see because you neither show debug output not did you say where you stumble over this EBADF. To track similar problems down I recommend to augment your code with more debug information on where you close a socket and where you try to process a socket because it is readable or writable so that you actually find the exact place of the race when looking at the debug output.
I have an idea like how basic communication between client and server is established. So serialize data streams can be passed between client and server. But I want to know, how socket objects can be passed between two clients: I want to know is it possible to pass socket objects between two clients and both share the same socket instance. Please suggest.
Client class:
import socket
import sys
# create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#connect the socket to the port where server is listening
server_address = ('localhost',2000)
print >>sys.stderr, 'connecting to %s port %s' % server_address
sock.connect(server_address)
#after connection is established, data can be through socket with sendall() and recv()
try:
#send data
message = 'This is Message. It will be repeated'
print >>sys.stderr, 'sending "%s"' % message
sock.sendall(message)
#llok for the response
amount_received = 0
amount_expected = len(message)
while amount_received < amount_expected:
data = sock.recv(16)
amount_received += len(data)
print amount_received
print >>sys.stderr, 'received "%s"' % data
finally:
print >>sys.stderr, 'closing socket'
sock.close()
Server class created to receive message from the client and revert with some message.
Server class:
import socket
import sys
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ("localhost",2000)
print >>sys.stderr, 'starting up on %s port %s' %server_address
sock.bind(server_address)
sock.listen(1)
while True:
print >>sys.stderr, 'waiting for connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', cleint_address
while True:
data = connection.recv(16)
print >>sys.stderr, 'received "%s"' % data
if data:
print >>sys.stderr, 'sending data back to the client'
connection.sendall(data)
else:
print >>sys.stderr, 'no more data from', client_address
break
finally:
connection.close()
After server started the client connects with server and displays suitable messages. Now instead of sending messages between client and server, I want to send socket object to another client which can be achieved using either TCP or UDP. In TCP, serialization of data is required. I want to know is there any way to wrap socket object and pass it over.
Socket objects can not be transported (or you know, teleported :D ) to another language or anything. At most, you can create a protocol by which an instance of the socket can be transferred to another language. But again, I don't see how it may help you.
You have a server socket listening on 2000 port. And another Java socket may connect to it using a client socket. So, what's the point of sending one of the socket to the another? the communication link is somehow twirled. Like, we can just eat ourself to regenerate us. But that would be impossible
Similarly, at most, you can send an instance of the server socket to the java socket. But on the same computer, the Java won't be able to recreate it, because the port is already being listened by another program.
Next, if two programs could listen on the same port, that would make stealing of data and forging quite easy. So, it is not possible for two programs to listen on the same port.
I think what you are looking for is that, two programs combinedly handle the I/O of the same socket. That is rational, at least.
For that, you should create some sort of bidirectional communication link between these two processes. Like another socket on a different port.
Like S is the Server (the sole owner of the socket S1) meanwhile A and B are the handlers.
S should be listening on two different ports. where only A and B are connected. Then any data that comes to S, at S' discretion would be A or B appropriate, then, A or B will reply to that request. And then S will respond appropriate.
Another approach would be S is the main server socket. and A and B are servers listening on different ports. Whenever data comes to S, S sends it to A or B depending on content.
Thirdly, and the most messy solution would be that, A is the server and it offloads some tasks to B via some sort of communication (server-client or threads or a subprocess) and they handle data appropriately.
The reason of calling it messy is that one has to handle two tasks and its harder to maintain its functionality.
But still, sharing a socket is like using the same page of a copy for two different tasks. Hope it helped
I have a problem, whenever I try to run this python script on an Raspberry PI:
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 10000)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
while True:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
# Receive the data in small chunks and retransmit it
while True:
data = connection.recv(16)
print >>sys.stderr, 'received "%s"' % data
if data:
print >>sys.stderr, 'sending data back to the client'
connection.sendall(data)
else:
print >>sys.stderr, 'no more data from', client_address
break
finally:
# Clean up the connection
connection.close()
I get this error:
File "server.py", line 20
try:
^
IndentationError: unexpected indent
Could you please tell me what's wrong here? The script is supposed to create a simple TCP/IP server, and I have no such problems with the client, so I really don't understand where is my mistake/s...
One of the unfortunate side-effects of Python's use of whitespace for denoting blocks is that sometimes you get scripts that have tabs and spaces mixed up throughout the source code.
Since this script is pretty small, you could try deleting the whitespace preceding each line's code and then reindent it properly.