I'm trying to adapt the Python Twisted - UDP examples to use UDP broadcast. I can send a message from the client and receive it on the server, however, it isn't sending a message back.
Client:
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
from socket import SOL_SOCKET, SO_BROADCAST
class EchoClientDatagramProtocol(DatagramProtocol):
strings = [
"Hello, world!",
"What a fine day it is.",
"Bye-bye!"
]
def startProtocol(self):
self.transport.socket.setsockopt(SOL_SOCKET, SO_BROADCAST, True)
self.transport.connect("255.255.255.255", 8000)
self.sendDatagram()
def sendDatagram(self):
if len(self.strings):
datagram = self.strings.pop(0)
self.transport.write(datagram)
else:
reactor.stop()
def datagramReceived(self, datagram, host):
print 'Datagram received: ', repr(datagram)
self.sendDatagram()
def main():
protocol = EchoClientDatagramProtocol()
#0 means any port
t = reactor.listenUDP(0, protocol)
reactor.run()
if __name__ == '__main__':
main()
Server:
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
class EchoUDP(DatagramProtocol):
def datagramReceived(self, datagram, address):
print "Received from address: " + str(address)
print str(datagram)
self.transport.write(datagram, address)
print "Finished sending reply."
print "Starting server."
reactor.listenUDP(8000, EchoUDP())
reactor.run()
Console Output:
Server:
Starting server.
Received from address ('192.168.1.137', 53737)
Hello, world!
Finished sending reply.
Client:
no output.
transport.connect creates a connected UDP socket
A connected UDP socket is slightly different from a standard one - it can only send and receive datagrams to/from a single address, but this does not in any way imply a connection. Datagrams may still arrive in any order, and the port on the other side may have no one listening. The benefit of the connected UDP socket is that it it may provide notification of undelivered packages. This depends on many factors, almost all of which are out of the control of the application, but it still presents certain benefits which occasionally make it useful.
I suspect that the response from the server is not being caught by the client as it is listening for responses from the broadcast address, not the specific address of the server.
Instead, just use the self.transport.write(data, (host, port)) form of write without initiating the connection first - this will allow the client to receive packets from any address.
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
from socket import SOL_SOCKET, SO_BROADCAST
class EchoClientDatagramProtocol(DatagramProtocol):
strings = [
"Hello, world!",
"What a fine day it is.",
"Bye-bye!"
]
def startProtocol(self):
self.transport.socket.setsockopt(SOL_SOCKET, SO_BROADCAST, True)
#self.transport.connect("255.255.255.255", 8000) <- not needed
self.sendDatagram()
def sendDatagram(self):
if len(self.strings):
datagram = self.strings.pop(0)
self.transport.write(datagram, ('255.255.255.255', 8000)) # <- write to broadcast address here
else:
reactor.stop()
def datagramReceived(self, datagram, host):
print 'Datagram received: ', repr(datagram)
self.sendDatagram()
def main():
protocol = EchoClientDatagramProtocol()
#0 means any port
t = reactor.listenUDP(0, protocol)
reactor.run()
if __name__ == '__main__':
main()
Related
I have a UDP socketserver program that I use to demonstrate how UDP works (code for the server and client are below). I run this on a server, then have the client.py program send a message and receive a reply. I am unfortunately running into an issue that seems to only occur on campus Wifi. On campus wifi, the client does not receive a response.
Troubleshooting with Wireshark shows the issue. For some reason the UDP server is responding with two UDP messages - one empty, and one containing the response message. These messages are recorded in Wireshark as coming in approximately 0.000002 seconds apart. On a wired network, the one with the response consistently comes first, and on Wifi, the empty message consistently comes first. Since the client is waiting for a single messages response, when the empty message returns, the client prints and exits, and the actual response is never seen.
I know I could write the client to listen for both messages and print out whichever one has the data, but I would rather try to figure out what's going on. Why is the socketserver responding with two messages in the first place, and how can I get it to only send one? OR at least to send the data first.
server.py:
import socketserver
class MyUDPRequestHandler(socketserver.DatagramRequestHandler):
def handle(self):
data = self.request[0].strip()
socket = self.request[1]
# just send back the same data, but lower-cased
socket.sendto(data.lower(), self.client_address)
if __name__ == "__main__":
with socketserver.UDPServer(("0.0.0.0", 9091), MyUDPRequestHandler) as server:
server.serve_forever()
client.py:
import socket
HOST, PORT = "localhost", 9091
message = "NOW I AM SHOUTING" # The UDP server will lowercase the message
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes(message + "\n", "utf-8"), (HOST, PORT))
received = str(sock.recv(1024), "utf-8")
print("Sent: {}".format(message))
print("Received: {}".format(received))
I've repeated the problem and it's socketserver. Notice the definition of DatagramRequestHandler below:
class DatagramRequestHandler(BaseRequestHandler):
"""Define self.rfile and self.wfile for datagram sockets."""
def setup(self):
from io import BytesIO
self.packet, self.socket = self.request
self.rfile = BytesIO(self.packet)
self.wfile = BytesIO()
def finish(self):
self.socket.sendto(self.wfile.getvalue(), self.client_address)
The packet is put into a buffer as rfile and should be read from there, then written back to the wfile buffer. finish sends the packet. The handler shouldn't call sendto itself:
import socketserver
class MyUDPRequestHandler(socketserver.DatagramRequestHandler):
def handle(self):
data = self.rfile.read()
self.wfile.write(data.strip().lower())
if __name__ == "__main__":
with socketserver.UDPServer(("0.0.0.0", 9091), MyUDPRequestHandler) as server:
server.serve_forever()
But just using a simple socket as the server works fine too:
import socket
s = socket.socket(type=socket.SOCK_DGRAM)
s.bind(('', 9091))
while True:
data, client = s.recvfrom(2048)
s.sendto(data.strip().lower(), client)
Note that UDP packets are not guaranteed to be delivered or delivered in the same order, so the original code's issue with the two packets changing order isn't surprising.
I am new to networking programming and python.
I am trying to figure out how to run different jobs at the server side.
For example, I want one function to create connections for incoming clients but in the same time I can still do some administration work from the terminal.
My code is as below but it doesn't work:
Edited: it doesn't work means it will get stuck in the init_conn() function
Like:
starting up on localhost port 8887
Thread: 0 Connected with 127.0.0.1:48080
# waiting
I am looking into SocketServer framework but don't know how that works.
from thread import *
import socket
def init_conn():
thread_count =0
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 8887)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(10)
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = sock.accept()
print 'Thread: '+ str(thread_count) + ' Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
thread_count +=1
sock.close()
def clientthread(conn):
# receive data from client and send back
def console():
print 'this is console'
option = raw_input('-v view clients')
if option == 'v':
print 'you press v'
def main():
start_new_thread( init_conn(),() )
start_new_thread( console(),() )
if __name__ == "__main__":
main()
Your problem is probably that you start the program, sometimes it prints "this is console" and then it ends.
The first bug is that you call the methods instead of passing the handle to start_new_thread. It must be:
start_new_thread( init_conn, () )
i.e. no () after the function name.
The program doesn't do much because start_new_thread() apparent starts a thread and then waits for it to stop. The documentation is pretty unclear. It's better to use the new threading module; See http://pymotw.com/2/threading/
def main():
t = threading.Thread( target=init_conn )
t.daemon = True
t.start()
console()
so the code will run until console() ends.
I suggest to split the server and the command line tool. Create a client which accepts commands from the command line and sends them to the server. That way, you can start the console from anywhere and you can keep the code for the two separate.
Seeing that you're new to python, have you tried taking a look at the threading module that comes with the standard library?
import threading
... #rest of your code
while conditions==True:
i = threading.Thread(target=init_conn)
c = threading.Thread(target=console)
i.start()
c.start()
Can't say I've done too much with networking programming with python, so I don't really have much to say in that manner, but at least this should get you started with adding multithreading to your project.
Using SocketServer you may implement a client/server system. The documentation gives small examples which may be useful for you. Here is an extended example from there:
server.py :
import SocketServer
import os
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
class MyServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
# By setting this we allow the server to re-bind to the address by
# setting SO_REUSEADDR, meaning you don't have to wait for
# timeouts when you kill the server and the sockets don't get
# closed down correctly.
allow_reuse_address = True
request_queue_size = 10
def __init__(self, port):
self.host = os.uname()[1]
self.port = port
SocketServer.TCPServer.__init__(self, (self.host,self.port), MyTCPHandler)
logging.info( "Server has been started on {h}:{p}".format(h=self.host,p=self.port) )
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
# max length is here 1024 chars
self.data = self.request.recv(1024).strip()
logging.info( "received: {d}".format(d=self.data) )
# here you may execute different functions according to the
# request string
# here: just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
PORT = 8887
if __name__ == "__main__":
# Create the server, binding to localhost on port 8887
#server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server = MyServer( PORT )
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
client.py
import socket
import sys
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
HOST, PORT = "workstation04", 8887
logging.info( "connect to server {h}:{p}".format(h=HOST,p=PORT ) )
# read command line
data = " ".join(sys.argv[1:])
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Connect to server and send data
sock.connect((HOST, PORT))
sock.sendall(data + "\n")
# Receive data from the server and shut down
received = sock.recv(1024)
finally:
sock.close()
logging.info( "Sent: {}".format(data) )
logging.info( "Received: {}".format(received) )
The output looks something like:
server side:
> python server.py
[2015-05-28 11:17:49,263] Server has been started on disasterarea:8887
[2015-05-28 11:17:50,972] received: my message
client side:
[2015-05-28 11:17:50,971] connect to server disasterarea:8887
[2015-05-28 11:17:50,972] Sent: my message
[2015-05-28 11:17:50,972] Received: MY MESSAGE
You can run several clients (from different consoles) in parallel. You may implement a request processor on the server side which processes the incoming requests and executes certain functions.
Alternatively, you may use the python module ParallelPython which executes python code locally on a multicore system or on a cluster and clusters. Check the http examples.
I had to force pip to install this module:
pip install --allow-external pp --allow-unverified pp pp
This question already has an answer here:
Sending data from one Protocol to another Protocol in Twisted?
(1 answer)
Closed 8 years ago.
The code below receives data through sockets from an iPhone, and then I want to send that received data to another python script running through a different socket. My attempt is below. The other server receives the message fine however I am getting an errno 9 bad file descriptor as soon as I send the second message. Is there anyway to change the below code so it can continuously send received data straight to another socket?
import os
from twisted.internet.protocol import Protocol, Factory
from twisted.internet import reactor
from threading import Thread
class IphoneChat(Protocol):
def connectionMade(self):
self.factory.clients.append(self)
print "A new client has connected"
def connectionLost(self, reason):
print "client disconnected"
def dataReceived(self, data):
print "Message Received: ", data
TCP_IP = '127.0.0.1'
TCP_PORT = 5000
BUFFER_SIZE = 1024
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send(data)
s.close()
def message(self, message):
self.transport.write(message + '\n')
factory = Factory()
factory.protocol = IphoneChat
factory.clients = []
reactor.listenTCP(3000, factory)
print "listening to 3000"
reactor.run()
There are a couple of errors in your IphoneChat code: a missing import socket and self.s = socket.socket(...) should just bind to a local variable s, not to the instance variable self.s.
Nevertheless, I think that the exception (socket.error: [Errno 9] Bad file descriptor) is actually being raised in the "other" Python script for which you don't show any code. It would be helpful to see that code, however, it is likely that you are closing the main server socket in response to the connection being closed by the IphoneChat script, rather than the per connection socket returned by socket.accept(). Here's some rough code that should work as the receiving server:
import socket
s = socket.socket()
s.bind(('127.0.0.1', 5000))
s.listen(1)
while True:
print "Waiting for connection... ",
conn, addr = s.accept()
print "Got connection from {}".format(addr)
while True:
msg = conn.recv(1024)
if msg == '':
print "Remote disconnected"
break
print "Got msg: %r" % msg
# N.B. close connection to remote peer, not the main server socket "s"
conn.shutdown(socket.SHUT_RDWR)
conn.close()
My guess is that you have code similar to the above, but that you are closing the main server socket instead of the remote connection.
Given that I have implemented a UDP Client in Twisted with the DatagramProtocol, and using it to communicate to a UDP Server, which at one point goes offline (due to a restart - so it does not change it's IP address), stopProtocol in my protocol is called, however the transport itself is set to None by Twisted.
How can I solve a simple reconnect in Twisted or re-initiate the transport?
I cannot connect again with udp according to the docs.
Given that in UDP the sender should be able to send packets even after the server is dead, and given that the protocol has it's own connection handling in the packets, I could reconnect the logical part entirely over the Packet layer, if the transport would not disappear.
I suppose running listenUDP again with a new protocol while the core is running won't work.
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
class UDPClientProtocol(DatagramProtocol):
def __init__(self, host, port):
self.host = host
self.port = port
def startProtocol(self):
# Called when transport is connected
self.transport.connect(self.host, self.port)
self.transport.write('initiate protocol') # pseudo code.
def stopProtocol(self):
print "I have lost connection and self.transport is gone!"
# wait some time and try to reconnect somehow?
t = reactor.listenUDP(0, UDPClientProtocol('127.0.0.1', 12345))
reactor.run()
I used following technique that I saw at Twisted DNS source file. It survives server disconnection and even network failures.
from twisted.internet import reactor, protocol, task
import time
class EchoClientDatagramProtocol(protocol.DatagramProtocol):
def __init__(self, host, port, reactor):
self.host = host
self.port = port
self._reactor = reactor
def startProtocol(self):
self.transport.connect(self.host, self.port)
def stopProtocol(self):
#on disconnect
self._reactor.listenUDP(0, self)
def sendDatagram(self):
datagram = ntp_packet
try:
self.transport.write(datagram, (self.host, self.port))
print "{:0.6f}".format(time.time())
except:
pass
def datagramReceived(self, datagram, host):
pass
#print 'Datagram received: ', repr(datagram)
#self.sendDatagram()
def main():
protocol = EchoClientDatagramProtocol('127.0.0.1', 8000, reactor)
t = reactor.listenUDP(0, protocol)
l = task.LoopingCall(protocol.sendDatagram)
l.start(1.0) # call every second
reactor.run()
if __name__ == '__main__':
main()
Interesting. That sounds like something that shouldn't happen. Is this due to a network interface restart or something? What conditions reproduce this?
The simple answer to your question is probably "call listenUDP again, with 'self'", but I am curious what could cause this error to happen in the first place.
I would like to listen on 2 different UDP port with the same server.
I use SocketServer lib for my server, and basicly it looks like that;
SocketServer.UDPServer(('', 7878),CLASSNAME)
I would like to listen on 7878 and 7879 with the same server and same file.
Is that possible? If yes how?
Sure you can, using threads. Here's a server:
import SocketServer
import threading
class MyUDPHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request[0].strip()
socket = self.request[1]
print "%s wrote:" % self.client_address[0]
print data
socket.sendto(data.upper(), self.client_address)
def serve_thread(host, port):
server = SocketServer.UDPServer((host, port), MyUDPHandler)
server.serve_forever()
threading.Thread(target=serve_thread,args=('localhost', 9999)).start()
threading.Thread(target=serve_thread,args=('localhost', 12345)).start()
It creates a server to listen on 9999 and another to listen on 12345.
Here's a sample client you can use for testing this:
import socket
import sys
HOST, PORT = "localhost", 12345
data = 'da bomb'
# SOCK_DGRAM is the socket type to use for UDP sockets
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# As you can see, there is no connect() call; UDP has no connections.
# Instead, data is directly sent to the recipient via sendto().
sock.sendto(data + "\n", (HOST, PORT))
received = sock.recv(1024)
print "Sent: %s" % data
print "Received: %s" % received
Note: this was taken from the docs of the SocketServer module, and modified with threads.
Nope. Consider using Twisted.
No need to use threads for something like this. Consider http://code.google.com/p/pyev/