the server-side code (tcp_server.py):
from SocketServer import TCPServer, ThreadingMixIn, StreamRequestHandler
class Server(ThreadingMixIn, TCPServer):
pass
class Handler(StreamRequestHandler):
def handle(self):
print 'got a connection from: ', self.request.getpeername()
print self.rfile.read(1024)
msg = 'hello'
self.wfile.write(msg)
server = Server(('127.0.0.1', 8888), Handler)
server.serve_forever()
the client-side code (tcp_client.py):
from socket import *
import threading
def do_connection():
s = socket(AF_INET, SOCK_STREAM)
s.connect(('127.0.0.1', 8888))
s.sendall('this is client')
print s.recv(1024)
ts = []
for x in xrange(100):
print x
t = threading.Thread(target=do_connection())
t.daemon = True
ts.append(t)
for t in ts:
t.start()
I runned tcp_server.py, and then tcp_client.py. tcp_client.py should have been over soon. However, tcp_client.py seemed just run only one thread and blocked, and tcp_server.py got only one connection. When I interrupted tcp_client.py，tcp_server.py got one message this is client。
Is there any mistake in my code ?
This line:
t = threading.Thread(target=do_connection())
Should be
t = threading.Thread(target=do_connection)
When you use do_connection(), you end up executing do_connection in the main thread, and then pass the return value of that call to the Thread object. What you want to do is pass the do_connection function object to Thread, so that the Thread object can execute do_connection in a new thread when you call t.start.
Also, note that starting 100 threads concurrently to connect to your server may not perform very well. You may want to consider starting with fewer threads, and working your way up to a higher number once you know things are working properly.
because the server is blocked by the first request, I try to change the read(1024) to
readline in the server.py and add a '\n' to the content sended from the client, it
works.
it seems the rfile.read(1024) will block the how process, so the goodway is use readline
or use the self.request.recv(1024)
server.py
from SocketServer import TCPServer, ThreadingMixIn, StreamRequestHandler
class Server(ThreadingMixIn, TCPServer):
pass
class Handler(StreamRequestHandler):
def handle(self):
print 'got a connection from: ', self.request.getpeername()
print self.rfile.readline()
#print self.request.recv(1024).strip()
msg = 'hello'
self.wfile.write(msg)
# Create the server, binding to localhost on port 9999
server = Server(("127.0.0.1", 8888), Handler)
server.serve_forever()
client.py
from socket import *
import threading
def do_connection():
print "start"
s = socket(AF_INET, SOCK_STREAM)
s.connect(('127.0.0.1', 8888))
s.sendall('this is client\n')
print s.recv(1024)
ts = []
for x in xrange(100):
print x
t = threading.Thread(target=do_connection)
ts.append(t)
for t in ts:
print "start t"
t.start()
Related
When i run main and client's py files not showing error but doesn't show the input to send the message. how to fix?
main.py result -- > Waiting... and connected 192.168.1.26 58767
clients.py result -- > nothing
this is main.py
import threading
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
host="192.168.1.26"
port=1111
threadcount=0
server.bind((host,port))
print("Waiting...")
server.listen(3)
def clients(connection):
connection.send(str.encode("welcome\n"))
while True:
data=connection.recv(2048)
reply="server says" + data.decode("utf-8")
if not data:
break
connection.sendall(str.encode(reply))
connection.close()
while True:
client,adress=server.accept()
print("connected"+ str(adress[0]) + " " + str(adress[1]))
threading.Thread(target=clients,args=(client,))
threadcount+=1
server.close()
this is clients.py
import socket
client_socket=socket.socket()
host="192.168.1.26"
port=1111
client_socket.connect((host,port))
response=client_socket.recv(1024)
print("hello")
while True:
input=input("say")
client_socket.send(str.encode(input))
Response=client_socket.recv(1024)
print(Response.decode("utf-8"))
client_socket.close()
It looks like your thread is never actually starting. One solution is to define a class that implements your thread logic in a run routine. The class should inherit threading.Thread. Then you can simply initialize the class as an object and call its start routine. Try the following variant of your original server code.
import threading
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
host="192.168.1.26"
port=1111
threadcount=0
server.bind((host,port))
print("Waiting...")
server.listen(3)
class ServerThread(threading.Thread):
def __init__(self, threadsocket):
threading.Thread.__init__(self)
self.threadsocket = threadsocket
def run(self):
self.threadsocket.send(str.encode("welcome\n"))
while True:
data=self.threadsocket.recv(2048)
reply="server says" + data.decode("utf-8")
print(reply)
if not data:
break
self.threadsocket.sendall(str.encode(reply))
self.threadsocket.close()
while True:
client,adress=server.accept()
print("connected"+ str(adress[0]) + " " + str(adress[1]))
thread = ServerThread(client)
thread.start()
threadcount+=1
server.close()
As far as the client, as I mentioned in a comment you should not use input as a variable name to store the result of the input function. Change it to something like data instead.
I have the following server program in Python which simulates a chat-room. The code accepts connections from clients and for each of them it launches a new thread. This thread will wait for messages from this client. The messages can be L so that the server will respond with a list of connected clients, ip:port msg the server will send the message msg to the client ip:port.
On client side there will be 2 threads, one for receiving messages from the server, the other for sending.
import socket
from threading import Thread
#from SocketServer import ThreadingMixIn
import signal
import sys
import errno
EXIT = False
address = []
address2 = []
# handler per il comando Ctrl+C
def sig_handler(signum, frame):
if (signum == 2):
print("Called SIGINT")
EXIT = True
signal.signal(signal.SIGINT, sig_handler) # setto l'handler per i segnali
# Multithreaded Python server : TCP Server Socket Thread Pool
class ClientThread(Thread):
def __init__(self,conn,ip,port):
Thread.__init__(self)
self.conn = conn
self.ip = ip
self.port = port
print ("[+] New server socket thread started for " + ip + ":" + str(port))
def run(self):
while True:
data = self.conn.recv(1024)
print ("Server received data:", data)
if (data=='L'):
#print "QUI",address2
tosend = ""
for i in address2:
tosend = tosend + "ip:"+str(i[0]) + "port:"+str(i[1])+"\n"
self.conn.send(tosend)
#mandare elenco client connessi
else:
#manda ip:port msg
st = data.split(" ")
msg = st[1:]
msg = ' '.join(msg)
print ("MSG 2 SEND: ",msg)
ipport = st[0].split(":")
ip = ipport[0]
port = ipport[1]
flag = False
print ("Address2:",address2)
print ("ip:",ip)
print ("port:",port)
for i in address2:
print (i[0],ip,type(i[0]),type(ip),i[1],type(i[1]),port,type(port))
if str(i[0])==str(ip) and str(i[1])==str(port):
i[2].send(msg)
self.conn.send("msg inviato")
flag = True
break
if flag == False:
self.conn.send("client non esistente")
if __name__ == '__main__':
# Multithreaded Python server : TCP Server Socket Program Stub
TCP_IP = '127.0.0.1'
TCP_PORT = 2004
TCP_PORTB = 2005
BUFFER_SIZE = 1024 # Usually 1024, but we need quick response
tcpServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServer.bind((TCP_IP, TCP_PORT))
tcpServerB = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServerB.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServerB.bind((TCP_IP, TCP_PORTB))
threads = []
tcpServer.listen(4)
tcpServerB.listen(4)
while True:
print("Multithreaded Python server : Waiting for connections from TCP clients...")
try:
(conn, (ip,port)) = tcpServer.accept()
except socket.error as e: #(code, msg):
if e.errno != errno.EINTR:
raise
else:
break
address.append((ip,port,conn))
(conn2, (ip2,port2)) = tcpServerB.accept()
address2.append((ip2,port2,conn2))
newthread = ClientThread(conn,ip,port)
newthread.start()
threads.append(newthread)
if EXIT==True:
break
print ("SERVER EXIT")
for t in threads:
t.join()
The code has a signal handler for SIGINT to make the exit cleaner (closing connections, sending a message to the client (still to be implemented) and so on ). The handler writes a global flag EXIT to make the infinite loops terminate.
The code runs both in Python2 and Python3. However there are some problems with SIGINT signal generated by CTRL-C. When there is no client connected the program launched with Python2 exits correctly while the one in Python3 does not. Why this behavioural difference?
Considering only running the program in Python2, when a client connects and I press CTRL-C, the main while exits, like the signal is catched always by the main thread and this interrupts the blocking system call accept. However the other threads do not, I think because of the blocking underlying system call data = self.conn.recv(1024). In C I would block SIGINT signals for one thread and then call pthread_cancel from the other thread. How to exit from all threads when SIGINT is generated in Python?
The client program that for the moment works in Python2 only and suffers from the same problem is:
# Python TCP Client A
import socket
from threading import Thread
class ClientThread(Thread):
def __init__(self,conn):
Thread.__init__(self)
self.conn = conn
def run(self):
while True:
data = self.conn.recv(1024)
print "Ricevuto msg:",data
host = socket.gethostname()
print "host:",host
port = 2004
portB = 2005
BUFFER_SIZE = 2000
tcpClientA = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpClientB = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpClientA.connect(('127.0.0.1', port))
tcpClientB.connect(('127.0.0.1', portB))
newthread = ClientThread(tcpClientB)
newthread.start()
while(True):
msg = raw_input("Inserisci comando: ")
tcpClientA.send (msg)
data = tcpClientA.recv(BUFFER_SIZE)
print "data received:",data
tcpClientA.close()
As for the difference in behavior with accept() in Python 3, look at the full description in the docs. I think this is the key statement:
Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an InterruptedError exception (see PEP 475 for the rationale).
The other problem, stated in your penultimate sentence:
How to exit from all threads when SIGINT is generated in Python 2?
Take a look at the threading documentation:
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.
I'm trying to do a client server exercise in Python and it has to be concurrent, so basically I got a main server and 3 other download servers. These download servers connect just fine to the main one but whenever I want to interact with them I just can't. I tried to cycle through the threads and execute a simple function that sends a different message from the main server to each of the download ones. The problem is that it only sends the message to the last one it cycles through, and it doesn't even send the right message, it sends the message meant to be delivered to the first server to the last one.
So here's my code:
import socket
from threading import Thread
from SocketServer import ThreadingMixIn
import logging
# Multithreaded Python server : TCP Server Socket Thread Pool
class ClientThread(Thread):
def __init__(self,ip,port):
Thread.__init__(self)
self.ip = ip
self.port = port
print "[+] New server socket thread started for " + ip + ":" + str(port)
def run(self):
while True :
data = conn.recv(2048)
print "Server received data:", data
#MESSAGE = raw_input("Multithreaded Python server : Enter Response from Server/Enter exit:")
if not data:
break
if data == 'exit':
break
#conn.send(MESSAGE) # echo
class DServerThread(Thread):
def __init__(self,ip,port):
Thread.__init__(self)
self.ip = ip
self.port = port
def run(self):
while True :
data = conn.recv(2048)
print "Server received data:", data
if not data:
break
if data == 'exit':
break
#conn.send(MESSAGE) # echo
def sendmsg(self,str):
conn.send(str)
# Multithreaded Python server : TCP Server Socket Program Stub
TCP_IP = '0.0.0.0'
TCP_PORT = 2004
sport = 5000
BUFFER_SIZE = 20
dServer=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
dServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
dServer.bind((TCP_IP, sport))
tcpServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServer.bind((TCP_IP, TCP_PORT))
cthreads = [] #client threads
sthreads = [] #download server threads
controlthreads = [] #Threads to manage the background processes
for i in range(1, 4):
dServer.listen(3)
(conn, (ip,port)) = dServer.accept()
newthread = DServerThread(ip,port)
newthread.start()
sthreads.append(newthread)
for t in sthreads:
print t.getName()
t.sendmsg(t.getName())
The problem to me is on that last loop, but I don't quite get why.
TL;DR: Last loop isn't working as intended, it seems to only execute the function on the last iteration and send the message meant for the first thread.
I am new to networking programming and python.
I am trying to figure out how to run different jobs at the server side.
For example, I want one function to create connections for incoming clients but in the same time I can still do some administration work from the terminal.
My code is as below but it doesn't work:
Edited: it doesn't work means it will get stuck in the init_conn() function
Like:
starting up on localhost port 8887
Thread: 0 Connected with 127.0.0.1:48080
# waiting
I am looking into SocketServer framework but don't know how that works.
from thread import *
import socket
def init_conn():
thread_count =0
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 8887)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(10)
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = sock.accept()
print 'Thread: '+ str(thread_count) + ' Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
thread_count +=1
sock.close()
def clientthread(conn):
# receive data from client and send back
def console():
print 'this is console'
option = raw_input('-v view clients')
if option == 'v':
print 'you press v'
def main():
start_new_thread( init_conn(),() )
start_new_thread( console(),() )
if __name__ == "__main__":
main()
Your problem is probably that you start the program, sometimes it prints "this is console" and then it ends.
The first bug is that you call the methods instead of passing the handle to start_new_thread. It must be:
start_new_thread( init_conn, () )
i.e. no () after the function name.
The program doesn't do much because start_new_thread() apparent starts a thread and then waits for it to stop. The documentation is pretty unclear. It's better to use the new threading module; See http://pymotw.com/2/threading/
def main():
t = threading.Thread( target=init_conn )
t.daemon = True
t.start()
console()
so the code will run until console() ends.
I suggest to split the server and the command line tool. Create a client which accepts commands from the command line and sends them to the server. That way, you can start the console from anywhere and you can keep the code for the two separate.
Seeing that you're new to python, have you tried taking a look at the threading module that comes with the standard library?
import threading
... #rest of your code
while conditions==True:
i = threading.Thread(target=init_conn)
c = threading.Thread(target=console)
i.start()
c.start()
Can't say I've done too much with networking programming with python, so I don't really have much to say in that manner, but at least this should get you started with adding multithreading to your project.
Using SocketServer you may implement a client/server system. The documentation gives small examples which may be useful for you. Here is an extended example from there:
server.py :
import SocketServer
import os
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
class MyServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
# By setting this we allow the server to re-bind to the address by
# setting SO_REUSEADDR, meaning you don't have to wait for
# timeouts when you kill the server and the sockets don't get
# closed down correctly.
allow_reuse_address = True
request_queue_size = 10
def __init__(self, port):
self.host = os.uname()[1]
self.port = port
SocketServer.TCPServer.__init__(self, (self.host,self.port), MyTCPHandler)
logging.info( "Server has been started on {h}:{p}".format(h=self.host,p=self.port) )
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
# max length is here 1024 chars
self.data = self.request.recv(1024).strip()
logging.info( "received: {d}".format(d=self.data) )
# here you may execute different functions according to the
# request string
# here: just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
PORT = 8887
if __name__ == "__main__":
# Create the server, binding to localhost on port 8887
#server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server = MyServer( PORT )
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
client.py
import socket
import sys
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
HOST, PORT = "workstation04", 8887
logging.info( "connect to server {h}:{p}".format(h=HOST,p=PORT ) )
# read command line
data = " ".join(sys.argv[1:])
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Connect to server and send data
sock.connect((HOST, PORT))
sock.sendall(data + "\n")
# Receive data from the server and shut down
received = sock.recv(1024)
finally:
sock.close()
logging.info( "Sent: {}".format(data) )
logging.info( "Received: {}".format(received) )
The output looks something like:
server side:
> python server.py
[2015-05-28 11:17:49,263] Server has been started on disasterarea:8887
[2015-05-28 11:17:50,972] received: my message
client side:
[2015-05-28 11:17:50,971] connect to server disasterarea:8887
[2015-05-28 11:17:50,972] Sent: my message
[2015-05-28 11:17:50,972] Received: MY MESSAGE
You can run several clients (from different consoles) in parallel. You may implement a request processor on the server side which processes the incoming requests and executes certain functions.
Alternatively, you may use the python module ParallelPython which executes python code locally on a multicore system or on a cluster and clusters. Check the http examples.
I had to force pip to install this module:
pip install --allow-external pp --allow-unverified pp pp
I'm trying to implement UDP socket's threading.
I want to be able to wait for clients to send me some data in a thread and wait for first datas in an other.
import threading
import socket
class Broker():
def __init__(self):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.sock.bind(('127.0.0.1', 4242))
self.clients_list = []
def talkToClient(self, ip):
self.sock.sendto("ok", ip)
def listen_clients(self):
while True:
msg, client = self.sock.recvfrom(1024)
t = threading.Thread(None, self.talkToClient, None, (client,), None)
b = Broker()
b.listen_clients()
and my client
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
sock.sendto("connection", ('127.0.0.1', 4242))
while True:
msg, b = sock.recvfrom(1024)
print msg
Problem is that my client is never receiving "ok"
Your main problem is that you are not starting the thread that you have created.
t.start()
Should do it. Please make sure you are using four spaces for indentation as well.
I didn't see the error first myself, but once I added some logging statements it was pretty obvious. The code ended up looking like this:
import threading
import socket
import logging
class Broker():
def __init__(self):
logging.info('Initializing Broker')
self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.sock.bind(('127.0.0.1', 4242))
self.clients_list = []
def talkToClient(self, ip):
logging.info("Sending 'ok' to %s", ip)
self.sock.sendto("ok", ip)
def listen_clients(self):
while True:
msg, client = self.sock.recvfrom(1024)
logging.info('Received data from client %s: %s', client, msg)
t = threading.Thread(target=self.talkToClient, args=(client,))
t.start()
if __name__ == '__main__':
# Make sure all log messages show up
logging.getLogger().setLevel(logging.DEBUG)
b = Broker()
b.listen_clients()
I'm afraid you will run into other problems however, because of your threaded solution. Most python modules are not thread-safe by default, unfortunately this is true for the socket module as well. I'm pretty sure that eventually your socket's internal state will be corrupted since you are reading in one thread and writing in another, or potentially in many others since you spawn a new process for each client.
If you look at multi-threaded socket code examples in Python, a socket is usually owned and used by only one thread. The key is to not reuse the listening socket for clients, but to use socket.accept to create a new socket for each client once it has connected.