Develop Reference

Send Continuous Data to Client from Server python

I am writing a program to send the data continuously to the Client from the Server. In here, i am using a timestamp for sample to send it to the multiple Clients who are connected. I have used multi-threading to support multiple Clients. I want the time to be sent every 10 seconds to the client. but in my code, the client stops after receiving the first data. How to make client continuously receive the data. I tried adding while loop in Client Side but it doesn't make it possible. Any suggestions please
Here's the sample Code:
Server Side:
import socket
import os
from threading import Thread
import thread
import threading
import time
import datetime
def listener(client, address):
print "Accepted connection from: ", address
with clients_lock:
clients.add(client)
try:
while True:
data = client.recv(1024)
if not data:
break
else:
print repr(data)
with clients_lock:
for c in clients:
c.sendall(data)
finally:
with clients_lock:
clients.remove(client)
client.close()
clients = set()
clients_lock = threading.Lock()
host = socket.gethostname()
port = 10016
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host,port))
s.listen(3)
th = []
while True:
print "Server is listening for connections..."
client, address = s.accept()
timestamp = datetime.datetime.now().strftime("%I:%M:%S %p")
client.send(timestamp)
time.sleep(10)
th.append(Thread(target=listener, args = (client,address)).start())
s.close()
Client Side:
import socket
import os
from threading import Thread
import socket
import time
s = socket.socket()
host = socket.gethostname()
port = 10016
s.connect((host, port))
print (s.recv(1024))
s.close()
# close the connection
My output:
01:15:10
Required Output on clients:
01:15:10
01:15:20
01:15:30
#and should go on

Server side
while True:
client, address = s.accept()
th.append(Thread(target=listener, args = (client,address)).start())
s.close()
In def listener() change while loop to continuously send data for each thread like this
while True:
data = client.recv(1024)
if data == '0':
timestamp = datetime.datetime.now().strftime("%I:%M:%S %p")
client.send(timestamp)
time.sleep(2)
at client side add this line in while loop to send some data to satisfy the if condition
s.connect((host, port))
while True:
s.send('0')
print(s.recv(1024))
#s.close()

Python: Accept connections into a list, and have the user select a specific socket to write to?

Python: Accept connections into a list, and have the user select a specific socket to write to?
I want to program a simple Python server that accepts incoming connections, stores them into a list, and has a user-interface that lets the user select which socket to write to.
So, while the server accepts new connections and prints received data to screen, the user should have a UI like:
> select 2 # select connection 2
> helloworld # writes 'helloworld' to the socket
How can I do this using threads or subprocesses? (what is the best approach?)
Bonus: How can I do so, that received data don't mess with the UI prompt?
This is my code using select.select():
#!/usr/bin/env python3
import os
import sys
import select
import socket
TCP_IP = '0.0.0.0'
TCP_PORT = 80
BUF_SZ = 1024
sock_list = []
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((TCP_IP,TCP_PORT))
server.listen(10)
inputs = [server]
outputs = []
while True:
read_socks, write_socks, error_socks = select.select(inputs, outputs, inputs)
for sock in read_socks:
if sock is server:
conn,addr = server.accept()
inputs.append(conn)
print('Connection %s' % conn)
else:
data = sock.recv(BUF_SZ)
print(data)
for sock in write_socks:
None

To use threading you can simply wait on socket.accept() and when you receive a new connection handle it using a new thread. A sample code can be like this:
import socket
import sys
import threading
def worker(conn, ip, port, max_buffer_size=4096):
print('accepting connection from ip={} and port={}'.
format(ip, port, conn))
data = conn.recv(max_buffer_size)
# Do something
conn.close()
def init_server():
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind((HOST, PORT))
server_socket.listen(PARALLEL_CONNECTIONS)
return server_socket
def run():
try:
s = init_server()
print('successfully created server socket.')
except Exception as e:
print('failed to create socket because {}'.format(e))
sys.exit()
while True:
clientsocket, addr = s.accept()
ip, port = str(addr[0]), str(addr[1])
t = threading.Thread(target=worker, args=(clientsocket, ip, port,))
t.start()

Server process started by multiprocessing.Manager() makes piped socket not closed immediately

I have following code, server accept net connection, pass it to child to process with Manager().Queue():
q = Manager().Queue()
class Server:
def run(self, host, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(1)
print('parent', os.getpid())
while True:
c, _ = s.accept()
q.put(c)
c.close()
def handle_request():
print('child', os.getpid())
while True:
c = q.get()
time.sleep(1)
print(c.recv(4))
c.close()
Process(target=handle_request, args=()).start()
Server().run('127.0.0.1', 10000)
close doesn't work as expected, I think it is because Manager's server process sill have a reference on that socket, lsof -i confirmed. How to deal with this? I found there is not a way to close the socket in Manager process, shutdown could do the trick but not what I want.

Interesting problem.
I am not sure if this is of any help, but I found your code somewhat odd in the beginning, as sending socket objects using Manager().Queue() to another process does not sound like it is supported. It may be, but sending a file descriptor to another process needs a couple of hoops. I changed your code a bit to do it as I would do it - basically reducing and reconstructing handles.
from multiprocessing import Manager, Process
from multiprocessing.reduction import reduce_handle, rebuild_handle
import socket
import os
from time import sleep
q = Manager().Queue()
class Server:
def run(self, host, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))
s.listen(1)
print('parent', os.getpid())
while True:
c, _ = s.accept()
foo = reduce_handle(c.fileno())
q.put(foo)
c.close()
def handle_request():
print('child', os.getpid())
while True:
bar = q.get()
sleep(1)
barbar = rebuild_handle(bar)
c = socket.fromfd(barbar, socket.AF_INET, socket.SOCK_STREAM)
print(c.recv(4))
c.shutdown(socket.SHUT_RDWR)
Process(target=handle_request, args=()).start()
Server().run('127.0.0.1', 10000)
This does not leave any sockets behind in CLOSE_WAIT at least when I ran it, and it works as I would expect it to work.

an empty file after sending it to tcp server in python

I am new to python and i am trying to make a multithreded tcp server and client to be able to send files between them. I did write some simple codes for these two programs but every time I get empty file on server's site. The file does create in the folder but when I open it it is blank inside. I also tried to send .png files but windows photoviewer doesn't open them saying they are empty. I didn't find anyone encourting such problem so that's why i am asking
Client.py
import socket # Import socket module
HOST = "localhost" # Host address / name
PORT = 2137 # Reserves port for the service
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((HOST, PORT))
fileToSend = open('test.txt', 'rb')
print "File ready to be sent"
l = fileToSend.read(1024)
while l:
print "Sending the file"
client.send(l)
l = fileToSend.read(1024)
fileToSend.close() print "done"
client.close()
Server.py
import socket
from threading import Thread
from SocketServer import ThreadingMixIn
import sys
TCPHOST = "localhost"
TCPPORT = 2137
BUFFER_SIZE = 20
class ClientThread(Thread):
def __init__(self, HOST, PORT):
Thread.__init__(self)
self.HOST = HOST
self.PORT = PORT
print "New thread started for " + HOST + " on port " + str(PORT)
def run(self):
f = open('received.py', 'wb')
while True:
try:
data = conn.recv(1024)
except socket.error, e:
print "Error receiving data: %s" % e
sys.exit(1)
while data:
print "Receiving"
f.write(data)
data = conn.recv(1024)
f.close()
try:
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind((TCPHOST, TCPPORT))
print "Socket created"
except socket.error, err:
print "Failed to create socket" % err
threads = []
while True:
server.listen(4)
print "Waiting for connections"
(conn, (HOST, PORT)) = server.accept()
thread = ClientThread(HOST, PORT)
thread.start()
threads.append(thread)
for t in threads:
t.join()

I am not sure what you actually want to do, because I see that you import SocketServer however you are not using it all.
If you are trying to run a simple socket server then the class ClientThread and all the other stuff about threads in that file are not necessary.
The following code in server.py will do the job
import socket
import sys
TCPHOST = "localhost"
TCPPORT = 2137
BUFFER_SIZE = 20
try:
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind((TCPHOST, TCPPORT))
server.listen(4)
print "Socket created"
except socket.error, err:
print "Failed to create socket" % err
while True:
print "Waiting for connections"
(conn, (TCPHOST, TCPPORT)) = server.accept()
try:
while True:
data = conn.recv(1024)
f = open('received.py', 'wb')
if data:
print "Receiving " + data
f.write(data)
else:
f.close()
break;
except socket.error, e:
#pass
print "Error receiving data: %s" % e
#sys.exit(1)
finally:
conn.close()
However if you are trying to implement a threaded TCPServer using the ThreadingMixIn then you need to create a class that subclasses SocketServer and override its handle() function
Python documentation is quite helpful on this
https://docs.python.org/3.5/library/socketserver.html
(ThreadingMixin is at the bottom of the page)

When combine TCPServer with ThreadingMixIn, it block

the server-side code (tcp_server.py):
from SocketServer import TCPServer, ThreadingMixIn, StreamRequestHandler
class Server(ThreadingMixIn, TCPServer):
pass
class Handler(StreamRequestHandler):
def handle(self):
print 'got a connection from: ', self.request.getpeername()
print self.rfile.read(1024)
msg = 'hello'
self.wfile.write(msg)
server = Server(('127.0.0.1', 8888), Handler)
server.serve_forever()
the client-side code (tcp_client.py):
from socket import *
import threading
def do_connection():
s = socket(AF_INET, SOCK_STREAM)
s.connect(('127.0.0.1', 8888))
s.sendall('this is client')
print s.recv(1024)
ts = []
for x in xrange(100):
print x
t = threading.Thread(target=do_connection())
t.daemon = True
ts.append(t)
for t in ts:
t.start()
I runned tcp_server.py, and then tcp_client.py. tcp_client.py should have been over soon. However, tcp_client.py seemed just run only one thread and blocked, and tcp_server.py got only one connection. When I interrupted tcp_client.py，tcp_server.py got one message this is client。
Is there any mistake in my code ?

This line:
t = threading.Thread(target=do_connection())
Should be
t = threading.Thread(target=do_connection)
When you use do_connection(), you end up executing do_connection in the main thread, and then pass the return value of that call to the Thread object. What you want to do is pass the do_connection function object to Thread, so that the Thread object can execute do_connection in a new thread when you call t.start.
Also, note that starting 100 threads concurrently to connect to your server may not perform very well. You may want to consider starting with fewer threads, and working your way up to a higher number once you know things are working properly.

because the server is blocked by the first request, I try to change the read(1024) to
readline in the server.py and add a '\n' to the content sended from the client, it
works.
it seems the rfile.read(1024) will block the how process, so the goodway is use readline
or use the self.request.recv(1024)
server.py
from SocketServer import TCPServer, ThreadingMixIn, StreamRequestHandler
class Server(ThreadingMixIn, TCPServer):
pass
class Handler(StreamRequestHandler):
def handle(self):
print 'got a connection from: ', self.request.getpeername()
print self.rfile.readline()
#print self.request.recv(1024).strip()
msg = 'hello'
self.wfile.write(msg)
# Create the server, binding to localhost on port 9999
server = Server(("127.0.0.1", 8888), Handler)
server.serve_forever()
client.py
from socket import *
import threading
def do_connection():
print "start"
s = socket(AF_INET, SOCK_STREAM)
s.connect(('127.0.0.1', 8888))
s.sendall('this is client\n')
print s.recv(1024)
ts = []
for x in xrange(100):
print x
t = threading.Thread(target=do_connection)
ts.append(t)
for t in ts:
print "start t"
t.start()