Develop Reference

Persistent socket connection in Lua/Python

I'm trying to create a persistent socket connection between a Lua client and Python server. Effectively a script that'll constantly ping the server with keepalive messages
My current issue is that the socket closes after each connection without a means to reopen it for transmission.
Lua Client:
local HOST, PORT = "localhost", 9999
local socket = require('socket')
-- Create the client and initial connection
client, err = socket.connect(HOST, PORT)
client:setoption('keepalive', true)
-- Attempt to ping the server once a second
start = os.time()
while true do
now = os.time()
if os.difftime(now, start) >= 1 then
data = client:send("Hello World")
-- Receive data from the server and print out everything
s, status, partial = client:receive()
print(data, s, status, partial)
start = now
end
end
Python Server:
import socketserver
class MyTCPHandler(socketserver.BaseRequestHandler):
def handle(self):
self.data = self.request.recv(1024).strip()
print("{} wrote".format(self.client_address[0]))
print(self.data)
print(self.client_address)
# Send back some arbitrary data
self.request.sendall(b'1')
if __name__ == '__main__':
HOST, PORT = "localhost", 9999
# Create a socketserver and serve is forever
with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server:
server.serve_forever()
The expected result is a keepalive ping every second to ensure the client is still connected to the server.

I ended up finding a solution.
The problem seems to have been with the socketserver library in Python. I switched it to raw sockets and things began working how I wanted them to. From there I created threads to handle the back and forth in the background
Python Server:
import socket, threading
HOST, PORT = "localhost", 9999
# Ensures the connection is still active
def keepalive(conn, addr):
print("Client connected")
with conn:
conn.settimeout(3)
while True:
try:
data = conn.recv(1024)
if not data: break
message = data.split(b',')
if message[0] == b'ping':
conn.sendall(b'pong' + b'\n')
except Exception as e:
break
print("Client disconnected")
# Listens for connections to the server and starts a new keepalive thread
def listenForConnections():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind((HOST, PORT))
while True:
sock.listen()
conn, addr = sock.accept()
t = threading.Thread(target=keepalive, args=(conn, addr))
t.start()
if __name__ == '__main__':
# Starts up the socket server
SERVER = threading.Thread(target=listenForConnections)
SERVER.start()
# Run whatever code after this
The Lua client didn't change in this scenario

Send messages received by server to multiple clients in python 2.7 with socket programming

So I have created a socket program for both client and server as a basic chat. I made it so the server accepts multiple clients with threading, so that is not the problem. I am having trouble sending messages to each client that is connected to the server. I am not trying to have the server send a message it created but rather have client1 sending a message to client2 by going through the server. For some reason it will only send it back to client1.
For example, client1 will say hello and the server will send the same message back to client1 but nothing to client2. I fixed this slightly by making sure the client doesn't receive its own message but client2 is still not receiving the message from the client1.
Any help will be appreciated.
I have tried multiple changes and nothing seems to work. You can look at my code for specifics on how I did things but ask if there are any questions.
Also, there is a question where someone has asked that is similar and I thought it would give me an answer but the responses stopped going through and a solution was never fully given, so please don't just refer me to that question. that is located here: Python 3: Socket server send to multiple clients with sendto() function.
Here's the code:
CLIENT:
import socket
import sys
import thread
#Create a socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Enter username to identify self to others
name = raw_input("Enter username: ") + ": "
#Connect socket to ip and port
host = socket.gethostname()
#host = '192.168.1.10'
server_address = (host, 4441)
sock.connect(server_address)
#function waiting to receive and print a message
def receive(nothing):
while True:
data = sock.recv(1024)
if message != data:
print data
# Send messages
while True:
#arbitrary variable allowing us to have a thread
nothing = (0, 1)
message = name + raw_input("> ")
sock.sendall(message)
#thread to receive a message
thread.start_new_thread(receive, (nothing,))
SERVER:
import socket
import sys
import thread
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
host = socket.gethostname()
server_address = (host, 4441)
sock.bind(server_address)
#Listen for incoming connections
sock.listen(5)
print "Waiting for connection..."
#Variable for the number of connections
numbOfConn = 0
#Name of list used for connections
addressList = []
#Function that continuosly searches for connections
def clients(connection, addressList):
while True:
message = connection.recv(1024)
print message
#connection.sendall(message)
#for loop to send message to each
for i in range(0,numbOfConn - 1):
connection.sendto(message, addressList[i])
connection.close()
while True:
#accept a connection
connection, address = sock.accept()
print 'Got connection from', address
numbOfConn += 1
addressList.append((address))
#Thread that calls the function: clients and stores them in a tuple called connection
thread.start_new_thread(clients, (connection, addressList))
sock.close()
Please help me if you can!
EDIT:
I was able to fix it to a certain extent. It is still a little buggy but I am able to send messages back and forth now. I needed to specify the connection socket as well as the address. Here's the updated code:
SERVER
import socket
import sys
import thread
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
host = socket.gethostname()
server_address = (host, 4441)
sock.bind(server_address)
#Listen for incoming connections
sock.listen(5)
print "Waiting for connection..."
#Variable for the number of connections
numbOfConn = 0
#Name of list used for connections
addressList = []
connectionList = []
#Function that continuosly searches for connections
def clients(connectionList, addressList):
while True:
for j in range(0,numbOfConn):
message = connectionList[j].recv(1024)
print message
#for loop to send message to each
for i in range(0,numbOfConn):
connectionList[i].sendto(message, addressList[i])
connection.close()
while True:
#accept a connection
connection, address = sock.accept()
print 'Got connection from', address
numbOfConn += 1
addressList.append((address))
connectionList.append((connection))
#Thread that calls the function: clients and stores them in a tuple called connection
thread.start_new_thread(clients, (connectionList, addressList))
sock.close()

How to send and receive from the same socket in Python?

I'm am trying to write a client program in Python that can send and receive from the same socket, but it is always giving me the same error which address is already in use. Here is the function I'm trying to write.
def Login():
username=raw_input()
password=raw_input()
message=raw_input()
array=[username,password,message]
TCP_IP = '127.0.0.1'
TCP_PORT = 5563
BUFFER_SIZE = 1024 # Normally 1024, but we want fast response
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((TCP_IP, TCP_PORT))
array_string=pickle.dumps(array)
sock.send(array_string)
sock.close()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((TCP_IP, TCP_PORT))
sock.listen(1)
conn, info = sock.accept()
while 1:
data = serverSocket.recv(1024)
if not data:break
conn.send(data)
conn.close()

There is a bunch of truly newbie errors here.
You can't ever connect a TCP socket to itself. There must be two different sockets.
If you really want to get the data you sent earlier at a listening socket, this listening socket must be created, bound and configured to listen before the client side connects (or, at least, in parallel to this connect attempt, in a few seconds, so the connect attempt will try - but this very likely won't work on localhost).
You can't wait on connect and on accept in the same thread if both are blocking. The simplest approach is to separate the client side and the server side to 2 different programs and run them manually in parallel. Then, after successful debugging, you will be able to do this in different threads of the same process, or using an event-driven engine.

While you may not be able to connect a socket to itself to send and receive data, you might be able to learn from the following example inspired by your code that attempts to do something similar.
import _thread
import pickle
import socket
import time
def main():
"""Run a server in a thread and start a client to talk to it."""
_thread.start_new_thread(run_server, ('', 5563))
run_client('localhost', 5563)
def run_server(host, port):
"""Handle all incoming connections by spawning worker threads."""
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((host, port))
server.listen(5)
while True:
_thread.start_new_thread(handle_connection, server.accept())
def handle_connection(client, address):
"""Answer an incoming question from the connected client."""
print('Incoming connection from', address)
client.settimeout(0.1)
data = recvall(client)
client.shutdown(socket.SHUT_RD)
question = pickle.loads(data)
answer = '''len(username) = {}
len(password) = {}
len(message) = {}'''.format(*map(len, question))
client.sendall(answer.encode())
client.shutdown(socket.SHUT_WR)
client.close()
print('Finished with', address)
def recvall(connection):
"""Receive all data from a socket and return as a bytes object."""
buffer = bytearray()
while True:
try:
data = connection.recv(1 << 12)
except socket.timeout:
pass
else:
if data:
buffer.extend(data)
else:
return bytes(buffer)
def run_client(host, port):
"""Collect information from question and display returned answer."""
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
time.sleep(0.1) # wait for server to start listening for clients
client.connect((host, port))
time.sleep(0.1) # wait for handler thread to display connection
username = input('Username: ')
password = input('Password: ')
message = input('Message: ')
question = pickle.dumps((username, password, message))
client.sendall(question)
client.shutdown(socket.SHUT_WR)
answer = recvall(client)
client.shutdown(socket.SHUT_RD)
client.close()
print(answer.decode())
time.sleep(0.1) # wait for handler to cleanly terminate execution
if __name__ == '__main__':
main()

Why does socket close when sending message over socket?

Why does my connection stop after 3 print statements if I have the conn.send() line? If that line is commented out, the connection stays open indefinitely. It is hitting the exception for some reason, but I don't know why and I am inexperienced with python.
server.py:
import random
import signal
import socket
import struct
import sys
import time
PORT = 1234
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', PORT))
s.listen(1)
print("Server started on port %u" % PORT)
try:
while True:
(conn, addr) = s.accept()
conn.setblocking(0)
print("Client connected: %s:%d" % addr)
while True:
print "hey"
conn.send("random")
time.sleep(1)
except:
s.close()
print "exception"
client.py:
import socket # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 1234 # Reserve a port for your service.
s.connect((host, port))
print s.recv(1024)
s.close

The reason for this behavior is the socket is actually closed after the clients first recv(), your server just doesn't realize it until the third attempt at send(). Calling close() in your client didn't change anything because previously the program was terminating which closed the socket anyway. This thread explains why this is very likely what's happening. To test this theory, you could try having the client sleep (or select) and print more recv() calls, before closing the socket.

How to keep a socket open until client closes it?

I have simple python server and client.
Server:
import SocketServer
import threading
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.data = self.request.recv(1024).strip()
print str(self.client_address[0]) + " wrote: "
print self.data
self.request.send(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 3288
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server.serve_forever()
Client:
import socket
import sys
from time import sleep
HOST, PORT = "localhost", 3288
data = "hello"
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((HOST, PORT))
sock.send(data + "\n")
received = sock.recv(1024)
sleep(10)
sock.send(data + "\n")
received = sock.recv(1024)
sleep(10)
sock.send(data + "\n")
received = sock.recv(1024)
finally:
sock.close()
Here is the output I get:
Server:
>python server.py
127.0.0.1 wrote:
hello
Client:
>python client.py
Traceback (most recent call last):
File "client.py", line 18, in <module>
received = sock.recv(1024)
socket.error: [Errno 10053] An established connection was aborted by the software in your host machine
I tried it on a linux machine as well. The server only receives one message and then I get an error on the recv statement of second message. I have just started learning networking on python but I think the server is closing the socket for some reason. How do I correct this?

A MyTcpHandler object is created for each connection, and handle is called to deal with the client. The connection is closed when handle returns, so you have to handle the complete communication from the client within the handle method:
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
while 1:
self.data = self.request.recv(1024)
if not self.data:
break
self.data = self.data.strip()
print str(self.client_address[0]) + " wrote: "
print self.data
self.request.send(self.data.upper())
NOTE: recv returns '' when the client closes the connection, so I moved .strip() after the recv so there is no false alarm due to the client sending only white space.

I'll first admit that it's been years since I last used SocketServer, so there might be more idiomatic approaches to solve your problem.
Note that your client opens a single connection and sends three sets of data and receives three sets of data. (Hopefully the TCP stack will send buffered data once you call receive() on the socket.)
Your server is expecting to handle a client connection completely, from start to finish, when it is called from the SocketServer callback mechanism. Your current class does a little bit of IO and then quits. You just need to extend your server callback to do more:
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.data = self.request.recv(1024).strip()
print str(self.client_address[0]) + " wrote: "
print self.data
self.request.send(self.data.upper())
foo = self.request.recv(1024).strip()
self.request.send(foo.lower())
bar = self.request.recv(1024).strip()
self.request.send("goodbye " + bar)

TO a similar problem here error: [Errno 10053]
I also tried the same thing and got the same error.
If there is a simple code like this to demonstrate this error:
import socket
host = 'localhost'
port = 5001
size = 102400
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
for msg in ['Hello, world','Test','anything goes here']:
s.send(msg)
data = s.recv(size)
print 'Received:', data
s.close()
If you create a socket object and the amt it can send and echo back from server to see how much it receivers, if you vary that, say 1024 to 102400(in this code);
Which means the socket should not get closed but again in my Windows OS, the server side keeps listening and printing any data that client sends but on the Client side you get this error;
However if it is that the client can connect only once and send and receive only once, then that is how it was designed. Trying this works without any errors:
for msg in ['Hello, world','Test','anything goes here']:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
s.send(msg)
data = s.recv(size)
s.close()
print 'Received:', data
I am not sure if one socket object works only once to send and recieve data.
UPDATE
I think the issue was the capacity per client socket to receive data as per the buffersize fixed;
That's why the second code snippet above works thus creating new client connection sockets on the server. But that way lots of sockets are going to get used up.
Instead the following code fixed that problem by checking the amt of size being used up. If it exceeds the given amount, it creates a new socket at clients' but makes sure the message is sent; Actually the problem was with the server code but fixed it.
size = 10
This is a quick baby attempt at the code. I am sure you would understand and optimize it for the better!
client code:
messag = ['Hello, world', 'Test', 'anything goes here']
def client_to_server(messag,host,port,size):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
countmsg = 0
restmsg = ''
for msg in messag:
strl = tmsg = msg
if len(restmsg):
tmsg = restmsg + ' ' + msg
countmsg = len(tmsg)
if countmsg <= size:
pass
else:
restmsg = tmsg[size:]
tmsg = tmsg[:size]
#s.close()
countmsg = len(tmsg)
#s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#s.connect((host, port))
print 'Sending to server msg {}'.format(tmsg)
s.send(tmsg)
# s.settimeout(1)
try:
data = s.recv(size)
print 'Received:', data
if strl == data:
print strl,data
countmsg = 0
restmsg = ''
except (socket.error), e:
print e.args,e.message
s.close()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.close()
if restmsg:
client_to_server([restmsg],host,port,size)
return
client_to_server(messag,host,port,size)
Server Code:
size = 1024 #This has to be bigger than client buf size!
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(backlog)
while True:
#this is what accepts and creates a P2P dedicated client socket per socket
client, address = s.accept()
try:
data = client.recv(size)
while data or 0:
print "Client sent {} | Server sending data to client address {}".format(data, address)
client.send(data)
data = client.recv(size)
else: client.close()
except (socket.error), e:
client.close()
print e.args, e.message
Try it out. This uses the same socket.