I'm teaching myself Python networking, and I recalled that back when I was teaching myself threading, I came across this page, so I copied the scripts, updated them for Python 3.1.1 and ran them. They worked perfectly.
Then I made a few modifications. My goal is to do something simple:
The client pickles an integer and sends it to the server.
The server receives the pickled integer, unpickles it, doubles it, then pickles it and sends it back to the client.
The client receives the pickled (and doubled) integer, unpickles it, and outputs it.
Here's the server:
import pickle
import socket
import threading
class ClientThread(threading.Thread):
def __init__(self, channel, details):
self.channel = channel
self.details = details
threading.Thread.__init__ ( self )
def run(self):
print('Received connection:', self.details[0])
request = self.channel.recv(1024)
response = pickle.dumps(pickle.loads(request) * 2)
self.channel.send(response)
self.channel.close()
print('Closed connection:', self.details [ 0 ])
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('', 2727))
server.listen(5)
while True:
channel, details = server.accept()
ClientThread(channel, details).start()
And here is the client:
import pickle
import socket
import threading
class ConnectionThread(threading.Thread):
def run(self):
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 2727))
for x in range(10):
client.send(pickle.dumps(x))
print('Sent:',str(x))
print('Received:',repr(pickle.loads(client.recv(1024))))
client.close()
for x in range(5):
ConnectionThread().start()
The server runs fine, and when I run the client it successfully connects and starts sending integers and receiving them back doubled as expected. However, very quickly it exceptions out:
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Python30\lib\threading.py", line 507, in _bootstrap_inner
self.run()
File "C:\Users\Imagist\Desktop\server\client.py", line 13, in run
print('Received:',repr(pickle.loads(client.recv(1024))))
socket.error: [Errno 10053] An established connection was aborted by the softwar
e in your host machine
The server continues to run and receives connections just fine; only the client crashes. What's causing this?
EDIT: I got the client working with the following code:
import pickle
import socket
import threading
class ConnectionThread(threading.Thread):
def run(self):
for x in range(10):
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 2727))
client.send(pickle.dumps(x))
print('Sent:',str(x))
print('Received:',repr(pickle.loads(client.recv(1024))))
client.close()
for x in range(5):
ConnectionThread().start()
However, I still don't understand what's going on. Isn't this just opening and closing the socket a bunch of times? Shouldn't there be time limitations to that (you shouldn't be able to open a socket so soon after closing it)?
Your client is now correct - you want to open the socket send the data, receive the reply and then close the socket.
The error original error was caused by the server closing the socket after it sent the first response which caused the client to receive a connection closed message when it tried to send the second message on the same connection.
However, I still don't understand
what's going on. Isn't this just
opening and closing the socket a bunch
of times?
Yes. This is acceptable, if not the highest performance way of doing things.
Shouldn't there be time
limitations to that (you shouldn't be
able to open a socket so soon after
closing it)?
You can open a client socket as quickly as you like as every time you open a socket you will get a new local port number, meaning that the connections won't interfere. In the server code above, it will start a new thread for each incoming connection.
There are 4 parts to every IP connection (source_address, source_port, destination_address, destination_port) and this quad (as it is known) must change for ever connection. Everything except source_port is fixed for a client socket so that is what the OS changes for you.
Opening server sockets is more troublesome - if you want to open a new server socket quickly, your
server.bind(('', 2727))
Above then you need to read up on SO_REUSEADDR.
Related
So I'm studying Python from the book Fundamentals of Python by Kenneth Lambert and I'm having trouble with an error from one of the programs in the book.
Here in chapter 10 talks about Clients and Servers. My professor asked us to key those programs in Python to see how they work. The first programs worked perfectly, but in a program I'm getting an error which seems to be a Windows error instead of a Python one.
This is the program in page 339:
from socket import *
from time import ctime
from threading import Thread
class ClientHandler(Thread):
"""Handles a client request."""
def __init__(self, client):
Thread.__init__(self)
self._client = client
def run(self):
self._client.send(bytes(ctime() + '\nHave a nice day!' , 'ascii'))
self._client.close()
HOST = "localhost"
PORT = 5000
BUFSIZE = 1024
ADDRESS = (HOST, PORT)
server = socket(AF_INET, SOCK_STREAM)
server.bind(ADDRESS)
server.listen(5)
# The server now just waits for connections from clients
# and hands sockets off to client handlers
while True:
print('Waiting for connection')
client, address = server.accept()
print('...connected from:', address)
handler = ClientHandler(client)
handler.start()
When I run this program, it displays the "Waiting for connection" message in the Shell. However, when I try to connect to the program using the Command Prompt, it displays the following error:
C:\Python33>python multi-client-server.py
Traceback (most recent call last):
File "multi-client-server.py", line 30, in <module>
server.bind(ADDRESS)
OSError: [WinError 10048] Only one usage of each socket address (protocol/networ
k address/port) is normally permitted
We haven't studied this in class a lot. So I'm just wondering why this happens and how to fix it.
Thanks!
So, as per your questions:
We haven't studied this in class a lot. So I'm just wondering why this
happens and how to fix it.
Why:
You are trying to run the same code snippet from two different CMD on a Windows OS. So, when you initially execute the code snippet, the server starts listening on the port number 5000, then when you execute the same code snippet from the second CMD wndow it conflicts with the socket that is already being used by the first one.
I tested this on Windows 8.
How to fix:
To fix this issue, you have to simply use a different port number when you execute the code snippet for the second time, so that the socket(IP+port) doesn't conflicts with the previous one. Simply edit your code and put PORT = 15200 and save this file with a different name.(I have provided the code below too.) Now try executing the first code snippet file from a CMD windows and then execute the second code snippet file that you created right now from the second CMD window. The issue will be solved!
Code:
from socket import *
from time import ctime
from threading import Thread
class ClientHandler(Thread):
"""Handles a client request."""
def __init__(self, client):
Thread.__init__(self)
self._client = client
def run(self):
self._client.send(bytes(ctime() + '\nHave a nice day!' , 'ascii'))
self._client.close()
HOST = "localhost"
PORT = 15200 # Port number was changed here
BUFSIZE = 1024
ADDRESS = (HOST, PORT)
server = socket(AF_INET, SOCK_STREAM)
server.bind(ADDRESS)
server.listen(5)
# The server now just waits for connections from clients
# and hands sockets off to client handlers
while True:
print('Waiting for connection')
client, address = server.accept()
print('...connected from:', address)
handler = ClientHandler(client)
handler.start()
If you prefer then have a look here for the basic client-server issues.
I am writing a UDP server application that serves as a back end to Teltonika FMB630 car mounted devices.
I already took care of the protocol specifics and decoding, the problem I am facing relates to the UDP socket used.
My UDP server has to send an acknowledgement to the client device upon receiving a message (that is the protocol), however, if I send those ACKs, the server socket stops receiving data after a while.
The server's UDP socket object is passed to an concurrent.futures.ThreadPoolExecutor that fires a function (send_ack) that sends the ACK, however this is not the issue because I tried calling send_ack in the main thread, after receiving data and the same issue occurs.
I suspect the problem is the remote device somehow breaks the connection or the ISP or MNO doesn't route the reply packet (this is a GPRS device) and then the socket.send() method that is used to send the acknowledge, somehow freezes other socket operations, specifically recvfrom_into called in the main thread loop.
I wrote two scripts to illustrate the situation:
udp_test_echo.py :
#!/usr/env/bin python
import socket
import concurrent.futures
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.connect(addr)
print("connected to {}".format(addr))
sock.send(ack)
print("ACK sent to {}".format(addr))
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 1337))
data = bytearray([0] * 10)
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
while True:
print("listening")
nbytes, address = s.recvfrom_into(data)
print("Socket Data received {} bytes Address {}".format(nbytes, address))
print("Data received: ", data, " Echoing back to client")
executor.submit(send_ack, s, address, data[:nbytes])
udp_test_client.py:
#!/usr/env/bin python
import socket
import time
import random
def get_random_bytes():
return bytearray([random.randint(0,255) for b in range(10)])
ip = "127.0.0.1"
port = 1337
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect((ip, port))
while True:
stuff_to_send = get_random_bytes()
print("Sending stuff", stuff_to_send)
s.sendall(stuff_to_send)
print("reply: ", s.recvfrom(10))
time.sleep(0.1)
Running udp_test_echo.py in one terminal and udp_test_client.py in another, we see normal operation but if you Ctrl+C the test client and re run it, you will see that the server doesn't respond until it is restarted.
Is there a way to timeout a specific sending operation from a specific call to socket.send() method without affecting other calls ? (I want my socket.recvfrom_into call to block on the main thread)
If I settimeout on the entire socket object, I am going to have to deal with many exceptions while waiting for data in the main thread and I don't like to have to rely on exceptions for proper program operation.
The culprit was the socket.connect() call in send_ack, when being called on the server's socket object it causes the socket to no longer be bound and listen on the port specified in the start of the program.
Instead the send_ack function was changed to be:
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.sendto(ack, addr)
print("ACK sent to {}".format(addr))
socket.sendto(data, address) uses the existing connection instead of starting a new one.
I have a client-server model where the client will constantly checking a log file and as soon as a new line comes in the log file it sends that line to the server.
Somehow I managed to work this thing using the following code.
server.py
import SocketServer
class MyTCPSocketHandler(SocketServer.BaseRequestHandler):
def handle(self):
# self.request is the TCP socket connected to the client
data = self.request.recv(1024).strip()
print data
# process the data..
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPSocketHandler)
server.serve_forever()
client.py
import time
import socket
def follow(thefile):
thefile.seek(0, 2)
while True:
line = thefile.readline()
if not line:
time.sleep(0.1)
continue
yield line
def connect_socket():
HOST, PORT = "localhost", 9999
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
return sock
if __name__ == '__main__':
logfile = open("my_log.log")
loglines = follow(logfile)
for line in loglines:
sock = connect_socket()
# send data
sock.sendall(bytes(line))
the problem is every time I need to call the connect_socket() method to send a new line.
I'm quite new to this topic so somebody please let me know is there any workaround for this to work in a such a way that once the connection is established between client and server I need to send data continuously to the server without making a new connection again and again.
If I'm connecting only one time and using the same socket object to send data it was throwing
socket.error: [Errno 32] Broken pipe
Some StackOverflow links which I have followed are given below,
1, 2, 3
One thing I found is
Broken Pipe occurs when one end of the connection tries sending data while the other end has already closed the connection.
How can I keep the connection open on both ends?
For this use case should I go for an asynchronous method and if so which framework will be the best match tornado or twisted?
After a line is transmitted, you close the connection on the client, but don't close it on the server.
From the docs:
RequestHandler.finish()
Called after the handle() method to perform any clean-up actions required. The default implementation does nothing.
So you should implement finish as well and close the socket (self.request) there.
Another option is not to close the connection on the client:
sock = connect_socket()
for line in loglines:
# send data
sock.sendall(bytes(line))
sock.sendall(b'bye') # EOF
sock.close()
However in this case you should modify the server code and make sure it can serve multiple clients and it understands when to close the socket. With all these difficulties it is nevertheless the preferable way of doing things. Ideally, a client has a single TCP connection per session for they're costly to establish.
I am new to python and I am using socketserver to try to create a server that broadcasts all the received message from one client to all the connected clients, but facing a problem. I get the following error in the end:
Exception happened during processing of request from ('127.0.0.1', 14872)
Traceback (most recent call last):
File "C:\Users\umair\AppData\Local\Programs\Python\Python35-32\lib\socketserver.py", line 628, in process_request_thread
self.finish_request(request, client_address)
File "C:\Users\umair\AppData\Local\Programs\Python\Python35-32\lib\socketserver.py", line 357, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\umair\AppData\Local\Programs\Python\Python35-32\lib\socketserver.py", line 684, in init
self.handle()
File "C:\Users\umair\Desktop\socketserverthread_server.py", line 52, in handle
clients.send(data)
OSError: [WinError 10038] An operation was attempted on something that is not a socket
My server code is as follows:
import socketserver
import threading
host = '127.0.0.1'
port = 6666
all_clients = []
class ThreadingHandler (socketserver.BaseRequestHandler):
def handle(self):
if (self.request) not in all_clients:
all_clients.append(self.request)
data = self.request.recv(1024)
print('%s writes: ' % str(self.client_address), end = " ")
print(data.decode())
for clients in all_clients:
clients.send(data)
class ThreadingServer (socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
myserv = ThreadingServer((host, port), ThreadingHandler)
t = threading.Thread(target = myserv.serve_forever)
t.setDaemon(True)
t.start()
print('The server is online')
The client code is:
import socket
host = '127.0.0.1'
port = 6666
while True:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
data = input('You: ')
sock.send(data.encode())
received = sock.recv(1024)
print('Received: ', received.decode())
I am running two of these client codes to test it. The first message from a client doesn't get broadcast, and is just received by the same client. On sending the second message, I receive the above mentioned error. Is it the problem due to the fact that I am creating a new socket in every cycle? I have to create the socket in the loop because if I don't, then I cannot send and receive continuously. I think the socket object is destroyed after a request and a response. I have no idea what's going on. So any help is appreciated. Thank you.
On sending the second message, I receive the above mentioned error. Is
it the problem due to the fact that I am creating a new socket in
every cycle? … I think the socket
object is destroyed after a request and a response. I have no idea
what's going on.
You do have an idea what's going on; indeed the client's socket object sock from a previous loop cycle is destroyed at the time when the new socket is assigned to the variable, which causes the old socket to be closed. But independently thereof the server already closes its request socket due to the use of socketserver, just after one request has been handled, i. e. after your handle(self) returns. The above mentioned error arises from the omission to take this closed socket object in all_clients into account, which could be done e. g. so:
for clients in all_clients[:]:
if clients._closed: all_clients.remove(clients)
else: clients.send(data)
The first message from a client doesn't get broadcast, and is just
received by the same client.
The message does get broadcast (sent to each client), it's just that the other client doesn't bother about receiving it, because it is waiting in input('You: ').
I have to create the socket in the loop because if I don't, then I
cannot send and receive continuously.
That's only true in a sense because you use socketserver, which closes the connection after each request - not quite helpful in case of a chat server.
See the question "Handle multiple requests with select" for an example server without the hassle brought by socketserver et al.
So I'm studying Python from the book Fundamentals of Python by Kenneth Lambert and I'm having trouble with an error from one of the programs in the book.
Here in chapter 10 talks about Clients and Servers. My professor asked us to key those programs in Python to see how they work. The first programs worked perfectly, but in a program I'm getting an error which seems to be a Windows error instead of a Python one.
This is the program in page 339:
from socket import *
from time import ctime
from threading import Thread
class ClientHandler(Thread):
"""Handles a client request."""
def __init__(self, client):
Thread.__init__(self)
self._client = client
def run(self):
self._client.send(bytes(ctime() + '\nHave a nice day!' , 'ascii'))
self._client.close()
HOST = "localhost"
PORT = 5000
BUFSIZE = 1024
ADDRESS = (HOST, PORT)
server = socket(AF_INET, SOCK_STREAM)
server.bind(ADDRESS)
server.listen(5)
# The server now just waits for connections from clients
# and hands sockets off to client handlers
while True:
print('Waiting for connection')
client, address = server.accept()
print('...connected from:', address)
handler = ClientHandler(client)
handler.start()
When I run this program, it displays the "Waiting for connection" message in the Shell. However, when I try to connect to the program using the Command Prompt, it displays the following error:
C:\Python33>python multi-client-server.py
Traceback (most recent call last):
File "multi-client-server.py", line 30, in <module>
server.bind(ADDRESS)
OSError: [WinError 10048] Only one usage of each socket address (protocol/networ
k address/port) is normally permitted
We haven't studied this in class a lot. So I'm just wondering why this happens and how to fix it.
Thanks!
So, as per your questions:
We haven't studied this in class a lot. So I'm just wondering why this
happens and how to fix it.
Why:
You are trying to run the same code snippet from two different CMD on a Windows OS. So, when you initially execute the code snippet, the server starts listening on the port number 5000, then when you execute the same code snippet from the second CMD wndow it conflicts with the socket that is already being used by the first one.
I tested this on Windows 8.
How to fix:
To fix this issue, you have to simply use a different port number when you execute the code snippet for the second time, so that the socket(IP+port) doesn't conflicts with the previous one. Simply edit your code and put PORT = 15200 and save this file with a different name.(I have provided the code below too.) Now try executing the first code snippet file from a CMD windows and then execute the second code snippet file that you created right now from the second CMD window. The issue will be solved!
Code:
from socket import *
from time import ctime
from threading import Thread
class ClientHandler(Thread):
"""Handles a client request."""
def __init__(self, client):
Thread.__init__(self)
self._client = client
def run(self):
self._client.send(bytes(ctime() + '\nHave a nice day!' , 'ascii'))
self._client.close()
HOST = "localhost"
PORT = 15200 # Port number was changed here
BUFSIZE = 1024
ADDRESS = (HOST, PORT)
server = socket(AF_INET, SOCK_STREAM)
server.bind(ADDRESS)
server.listen(5)
# The server now just waits for connections from clients
# and hands sockets off to client handlers
while True:
print('Waiting for connection')
client, address = server.accept()
print('...connected from:', address)
handler = ClientHandler(client)
handler.start()
If you prefer then have a look here for the basic client-server issues.