I need to create a communication between a client and a server with TCP. But I'd like to send and work with "headers". So from the client I'd like to send a header "COMMAND1" and the server returns me something.
I have the following code:
Server
import socket
import threading
bind_ip = '0.0.0.0'
bind_port = 9998
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((bind_ip, bind_port))
server.listen(5) # max backlog of connections
print ('Listening on {}:{}'.format(bind_ip, bind_port))
def handle_client_connection(client_socket):
request = client_socket.recv(1024)
print ('Received {}'.format(request))
client_socket.send('Response1!'.encode('utf-8'))
client_socket.close()
while True:
client_sock, address = server.accept()
print ('Accepted connection from {}:{}'.format(address[0], address[1]))
client_handler = threading.Thread(
target=handle_client_connection,
args=(client_sock,) # without comma you'd get a... TypeError: handle_client_connection() argument after * must be a sequence, not _socketobject
)
client_handler.start()
Client
import socket
hostname, sld, tld, port = 'www', 'integralist', 'co.uk', 80
target = '{}.{}.{}'.format(hostname, sld, tld)
# create an ipv4 (AF_INET) socket object using the tcp protocol (SOCK_STREAM)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# connect the client
# client.connect((target, port))
client.connect(('0.0.0.0', 9998))
# send some data (in this case a HTTP GET request)
client.send('hi'.encode('utf-8'))
# receive the response data (4096 is recommended buffer size)
response = client.recv(4096)
print (response)
Anyone knows the best way to return "Response1!" when the header is "COMMAND1" and "Response2!" when the header is "COMMAND2"?
I can't find examples on how to use headers
EDIT: It doesn't have to be "COMMAND1" or "COMMAND2" it can be a "0" or "1", or anything else.
If you want to add your own header, you just have to:
Make sure your programm finds the start of your message (like, every message beginns "!?&")
Send your own header-data just after the start-symbol of your message.
Maybe mark the end of your message with something or pass a length in your header.
Since TCP will give you a stream of data, it might come to a case, where it just gives you 2 or 3 messages at once. You have to separate these messages by yourself (e.g. by using "?!&" as start of every message).
You can always create your own protocoll as payload of another protocoll. Just as TCP is just payload from the ethernet point of view.
You can do something i have done with my program to accept such headers
Use pickle library to encode a dict headers and send it through socket.
Code will look something like this.
import pickle
def handleSocket(headers:dict):
message = pickle.dumps(headers)
socket.send(message)
For server side, you will be handling it
Gonna initialise the socket recv to 100 kb
def handleReceivedSocket(socket):
message:dict = pickle.loads(socket.recv(102400))
Another way to do this. Is sending a raw json string to the server (just change pickle.dumps,pickle.loads by json.dumps,json.loads
But it will be in raw and less secure.
Last way you can do it is uri encoding. Check w3schools
Related
So in other words it's just stuck after 1st response from the browser.
I'm new to socket programming, but here is stupid question, do I need
to close client socket after the response from the server is sent? In
handleData after self.client.send(). Or what could be the reason
of this behaviour? Tried to use select here to make it asynchronous
but it didn't help. Same issue. After I get request from the client
with headers, I go to the next iteration of the loop, exactly to the
recv() and it is stuck. The client is not sending info anymore and
doesn't show the info that is sent from the server. In this example it
is "Info/Message was retrieved!".
P.S. DEFAULT_ENCODING = "utf-8"
DEFAULT_CLIENTS_AMOUNT = 5
SERVER_IP = "127.0.0.1"
SERVER_PORT = 8080
class Server:
def __init__(self, socket_family: int = socket.AF_INET, socket_type: int = socket.SOCK_STREAM, ip_address: str = SERVER_IP, port: int = SERVER_PORT):
self.addr: Tuple[str, int] = ip_address, port
self.serv_heart: socket.socket = socket.socket(socket_family, socket_type)
self.client: Optional[socket.socket] = None
def _launchSyncServer(self, clients: int = DEFAULT_CLIENTS_AMOUNT):
print("Starting server...")
self.serv_heart.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
self.serv_heart.bind(self.addr)
self.serv_heart.listen(clients)
print(f"Server started at {str(self.addr)}")
print(f"Visible: http://{self.addr[0]}:{self.addr[1]}")
def _acceptConnections(self):
client_socket, client_addr = self.serv_heart.accept()
print(f"Client connected from {str(client_addr)}")
self.client = client_socket
def _handleData(self):
while True:
rawData: str = self.client.recv(128).decode(DEFAULT_ENCODING)
if not rawData:
self.client.close()
break
self.client.send("Info/Message was retrieved!".encode(DEFAULT_ENCODING))
self.client.close()
def serverLoop(self, clients: int = DEFAULT_CLIENTS_AMOUNT):
self._launchSyncServer(clients=clients)
while True:
self._acceptConnections()
self._handleData()
it's just stuck after 1st response from the browser.
Your server doesn't implement HTTP nor any other browser protocol, so a browser is not an appropriate client for it; better use something like nc localhost 8080.
do I need to close client socket after the response from the server is sent?
That depends on how you want the communication to be conducted. If you want the client to only send one message, you can close the server's connection right after responding. If you want the client to be able to send multiple messages, the server as it is is prepared for that, and the client needs to indicate that it's finished by closing its socket or at least shutting down writing.
Note that in general, you can't be sure that a message is reveived in whole with a single recv call - it might be split and returned piece by piece through several calls of recv.
I have the following problem: I want a sever to send the contents of a textfile
when requested to do so. I have writen a server script which sends the contents to the client and the client script which receives all the contents with a revcall loop. The recvall works fine when
I run the server and client from the same device for testing.
But when I run the server from a different device in the same wifi network to receive the textfile contents from the server device, the recvall doesn't work and I only receive the first 1460 bytes of the text.
server script
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("", 5000))
server.listen(5)
def send_file(client):
read_string = open("textfile", "rb").read() #6 kilobyte large textfile
client.send(read_string)
while True:
client, data = server.accept()
connect_data = client.recv(1024)
if connect_data == b"send_string":
send_file(client)
else:
pass
client script
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(("192.168.1.10", 5000))
connect_message = client.send(b"send_string")
receive_data = ""
while True: # the recvall loop
receive_data_part = client.recv(1024).decode()
receive_data += receive_data_part
if len(receive_data_part) < 1024:
break
print(receive_data)
recv(1024) means to receive at least 1 and at most 1024 bytes. If the connection has closed, you receive 0 bytes, and if something goes wrong, you get an exception.
TCP is a stream of bytes. It doesn't try to keep the bytes from any given send together for the recv. When you make the call, if the TCP endpoint has some data, you get that data.
In client, you assume that anything less than 1024 bytes must be the last bit of data. Not so. You can receive partial buffers at any time. Its a bit subtle on the server side, but you make the same mistake there by assuming that you'll receive exactly the command b"send_string" in a single call.
You need some sort of a protocol that tells receivers when they've gotten the right amount of data for an action. There are many ways to do this, so I can't really give you the answer. But this is why there are protocols out there like zeromq, xmlrpc, http, etc...
The task is building two files client.py and server.py. I am able to connect the client to the server. The problem I encounter is when I trying to send a get request like client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n"), I do not how to return the file suc.txt to the client from the server side. The scene is a client request file from a server and what the server returns is the respond header and the requested file.
What I wrote so far :
Client:
import socket
target_host = "127.0.0.1"
target_port = 5050
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target_host,target_port))
client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n")
response = client.recv(1024)
print(response.decode())
Server:
import socket
import threading
import urllib.request
HEADER = 64
PORT = 5050
HOST = socket.gethostbyname(socket.gethostname())
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST,PORT))
def handleClient(conn, addr):
print (f"[NEW CONNECTION {addr} connected. ")
connected = True
while connected:
conn.send()
def start():
server.listen()
while True:
conn, addr = server.accept()
thread = threading.Thread(target=handleClient, args=(conn,addr))
thread.start()
print(f"[ACTIVE CONNECTIONS] {threading.activeCount()} ")
print ("server is starting...")
start()
client.send("bGET /suc.txt HTTP/1.1\r\nHost:127.0.0.1\r\n\r\n")
The "b..." should be b"...", i.e. you want to specify a sequence of bytes and not a string with a b as first character.
I do not how to return the file suc.txt to the client from the server side
You basically ask very broadly how to read an HTTP request, extract information from it, create a proper response and send it. All what you code so far does is create a listener socket, so you are far away from your ultimate goal.
There are two major ways to tackle this: the easy one is to use a library like http.server to implement the complexity of HTTP for you. The documentation contains actual examples on how to do this and there are many more examples on the internet for this.
The harder option is to study the actual HTTP standard and implement everything yourself based on this standard. Expecting that somebody explains the complex standard here and describes how to implement it would be a too broad question.
I have a simple Python HTTP server which also connects other HTTP servers to fetch some data. While, connecting to other servers, my server acts as an http client, but the socket created for incoming connection requests still keeps listening from port 8080 (I have a different socket for the client).
The list of other servers that I need to connect and fetch data is stored in a JSON file and I have code like this
with open(myjsonfile, 'r') as json_file:
entries = json.load(json_file)
for entry in entries.keys():
address = entries[entry]['address']
port = int(entries[entry]['port'])
client_port = config.server_port + 50000
host = gethostname()
# request the TXT file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.txt_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = ''
response = sock.recv(2048).decode()
pos = response.find("\r\n\r\n")
txt_data = response[pos+4:]
# processing the received data here
sock.close()
# request the IMG file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.img_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = b''
while True:
recvdata = sock.recv(2048)
if (len(recvdata) < 1):
break
response = response + recvdata
pos = response.find(b"\r\n\r\n")
img_data = response[pos+4:]
# working with the image here
sock.close()
I have to use a set port number for my client because this is how the server identifies me. However, I sometimes get an "Address already in use" error for the second socket.bind() call (the one for the image). Without the bind() calls, my code works fine.
I tried setting socket options (commented out in the code above) and using pycurl with the LOCALPORT property set to client_port value above, but still getting the same error.
What could be the reason behind the error message? I think I open and close the sockets so the operating system should free the port for further use (I think)?
Thanks
PS : This is a small project, not a production system, hence do not bother with "why use port numbers to identify clients"
There is a TIME_WAIT after the session is shutdown to make sure that there are still no live packets in the network.When you re-create the same tuple and one of those packets shows up, it would be treated as a valid packet for your connection this will cause an error state.Usually 2xpacket max age, before the packet is discarded
Before you create a connection with the same tuple, all the packets from the previous session must be dead.
Try using;
...
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.listen([backlog])
sock.bind((host, client_port))
...
socket.listen([backlog])
I have a SocketServer.StreamRequestHandler server that calls self.rfile.readline() to read a request and then calls self.wfile.write(data) to send back some data:
class FileServerHandler(SocketServer.StreamRequestHandler):
def handle(self):
# self.rfile is a file-like oject created by the handler
data = self.rfile.readline()
if data == "msg":
self.wfile.write(someOtherData)
I want my client to be able to send the request and receive the "someOtherData" from the server:
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.send("msg")
print sock.recv(1024)
sock.close()
But the client hangs when I try this. Where am I going wrong? Also is it necessary to know how much data the socket recv's or is there a way to just receive all the data the server writes?
As your server is doing a self.rfile.readline() it is constantly reading until it receives a newline ("\n") character. Thus your client needs to send sock.send("msg\n") for it to terminate the read command.
beside Jan answers I like to mention if you want to receive your exact message, you need to use strip to drop '\n' that you have put at the end of your string .