I'm creating a TCP client and server using Python 3.3 and I'm new to using both Python and sockets. My issue is that I need to be able to store anything passed across the connection as a separate variable for writing to various files, etc.
I'm unsure how to do this as it all seems to be one stream of data that I cannot store separately. Below is my latest piece of working code and all it does is send the data I need across the connection. Should I be trying to send all the data across as a list and de-construct it into separate variables? Can I already access them separately and I just haven't figured it out yet? Is my approach all wrong?
Server code:
import os
from socket import *
HOST = '' #We are the host
PORT = 29876
ADDR = (HOST, PORT)
BUFFSIZE = 4096
serv = socket( AF_INET,SOCK_STREAM)
serv.bind(ADDR,) #Create tuple with one element
serv.listen(5)
print ('listening...')
conn,addr = serv.accept()
print (conn,addr)
print ('...connected')
with open(os.path.expanduser("~/.ssh/id_rsa.pub")) as f:
key = f.read()
conn.sendall(key)
print(key)
while True:
data = conn.recv(BUFFSIZE)
print(data)
conn.close()
Client code:
from socket import *
import os
import socket
HOST = 'xxx.xxx.xxx.xxx'
PORT = 29876 #Must match server.py
ADDR = (HOST,PORT)
BUFFSIZE = 4096
cli = socket.socket( AF_INET, SOCK_STREAM)
cli.connect(ADDR,)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(("gmail.com",80))
ip = ((s.getsockname()[0]))
ip = ip.encode('utf-8')
cli.send(ip)
while True:
data = cli.recv(BUFFSIZE)
print (data)
cli.close()
server:
from socket import *
from os.path import isfile
s = socket()
s.bind(('', 1234))
s.listen(4)
ns, na = s.accept()
loopdata = {}
i = 0
while 1:
try:
data = ns.recv(8192)
except:
break
for line data.split('\n'):
if line == 'version':
print na[0] + ' requested a version'
ns.send('1.0\n')
elif line == 'key':
print na[0] + ' is requesting a key'
if isfile(na[0] + '.key'):
with open(na[0] + '.key') as f:
ns.send(f.read())
else:
ns.send('Missing key file!\n')
loopdata[i] = line
#ns.send('OK\n')
i += 1
ns.close()
s.close()
print loopdata # <- Print all the lines
client:
from socket import *
s = socket()
s.connect(('127.0.0.1', 1234))
s.send('version\n')
print 'Client got:', s.recv(8192)
s.close()
I'm not sure as to what you want to save/store/respond to,
You mentioned something about a key in your code and in your client code you created multiple sockets but only really used one.. and that was for printing whatever the server was sednding?
Begin clean, try to figure out things before mixing it all up.
And state a clear goal that you want to achieve with your code? what do you want to send? and when sending X, what do you want the server to respond?
Tactics:
1. You want to define a protocol,
1.1 Command separator
1.2 Command structure (ex: command:parameter:data\n)
1.3 sates (ex login state etc etc..)
After you know what you want to do, ex file share:
c->s: get:file:/root/storage/file.txt\n
c<-s: file:content\n
c<-s: <file data>\n\n
c<-s: file:close\n
c->s: file:recieved
For instance.
Related
I am trying to learn multithreading programming with Python and specifically I am trying to build a programm that 1 Client sends data to multiple servers and get some messages back. In the first version of my programm I want my client to communicate back and forth with each server that I have spawned with each thread, till I type 'bye'. Now I have 2 issues with my implementation that I don't understand how to deal with. The first one is that I don't want the connection with the server to close after I type 'bye' (I want to add extra functionality after that) and the second one is that the servers doesn't get the messages that I type at the same type but I can communicate with the second server only if the first thread terminates (which as I said, I don't want to terminate). Any suggestions would be appreciated. Cheers!
Client.py
import sys
import threading
from _thread import *
import socket
host_1 = '127.0.0.1'
port_1 = 6000
host_2 = '127.0.0.2'
port_2 = 7000
def connect_to_server(host, port):
client_socket = socket.socket() # instantiate
client_socket.connect((host, port)) # connect to the server
message = input(" -> ") # take input
while message.lower().strip() != 'bye':
client_socket.send(message.encode()) # send message
data = client_socket.recv(1024).decode() # receive response
print('Received from server: ' + data) # show in terminal
message = input(" -> ") # again take input
threads_dict = {}
th_1 = threading.Thread(target=connect_to_server, args=(host_1, port_1))
th_2 = threading.Thread(target=connect_to_server, args=(host_2, port_2))
th_1.start()
th_2.start()
th_1.join()
th_2.join()
Server.py
import socket
import sys
def server_program():
host = sys.argv[1] # '127.0.0.1', '127.0.0.2'
port = int(sys.argv[2]) # 6000, 7000
server_socket = socket.socket() # get instance
server_socket.bind((host, port)) # bind host address and port together
# configure how many client the server can listen simultaneously
server_socket.listen(2)
conn, address = server_socket.accept() # accept new connection
print("Connection from: " + str(address))
while True:
# receive data stream. it won't accept data packet greater than 1024 bytes
data = conn.recv(1024).decode()
if not data:
# if data is not received break
break
print("from connected user: " + str(data))
data = input(' -> ')
conn.send(data.encode()) # send data to the client
if __name__ == '__main__':
server_program()
I am capturing screenshots from the server, then sending it to the client, but the images get all sent as one big file to the client that keeps expanding in size. This only happens when i send from one machine to another (I am working on a local netwrok) but when running both client and server from my machine they work fine.
Note: for the client on the other machine, I packaged it into an exe using pyinstaller, since this machine does not have python.
server code:
host="192.168.43.79" # Set the server address to variable host
port=4446 # Sets the variable port to 4446
import time
import pyautogui
from socket import *
import os
s=socket(AF_INET, SOCK_STREAM)
s.bind((host,port))
print("Listening for connections.. ")
q,addr=s.accept()
i = 0
while True:
screenshot = pyautogui.screenshot()
screenshot.save(str(i) + ".jpg")
with open(str(i) + ".jpg", "rb") as f:
data = f.read(4096)
while data:
q.send(data)
data = f.read(4096)
q.send(b"full")
i += 1
time.sleep(0.3)
client code:
host="192.168.43.79" # Set the server address to variable host
port=4446 # Sets the variable port to 4446
from multiprocessing.reduction import recv_handle
from socket import * # Imports socket module
s=socket(AF_INET, SOCK_STREAM) # Creates a socket
s.connect((host,port))
i = 0
while True:
with open(str(i) + "s.jpg", "wb") as f:
recv_data = s.recv(4096)
while recv_data:
f.write(recv_data)
recv_data = s.recv(4096)
if(recv_data == b"full"):
break
i += 1
There various wrong assumptions here which lead to the problem you see. The wrong assumptions are:
that send(data) will write all data It might send less. You need to check the return value or use sendall to be sure.
that a single send in the sender is matched by exactly a single recv in the recipientTCP is only an unstructured byte stream. send does not add message semantics, so a single send might lead to multiple recv, multiple send might lead to a single recv etc. Specifically send("data") followed by send("full") might be recv(4096) as "datafull", thus missing your code to detect end of image.
As for why does it work on the local machine but not on the remote - the chance in the latter case is higher that send get combined together and recv as one.
As stated by Steffen Ulrich you should use sendall for sending and for receiving we create a specialized function my_recv that will repeatedly call socket.recv until the expected number of bytes have been received. Also, a 4-byte header (you can make the length greater if your file sizes warrant this) that contains a binary representation of the file length precedes the sending of the actual file data. In this way the client knows exactly how much data it should receive for each file.
Server Code
host="192.168.43.79" # Set the server address to variable host
port=4446 # Sets the variable port to 4446
import time
import pyautogui
from socket import *
import os
s=socket(AF_INET, SOCK_STREAM)
s.bind((host,port))
s.listen(1) # This should be called
print("Listening for connections.. ")
q,addr=s.accept()
i = 0
while True:
screenshot = pyautogui.screenshot()
screenshot.save(str(i) + ".jpg")
with open(str(i) + ".jpg", "rb") as f:
# Get length by positioning to end of file
image_length = f.seek(0, 2)
f.seek(0, 0) # Seek back to beginning of file
# Convert image length to a 4-byte array:
image_length_bytes = image_length.to_bytes(4, 'big')
q.sendall(image_length_bytes)
data = f.read(4096)
while len(data):
q.sendall(data)
data = f.read(4096)
i += 1
Client Code
host="192.168.43.79" # Set the server address to variable host
port=4446 # Sets the variable port to 4446
from multiprocessing.reduction import recv_handle
from socket import * # Imports socket module
s=socket(AF_INET, SOCK_STREAM) # Creates a socket
s.connect((host,port))
def my_recv(msg_length):
chunks = []
bytes_to_recv = msg_length
while bytes_to_recv:
chunk = s.recv(bytes_to_recv)
if chunk == b'':
raise RuntimeError("socket connection broken")
chunks.append(chunk)
bytes_to_recv -= len(chunk)
return b''.join(chunks)
i = 0
while True:
image_length_bytes = my_recv(4)
image_length = int.from_bytes(image_length_bytes, 'big')
with open(str(i) + "s.jpg", "wb") as f:
bytes_to_recv = image_length
while bytes_to_recv:
recv_data = my_recv(min(4096, bytes_to_recv))
f.write(recv_data)
bytes_to_recv -= len(recv_data)
i += 1
I am trying to write a client/server program in python that will accept multiple connections and manage them using threading. The server and client both run, and the client will receive the "welcome" message from the servers "processClient" function, which means the connection is being made and the thread is being started. However, any subsequent receive or send on the connection object after the welcome message fails with an "OSError: [Errno 9] Bad file descriptor" error. I've done some searching on the error, and most problems seem to result from someone trying to use a socket or connection that's been previously closed-which should not be the case here. Does anyone know what could be causing the error? Running python version 3.5.2
Server code:
#!/usr/bin/env python3
import socket
import sys
import os
import datetime
import threading
import random
PORT = 65432 # Port to listen on (non-privileged ports are > 1023)
def processClient(conn, id):
welcome = "Hello, you are client number " + str(id)
welcome = bytes(welcome, 'utf-8')
conn.sendall(welcome)
while True:
data = conn.recv(1024)
print(rpr(data))
time = str(datetime.datetime.now())
arr = bytes(time, 'utf-8')
if data == b'time':
conn.sendall(arr)
elif data == b'':
conn.close()
return
else:
temp = data.decode("utf-8")
temp = temp.upper()
temp = bytes(temp, 'utf-8')
conn.sendall(temp)
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except:
print("unable to create socket connection, shutting down.")
quit()
s.bind(('0.0.0.0', PORT))
s.listen()
sys.stdout.write("Server is running \n")
runningThreads = []
threadID = 0
while True:
conn, addr = s.accept()
with conn:
#conn.setblocking(False)
print('Connected by', addr)
threadID += 1
threadTemp = threading.Thread(target = processClient, args=(conn, threadID))
threadTemp.start()
runningThreads.append(threadTemp)
for t in runningThreads:
if not t.isAlive():
# get results from thtead
t.handled = True
threadID -= 1
else:
t.handled = False
runningThreads = [t for t in runningThreads if not t.handled]
Client code:
#!/usr/bin/env python3
import socket
import sys
import os
import datetime
HOST = 0
while HOST == 0 or HOST == "":
HOST = input("Please enter host IP: ")
PORT = 65432 # The port used by the server
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
data = s.recv(1024)
print(repr(data))
while True:
inputString = input("Please input a string: ")
temp = bytes(inputString, 'utf-8')
s.sendall(temp)
if inputString == "":
quit()
data = s.recv(1024)
if data:
print(rpr(data))
for anyone else who stumbles across this: I did finally work out the problem. The server was not waiting on input from the client before it attempts to read data from the connection, which was triggering the error (the error message was particularly unhelpful in diagnosing this issue). I rewrote this to use python selectors rather than threads-selectors includes very handy polling functionality that can be used to "pause" until there is data to be read. I could have built this into the program myself, but why do so when there's already a language feature that does it for you?
I am having the above issue. The client is suppose to ask for a filename and send the file name to the server after which the server will open the file and display it. Problem is that the server isn't opening the file and displaying it.
Below is the client.
#!/usr/bin/env python3
import socket, os.path, datetime, sys
def Main():
host = '127.0.0.1'
port = 50001
s = socket.socket()
s.connect((host, port))
Filename = input("Type in ur file ")
s.send(Filename.encode('utf-8'))
data = s.recv(1024).decode('utf-8')
s.close()
if __name__ == '__main__':
Main()
Below is the server
#!/usr/bin/env python3
import socket
import os
import sys
def Main():
host = '127.0.0.1'
port = 50001
s = socket.socket()
s.bind((host,port))
print("server Started")
s.listen(1)
c, addr = s.accept()
print("Connection from: " + str(addr))
while True:
data = c.recv(1024).decode('utf-8')
myfile = open(data, "r")
if not data:
break
print("from connected user: " + myfile)
c.close()
if __name__ == '__main__':
Main()
I've made few minimal adjustments to your code with which it runs as so that server.py continuously listens on a given port and sends back data which each invocation of client.py asks for.
server.py
#!/usr/bin/env python3
import socket
import os
import sys
def Main():
host = '127.0.0.1'
port = 50001
s = socket.socket()
s.bind((host,port))
print("server Started")
s.listen(1)
while True:
c, addr = s.accept()
print("Connection from: " + str(addr))
filename = ''
while True:
data = c.recv(1024).decode('utf-8')
if not data:
break
filename += data
print("from connected user: " + filename)
myfile = open(filename, "rb")
c.send(myfile.read())
c.close()
if __name__ == '__main__':
Main()
client.py
#!/usr/bin/env python3
import socket, os.path, datetime, sys
def Main():
host = '127.0.0.1'
port = 50001
s = socket.socket()
s.connect((host, port))
Filename = input("Type in ur file ")
s.send(Filename.encode('utf-8'))
s.shutdown(socket.SHUT_WR)
data = s.recv(1024).decode('utf-8')
print(data)
s.close()
if __name__ == '__main__':
Main()
And now a bit of explanation.
On the server side. The outer loop accepts a connection, then reads from the connection until done (more on this later). Prints your debugging info, but note you were trying to print the file object and not the filename (which would fail trying to concatenate). I also open the file in binary mode (that way I can skip the str -> bytes translation.
On the client side. I've added closing the writing end of the socket when the file has been sent. Note you might want to use sendall instead of send for this use case: check those docs links for details. And I've added a print for the incoming data.
Now that bit with shutting down the writing end in the client and the inner loop reading (and also related to the sendall hint. Which BTW also holds true for the server side, otherwise you should loop, as you might see your content truncated; other option is to also have a sending loop.). Stream sockets will guarantee you get your bytes in in order you've send them. On itself it has no idea whether your message is complete and it also does not guarantee in how many and how large chunks will the data be sent and received (resp.).
The inner loop of server keep reading until we see an EOF (we've receive zero length string in python socket). This would happen (be returned by recv when the remote socket (or at least its writing end) has been shut down. Since we still want to reuse the connection, we only do that on the sending end in the client. Hope this helps you to move ahead.
I am implementing a program with a server and multiple clients. All clients send data to the server and a server check out the step of each client. If all client's steps are the same, a server sends new data to all clients to do next step. And it continues this procedure again and again.
However, when I run my program, it cannot communicate each other. Here are my code. Would you give me some hints?
client & server
#client
from socket import *
from sys import *
import time
import stat, os
import glob
# set the socket parameters
host = "localhost"
port = 21567
buf = 1024
data = ''
addr = (host, port)
UDPSock = socket(AF_INET, SOCK_DGRAM)
UDPSock.settimeout(100)
def_msg = "=== TEST ==="
#FILE = open("test.jpg", "w+")
FILE = open("data.txt","w+")
while (1):
#data, addr = UDPSock.recvfrom(buf)
print "Sending"
UDPSock.sendto(def_msg, addr)
#time.sleep(3)
data, addr = UDPSock.recvfrom(buf)
if data == 'done':
FILE.close()
break
FILE.write(data)
print "Receiving"
#time.sleep(3)
UDPSock.close()
# server program for nvt
from socket import *
import os, sys, time, glob
#import pygame
import stat
host = 'localhost'
port = 21567
buf = 1024
addr = (host, port)
print 'test server'
UDPSock = socket(AF_INET, SOCK_DGRAM)
UDPSock.bind(addr)
msg = "send txt file to all clients"
#FILE = open("cam.jpg", "r+")
FILE = open("dna.dat","r+")
sending_data = FILE.read()
FILE.close()
tmp_data = sending_data
while (1):
#UDPSock.listen(1)
#UDPSock.sendto(msg, addr)
#FILE = open("gen1000.dat","r+")
#sending_data = FILE.read()
#FILE.close()
#print 'client is at', addr
data, addr = UDPSock.recvfrom(buf)
#time.sleep(3)
print data
#msg = 'hello'
#
tmp, sending_data = sending_data[:buf-6], sending_data[buf-6:]
if len(tmp) < 1:
msg = 'done'
UDPSock.sendto(msg, addr)
print "finished"
sending_data = tmp_data
UDPSock.sendto(tmp, addr)
print "sending"
#time.sleep(3)
UDPSock.close()
A server must perform the sequence socket(), bind(), listen(), accept() (possibly repeating the accept() to service more than one client), while a client only needs the sequence socket(), connect().
Your missing listen() i saw first. Listen for connections made to the socket.
More on this here: link text
Look at this: http://heather.cs.ucdavis.edu/~matloff/Python/PyNet.pdf
It's a very good Python networking tutorial including working examples of a client and server. Now, I'm not an expert on this, but it looks to me like your code is overcomplicated. And what's the deal with all the commented-out lines?
Quote from question:
#UDPSock.listen(1)
#UDPSock.sendto(msg, addr)
#FILE = open("gen1000.dat","r+")
#sending_data = FILE.read()
#FILE.close()
End quote
Those look like some pretty important lines to me.
Also, make sure the computers are connected. From a prompt run:
ping [IP]
where [IP] is the IP address of the other machine(Note: if you're not connected to the same LAN, this becomes much harder as you might then need port forwarding and possibly static IPs).