I have a sort of strange issue here. I am somewhat new to python and especially parallel TCP sockets in Python. I followed a guide to make a multi-client TCP server that runs using multiprocessing so I can maintain a front end control.
My understanding of passing variables in Python multiprocessing is using the args parameter when calling the multiprocessing.Process function.
However to start my TCP server my target isn't actually a def (function) and I am unsure how to properly pass a boolean value that will allow me to enable a debug mode in my code on the server process.
See code below:
import os
import datetime
import socket
import jetson.inference
import jetson.utils
import sys
import multiprocessing
import socketserver
from cv2 import *
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler): #Class to process multiple clients if needed, handles sending and receiving messages as well as interperating them
def handle(self):
#cam_port = 0
if not os.path.exists("Images"):
os.makedirs("Images")
net = jetson.inference.detectNet("ssd-inception-v2", sys.argv, 0.5)
inputC = jetson.utils.videoSource("/dev/video0", argv=sys.argv)
# create video sources & outputs
#input = jetson.utils.videoSource(opt.input_URI, argv=sys.argv)
while True:
#print("Got here")
# self.request is the TCP socket connected to the client
self.data = self.request.recv(250).strip()
print("{} wrote:".format(self.client_address[0]))
rec = self.data.decode('utf-8')
print(rec)
if rec == "TRIGGER":
#cam = VideoCapture(cam_port)
#result, image = cam.read()
#processTrigger(cam, result, image, net)
processImage(net, inputC, debug)
double = "DOUBLE"
msg = double.encode('utf-8')
self.request.sendall(msg)
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer): #Class to bind each client to a thread
pass
def openServer(thread, server, isCreate): #function that handles opening and closing the tcp server
if isCreate:
with server:
print("Server running as ", thread.name, "on", server.server_address)
return thread
if not isCreate:
with server:
thread.terminate()
print("Server and Process terminated")
def processImage(network, inputC, debug):
# saving image in local storage
img = inputC.Capture()
detections = network.Detect(img, overlay="box,labels,conf")
print("detected {:d} objects in image".format(len(detections)))
print(debug)
for detection in detections:
print(detection)
if debug:
dateNow = datetime.datetime.now()
currentMSStr = dateNow.strftime("%f")
currentMS = round(int(currentMSStr)/1000)
outputFile = "Images/Capture_"+dateNow.strftime("%y_%m_%d_%H_%M_%S")+"_"+str(currentMS)+".png"
output = jetson.utils.videoOutput(outputFile, argv=sys.argv+['--headless'])
network.PrintProfilerTimes()
output.Render(img)
print("Warning! Debug mode is enabled, this should not be used during normal operation!")
debug = False #global debug boolean
if __name__ == "__main__":
enabled = True
serverOpen = False
HOST, PORT = "", 3334
print("Welcome to the Fives Vision Solution")
print("Press 1 to enable the server")
print("Press 2 to shutdown the server")
print("Press 3 to change server port")
print("Press 4 to enable debug mode. WARNING impacts performance!")
print("Press 5 to exit")
while enabled:
choice = input("Make a selection... \n")
if choice == '1': #Create socket and send to process to run in background
if serverOpen:
print("Server is already active")
if not serverOpen:
try:
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
server_proc = multiprocessing.Process(target=server.serve_forever)
server_proc.daemon = True
server_proc.start()
openServer(server_proc,server,True)
serverOpen = True
except Exception as e:
print("Error occured: ", e)
elif choice == '2': #Terminate the server and close process
openServer(server_proc,server,False)
serverOpen = False
elif choice == '3': #Change port
if serverOpen:
print("Please close the server first.")
if not serverOpen:
print("Please enter a port:")
try:
port = input()
if int(port) > 1024 and int(port) < 65500:
PORT = int(port)
print("Port is now set to: ", PORT)
if int(port) < 1024 or int(port) > 65500:
print("Invalid port, must be a number between 1023 and 65500")
except:
print("Invalid port, must be a number between 1023 and 65500")
elif choice == '4': #Enable or disable debug mode
if debug:
debug = False
print("Debug set to: " ,debug)
continue
if not debug:
debug = True
print("Debug set to: " ,debug)
continue
elif choice == '5': #Terminate server is open and close the program
enabled = False
if (serverOpen):
openServer(server_proc,server,False)
else:
print("Invalid choice")
As you can see, line 87 initiates the server, and line 88 calls the actual server to serve in another process:
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
server_proc = multiprocessing.Process(target=server.serve_forever)
How would I go about passing my global variable "debug" into my actual handle(self) function in my server class, which actually does the processing of the incoming message, calls a machine learning model and then returns the result back to the client.
Like I said, I'm pretty new to python so forgive me if there are better ways to do what I'm doing.
About the code:
My main issue is that I need debug mode because I want to be able to enable saving the images I process but doing so takes my total round trip of receiving a message, processing, and returning a result to around 200-250ms, and I need to run in less than 100ms.
When I disable the code within the debug if statement, my total receive/return time is around 40ms which is great, but if the system has issues I want to be able to store the results in image form for debugging camera issues and so on.
Thanks for any help!
Related
I am opening the same program multiple times (10 to be exact) and i they are all trying to solve the random number first. When they solve this number they open a server which all of the other same programs are listening to on separate threads waiting to connect and when they do finally connect i want them all to shutdown.
import random
import os
import socket
import threading
import time
host = '192.168.1.139'
port = 8011
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def verified():
print('Connection made successfuly')
time.sleep(10)
quit()
def network():
try:
s.connect(('192.168.1.135',8014))
f = s.recv(50)
finished = f.decode('utf-8')
if finished == 'completed':
quit()
except:
pass
def winner():
x = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
x.bind(('192.168.1.135', 8014))
x.listen(10)
con, address = x.accept()
con.send(bytes('completed', 'utf-8'))
def mine():
global guess
time.sleep(.000001)
guess = random.randint(0,1000)
print(guess)
tran = 500
while True:
mine()
if guess == 500:
print("Solved")
winner()
break;
else:
x = threading.Thread(target=network)
x.start()
For some reason it is connecting to the server but it isn't quitting the program, could this be because it is connecting to itself and exiting to fast? Please help, Thank You!
Actually, I was consfused, but apparently when the quit() is in a function it can't end the whole process, so what I would suggest you to do is to make the while loop with a variable that network would be able to change (or even use guess):
guessing = True
while guessing:
mine()
if guess == 500:
print("Solved")
winner()
break;
else:
x = threading.Thread(target=network)
x.start()
and change the network to:
def network():
try:
s.connect(('192.168.1.135',8014))
f = s.recv(50)
finished = f.decode('utf-8')
if finished == 'completed':
#quit()
global guessing
guessing = False
except:
pass
I saw that the two constants host and port were not used
Unfortunately the winner function is just accepting one answer
(she's a little introvert)
but we can solve that problem:
def winner():
x = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
x.bind(('192.168.1.135', 8014))
x.listen(10)
connected = 0
while connected < 10:
con, address = x.accept()
con.send(bytes('completed', 'utf-8'))
connected += 1
and yay, she is making friends
ok, but now she is in the outer world where she needs to talk to a lot of people
not just limited 10, so she goes to the bus stop (she's going home), she waits a little bit to see if someone will talk to her (for any reason), no one said anything for some time, so, she puts her airpods and starts listening to some good {insert here a music that a computer function likes to listen to}
So, to implement this:
def winner():
x = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
x.settimeout(5) # WAITS FOR FIVE SECONDS
x.bind(('192.168.1.135', 8014))
x.listen()
while True:
try:
con, address = x.accept()
con.send(bytes('completed', 'utf-8'))
except socket.timeout: # IF IN FIVE SECONDS, RECEIVES NO CONNECTION
break
the x.settimeout(5) will make the connections give an socket.timeout error if the time set (in case 5) is exceeded without any response of the other side.
now I remembered that using threading we could push clients to another function and "free" the queue of the clients, but I think that for this special case this solution is better
This is a python chat room that I've been working on and it enables to you to chat to other people on the same network through python
Host:
import socket
import sys
import time
s = socket.socket()
host = socket.gethostname()
port = 8080
s.bind((host,port))
print("")
print("Sever adress is", host)
print("")
name = input(str("Please enter your username : "))
s.listen(1)
print("")
print("Waiting for any incoming connections ... ")
print("")
conn, addr = s.accept()
print("Recieved connection")
#connection done ###
s_name = conn.recv(1024)
s_name = s_name.decode()
print("")
print(s_name, "has connected to the chat room")
print("")
conn.send(name.encode())
## messaging loop ##
while 1:
message = input(str("Please enter enter your message : "))
print("")
conn.send(message.encode())
message = conn.recv(1024)
message = message.decode()
print("")
print(name,": ",message)
print("")
Client:
import socket
import sys
import time
print("Welcome to python chat ")
print("")
print("Initiallsing....")
time.sleep(1)
s = socket.socket()
print("")
host = input(str("Please enter server adress : "))
print("")
name = input(str("Please enter your name : "))
port = 8080
print("")
time.sleep(1)
s.connect((host,port))
print("Connected...")
## Conection done ##
s.send(name.encode())
s_name = s.recv(1024)
s_name = s_name.decode()
print("")
print( s_name, "has joined the chat room ")
while 1:
message = s.recv(1024)
message = message.decode()
print("")
print(name,": ",message)
print("")
message = input(str("Please enter your enter message : "))
print("")
s.send(message.encode())
I have 2 problems, the first problem is that it only allows one person to speak at a time, what I mean by this say that if you send a message first you won't be allowed to send another message until the other person has responded. The second problem is that this code only works for 2 users, I want it so it works for multiple users
Edit:
Also could some one acutually give a proper solution instead of telling me to start all over again please because it took me so long to figure out how to make this system in the place.
Thank you
You need to create two separate threads for sending and receiving. The way you have written the loop will not work for both way communication simultaneously. Because after sending a message the loop is waiting to receive something. [If you want to run the code over internet, replace localhost with the desired IP address in the line HOST = 'localhost'] Let me share a solution (this is a sample solution I have done while TAing an undergraduate class on networking):
I have tested the code on a Linux machine (Ubuntu 18.04). I have got students who have successfully ran this on their Mac. I am not sure if it runs on a windows machine. Even if it does not work on a Windows machine, a few minor modifications should do the trick.
Server sider code (you need to run this first): chatServerDuplex.py
# Import socket module
from socket import *
import threading
import sys # In order to terminate the program
FLAG = False # this is a flag variable for checking quit
# function for receiving message from client
def recv_from_client(conn):
global FLAG
try:
# Receives the request message from the client
while True:
if FLAG == True:
break
message = conn.recv(1024).decode()
# if 'q' is received from the client the server quits
if message == 'q':
conn.send('q'.encode())
print('Closing connection')
conn.close()
FLAG = True
break
print('Client: ' + message)
except:
conn.close()
# function for receiving message from client
def send_to_client(conn):
global FLAG
try:
while True:
if FLAG == True:
break
send_msg = input('')
# the server can provide 'q' as an input if it wish to quit
if send_msg == 'q':
conn.send('q'.encode())
print('Closing connection')
conn.close()
FLAG = True
break
conn.send(send_msg.encode())
except:
conn.close()
# this is main function
def main():
threads = []
global FLAG
# TODO (1) - define HOST name, this would be an IP address or 'localhost' (1 line)
HOST = 'localhost'
# TODO (2) - define PORT number (1 line) (Google, what should be a valid port number)
# make sure the ports are not used for any other application
serverPort = 6789
# Create a TCP server socket
#(AF_INET is used for IPv4 protocols)
#(SOCK_STREAM is used for TCP)
# TODO (3) - CREATE a socket for IPv4 TCP connection (1 line)
serverSocket = socket(AF_INET, SOCK_STREAM)
# Bind the socket to server address and server port
# TODO (4) - bind the socket for HOSR and serverPort (1 line)
serverSocket.bind((HOST, serverPort))
# Listen to at most 1 connection at a time
# TODO (5) - listen and wait for request from client (1 line)
serverSocket.listen(1)
# Server should be up and running and listening to the incoming connections
print('The chat server is ready to connect to a chat client')
# TODO (6) - accept any connection request from a client (1 line)
connectionSocket, addr = serverSocket.accept()
print('Sever is connected with a chat client\n')
t_rcv = threading.Thread(target=recv_from_client, args=(connectionSocket,))
t_send = threading.Thread(target=send_to_client, args=(connectionSocket,))
# call the function to receive message server
#recv_from_server(clientSocket)
threads.append(t_rcv)
threads.append(t_send)
t_rcv.start()
t_send.start()
t_rcv.join()
t_send.join()
# closing serverScoket before exiting
print('EXITING')
serverSocket.close()
#Terminate the program after sending the corresponding data
sys.exit()
# This is where the program starts
if __name__ == '__main__':
main()
Client side code: chatClientDuplex.py
from socket import *
import threading
import sys
FLAG = False # this is a flag variable for checking quit
# function for receiving message from client
def send_to_server(clsock):
global FLAG
while True:
if FLAG == True:
break
send_msg = input('')
clsock.sendall(send_msg.encode())
# function for receiving message from server
def recv_from_server(clsock):
global FLAG
while True:
data = clsock.recv(1024).decode()
if data == 'q':
print('Closing connection')
FLAG = True
break
print('Server: ' + data)
# this is main function
def main():
threads = []
# TODO (1) - define HOST name, this would be an IP address or 'localhost' (1 line)
HOST = 'localhost' # The server's hostname or IP address
# TODO (2) - define PORT number (1 line) (Google, what should be a valid port number)
PORT = 6789 # The port used by the server
# Create a TCP client socket
#(AF_INET is used for IPv4 protocols)
#(SOCK_STREAM is used for TCP)
# TODO (3) - CREATE a socket for IPv4 TCP connection (1 line)
clientSocket = socket(AF_INET, SOCK_STREAM)
# request to connect sent to server defined by HOST and PORT
# TODO (4) - request a connection to the server (1 line)
clientSocket.connect((HOST, PORT))
print('Client is connected to a chat sever!\n')
# call the function to send message to server
#send_to_server(clientSocket)
t_send = threading.Thread(target=send_to_server, args=(clientSocket,))
# call the function to receive message server
#recv_from_server(clientSocket)
t_rcv = threading.Thread(target=recv_from_server, args=(clientSocket,))
threads.append(t_send)
threads.append(t_rcv)
t_send.start()
t_rcv.start()
t_send.join()
t_rcv.join()
print('EXITING')
sys.exit()
# This is where the program starts
if __name__ == '__main__':
main()
Your first problem is likely due to the fact that python sockets are blocking by default.
What this means is that, for example, on the line message = s.recv(1024), your program will keep listening and won't move on to the rest of your script until it receives something.
If you want two people to be able to receive and send at the same time, you might want to look into non-blocking sockets and some asynchronous programming.
This how-to from the official documentation might help you: https://docs.python.org/2/howto/sockets.html#non-blocking-sockets
System123456 the problem is that you built a client-server system when the server listens and the client connects to it. Try looking at peer-to-peer systems instead where each node is an equal. For building a chat room you might review DHT nodes.
Im sure there are easier ways with particular python modules, but for an assignment I need to create a program that can act as a client/server. As of right now I have it working to the point of only being able to send a message if the reciever has responded. I need it to just send and appear on the respective client/server terminal when enter is pressed. Any help would be greatly appreciated!
These are pictures of what happens as of now
https://i.stack.imgur.com/T9CsJ.png
import sys
import socket
import getopt
def usage(script_name):
print('Usage: py' + script_name + '-l' +' <port number>' + '[<server>]')
def sockObj():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
return sock
def serversockConn(serversocket,port):
serversocket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
# bind the serversocket to address
serversocket.bind(('',int(port)))
# enable a server to accept connections
serversocket.listen(5)
# wait for a connection and accept it
sock,addr = serversocket.accept()
return sock
def connToServersock(sock,server, port):
# connect to a serversocket
if server:
sock.connect((server, int(port)))
else:
sock.connect(('localhost', int(port)))
return sock
if __name__ == '__main__':
## get the command line arguments
try:
options, non_option_args = getopt.getopt(sys.argv[1:],'l')
except getopt.GetoptError as err:
print(err)
sys.exit(2)
#check if '-l' is present in command line agrument
serverSide = False
for opt,arg in options:
if opt == "-l":
serverSide = True
# port number
port = non_option_args[0]
# server address
server = None
hostLen = len(non_option_args)
if hostLen == 2:
server = non_option_args[1]
# create a communicator object, and make a connection between server and client
# server
if serverSide:
serversocket = sockObj()
sock = serversockConn(serversocket,port)
# client
else:
sock = sockObj()
sock = connToServersock(sock,server,port)
while True:
## read a message from standard input
message = sys.stdin.readline().encode()
if len(message) != 0:
sock.send(message)
return_msg = sock.recv( 1024 )
if return_msg:
print("Message recieved: " + return_msg.decode())
else:
print("Other side shut down")
else:
try:
sock.shutdown(socket.SHUT_WR)
sock.close()
except:
pass
I think your issue is that there are two places in your event loop where you block:
message = sys.stdin.readline().encode()
Here, you block until the user has pressed return -- during this time, your program is unable to respond to any data received over the network, because it is blocked waiting for data from stdin.
... and:
return_msg = sock.recv( 1024 )
Here, you are waiting for data to be received from the network -- during this time, your program is unable to respond to any data received from stdin, because it is blocked waiting for data from the network.
The behavior you'd ideally like to have is for your program to wait for both stdin and network traffic simultaneously -- i.e. have it block until either the user has pressed return, or some network data has been received, whichever comes first.
The easiest way to achieve that behavior is to use select(); its purpose is to block until at least one of several file descriptors is ready to be acted on. (Note, however, that Windows does not support using select() on stdin, so if your program needs to run under Windows you will probably have to spawn a second thread instead).
To implement the event loop using select(), add import select to the top of your script, then replace your event loop with something like this instead:
while True:
## block here until either sock or sys.stdin has data ready for us
readable, writable, exceptional = select.select([sock, sys.stdin], [], [])
if sys.stdin in readable:
## read a message from standard input
message = sys.stdin.readline().encode()
if len(message) != 0:
sock.send(message)
if sock in readable:
## read a message from the network
try:
return_msg = sock.recv( 1024 )
if (return_msg):
print("Message received: " + return_msg.decode())
else:
print("Other side shut down")
break
except:
print("recv() threw an exception")
break
I have a basic HTTP server-client written in Python wherein the server has different modules (campaigns) to run. I want the client to interact with the server only when a certain campaign is up. I was thinking I could maybe create a flag whose value would depend upon the campaign and send this flag to the client when it connects and then depending on the flag the client-server can interact. But how do I do that?
Example:
#Server
def run(server_class, handler_class):
server_add = (host, port)
httpd = server_class(server_add, handler_class)
httpd.serve_forever()
def campaign():
c = input("Choose Campaign: \r\n 1. Cam1 \r\n 2. Cam2 \r\n")
if c == 1:
flag = "Run Cam1" #Set flag
#Send flag to client
run()
do_cam1() #Call the defined function for Campaign 1
elif c == 2:
flag = "Run Cam2"
#Send flag to client
run()
do_cam2()
else:
print "Invalid Campaign"
#Client
def client():
server = ''
c = httplib.HTTPConnection(server)
#Read the flag here
if flag == "Run Cam1":
#Do Something
elseif flag == "Run Cam2":
#Do Something
else:
#Throw Some Error
c.close()
I'm trying to create a client/server console game, but I'm having trouble keeping the socket alive, it seems to be closing before I've called close() and I can't figure out why.
I've read the thread here but I have called connect() outside of the while loop since the logic of that already made sense to me before I attempted to run it, but I'm still getting the error from the server shell though:
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
So far I'm not getting an error on the client shell, it just exits running right after launching which is also when the server error occurs.
After fixing mistakes noted in the comments, the client now also shows an error stating that connected is undefined when attempting to start the while loop but should be since Connect() is run before entering the loop which should have set connected = True thus the loop should run but does not. I doubt this is related to the server issue at all, but what maybe the problem here?
I'll put the code for both below:
Client
import socket
def GetData():
data = s.recv(1000).decode()
if (data == "closing"):
connected = False
else:
print(data)
def SendData():
tdata = input("")
s.send(data.encode())
def Connect():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 2222))
connected = True
global connected
Connect()
while (connected == True):
GetData()
if (connected == False):
s.close()
break
SendData()
Server
import socket
import random
def TheMainGame():
print ("started TheMainGame") # Remove after debug
def within(val, goal):
print ("Runing 'within' function") # Remove after debug
i = abs(val - goal)
if (i <= 5):
return True
else:
return False
def Guess():
print ("Running 'Guess' function") # Remove after debug
while True:
print ("Entered while loop") # Remove after debug
data = "What is your guess?"
c.send(data.encode())
print ("Sent data") # Remove after debug
t = c.recv(1000).decode()
print ("Got data") # Remove after debug
if (t == x):
data = "Correct!"
c.send(data.encode())
data = "closing"
c.send(data.encode())
c.close()
break
elif (within(t,x) == True):
data = "You're close!"
c.send(data.encode())
elif (within(t,x) == False):
data = "Try again"
c.send(data.encode())
else:
data = "There was an error computing the value"
c.send(data.encode())
c.close()
break
x = random.randrange(1, 20, 1)
Guess()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 2222))
s.listen(5)
global data
while True:
(c,a) = s.accept()
print("Received connection from", a) # Remove after debug
data = "Hello %s" %a[0] # Remove after debug
c.send(data.encode()) # Remove after debug
TheMainGame()
The error in the server shell relates to line 19 which is why the server attempts to ask the client the question, which is also the second time it attempts to send data to the client but the attempt never successfully happens, which is why I think the socket is being closed even though I never told it to anywhere before that point since it hasn't even been able to check if t == x. What is causing the socket to close?
You're mis-using global in the client.
in the main body you have global connected. This is incorrect.
Instead, at the top of the file put:
import socket
connected = False
Then, in the Connect function, pull in the global scoped variable:
def Connect():
global connected
...
The keyword global is to tell a function to use a variable from the global scope, not to define them.
Note:
There are other errors in this code which you will need to look at and address.
You've done the same in server.py
You're trying to reference socket s in the GetData and SendData functions but s is local to within the connect function only (meaning that the connection will drop once the function exists.