I am using glade as my gui and creating a process to run my gui in. This app will open a socket when 'on' is clicked. When i press 'send', it will send whatever is in an textfield to the socket. The socket receives this data and sends it back. The problem is after i send data to the socket the thread doesn't terminate. Also after i close my gui it calls a sys.exit() but also leaves a process and doesn't terminate. I believe the error is in how i am implementing my processes or all my processing in general. Can anyone shine some light on this? It also relates to my last post as well. Thanks
main.py
// Main thread that create a new process for my gui and displays it
import socket, thread, gtk, Handler, sys, os, multiprocessing
sys.setrecursionlimit(10000)
if __name__ == '__main__':
builder = gtk.Builder()
#32bit template.glade 64bit template-2.22
# #todo add switching between architectures
#
builder.add_from_file("template/template-2.22.glade")
builder.connect_signals(Handler.Handler(builder))
window = builder.get_object("window1")
window.show_all()
try:
p = multiprocessing.Process(target=gtk.main())
p.start()
except:
print "Error Starting new Thread"
handler.py
// Handler for gtk glade signals, creates new threads and handles button and stuff
import thread, threading, os, server, client,multiprocessing, time
import sys, gtk
class Handler(object):
'''
classdocs
'''
myobject = ''
def __init__(self,object1):
#Getting glade builder
self.myobject = object1
'''
Constructor
'''
def clickme(self,value):
myserver = server.Server()
try:
p = multiprocessing.Process(target=myserver.run)
p.start()
except:
pass
def sendmessage(self,value):
text = self.myobject.get_object('entry1').get_text()
print text
msg = client.MyClass()
p = multiprocessing.Process(target=msg.run,args=([text]))
p.start()
server.py
// Opens a socket and listens for incoming data and sends it back
import socket,multiprocessing, gtk, sys
class Server:
'''
classdocs
'''
def __init__(self):
'''
Constructor
'''
def run(self):
try:
while 1:
HOST = 'localhost' # Symbolic name meaning the local host
PORT = 50006 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
s.listen(5)
conn, addr = s.accept()
print 'Connected by', addr
while True:
data = conn.recv(1024)
if not data:
conn.close()
sys.exit()
break
elif data != '':
conn.sendall(data)
break
print "Closing"
#conn.close()
finally:
print "End"
pass
client.py
// Sends whatever is inside text area to socket
import time
class MyClass:
'''
classdocs
'''
def __init__(self):
'''
Constructor
'''
def run(self,text):
try:
import socket
HOST = 'localhost' # The localhost
PORT = 50006 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send(text)
data = s.recv(1024)
while 1:
if data != '':
print 'Received', repr(data)
break
finally:
pass
This is just wrong:
p = multiprocessing.Process(target=gtk.main())
p.start()
First, you can't start the gtk main loop in a subprocess, even if you did it rigth. Fortunately the process never really tries to start main as you call gtk.main(), which will block until the main loop exits and then return None. So what you're actually doing is:
gtk.main()
p = multiprocessing.Process(target=None)
p.start()
Througout the rest of your code you keep creating new processes and then forgetting about them. If you would keep a reference to them, you could at least try to send the TERM signal to them to shut them down (using Process.terminate, or set the daemon flag). If you want to shut down the subprocess cleanly, you either need to handle that signal in the subprocess, or use other IPC mechanisms to get it to shut down cleanly (like mutliprocessing.Event, ...).
Then there is this:
while True:
data = conn.recv(1024)
if not data:
conn.close()
sys.exit()
break
elif data != '':
conn.sendall(data)
break
This while loop will never loop (unless recv magically returns something else then a string). The first execution path ends with sys.exit() (taking the whole server down - the break is unreachable), the second ends with break, so the loop is useless.
A few lines below you have the exact opposite:
data = s.recv(1024)
while 1:
if data != '':
print 'Received', repr(data)
break
Unless data was '' in the first line, this will be an endless loop, as data's value won't change anymore.
Generally you don't really need multiprocessing for most of this. Starting a server in a different process may be ok if if has to do a lot of work, but spawing a subprocess just to send some data is overkill. Sending and receiving using sockets are IO bound, using threading here would be more reasonable.
You have two classes (Server and Handler) which have only two methods, one of which is __init__, and the other one is only used as target for a subprocess:
myserver = server.Server()
try:
p = multiprocessing.Process(target=myserver.run)
and:
msg = client.MyClass()
p = multiprocessing.Process(target=msg.run,args=([text]))
That's a sign that these shouldn't be classes but functions.
Related
Im sure there are easier ways with particular python modules, but for an assignment I need to create a program that can act as a client/server. As of right now I have it working to the point of only being able to send a message if the reciever has responded. I need it to just send and appear on the respective client/server terminal when enter is pressed. Any help would be greatly appreciated!
These are pictures of what happens as of now
https://i.stack.imgur.com/T9CsJ.png
import sys
import socket
import getopt
def usage(script_name):
print('Usage: py' + script_name + '-l' +' <port number>' + '[<server>]')
def sockObj():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
return sock
def serversockConn(serversocket,port):
serversocket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
# bind the serversocket to address
serversocket.bind(('',int(port)))
# enable a server to accept connections
serversocket.listen(5)
# wait for a connection and accept it
sock,addr = serversocket.accept()
return sock
def connToServersock(sock,server, port):
# connect to a serversocket
if server:
sock.connect((server, int(port)))
else:
sock.connect(('localhost', int(port)))
return sock
if __name__ == '__main__':
## get the command line arguments
try:
options, non_option_args = getopt.getopt(sys.argv[1:],'l')
except getopt.GetoptError as err:
print(err)
sys.exit(2)
#check if '-l' is present in command line agrument
serverSide = False
for opt,arg in options:
if opt == "-l":
serverSide = True
# port number
port = non_option_args[0]
# server address
server = None
hostLen = len(non_option_args)
if hostLen == 2:
server = non_option_args[1]
# create a communicator object, and make a connection between server and client
# server
if serverSide:
serversocket = sockObj()
sock = serversockConn(serversocket,port)
# client
else:
sock = sockObj()
sock = connToServersock(sock,server,port)
while True:
## read a message from standard input
message = sys.stdin.readline().encode()
if len(message) != 0:
sock.send(message)
return_msg = sock.recv( 1024 )
if return_msg:
print("Message recieved: " + return_msg.decode())
else:
print("Other side shut down")
else:
try:
sock.shutdown(socket.SHUT_WR)
sock.close()
except:
pass
I think your issue is that there are two places in your event loop where you block:
message = sys.stdin.readline().encode()
Here, you block until the user has pressed return -- during this time, your program is unable to respond to any data received over the network, because it is blocked waiting for data from stdin.
... and:
return_msg = sock.recv( 1024 )
Here, you are waiting for data to be received from the network -- during this time, your program is unable to respond to any data received from stdin, because it is blocked waiting for data from the network.
The behavior you'd ideally like to have is for your program to wait for both stdin and network traffic simultaneously -- i.e. have it block until either the user has pressed return, or some network data has been received, whichever comes first.
The easiest way to achieve that behavior is to use select(); its purpose is to block until at least one of several file descriptors is ready to be acted on. (Note, however, that Windows does not support using select() on stdin, so if your program needs to run under Windows you will probably have to spawn a second thread instead).
To implement the event loop using select(), add import select to the top of your script, then replace your event loop with something like this instead:
while True:
## block here until either sock or sys.stdin has data ready for us
readable, writable, exceptional = select.select([sock, sys.stdin], [], [])
if sys.stdin in readable:
## read a message from standard input
message = sys.stdin.readline().encode()
if len(message) != 0:
sock.send(message)
if sock in readable:
## read a message from the network
try:
return_msg = sock.recv( 1024 )
if (return_msg):
print("Message received: " + return_msg.decode())
else:
print("Other side shut down")
break
except:
print("recv() threw an exception")
break
I have the following server program in Python which simulates a chat-room. The code accepts connections from clients and for each of them it launches a new thread. This thread will wait for messages from this client. The messages can be L so that the server will respond with a list of connected clients, ip:port msg the server will send the message msg to the client ip:port.
On client side there will be 2 threads, one for receiving messages from the server, the other for sending.
import socket
from threading import Thread
#from SocketServer import ThreadingMixIn
import signal
import sys
import errno
EXIT = False
address = []
address2 = []
# handler per il comando Ctrl+C
def sig_handler(signum, frame):
if (signum == 2):
print("Called SIGINT")
EXIT = True
signal.signal(signal.SIGINT, sig_handler) # setto l'handler per i segnali
# Multithreaded Python server : TCP Server Socket Thread Pool
class ClientThread(Thread):
def __init__(self,conn,ip,port):
Thread.__init__(self)
self.conn = conn
self.ip = ip
self.port = port
print ("[+] New server socket thread started for " + ip + ":" + str(port))
def run(self):
while True:
data = self.conn.recv(1024)
print ("Server received data:", data)
if (data=='L'):
#print "QUI",address2
tosend = ""
for i in address2:
tosend = tosend + "ip:"+str(i[0]) + "port:"+str(i[1])+"\n"
self.conn.send(tosend)
#mandare elenco client connessi
else:
#manda ip:port msg
st = data.split(" ")
msg = st[1:]
msg = ' '.join(msg)
print ("MSG 2 SEND: ",msg)
ipport = st[0].split(":")
ip = ipport[0]
port = ipport[1]
flag = False
print ("Address2:",address2)
print ("ip:",ip)
print ("port:",port)
for i in address2:
print (i[0],ip,type(i[0]),type(ip),i[1],type(i[1]),port,type(port))
if str(i[0])==str(ip) and str(i[1])==str(port):
i[2].send(msg)
self.conn.send("msg inviato")
flag = True
break
if flag == False:
self.conn.send("client non esistente")
if __name__ == '__main__':
# Multithreaded Python server : TCP Server Socket Program Stub
TCP_IP = '127.0.0.1'
TCP_PORT = 2004
TCP_PORTB = 2005
BUFFER_SIZE = 1024 # Usually 1024, but we need quick response
tcpServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServer.bind((TCP_IP, TCP_PORT))
tcpServerB = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServerB.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServerB.bind((TCP_IP, TCP_PORTB))
threads = []
tcpServer.listen(4)
tcpServerB.listen(4)
while True:
print("Multithreaded Python server : Waiting for connections from TCP clients...")
try:
(conn, (ip,port)) = tcpServer.accept()
except socket.error as e: #(code, msg):
if e.errno != errno.EINTR:
raise
else:
break
address.append((ip,port,conn))
(conn2, (ip2,port2)) = tcpServerB.accept()
address2.append((ip2,port2,conn2))
newthread = ClientThread(conn,ip,port)
newthread.start()
threads.append(newthread)
if EXIT==True:
break
print ("SERVER EXIT")
for t in threads:
t.join()
The code has a signal handler for SIGINT to make the exit cleaner (closing connections, sending a message to the client (still to be implemented) and so on ). The handler writes a global flag EXIT to make the infinite loops terminate.
The code runs both in Python2 and Python3. However there are some problems with SIGINT signal generated by CTRL-C. When there is no client connected the program launched with Python2 exits correctly while the one in Python3 does not. Why this behavioural difference?
Considering only running the program in Python2, when a client connects and I press CTRL-C, the main while exits, like the signal is catched always by the main thread and this interrupts the blocking system call accept. However the other threads do not, I think because of the blocking underlying system call data = self.conn.recv(1024). In C I would block SIGINT signals for one thread and then call pthread_cancel from the other thread. How to exit from all threads when SIGINT is generated in Python?
The client program that for the moment works in Python2 only and suffers from the same problem is:
# Python TCP Client A
import socket
from threading import Thread
class ClientThread(Thread):
def __init__(self,conn):
Thread.__init__(self)
self.conn = conn
def run(self):
while True:
data = self.conn.recv(1024)
print "Ricevuto msg:",data
host = socket.gethostname()
print "host:",host
port = 2004
portB = 2005
BUFFER_SIZE = 2000
tcpClientA = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpClientB = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpClientA.connect(('127.0.0.1', port))
tcpClientB.connect(('127.0.0.1', portB))
newthread = ClientThread(tcpClientB)
newthread.start()
while(True):
msg = raw_input("Inserisci comando: ")
tcpClientA.send (msg)
data = tcpClientA.recv(BUFFER_SIZE)
print "data received:",data
tcpClientA.close()
As for the difference in behavior with accept() in Python 3, look at the full description in the docs. I think this is the key statement:
Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an InterruptedError exception (see PEP 475 for the rationale).
The other problem, stated in your penultimate sentence:
How to exit from all threads when SIGINT is generated in Python 2?
Take a look at the threading documentation:
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.
I want make a multiprocess UDP server with Python, listen one port for each process from a class:
processListener.py:
import multiprocessing
import socket
class processListener(multiprocessing.Process):
def __init__(self):
multiprocessing.Process.__init__(self)
self.data = None
def run(self):
self.startServer()
return
def startServer(self):
udpSocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
address = ('', self.port)
udpSocket.bind(address)
while 1:
data, client = udpSocket.recvfrom(1024)
print self.data, '>>>', data.strip()
self.data = data.strip()
udpSocket.sendto('ACK', client)
return
and my main file is server.py:
from processListener import *
# Variable Definition
port = 4000
# Sever Initialization
if __name__ == '__main__':
process = processListener()
process.port = port
process.start()
while True:
command = raw_input()
if command == 'showdata':
print 'Last Data is:', process.data
When the server is running and I send data to localhost:4000 from UDP
shell$
None >>> Test Data
But the problem starts when I use the command showdata
shell$
None >>> Test Data
showdata
Last Data is: None
Multiple processes do not share state by default.
You are accessing processListener instance from server.py process. processListener.py process has its own processListener instance (that has non-None data attribute).
To demonstrate it, replace multiprocessing.Process by threading.Thread. Multiple threads share objects by default and you should see non-None data.
related to my last post (Which somehow got marked off and closed),
I wrote some code to create a thread for a command handler for my python TCP listener. What basically happens is that I send in some data and it goes in the TCP connecter. Then the TCP connector creates another process and sends the data that it received through the process to the function in the command listener. I do not know what is going on. Please help!
import socket
import sys
import errno
from multiprocessing import Process, Queue # #UnresolvedImport
import CommandHandler
class tcpconnection:
def tcp(self):
data = ''
q = Queue()
p = Process(target=CommandHandler.CommandHandler.commands(), args=(self, data))
#
HOST = '' # Symbolic name meaning all available interfaces
PORT = 9999 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print('Connected by', addr)
while True:
data = conn.recv(1024)
p.start()
p.join()
if not data: break
conn.send(data)
conn.close()
Says that:
p = Process(target=CommandHandler.CommandHandler.commands(), args=(self, data))
TypeError: unbound method commands() must be called with CommandHandler instance as first argument (got nothing instead)
target=CommandHandler.CommandHandler.commands() sets the target argument to the result of the invocation of the commands method. What you probably intended to do was to use that method as tharget, so you should use:
p = Process(target=CommandHandler.CommandHandler.commands, args=(self, data))
Also the error tells you that you're trying to call an unbound method on a class, but you need an object to call it on, probably:
p = Process(target=CommandHandler.CommandHandler().commands, args=(self, data))
but that's not your only issue:
while True:
data = conn.recv(1024)
p.start()
p.join()
This loop will fail after the first invocation, because you can't restart an already started process.
I try to write a daemon in python. But I have no idea how can I use a thread to start parallel tcp server in this daemon. And even what type of server I should use : asyncore?SocketServer?socket?
this is part of my code:
import os
def demonized():
child_pid = os.fork()
if child_pid == 0:
child_pid = os.fork()
if child_pid == 0: #fork twice for demonize
file = open('###', "r") # open file
event = file.read()
while event:
#TODO check for changes put changes in list variable
event = file.read()
file.close()
else:
sys.exit(0)
else:
sys.exit(0)
if __name__ == "__main__":
demonized()
So in a loop I have a list variable with some data appended every circle, and I want to start a thread with tcp server that wait for connection in the loop and if client connects send it this data(with zeroing variable). So I do not need to handle multiple clients, the client will be only one at time. What is the optimal way to implement this?
Thank you.
In case you want to avoid repeating boilerplate, Python will soon have a standard module that does the fork() pair and standard-I/O manipulations (which you have not added to your program yet?) that make it a daemon. You can download and use this module right now, from:
http://pypi.python.org/pypi/python-daemon
Running a TCP server in a separate thread is often as simple as:
import threading
def my_tcp_server():
sock = socket.socket(...)
sock.bind(...)
sock.listen()
while True:
conn, address = sock.accept()
...
... talk on the connection ...
...
conn.close()
def main():
...
threading.Thread(target=my_tcp_server).start()
...
I strongly recommend against trying to get your file-reader thread and your socket-answering thread talking with a list and lock of your own devising; such schemes are hard to get working and hard to keep working. Instead, use the standard library's Queue.Queue() class which does all of the locking and appending correctly for you.
Do you want to append items to the list in while event:... loop and serving this list simultaneously? If so then you have two writers and you must somehow protect your list.
In the sample SocketServer.TCPServer and threading.Lock was used:
import threading
import SocketServer
import time
class DataHandler(SocketServer.StreamRequestHandler):
def handle(self):
self.server.list_block.acquire()
self.wfile.write(', '.join(self.server.data))
self.wfile.flush()
self.server.data = []
self.server.list_block.release()
if __name__ == '__main__':
data = []
list_block = threading.Lock()
server = SocketServer.TCPServer(('localhost', 0), DataHandler)
server.list_block = list_block
server.data = data
t = threading.Thread(target=server.serve_forever)
t.start()
while True:
list_block.acquire()
data.append(1)
list_block.release()
time.sleep(1)