Develop Reference

Connecting multiple processes on the same socket

In this question, the questioner said that we can not use the same socket for the same process (Monitor creates a new process for each message with unseen id), but he did the opposite in the code as each time he created a monitor, he created it on the same socket? I guess there should be no problem if we create multiple processes and connect them to the same socket using socket.bind("localhost:8888") please?
import zmq
from zmq.eventloop import ioloop
from zmq.eventloop.zmqstream import ZMQStream
class Monitor(object):
def __init(self)
self.context = zmq.Context()
self.socket = self.context.socket(zmq.DEALER)
self.socket.connect("tcp//127.0.0.1:5055")
self.stream = ZMQStream(self._socket)
self.stream.on_recv(self.somefunc)
def initialize(self,id)
self._id = id
def somefunc(self, something)
"""work here and send back results if any """
import json
jdecoded = json.loads(something)
if self_id == jdecoded['_id']
""" good im the right monitor for you """
work = jdecoded['message']
results = algorithm (work)
self.socket.send(json.dumps(results))
else:
"""let some other process deal with it, not mine """
pass
class Prefect(object):
def __init(self, id)
self.context = zmq.Context()
self.socket = self.context.socket(zmq.DEALER)
self.socket.bind("tcp//127.0.0.1:5055")
self.stream = ZMQStream(self._socket)
self.stream.on_recv(self.check_if)
self._id = id
self.monitors = []
def check_if(self,message):
"""find out from message's id whether we have
started a proces for it previously"""
import json
jdecoded = json.loads(message)
this_id = jdecoded['_id']
if this_id in self.monitors:
pass
else:
"""start new process for it should have its won socket """
new = Monitor()
import Process
newp = Process(target=new.initialize,args=(this_id) )
newp.start()
self.monitors.append(this_id) ## ensure its remembered
Original question: here.

How do I refer to a socket as a class variable that I can use later?

In my case I have a texting app that has a receiver class that accepts incoming connections, and it has a pyqt signal connected to the main app class that will add messages to the screen upon being received. I want to be able to have two of these running and connect to eachother on different ports and send messages back and forth. This also means I want it to be able to run and listen for connections before itself connecting back.
I thought I'd create a connecter class, with the socket as a class variable, and then later instantiate that within the main app and link the send button pyqt signal to a send function that sends over that socket. It doesn't seem to like this. I get the error "socket.socket(socket.AF_INET, socket.SOCK_STREAM) as self.s" invalid syntax.
This is my code:
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
import socket
import numpy as np
from collections import deque, namedtuple
from threading import Thread
from time import sleep
HOST = '127.0.0.1'
inPORT = int(input('Input port to receive from: '))
outPORT = int(input('Input port to send to: '))
class Receiver(QThread):
received = pyqtSignal(object)
def __init__(self):
# self.outPORT = input('Input port to connect to: ')
super().__init__()
def run(self):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, inPORT))
s.listen()
print('Listening...')
conn, addr = s.accept()
with conn:
print('Connected by', addr)
while True:
data = conn.recv(1024)
self.received.emit(data)
class Connecter():
def __init__(self):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as self.c
self.c.connect((HOST, outPORT))
class ChatApp(QMainWindow):
def __init__(self):
super().__init__()
self.text_area = QTextEdit()
self.text_area.setReadOnly(True)
self.message_line = QLineEdit()
self.message_line.setFocus()
self.button = QPushButton('Send')
outerLayout = QVBoxLayout()
outerLayout.addWidget(self.text_area)
bottomLayout = QHBoxLayout()
bottomLayout.addWidget(self.message_line)
bottomLayout.addWidget(self.button)
outerLayout.addLayout(bottomLayout)
widget = QWidget()
widget.setLayout(outerLayout)
self.setWindowTitle("Chat App")
self.setWindowIcon(QIcon('ChatIcon.png'))
self.setCentralWidget(widget)
self.name = 'frd'
receiver = Receiver()
receiver.received.connect(self.addMessages)
self.threads = [receiver]
connecter = Connecter()
self.button.clicked.connect(self.sendMessage)
self.message_line.returnPressed.connect(self.sendMessage)
receiver.start()
# def askName(self):
# name, ask = QInputDialog.getText(self, 'input dialog', 'Enter username: ')
# return name
def sendMessage(self):
message = self.message_line.text()
connecter.c.sendall(message)
# headerArr = np.array([len(message)], dtype=np.uint16)
# header = headerArr.tobytes
def addMessages(self, message):
self.text_area.append(message.decode('utf-8'))
app = QApplication([])
myApp = ChatApp()
myApp.show()
app.exec()

The point of with is it creates something, then closes it again after the code inside the with is done. That doesn't apply here. It's not what you want the computer to do.
You can assign the socket to the variable like so:
self.c = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
and when you are done with the socket, close it like so:
self.c.close()

How to open serial port from a thread in Python

For a few weeks I'm stuck with, how to open a serial COM port from a thread,
to be able to write/read it from another thread. In my example, when I write to the port from another thread, there is access denied.
When I open the port from Main GUI, it works fine, I can write it from the thread, but I need to give a user possibility to choose the COM number.
This is my code, if anybody could take a look and help, it would be great...
class Connect(QThread):
connectResult = QtCore.pyqtSignal(str)
position1 = QtCore.pyqtSignal(str)
actuPosResult = QtCore.pyqtSignal(str)
def __init__(self, myvar, parent=None):
QThread.__init__(self, parent)
self.myvar = str(myvar) # value from spinbox
def run(self):
self.pserial = serial.Serial()
try:
COMnumber= self.myvar
self.pserial = serial.Serial('COM'+COMnumber, 115200,timeout=None)
r='COM '+COMnumber+' connected.'
self.pserial.write("c".encode('ascii')+"\n".encode('ascii'))
incomingByte = self.pserial.read()
decodedByte = incomingByte.decode("utf-8")
if decodedByte == ('c'):
r='Atmega On-Line'
self.connectResult.emit(r)
pos1='---'
self.position1.emit(pos1)
else :
r=' No answer from Atmega.'
self.connectResult.emit(r)
def stop(self):
self.terminate()
class ReadingEncoder(QThread):
actuPosResult = QtCore.pyqtSignal(str)
def __init__(self, mojazmienna, parent=None):
QThread.__init__(self, parent)
self.mojazmienna = str(mojazmienna)
def run(self):
Try:
self.pserial = serial.Serial()
self.pserial = serial.Serial('COM3', 115200,timeout=1)
self.pserial.write("p".encode('ascii')+"\n".encode('ascii'))
incomingByte = self.pserial.read()
decodedByte = incomingByte.decode("utf-8")
actualPos = ''
if decodedByte == ('a'):
while decodedByte != ('\n'):
incomingByte = self.pserial.read()
decodedByte = incomingByte.decode("utf-8")
actualPos = actualPos + decodedByte
pos= actualPos.rstrip('\n')# pozycja w formacie string
print(pos)
self.actuPosResult.emit(pos)
except (EOFError, OSError, IOError, ValueError, RuntimeError, BrokenPipeError, InterruptedError, TimeoutError):
print('Thread readingEncoder error')
self.pserial.close()

You just open the serial port and start the thread.
import atexit
class SerialPort(QThread):
connectResult = QtCore.pyqtSignal(str)
position1 = QtCore.pyqtSignal(str)
actuPosResult = QtCore.pyqtSignal(str)
def __init__(self, port=None, baud=115200, timeout=1):
super().__init__()
self.ser = serial.Serial()
self.ser.port = port
self.ser.baudrate = baud
self.ser.timeout = timeout
self.running = False
atexit.register(self.ser.close) # Make sure the serial port closes when you quit the program.
def set_port(port_num):
self.ser.port = "COM"+str(port_num)
def start(self, *args, **kwargs):
self.running = True
self.ser.open()
super().start()
self.ser.write("c\n".encode("ascii"))
def run(self):
while self.running:
try:
incomingByte = self.ser.read()
decodedByte = incomingByte.decode("utf-8")
if decodedByte == ('c'):
r='Atmega On-Line'
self.connectResult.emit(r)
pos1='---'
self.position1.emit(pos1)
else:
r=' No answer from Atmega.'
self.connectResult.emit(r)
except:
pass
# time.sleep(0.01) # You may want to sleep or use readline
You use this class by having a button call the start method.
serial = SerialPort("COM3")
btn = QtGui.QPushButton("Connect")
btn.clicked.connect(serial.start)
You can typically only open a serial port once unless you know that you have both ends available to you.
The serial port exists on the main thread and exists the entire time. You don't need to continuously open and close the serial port. All of the reading happens in the thread. To write just call serial.write(b"c\n"). You shouldn't need the writing to happen in the thread.
To have the user select the com port just use a QLineEdit.
myserial = QtGui.QLineEdit("3")
myserial.textChanged.connect(serial.set_port)

You're trying to open the port multiple times.
Instead of doing that, you have a couple of alternatives;
Create a Serial object once, and pass it to the thread that has to work with it.
Provide the name of the serial port to use to the the thread that has to work with it, and let it open the port.
In both cases make sure that your thread closes the port properly when it exits!

asyncore close old sockets

I have got tcp server on python with asyncore:
class AsyncClientHandler(asyncore.dispatcher_with_send):
def __init__(self,sock):
asyncore.dispatcher_with_send.__init__(self,sock)
self.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
self.message=""
self.protocol=Protocol(DBSession, logger)
def handle_read(self):
data = self.recv(8192)
if data:
self.message+=data
while TERMINATOR in self.message:
index=self.message.index(TERMINATOR)
msg=self.message[:index]
self.message=self.message[index+len(TERMINATOR):]
answer=self.protocol.process_msg(msg, DBSession, tarif_dict)
if answer:
msg = HEADER+answer+TERMINATOR
self.send(msg)
def handle_close(self):
self.close()
class AsyncServer(asyncore.dispatcher):
def __init__(self, host, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
self.set_reuse_addr()
self.bind((host, port))
self.listen(5)
def handle_accept(self):
pair = self.accept()
if pair is None:
pass
else:
sock, addr = pair
logging.info("Incoming connection from %s",repr(addr))
AsyncClientHandler(sock)
Some clients do not close the connection, so at some point the server crashes due to the large number of sockets.
How can I close an inactive socket after some time? settimeout not work.

To achieve this you could use TCP's Keepalive (like you already did) and set its delay, pings... But this apporach should only be used for long-lasting connections and is only available on Unixes. Have some read here.
You can also make some scheduling of sockets, closing them when some time passes or delay them when they're active. I made an example working with your code:
import sched, time
class SocketSched(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
self.to_run = []
self.scheds = {}
self.start()
def add(self, what):
self.to_run.append(what.values()[0])
self.scheds.update(what)
def run(self):
while True:
if self.to_run:
run = self.to_run.pop()
if not run.empty(): run.run()
else: self.to_run.append(run)
Here we define new class of scheduler in different thread - this is important, sched module would continuously block, like asyncore.loop().
This needs modificating your code a bit:
class AsyncClientHandler(asyncore.dispatcher_with_send):
def __init__(self,sock, sch_class):
...
self.delay = 10
self.sch_class = sch_class
self.sch = sched.scheduler(time.time, time.sleep)
self.sch_class.add({self.fileno(): self.sch})
self.event = self.sch_class.scheds[self.fileno()].enter(self.delay, 1, self.handle_close, ())
...
def delay_close(self):
self.sch_class.scheds[self.fileno()].cancel(self.event)
self.event = self.sch_class.scheds[self.fileno()].enter(self.delay, 1, self.handle_close, ())
...
def handle_close(self):
try:
self.sch_class.scheds[self.fileno()].cancel(self.event)
except:
pass
...
self.delay is a timeout in seconds. After this time passes, and no action delays it, socket will be closed. Line in handle_close() ensures it won't be called twice due to task in scheduler.
Now you have to add self.delay_close() to the beginning of every method that ensures socket is active, eg. handle_read().
Server class (gets instance of SocketSched and passes it to new channels):
class AsyncServer(asyncore.dispatcher):
def __init__(self, host, port, sch_class):
...
self.sch_class = sch_class
...
def handle_accept(self):
...
AsyncClientHandler(sock, self.sch_class)
Ready. Using this:
server = AsyncServer('', 1337, SocketSched())
asyncore.loop()
This solution works, but can be error-prone on some close events. Anyway, sockets will read, delay, and close when given timeout occurs. Unfortunately running such scheduling loop uses some CPU.

Sending pygames-event data to a twisted-server?

I'm running a pygames program with twisted and I'm having problems when it comes to sending data from within a pygame event.
First here is the server:
from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor
import simplejson
class Game_Data(LineReceiver):
def __init__(self, players, clients):
self.players = players
self.clients = clients
## the connectionMade method of LineReceiver is being used to create ##
## protocol instances of each client so it can send back any data it gets ##
def connectionMade(self):
new_player = 'player_' + str(len(self.players) + 1)
self.clients.append(self)
self.players.append(new_player)
self.players = simplejson.dumps(self.players)
for client in self.clients:
client.sendLine(self.players)
## what ever data the server receives it sends right back to any clients ##
def lineReceived(self,line):
self.line = line
print self.line
for client in self.clients:
client.sendLine(self.line)
class BlackFactory(Factory):
def __init__(self):
self.players = []
self.clients = []
def buildProtocol(self, addr):
return Game_Data(self.players, self.clients)
reactor.listenTCP(6000, BlackFactory())
Now for the client:
import pygame
from twisted.internet.protocol import Protocol, ClientFactory
from twisted.protocols.basic import LineReceiver
from twisted.internet.task import LoopingCall
from twisted.internet import reactor
class BlackClientProtocol(LineReceiver):
def __init__(self, recv, host):
self.recv = recv
self.host = host
def lineReceived(self, line):
self.recv(line)
print line ## prints out as expected ##
def connectionMade(self):
self.host = self
print self.host ## prints out as expected ##
class BlackClient(ClientFactory):
def __init__(self, recv, host):
self.recv = recv
self.host = host
def buildProtocol(self, addr):
return BlackClientProtocol(self.recv, self.host)
class Client(object):
def __init__(self):
pygame.init()
self.screen = pygame.display.set_mode((800, 600))
pygame.display.flip()
reactor.callLater(0.1, self.tick)
def new_line(self, line):
self.line = line
## this trial_func was to see if I could pass the BlackClient another argument ##
## to return an instance of the protocol to be called later in this class ##
def trial_func(self, host):
self.host = host
def tick(self):
for event in pygame.event.get():
if event.type == pygame.KEYDOWN and event.key == pygame.K_ESCAPE:
print self.line ## prints out as expected##
print self.host ## does not print out ???##
if __name__ == '__main__':
c = Client()
lc = LoopingCall(c.tick)
lc.start(0.1)
reactor.connectTCP('192.168.1.2', 6000, BlackClient(c.new_line, c.trial))
reactor.run()
Edit: This is a more explicit example of whats going on. With the comments being the hang-ups. As long as you have pygames installed this will run, with the pygame event being a simple KEYDOWN (escape key) event as the trigger for the client to send data.

I looked at your code and this seems to be the relevant part:
def tick(self):
flag = 0
for event in pygame.event.get():
...
## call an instance of the protocol and sendLine ##
BlackClient(self.new_line(allhands)) ## this line does not work ##
## send allhands data ##
...
You want so send data back to the server, but here you're just creating a new instance of BlackClient.
You probably want to use sendLine to send the line, but you need a reference to the current protocoll instance (or at least to this method).
This is an example how you could achieve this:
class Client(object):
...
# this method is going to be replaced
def sendLine(self, line):
pass
def tick(self):
flag = 0
for event in pygame.event.get():
...
## call sendLine, which is replaced by the sendLine method from the protocol ##
self.sendLine(yourStuff)
class BlackClient(ClientFactory):
def __init__(self, client):
# keep instance of client
self.client = client
def buildProtocol(self, addr):
# give client.new_line method to protocol
proto = BlackClientProtocol(self.client.new_line)
# replace client's sendLine with proto.sendLine
self.client.sendLine = proto.sendLine
return proto
if __name__ == '__main__':
c = Client()
lc = LoopingCall(c.tick)
lc.start(0.1)
protocoll =
reactor.connectTCP('192.168.1.2', 6000, BlackClient(c))
reactor.run()
This is just an example; you could probably clean up the code a bit. Maybe you want to use some kind of dispatcher (like pydispatch)