I am using multithreading for the first time and here is the problem in which I am stuck.
I have a class Workstation and want to use a separate thread for each instance of the class. Each instance has to draw some cell phone models defined in the global list, but multithreading does not work for all instances, it works well only for one object and the rest seem to hang.
Here is code for my class and in the main file I am using another thread(code for that also attached below the class code) that runs a while loop forever and inside that loop, I am calling "drawing_thread"
function for each object of workstation-class.
why threading only works for the first object, not for others
import requests, time, db_quries, threading
from datetime import datetime
import helper_functions as HF
LocIP = '192.168.100.100'
# Local port for server
LocPort = 4321
class Workstation:
def __init__(self,FCell):
"""
constructor for class workstation
:param FCell:
"""
self.FCell = FCell # cell number
#initial values of component's flag
self.frame_done = True
self.screen_done = False
self.kpad_done = False
#flags for order compeltion
self.make = 1
self.complete = 0
self.thread_flag=True
#mutators for object variables
def make_setter(self):
self.make = self.make+1
#print('FL1: Make ',self.make)
def set_m(self,m):
self.make=m
def complete_setter(self,complete):
self.complete =complete
#print('FL1: complete ',self.complete)
def set_frame(self, value):
self.frame_done = value
def set_screen(self, value):
self.screen_done = value
def set_kpad(self, value):
self.kpad_done = value
def set_thread_flag(self,value):
self.thread_flag = value
def drawing_thread(self,param):
self.thread_flag = False
drawing = threading.Thread(target=HF.production, args=param)
drawing.start()
function that called by thread in main file
def process_thread():
while True:
#print(len(OrderList))
if WS_obj_list[0].thread_flag and WS_obj_list[0].get_zone_status('Z1') != '-1':
WS_obj_list[0].drawing_thread((WS_obj_list[0], OrderList))
if WS_obj_list[1].thread_flag and WS_obj_list[1].get_zone_status('Z1') != '-1':#
WS_obj_list[1].drawing_thread((WS_obj_list[1], OrderList))
if WS_obj_list[2].thread_flag and WS_obj_list[2].get_zone_status('Z1') != '-1':
WS_obj_list[2].drawing_thread((WS_obj_list[2], OrderList))
time.sleep(1)
Related
I have the following situation:
class Test:
cities_visited: dict
#staticmethod
def prepare_city_dict(persons):
Test.cities_visited = {}
for i in range(len(persons)):
name = persons[i].surname
Test.cities_visited[name] = Test.create_visit()
#staticmethod
def create_visit():
counter: dict = {"City1": 0, "City2": 0, "City3": 0}
return counter
#staticmethod
def increment_visit(surname: str, key):
counter_visit = Test.cities_visited[surname]
current_value = counter_visit[key]
print(current_value)
counter_visit[key] = current_value + 1
Test.cities_visited[surname] = counter_visit
At start-up I am calling Test.prepare_city_dict, and then I create a thread and do a lock and call other stuff, at some point I try to increment 2 cities:
Test.increment_visit("Dummy", "City1")
Test.increment_visit("Dummy", "City2")
If I am trying to log how many times a city was visited, only the 'City1' is correctly implemented.
I am coming from a different language (which is pretty obvious I think :D), running my code in a docker container on the Windows OS, everything is incremented properly.
Running the same configuration (container) under Linux OS, only the first 'City1' is properly incremented.
I taught it was a race condition, but unfortunately I cannot reproduce it and I cannot figure out what is going on.
+++ UPDATE:
class TestClass:
def main():
Test.prepare_city_dict(persons)
lock = threading.Lock()
thread = threading.Thread(target=TestClass.process_message,
args=(lock, persons,))
thread.start()
def process_message(lock, persons):
lock.acquire()
Test.increment_visit("Dummy", "City1")
..... -> lots of calculations
Test.increment_visit("Dummy", "City2")
lock.release()
I am having trouble implementing the following scheme :
class A:
def __init__(self):
self.content = []
self.current_len = 0
def __len__(self):
return self.current_len
def update(self, new_content):
self.content.append(new_content)
self.current_len += 1
class B:
def __init__(self, id):
self.id = id
And I also have these 2 functions that will be called later in the main :
async def do_stuff(first_var, second_var):
""" this function is ideally called from the main in another
process. Also, first_var and second_var are not modified so it
would be nice if they could be given by reference without
having to copy them """
### used for first call
yield None
while len(first_var) < CERTAIN_NUMBER:
time.sleep(10)
while True:
## do stuff
if condition_met:
yield new_second_var ## which is a new instance of B
## continue doing stuff
def do_other_stuff(first_var, second_var):
while True:
queue = multiprocessing.JoinableQueue()
results = multiprocessing.Queue()
### do stuff
first_var.update(results)
The main looks like this at the moment :
first_var = A()
second_var = B()
while True:
async for new_second_var in do_stuff(first_var, second_var):
if new_second_var:
## stop the do_other_stuff that is currently running
## to re-launch it with the updated new_var
do_other_stuff(first_var, new_second_var)
else: ## used for the first call
do_other_stuff(first_var, second_var)
Here are my questions :
Is there a better solution to make this scheme work?
How can I implement the "stopping" part since there is a while True loop that fills first_var by reference?
Will the instance of A (first_var) be passed by reference to do_stuff if first_var doesn't get modified inside it?
Is it even possible to have an asynchronous generator in another process?
Is it even possible at all?
This is using Python 3.6 for the async generators.
I hope this is somewhat clear! Thanks a lot!
I have a program that have a gui in PyQt in the main thread. It communicates to a photo-detector and gets power readings in another thread, which sends a signal to the main thread to update the gui's power value.
Now I want to use a motor to automatically align my optical fiber, getting feedback from the photo-detector.
So I created a class that controls the motors, but I have to somehow pass the photo-detector readings to that class. First, I tried to access parent's power variable but it didn't work.
Then I created a method in my gui to return the variable's value and tried to access it from the motor class. I got a problem saying that I couldn't use parent's method without using its __init__ first. Is there a way to bypass it? I can't call the gui __init__ again, I just want to use one of its methods from within the child class.
If there is an alternative way to do this, I'd be happy as well.
PS: I guess I can't give the child class the photo-detector object because it is in another thread, right?
--Edit--
The gui code is:
class MyApp(QtGui.QMainWindow, Ui_MainWindow):
self.PDvalue = 0 #initial PD value
self.PDState = 0 #control the PD state (on-off)
self.PDport = self.dialog.pm100d.itemText(self.dialog.pm100d.currentIndex()) #gets pot info
def __init__(self):
... #a lot of other stuff
self.nano = AlgoNanoMax.NanoMax('COM12') #creates the motor object
self.nano_maxX.clicked.connect(self.NanoMaximizeX) #connect its fun to a buttom
self.actionConnect_PM100D.triggered.connect(self.ActionConnect_PM100D) #PD buttom
def NanoMaximizeX(self):
self.nano.maximize_nano_x() #uses motor object function
def ActionConnect_PM100D(self):
if self.PDState == 0: #check if PD is on
self.PD = PDThread(self.PDState, self.PDport) #creates thread
self.PD.valueupdate.connect(self.PDHandler) #signal connect
self.PD.dialogSignal.connect(self.PDdialog) #create error dialog
self.threads = []
self.threads.append(self.PD)
self.PD.start() #start thread
else:
self.PDState = 0
self.PD.state = 0 #stop thread
self.startpd.setText('Start PD') #change buttom name
def PDHandler(self, value):
self.PDvalue = value #slot to get pow from thread
def ReturnPow(self):
return self.PDvalue #return pow (I tried to use this to pass to the motor class)
def PDdialog(self):
self.dialog.set_instrument('PM100D') #I have a dialog that says error and asks you to type the right port
if self.dialog.exec_() == QtGui.QDialog.Accepted: #if Ok buttom try again
ret = self.dialog.pm100d.itemText(self.dialog.pm100d.currentIndex()) #new port
self.PD.port = str(ret)
self.PD.flagWhile = False #change PD stop loop condition to try again
else: #pressed cancel, so it gives up
self.PD.photodetector.__del__() #delete objects
self.PD.terminate() #stop thread
self.PD.quit()
Now the PD class, which is in another thread but in the same file as gui:
class PDThread(QtCore.QThread):
valueupdate = QtCore.pyqtSignal(float) #creating signals
dialogSignal = QtCore.pyqtSignal() #signal in case of error
state = 1 #used to stop thread
def __init__(self, state, port):
QtCore.QThread.__init__(self)
self.photodetector = PM100D() #creates the PD object
self.port = port
def run(self):
while True:
self.flagWhile = True #used to leave while
try:
self.photodetector.connect(self.port) #try to connect
except:
self.dialogSignal.emit() #emit error signal
while self.flagWhile == True:
time.sleep(0.5) #wait here until user press something in the dialog, which is in another thread
else:
break #leave loop when connected
window.PDState = 1 #change state of main gui buttom (change functionality to turn off if pressed again)
window.startpd.setText('Stop PD') #change buttom label
while self.state == 1:
time.sleep(0.016)
value = self.photodetector.get_pow() #get PD pow
self.valueupdate.emit(value) #emit it
The AlgoNanoMax file:
import gui
from NanoMax import Nano
class NanoMax(gui.MyApp): #inheriting parent
def __init__(self, mcontroller_port):
self.mcontroller = Nano(mcontroller_port) #mcontroller is the communication to the motor
def maximize_nano_x(self, step=0.001, spiral_number=3):
''' Alignment procedure with the nano motor X'''
print 'Optimizing X'
power = super(NanoMax, self).ReturnPow() #here I try to read from the photodetector
xpos = self.mcontroller.initial_position_x
position = []
position = [[power, xpos]]
xsign = 1
self.mcontroller.move_relative(self.mcontroller.xaxis, (-1) * spiral_number * step)
print 'X nano move: '+ str((-1) * spiral_number * step * 1000) + ' micrometers'
time.sleep(4)
power = super(NanoMax, self).ReturnPow()
xpos += (-1) * spiral_number * step
position.append([power, xpos])
for _ in xrange(2*spiral_number):
self.mcontroller.move_relative(self.mcontroller.xaxis, xsign * step)
print 'X nano move: '+ str(xsign * step * 1000) + ' micrometers'
time.sleep(5)
power = super(NanoMax, self).ReturnPow()
xpos += xsign * step
position.append([power, xpos])
pospower = [position[i][0] for i in xrange(len(position))]
optimalpoint = pospower.index(max(pospower))
x_shift = (-1) * (xpos - position[optimalpoint][1])
print 'Maximum power: ' + str(max(pospower)) + ' dBm'
print 'Current power: ' + str(super(NanoMax, self).ReturnPow()) + ' dBm'
self.mcontroller.move_relative(self.mcontroller.xaxis, x_shift)
The __init__ for NanoMax and MyApp should call super().__init__() to ensure initialization is done for all levels (if this is Python 2, you can't use no-arg super, so it would be super(NanoMax, self).__init__() and super(MyApp, self).__init__() respectively). This assumes the PyQT was properly written with new-style classes, and correct use of super itself; you're using super in other places, so presumably at least the former is true. Using super appropriately in all classes will ensure all levels are __init__-ed once, while manually listing super classes won't work in certain inheritance patterns, or might call some __init__s multiple times or not at all.
If there is a possibility that many levels might take arguments, you should also accept *args/**kwargs and forward them to the super().__init__ call so the arguments are forwarded where then need to go.
Combining the two, your code should look like:
class MyApp(QtGui.QMainWindow, Ui_MainWindow):
def __init__(self, *args, **kwargs):
super(MyApp, self).__init__(*args, **kwargs)
... rest of __init__ ...
class PDThread(QtCore.QThread):
def __init__(self, state, port, *args, **kwargs):
super(PDThread, self).__init__(*args, **kwargs)
...
class NanoMax(gui.MyApp): #inheriting parent
def __init__(self, mcontroller_port, *args, **kwargs):
super(NanoMax, self).__init__(*args, **kwargs)
self.mcontroller = Nano(mcontroller_port) #mcontroller is the communication to the motor
Note: If you've overloaded methods that the super class might call in its __init__ and your overloads depend on state set in your own __init__, you'll need to set up that state before, rather than after the super().__init__(...) call. Cooperative multiple inheritance can be a pain that way. Also note that using positional arguments for anything but the lowest level class can be ugly with multiple inheritance, so it may make sense to pass all arguments by keyword, and only accept and forward **kwargs, not *args, so people don't pass positional arguments in ways that break if the inheritance hierarchy changes slightly.
class MyApp(QtGui.QMainWindow, Ui_MainWindow):
self.PDvalue = 0 #initial PD value
self.PDState = 0 #control the PD state (on-off)
In the above code it is setting a variable outside of a function. To do this in a class don't put the self keyword in front of it. This way you can just have in the class definition
class MyApp(QtGui.QMainWindow, Ui_MainWindow):
PDvalue = 0 #initial PD value
PDState = 0 #control the PD state (on-off)
and in the super line
power = super(NanoMax, self).PDvalue
For example:
>>> class Hi:
H = 5
def __init__(self):
self.g = 6
>>> class Bye(Hi):
def H(self):
print(super(Bye, self).H)
>>> e = Bye()
>>> e.H()
5
>>>
So the situation is that I have multiple methods, which might be threaded simaltenously, but all need their own lock
against being re-threaded until they have run. They are established by initialising a class with some dataprocessing options:
class InfrequentDataDaemon(object): pass
class FrequentDataDaemon(object): pass
def addMethod(name):
def wrapper(f):
setattr(processor, f.__name__, staticmethod(f))
return f
return wrapper
class DataProcessors(object):
lock = threading.Lock()
def __init__(self, options):
self.common_settings = options['common_settings']
self.data_processing_configurations = options['data_processing_configurations'] #Configs for each processing method
self.data_processing_types = options['data_processing_types']
self.Data_Processsing_Functions ={}
#I __init__ each processing method as a seperate function so that it can be locked
for type in options['data_processing_types']:
def bindFunction1(name):
def func1(self, data=None, lock=None):
config = self.data_processing_configurations[data['type']] #I get the right config for the datatype
with lock:
FetchDataBaseStuff(data['type'])
#I don't want this to be run more than once at a time per DataProcessing Type
# But it's fine if multiple DoSomethings run at once, as long as each DataType is different!
DoSomething(data, config)
WriteToDataBase(data['type'])
func1.__name__ = "Processing_for_{}".format(type)
self.Data_Processing_Functions[func1.__name__] = func1 #Add this function to the Dictinary object
bindFunction1(type)
#Then I add some methods to a daemon that are going to check if our Dataprocessors need to be called
def fast_process_types(data):
if not example_condition is True: return
if not data['type'] in self.data_processing_types: return #Check that we are doing something with this type of data
threading.Thread(target=self.Data_Processing_Functions["Processing_for_{}".format(data['type'])], args=(self,data, lock)).start()
def slow_process_types(data):
if not some_other_condition is True: return
if not data['type'] in self.data_processing_types: return #Check that we are doing something with this type of data
threading.Thread(target=self.Data_Processing_Functions["Processing_for_{}".format(data['type'])], args=(self,data, lock)).start()
addMethod(InfrequentDataDaemon)(slow_process_types)
addMethod(FrequentDataDaemon)(fast_process_types)
The idea is to lock each method in
DataProcessors.Data_Processing_Functions - so that each method is only accessed by one thread at a time (and the rest of the threads for the same method are queued). How does Locking need to be set up to achieve this effect?
I'm not sure I completely follow what you're trying to do here, but could you just create a separate threading.Lock object for each type?
class DataProcessors(object):
def __init__(self, options):
self.common_settings = options['common_settings']
self.data_processing_configurations = options['data_processing_configurations'] #Configs for each processing method
self.data_processing_types = options['data_processing_types']
self.Data_Processsing_Functions ={}
self.locks = {}
#I __init__ each processing method as a seperate function so that it can be locked
for type in options['data_processing_types']:
self.locks[type] = threading.Lock()
def bindFunction1(name):
def func1(self, data=None):
config = self.data_processing_configurations[data['type']] #I get the right config for the datatype
with self.locks[data['type']]:
FetchDataBaseStuff(data['type'])
DoSomething(data, config)
WriteToDataBase(data['type'])
func1.__name__ = "Processing_for_{}".format(type)
self.Data_Processing_Functions[func1.__name__] = func1 #Add this function to the Dictinary object
bindFunction1(type)
#Then I add some methods to a daemon that are going to check if our Dataprocessors need to be called
def fast_process_types(data):
if not example_condition is True: return
if not data['type'] in self.data_processing_types: return #Check that we are doing something with this type of data
threading.Thread(target=self.Data_Processing_Functions["Processing_for_{}".format(data['type'])], args=(self,data)).start()
def slow_process_types(data):
if not some_other_condition is True: return
if not data['type'] in self.data_processing_types: return #Check that we are doing something with this type of data
threading.Thread(target=self.Data_Processing_Functions["Processing_for_{}".format(data['type'])], args=(self,data)).start()
addMethod(InfrequentDataDaemon)(slow_process_types)
addMethod(FrequentDataDaemon)(fast_process_types)
I have a class for rooms. I want that every time I create an object using that class the object would be added to a list of all rooms.
Rooms class:
class Rooms:
"""Room class, takes type,days, occupied or not and when it frees up"""
def __init__(self, room_type, days, occupied, when_free):
self.room_type = room_type
self.days = days
self.occupied = occupied
self.when_free = arrow.get(when_free,'YYYY-MM-DD')
Any other feedback is appreciated as well!
also not sure if I should create new topic on this but is it possible that when the object is created and True on occupied is passed to the object you wouldn't need to pass 4th variable and it would take it as the current date instead? in short if there is no 4th variable it passes arrow.get(str(arrow.utcnow()),'YYYY-MM-DD') instead
figured out my second issue. I did change the init to:
def __init__(self, room_type, days, occupied, when_free=str(arrow.get(str(arrow.utcnow()),'YYYY-MM-DD'))):
self.room_type = room_type
self.days = days
self.occupied = occupied
self.when_free = arrow.get(when_free,'YYYY-MM-DD')
I would suggest a slightly more elegant and logical way than the above:
class Building(object):
def __init__(self):
self.rooms = []
class Room(object):
def __init__(self, building=None)
if building:
building.rooms.append(self)
self.building = building
b = Building()
r = Room(b)
That way, you don't need every time call b.rooms.append and now it more agreese with OOP.
Ideally, you would want the scope of your room list to be where you plan to use it. Not as part of a room itself. So, if you have a building with rooms:
class Building():
def __init__(self):
self.rooms = []
b = Building()
b.rooms.append(Room(room_type, days, occupied, when_free))
The building is just for an example. The important part is rooms.append(). That should be declared and used wherever you actually need to use the list of rooms.
Might be better just to make the list a class variable:
class Room(object):
rooms = []
def __init__(self, room_type, days, occupied, when_free):
self.room_type = room_type
self.days = days
self.occupied = occupied
self.when_free = arrow.get(when_free,'YYYY-MM-DD')
Room.rooms.append(self)
r = Room('x', 1,2, True)
Room.rooms
[<Room object at 0x00000000C6325550>]
r.rooms
[<Room object at 0x00000000C6325550>]
Since it's a class variable, you can get to it through any class instance, or the class type itself.
edited to go through 'Room' instead of 'self', which is safer...
I was thinking you could decorate the __init__ method with a decorator that appends the instance to a list, to avoid cluttering the __init__ method with the instance registering. Now you only have to add one decorator to each class' init method if you want to keep track of the instances. Something like:
#!/usr/bin/env python3
import sys
class InstanceRegister:
def __call__(self, init):
def register(instance, *args, **kwargs):
init(instance, *args, **kwargs)
try :
instance.__class__.__instances__
except:
instance.__class__.__instances__ = []
instance.__class__.__instances__.append(instance)
return register
class Room:
"""Room class, takes type,days, occupied or not and when it frees up"""
#InstanceRegister()
def __init__(self, room_type, days, occupied, when_free):
self.room_type = room_type
self.days = days
self.occupied = occupied
self.when_free = arrow.get(when_free,'YYYY-MM-DD')
def __str__(self):
return "Room of type {:s}".format(self.room_type)
def main():
r1 = Room('type_a', 1, True, '1999-12-30')
r2 = Room('type_b', 2, True, '2000-12-30')
r3 = Room('type_c', 3, True, '2001-01-30')
for room in Room.__instances__:
print("{:s}".format(room))
return 0
if __name__ == '__main__':
sys.exit(main())
More on decorators at Understanding Python Decorators in 12 Easy Steps!