I'm having an issue with threading that I can't solve in any way I've tried. I searched in StackOverflow too, but all I could find was cases that didn't apply to me, or explanations that I didn't understand.
I'm trying to build an app with BottlePy, and one of the features I want requires a function to run in background. For this, I'm trying to make it run in a thread. However, when I start the thread, it runs twice.
I've read in some places that it would be possible to check if the function was in the main script or in a module using if __name__ == '__main__':, however I'm not able to do this, since __name__ is always returning the name of the module.
Below is an example of what I'm doing right now.
The main script:
# main.py
from MyClass import *
from bottle import *
arg = something
myObject = Myclass(arg1)
app = Bottle()
app.run('''bottle args''')
The class:
# MyClass.py
import threading
import time
class MyClass:
def check_list(self, theList, arg1):
a_list = something()
time.sleep(5)
self.check_list(a_list, arg1)
def __init__(self, arg1):
if __name__ == '__main__':
self.a_list = arg.returnAList()
t = threading.Thread(target=self.check_list, args=(a_list, arg1))
So what I intend here is to have check_list running in a thread all the time, doing something and waiting some seconds to run again. All this so I can have the list updated, and be able to read it with the main script.
Can you explain to me what I'm doing wrong, why the thread is running twice, and how can I avoid this?
This works fine:
import threading
import time
class MyClass:
def check_list(self, theList, arg1):
keep_going=True
while keep_going:
print("check list")
#do stuff
time.sleep(1)
def __init__(self, arg1):
self.a_list = ["1","2"]
t = threading.Thread(target=self.check_list, args=(self.a_list, arg1))
t.start()
myObject = MyClass("something")
Figured out what was wrong thanks to the user Weeble's comment. When he said 'something is causing your main.py to run twice' I remembered that Bottle has an argument that is called 'reloader'. When set to True, this will make the application load twice, and thus the thread creation is run twice as well.
Related
I'm working on terminal based PyQt application using Windows OCX api.
And I'm having trouble over implement(calling exec) QEventLoop in QAxWidget instance.
If I call exec() in main QApplication main loop, everthing is fine.
But, calling exec() in instance of QAxWidget which is instantiated in main loop, it does not work as expected.
When exec() called(_on_event_connect in code), error message showed up like
"QEventLoop:exec: instance 0x18479d8 has already called exec()"
And on the moment of calling exit(), QEventLoop seems not running.
And the application is hang after this.
The below is simplified example of my code, It tries call QAxWidget method "dynamicCall" as soon as log-in process finished.
It consists of pyQt signal/slot mechanism, get event from API server.
Please understand that you can not execute this code because of it needs specific APIs installation and registered user-id.
But I hope you can see the points of problem.
main.py
import sys
from PyQt5.QtWidgets import QApplication
from api import OpenApi
class Main():
def __init__(self):
self.api = OpenApi()
self.api.comm_connect()
# self.api.req_basic_stock_info("035420") -> if I call here, works well
if __name__ == "__main__":
app = QApplication(sys.argv)
main = Main()
sys.exit(app.exec_())
api.py
import sys
from PyQt5.QtWidgets import *
from PyQt5.QAxContainer import *
from PyQt5.QtCore import *
class OpenApi(QAxWidget):
def __init__(self):
super().__init__()
self.setControl("KHOPENAPI.KHOpenAPICtrl.1")
self.OnEventConnect.connect(self._on_event_connect)
self.OnReceiveTrData.connect(self._on_receive_tr_data)
self._qloop = QEventLoop()
def loop(self):
if not self._qloop.isRunning():
logger.info(f"-->exec")
self._qloop.exec()
else:
logger.info("exec skipped")
def unloop(self):
if self._qloop.isRunning():
logger.info("-->exit")
self._qloop.exit()
else:
logger.info(f"exit skipped")
def comm_connect(self):
self.dynamicCall("CommConnect()")
self.loop()
def comm_rq_data(self, rqname, trcode, next, screen_no):
self.dynamicCall("CommRqData(QString, QString, int, QString)", rqname, trcode, next, screen_no)
self.loop()
def _on_event_connect(self, err_code):
if err_code == 0:
logger.info("logged in")
else:
logger.info("log-in failed")
self.unloop()
# If I call this function here, QEventLoop fails
self.req_basic_stock_info("000660")
def _on_receive_tr_data(self, screen_no, rqname, trcode, record_name, next,
unused1, unused2, unused3, unused4):
logger.info(f"OnReceivTrData: {rqname}")
self.unloop()
def req_basic_stock_info(self, code):
# I want to call this function inside instance on certain condition.
self.set_input_value("item", code)
scrno = "2000"
self.comm_rq_data("ITEM_INFO_REQ", "opt10080", "0", scrno)
And output is like below.
12:42:53,54 test INFO -->exec
12:42:58,961 test INFO logged in
12:42:58,962 test INFO -->exit
12:42:58,977 test INFO -->exec
QEventLoop::exec: instance 0x35c34a0 has already called exec()
12:42:59,33 test INFO OnReceivTrData:ITEM_INFO_REQ
12:42:59,35 test INFO exit skipped
As eyllanesc told me, I changed functions related assign/deassign QEventLoop like below. And it works.
def loop(self):
self._qloop = QEventLoop()
self._qloop.exec()
def unloop(self):
try:
self._qloop.exit()
except AttributeError as e:
logger.error(e)
update
I'm afraid this issue has not been solved yet. It was successful when req_basic_stock_info function is called on _on_event_connect event, But I call it at another event function, it hang again without any error message. There is no difference in calling mechanism.
Calling the exit() function does not imply that the QEventLoop can be reused, therefore the error message is signaled. In this case it is better to use a new QEventLoop:
def unloop(self):
if self._qloop.isRunning():
logger.info("-->exit")
self._qloop.exit()
self._qloop = QEventLoop()
else:
logger.info(f"exit skipped")
I'm asking this question in a more broad spectrum because I'm not facing this specific issue right now, but I'm wondering how to do it in the future.
If I have a long running python script, that is supposed to do something all the time (could be a infine loop, if that helps). The code is started by running python main.py command on a terminal.
The code doesn't have an ending, so there will be no sys.exit().
I don't want to use KeyboardInterrupt and I don't want to kill the task. Because those options are abrupt, and you can't predict precisely at what point you are stoping the code.
Is there a way to 'softly' terminate the code when I eventually decide to fo it? For example using another command, preparing a class or running another script?
What would be the best practice for this?
PS.: Please, bear in mind that I'm a novice coder.
EDIT:
I'm adding some generic code, in order to make my question clearer.
import time,csv
import GenericAPI
class GenericDataCollector:
def __init__(self):
self.generic_api = GenericAPI()
def collect_data(self):
while True: #Maybe this could be a var that is changed from outside of the class?
data = self.generic_api.fetch_data() #Returns a JSON with some data
self.write_on_csv(data)
time.sleep(1)
def write_on_csv(self, data):
with open('file.csv','wt') as f:
writer = csv.writer(f)
writer.writerow(data)
def run():
obj = GenericDataCollector()
obj.collect_data()
if __name__ == "__main__":
run()
In this particular case, the class is collecting data from some generic API (that comes in JSON) and writing it in a csv file, in a infinite loop. How could I code a way (method?) to stop it (when called uppon, so unexpected), without abruptly interrupting (Ctrl+C or killing task).
I would recommend use the signal module. This allows you to handle signal interrupts (SIGINT) and clean up the program before your exit. Take the following code for example:
import signal
running = True
def handle(a, b):
global running
running = False
# catch the SIGINT signal and call handle() when the process
# receives it
signal.signal(signal.SIGINT, handle)
# your code here
while running:
pass
You can still exit with a Ctrl+C, but what you put in the while loop will not be cut off half way.
Based on #Calder White, how about this (not tested):
import signal
import time,csv
import GenericAPI
class GenericDataCollector:
def __init__(self):
self.generic_api = GenericAPI()
self.cont = True
def collect_data(self):
while self.cont:
signal.signal(signal.SIGINT, self.handle)
data = self.generic_api.fetch_data() #Returns a JSON with some data
self.write_on_csv(data)
time.sleep(1)
def handle(self):
self.cont = False
def write_on_csv(self, data):
with open('file.csv','wt') as f:
writer = csv.writer(f)
writer.writerow(data)
def run():
obj = GenericDataCollector()
obj.collect_data()
if __name__ == "__main__":
run()
I have a situation where i have to call two different methods to run in parallel. I am using Python threading module to achieve this. But, instead of two methods running in parallel, they are running sequentially. Can someone please guide me in what is the wrong in my code?
This is for Python 3.5, using threading module and have a class with two different methods which has to be run in parallel.
## This is in template.py
from threading import Thread
import time
class createTemplate:
def __init__(self,PARAM1):
self.PARAM1=PARAM1
def method1(self):
print("Method1-START")
time.sleep(120)
print("Method1-END")
def method2(self):
print("Method2-START")
time.sleep(120)
print("Method2-END")
def final_method(self):
if self.PARAM1=="1":
m1=Thread(target=self.method1)
m1.run()
if self.PARAM1=="1":
m2=Thread(target=self.method2)
m2.run()
## This is in createTemplate.py
from template import createTemplate
template = createTemplate("1")
template.final_method()
Actual Output:
Method1-START
Method1-END
Method2-START
Method2-END
Expected Output:
Method1-START
Method2-START
Method1-END
Method2-END
instead of .run() you should call .start()
Thread.run() will run the code in the context of the current thread, but Thread.start() will actually spawn a new thread and run the code on it in parallel to existing threads.
try this:
from threading import Thread
import time
class createTemplate:
def __init__(self,PARAM1):
self.PARAM1=PARAM1
def method1(self, arg):
print("Method1-START",arg)
time.sleep(5)
print("Method1-END",arg)
def method2(self,arg):
print("Method2-START",arg)
time.sleep(5)
print("Method2-END",arg)
def final_method(self):
if self.PARAM1=="1":
m1=Thread(target=self.method1, args=("A", )) # <- this is a tuple of size 1
m1.start()
if self.PARAM1=="1":
m2=Thread(target=self.method2, args=("B", )) # <- this is a tuple of size 1
m2.start()
I have a function that calls a class, after the class has been initilized it will continue down the function, as it should. However i want to pause after the class has been initilized until the asyncio is closed from inside the class.
Something like this:
from PyHook3 import HookManager
import asyncio
class Keystroke_Watcher(object):
def __init__(self, event_loop):
self.event_loop = event_loop
self.hm = HookManager()
self.hm.KeyDown = self.on_keyboard_event
self.hm.HookKeyboard()
def on_keyboard_event(self, event):
if event.KeyID == 65: # A
self.shutdown()
return True
def shutdown(self):
self.hm.UnhookKeyboard()
self.event_loop.stop()
if __name__ == '__main__':
loop = asyncio.get_event_loop()
watcher = Keystroke_Watcher(loop)
loop.run_forever()
print("Done!")
The idea is the "main" code shouldn't continue until after the "A" key has been pressed. However with this current code it will freeze all keyboard actions, and it won't stop even after A is pressed.
Am I using the wrong asyncio loop?
Doesn't it work well with classes?
What would be the best solution to this problem? (doesn't necessarily to involve asyncio in solution)
Note: I know that Qt has something that does exactly this, however i don't feel like including the entire qt library. QEventLoop().exec() to start it and QEventLoop().exit() to end it.
I'm trying out threads in python. I want a spinning cursor to display while another method runs (for 5-10 mins). I've done out some code but am wondering is this how you would do it? i don't like to use globals, so I assume there is a better way?
c = True
def b():
for j in itertools.cycle('/-\|'):
if (c == True):
sys.stdout.write(j)
sys.stdout.flush()
time.sleep(0.1)
sys.stdout.write('\b')
else:
return
def a():
global c
#code does stuff here for 5-10 minutes
#simulate with sleep
time.sleep(2)
c = False
Thread(target = a).start()
Thread(target = b).start()
EDIT:
Another issue now is that when the processing ends the last element of the spinning cursor is still on screen. so something like \ is printed.
You could use events:
http://docs.python.org/2/library/threading.html
I tested this and it works. It also keeps everything in sync. You should avoid changing/reading the same variables in different threads without synchronizing them.
#!/usr/bin/python
from threading import Thread
from threading import Event
import time
import itertools
import sys
def b(event):
for j in itertools.cycle('/-\|'):
if not event.is_set():
sys.stdout.write(j)
sys.stdout.flush()
time.sleep(0.1)
sys.stdout.write('\b')
else:
return
def a(event):
#code does stuff here for 5-10 minutes
#simulate with sleep
time.sleep(2)
event.set()
def main():
c = Event()
Thread(target = a, kwargs = {'event': c}).start()
Thread(target = b, kwargs = {'event': c}).start()
if __name__ == "__main__":
main()
Related to 'kwargs', from Python docs (URL in the beginning of the post):
class threading.Thread(group=None, target=None, name=None, args=(), kwargs={})
...
kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.
You're on the right track mostly, except for the global variable. Normally you'd needed to coordinate access to shared data like that with a lock or semaphore, but in this special case you can take a short-cut and just use whether one of the threads is running or not instead. This is what I mean:
from threading import Thread
from threading import Event
import time
import itertools
import sys
def monitor_thread(watched_thread):
chars = itertools.cycle('/-\|')
while watched_thread.is_alive():
sys.stdout.write(chars.next())
sys.stdout.flush()
time.sleep(0.1)
sys.stdout.write('\b')
def worker_thread():
# code does stuff here - simulated with sleep
time.sleep(2)
if __name__ == "__main__":
watched_thread = Thread(target=worker_thread)
watched_thread.start()
Thread(target=monitor_thread, args=(watched_thread,)).start()
This is not properly synchronized. But I will not try to explain it all to you right now because it's a whole lot of knowledge. Try to read this: http://effbot.org/zone/thread-synchronization.htm
But in your case it's not that bad that things aren't synchronized correctyl. The only thing that could happen, is that the spining bar spins a few ms longer than the background task actually needs.