i write a program to scrape data from website using requests and tkinter
there are start butoon that call Start_collecting_data function and it work good
i tried to add thread to my code to work faster so i changed command in start button to call threading_system function. the code give me no responce after add thread to threads list
def threading_system(self):
print('start threading')
self.threads = []
for i in range(100):
print(f'add thread number {i}')
i = threading.Thread(target=self.Start_collecting_data)
print('thread added')
i.start()
print('thread start')
self.threads.append(i)
print('thread append')
for thread2 in self.threads:
print('start joining')
thread2.join()
print('thread join')
print('end of function')
all {for loops} work correctly and print all strings, except the last line {print('end of function')} not be printed
Related
I'm writing a timer in python. When the timer reaches 0, I want the thread I made to automatically exit.
class Rollgame:
timer = 0
def timerf(self, timer):
self.timer = timer
while self.timer > 0:
time.sleep(0.1)
self.timer -= 0.1
sys.exit(0)
Is this a valid way to exit a thread? It seems to be working in the context of the program im building, however I'm not sure if it's a good way to do it.
If I ever choose to implement this in something like a flask/django app, will this still be valid?
Sorry if the question seems stupid or too simple, I've never worked with threading in python before.
In general, killing threads abruptly is considered a bad programming practice. Killing a thread abruptly might leave a critical resource that must be closed properly, open. But you might want to kill a thread once some specific time period has passed or some interrupt has been generated. There are the various methods by which you can kill a thread in python.
Set/Reset stop flag :
In order to kill a threads, we can declare a stop flag and this flag will be check occasionally by the thread. For Example:
# Python program showing
# how to kill threads
# using set/reset stop
# flag
import threading
import time
def run():
while True:
print('thread running')
global stop_threads
if stop_threads:
break
stop_threads = False
t1 = threading.Thread(target = run)
t1.start()
time.sleep(1)
stop_threads = True
t1.join()
print('thread killed')
In the above code, as soon as the global variable stop_threads is set, the target function run() ends and the thread t1 can be killed by using t1.join(). But one may refrain from using global variable due to certain reasons. For those situations, function objects can be passed to provide a similar functionality as shown below:
# Python program killing
# threads using stop
# flag
import threading
import time
def run(stop):
while True:
print('thread running')
if stop():
break
def main():
stop_threads = False
t1 = threading.Thread(target = run, args =(lambda : stop_threads, ))
t1.start()
time.sleep(1)
stop_threads = True
t1.join()
print('thread killed')
main()
Using traces to kill threads :
This methods works by installing traces in each thread. Each trace terminates itself on the detection of some stimulus or flag, thus instantly killing the associated thread. For Example:
# Python program using
# traces to kill threads
import sys
import trace
import threading
import time
class thread_with_trace(threading.Thread):
def __init__(self, *args, **keywords):
threading.Thread.__init__(self, *args, **keywords)
self.killed = False
def start(self):
self.__run_backup = self.run
self.run = self.__run
threading.Thread.start(self)
def __run(self):
sys.settrace(self.globaltrace)
self.__run_backup()
self.run = self.__run_backup
def globaltrace(self, frame, event, arg):
if event == 'call':
return self.localtrace
else:
return None
def localtrace(self, frame, event, arg):
if self.killed:
if event == 'line':
raise SystemExit()
return self.localtrace
def kill(self):
self.killed = True
def func():
while True:
print('thread running')
t1 = thread_with_trace(target = func)
t1.start()
time.sleep(2)
t1.kill()
t1.join()
if not t1.isAlive():
print('thread killed')
In this code, start() is slightly modified to set the system trace function using settrace(). The local trace function is defined such that, whenever the kill flag (killed) of the respective thread is set, a SystemExit exception is raised upon the excution of the next line of code, which end the execution of the target function func. Now the thread can be killed with join().
Finally, Using the multiprocessing module to kill threads :
The multiprocessing module of Python allows you to spawn processes in the similar way you spawn threads using the threading module. The interface of the multithreading module is similar to that of the threading module. For Example, in a given code we created three threads(processes) which count from 1 to 9. Now, suppose we wanted to terminate all of the threads. You could use multiprocessing to do that.
# Python program killing
# a thread using multiprocessing
# module
import multiprocessing
import time
def func(number):
for i in range(1, 10):
time.sleep(0.01)
print('Processing ' + str(number) + ': prints ' + str(number*i))
# list of all processes, so that they can be killed afterwards
all_processes = []
for i in range(0, 3):
process = multiprocessing.Process(target=func, args=(i,))
process.start()
all_processes.append(process)
# kill all processes after 0.03s
time.sleep(0.03)
for process in all_processes:
process.terminate()
To sum it up, there are many ways to terminate threads, but I peronally wouldn't use sys.exit().
I have a function text_to_speach that receives a text and speak it with gtts module:
def text_to_speach(text):
try:
tts = gTTS(text=text, lang='fr', tld="com", slow=False)
tts.save("audio.mp3")
playsound.playsound('audio.mp3', False)
os.remove('audio.mp3')
except:
print('Check Speak issue !!!')
That function runs inside a Thread:
def speak(reply):
thread = Thread(target=text_to_speach, args=(reply,), daemon=True)
thread.start()
Now every time I run the speak() function it creates a Thread
and I don't want it to create multiple Threads.
So I want every time I run speak function the Thread will end after that.
Example:
speak("some text")
#Thread end
speak("some text 2")
#Thread end
speak("some text 3")
#Thread end
So my question is how to end the thread?
Blocking solution:
def speak(reply):
thread = Thread(target=text_to_speach, args=(reply,), daemon = True)
thread.start()
thread.join() # it will block till thread ends
Python3.6
First I put some items in a queue, then start a thread and called join() of the queue in the main thread, then called get() in the thread loop, when the size of queue == 0, I called task_done() and break loop and exit from the thread. But the join() method still blocked in the main thread. I can not figure out what`s wrong.
Below is the code
Thanks
import queue
import threading
def worker(work_queue):
while True:
if work_queue.empty():
print("Task 1 Over!")
work_queue.task_done()
break
else:
_ = work_queue.get()
print(work_queue.qsize())
# do actual work
def main():
work_queue = queue.Queue()
for i in range(10):
work_queue.put("Item %d" % (i + 1))
t = threading.Thread(target=worker, args=(work_queue, ))
t.setDaemon(True)
t.start()
print("Main Thread 1")
work_queue.join()
print("Main Thread 2")
t.join()
print("Finish!")
if __name__ == "__main__":
main()
task_done should be called for each work item which is dequeued and processed, not once the queue is entirely empty. (There'd be no reason for that-- the queue already knows when it's empty.) join() will block until task_done has been called as many times as put was called.
So:
def worker(work_queue):
while True:
if work_queue.empty():
print("Task 1 Over!")
break
else:
_ = work_queue.get()
print(work_queue.qsize())
# do actual work
Note that it's weird for a worker to exit as soon as it sees an empty queue. Normally it would get() with blocking, and only exit when it got a "time to exit" work item out of the queue.
I am trying to implement a timer for calling the function for printing the queue after specific time .I am also able to cancel the timer if the queue gets filled before that specified time and print the queue.But after that my timer object behaves abruptly causing timers to overlap for example if the queue gets filled in 2 sec then it prints the queue in 2,8,2,8... time interval instead of 2,10 time interval
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1'))
channel = connection.channel()
channel.queue_declare(queue='final', durable=True)
global msg_queue
global t
msg_queue=queue.Queue(maxsize=6)
def empty_queue():
print(time.time())
l=[]
i=int(msg_queue.qsize())
while i!=0:
l.append(msg_queue.get())
i-=1
t=threading.Timer(10,empty_queue)
print(l)
t.start()
t=threading.Timer(10,empty_queue)
print(time.time())
t.start()
while True:
if int(msg_queue.qsize())<6:
consume_generator = channel.consume(queue='final', no_ack=True)
result=next(consume_generator)
msg_queue.put(json.loads(result[2].decode("utf-8")))
else:
print("more",time.time())
t.cancel()
empty_queue()
I have solved the issue by cancelling the timer to prevent its own duplicacy
def empty_queue():
global t
print(time.time())
l=[]
i=int(msg_queue.qsize())
while i!=0:
l.append(msg_queue.get())
i-=1
if t.isAlive():
t.cancel()
t=threading.Timer(10,empty_queue)
print(l)
t.start()
You could nest a if statement inside the timer that says if queue is full then disable this if statement then let the timer continue until its done without affecting the queue. I don't think it will cause conflict with your program because the timer is probably a closure.
I am trying to do multi threading to check the network connection. My code is:
exitFlag = 0
lst_doxygen=[]
lst_sphinx=[]
class myThread (threading.Thread):
def __init__(self, counter):
threading.Thread.__init__(self)
self.counter=counter
def run(self):
print "Starting thread"
link_urls(self.counter)
def link_urls(delay):
global lst_doxygen
global lst_sphinx
global exitFlag
while exitFlag==0:
try:
if network_connection() is True:
try:
links = lxml.html.parse(gr.prefs().get_string('grc', 'doxygen_base_uri', '').split(',')[1]+"annotated.html").xpath("//a/#href")
for url in links:
lst_doxygen.append(url)
links = lxml.html.parse(gr.prefs().get_string('grc', 'sphinx_base_uri', '').split(',')[1]+"genindex.html").xpath("//a/#href")
for url in links:
lst_sphinx.append(url)
exitFlag=1
except IOError, AttributeError:
pass
time.sleep(delay)
print "my"
except KeyboardInterrupt:
exitFlag=1
def network_connection():
network=False
try:
response = urllib2.urlopen("http://google.com", None, 2.5)
network=True
except urllib2.URLError, e:
pass
return network
I have set a flag to stop the thread inside while loop. I also want to exit the thread by pressing Ctrl-C. So I have used try-except but thread is still working and does not exit. If I try to use
if KeyboardInterrupt:
exitFlag=1
instead of try-except, thread just works for first time execution of while loop and then exist.
p.s.
I have created the instance of myThread class in another module.
Finally, I got the answer of my question. I need to flag my thread as Daemon. So when I will create the instance if myThread class, I will add one more line:
thread1.myThread(2)
thread1.setDaemon(True)
thread1.start()
You only get signals or KeyboardInterrupt on the main thread. There are various ways to handle it, but perhaps you could make exitFlag a global and move the exception handler to your main thread.
Here is how I catch a CTRL-C in general.
import time
import signal
import sys
stop = False
def run():
while not stop:
print 'I am alive'
time.sleep(3)
def signal_handler(signal, frame):
global stop
print 'You pressed Ctrl+C!'
stop = True
t1 = threading.Thread(target=run)
t1.start()
signal.signal(signal.SIGINT, signal_handler)
print 'Press Ctrl+C'
signal.pause()
output:
python threads.py
Press Ctrl+C
I am alive
I am alive
^CYou pressed Ctrl+C!