Develop Reference

Python - Multithreading not working (Pulse ProgressBar)

In my programm (PyQT) I need to run a heavy operation, and I want to display an indicator (in this case a Progress Bar) during the process. The Progress Bar can be set to "Waiting mode" by doing self.progressBar.setRange(0,0).
The Problem: My bar always goes from 0 (not even started) to 100 (Done). Never is set to "Waiting mode". I am using a global variable to see if the process is done, and give the information to both classes or threads.
Here´s the code.
# Link to GUI
qtCreatorFile = "interaface.ui"
Ui_MainWindow, QtBaseClass = uic.loadUiType(qtCreatorFile)
ready = False
class Process(QtGui.QMainWindow, Ui_MainWindow):
def __init__(self, parent = None):
# Setting-ip UI
QtGui.QMainWindow.__init__(self)
Ui_MainWindow.__init__(self)
self.setupUi(self)
# Setting Progress Bar to 0 and initial message
self.progressBar.setValue(0)
self.resultLabel.setText("Not ready")
# Button
self.runButton.clicked.connect(self.onStart)
# Thread
self.myLongTask = TaskThread()
self.myLongTask.taskFinished.connect(self.onFinished)
def onStart(self):
global ready
ready = False
print "Activate Waiting Bar Mode and message"
self.progressBar.setRange(0,0)
self.resultLabel.setText("Wait")
print "Deactivate Waiting Bar Mode and message"
print "INITIALIZE THREAD"
self.myLongTask.start()
# Simulate heavy process
print "DOING TASK..."
i = 1
for i in range (9999999):
u = i*2
ready = True
print "TASK DONE"
def onFinished(self):
print "onFinished executed and set Progress Bar to 1"
self.progressBar.setRange(0,1)
self.progressBar.setValue(1)
self.resultLabel.setText("Done")
class TaskThread(QtCore.QThread):
taskFinished = QtCore.pyqtSignal()
def run(self):
global ready
temp = True
while temp == True:
if ready == False:
print "(THREAD)Process not finished yet"
else:
print "(THREAD)Process done"
self.taskFinished.emit()
temp = False
def main():
app = QtGui.QApplication(sys.argv)
window = BagTool()
window.show()
sys.exit(app.exec_())
if __name__ == '__main__':
main()
PS. I use an .ui for the Interface. I have just a QPushButton and QProgressBar
The process should be:
1- Initially bar at 0
2- Push button. Start process and bar starts moving
3- When process is over, the bar is complete.
If you run the code (with an interface having these two QWidgets) you will see that the process takes maybe one second, BUT during this second, the Progress BAR always stays at 0 (it should move from one side to the other in this waiting mode). When it finishes, the bar shows 1 and the "Done" message.
What am I doing wrong? Thanks for the help!

How to pause a thread (python)

The context:
I'm building a Graphical Interface with Qt creator and the "behaviour" file in python. A test version of my GUI is:
The expected behaviour:
I am running 2 different threads which are referred to the same function with different input arguments. With the SELECTOR button I can assign the value of 1 or 2 to a variable (and display it)
The button Start thread enables the correct thread to start (the first time).
The loop should be turned off by the stop button by modifying the global running variable.
This is my code
# -*- coding: utf-8 -*-
from PyQt4 import QtCore, QtGui, uic
import sys
import threading
import time
import Queue
running = False
first_thread = None
second_thread = None
form_class = uic.loadUiType("simple2.ui")[0]
q = Queue.Queue()
select = 0
def action(string, queue): #function called by threads
global running
while(running):
phrase = string
if queue.qsize() < 10:
queue.put(phrase)
#else:
# print queue.qsize()
class MyWindowClass(QtGui.QMainWindow, form_class):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.setupUi(self)
#buttons
self.startButton.clicked.connect(self.start_clicked)
self.stopButton.clicked.connect(self.stop_clicked)
self.selector.clicked.connect(self.sel_click)
#variables
self.first = False
self.second = False
#queue
self.timer = QtCore.QTimer(self)
self.timer.timeout.connect(self.update_phrase)
self.timer.start(1)
def start_clicked(self): #start button callback
global select
if select > 0:
global running
running = True
print "started"
if (not self.first) & (select == 1):
first_thread.start()
self.first = True
if (not self.second) & (select == 2):
second_thread.start()
self.second = True
self.startButton.setEnabled(False)
self.startButton.setText('Starting...')
def stop_clicked(self): #stop button callback
global running
running = False
print "stopped"
self.startButton.setEnabled(True)
self.startButton.setText('Start Thread')
def sel_click(self): #selector button callback
global select
if select < 2:
select = select + 1
else:
select = 1
self.thread_counter.setText(str(select))
def update_phrase(self): #looping function
global running
if (not q.empty()) & running:
self.startButton.setText('Thread on')
abc = q.get()
print abc
def closeEvent(self, event):
global running
running = False
if __name__ == "__main__":
first_thread = threading.Thread(target=action, args = ("first", q))
second_thread = threading.Thread(target=action, args = ("second", q))
app = QtGui.QApplication(sys.argv)
w = MyWindowClass(None)
w.setWindowTitle('Multiple threads test in python')
w.show()
app.exec_()
For now, each thread should simple print on terminal their arguments ("First" or "Second").
If threads are started for the first time, my code works. But I would like to switch between threads infinite times.
Since threads cannot be stopped, is there a way to "pause" them?
I cannot find a solution, I hope someone will help me also with a piece of code. Thank you in advance

You can use Lock class to do that, a simple example would be:
import threading
lock = threading.Lock()
//here it will be lock
lock.acquire() # will block if lock is already held
...
then in other side do
//this will wake up
lock.release()
you can read more here http://effbot.org/zone/thread-synchronization.htm

How to run a function/thread in a different terminal window in python?

I have a program like this:
from threading import Thread
def foo1(arg):
print("foo1 >>> Something")
input("foo1 >>> Enter Something")
...
def foo2(arg):
print("foo2 >>> Something")
input("foo2 >>> Enter Something")
...
def main():
th1 = Thread(target= foo1)
th1.start()
th2 = Thread(target= foo2)
th2.start()
This program runs both the functions(foo1 and foo2) in the same terminal window. Can I in some way run them in a different terminal window. What I don't wish is to re-run the program. The reason is that they print and take input at the same place and same time. I don't want. Any method?

What you are trying to accomplish isn't possible with just threads, when you create a new Thread it shares all the variables with other threads in your program, including sys.stdout / sys.stdin.
Normally you don't have to worry about PIPES in python programs because it takes care of it for you. print sends the text to sys.stdout and input grabs text from sys.stdin (and error messages are sent to sys.stderr)
So running one program in two terminal windows would mean you would have to have more then one input/output streams, to which there are two solutions:
run a completely separate program with subprocess.Popen like the other fellow described and figure out how to bridge information across the two which is a real pain.
or 2. create your own terminal window with something like tkinter, which is difficult from scratch but luckily IDLE has the majority of the code available in the standard library.
Here is an adapted version of PyShell from idlelib.PyShell to run a Thread instead of the interactive interpretive:
from idlelib import PyShell,EditorWindow
import threading,sys
try:
import tkinter as tk #python 3.X
except ImportError:
import Tkinter as tk #python 2
import tkMessageBox as messagebox
tk.messagebox = messagebox
class ThreadShell(PyShell.PyShell):
"""mostly copied from idlelib.PyShell module but adapted to work with threads"""
#__adapted_by__ = "Tadhg McDonald-Jensen"
def __init__(self, tk_root,target=None):
#not sure exactly what the FileList object is for but it is required by the shell
flist = PyShell.PyShellFileList(tk_root)
super(ThreadShell,self).__init__(flist)
#internal event flag for input, allows thread waiting for input to wait until a tk event handles it
self.__input_flag = threading.Event()
#target is stored and called in .run_command() which also deals with finishing the shell
self.target = target
self.thread = threading.Thread(target=self.run_command)
#tk_root.after makes the .start method call when the program starts (after 0 miliseconds)
tk_root.after(0,self.start)
def start(self):
"""starts executing the Thread"""
super(ThreadShell,self).beginexecuting()
try:
self.thread.start()
except RuntimeError:
self.executing = 0
self.canceled = 0
#self.top.quit() #this causes double deletion warnings with better Implementation of mainloop
beginexecuting = start
def run_command(self):
"""calls target from constructor with self as argument then cleans up shell"""
if self.target:
self.target(self)
self.prompt_exit()
self.executing = 0
self.canceled = 0
try:
self.text.after(1,self.close)
except RuntimeError:
pass #tkinter has issues with changing threads so often after closing one shell others will throw this error
def printf(self,*stuff,**kw):
"""works just like python 3.x print function but writes to shell's .stdout file"""
if self.executing:
## if USING_OLD_METHOD: #Pretty sure this would do exact same thing
## kw.setdefault("file",self.stdout)
## print(*stuff,**kw), self.resetoutput()
## return
sep = kw.get("sep"," ")
end = kw.get("end","\n")
text = sep.join(stuff) + end
self.stdout.write(text)
self.resetoutput()
def input(self,prompt="",timeout=None):
"""python 2 equivelent to raw_input or py 3+ input
Prompts user for input and freezes thread until input is given
Will return "" if .executing is False or it timed out from optional timeout argument"""
if self.executing or self.closing:
if prompt:
self.stdout.write(prompt)
self.__in_buffer = ""
self.__input_flag.clear()
self.reading=True
self.__input_flag.wait(timeout)
#input is inserted into .__in_buffer by other events
#then set __input_flag so that it can be delivered to thread
self.reading = False
return self.__in_buffer.strip("\n")
else:
raise RuntimeError("cannot take input after finished")
def prompt_exit(self):
"""writes press enter to quit" to the console colour then waits for input"""
self.executing = False
self.closing = True
self.console.write("\n press enter to quit")
self.input()
def join_thread(self,timeout=None):
"""sets .executing label to False then waits to join thead,
returns True if thread finished or False if timeout activated"""
self.executing = False
self.closing = True
if self.thread:
self.thread.join(timeout)
return not self.thread.is_alive()
def _close(self):
"Extend EditorWindow._close(), joins thread to close it"
# Restore std streams
sys.stdout = self.save_stdout
sys.stderr = self.save_stderr
sys.stdin = self.save_stdin
# Break cycles
self.interp = None
self.console = None
self.flist.pyshell = None
self.history = None
EditorWindow.EditorWindow._close(self)
self.join_thread()
def stop_readline(self):
self.__in_buffer = ""
self.__input_flag.set()
def update_in(self):
"""updates input from user, I think some of the labels are probably unnecessary but it is easier to leave it alone"""
line = self.text.get("iomark", "end-1c")
if len(line) == 0: # may be EOF if we quit our mainloop with Ctrl-C
line = "\n"
self.resetoutput()
if self.canceled:
self.canceled = 0
if self.endoffile:
self.endoffile = 0
line = ""
self.__in_buffer = line
self.__input_flag.set()
def cancel_callback(self, event=None):
try:
if self.text.compare("sel.first", "!=", "sel.last"):
return # Active selection -- always use default binding
except:
pass
if not (self.executing or self.reading):
return "break"
self.endoffile = 0
self.canceled = 1
if self.reading:
self.update_in()
return "break"
def eof_callback(self, event):
if self.executing and not self.reading:
return # Let the default binding (delete next char) take over
if not (self.text.compare("iomark", "==", "insert") and
self.text.compare("insert", "==", "end-1c")):
return # Let the default binding (delete next char) take over
if not self.executing:
self.resetoutput()
self.close()
else:
self.canceled = 0
self.endoffile = 1
self.update_in()
return "break"
def enter_callback(self, event):
"""called when the enter/return key is pressed,
only the recursive self.top.mainloop() / self.top.quit() had to be changed for support"""
# it is very long to copy/paste for the one line change, so I override the method temporarily
save = self.top.quit
self.top.quit = self.update_in
super(ThreadShell,self).enter_callback(event)
self.top.quit = save
#stupid module depends on this being set from the main function, so it needs to be done manually
PyShell.use_subprocess = True
#this defines the root tkinter window and sets it up
root = tk.Tk()
EditorWindow.fixwordbreaks(root)
root.withdraw()
#I need this to work on my mac, not sure if there are other OS specific stuff that should be included
try:
from idlelib import macosxSupport
macosxSupport.setupApp(root, None)
except (ImportError,AttributeError):
pass
##!!!!!!!!!!!!!!!!!!!! And This Is The Part You Need To Worry About !!!!!!!!!!!!!!!!!!!!##
switch = threading.Event()
switch.clear()
def foo(shell):
global x
x = shell.input("enter a message: ")
switch.set()
shell.printf("message sent")
def foo2(shell):
shell.printf("waiting for message...")
while shell.executing and not switch.is_set():
switch.wait(2) # by using shell.executing in the loop it will occasionally check
# if the program should quit because the window was closed
if shell.executing:
shell.printf("message recieved: ",x)
shell1 = ThreadShell(root,foo)
shell2 = ThreadShell(root,foo2)
first_time = True
while shell1.executing or shell2.executing or first_time:
first_time = False
root.mainloop()
root.destroy()

#!/usr/bin/env python
"""Show messages in two new console windows simultaneously."""
import sys
import platform
from subprocess import Popen
messages = 'This is Console1', 'This is Console2'
def randomFunction():
return "import sys; print(sys.argv[1]); input('Press Enter..')"
# define a command that starts new terminal
if platform.system() == "Windows":
new_window_command = "cmd.exe /c start".split()
else: #XXX this can be made more portable
new_window_command = "x-terminal-emulator -e".split()
# open new consoles, display messages
echo = [sys.executable, "-c",randomFunction()
]
processes = [Popen(new_window_command + echo + [msg]) for msg in messages]
# wait for the windows to be closed
for proc in processes:
proc.wait()

Find working solution for your problem, I haven't used thread, but can be done. And this solution is motivated from solution provided by "Miodrag Novakovic"
You have to change few paths as per your env. Below code is tested on windows
test_code.py -
import sys
import platform
from subprocess import Popen
messages = 'This is Console1', 'This is Console2'
def foo1():
print "In foo1"
i = input("Enter Something - ")
print i
input("Enter to exit")
def foo2():
print "In foo2"
i = input("Enter Something - ")
print i
input("Enter to exit")
def run_foo1():
print("foo1 >>> Something")
return "import sys; sys.path.append('path_to_your_program_folder'); from test_code import foo1; foo1()"
def run_foo2():
print("foo2 >>> Something")
return "import sys; sys.path.append('path_to_your_program_folder'); from test_code import foo2; foo2()"
# define a command that starts new terminal
if platform.system() == "Windows":
new_window_command = "cmd.exe /c start".split()
else: #XXX this can be made more portable
new_window_command = "x-terminal-emulator -e".split()
if __name__ == '__main__':
# open new consoles, display messages
echos = [[sys.executable, "-c",run_foo1()],
[sys.executable, "-c",run_foo2()]
]
processes = [Popen(new_window_command + echo) for echo in echos]
# wait for the windows to be closed
for proc in processes:
proc.wait()

Python PyQT: How to call a GUI function from a worker thread?

I have a pyqt gui and calling a long process (ffmpeg) which I put on a separate thread to not block the gui. I then want to update a progress bar when one command of a longer list of commands finishes. The problem is, that I can't call a function in the gui thread out of the worker thread. So I let run a ticker in the worker thread, but when I update the progress bar with a while loop and reading the ticker value, the gui gets blocked again. How can I solve this. I used currently python threading and not Qthread.
Thx for any help!
import threading, pexpect
self.cmd_list = ['ffmpeg -i file outfile','and so on']
self.stop_proc = False
self.executeCMD()
def spawn_ffmpeg_cmd(self):
for cmd in self.cmd_list:
if self.stop_proc == False:
thread = pexpect.spawn(cmd)
print "\nstarted: %s" % cmd
cpl = thread.compile_pattern_list([pexpect.EOF,"frame= *\d+ fps=*\d+",'(.+)'])
while True:
i = thread.expect_list(cpl, timeout=None)
if i == 0: # EOF
print "the sub process exited"
self.pgticker += 1
break
elif i == 1:
frame_number_fps = thread.match.group(0)
print frame_number_fps
thread.close
elif i == 2:
pass
self.startButton.setEnabled(True)
def executeCMD(self):
self.startButton.setEnabled(False)
self.pgticker = 0
threading.Thread(target=self.spawn_ffmpeg_cmd, name="_proc").start()
def stopprocess(self):
self.stop_proc = True
self.cmd_list = []
os.system('pkill ffmpeg')
self.pgticker = len(self.cmd_list)
self.startButton.setEnabled(True)
def updateProgress(self):
pgfactor = 100 / len(self.cmd_list)
progress = 0.0
progress = pgfactor*int(self.pgticker)
self.progressBar.setProperty("value", progress)

In short: Move to QThread and use Qt's signals and slots, they are the preferred way to communicate between threads.
This answer provides some examples how this could look like:
https://stackoverflow.com/a/6789205/2319400
In your case, using the "SomeObject" version from the above could look like this:
class Worker(QtCore.QObject):
madeProgress = QtCore.pyqtSignal([int])
finished = QtCore.pyqtSignal()
def __init__(self, cmdlist):
self.cmdlist = cmdlist
def run(self):
for icmd, cmd in enumerate(self.cmdlist):
# execute your work
# processCommand(cmd)
# signal that we've made progress
self.madeProgress.emit(icmd)
# emit the finished signal - we're done
self.finished.emit()
Then move this worker to a QThread instance you create.
Following the pattern from the linked answer, you can then connect the madeProgress signal
to the setValue slot of a progressbar:
workerThread = QThread()
workerObject = Worker(cmdlist)
workerObject.moveToThread(workerThread)
workerThread.started.connect(workerObject.run)
workerObject.finished.connect(workerThread.quit)
# create a progressbar with min/max according to
# the length of your cmdlist
progressBar = QProgressBar()
progressBar.setRange(0, len(cmdlist))
# connect the worker's progress signal with the progressbar
workerObject.madeProgress.connect(progressBar.setValue)
# start the thread (starting your worker at the same time)
workerThread.start()

python add time to a countdown already running

I want to have an app where if I click a button I add X amount of time to my running countdown timer.
I'm guessing I have to use threads for this but am not sure how to implement it..
Here is the code I have so far:
def countdown_controller(add_time):
end_it = False
def timer(time_this):
start = time.time()
lastprinted = 0
finish = start + time_this
while time.time() < finish:
now = int(time.time())
if now != lastprinted:
time_left = int(finish - now)
print time_left
lastprinted = now
if end_it == True:
now = finish
time.sleep(0.1)
# Check if the counter is running otherwise just add time.
try:
time_left
except NameError:
timer(add_time)
else:
if time_left == 0:
timer(add_time)
else:
add_this = time_left
end_it = True
while now != finish:
time.sleep(0.1)
timer(add_time + add_this)
Obviously this will not work, because every time I call countdown_controller(15) fx, it will start counting down for 15 seconds and if I click my button nothing happens until the timer is ended.
Help would be greatly appreciated.

I would say that there is a flaw in the design of the code, because your screen output blocks down the entire program doing nothing (time.sleep(0.1)).
Typically what you want to to do in these cases is having a main loop in your program that cycles through the various operations that make your program run. This guarantees a sensible distribution of system resources between the various tasks.
In your specific case, what you would like to have in your main loop is:
Check user input (has extra time been added?)
Update output of the countdown
Example implementation:
import time
import curses
# The timer class
class Timer():
def __init__(self):
self.target = time.time() + 5
def add_five(self):
self.target += 5
def get_left(self):
return int(self.target-time.time())
# The main program
t = Timer()
stdscr = curses.initscr()
stdscr.nodelay(True)
curses.noecho()
# This is the main loop done in curses, but you can implement it with
# a GUI toolkit or any other method you wish.
while True:
left = t.get_left()
if left <= 0:
break
stdscr.addstr(0, 0, 'Seconds left: %s ' % str(left).zfill(3))
c = stdscr.getch()
if c == ord('x') :
t.add_five()
# Final operations start here
stdscr.keypad(0)
curses.echo()
curses.endwin()
print '\nTime is up!\n'
The above program will increase the counter of 5 seconds if you press the x key (lowercase). Most of the code is boilerplate to use the curses module, but of course if you use PyGTK, PySide or any other graphical toolkit, it will be different.
EDIT: As a rule of thumb, in python you want to avoid threading as much as you can, both because it often (but not always) slows down programs (see "Global Interpreter Lock") and because it makes software harder to debug/maintain.
HTH!

I would probably have a Timer object with a finish attribute that I could simply add an int to. Have that timer running in another thread that you can then query for the current time remaining from your GUI.
class Timer(object):
def __init__(self, length):
self.finish = time.time() + length
def get_time(self):
return time.time() >= self.finish