Develop Reference

Non-blocking PyQT as an additional GUI that visualizes the result from a main process?

I have a main process that does some stuff (e.g. analyzing data) and it runs alone just fine. I also want to make a simple GUI that displays the result of the main task using PyQT5. The idea is that the GUI should not interfere in the main process, that is, if I remove the GUI it shouldn't cause any issue to the main process.
The code of the main process is quite simple:
if __name__ == '__main__':
# initialize the object that performs the main task
tasker = Task()
# the graphical interface to visualize the result of tasker
gui = GUI(task) # GUI is a separate class that keeps a reference to tasker
# read the input data and do stuff on each new data instance
for f in listdir(inrepo):
data = read_new_data(f) # an utility function that reads new data from file
result = tasker.process(data) # tasker processes the new data and return some results
gui.update(data, result) # pass the data and result in the GUI to update it
The code of the GUI class is quite long so I only paste a few lines here, but the lines I skip are just to create widgets, nothing fancy (I didn't connect any event yet)
class GUI(QApplication):
def __init__(self, tasker):
"""Initialize the application"""
super().__init__([])
self.tasker = tasker
# define the main window
self.window = QWidget()
self.window.setWindowTitle('GUI')
... # layout and components etc.
# show stuff
self.window.show()
self.exec()
So I want the GUI to be completely independent from my main process. For example, if I don't need the GUI anymore, I can just comment out the 2 lines gui = GUI(task) and gui.update(data, result).
However, the problem is that starting the GUI blocks the entire process (I assume it's because of self.exec() in GUI.__init__, so my main process cannot proceed to loop over the data. Could you please show me how to make PyQT non-blocking? Is it even feasible?
Some options I have considered:
Threading: it seems more complicated than necessary for my use case and it may make referencing to the task instance difficult from a thread. All new updates to task should be reflected in the GUI. If I'm not mistaken, PyQT's application already runs on a thread. So multi-level threading may be troublesome.
Run the GUI from another Python process, communicated via shared folders: may incur high latency. Any new data and result from task should be immediately reflected in the GUI. Writing to file then reading from file then updating the GUI will cause some delay.
Perform the task in GUI: I could use some timeout event to read new data periodically and run task on them, but then everything depends heavily on the GUI, and I can't just comment it out if I don't need the GUI anymore.
Any suggestion is very much appreciated! Thank you very much!

Switching your program between GUI mode/console mode is often not as simple as commenting out some lines. PyQt in particular does not allow you to run a GUI loop from anything other than a main thread. Not all hope is lost, though - this simply means that you should decide as early as possible whether your program is going to run as a console application or as a GUI.
Instead of relying in commenting out code, you can create a "switch" of sorts inside your code that tells your code how to execute. One way to do this is to check the command-line arguments when you execute the code, e.g.:
import sys
if "--headless" in sys.argv[1:]: # checking the command-line arguments
run_code_without_gui()
else:
run_code_with_gui()
This way, executing your code as python mycode.py --headless will execute it without the GUI (through the run_code_without_gui function), while executing python mycode.py will run it as a GUI (through the run_code_with_gui function). (Although if you're actually going to parse command-line arguments, I recommend using the argparse library).
You can even keep the analysis code completely decoupled from the GUI code, so that you simply run a function from, say, analysis.py when executing without the GUI, and have the GUI call that exact same function when e.g. the user clicks on the "Analyze" button.
Another thing to note is that if your analysis code takes long to execute, it may inadvertently block the GUI. In this case, you should run the analysis code in a separate "worker" thread that spawns upon clicking the "Analyze" button, keeping the GUI responsive while it executes. This might be the way to go if you want the analysis to keep running alongside the GUI indefinitely - create the worker thread for the analysis at the same time that you display the GUI to the user.

How do I stop Pyzo from calling mainloop?

I used to use the Interactive Editor for Python, and then I "upgraded" to Pyzo (since IEP was merged into Pyzo). One of my programs uses tkinter to create a GUI. The code used to work just fine: I would run the file as a script, and then, in the interpreter, I would call main, which would launch the application.
The code skeleton looks like this.
def main():
class Application(tk.Frame):
def __init__(self, master=None):
# a bunch of stuff
# several more methods here
front=Application()
front.mainloop()
# then I can either call main in the interpreter, or I can add this:
# the name==main part is to handle some multiprocessing that occurs within the application class
if __name__=="__main__":
main()
This worked like a charm in IEP. However, in Pyzo, main() never launches, or, rather, it launches, but the gui never appears and it doesn't let me do anything. Instead, I get this message: Note: The GUI event loop is already running in the pyzo kernel. Be aware that the function to enter the main loop does not block.
This message occurs in Pyzo when I am using the CPython 3 or PyPy interpreter, but not when I'm using Anaconda 3 (I actually need to use PyPy because the work I'm doing is computationally expensive).
The other alternative is to not use Pyzo, but that's no fun.

I discovered the answer a while ago but I didn't get back to posting the answer until now. Essentially, there is a setting in Pyzo itself which attempts to detect GUIs. Switching that setting from auto to none fixed the problem.

Can I safely use a thread to call a script from a Tkinter application with subprocess.call?

So I have a Tkinter application that I use at work. I wrote another Tkinter application that I wanted to call from the main Tkinter application. I know that Tkinter isn't 'thread safe', but I'm not 100% sure what that means. Does it mean that it can 'work', but it isn't guaranteed to work as written? Or should it just not work at all?
The reason I ask is because when I run the code below(self.thread_easy_imaging() is triggered by a filemenu option), it works just fine. So far I haven't encountered any issues and both GUIs are operational so far as I can tell. Is this because I'm using subprocess.call to call the script? I'm having a hard time wrapping my head around this because I know you can't use thread in a Tkinter app to do something like run a function while still being able to use buttons in the GUI, but for some reason when I use a thread to call the script with subprocess.call it does it just fine. When I wrote it, I assumed it wouldn't work, but for some odd reason it does. Here is an example of the code I'm using:
def thread_easy_imaging(self):
thread.start_new_thread(self.start_easy_imaging, ('EASY-IMAGING-1', 0))
def start_easy_imaging(self, thread_name, delay):
time.sleep(delay) #have to have args for some reason? I just did a delay of 0 seconds so I could use the tuple.
subprocess.call(['c:/python27/python.exe', 'EasyImaging.py'])
self.thread_easy_imaging()
I have only used this on a windows machine, but I'm guessing that it should work on any OS.
Edit: The scripts don't have to interact at all, I just want to add my other GUI application into the filemenu so I can call it as a separate application. If needed, I'll just use a Toplevel widget to recreate the application I want to call because it won't muck up the main thread at all.
Thanks in advance!

Executing a python script in a subprocess - with graphics

I've seen a lot of stuff about running code in subprocesses or threads, and using the multiprocessing and threading modules it's been really easy. However, doing this in a GUI adds an extra layer of complication.
From what I understand, the GUI classes don't like it if you try and manipulate them from multiple threads (or processes). The workaround is to send the data from whatever thread you created it in to the thread responsible for the graphics and then render it there.
Unfortunately, for the scenario I have in mind this is not an option: The gui I've created allows users to write their own plotting code which is then executed. This means I have no control over how they plot exactly, nor do I want to have it. (Update: these plots are displayed in separate windows and don't need to be embedded anywhere in the main GUI. What I want is for them to exist separated from the main GUI, without sharing any of the underlying stack of graphics libraries.)
So what I'm wondering now is
Is there some clean(ish) way of executing a string of python code in a whole new interpreter instance with its own ties to the windowing system?
In response to the comments:
The current application is set up as follows: A simple python script loads a wxPython gui (a wx.App). Using this gui users can set up a simulation, part of which involves creating a script in plain python that runs the simulation and post-processes the results (which usually involves making plots and displaying them). At the moment I'm doing this by simply calling exec() on the script code. This works fine, but the gui freezes while the simulation is running. I've experimented with running the embedded script in a subprocess, which also works fine, right up until you try to display the created graphs (usually using matplotlib's show()). At this point some library deep down in the stack of wxPython, wx, gtk etc starts complaining because you cannot manipulate it from multiple threads.
The set-up I would like to have is roughly the same, but instead of the embedded script sharing a GUI with the main application, I would like it to show graphics in an environment of its own.
And just to clarify:
This is not a question about "how do I do multithreading/multiprocessing" or even "how do I do multithreading/multiprocessing within a single wxpython gui". The question is how I can start a script from a gui that loads an entirely new gui. How do I get the window manager to see this script as an entirely separate application?
The easiest way would be to generate it in a temporary folder somewhere and then make a non-blocking call to the python interpreter, but this makes communication more difficult and it'd be quite hard to know when I could delete the temp files again. I was hoping there was a cleaner, dynamical way of doing this.

Can you simply use subprocess to run 'python.exe' and pipe the script in?
Alternatively, the multiprocessing package should suffice if you want to move some (pickle-able) data over to the new process in which you run the script. Just create a function/callable that runs the script, and create a Process object with the callable as target. That way, you should be able to pass some data over, without having GUI issues.
Capturing text with either is easy, subprocess allows that and no more. With multiprocess, you can pass python objects back and forth more easily.

On Windows, you can create window with a parent window from another process, and draw to that.
See the hWndParent argument to CreateWindowEx.
If wxWindows supports getting/setting that explicitly, then you should be good to go.
Depending on your platform, something similar might be possible in any windows system.
So, just giving your users the ability to find the handle of your apps window should give them the option to plot away at views embedded in your app, while running in their own processes.

I don't no much about wx, I work with jython(python implemented in java and you can use java) and swing. Swing has its own worker thread, and if you do gui updates you wrap your code into a runnable and invoke it with swing.invokelater.
You could see if wx has something like that, if you however are only allowed to manipulate the gui from the thread in which you created it try something similar. create a proxy object for your gui, which forwards all your calls to your thread which forwards them to the gui.
But proxying like this gets messy. how about you let them define classes, with an 'updateGui' function, that they should hand back to you over a queue and that you will execute in your gui thread.

In wxPython land when you use threads, you have to use its thread-safe methods to communicate with the GUI: wx.CallAfter, wx.CallLater or wx.PostEvent. In your case, I would run any long running code in a separate thread/process and when it's done its processing, send the result to the GUI. The GUI can instantiate a new frame and use matplotlib or PyPlot to show the plot, depending on which way you want to go. I've heard you can draw the plot using FloatCanvas too.
Anyway, if you instantiate the new frame correctly, then you can instantiate N frames and show them and be fine. See the wxPython wiki for a few examples of using Threads with wx: http://wiki.wxpython.org/LongRunningTasks

PyQt restarts shell

If I'm trying to create a window or prompt a file dialog in the IDLE shell, nothing opens and the shell restarts. Is this a bug of some kind? I can't find anything about it. I'm new to PyQt (and Python in general) but had been able to get tutorials to work correctly. The last day or so, if I open IDLE and import PyQt4, QtGui, etc and then run something simple like QFileDialog.getOpenFileName, the shell just restarts. Any ideas?

You need to have a QApplication before you can use anything else from PyQt. Try rereading some of the tutorials you followed, or do a few more. This one for example.
In the first code sample of the above tutorial, pay special attention to these lines (I've included the comments from the tutorial for convenience):
app = QtGui.QApplication(sys.argv)
Every PyQt4 application must create an application object. The
application object is located in the QtGui module. The sys.argv
parameter is a list of arguments from the command line. Python scripts
can be run from the shell. It is a way, how we can control the startup
of our scripts.
and
sys.exit(app.exec_())
Finally, we enter the mainloop of the application. The event handling
starts from this point. The mainloop receives events from the window
system and dispatches them to the application widgets. The mainloop
ends, if we call the exit() method or the main widget is destroyed.
The sys.exit() method ensures a clean exit. The environment will be
informed, how the application ended.
The exec_() method has an underscore. It is because the exec is a
Python keyword. And thus, exec_() was used instead.
It appears you might have forgotten about these. Or maybe you haven't realized that this means that you normally can't use PyQt with a running event loop in the interactive shell. However, there is a trick for that, see here.