I've seen a lot of stuff about running code in subprocesses or threads, and using the multiprocessing and threading modules it's been really easy. However, doing this in a GUI adds an extra layer of complication.
From what I understand, the GUI classes don't like it if you try and manipulate them from multiple threads (or processes). The workaround is to send the data from whatever thread you created it in to the thread responsible for the graphics and then render it there.
Unfortunately, for the scenario I have in mind this is not an option: The gui I've created allows users to write their own plotting code which is then executed. This means I have no control over how they plot exactly, nor do I want to have it. (Update: these plots are displayed in separate windows and don't need to be embedded anywhere in the main GUI. What I want is for them to exist separated from the main GUI, without sharing any of the underlying stack of graphics libraries.)
So what I'm wondering now is
Is there some clean(ish) way of executing a string of python code in a whole new interpreter instance with its own ties to the windowing system?
In response to the comments:
The current application is set up as follows: A simple python script loads a wxPython gui (a wx.App). Using this gui users can set up a simulation, part of which involves creating a script in plain python that runs the simulation and post-processes the results (which usually involves making plots and displaying them). At the moment I'm doing this by simply calling exec() on the script code. This works fine, but the gui freezes while the simulation is running. I've experimented with running the embedded script in a subprocess, which also works fine, right up until you try to display the created graphs (usually using matplotlib's show()). At this point some library deep down in the stack of wxPython, wx, gtk etc starts complaining because you cannot manipulate it from multiple threads.
The set-up I would like to have is roughly the same, but instead of the embedded script sharing a GUI with the main application, I would like it to show graphics in an environment of its own.
And just to clarify:
This is not a question about "how do I do multithreading/multiprocessing" or even "how do I do multithreading/multiprocessing within a single wxpython gui". The question is how I can start a script from a gui that loads an entirely new gui. How do I get the window manager to see this script as an entirely separate application?
The easiest way would be to generate it in a temporary folder somewhere and then make a non-blocking call to the python interpreter, but this makes communication more difficult and it'd be quite hard to know when I could delete the temp files again. I was hoping there was a cleaner, dynamical way of doing this.
Can you simply use subprocess to run 'python.exe' and pipe the script in?
Alternatively, the multiprocessing package should suffice if you want to move some (pickle-able) data over to the new process in which you run the script. Just create a function/callable that runs the script, and create a Process object with the callable as target. That way, you should be able to pass some data over, without having GUI issues.
Capturing text with either is easy, subprocess allows that and no more. With multiprocess, you can pass python objects back and forth more easily.
On Windows, you can create window with a parent window from another process, and draw to that.
See the hWndParent argument to CreateWindowEx.
If wxWindows supports getting/setting that explicitly, then you should be good to go.
Depending on your platform, something similar might be possible in any windows system.
So, just giving your users the ability to find the handle of your apps window should give them the option to plot away at views embedded in your app, while running in their own processes.
I don't no much about wx, I work with jython(python implemented in java and you can use java) and swing. Swing has its own worker thread, and if you do gui updates you wrap your code into a runnable and invoke it with swing.invokelater.
You could see if wx has something like that, if you however are only allowed to manipulate the gui from the thread in which you created it try something similar. create a proxy object for your gui, which forwards all your calls to your thread which forwards them to the gui.
But proxying like this gets messy. how about you let them define classes, with an 'updateGui' function, that they should hand back to you over a queue and that you will execute in your gui thread.
In wxPython land when you use threads, you have to use its thread-safe methods to communicate with the GUI: wx.CallAfter, wx.CallLater or wx.PostEvent. In your case, I would run any long running code in a separate thread/process and when it's done its processing, send the result to the GUI. The GUI can instantiate a new frame and use matplotlib or PyPlot to show the plot, depending on which way you want to go. I've heard you can draw the plot using FloatCanvas too.
Anyway, if you instantiate the new frame correctly, then you can instantiate N frames and show them and be fine. See the wxPython wiki for a few examples of using Threads with wx: http://wiki.wxpython.org/LongRunningTasks
Related
I have a main process that does some stuff (e.g. analyzing data) and it runs alone just fine. I also want to make a simple GUI that displays the result of the main task using PyQT5. The idea is that the GUI should not interfere in the main process, that is, if I remove the GUI it shouldn't cause any issue to the main process.
The code of the main process is quite simple:
if __name__ == '__main__':
# initialize the object that performs the main task
tasker = Task()
# the graphical interface to visualize the result of tasker
gui = GUI(task) # GUI is a separate class that keeps a reference to tasker
# read the input data and do stuff on each new data instance
for f in listdir(inrepo):
data = read_new_data(f) # an utility function that reads new data from file
result = tasker.process(data) # tasker processes the new data and return some results
gui.update(data, result) # pass the data and result in the GUI to update it
The code of the GUI class is quite long so I only paste a few lines here, but the lines I skip are just to create widgets, nothing fancy (I didn't connect any event yet)
class GUI(QApplication):
def __init__(self, tasker):
"""Initialize the application"""
super().__init__([])
self.tasker = tasker
# define the main window
self.window = QWidget()
self.window.setWindowTitle('GUI')
... # layout and components etc.
# show stuff
self.window.show()
self.exec()
So I want the GUI to be completely independent from my main process. For example, if I don't need the GUI anymore, I can just comment out the 2 lines gui = GUI(task) and gui.update(data, result).
However, the problem is that starting the GUI blocks the entire process (I assume it's because of self.exec() in GUI.__init__, so my main process cannot proceed to loop over the data. Could you please show me how to make PyQT non-blocking? Is it even feasible?
Some options I have considered:
Threading: it seems more complicated than necessary for my use case and it may make referencing to the task instance difficult from a thread. All new updates to task should be reflected in the GUI. If I'm not mistaken, PyQT's application already runs on a thread. So multi-level threading may be troublesome.
Run the GUI from another Python process, communicated via shared folders: may incur high latency. Any new data and result from task should be immediately reflected in the GUI. Writing to file then reading from file then updating the GUI will cause some delay.
Perform the task in GUI: I could use some timeout event to read new data periodically and run task on them, but then everything depends heavily on the GUI, and I can't just comment it out if I don't need the GUI anymore.
Any suggestion is very much appreciated! Thank you very much!
Switching your program between GUI mode/console mode is often not as simple as commenting out some lines. PyQt in particular does not allow you to run a GUI loop from anything other than a main thread. Not all hope is lost, though - this simply means that you should decide as early as possible whether your program is going to run as a console application or as a GUI.
Instead of relying in commenting out code, you can create a "switch" of sorts inside your code that tells your code how to execute. One way to do this is to check the command-line arguments when you execute the code, e.g.:
import sys
if "--headless" in sys.argv[1:]: # checking the command-line arguments
run_code_without_gui()
else:
run_code_with_gui()
This way, executing your code as python mycode.py --headless will execute it without the GUI (through the run_code_without_gui function), while executing python mycode.py will run it as a GUI (through the run_code_with_gui function). (Although if you're actually going to parse command-line arguments, I recommend using the argparse library).
You can even keep the analysis code completely decoupled from the GUI code, so that you simply run a function from, say, analysis.py when executing without the GUI, and have the GUI call that exact same function when e.g. the user clicks on the "Analyze" button.
Another thing to note is that if your analysis code takes long to execute, it may inadvertently block the GUI. In this case, you should run the analysis code in a separate "worker" thread that spawns upon clicking the "Analyze" button, keeping the GUI responsive while it executes. This might be the way to go if you want the analysis to keep running alongside the GUI indefinitely - create the worker thread for the analysis at the same time that you display the GUI to the user.
My engine is written in GO and I already use SQLITE because I generate some plots that I want to display in a GUI. So far, I think that GO does not have a mature GUI package, so I am building my GUI in Python, with Tkinter that I am familiar with.
When I run my engine in GO with a certain flag, it calls the Python script for the GUI and it generates the content on the background continuously. There are certain modes that the GO engine runs and normally I have to change them by going back to the terminal that runs GO (different plots etc).
What I want, is to create action buttons in my Python GUI that they will change those flags/variables values and the GO script will generate different plots for example, without me going back to the terminal that runs the GO engine and manually do it.
So far, the only implementation from my side, was to create another SQLITE database and save my settings there. Then, when I press an action button at my GUI, it changes the value at my Settings Database and GO reads this value from there all the time, instead of waiting me to input it.
Is there any better way of doing it, passing a variable value(flag etc) from one programming language to the other(Python-GO) without using a database?
The variables that I am changing from the GUI are strings in general and I am asking this question because I think that it is a waste of resources to do it the way I do it.
I've got the following problem:
I've written a script which is running up to four processes at the same time. Works like a charm when using it via command line.
Then I made the decision to write a GUI with wxPython and I was quickly figuring out that the GUI and the script need to run in different processes so both stay usable while the other is doing something. (i.e. being able to press a stop button while the script is running) This is also working perfectly.
Now the problem:
I just cannot communicate with the GUI while the script is running or at least I have no idea how. I'm trying to write output in a text window by passing "self" (the gui) to the script and in the script I try to do things like "self.outputWindow.WriteText('the script is doing bla 1 of bla 10')"
I even figured out why this won't work: self (the gui object) is not pickable and that's mandatory for multiprocessing, but I don't know how else I should do it.
You can use my tutorial on wxPython and threads, although I'm not sure if Python spreads those threads evenly to all the cores. I suspect it doesn't.
Fortunately, there are examples of using the multiprocessing module with wxPython. See the following links:
http://wiki.wxpython.org/MultiProcessing
wxPython non-blocking GUI threading AND multiprocessing?
wxpython GUI and multiprocessing - how to send data back from the long running process
http://www.blog.pythonlibrary.org/2012/08/03/python-concurrency-porting-from-a-queue-to-multiprocessing/
I hope those are helpful!
I have looked at similar questions that may answer my question but I am still very unclear on how to go about the following:
I can create programs to run in the Python Shell in Idle and I can also set up windows with widgets in Tkinter, but whatever I create in Tkinter is pointless because I cannot figure out how to take my Python Shell code and "wrap" it in the Tkinter GUI.
I have assumed that it cannot be done, and that entirely new code must be written to assist the language that is specific to Tkinter. I am very confused on how to create a well-rounded program without being left with just a GUI "skeleton" with random buttons, labels, entries, etc. and a Python program that is very unappealing and can only run in the ugly little Shell.
What you create with Tkinter is not pointless. It sounds to me like you're trying to compile a stand-alone program in Python, using the Tkinter library to provide the GUI. Once you have a script working, you can use a program to compile into a standalone program. Look into using py2app on a mac, or py2exe on Windows. Google them and see if that's what you're looking for.
Porting an application from command line to GUI might require some rework (depending on degree of interactivity you want to achieve).
Basically, in a GUI application, you build a few widgets (buttons...) at startup, and then perfom all your actions "on reaction" of user input. You typically do this by binding callbacks onto your widgets (button, input field), and then enter a mainloop (or eventloop).
You might read this chapter about events and binding.
If your application is mainly computing oriented, providing a gui with a "launch" button, and an output field is straightforward. If you perform some command line input, you can switch to widget input at low cost. More interactive apps will require to be architectured toward interaction capabilities.
Continued from How to use wxSpellCheckerDialog in Django?
I have added spell checking to Django application using pyenchant.
It works correctly when first run. But when I call it again (or after several runs) it gives following error.
PyAssertionError at /quiz/submit/
C++ assertion "wxThread::IsMain()"
failed at ....\src\msw\evtloop.cpp(244) in wxEventLoop::Dispatch():
only the main thread can process Windows messages
How to fix this?
You don't need wxPython to use pyEnchant. And you certainly shouldn't be using wx stuff with django. wxPython is for desktop GUIs, while django is a web app framework. As "uhz" pointed out, you can't call wxPython methods outside the main thread that wxPython runs in unless you use its threadsafe methods, such as wx.CallAfter. I don't know why you'd call wxPython from Django though.
It seems you are trying to use wx controls from inside Django code, is that correct? If so you are doing a very weird thing :)
When you write a GUI application with wxPython there is one main thread which can process Window messages - the main thread is defined as the one where wx.App was created. You are trying to do a UI thing from a non-UI thread. So probably at first run everything works (everything is performed in the GUI thread) but on second attempt a different python thread (spawned by django?) is performing some illegal GUI actions. You could try using wx.CallAfter which would execute a function from the argument in GUI thread but this is non-blocking. Also I've found something you might consider: wxAnyThread wxAnyThread. But I didn't use it and I don't know if it applies in your case.