Develop Reference

GUI extremly slow when ethernet_ip connection is established

I have a GUI (tkinter) where i monitor data from a few sensors. The monitoring is realized as a realtime plot, which gets redrawn every 100 ms. The data from sensors is acquired via ethernet/ip connection. Therefore i use the get_attribute_single function from this package (https://github.com/rossmann-engineering/eeip.py). First i register a session with my network-communication unit (Keyence NU-EP1). Afterwards i can use the get_attribute-single to get the single data from each sensor. The script works perfect as expected, but the GUI is interactive only if i don't establish the ethernet_ip connection. If i use fake values (random.randint()) the gui and plot are working fine with an interactive GUI. If i use the ethernet/ip connection to acquire and plot real data, the GUI is getting very very slow. After clicking a button, it's color changes. But the color change is performed about a few seconds later, so really very slow.
Any ideas how to solve this problem? Thanks in advance.

Unless I'm missing something (I'm not familiar with tkinter) it sounds like you are doing everything in a single thread. That would create the described behavior since the GUI has to wait for the data to be fetched before it can update.
To solve this kind of issue you should look into concurrent programming, for example concurrent.futures.
Create a second thread that fetches the data and let the main thread handle the GUI.

How to handle blocking calls / while 1-s, inside a tkinter's mainloop? ZMQ / Tkinter

I am using ZeroMQ, which is a messaging library (presumably async I/O), if you don't know what it is you can think of it as similar to socket library in python, the sockets used for messaging are usually run within an infinite while loop with a small sleep for keep everything cool.
I have the code written in a separate file and I have a GUI based on the working of that code separate, I want to integrate the two codes.
But the issue I come across is that I can not possibly put a while True, or a blocking socket.recv() inside tkinter's .mainloop().
I want to receive on on a socket, which is blocking - BUT I can manage that part of the issue, zmq sockets can either be polled on (check periodically to see if we have any pending messages to process) or equivalently you can use zmq.DONTWAIT which does the same thing.
The issue remaining however is that I need a while True, so that the socket is constantly polled, say every millisecond to see if we have messages.
How do I put a while True inside the tkinter .mainloop() that allows me to check the state of that socket constantly?
I would visualize something like this :
while True:
update_gui() # contains the mainloop and all GUI code
check_socket() # listener socket for incoming traffic
if work:
# # do a work, while GUI will hang for a bit.
I have checked the internet, and came across solution on SO, which says that you can use the After property of widgets but I am not sure how that works. If someone could help me out I would be super grateful !
Code for reference :
zmq.DONTWAIT throws an exception if you do not have any pending messages which makes us move forward in the loop.
while 1:
if socket_listen and int(share_state):
try:
msg = socket_listen.recv_string(zmq.DONTWAIT)
except:
pass
time.sleep(0.01)
I would like that I could put this inside the .mainloop() and along with the GUI this also gets checked every iteration.
Additional info : Polling here equates to :
check if we have messages on socket1
if not then proceed normally
else do work.

How do I put a while True inside the tkinter .mainloop() that allows me to check the state of that socket constantly?
Do not design such part using an explicit while True-loop, better use the tkinter-native tooling: asking .after() to re-submit the call not later than a certain amount of time ( let for other things to happen concurrently, yet having a reasonable amount of certainty, your requested call will still be activated no later than "after" specified amount of milliseconds ).
I love Tkinter architecture of co-existing event processing
So if one keeps the Finite-State-Automata ( a game, or a GUI front-end ) clean crafted on the Tkinter-grounds, one can enjoy delivering ZeroMQ-messages data being coordinated "behind" the scene, right by Tkinter-native tools, so no imperative-code will be needed whatsoever. Just let the messages get translated into tkinter-monitored-variables, if you need to have indeed smart-working GUI integration.
aScheduledTaskID = aGuiRelatedOBJECT.after( msecs_to_wait,
aFunc_to_call = None,
*args
)
# -> <_ID_#_>
# ... guarantees a given wait-time + just a one, soloist-call
# after a delay of at least delay_ms milliseconds.
# There is no upper limit to how long it will actually take, but
# your callback-FUN will be called NO SOONER than you requested,
# and it will be called only once.
# aFunc_to_call() may "renew" with .after()
#
# .after_cancel( aScheduledTaskID ) # <- <id> CANCELLED from SCHEDULER
#
# .after_idle() ~ SCHEDULE A TASK TO BE CALLED UPON .mainloop() TURNED-IDLE
#
# aScheduledTaskOnIdleID = aGuiOBJECT.after_idle( aFunc_to_call = None,
# *args
# )
# -> <_ID_#_>
That's cool on using the ready-to-reuse tkinter native-infrastructure scheduler tools in a smart way, isn't it?
Epilogue:
( Blocking calls? Better never use blocking calls at all. Have anyone ever said blocking calls here? :o) )
a while True, or a blocking socket.recv() inside tkinter's .mainloop().
well, one can put such a loop into a component aligned with tkinter native-infrastructure scheduler, yet this idea is actually an antipattern and can turn things into wreck havoc ( not only for tkinter, in general in any event-loop handler it is a bit risky to expect any "competitive" event-handler loop to somehow tolerate or behave in a peacefull the co-existence of adjacent intentions - problems will appear ( be it from a straight blocking or due to one being a just too much dominant in scheduling resources or other sorts of a war on time and resources ) ).

should i be using threads multiprocessing or asycio for my project?

I am trying to build a temperature control module that can be controlled over a network or with manual controls. the individual parts of my program all work but I'm having trouble figuring out how to make them all work together.also my temperature control module is python and the client is C#.
so far as physical components go i have a keypad that sets a temperature and turns the heater on and off and an lcd screen that displays temperature data and of course a temperature sensor.
for my network stuff i need to:
constantly send temperature data to the client.
send a list of log files to the client.
await prompts from the client to either set the desired temperature or send a log file to the client.
so far all the hardware works fine and each individual part of the network functions work but not together. I have not tried to use both physical and network components.
I have been attempting to use threads for this but was wondering if i should be using something else?
EDIT:
here is the basic logic behind what i want to do:
Hardware:
keypad takes a number inputs until '*' it then sets a temp variable.
temp variable is compared to sensor data and the heater is turned on or off accordingly.
'#' turns of the heater and sets temp variable to 0.
sensor data is written to log files while temp variable is not 0
Network:
upon client connect the client is sent a list of log files
temperature sensor data is continuously sent to client.
prompt handler listens for prompts.
if client requests log file the temperature data is halted and the file sent after which the temperature data is resumed.
client can send a command to the prompt handler to set the temp variable to trigger the heater
client can send a command to the prompt handler to stop the heater and set temp variable to 0
commands from either the keypad or client should work at all times.

Multiprocessing is generally for when you want to take advantage of the computational power of multiple processing cores. Multiprocessing limits your options on how to handle shared state between components of your program, as memory is copied initially on process creation, but not shared or updated automatically. Threads execute from the same region of memory, and do not have this restriction, but cannot take advantage of multiple cores for computational performance. Your application does not sound like it would require large amounts of computation, and simply would benefit from concurrency to be able to handle user input, networking, and a small amount of processing at the same time. I would say you need threads not processes. I am not experienced enough with asyncio to give a good comparison of that to threads.
Edit: This looks like a fairly involved project, so don't expect it to go perfectly the first time you hit "run", but definitely very doable and interesting.
Here's how I would structure this project...
I see effectively four separate threads here (maybe small ancillary dameon threads for stupid little tasks)
I would have one thread acting as your temperature controller (PID control / whatever) that has sole control of the heater output. (other threads get to make requests to change setpoint / control mode (duty cycle / PID))
I would have one main thread (with a few dameon threads) to handle the data logging: Main thead listens for logging commands (pause, resume, get, etc.) dameon threads to poll thermometer, rotate log files, etc..
I am not as familiar with networking, and this will be specific to your client application, but I would probably get started with http.server just for prototyping, or maybe something like websockets and a little bit of asyncio. The main thing is that it would interact with the data logger and temperature controller threads with getters and setters rather than directly modifying values
Finally, for the keypad input, I would likely just make up a quick tkinter application to grab keypresses, because that's what I know. Again, form a request with the tkinter app, but don't modify values directly; use getters and setters when "talking" between threads. It just keeps things better organized and compartmentalized.

Python: Threading/multiprocessing with matplotlib and user input

I am currently working on a code that will continuous plot data retrieved via serial communication while also allowing for user input, in the form of raw_input, to control the job, such as starting/stopping/clearing the plot and setting the save file names for the data. Currently, I'm trying to do this by having an extra thread that will just read user input and relay it to the program while it continuously plots and saves the data.
Unfortunately, I have run into some errors where commands that are entered during the plotting loop freeze the program for 2 minutes or so, which I believe has to do with matplotlib not being thread safe, where a command entered while the loop is not working with the plotting libraries will lead to a response in 1-2 seconds.
I have attempted switching from threading to the multiprocessing library to try to alleviate the problem to no avail, where the program will not show a plot, leading me to believe the plotting process never starts (the plotting command is the first command in it). I can post the codes or the relevant parts of either program if necessary.
I wanted to know if there was any way around these issues, or if I should start rethinking how I want to program this. Any suggestions on different ways of incorporating user input are welcome too.
Thanks

If matplotlib is not thread-safe, the right thing to do is to serialize all the inputs into matplotlib through a single event queue. Matplotlib can retrieve items off the queue until the queue is empty, and then process all the new parameters.
Your serial communication code and your raw_input should simply put data on this queue, and not attempt direct communication with matplotlib.
Your matplotlib thread will be doing one of three things: (1) waiting for new data; (2) retrieving new data and processing it (e.g. appending it to the arrays to be plotted or changing output file names) and staying in this state as long as the queue is not empty, or moving on to state (3) if it is; or (3) invoking matplotlib to do the plotting, then looping back to state (1).
If you are implementing multiple action commands from your raw_input, you can add some auxiliary state variables. For example, if 'stop' is read from the queue, then you would set a variable that would cause state (3) to skip the plotting and go straight to state (1), and if 'start' is read from the queue, you would reset this variable, and would resume plotting when data is received.
You might think you want to do something fancy like: "if I see data, wait to make sure more is not coming before I start to plot." This is usually a mistake. You would have to tune your wait time very carefully, and then would still find times when your plotting never happened because of the timing of the input data. If you have received data (you are in state 2), and the queue is empty, just plot! In the time taken to do that, if 4 more data points come in, then you'll plot 4 more next time...

How to prebuffer an incoming network stream with gstreamer?

I'm using gstreamer to stream audio over the network. My goal is seemingly simple: Prebuffer the incoming stream up to a certain time/byte threshold and then start playing it.
I might be overlooking a really simple feature of gstreamer, but so far, I haven't been able to find a way to do that.
My (simplified) pipeline looks like this: udpsrc -> alsasink. So far all my attempts at achieving my goal have been using a queue element in between:
Add a queue element in between.
Use the min-threshold-time property. This actually works but the problem is, that it makes all the incoming data spend the specified minimum amount of time in the queue, rather than just the beginning, which is not what I want.
To work around the previous problem, I tried to have the queue notify my code when data enters the audio sink for the first time, thinking that this is the time to unset the min-time property that I set earlier, and thus, achieving the "prebuffering" behavior.
Here's is a rough equivalent of the code I tried:
def remove_thresh(pad, info, queue):
pad.remove_data_probe(probe_id)
queue.set_property("min-threshold-time", 0)
queue.set_property("min-threshold-time", delay)
queue.set_property("max-size-time", delay * 2)
probe_id = audiosink.get_pad("sink").add_data_probe(remove_thresh, queue)
This doesn't work for two reasons:
My callback gets called way earlier than the delay variable I provided.
After it gets called, all of the data that was stored in the queue is lost. the playback starts as if the queue weren't there at all.
I think I have a fundamental misunderstanding of how this thing works. Does anyone know what I'm doing wrong, or alternatively, can provide a (possibly) better way to do this?
I'm using python here, but any solution in any language is welcome.
Thanks.

Buffering has already been implemented in GStreamer. Some elements, like the queue, are capable of building this buffer and post bus messages regarding the buffer level (the state of the queue).
An application wanting to have more network resilience, then, should listen to these messages and pause playback if the buffer level is not high enough (usually, whenever it is below 100%).
So, all you have to do is set the pipeline to the PAUSED state while the queue is buffering. In your case, you only want to buffer once, so use any logic for this (maybe set flag variables to pause the pipeline only the first time).
Set the "max-size-bytes" property of the queue to the value you want.
Either listen to the "overrun" signal to notify you when the buffer becomes full or use gst_message_parse_buffering () to find the buffering level.
Once your buffer is full, set the pipeline to PLAYING state and then ignore all further buffering messages.
Finally, for a complete streaming example, you can refer to this tutorial: https://gstreamer.freedesktop.org/documentation/tutorials/basic/streaming.html
The code is in C, but the walkthroughs should help you with you want.

I was having the exact same problems as you with a different pipeline (appsrc), and after spending days trying to find an elegant solution (and ending up with code remarkably similar to what you posted)... all I did was switch the flag is-live to False and the buffering worked automagically. (no need for min-threshold-time or anything else)
Hope this helps.