I have a python GUI interface written in Tkinter. The main point of the program is to setup many different variables for a final calculation on a hyperspectral image (geography stuff). Now, some further specifications have developed where the user would like to be able to actively input some parameters for groups of pixels to be smoothed. This information would be input in the Python GUI and the C programs that handle the image modifications need this as input. Since the images can be giant, I want to try and avoid always re-running the C program (which involves memory allocation, reading a giant file, etc.) with a call such as
os.system(./my_C_Program param1 param2 param3....)
I'd prefer to have a system where once I've called my_C_Program, it can be in a state of waiting after having loaded all the resources into memory. I was thinking something involving getchar() would be what I want, but I don't know how I can get the output from python to go my_C_Program. I've seen a few similar questions about this on SO, but I wasn't able to determine quite how those scenarios would help mine specifically.
If getchar() is the answer, can someone please explain how stdout works with multiple terminals open?
As well, I'm trying to keep this program easily multiplatform across linux/mac/windows.
To summarize, I want the following functionality:
User selects certain input from python GUI
That input becomes the input for a C program
That C program can handle more input without having to be run again from the start (avoiding file I/O, etc).
The first thing you should probably do is start using Python's subprocess module, rather than os.system. Once you've done that, then you can change it so the C program's stdin is something you can write to in Python, rather than inheriting the Python script's stdin.
After that, you could just have Python send data over that the C program can interpret. For example, you might want to use a bunch of JSON chunks, one per line, like Twitter's streaming API1; the Python script makes a request dictionary, serializes it with json.dump, and then writes a newline. The C program reads a line, parses the JSON, and handles the request.
1 Upon reading the documentation, it looks like their implementation is a little more complex. You could adopt how they do it or just do it like I described.
icktoofay and JasonFruit have suggested decent approaches; I'm going to suggest something to decouple the two programs a little further.
If you write your C program as a server that listens for requests and replies with answers on a TCP socket, you can more easily change clients, make it support multiple simultaneous clients, perform near-seamless upgrades of clients or servers without necessarily needing to modify the other, or you could move the C program to more powerful hardware without needing to do more than slightly modify configurations.
Once you have opened a listening socket and accepted a connection, the rest of your program could continue as if you were just interacting over standard input and standard output. This works well enough, but you might prefer to encode your data in some standardized format such as JSON or ASN.1, which can save some manual string handling.
Could you do something with pexpect? It lets you provide input to a command-line program, waiting for specified prompts from it before continuing. It also lets you read the intervening output, so you could respond to that as needed.
If you're on Windows (as I note from your comment that you are), you could try winpexpect, which is similar.
Related
I have a flow sensor that I have to read with c because python isn't fast enough but the rest of my code is python. What I want to do is have the c code running in the background and just have the python request a value from it every now and then. I know that popen is probably the easiest way to do this but I don't fully understand how to use it. I don't want completed code I just want a way to send text/numbers back and forth between a python and a c code. I am running raspbian on a raspberry pi zero w. Any help would be appreciated.
Probably not a full answer, but I expect it gives some hints and it is far too long for a comment. You should think twice about your requirements, because it will probably not be that easy depending on your proficiency in C and what OS you are using.
If I have correctly understood, you have a sensor that sends data (which is already weird unless the sensor is an intelligent one). You want to write a C program that will read that data and either buffer it, and retain only last (you did not say...) and at the same time will wait for requests from a Python script to give it back what it has received (and kept) from the sensor. That probably means a dual thread program with quite a bit of synchronization.
You will also need to specify the communication way between C and Python. You can certainly use the subprocess module, but do not forget to use unbuffered output in C. But you could also imagine an independant program that uses a FIFO or a named piped with a well defined protocol for external requests, in order to completely separate both problems.
So my opinion is that this is currently too broad for a single SO question...
I have a main program X which is getting feed from my webcam.
I want to configure X in real-time while it's executing.
I understand that one of the common ways of doing that is using IPC like named-pipes/Unix sockets/Internet sockets, etc. But I want to avoid each caller to have to separately open a socket/named-pipe, and communicate each time.
In short, I want a helper program by the name Y, which I can use in the following manner:
Y set-fps=15
Y show-frame=true
Y get-fps (should return 15)
I would want to play this helper program Y in /usr/bin/* (or rather place it in one of $PATH directoreis) so that it's executable from the command-line.
What are my options for obtaining this functionality. My constraints are as under:
(i) Program X could be either C++/Python.
(ii) Multiple clients could call Y simultaneously.
I guess such systems are common on linux where you have programs like nmcli interacting with services like the network-manager ?
sorry to put that post in answer, i still can't comment.
one of the way i could see to do such is the following :
as your program takes frames from the webcam, there is certainly a loop or a redundant automat.
In this way, you could allocate some registers in the kernel memory in one block, each of registers being one of the parameter.
then, at each loop, your X program could read the different parameters from that kernel space. On the other side, each client who wants to modify the parameters will access this kernel space too, and modify the required values.
Then, you have to protect the kernel space with a semaphore, for any write or read operation.(a mutex is ok there, however you may want some writer/reader implementation)
Of course, this is not optimal in performance compared to a protection for each one of the parameters, but with that you have to handle only one semaphore, and a simple mutex would do the job. It's certainly also less beautiful than a communication with pipes or sockets, but with them, you would still have to protect the read and write of your parameters...
I've got a python program that reads input from a midi device and produces text output derived from the incoming MIDI messages. As a simple example, let's say that it's simply mapping MIDI Note On events to note names e.g. note_on(60) --> 'C'. I'd like to capture the output in real time to a GVIM (actually MacVim) window without losing the ability to edit the output with a computer keyboard, i.e. I need for MacVim to read from both an external program and from the computer keyboard.
What's the cleanest general way to implement that under the assumption that the MIDI reader will never generate output while I'm trying to type and vice-versa? I'd prefer to be able to give the python script a filename and have it start MacVim with that file open, but doing it with shell commands or connecting from within MacVim would also be acceptable.
Based on the answers to How do I read and write repeatedly from a process in vim?, it looks like vim does not easily support 2 input sources asynchronously. I'll leave the question open in case someone happens to know an elegant solution, but for now it seems like the best approach is have my python program write to a normal file, use 'tail -f' for real-time viewing, and edit afterwards.
I'm in the process of rewriting a server program, and I want to add into it a simple console input.
At the moment, it just serves data and prints out one or two lines for each thing it does, as a descriptive measure for anyone watching/debugging.
What I want is to have a "sticky" input bar that is always at the bottom, above which my debug prints appear, so that I can enter commands at any point while the program is printing. This would look a bit like:
...
[88.88.88.88] Handling Connection on Port 11452
[12.12.12.12] Received Data
[44.44.44.44] Sending Disconnect Sequence
>>>Enter Data Here at Any Time
Ideally, this would be done without curses as this would complicate matters. I feel like I must be missing a simple solution.
Thanks in Advance,
Freddy.
If you're looking for a slightly higher-level library than curses, there are a few, like urwid (although of course they don't come with Python, and have to be installed).
If you really want to avoid a windowing library, I suppose you could run the server under screen, configured with an input bar and a rest-of-the-window, and give the server the two virtual TTYs instead of normal input… but that would be a lot more work, not less.
If you want to go lower level, you can always use termios, search termcap, and manually write control sequences to manually take care of scrolling everything but the last line, moving the input cursor, and so on. But that's going to be even more work.
And of course there's always the super-hacky possibility: For every output, and after every input, "redraw" the whole screen by spamming a few hundred blank lines, re-writing the output, and writing the input line. This will look terrible, and not work in a variety of edge cases, but it does avoid any kind of windowing.
However, I think there's a much better way to solve this problem.
You've already got a server. Why not just serve the console API on another "control" port (maybe just on localhost) instead of on the stdin/stdout/tty? That will work even if the server is, e.g., running as a daemon. And it means you can use your favorite fancy Telnet or similar client as a console, instead of having to write something from scratch.
I just made a little chat CLI-application that works pretty fine actually!
Unfortunately, when one types a message and receives other's messages at the same time, both interlacing... Which renders quite awful:
elton: Hey!
john: how are you doing?
fine anjohn: still not to bed?!
d john: haha
you?elton: fine and you?
I'm looking for a way to avoid this kind of problem. Such as "reserving" the last line for user-input or process some actions when data are received to recalculate the position of user-input.
After some research I found that I shall retrieve each character one by one with getch(). So that I can check regularly if a new message is waiting to be displayed and handle this case.
But if I use getch() I have to redefine manually basic actions (such as backspace, move left and right...), some characters take more than one byte. In brief, it's unusable.
As far as I can tell, you have a few options here. Normally just calling getch you're running the terminal in what's called "cooked" mode. This means that the terminal just displays characters in the order they arrive, nothing too special. However you run in to race conditions as you've discovered.
Option 1: you read things in one character at a time, buffering the entire line. When a new line arrives and you want to print it out, you'd have to (a) grab hold of some kind of print mutex (internal to the program), (b) clear out the current line (print '\r[space][space][space]...\r'), (c) print the incoming line + '\n', and (d) restore the previous buffer to the screen (via print) and unlock the mutex.
This gets ugly. Fast. As you discovered, there's no line editing support or anything fancy. The only good thing about this approach is that it'll probably work in about 99% of terminals.
Option 2: you put the terminal in to raw mode. In this mode, you have complete control over the terminal, but you lose very basic functionality (e.g. typing a key won't display it unless you manually output it). You would still have to manually implement things like navigation, but at least it would work. This method is probably the most flexible and give you the most control, however it's very difficult to work like this, which is why they invented...
Option 3: you use ncurses (or a similar terminal library). The concepts are a little difficult, but FAR easier to learn than doing things in pure raw mode. Editing and windowing (or as you put it, reserving a line) are built-in. The interface can be much prettier than you'd get from cooked mode.
EDIT I'm talking a lot about *nix terminals here, most of which doesn't apply for Windows.
The usual way to do this kind of thing is by writing a "console-mode GUI" instead of a pure CLI.
For example, you can very easily create a very simple interface with a ROWS-2xCOLS output window, a 1xCOLS divider or mode line, and a 1xCOLS input window—as seen in things like IRC clients, emacs, etc. For example:
Or you could do a split view, with a ROWS/2xCOLS window for the other person/people's chat, and a ROWS/2xCOLS window below that for the user's current and previous messages, as seen in most of the early Unix chat programs.
Or something fancy with windows all over the place that pop up and go away as appropriate, as some modern IRC clients do.
Python has curses support built-in—but not on Windows. If you want a cross-platform solution, or a Windows-only solution, see curses-like library from cross-platform console app in python, and the answer is probably there.
Here's a screenshot from the IRC client BitchX, to give you an idea of what's possible without too much work: