I am writing a python script that I want to use in a unix pipeline. My goal is to write to the screen using curses (which should only be seen by the person running the command, not the pipe), and then write the "return value" to stdout at the end so it can continue down the pipeline, something along the lines of ./myscript.py | consumer_script
This was failing in mysterious ways until I found This. The suggested solution was to use newterm instead of init_scr.
My problem is that I am using python, and from what I could find in the documentation, newterm doesnt exist. All I was able to find was a single reference to newterm, and it didn't come with a link.
Could someone please either point me towards the python newterm, or suggest another way of working with pipes and curses.
I think you're making this more complicated than it needs to be... the simple answer is to write the curses stream to another handle than stdout. If it works for you, stderr is the obvious choice. In short, anything that gets written to stdout goes into the pipeline, and if you don't want it there, you need a different handle.
Check out this thread for ways to write to stderr in python:
How to print to stderr in Python?
This Python program enters fdisk. I see the output. fdisk is an interactive program. How do I get the Python program to pass an "m" to the first field and press enter?
import subprocess
a = "dev/sda"
x = subprocess.call(["fdisk", a])
print x
I'd rather not import a new module/library, but I could. I've tried different syntax with subprocess.call() and extra parameters in the above. Nothing seems to work. I get different errors. I've reviewed Python documentation. I want to feed input and press Enter in the subsequent, interactive menu options of fdisk.
Check out the pexpect library (I know you didn't want an extra module, but you want to use the best tool for the job). It's pure Python, with no compiled submodules, so installation is a snap. Basically, it does the same thing in Python as the classic Unix utility expect - spawns child applications, controls them, and responds to expected patterns in their output. It's great for automation, and especially application testing, where you can quickly feed the newest build of a command-line program a series of inputs and guide the interaction based on what output appears.
In case you just don't want another module at all, you can always fall back on the subprocess module's Popen() constructor. It spawns and creates a connection to a child process, allowing you to communicate with it as needed, and in fact pexpect relies a great deal on it. I personally think using pexpect is more intuitive than subprocess.Popen(), but that's just me. YMMV.
Here's a general example of what I need to do:
For example, I would initiate a back trace by sending the command "bt" to GDB from the program. Then I would search for a word such as "pardrivr" and get the line number associated with it by using regular expressions. Then I would input "f [line_number_of_pardriver]" into GDB. This process would be repeated until the correct information is eventually extracted.
I want to use named pipes in bash or python to accomplish this.
Could someone please provide a simple example of how to do this?
My recommendation is not to do this. Instead there are two more supportable ways to go:
Write your code in Python directly in gdb. Gdb has been extensible in Python for several years now.
Use the gdb MI ("Machine Interface") approach. There are libraries available to parse this already (not sure if there is one in Python but I assume so). This is better than parsing gdb's command-line output because some pains are taken to avoid gratuitous breakage -- this is the preferred way for programs to interact with gdb.
how can I write/append to a file by calling nano using subprocess and get it saved automatically .For example I have a file and I want to open it and append something at the end of it so I write
>>> import tempfile
>>> file = tempfile.NamedTemporaryFile(mode='a')
>>> example = file.name
>>> f.close()
>>> import subprocess
>>> subprocess.call(['nano', example])
Now once the last line gets executed the file gets open and I can write anything and then save it by hitting Ctrl+O and Ctrl+X
Instead I want that I send the input through a stdin PIPE and and the file gets saved by itself ie there could be any mechanism that hits Ctrl+O and Ctrl+X automayically by itself ?
Can help me in solving this issue ?
A ctrl-O is just a character, same as any other. You can send it by writing '\x0f' (or, in Python 3, b'\x0f').
However, that probably isn't going to do you any good. Most programs that provide an interactive GUI in the terminal, like nano, cannot be driven by stdin. They need to take control of the terminal, and to do that, they will either check that stdin isatty and then tcsetattr it, or just open /dev/tty,
You can deal with this by creating a pseudo-terminal with os.openpty, os.forkpty, or pty.
But it's often easier to use a library like pexpect to deal with interactive programs, GUI or otherwise.
And it's even easier to not try to drive an interactive program in the first place. For example, unlike nano, ed is designed to be driven in "batch mode" by a script, and sed even more so.
And it's even easier to not try to drive a program at all when you're trying to do something that can be just as easily done directly in Python. The easiest way to append something to a file is to open it in 'a' mode and write to it. No need for an external program at all. For example:
new_line = input('What do you want to add?')
with open(fname, 'a') as f:
f.write(new_line)
If the only reason you were using nano is because you needed something to sudo… there's really no reason for that. You can sudo anything else—like sed, or another Python script—just as easily. Using nano is just making things harder for yourself for absolutely no reason.
The big question here is: why do you have a file that's not writable by your Python script, but which you want arbitrary remote users to be able to append to? That sounds like a very bad system design. You make files non-writable because you want to restrict normal users from modifying them; if you want your Python script to be able to modify it on behalf of your remote users, why isn't it owned by the same user that the script runs as?
In the (unlikely) event that you still find that you need to control nano or some other interactive program from a Python process, I'm going to suggest the same thing here that I suggested for this question: Using python subprocess.call() to launch an ncurses process ...
... don't use subprocess for controlling curses/full-screen interactive processes. use pexpect. That's what it's for.
(On the other hand I also agree with the many comments here regarding better ways to work around the permissions issue. Write some sort of script (in Python, bash, sed or whatever) which can be run under sudo and which can make the in-place edits or appendices to your data file directly.
I'm trying to run an external, separate program from Python. It wouldn't be a problem normally, but the program is a game, and has a Python interpreter built into it. When I use subprocess.Popen, it starts the separate program, but does so under the original program's Python instance, so that they share the first Python console. I can end the first program fine, but I would rather have separate consoles (mainly because I have the console start off hidden, but it gets shown when I start the program from Python with subprocess.POpen).
I would like it if I could start the second program wholly on its own, as though I just 'double-clicked on it'. Also, os.system won't work because I'm aiming for cross-platform compatibility, and that's only available on Windows.
I would like it if I could start the second program wholly on its own, as though I just 'double-clicked on it'.
As of 2.7 and 3.3, Python doesn't have a cross-platform way to do this. A new shutil.open method may be added in the future (possibly not under that name); see http://bugs.python.org/issue3177 for details. But until then, you'll have to write your own code for each platform you care about.
Fortunately, what you're trying to do is simpler and less general than what shutil.open is ultimately hoped to provide, which means it's not that hard to code:
On OS X, there's a command called open that does exactly what you want: "The open command opens a file (or a directory or URL), just as if you had double-clicked the file's icon." So, you can just popen open /Applications/MyGame.app.
On Windows, the equivalent command is start, but unfortunately, that's part of the cmd.exe shell rather than a standalone program. Fortunately, Python comes with a function os.startfile that does the same thing, so just os.startfile(r'C:\Program Files\MyGame\MyGame.exe').
On FreeDesktop-compatible *nix systems (which includes most modern linux distros, etc.), there's a very similar command called xdg-open: "xdg-open opens a file or URL in the user's preferred application." Again, just popen xdg-open /usr/local/bin/mygame.
If you expect to run on other platforms, you'll need to do a bit of research to find the best equivalent. Otherwise, for anything besides Mac and Windows, I'd just try to popen xdg-open, and throw an error if that fails.
See http://pastebin.com/XVp46f7X for an (untested) example.
Note that this will only work to run something that actually can be double-clicked to launch in Finder/Explorer/Nautilus/etc. For example, if you try to launch './script.py', depending on your settings, it may just fire up a text editor with your script in it.
Also, on OS X, you want to run the .app bundle, not the UNIX executable inside it. (In some cases, launching a UNIX executable—whether inside an .app bundle or standalone—may work, but don't count on it.)
Also, keep in mind that launching a program this way is not the same as running it from the command line—in particular, it will inherit its environment, current directory/drive, etc. from the Windows/Launch Services/GNOME/KDE/etc. session, not from your terminal session. If you need more control over the child process, you will need to look at the documentation for open, xdg-open, and os.startfile and/or come up with a different solution.
Finally, just because open/xdg-open/os.startfile succeeds doesn't actually mean that the game started up properly. For example, if it launches and then crashes before it can even create a window, it'll still look like success to you.
You may want to look around PyPI for libraries that do what you want. http://pypi.python.org/pypi/desktop looks like a possibility.
Or you could look through the patches in issue 3177, and pick the one you like best. As far as I know, they're all pure Python, and you can easily just drop the added function in your own module instead of in os or shutil.
As a quick hack, you may be able to (ab)use webbrowser.open. "Note that on some platforms, trying to open a filename using this function, may work and start the operating system’s associated program. However, this is neither supported nor portable." In particular, IIRC, it will not work on OS X 10.5+. However, I believe that making a file: URL out of the filename actually does work on OS X and Windows, and also works on linux for most, but not all, configurations. If so, it may be good enough for a quick&dirty script. Just keep in mind that it's not documented to work, it may break for some of your users, it may break in the future, and it's explicitly considered abuse by the Python developers, so I wouldn't count on it for anything more serious. And it will have the same problems launching 'script.py' or 'Foo.app/Contents/MacOS/foo', passing env variables, etc. as the more correct method above.
Almost everything else in your question is both irrelevant and wrong:
It wouldn't be a problem normally, but the program is a game, and has a Python interpreter built into it.
That doesn't matter. If the game were writing to stdout from C code, it would do the exact same thing.
When I use subprocess.Popen, it starts the separate program, but does so under the original program's Python instance
No it doesn't. It starts an entirely new process, whose embedded Python interpreter is an entirely new instance of Python. You can verify that by, e.g., running a different version of Python than the game embeds.
so that they share the first Python console.
No they don't. They may share the same tty/cmd window, but that's not the same thing.
I can end the first program fine, but I would rather have separate consoles (mainly because I have the console start off hidden, but it gets shown when I start the program from Python with subprocess.POpen).
You could always pipe the child's stdout and stderr to, e.g., a logfile, which you could then view separately from the parent process's output, if you wanted to. But I think this is going off on a tangent that has nothing to do with what you actually care about.
Also, os.system won't work because I'm aiming for cross-platform compatibility, and that's only available on Windows.
Wrong; os.system is available on "Unix, Windows"--which is probably everywhere you care about. However, it won't work because it runs the child program in a subshell of your script, using the same tty. (And it's got lots of other problems—e.g., blocking until the child finishes.)
When I use subprocess.Popen, it starts the separate program, but does so under the original program's Python instance...
Incorrect.
... so that they share the first Python console.
This is the crux of your problem. If you want it to run in another console then you must run another console and tell it to run your program instead.
... I'm aiming for cross-platform compatibility ...
Sorry, there's no cross-platform way to do it. You'll need to run the console/terminal appropriate for the platform.