I'm trying to run an external, separate program from Python. It wouldn't be a problem normally, but the program is a game, and has a Python interpreter built into it. When I use subprocess.Popen, it starts the separate program, but does so under the original program's Python instance, so that they share the first Python console. I can end the first program fine, but I would rather have separate consoles (mainly because I have the console start off hidden, but it gets shown when I start the program from Python with subprocess.POpen).
I would like it if I could start the second program wholly on its own, as though I just 'double-clicked on it'. Also, os.system won't work because I'm aiming for cross-platform compatibility, and that's only available on Windows.
I would like it if I could start the second program wholly on its own, as though I just 'double-clicked on it'.
As of 2.7 and 3.3, Python doesn't have a cross-platform way to do this. A new shutil.open method may be added in the future (possibly not under that name); see http://bugs.python.org/issue3177 for details. But until then, you'll have to write your own code for each platform you care about.
Fortunately, what you're trying to do is simpler and less general than what shutil.open is ultimately hoped to provide, which means it's not that hard to code:
On OS X, there's a command called open that does exactly what you want: "The open command opens a file (or a directory or URL), just as if you had double-clicked the file's icon." So, you can just popen open /Applications/MyGame.app.
On Windows, the equivalent command is start, but unfortunately, that's part of the cmd.exe shell rather than a standalone program. Fortunately, Python comes with a function os.startfile that does the same thing, so just os.startfile(r'C:\Program Files\MyGame\MyGame.exe').
On FreeDesktop-compatible *nix systems (which includes most modern linux distros, etc.), there's a very similar command called xdg-open: "xdg-open opens a file or URL in the user's preferred application." Again, just popen xdg-open /usr/local/bin/mygame.
If you expect to run on other platforms, you'll need to do a bit of research to find the best equivalent. Otherwise, for anything besides Mac and Windows, I'd just try to popen xdg-open, and throw an error if that fails.
See http://pastebin.com/XVp46f7X for an (untested) example.
Note that this will only work to run something that actually can be double-clicked to launch in Finder/Explorer/Nautilus/etc. For example, if you try to launch './script.py', depending on your settings, it may just fire up a text editor with your script in it.
Also, on OS X, you want to run the .app bundle, not the UNIX executable inside it. (In some cases, launching a UNIX executable—whether inside an .app bundle or standalone—may work, but don't count on it.)
Also, keep in mind that launching a program this way is not the same as running it from the command line—in particular, it will inherit its environment, current directory/drive, etc. from the Windows/Launch Services/GNOME/KDE/etc. session, not from your terminal session. If you need more control over the child process, you will need to look at the documentation for open, xdg-open, and os.startfile and/or come up with a different solution.
Finally, just because open/xdg-open/os.startfile succeeds doesn't actually mean that the game started up properly. For example, if it launches and then crashes before it can even create a window, it'll still look like success to you.
You may want to look around PyPI for libraries that do what you want. http://pypi.python.org/pypi/desktop looks like a possibility.
Or you could look through the patches in issue 3177, and pick the one you like best. As far as I know, they're all pure Python, and you can easily just drop the added function in your own module instead of in os or shutil.
As a quick hack, you may be able to (ab)use webbrowser.open. "Note that on some platforms, trying to open a filename using this function, may work and start the operating system’s associated program. However, this is neither supported nor portable." In particular, IIRC, it will not work on OS X 10.5+. However, I believe that making a file: URL out of the filename actually does work on OS X and Windows, and also works on linux for most, but not all, configurations. If so, it may be good enough for a quick&dirty script. Just keep in mind that it's not documented to work, it may break for some of your users, it may break in the future, and it's explicitly considered abuse by the Python developers, so I wouldn't count on it for anything more serious. And it will have the same problems launching 'script.py' or 'Foo.app/Contents/MacOS/foo', passing env variables, etc. as the more correct method above.
Almost everything else in your question is both irrelevant and wrong:
It wouldn't be a problem normally, but the program is a game, and has a Python interpreter built into it.
That doesn't matter. If the game were writing to stdout from C code, it would do the exact same thing.
When I use subprocess.Popen, it starts the separate program, but does so under the original program's Python instance
No it doesn't. It starts an entirely new process, whose embedded Python interpreter is an entirely new instance of Python. You can verify that by, e.g., running a different version of Python than the game embeds.
so that they share the first Python console.
No they don't. They may share the same tty/cmd window, but that's not the same thing.
I can end the first program fine, but I would rather have separate consoles (mainly because I have the console start off hidden, but it gets shown when I start the program from Python with subprocess.POpen).
You could always pipe the child's stdout and stderr to, e.g., a logfile, which you could then view separately from the parent process's output, if you wanted to. But I think this is going off on a tangent that has nothing to do with what you actually care about.
Also, os.system won't work because I'm aiming for cross-platform compatibility, and that's only available on Windows.
Wrong; os.system is available on "Unix, Windows"--which is probably everywhere you care about. However, it won't work because it runs the child program in a subshell of your script, using the same tty. (And it's got lots of other problems—e.g., blocking until the child finishes.)
When I use subprocess.Popen, it starts the separate program, but does so under the original program's Python instance...
Incorrect.
... so that they share the first Python console.
This is the crux of your problem. If you want it to run in another console then you must run another console and tell it to run your program instead.
... I'm aiming for cross-platform compatibility ...
Sorry, there's no cross-platform way to do it. You'll need to run the console/terminal appropriate for the platform.
Related
When reading a book or just coding on terminal/IDLE it's common to make typo, forgot brace or comma etc. After I got error and all what I wrote before is lost.
Then I have to write down code again..
Is there any way/option to return back all what write before and just edit mistake and continue to code?
In Idle (at least my version, Python 2.7.10 on windows), you can simply copy paste your code. In the python interpreter, you can't afaik, however you can use the up/down arrow keys to recall lines you previously "submitted" (i.e. typed and pressed enter).
If I understood correctly, IDLE is a GUI (graphical user interface - a visual representation of a program rather just through text) made to have a bit more features for programming in Python. You can use IDLE interactively, like in Terminal (a.k.a command line), or use it to write your script rather than in a separate text editor. Then once you save your script/program you can do neat things like run it directly from IDLE. There's nothing more special about the Terminal, you just have to do some more work.
Furthermore, all the code you have written on your GUI is on the cache memory which is used in system to store information recently accessed by a processor. So, I suggest you write again your code you can't recover them without saving.
To avoid these kind of problems use Git!
Git is a version control system that is used for software development and other version control tasks.
IDLE's Shell window is statement rather that line oriented. One can edit any line of a statement before submitting it for execution. After executing, one may recall any statement by either a) placing the cursor anywhere on the statement and hitting Enter, or b) using the history-next and history-prev actions. On Windows, these are bound, by default, to Alt-p and Alt-p. To check on your installation, Select Options => IDLE preferences on the menu. In the dialog, select the Keys tab. Under Custom Key Bindings, find the 'histor-xyz' actions in the alphabetical list.
For short, one-off scripts, I have a scratch file called tem.py. Since I use it often, it is usually accessible via File => Recent files.
This Python program enters fdisk. I see the output. fdisk is an interactive program. How do I get the Python program to pass an "m" to the first field and press enter?
import subprocess
a = "dev/sda"
x = subprocess.call(["fdisk", a])
print x
I'd rather not import a new module/library, but I could. I've tried different syntax with subprocess.call() and extra parameters in the above. Nothing seems to work. I get different errors. I've reviewed Python documentation. I want to feed input and press Enter in the subsequent, interactive menu options of fdisk.
Check out the pexpect library (I know you didn't want an extra module, but you want to use the best tool for the job). It's pure Python, with no compiled submodules, so installation is a snap. Basically, it does the same thing in Python as the classic Unix utility expect - spawns child applications, controls them, and responds to expected patterns in their output. It's great for automation, and especially application testing, where you can quickly feed the newest build of a command-line program a series of inputs and guide the interaction based on what output appears.
In case you just don't want another module at all, you can always fall back on the subprocess module's Popen() constructor. It spawns and creates a connection to a child process, allowing you to communicate with it as needed, and in fact pexpect relies a great deal on it. I personally think using pexpect is more intuitive than subprocess.Popen(), but that's just me. YMMV.
I am in the works of creating a python program similar to this
. Anyway what i want to do is have users be able to modify there own programs but i need help understanding how this works. I have looked through the source code and am confused where this happens even if someone could just point me towards that that would be very helpful. I know that the programs will not be sandboxed but that is not something im worried about at the moment. If you could point me in any direction that would be great! Thank you!
The "robot programs" are just stored as plain text files.
There's a general-purpose text editor in editor.py. When you open a robot in a given view, e.g., the Qt4 view in qt4view.py, it just instantiates a text editor and hands it the robot's file. Again, the fact that the robot's file is a Python script doesn't matter; it just edits it as a text file.
The battle code, meanwhile, opens the same robot files as Python code that the text editor opens as text files. You can see this code in game.py: It just uses the subprocess module to run Python, passing the robot file as an argument.
My other answer deals with what you actually asked. But I don't think it's what you really wanted to know.
You just want to know how to run some Python script, that you've got a pathname for, in a separate Python interpreter, right?
While it's possible to figure that out from the pybotwar code, there's a whole lot of extra stuff that will get in the way of understanding it—the conf.py file, the configurable extra flags, etc.
But the answer is simple: Use the subprocess module, just as you would for running any program. In this case, the Python interpreter is the executable (usually you want sys.executable, the same Python interpreter you're using), and the script you want to run as an argument. For example:
script_output = subprocess.check_output([sys.executable, script_path])
The subprocess documentation explains all the different options very nicely.
Per Python documentation, subprocess.call should be blocking and wait for the subprocess to complete. In this code I am trying to convert few xls files to a new format by calling Libreoffice on command line. I assumed that the call to subprocess call is blocking but seems like I need to add an artificial delay after each call otherwise I miss few files in the out directory.
what am I doing wrong? and why do I need the delay?
from subprocess import call
for i in range(0,len(sorted_files)):
args = ['libreoffice', '-headless', '-convert-to',
'xls', "%s/%s.xls" %(sorted_files[i]['filename'],sorted_files[i]['filename']), '-outdir', 'out']
call(args)
var = raw_input("Enter something: ") # if comment this line I dont get all the files in out directory
EDIT It might be hard to find the answer through the comments below. I used unoconv for document conversion which is blocking and easy to work with from an script.
It's possible likely that libreoffice is implemented as some sort of daemon/intermediary process. The "daemon" will (effectively1) parse the commandline and then farm the work off to some other process, possibly detaching them so that it can exit immediately. (based on the -invisible option in the documentation I suspect strongly that this is indeed the case you have).
If this is the case, then your subprocess.call does do what it is advertised to do -- It waits for the daemon to complete before moving on. However, it doesn't do what you want which is to wait for all of the work to be completed. The only option you have in that scenario is to look to see if the daemon has a -wait option or similar.
1It is likely that we don't have an actual daemon here, only something which behaves similarly. See comments by abernert
The problem is that the soffice command-line tool (which libreoffice is either just a link to, or a further wrapper around) is just a "controller" for the real program soffice.bin. It finds a running copy of soffice.bin and/or creates on, tells it to do some work, and then quits.
So, call is doing exactly the right thing: it waits for libreoffice to quit.
But you don't want to wait for libreoffice to quit, you want to wait for soffice.bin to finish doing the work that libreoffice asked it to do.
It looks like what you're trying to do isn't possible to do directly. But it's possible to do indirectly.
The docs say that headless mode:
… allows using the application without user interface.
This special mode can be used when the application is controlled by external clients via the API.
In other words, the app doesn't quit after running some UNO strings/doing some conversions/whatever else you specify on the command line, it sits around waiting for more UNO commands from outside, while the launcher just runs as soon as it sends the appropriate commands to the app.
You probably have to use that above-mentioned external control API (UNO) directly.
See Scripting LibreOffice for the basics (although there's more info there about internal scripting than external), and the API documentation for details and examples.
But there may be an even simpler answer: unoconv is a simple command-line tool written using the UNO API that does exactly what you want. It starts up LibreOffice if necessary, sends it some commands, waits for the results, and then quits. So if you just use unoconv instead of libreoffice, call is all you need.
Also notice that unoconv is written in Python, and is designed to be used as a module. If you just import it, you can write your own (simpler, and use-case-specific) code to replace the "Main entrance" code, and not use subprocess at all. (Or, of course, you can tear apart the module and use the relevant code yourself, or just use it as a very nice piece of sample code for using UNO from Python.)
Also, the unoconv page linked above lists a variety of other similar tools, some that work via UNO and some that don't, so if it doesn't work for you, try the others.
If nothing else works, you could consider, e.g., creating a sentinel file and using a filesystem watch, so at least you'll be able to detect exactly when it's finished its work, instead of having to guess at a timeout. But that's a real last-ditch workaround that you shouldn't even consider until eliminating all of the other options.
If libreoffice is being using an intermediary (daemon) as mentioned by #mgilson, then one solution is to find out what program it's invoking, and then directly invoke it yourself.
I like the python-send-buffer command, however I very often use Python embedded in applications, or launch Python via a custom package management system (to launch Python with certain dependencies).. In other words, I can't just run "python" and get a useful Python instance (something that python-send-buffer relies on)
What I would like to achieve is:
in any Python interpreter (or application that allows you to evaluate Python code), import a magic_emacs_python_server.py module (appending to sys.path as necessary)
In emacs, run magic-emacs-python-send-buffer
This would evaluate the buffer in the remote Python instance.
Seems like it should be pretty simple - the Python module listens on a socket, in a thread. It evaluates in the main thread, and returns the repr() of the result (or maybe captures the stdout/stderr, or maybe both). The emacs module would just send text to the socket, waits for a string in response, and displays it in a buffer.
Sounds so simple something like this must exist already... IPython has ipy_vimserver, but this is the wrong way around. There is also swank, while it seems very Lisp-specific, there is a Javascript backend which looks very like what I want... but searching finds almost nothing, other than some vague (possibly true) claims that SLIME doesn't work nicely with non-Lisp languages
In short:
Does a project exist to send code from an emacs buffer to an existing Python process?
If not, how would you recommend I write such a thing (not being very familiar with elisp) - SWANK? IPython's server code? Simple TCP server from scratch?
comint provides most of the infrastructure for stuff like this. There's a bunch of good examples, like this or this
It allows you to run a command, provides things comint-send-string to easily implement send-region type commands.
dbr/remoterepl on Github is a crude proof-of-concept of what I described in the question.
It lacks any kind of polish, but it mostly works - you import the replify.py module in the target interpreter, then evaluate the emacs-remote-repl.el after fixing the stupid hardcoded path to client.py
Doesn't shell-command give you what you are looking for? You could write a wrapper script or adjust the #! and sys.path appropriately.