I have a weird behaviour while managing a third party executable in my python code. Conceptually I have the following code in python:
import subprocess
p = subprocess.Popen([r'c:\path\to\programme.exe', '-d'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
print p.returncode, out, err
And the tool crashes showing in out its traceback, and returning an error code that means "unhandled exception". I have tried with a simple os.system(...) with the same results.
But, here comes the fun part, when I just paste the command in the windows shell, it works perfectly...
C:\> c:\path\to\programme.exe -d
The python interpreter is a 32bit 2.7.2 version.
So... what can be the difference between these two calls that leads to the crash? thanks in advance.
Extra info
I am not quite sure if this helps, but this external tool connects to a database and performs some operations. With some RDBMS it works when called from python code, but when it connects to an Oracle DB, it crashes. So the python code seems to be right, there is just a factor or difference that I don't know.
Well, you really don´t provide much info. I will make a guess based on my own experience dealing with situations like this.
Make sure you´re running the Python app as admin if the third party app require priveleges.
Check there is no problem with the working dir. Meaning, if the program opens some file or in any way it references to some relative path, you must change your working directory when executing from python. See code below for how to do this.
If the programm you're executing is a builtin windows shell app (dir, copy, etc...) consider using shell=True when creating the Popen object. See Popen constructor reference.
Python sets or modifies some environment variable needed/used by your third party application.
Code for changing working directory within the running Python app.
import os
os.chdir('/path_you_need/python/work_from')
You are supposed to use raw string.
p = subprocess.Popen([r'c:\path\to\programme.exe', '-d'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
else you can use \\ instead of using raw string like this:-
p = subprocess.Popen(['c:\\path\\to\\programme.exe', '-d'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Related
So, I'm trying to build a python script that opens Apache Karaf and automatically installs some features. I've tried using Subprocess' Popen but I found it surprisingly complicated. I was suggested to use the pexpect library.
So far I've come up with this:
karaf = "bash /home/karaf_test/bin/karaf"
p = pexpect.spawn(karaf)
p.logfile = p.stdout.buffer
p.expect("root", timeout=10)
p.sendline(feature_install)
Karaf's subshell opens correctly and the feature_install string appears to go through, but the process dies right after without actually executing the command.
There's a screenshot for clarity.
I've looked at the documentation but everything seems to be okay, I have no idea why it won't execute. I have tried running send() instead of sendline() to no avail. If anyone has any advice on this I'd be really thankful.
p.expect("root", timeout=10) expect the pattern not in string.so change your code like this
p.expect(".*#.*()>",timeout=10)
I have a command-line Golang executable on Windows named cnki-downloader.exe (open-sourced here: https://github.com/amyhaber/cnki-downloader). I want to run this executable in Python, and interact with it (get its output, then input something, then get output, and so on)
It's a command-line program, so I thought it's the same as a normal Windows command-line program built by MSVC. My code is like this:
# coding=gbk
from subprocess import Popen, PIPE
p = Popen(["cnki-downloader.exe"], stdin=PIPE, stdout=PIPE)
#p = Popen(["WlanHelper.exe"], stdin=PIPE, stdout=PIPE )
p.stdin.write( 'XXXXXX\n' )
result1 = p.stdout.read() # <---- we never return here
print result1
p.stdin.write( '1\n' )
result2 = p.stdout.read()
print result2
My Python code halts at the Popen call with cnki-downloader.exe parameter. Then I tried a C command-line program built by MSVC (named WlanHelper.exe), it runs well. My script can get the output from the exe.
So I suspect that the command-line mechanism of the Golang executable is different from a native C/C++ program, and it's difficult for other languages (like Python) to call and interact with it.
So I want to know how to interact with a Golang executable on Windows in other languages like Python?. If this is impossible, I can also consider modifying the Golang program's source code (because it's open source). But I hope I won't go that step. Thanks!
NOTE:
If possible, I want to call this Golang executable directly, instead of modifying it into a library and let Python import it. If I have to modify the Golang into a library, why not just remove the interactive manner and make everything into command-line parameters? There's no need to bother writing a Golang library. So please let's assume the Golang program is closed-sourced. I don't think there's no way for Python to call a command-line Golang program. If this is the case, then I think Golang REALLY needs to be improved in the interoperability with other languages.
Looking at the documentation for the subprocess module you are using, it seems they have a warning about deadlocks using stdout.read() and friends:
Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE
and the child process generates enough output to a pipe such that it
blocks waiting for the OS pipe buffer to accept more data. Use
communicate() to avoid that.
So, maybe try using communicate instead?
I created the simple python script using pexpect, created one spwan process using
CurrentCommand = "ssh " + serverRootUserName + "#" + serverHostName
child = pexpect.spawn(CurrentCommand)
Now I am running some command like ls-a or "find /opt/license/ -name '*.xml'"
using code
child.run(mycommand)
it works fine if running from Pycharm but if running from terminal it is not working it is not able to find any file, I think it is looking into my local system.
Can anyone suggest me something. Thanks
As a suggestion, have a look at the paramiko library (or fabric, which uses it, but has a specific purpose), as this is a python interface to ssh. It might make your code a bit better and more resilient against bugs or attacks.
However, I think the issue comes from your use of run.
This function runs the given command; waits for it to finish; then returns all output as a string. STDERR is included in output. If the full path to the command is not given then the path is searched.
What you should look at is 'expect'. I.e. your spawn with spawn then you should use expect to wait for that to get to an appropiate point (such as connected, terminal ready after motd pushed etc (because ouy might have to put a username and password in etc).
Then you want to run sendline to send a line to the program. See the example:
http://pexpect.readthedocs.io/en/latest/overview.html
Hope that helps, and seriously, have a look at paramiko ;)
I am trying to execute a shell command from a Python script.
I have tried the usual suspects, subprocess.call, Popen, os.system etc.
The command i am trying to execute is admittedly rather long (7k characters), since one of the parameters is a json string. From what I've read length should not be the issue here.
The command looks like this:
phantomjs /some/path/visualizer_interface.js -path /another/path/chart.svg -type chart_pie -id 0 -language de -data '{...}'
The visualizer interface is a script i wrote myself, that basically renders a requested chart in a Phantom JS context, grabs the svg and writes it to the specified path. When i execute the exact same command i get a flawless chart, but in Python the subprocess never return, and i don't get any form of feedback, not even on the subprocesses stdout.
with open('/home/max/stdout.txt', 'w') as out:
res = subprocess.Popen(command, shell=True, stdout=out)
res.wait()
I am able to execute other shell commands, so it's not a fundamental Python problem.
Any ideas very much appreciated.
Turns out i had a very slight error in the phantomjs script, that behaved differently depending on where it was executed.
longtime lurker, first time poster.
I know there are quite a few examples throughout the interweb on using subprocess, however I am yet to find one that explains the steps I need to take to birth a new terminal window, and send it commands. There are plenty of posts that give workarounds to launch tools and scripts via a direct subprocess call, but I have not found any that actually answer the original questions of how to send a command properly to terminal.
In my case, I need to open a new terminal window, then send the path to a particular version of an application, and finally the path to the file I wish to open in that application.
I know how to use subprocess to call the applications needed directly (without opening a visible terminal), how to open a new terminal with subprocess, and how to call either the application path OR the file path (have not been able to get both to execute together using --args for open() or any other workaround I have found).
I have been unable to send terminal a command once I have opened it. The following is a simple version of opening a new instance of terminal and sending it ls, which does not work.
from subprocess import Popen, PIPE, STDOUT
p = Popen(['open', '-a', 'Terminal', '-n'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
output = p.communicate(input='ls')
print(output)
This is most likely a trivial issue and I am simply missing something, but I have been unable to find the information or an example that illustrates what I need and I am beginning to get frustrated with it, so I figured I would ask for help.
Any assistance is greatly appreciated! TIA
First, I doubt that command you are trying to run will run at all.
Did you try it in terminal first? open -an Terminal will give you
an error. It probably should be something like open -n
/Applications/Utilities/Terminal.app
Second, #korylprince is right: open itself will create new process
of Terminal and exit. So you are linking pipe with wrong process.
Third, at the moment of passing ls to the stdin that process
doesn't exist already (unless you will pass -W option to the open,
but it certainly will not help due to the 2 problem).
So I see only one opportunity to do this: via AppleScript. You can create an AppleScript string, something like next:
tell application "System Events"
tell process "Terminal"
keystroke "ls"
keystroke return
end tell
end tell
and then run this script via osascript -e '<your_script>' via Popen.
Yes it is quite tricky (I'd say it is a hack)
Yes there probably will be problems with passing multiline string to Popen and with determining correct Terminal window.
But it is possible.
#cody
My response to your answer was too long, therefore I am making an answer to respond:
You are correct, if you enter it the way you offered, it flags an error, and if you put the -n before Terminal it still flags an error. However, if you enter it the way I showed in the first example (-n after Terminal) "open" calls a new instance of app bundle Terminal, even if one is already open.
As for 2-3, that was kind of what my research was leading me to believe, but I was hoping I was wrong or missed something somewhere and someone here could clarify. Sadly, I wasn't mistaken…
I should probably expand on what I am trying to do, as maybe it will help generate a better way to accomplish it via Python.
I have created a tool that launches application files based on the movie, scene, and shot an artist is working on. For some applications, like Nuke and Houdini, opening from Terminal gives you a wealth of information that the artist would be blind to otherwise, so we would like to give the artist the option to launch the file they have chosen in a Terminal. That terminal has to be standalone, and a new instance of Terminal, because the app I have created must persist after the launch in order to open other shots in different applications without making the user routinely open the app.
Parsing the necessary info, building the commands, and launching a new Terminal that launches the desired application were all trivial. Doing the same with the desired file was trivial as well. The issue arises when a particular version of the app is chosen, and I have not been able to pass the newly birthed instance of Terminal with more than a single command (honestly the syntax of my OSX command may be the issue as well, will post further down).
I can get the following two commands to work without issue:
p = Popen(['open', '-a', 'Terminal', '-n', '--args', '/Applications/Nuke6.3v8/Nuke6.3v8.app/Nuke6.3v8'])
p = Popen(['open', '-a', 'Terminal', '-n', '--args', '/Path/to/Nuke/File.nk'])
I cannot get the following to work properly:
p = Popen(['open', '-a', 'Terminal', '-n', '--args', '/Applications/Nuke6.3v8/Nuke6.3v8.app/Nuke6.3v8', '/Path/to/Nuke/File.nk'])
From there my thought was possibly that I should launch the Terminal in the Popen, then pass the commands I needed. That did not work, and then I came here lol
Thanks again for any help! Just knowing that I cannot send to the commands I want to Terminal is saving me a ton of time that would have been spent on continuos frustrated research.