I'm getting a slightly weird result from calling subprocess.Popen that I suspect has a lot to do with me being brand-new to Python.
args = [ 'cscript', '%USERPROFILE%\\tools\\jslint.js','%USERPROFILE%\\tools\\jslint.js' ]
p = Popen(args, stdout=PIPE, shell=True).communicate()[0]
Results in output like the following (the trailing double \r\n is there in case it's important)
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.\r\n\r\n
If I run that command from an interactive Python shell it looks like this
>>> args = ['cscript', '%USERPROFILE%\\tools\\jslint.js', '%USERPROFILE%\\tools\jslint.js']
>>> p = subprocess.Popen(args, stdout=subprocess.PIPE, shell=True).communicate()[0]
Lint at line 5631 character 17: Unexpected /*member 'OpenTextFile'.
f = fso.OpenTextFile(WScript.Arguments(0), 1),
...
Lint at line 5649 character 17: Unexpected /*member 'Quit'.
WScript.Quit(1);
So there's all the output I really care about, but if I dump the value of the "p" variable I just set up...
>>> p
'Microsoft (R) Windows Script Host Version 5.8\r\nCopyright (C) Microsoft Corpor
ation. All rights reserved.\r\n\r\n'
>>>
Where'd all that data I want end up going? It definitely didn't end up in "p". Looks like it's going to stdout, but I didn't I explictly tell it not to do that?
I'm running this on Windows 7 x64 with Python 2.6.6
Is it going to stderr? Try redirecting:
p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True).communicate()[0]
It's probably going to stderr, as SimonJ suggested.
Also, the docs say not to use shell=True in Windows for your case:
The executable argument specifies the
program to execute. It is very seldom
needed: Usually, the program to
execute is defined by the args
argument. If shell=True, the
executable argument specifies which
shell to use. On Unix, the default
shell is /bin/sh. On Windows, the
default shell is specified by the
COMSPEC environment variable. The only
reason you would need to specify
shell=True on Windows is where the
command you wish to execute is
actually built in to the shell, eg
dir, copy. You don’t need shell=True
to run a batch file, nor to run a
console-based executable.
Later: oh wait. Are you using the shell to get those environment variables expanded? Okay, I take it back: you do need the shell=True.
Related
If I want to run my Python 3.8 program in Windows 10, I have to firstly load below commands in PowerShell:
PS> cmd
CMD> call [somePath]/environmentSettingScript.bat
CMD> set Path=%Path%;D:[someOtherPath]\win64
After that, I am able to run Python in the same window (still CMD running in PS) and load my third party libraries.
CMD> python
>>> from thirdPartyLibrary import something
or
CMD> python programThatUseThirdPartyLibrary.py
Of course, it works also, if I make call and set procedures from first snippet directly in CMD. Nevertheless, I don't like such kind of solutions. I want to automatize this procedure, in programThatUseThirdPartyLibrary.py code directly with Spyder IDE usage. To do this I try write something like that:
import subprocess
pipe = subprocess.Popen(r"cmd \c call [somePath]/environmentSettingScript.bat \c set Path=%Path%;D:[someOtherPath]\win64",
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
encoding='utf8',
shell=True)
stdout, stderr = pipe.communicate()
print("No stderr" if pipe.returncode == 0 else "Some stderr")
from thirdPartyLibrary import something
Unfortunately, stdout shows only the information about loading first line of cmd command:
Microsoft Windows [Version 10.0.17763.1697]
(c) 2018 Microsoft Corporation. All rights reserved.
D:\[workDir]>
and it looks like, that the rest of commands stuck in the pipe. I am quite confused, because print() and stderr shows that, everything goes ok - but import at the end of the script fails with error
ModuleNotFoundError: No module named thirdPartyLibrary
I'm trying to write a script that opens a new terminal then runs a separate python script from that terminal.
I've tried:
os.system("gnome-terminal 'python f.py'")
and
p = Popen("/usr/bin/gnome-terminal", stdin=PIPE)
p.communicate("python f.py")
but both methods only open a new terminal and do not run f.py. How would I go about opening the terminal AND running a separate script?
Edit:
I would like to open a new terminal window because f.py is a simply server that is running serve_forever(). I'd like the original terminal window to stay "free" to run other commands.
Like most terminals, gnome terminal needs options to execute commands:
gnome-terminal [-e, --command=STRING] [-x, --execute]
You probably need to add -x option:
x, --execute
Execute the remainder of the command line inside the terminal.
so:
os.system("gnome-terminal -x python f.py")
That would not run your process in the background unless you add & to your command line BTW.
The communicate attempt would need a newline for your input but should work too, but complex processes like terminals don't "like" being redirected. It seems like using an interactive tool backwards.
And again, that would block until termination. What could work would be to use p.stdin.write("python f.py\n") to give control to the python script. But in that case it's unlikely to work.
So it seems that you don't even need python do to what you want. You just need to run
python f.py &
in a shell.
As of GNOME Terminal 3.24.2 Using VTE version 0.48.4 +GNUTLS -PCRE2
Option “-x” is deprecated and might be removed in a later version of gnome-terminal.
Use “-- ” to terminate the options and put the command line to execute after it.
Thus the preferred syntax appears to be
gnome-terminal -- echo hello
rather than
gnome-terminal -x echo hello
Here is a complete example of how you would call a executable python file with subprocess.call Using argparse to properly parse the input.
the target process will print your given input.
Your python file to be called:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--file", help="Just A test", dest='myfile')
args = parser.parse_args()
print args.myfile
Your calling python file:
from subprocess import call
#call(["python","/users/dev/python/sandboxArgParse.py", "--file", "abcd.txt"])
call(["gnome-terminal", "-e", "python /users/dev/python/sandboxArgParse.py --file abcd.txt"])
Just for information:
You probably don't need python calling another python script to run a terminal window with a process, but could do as follows:
gnome-terminal -e "python /yourfile.py -f yourTestfile.txt"
The following code will open a new terminal and execute the process:
process = subprocess.Popen(
"sudo gnome-terminal -x python f.py",
stdout=subprocess.PIPE,
stderr=None,
shell=True
)
I am running a uWS server with this.In my case Popen didn't help(Even though it run the executable, still it couldn't communicate with a client -: socket connection is broken).This is working.Also now they recommends to use "--" instead of "-e".
subprocess.call(['gnome-terminal', "--", "python3", "server_deployment.py"])
#server_deployment.py
def run():
execution_cmd = "./my_executable arg1 arg2 dll_1 dll_2"
os.system(execution_cmd)
run()
I am trying to run the following subprocess.Popen() command in python 2.6
output = subprocess.Popen("psexec -accepteula \\machineName -u username -p password cmd.exe /C fsutil fsinfo drives", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
print(output.communicate())
But I get the following result with output.communicate()
('\r\n', '\r\nPsExec v2.11 - Execute processes remotely\r\nCopyright (C) 2001-2014 Mark Russinovich\r\nSysinternals - www.sysinternals.com\r\n\r\nConnecting to machineNameHere...\r\r\rStarting PSEXESVC service on machineNameHere...\r\r\rConnecting with PsExec service on machineNameHere...\r\r\rStarting cmd.exe on machineNameHere...\r\r\r\r\ncmd.exe exited on machineNameHere with error code 0.\r\n')
When I run the same psexec command from cmd line in windows, I get the correct output.
PsExec v2.11 - Execute processes remotely
Copyright (C) 2001-2014 Mark Russinovich
Sysinternals - www.sysinternals.com
**Drives: A:\ C:\ D:\**
cmd.exe exited on machineNameHere with error code 0.
I am looking for the output **Drives: A:\ C:\ D:** even while running the psexec command using subprocess.Popen(). Any way I can do it?
Now I narrowed down the issue by running different commands like dir and echo "test". The issue seems to be that Popen is reading only first line into stdout and not the complete output. Any suggestions on how to fix this?
(expanding on John Gordon's comment on the question)
you are trying to run this command through a shell:
"psexec -accepteula \\machineName -u username -p password cmd.exe /C fsutil fsinfo drives"
notice that you have double backslashes within the command (\\machineName). The backslash happens to be an escape character in string literals (used to escape characters that otherwise have a special meaning). Therefore, \\machineName is getting translated to \machineName before it is passed to the spawned process.
here are 2 ways to handle this:
1) prepend an escape character (another backslash) before each backslash:
"psexec -accepteula \\\\machineName"
2) add an 'r' before the string literal, which makes it a a raw string, and will not interpret the backlsashes as escape chars:
r"psexec -accepteula \\machineName"
Here is an example in the python interpreter that should make it clear. Notice the output that gets printed:
>>> print("\\machineName")
\machineName
>>> print("\\\\machineName")
\\machineName
>>> print(r"\\machineName")
\\machineName
For whatever reason,
from subprocess import Popen, PIPE, STDOUT
handle = Popen('echo %CD%', shell=True, stdout=PIPE, stderr=STDOUT, stdin=PIPE, cwd=r'C:\test')
print 'STDOUT::',handle.stdout.readline().strip()
Gives me:
STDOUT:: C:\Documents and Settings\myusername\Desktop
Windows XP (for whatever reason)
Python 2.6.6
Does this perhaps have any similarities to how windows executes commands in row such as:
C:\> cd "C:\test\" && echo %CD%
C:\
Which, is wrong.
If you are just trying to change the current working directory, you could use:
os.chdir('c:\\test')
Note the double backslash as the backslash is a special character in Python that needs to be escaped.
Also, to check your current working directory, you can use:
os.getcwd()
Tested with Win7 and Python 2.6 and 2.7, works as expected (STDOUT:: C:\test).
So this seems to be related to WinXP.
I am writing a small python script that needs to execute git commands from inside a given directory
The code is as follows:
import subprocess, os
pr = subprocess.Popen(['/usr/bin/git', 'status'],
cwd=os.path.dirname('/path/to/dir/'),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True)
(out, error) = pr.communicate()
print out
But it shows git usage as the output.
If the command doesn't involve git for eg. ['ls'] then it shows the correct output.
Is there anything I am missing ?
python version - 2.6.6
Thanks.
subprocess.Popen:
On Unix, with shell=True: […] If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.
You don't want shell=True and also a list of arguments. Set shell=False.