Develop Reference

How to save output of subprocess.run into a string?

There is a git command that I am using
git log --format=%H 3c2232a5583711aa5f37d0f21014934f67913202
Here the long string at the end is the commit id. This command gives the list of previously commit ids of the branch. The output is like,
3c2232a5583711aa5f37d0f21014934f67913202
9i45e2a5583711aa5f37d0f21014934f679132de
I am trying to issue the same command in python and trying to store the output in a string as the following,
import subprocess
result = subprocess.run(
[
"cd",
"/Users/XYZ/Desktop/gitrepo",
"git",
"log",
"3c2232a5583711aa5f37d0f21014934f67913202",
],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
print(result.stdout.decode("utf-8"), type(result.stdout.decode("utf-8")))
But the output of the print is empty! I tried subprocess.run with ["-ls", "-l"] and it worked well. The git command works on command line but I am not able to capture it in a string. When I print the result alone,
CompletedProcess(args=['cd', '/Users/XYZ/Desktop/gitrepo', 'git', 'log', '3c2232a5583711aa5f37d0f21014934f67913202'], returncode=0, stdout=b'')
How can I save the git command's output in a string? I am issuing two commands in one line. Should I issue the commands separately? If I should then, how can I (a) navigate to git folder and (b) issue git command there?

Your code runs cd "/Users/XYZ/Desktop/gitrepo" "git" "log" "3c2232a5583711aa5f37d0f21014934f67913202" which is probably not what you intended.
The best way is not to interpret changing the working directory as a separate command but as part of the setup of the environment to run the git command. The subprocess module has the keyword argument cwd for that.
If cwd is not None, the function changes the working directory to cwd
before executing the child. In particular, the function looks for
executable (or for the first item in args) relative to cwd if the
executable path is a relative path.
This is only documented for the Popen constructor but the subprocess.run documentation has this paragraph:
The arguments shown above are merely the most common ones, described
below in Frequently Used Arguments (hence the use of keyword-only
notation in the abbreviated signature). The full function signature is
largely the same as that of the Popen constructor - apart from
timeout, input and check, all the arguments to this function are
passed through to that interface.
So you can rewrite your code like this:
import subprocess
result = subprocess.run(
[
"git",
"log",
"3c2232a5583711aa5f37d0f21014934f67913202",
],
cwd="/Users/XYZ/Desktop/gitrepo"
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
print(result.stdout.decode("utf-8"), type(result.stdout.decode("utf-8")))

Execute bash-command with "at" (<<<) via python: syntax error, last token seen

I'm using a radio sender on my RPi to control some light-devices at home. I'm trying to implement a time control and had successfully used the program "at" in the past.
#!/usr/bin/python
import subprocess as sp
##### some code #####
sp.call(['at', varTime, '<<<', '\"sudo', './codesend', '111111\"'])
When I execute the program, i receive the
errmsg:
syntax error. Last token seen: <
Garbled time
This codesnipped works fine with every command by itself (as long every parameter is from type string).
It's neccessary to call "at" in this way: at 18:25 <<< "sudo ./codesend 111111" to hold the command in the queue (viewable in "atq"),
because sudo ./codesend 111111 | at 18:25 just executes the command directly and writes down the execution in "/var/mail/user".
My question ist, how can I avoid the syntax error.
I'm using a lot of other packages in this program, so I have to stay with Python
I hope someone has a solution for this problem or can help to find my mistake.
Many thanks in advance

Preface: Shared Code
Consider the following context to be part of both branches of this answer.
import subprocess as sp
try:
from shlex import quote # Python 3
except ImportError:
from pipes import quote # Python 2
# given the command you want to schedule, as an array...
cmd = ['sudo', './codesend', '111111']
# ...generate a safely shell-escaped string.
cmd_str = ' '.join(quote(x) for x in cmd))
Solution A: Feed Stdin In Python
<<< is shell syntax. It has no meaning to at, and it's completely normal and expected for at to reject it if given as a literal argument.
You don't need to invoke a shell, though -- you can do the same thing directly from native Python:
p = sp.Popen(['at', vartime], stdin=sp.PIPE)
p.communicate(cmd_str)
Solution B: Explicitly Invoke A Shell
Moreover, <<< isn't /bin/sh syntax -- it's an extension honored in bash, ksh, and others; so you can't reliably get it just by adding the shell=True flag (which uses /bin/sh and so guarantees only POSIX-baseline features). If you want it, you need to explicitly invoke a shell with the feature, like so:
bash_script = '''
at "$1" <<<"$2"
'''
sp.call(['bash', '-c', bash_script,
'_', # this is $0 for that script
vartime, # this is its $1
cmd_str, # this is its $2
])
In either case, note that we're using shlex.quote() or pipes.quote() (as appropriate for our Python release) when generating a shell command from an argument list; this is critical to avoid creating shell injection vulnerabilities in our software.

Execute batch file in different directory

I have a a file structure like the following (Windows):
D:\
dir_1\
batch_1.bat
dir_1a\
batch_2.bat
dir_2\
main.py
For the sake of this question, batch_1.bat simply calls batch_2.bat, and looks like:
cd dir_1a
start batch_2.bat %*
Opening batch_1.bat from a command prompt indeed opens batch_2.bat as it's supposed to, and from there on, everything is golden.
Now I want my Python file, D:\dir_2\main.py, to spawn a new process which starts batch_1.bat, which in turn should start batch_2.bat. So I figured the following Python code should work:
import subprocess
subprocess.Popen(['cd "D:/dir_1"', "start batch_1.bat"], shell=True)
This results in "The system cannot find the path specified" being printed to my Python console. (No error is raised, of course.) This is due to the first command. I get the same result even if I cut it down to:
subprocess.Popen(['cd "D:/"'], shell=True)
I also tried starting the batch file directly, like so:
subprocess.Popen("start D:/dir_1/batch_1.bat", shell=True)
For reasons that I don't entirely get, this seems to just open a windows command prompt, in dir_2.
If I forego the start part of this command, then my Python process is going to end up waiting for batch_1 to finish, which I don't want. But it does get a little further:
subprocess.Popen("D:/dir_1/batch_1.bat", shell=True)
This results in batch_1.bat successfully executing... in dir_2, the directory of the Python script, rather than the directory of batch_1.bat, which results in it not being able to find dir_1a\ and hence, batch_2.bat is not executed at all.
I am left highly confused. What am I doing wrong, and what should I be doing instead?

Your question is answered here: Python specify popen working directory via argument
In a nutshell, just pass an optional cwd argument to Popen:
subprocess.Popen(["batch_1.bat"], shell=True, cwd=r'd:\<your path>\dir1')

String parameter using subprocess module

I am using Python to simplify some commands in Maven. I have this script which calls mvn test in debug mode.
from subprocess import call
commands = []
commands.append("mvn")
commands.append("test")
commands.append("-Dmaven.surefire.debug=\"-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE\"")
call(commands)
The problem is with line -Dmaven.surefire.debug which accepts parameter which has to be in quotas and I don't know how to do that correctly. It looks fine when I print this list but when I run the script I get Error translating CommandLine and the debugging line is never executed.

The quotas are only required for the shell executing the command.
If you do the said call directly from the shell, you probably do
mvn test -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE"
With these " signs you (simply spoken) tell the shell to ignore the spaces within.
The program is called with the arguments
mvn
test
-Dmaven.surefire.debug=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE
so
from subprocess import call
commands = []
commands.append("mvn")
commands.append("test")
commands.append("-Dmaven.surefire.debug=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE")
call(commands)
should be the way to go.

Execute cmd-like command in Python

I want to make a Python code that will open a program like cmd would, then export a .txt file from the file menu. The code looks like this for cmd:
c:\ESG\Statsvis.exe \\192.168.100.222\c\ESG\S1-424\2012\06\29\S1-42420120629.dsf /output=C:\Users\jessica.macleod\Desktop\outfile.txt /param=RMS Amplitude
In cmd, the above line does exactly what I want. What would be the equivalent for Python?

See subprocess.Popen, like this:
subprocess.Popen(["/bin/ls", "-l"]
Or, depending on what you want to get as result (stdout, return code), use subprocess.call, subprocess.call_check, or other snippets in this module.

Another way would be os.system().
import os
os.system("c:\\ESG\\Statsvis.exe \\192.16...0629.dsf /output=C:\\...\\outfile.txt ...")

If you want to have exact shell/cmd behavior, then set the shell argument to True in a suprocess.Popen() call. However, from the documentation:
Warning
Invoking the system shell with shell=True can be a security hazard if
combined with untrusted input. See the warning under Frequently Used
Arguments for details.

If you need the output of the command use subprocess:
import subprocess
out = subprocess.check_output("dir c:\ /AD", shell = True)