I'm trying to figure out how to invoke 7zip, on Windows, from a Python program.
I'm trying:
stdout = subprocess.run(['C:\\Program Files\\7-Zip\\7z.exe', "a -t7z -mx0", "C:\\Users\\IanWo\\test.7z", "C:\\Users\\IanWo\\test.txt", "C:\\Users\\IanWo\\test2.txt"], shell=True, check=True, capture_output=True, text=True).stdout
print(stdout)
but am getting:
Traceback (most recent call last):
File "D:\Normal Backup\Code\ProcessRetrospectBackups\process.py", line 93, in <module>
stdout = subprocess.run(['C:\\Program Files\\7-Zip\\7z.exe', "a -t7z -mx0", "C:\\Users\\IanWo\\test.7z", "C:\\Users\\IanWo\\test.txt", "C:\\Users\\IanWo\\test2.txt"], shell=True, check=True, capture_output=True, text=True).stdout
File "C:\Users\IanWo\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Program Files\\7-Zip\\7z.exe', 'a -t7z -mx0', 'C:\\Users\\IanWo\\test.7z', 'C:\\Users\\IanWo\\test.txt', 'C:\\Users\\IanWo\\test2.txt']' returned non-zero exit status 7.
>>>
It invokes 7z.exe fine as long as I don't have any arguments. I've tried with and without shell=True with no change.
#ThiefMaster is right. Here's the correct call:
stdout = subprocess.run(['C:\\Program Files\\7-Zip\\7z.exe', "a", "-t7z", "-mx0", "C:\\Users\\IanWo\\test.7z", "C:\\Users\\IanWo\\test.txt", "C:\\Users\\IanWo\\test2.txt"], shell=True, check=True, capture_output=True, text=True).stdout
print(stdout)
Related
I'm trying to run multiple UNIX commands in a python script like this
import subprocess
cmds = ['sleep 3', 'uptime','time ls -l /']
p = subprocess.Popen(cmds,stdout=subprocess.PIPE,shell=True)
while p.poll() is None:
time.sleep(0.5)
tempdata = p.stdout.read()
print(tempdata)
However my output does not contain all output and doesn't seem to run all the commands. Setting shell=False also causes an error.
Traceback (most recent call last):
File "task1.py", line 32, in ?
p = subprocess.Popen(commands,stdout=subprocess.PIPE,stderr=subprocess.PIPE,shell=False)
File "/usr/lib64/python36/subprocess.py", line 550, in __init__
errread, errwrite)
File "/usr/lib64/python36/subprocess.py", line 996, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
When you create a new process, you don't pass it a list of commands to run; rather, you pass it a single command -- either as a string (with shell=True) or as a list of args (with shell=False).
import subprocess
cmds = ['sleep 1', 'uptime', 'ls -l /']
for cmd in cmds:
stdout = subprocess.check_output(cmd, shell=True)
print('\n# {}'.format(cmd))
print(stdout)
If you just want to collect stdout, subprocess.check_output() might be simpler than Popen() -- but either approach will work, depending on what you need to do with the process.
Your problem is 'sleep 3' causes the error you get from the traceback, when I removed that it worked.
To run all for these:
cmds = ['sleep 3', 'uptime','time ls -l /']
You have to call popen for each of them:
for cmd in cmds:
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
while p.poll() is None:
time.sleep(0.5)
output = p.stdout.read()
Or simpler:
for cmd in cmds:
output = subprocess.check_output(cmd, stdout=subprocess.PIPE, shell=True)
Second question: This captures all output written to stdout. To capture also stderr, redirect that into subprocess.PIPE as well.
I am trying to save git fetch output to file through python, using:
subprocess.check_output(["git", "fetch", "origin", ">>", "C:/bitbucket_backup/backup.log", "2>&1"], cwd='C:/bitbucket_backup/loopx')
but I believe there is something missing in subprocess.check_output args because when adding >> C:/bitbucket_backup/backup.log 2>&1 I receive this error:
Traceback (most recent call last):
File "<pyshell#28>", line 1, in <module>
subprocess.check_output(["git", "fetch", "origin", ">>", "C://bitbucket_backup//backup.log", "2>&1"], cwd='C://bitbucket_backup//loopx')
File "C:\Users\fabio\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\Users\fabio\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'fetch', 'origin', '>>', 'C://bitbucket_backup//backup.log', '2>&1']' returned non-zero exit status 128.
Quickfix: enable shell features to handle redirection arguments:
subprocess.check_output(["git", "fetch", "origin", ">>", "C:/bitbucket_backup/backup.log", "2>&1"], cwd='C:/bitbucket_backup/loopx', shell=True)
But that's really dirty as python is able to do that really nicely:
output = subprocess.check_output(["git", "fetch", "origin"], stderr=subprocess.STDOUT, cwd='C:/bitbucket_backup/loopx')
with open("C:/bitbucket_backup/backup.log","ab") as f: # append to file
f.write(output)
That said, if you're to rewrite all git commands in python, maybe you should use a git python API like GitPython for instance.
I am on Mac OS X Yosemite 10.10 and Python 2.7.
If I type the following: du -g -d1 /Users 2> /dev/null in the command line, everything works perfectly.
Now, my goal is to use that command in a python script.
My idea was to use the following:
import subprocess
output = subprocess.check_output(['du', '-g', '-d1', '/Users', '/dev/null'])
But I get this error:
Traceback (most recent call last):
File "./verifications.py", line 1, in <module>
output = subprocess.check_output(['du', '-g', '-d1', '/Users', '/dev/null'])
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 537, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['du', '-g', '-d1', '/Users', '/dev/null']' returned non-zero exit status 1
Also, when I run subprocess.check_output(['du', '-g', '-d1', '/Users', '/dev/null']) in local everything works fine, the error happens when I am logged on a shared iMac using Apple's Shared Screen tool. I have a feeling that the problem might be due to permissions, but I cannot find anything.
For 2>/dev/null, the appropriate way to control redirection of file descriptor 2 with the subprocess.Popen family of calls is stderr=:
# Python 2.x, or 3.0-3.2
output = subprocess.check_output(['du', '-g', '-d1', '/Users'],
stderr=open('/dev/null', 'w'))
...or, with a Python supporting subprocess.DEVNULL:
# Python 3.3 or newer
output = subprocess.check_output(['du', '-g', '-d1', '/Users'],
stderr=subprocess.DEVNULL)
By the way, personally, I'd suggest something more like this:
p = subprocess.Popen(['du', '-g', '-d1', '/Users'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
if p.returncode != 0:
raise Exception(stderr)
...which, instead of sending stderr to /dev/null, keeps it around to use in generating a useful exception in the event that the command fails. (Pick an appropriate subclass of Exception, obviously).
I'm having a strange issue when using subprocess.Popen.communicate(). For background, I want to execute an application from my python script. When I run the program from the command line, I do it like this (UNIX):
$ echo "input text" | /path/to/myapp
From my script, I also want to pipe the input into the application. So, I tried the following. But I get a "broken pipe" error when I try to send the input with communicate():
>>> cmd = ['/path/to/myapp']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
>>> out,err = p.communicate('input text')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.5/subprocess.py", line 670, in communicate
return self._communicate(input)
File "/usr/lib/python2.5/subprocess.py", line 1223, in _communicate
bytes_written = self._write_no_intr(self.stdin.fileno(), buffer(input, input_offset, 512))
File "/usr/lib/python2.5/subprocess.py", line 1003, in _write_no_intr
return os.write(fd, s)
OSError: [Errno 32] Broken pipe
To make matters stranger, if I leave out the input data, I don't get any errors. However, this isn't really a good workaround because the application needs input to work.
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
>>> out,err = p.communicate()
>>> print out
[error from myapp regarding lack of input]
Any idea what I'm missing?
Your observation suggests that myapp is terminating without reading (all of the) input. Not knowing anything about myapp, that's hard to confirm, but consider for example
$ echo 'hello world' | tr 'l' 'L'
heLLo worLd
now...:
>>> cmd = ['/usr/bin/tr']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
>>> out,err = p.communicate('hello world')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/subprocess.py", line 668, in communicate
return self._communicate(input)
File "/usr/lib/python2.5/subprocess.py", line 1218, in _communicate
bytes_written = self._write_no_intr(self.stdin.fileno(), buffer(input, input_offset, 512))
File "/usr/lib/python2.5/subprocess.py", line 997, in _write_no_intr
return os.write(fd, s)
OSError: [Errno 32] Broken pipe
because...:
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
>>> /usr/bin/tr: missing operand
Try `/usr/bin/tr --help' for more information.
and if we fix the bug:
>>> cmd = ['/usr/bin/tr', 'l', 'L']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
>>> out,err = p.communicate('hello world')>>> print out
heLLo worLd
>>> print err
None
...it fixes everything. What happens if you omit the stderr redirection -- do you perchance see any error messages from myapp...?
How do I run this command with subprocess?
I tried:
proc = subprocess.Popen(
'''ECHO bosco|"C:\Program Files\GNU\GnuPG\gpg.exe" --batch --passphrase-fd 0 --output "c:\docume~1\usi\locals~1\temp\tmptlbxka.txt" --decrypt "test.txt.gpg"''',
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
stdout_value, stderr_value = proc.communicate()
but got:
Traceback (most recent call last):
...
File "C:\Python24\lib\subprocess.py", line 542, in __init__
errread, errwrite)
File "C:\Python24\lib\subprocess.py", line 706, in _execute_child
startupinfo)
WindowsError: [Errno 2] The system cannot find the file specified
Things I've noticed:
Running the command on the windows
console works fine.
If I remove the
ECHO bosco| part, it runs fine the
the popen call above. So I think
this problem is related to echo or
|.
First and foremost, you don't actually need a pipe; you are just sending input. You can use subprocess.communicate for that.
Secondly, don't specify the command as a string; that's messy as soon as filenames with spaces are involved.
Thirdly, if you really wanted to execute a piped command, just call the shell. On Windows, I believe it's cmd /c program name arguments | further stuff.
Finally, single back slashes can be dangerous: "\p" is '\\p', but '\n' is a new line. Use os.path.join() or os.sep or, if specified outside python, just a forward slash.
proc = subprocess.Popen(
['C:/Program Files/GNU/GnuPG/gpg.exe',
'--batch', '--passphrase-fd', '0',
'--output ', 'c:/docume~1/usi/locals~1/temp/tmptlbxka.txt',
'--decrypt', 'test.txt.gpg',],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
stdout_value, stderr_value = proc.communicate('bosco')
You were right, the ECHO is the problem. Without the shell=True option the ECHO command cannot be found.
This fails with the error you saw:
subprocess.call(["ECHO", "Ni"])
This passes: prints Ni and a 0
subprocess.call(["ECHO", "Ni"], shell=True)