This question already has answers here:
Python running in Windows subprocess.call with spaces and parameters
(2 answers)
Closed 4 months ago.
To run a command in python, for Windows, I do:
import subprocess
subprocess.check_output(lsCommand, shell=True)
where lsCommand is a list of strings that make up the bash command. This works, except when it contains some input with spaces in it. For example, copying + changing a name:
To try and do cp "test 123" test123:
lsCommand = ['cp', 'test 123', 'test123']
subprocess.check_output(lsCommand, shell=True)
fails because it thinks I am trying to do cp "test" "123" test123. Error (doing google storage stuff):
python: can't open file 'c:\GSUtil\gsutil.py cp -n gs://folderl/test': [Errno 22] Invalid argument
Then I try
subprocess.check_output('cp "test 123" test123', shell=True)
Same shit. Any ideas?
cp is not an internal command and therefore you don't need shell=True (though you might need to specify a full path to cp.exe).
The internal interface for starting a new subprocess on Windows uses a string i.e., it is up to the specific application how to interpret a command-line. The default MS C runtime rules (imlemented in subprocess.list2cmdline() that is called implicitly if you pass a list on Windows) should work fine in this case:
#!/usr/bin/env python
from subprocess import check_call
check_call(['cp', 'test 123', 'test123'])
If you want to use shell=True then the program that interprets the command line is cmd.exe and you should use its escape rules (e.g., ^ is a meta-character) and pass the command as a string as is (as you see it in the Windows console):
check_call('copy /Y /B "test 123" test123', shell=True)
Obviously, you don't need to start an external process, to copy a file in Python:
import shutil
shutil.copy('test 123', 'test123')
for Ubuntu:
subprocess.check_output(['list', 'of', 'commands with spaces'])
for Windows:
subprocess.check_output('single command "string with spaces"')
Thanks for info that I don't need shell=True.
Related
I have a Python script that needs to execute an external program, but for some reason fails.
If I have the following script:
import os;
os.system("C:\\Temp\\a b c\\Notepad.exe");
raw_input();
Then it fails with the following error:
'C:\Temp\a' is not recognized as an internal or external command, operable program or batch file.
If I escape the program with quotes:
import os;
os.system('"C:\\Temp\\a b c\\Notepad.exe"');
raw_input();
Then it works. However, if I add a parameter, it stops working again:
import os;
os.system('"C:\\Temp\\a b c\\Notepad.exe" "C:\\test.txt"');
raw_input();
What is the right way to execute a program and wait for it to complete? I do not need to read output from it, as it is a visual program that does a job and then just exits, but I need to wait for it to complete.
Also note, moving the program to a non-spaced path is not an option either.
This does not work either:
import os;
os.system("'C:\\Temp\\a b c\\Notepad.exe'");
raw_input();
Note the swapped single/double quotes.
With or without a parameter to Notepad here, it fails with the error message
The filename, directory name, or volume label syntax is incorrect.
subprocess.call will avoid problems with having to deal with quoting conventions of various shells. It accepts a list, rather than a string, so arguments are more easily delimited. i.e.
import subprocess
subprocess.call(['C:\\Temp\\a b c\\Notepad.exe', 'C:\\test.txt'])
Here's a different way of doing it.
If you're using Windows the following acts like double-clicking the file in Explorer, or giving the file name as an argument to the DOS "start" command: the file is opened with whatever application (if any) its extension is associated with.
filepath = 'textfile.txt'
import os
os.startfile(filepath)
Example:
import os
os.startfile('textfile.txt')
This will open textfile.txt with Notepad if Notepad is associated with .txt files.
The outermost quotes are consumed by Python itself, and the Windows shell doesn't see it. As mentioned above, Windows only understands double-quotes.
Python will convert forward-slashed to backslashes on Windows, so you can use
os.system('"C://Temp/a b c/Notepad.exe"')
The ' is consumed by Python, which then passes "C://Temp/a b c/Notepad.exe" (as a Windows path, no double-backslashes needed) to CMD.EXE
At least in Windows 7 and Python 3.1, os.system in Windows wants the command line double-quoted if there are spaces in path to the command. For example:
TheCommand = '\"\"C:\\Temp\\a b c\\Notepad.exe\"\"'
os.system(TheCommand)
A real-world example that was stumping me was cloning a drive in VirtualBox. The subprocess.call solution above didn't work because of some access rights issue, but when I double-quoted the command, os.system became happy:
TheCommand = '\"\"C:\\Program Files\\Sun\\VirtualBox\\VBoxManage.exe\" ' \
+ ' clonehd \"' + OrigFile + '\" \"' + NewFile + '\"\"'
os.system(TheCommand)
For python >= 3.5 subprocess.run should be used in place of subprocess.call
https://docs.python.org/3/library/subprocess.html#older-high-level-api
import subprocess
subprocess.run(['notepad.exe', 'test.txt'])
import win32api # if active state python is installed or install pywin32 package seperately
try: win32api.WinExec('NOTEPAD.exe') # Works seamlessly
except: pass
I suspect it's the same problem as when you use shortcuts in Windows... Try this:
import os;
os.system("\"C:\\Temp\\a b c\\Notepad.exe\" C:\\test.txt");
For Python 3.7, use subprocess.call. Use raw string to simplify the Windows paths:
import subprocess
subprocess.call([r'C:\Temp\Example\Notepad.exe', 'C:\test.txt'])
Suppose we want to run your Django web server (in Linux) that there is space between your path (path='/home/<you>/<first-path-section> <second-path-section>'), so do the following:
import subprocess
args = ['{}/manage.py'.format('/home/<you>/<first-path-section> <second-path-section>'), 'runserver']
res = subprocess.Popen(args, stdout=subprocess.PIPE)
output, error_ = res.communicate()
if not error_:
print(output)
else:
print(error_)
[Note]:
Do not forget accessing permission: chmod 755 -R <'yor path'>
manage.py is exceutable: chmod +x manage.py
No need for sub-process, It can be simply achieved by
GitPath="C:\\Program Files\\Git\\git-bash.exe"# Application File Path in mycase its GITBASH
os.startfile(GitPath)
I have a Python script that needs to execute an external program, but for some reason fails.
If I have the following script:
import os;
os.system("C:\\Temp\\a b c\\Notepad.exe");
raw_input();
Then it fails with the following error:
'C:\Temp\a' is not recognized as an internal or external command, operable program or batch file.
If I escape the program with quotes:
import os;
os.system('"C:\\Temp\\a b c\\Notepad.exe"');
raw_input();
Then it works. However, if I add a parameter, it stops working again:
import os;
os.system('"C:\\Temp\\a b c\\Notepad.exe" "C:\\test.txt"');
raw_input();
What is the right way to execute a program and wait for it to complete? I do not need to read output from it, as it is a visual program that does a job and then just exits, but I need to wait for it to complete.
Also note, moving the program to a non-spaced path is not an option either.
This does not work either:
import os;
os.system("'C:\\Temp\\a b c\\Notepad.exe'");
raw_input();
Note the swapped single/double quotes.
With or without a parameter to Notepad here, it fails with the error message
The filename, directory name, or volume label syntax is incorrect.
subprocess.call will avoid problems with having to deal with quoting conventions of various shells. It accepts a list, rather than a string, so arguments are more easily delimited. i.e.
import subprocess
subprocess.call(['C:\\Temp\\a b c\\Notepad.exe', 'C:\\test.txt'])
Here's a different way of doing it.
If you're using Windows the following acts like double-clicking the file in Explorer, or giving the file name as an argument to the DOS "start" command: the file is opened with whatever application (if any) its extension is associated with.
filepath = 'textfile.txt'
import os
os.startfile(filepath)
Example:
import os
os.startfile('textfile.txt')
This will open textfile.txt with Notepad if Notepad is associated with .txt files.
The outermost quotes are consumed by Python itself, and the Windows shell doesn't see it. As mentioned above, Windows only understands double-quotes.
Python will convert forward-slashed to backslashes on Windows, so you can use
os.system('"C://Temp/a b c/Notepad.exe"')
The ' is consumed by Python, which then passes "C://Temp/a b c/Notepad.exe" (as a Windows path, no double-backslashes needed) to CMD.EXE
At least in Windows 7 and Python 3.1, os.system in Windows wants the command line double-quoted if there are spaces in path to the command. For example:
TheCommand = '\"\"C:\\Temp\\a b c\\Notepad.exe\"\"'
os.system(TheCommand)
A real-world example that was stumping me was cloning a drive in VirtualBox. The subprocess.call solution above didn't work because of some access rights issue, but when I double-quoted the command, os.system became happy:
TheCommand = '\"\"C:\\Program Files\\Sun\\VirtualBox\\VBoxManage.exe\" ' \
+ ' clonehd \"' + OrigFile + '\" \"' + NewFile + '\"\"'
os.system(TheCommand)
For python >= 3.5 subprocess.run should be used in place of subprocess.call
https://docs.python.org/3/library/subprocess.html#older-high-level-api
import subprocess
subprocess.run(['notepad.exe', 'test.txt'])
import win32api # if active state python is installed or install pywin32 package seperately
try: win32api.WinExec('NOTEPAD.exe') # Works seamlessly
except: pass
I suspect it's the same problem as when you use shortcuts in Windows... Try this:
import os;
os.system("\"C:\\Temp\\a b c\\Notepad.exe\" C:\\test.txt");
For Python 3.7, use subprocess.call. Use raw string to simplify the Windows paths:
import subprocess
subprocess.call([r'C:\Temp\Example\Notepad.exe', 'C:\test.txt'])
Suppose we want to run your Django web server (in Linux) that there is space between your path (path='/home/<you>/<first-path-section> <second-path-section>'), so do the following:
import subprocess
args = ['{}/manage.py'.format('/home/<you>/<first-path-section> <second-path-section>'), 'runserver']
res = subprocess.Popen(args, stdout=subprocess.PIPE)
output, error_ = res.communicate()
if not error_:
print(output)
else:
print(error_)
[Note]:
Do not forget accessing permission: chmod 755 -R <'yor path'>
manage.py is exceutable: chmod +x manage.py
No need for sub-process, It can be simply achieved by
GitPath="C:\\Program Files\\Git\\git-bash.exe"# Application File Path in mycase its GITBASH
os.startfile(GitPath)
I try to setup a cron job to rsync remote files (contains root-level files) into my local server, if I run the command in shell, it works. But if I run this in Python, I got into strange command not found error:
This works if run it in a shell:
rsync -ave ssh --rsync-path='sudo rsync' --delete root#192.168.1.100:/tmp/test2 ./test
But this Python script doesn't:
#!/usr/bin/python
from subprocess import call
....
for src_dir in backup_list:
call(["rsync", "-ave", "ssh", "--rsync-path='sudo rsync'", "--delete", src_host+src_dir, dst_dir])
It fails with:
local server:$ backup.py
bash: sudo rsync: command not found
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: remote command not found (code 127) at io.c(226) [Receiver=3.1.0]
...
It is most likely a spacing error or something small, the way I debug commands is to make sure to prints out. OS.system is a great alternative thats easier although subprocess is better. I am not around my computer to test it but you can either set your subprocess like that, or use this example. This is assuming your on Linux or Mac.
import os
cmd = ('rsync -ave --delete root' +str(src_host) + str(src_directory) + '' + str(dst_dir)) #variable you can call anytime
os.system(cmd) # actually performs the command
print x # how to test and make sure
Quotes around an argument with spaces like you have in "--rsync-path='sudo rsync'" are needed when the shell splits up a long string into arguments, to avoid treating rsync as a separate argument. In your call(), you're providing the individual arguments, so that splitting of a string into arguments is not performed. With your code as-is, the quotes end up as part of the argument passed to rsync. Just drop them. Here's a working example of the list passed to a call() for a very similar rsync invocation:
['rsync',
'-arvz',
'-delete',
'-e',
'ssh',
'--rsync-path=sudo rsync',
'192.168.0.17:/remote/directory/',
'/local/directory/']
I have been facing the same issue:
This piece of code work for me…
join the command while passing to call or Popen and add shell=True.
from subprocess import call
for src_dir in backup_list:
call( " ".join(["rsync", "-ave", "ssh", "--rsync-path='sudo rsync'", "--delete", src_host+src_dir, dst_dir]) , shell=True)
I was trying to use subprocess calls to perform a copy operation (code below):
import subprocess
pr1 = subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = True)
and I got an error saying:
cp: missing file operand
Try `cp --help' for more information.
When I try with shell=False , I get
cp: cannot stat `./testdir1/*': No such file or directory
How do I get around this problem?
I'm using RedHat Linux GNOME Deskop version 2.16.0 and bash shell and Python 2.6
P.S. I read the question posted in Problems with issuing cp command with Popen in Python, and it suggested using shell = True option, which is not working for me as I mentioned :(
When using shell=True, pass a string, not a list to subprocess.call:
subprocess.call('cp -r ./testdir1/* ./testdir2/', shell=True)
The docs say:
On Unix with shell=True, the shell defaults to /bin/sh. If args is a
string, the string specifies the command to execute through the shell.
This means that the string must be formatted exactly as it would be
when typed at the shell prompt. This includes, for example, quoting or
backslash escaping filenames with spaces in them. If args is a
sequence, the first item specifies the command string, and any
additional items will be treated as additional arguments to the shell
itself.
So (on Unix), when a list is passed to subprocess.Popen (or subprocess.call), the first element of the list is interpreted as the command, all the other elements in the list are interpreted as arguments for the shell. Since in your case you do not need to pass arguments to the shell, you can just pass a string as the first argument.
This is an old thread now, but I was just having the same problem.
The problem you were having with this call:
subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = False)
was that each of the parameters after the first one are quoted. So to the shell sees the command like this:
cp '-r' './testdir1/*' './testdir2/'
The problem with that is the wildcard character (*). The filesystem looks for a file with the literal name '*' in the testdir1 directory, which of course, is not there.
The solution is to make the call like the selected answer using the shell = True option and none of the parameters quoted.
I know that the option of shell=True may be tempting but it's always inadvisable due to security issues. Instead, you can use a combination of the subprocess and glob modules.
For Python 3.5 or higher:
import subprocess
import glob
subprocess.run(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])
For Python 3.4 or lower:
import subprocess
import glob
subprocess.call(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])
This question already has answers here:
How to redirect output with subprocess in Python?
(6 answers)
Closed 7 years ago.
In my python script,
I am trying to run a windows program that prints output.
But I would like to redirect that output to a text file.
I tried
command = 'program' + arg1 + ' > temp.txt'
subprocess.call(command)
Where program is my program name and arg1 is argument it takes.
but it does not redirect the output to the text file
It just prints that on the screen.
Can anyone help me how to do this?
Thank you!
Pass a file object to the stdout parameter of subprocess.call():
with open('myoutfilename', 'w') as myoutfile:
subprocess.call(cmd, stdout=myoutfile)
You can use shell=True in subprocess.call
However, a (much) better way to do this would be:
command = ['program',arg1]
with open('temp.txt','w') as fout:
subprocess.call(command,stdout=fout)
This removes the shell from the whole thing making it more system independent, and it also makes your program safe from "shell injection" attacks (consider arg1='argument; rm -rf ~' or whatever the windows equivalent is).
The context manager (with statement) is a good idea as it guarantees that your file object is properly flushed and closed when you leave the "context".
Note that it is important that if you're not using shell=True to a subprocess.Popen (or similar) class, you should pass the arguments as a list, not a string. Your code will be more robust that way. If you want to use a string, python provides a convenience function shlex.split to split a string into arguments the same way your shell would. e.g.:
import subprocess
import shlex
with open('temp.txt','w') as fout:
cmd = shlex.split('command argument1 argument2 "quoted argument3"'
#cmd = ['command', 'argument1', 'argument2', 'quoted argument3']
subprocess.call(cmd,stdout=fout)