Develop Reference

In Python, with subprocess.Popen, is it possible to pass literal quotes to the command to be run, when Popen's command line parameter is in list form?

In Python, with subprocess.Popen, is it possible to pass literal quotes as an argument, when the command and its parameters are in list form?
I'll explain further what I mean. Some commands can have literal quotes in their arguments e.g. I'm trying "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 1" Some might even require them.
Note that one answer points out that technically it is possible to get Chrome from the command line to launch whatever profile, without passing a literal quote C:\Users\User>"C:\Program Files....\chrome.exe" "--profile-directory=Profile 2"
Nevertheless, i'm asking about passing literal quotes so "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 1"
For simplicity's sake i'll use calc.exe since it's in the path.
import time
import subprocess
proc=subprocess.Popen("calc.exe"+" "+'--profile-directory="Profile 3"')
proc2=subprocess.Popen(["calc.exe",'--profile-directory="Profile 4"'])
time.sleep(3)
proc.wait()
proc2.wait()
Now look at the difference in the command line as visible in task manager or via wmic.
C:\Users\User>wmic process where caption="calc.exe" get commandline | findstr calc
c:\windows\system32\calc.exe --profile-directory="Profile 3"
c:\windows\system32\calc.exe "--profile-directory=\"Profile 4\""
C:\Users\User>
You can see this from the python interpreter
>>> subprocess.Popen(["c:/windows/system32/calc.exe","abc"+'"'+"def"])
...
>>>
>>> subprocess.run("C:\Windows\System32\wbem\WMIC.exe process where caption=\"calc.exe\" get commandline")
...
c:/windows/system32/calc.exe abc\"def
....
>>>
You see it's sticking a backslash in there.
Some comments regarding some suggestions given.
One suggestion assumes that --profile-directory="Profile 1" is the same as --profile-directory "Profile 1" i.e. the assumption that you can replace the = with a space and chrome will work the same. But that isn't the case. So writing subprocess.Popen(["C:\...\chrome.exe", "--profile-directory", "Profile 3"]) will indeed produce "C:\....\chrome.exe" --profile-directory "Profile 1" but that won't work.. it leads chrome to either not open at all, or to open a browser window that offers profiles to click on. The equals sign is necessary.
Another suggestion does
subprocess.Popen(
" ".join(
[
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
'--profile-directory="Person 1"',
]
)
That's not passing a list to Popen, that's passing a list to join, and join is converting it to a string.
Another suggestion is
subprocess.Popen('C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe --profile-directory="Profile 3"')
That's using a string. But as you see from my question, I managed it using a string. I'm asking about using a list.
Another suggestion suggested "--profile-directory='Profile 1'"
If I run chrome with --profile-directory="Profile 1" I get a particular profile that I use sometimes. But if running chrome with "--profile-directory='Profile 1'" Then it doesn't load up that profile. It loads up a blank profile. And going to chrome://version shows "'profile 1'" rather than "profile 1" It's like a different profile, like you may as well have said chrome.exe --profile-directory="profile A". And it also creates directories starting with ' like C:\Users\User\AppData\Local\Google\Chrome\User Data\'Profile 1234' that should be removed.
Another suggestion suggested
subprocess.Popen(
[
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
"--profile-directory=Profile 1",
]
That is interesting because it does "C:\...chrome.exe" "--profile-directory=Profile 1"
And it does infact load chrome with the specified profile. Though it doesn't try to pass literal quotes!
My question asks about when passing literal quotes. It's as if maybe it assumes it's a linux shell and inserts a backslash before it, which in a linux would ensure the quote makes it past the shell and to the program being run. Though i'm not sure it'd even go to the linux shell on linux. e.g. on Windows if I stick a cmd escape character in there like ^ so "--pro^file-directory=Profile 1" then the ^ just gets passed literally. So the cmd shell doesn't intervene.
Why is it that on Windows, subprocess.Popen calls list2cmdline when passed a list, which(and here's the big 'why'), then adds a backslash to any literal double quote within a string, meaning that when using the 'method' of passing a list to to Popen rather than passing a string to it, there is this problem, that you can't pass a literal double quote! So, why does it add that backslash!
I did here a suggestion that looking at argsv in windows vs linux might show a difference. I'm not sure that they would since both implement C.
I don't see why POpen in any situation should behave like Windows needs a backslash inserted more than Linux does.
$ cat ./testargs.py
#!/usr/bin/env python3
import sys
print(sys.argv)
C:\blah>type .\testargsw.py
import sys
print(sys.argv)
in both cases
C:\blah>.\testargsw.py abc\^"def
['C:\\Users\\User\\testargsw.py', 'abc"def']
>.\testargsw.py abc\"def
['C:\\Users\\User\\testargsw.py', 'abc"def']
C:\blah>
$ ./testargs.py abc\"def
['./testargs.py', 'abc"def']
Maybe Windows , specifically the MS C Runtime.. The Code responsible for sending a program's arguments received from the shell, to the main method into argv, is requiring an extra backslash, in a sense because after escaping the double quote, a backslash is then required. (And [here] is put in by the user).
That said, I have heard though that looking at what a shell does on Linux is basically misleading, because a major part of the purpose of the subprocess module is to ensure that you can avoid using a shell entirely.
The script example is perhaps not that relevant(it was just something somebody suggested I check), but my issue is that POpen when passed a list is adding in a backslash as shown by WMIC output(also visible in task manager in the command line column).
added
I spoke to a person that has used python for a long time. They said subprocess was added somewhere in 2.x They still use os.popen(). That takes a string not a list. There have been moves to shift people from os.popen to subprocess.Popen https://docs.python.org/3/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3
An issue with subprocess.Popen in Windows, is it has this list feature, that I think behaves funny.
The easy workaround to that is to not use the list feature of it. To not pass it a list. It's a new feature and not necessary. You can pass it a string.
The question includes an example from the python interpreter and shows how (on windows at least), python adds a backslash to the literal quote.
The person I spoke to pointed out to me two documents that relate to that.
A string is a sequence. A sequence could be a string or list or tuple, though in this document they use the term sequence to just mean list or tuple, and they don't mean string when they say sequence.
https://peps.python.org/pep-0324/
"class Popen(args........."
"args should be a string, or a sequence of program arguments"
It mentions about on unix, shell=True and shell=False
And then it says
"On Windows: the Popen class uses CreateProcess() to execute the child program, which operates on strings. If args is a sequence, it will be converted to a string using the list2cmdline method. Please note that not all MS Windows applications interpret the command line the same way: The list2cmdline is designed for applications using the same rules as the MS C runtime."
Technically a string is a sequence, but that document uses the term sequence in a funny way. But what it means is it's that on Windows, if args is not given a string, but is given a list or tuple, then it uses the list2cmdline method.
Be sure to use print otherwise it uses repr() of the string
>>> print(subprocess.list2cmdline(['a', '"b c"']))
a "\"b c\""
>>>
so that's the function that it's using behind the scenes, on windows, that is inserting a backslash in there.
The guy I spoke to pointed me to this document too
https://bugs.python.org/issue11827
a technical user comments, "list2cmdline() in subprocess is publicly accessible (doesn't begin with underscores) but it isn't documented."
And the point is made there that, let's say they made list2cmdline() private, the fact is that what Popen is doing to the list, in Windows, to get the command line, is undocumented.
So the question then becomes, what is the design trying to do, what is the justification for the insertion of backslash. If a programmer wanted to insert a backslash they could do so. It seeems to me to make more sense then to avoid passing a list to subprocess.POpen.
Windows cmd doesn't even use backslash as an escape character!!!! It uses caret.
C:\Users\User>echo \\
\\
C:\Users\User>echo ^\
\
C:\Users\User>
it's linux eg bash, that uses backslash as an escape character
$ echo \\
\
$
Some executables in windows might want a quote escaped and with a backslash, but then the technical user can do that just as a technical linux user does.
So given that they haven't even documented the "feature" (or bug), how they would justify it, I don't know, but they could start by documenting it!
So I don't understand why passing a list to subprocess.ppopen is adding a backslash?
I could take the list join it with space and pass it as a string to popen, so it won't add a backslash, but as mentioned, that'd be avoiding the question.

In Python, with subprocess.Popen, is it possible to pass literal
quotes as an argument, when the command and its parameters are in list
form?
I think yes, but to elements of argsv, not to the command line..
I'll explain further what I mean. Some commands can have literal
quotes in their arguments e.g. I'm trying "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 1" Some might even require them.
Note that one answer points out that technically it is possible to get
Chrome from the command line to launch whatever profile, without
passing a literal quote C:\Users\User>"C:\Program Files....\chrome.exe" "--profile-directory=Profile 2"
Nevertheless, i'm asking about passing literal quotes so
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 1"
I think there was some confusion, that I might have resolved now.
When running a binary executable in windows(so, an exe, not a bat file) and giving it arguments
People often speak of how linux and windows behave differently..
Windows doesn't separate arguments out, it just dumps stuff to the command line. And there's no concept of space separating out arguments.
Apparently in linux, the command line is parsed by the shell and split into arguments and the arguments are passed by the shell to a POSIX function called execv, which then get passed to main's argsv.
Whereas in windows, there's an aspect of C pertaining to Windows called the MS C Runtime, that parses the command line and splits it into arguments.
So when looking in task manager or from WMIC, at the command line, it's just a dump when it comes to quotes. The quotes aren't some special some literal. But when the MS C Runtime looks at it then the quotes will have that meaning of some special some literal, a special one will not go to argsv, and a literal one will. Usually they are all special, none are literal.
>calc """"""""""^G
>wmic process where caption="calc.exe" get commandline | findstr calc
calc """"""""""G
One we see what's in the command line then one can consider what the arguments are.. what would would go to argsv.
In the case of the two Chrome examples, there's actually no literal quote, in the sense of a literal quote that'd go to argsv. in either example
Example 1
"C:\Program Files(x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 1"
Example 2
"C:\Program Files....\chrome.exe" "--profile-directory=Profile 2"
Looking at "Example 1"
It's not that there's an argument --profile-directory="Profile 1" that has two literal quotes.
The quotes when read by MS CRT(MS C runtime), will just preserve space keeping that "1" as part of the same argument.
So a program that displays argv will show
C:\blah>type w.c
#include <stdio.h>
int main(int argc, char *argv[]) {
int i = 0;
while (argv[i]) {
printf("argv[%d] = %s\n", i, argv[i]);
i++;
}
return 0;
}
C:\blah>w.exe --profile-directory="Profile 1"
argv[0] = w.exe
argv[1] = --profile-directory=Profile 1
C:\blah>
See, no literal quotes there
Infact both different command lines, will produce the same thing going to argv
>w.exe "--profile-directory=Profile 1"
argv[0] = w.exe
argv[1] = --profile-directory=Profile 1
>w.exe --profile-directory="Profile 1"
argv[0] = w.exe
argv[1] = --profile-directory=Profile 1
>
So, how would you even get a literal quote there, well, this link mentions it https://learn.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments?view=msvc-170
the find command might have funny parsing, but if you are using find or a windows implementation of grep.. and wanted to look for a quote.. Then, you'd want one of the argsv parameters to contain a quote.
And that's when you'd put a backslash in there
>w.exe \"
argv[0] = w.exe
argv[1] = "
>
The command line would have a backslash.. So that the argv would have a literal quote.
And that's what passing a list to subprocess.Popen is all about..
You are putting in what you want to be in the argv.
It is then producing the command line that would result in those things being in the argv.
That's why it's inserting a backslash before the double quotes!
So..
When I trying to use list, I was under the mistaken impression that I needed to get literal quotes into the arguments. And I thought I was forming the command line. Both of those premises were wrong.
I suppose if a python script has two options.. one to run an executable for linux users, and one to run an executable for windows users. The command line might be different, because on the one hand, the windows command line (either what's typed or it, what windows dumps as it), and on the other hand, what's typed on the linux command line.
And with using a list where each element is an element of argsv, it lets the library do the work, in the case of windows, using the list2cmd function, to produce whatever the command line would be in order for the MS C Runtime to produce the result you want in argsv.
side note
In the testargs.py file used in the question, it'd not give expected output https://learn.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments?view=msvc-170 for some strings e.g. `"ab\"c" "\\" d` because it is giving the repl representation Adjust it as follows.
user#comp:~# cat ./testargs.py
#!/usr/bin/env python3
import sys
print(sys.argv)
user#comp:~# cat ./testargs_corrected.py
#!/usr/bin/env python3
import sys
# print(sys.argv) would behaves unintended for "ab\"c" "\\" d
# (it's escaping strings for output in an repr)
# so not using print(sys.argv), using below line instead.
# '//' that it shows, means / 'cos '//' is a single character in python
# but much clearer to show it not in the repl form.
for n, a in enumerate(sys.argv): print(f"argv[{n}] = {a}")
user#comp:~# ./testargs.py "ab\"c" "\\" d
['./testargs.py', 'ab"c', '\\', 'd']
user#comp:~#
user#comp:~# ./testargs_corrected.py "ab\"c" "\\" d
argv[0] = ./testargs_corrected.py
argv[1] = ab"c
argv[2] = \
argv[3] = d
user#comp:~#

Should do the trick (might wanna pass shell=True to Popen if it doesn't):
subprocess.Popen(["C:\Program Files (x86)\Google\Chrome\Application\chrome.exe", "--profile-directory", "Profile 3"]);
This is possible because --some-flag="some value" is the same as --some-flag "some value"
ChRomE solution (working, omg):
import subprocess
subprocess.Popen(
[
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
"--profile-directory=Profile 1",
]
)

python subprocess multiple commands with win path [duplicate]

This question already has answers here:
How do you activate an Anaconda environment within a Python Script?
(5 answers)
Closed 2 years ago.
I'm trying to trigger the execution of a python script via conda.
I would then capture the output and report it to command prompt where this is executed.
This is basically the concept in the easiest way
wrap.py - wrapper inted to execute multiple times the following script
import subprocess
def wrap():
while True:
cmd1=r"call C:\\Users\\my_user\\anaconda3\\Scripts\\activate.bat"
cmd2=r"cd C:\\myfolder\\mysubfolder"
cmd3=r"C:\\Users\\my_user\\anaconda3\\python.exe C:\\myfolder\\mysubfolder\\test.py"
proc = subprocess.run([cmd1,cmd2,cmd3])
if __name__ == '__main__':
wrap()
test.py - script that has to be executed
def mytest():
print("success")
if __name__ == '__main__':
mytest()
since mytest prints success once, I would like the output of the wrapper (run on anaconda) to be
(base) C:\myfolder\mysubfolder> python wrap.py
success
success
success
...
I tried with
1 - subprocess.Popen
2 - using shell=True or not
3 - using a list ["first command","second command","third command"] or a single string "first;second;third"
4 - using or removing "r" in front of the string, here the blanks are breaking the game
5 - using single or double ""
6- in my_user the underscore is also resulting in an encoding error
I actually tried to replicate at least 20 different stackoverflow "solutions" but none of them really worked for me. I also read properly the subprocessing page of python documentation, but this didn't help.
Any hint is appreciated, I'm lost.

The syntax subprocess.run([cmd1, cmd2, cmd3]) means run cmd1 with cmd2 and cmd3 as command-line arguments to cmd1. You instead want to execute a single sequence of shell commands; several of the things you are trying to do here require the shell, so you do want shell=True, which dictates the use of a single string as input, rather than a list consisting of a command and its arguments.
(Windows has some finicky processing behind the scenes which makes it not completely impossible to use a list of strings as the first argument with shell=True; but this really isn't portable or obvious. Just don't.)
Regarding the requirement for shell=True here, commands like call and cd (and source or . in Bourne-family shells) are shell built-ins which do not exist as separate binaries; if you don't have shell=True you will simply get "command not found" or your local equivalent. (Under other circumstances, you should generally avoid shell=True when you can. But this is not one of those cases; here, it really is unavoidable without major code changes.)
If your shell is cmd I guess the command might look like
subprocess.run(
r"call C:\Users\my_user\anaconda3\Scripts\activate.bat & C:\Users\my_user\anaconda3\python.exe C:\myfolder\mysubfolder\test.py",
shell=True)
or equivalently the same without r before the string and with all backslashes doubled; the only difference between an r"..." string and a regular "..." string is how the former allows you to put in literal backslashes, whereas the latter requires you to escape them; in the former case, everything in the string is literal, whereas in the latter case, you can use symbolic notations like \n for a newline character, \t for tab, etc.
In Python, it doesn't really matter whether you use single or double quotes; you can switch between them freely, obviously as long as you use the same opening and closing quotes. If you need literal single quotes in the string, use double quotes so you don't have to backslash-escape the literal quote, and vice versa. There's also the triple-quoted string which accepts either quoting character, but is allowed to span multiple lines, i.e. contain literal newlines without quoting them.
If your preferred shell is sh or bash, the same syntax would look like
subprocess.run(r"""
source C:\Users\my_user\anaconda3\Scripts\activate.bat &&
C:\Users\my_user\anaconda3\python.exe C:\myfolder\mysubfolder\test.py""",
shell=True)
I left out the cd in both cases because nothing in your code seems to require the subprocess to run in a particular directory. If you do actually have that requirement, you can add cwd=r'C:\myfolder\mysubfolder' after shell=True to run the entire subprocess in a separate directory.
There are situations where the facilities of subprocess.run() are insufficient, and you need to drop down to bare subprocess.Popen() and do the surrounding plumbing yourself; but this emphatically is not one of those scenarios. You should stay far away from Popen() if you can, especially if your understanding of subprocesses is not very sophisticated.

Creating singly escaped single and double quotes where newlines are also present / python

I need to transform a string containing single and double quotes and newline characters for use in a system call. Consider the following input string:
"""one'two\nthree"four"""
This should be transformed to the following output string:
"$'one\'two\nthree\"four'"
So that it can be submitted as a message argument in a git commit command:
git commit --message=$'one\'two\nthree\"four'
The odd syntax with the leading $ and surrounding single quotes ' is a bash construct described in the bash manpage in the section on quoting (search for QUOTING). I have tried many python functions including str.replace, re.sub, json.dumps, repr, and str.encode('unicode-escape'). But none have yielded the required transformation. It seems that, in this case, python is too high-level for its own good. Suggestions on how to proceed will be very gratefully received.
The system call itself will be made using code like this (omitting the try block for clarity):
import subprocess
call = ["git", "commit", "--message", "'one\'two\nthree\"four'"]
cpi = subprocess.run(call)
I may also use a git library of some description, but I have not done my homework yet for that.
Note: the unnecessary $ character in the last item in the above call list has been removed.

Your wanted command is erroneous at the moment, it is not a valid Python string since a right " is missing, it should be:
"$'one\'two\nthree\"four'"
This is easily constructed by a simple .format call:
>>> "$'{}'".format("""one'two\nthree"four""") == "$'one\'two\nthree\"four'"
True

Python subprocess arguments

For example I am using ffplay and want to run this command -bufsize[:stream_specifier] integer (output,audio,video)
At the moment I have this:
subprocess.call(["ffplay", "-vn", "-nodisp","-bufsize 4096", "%s" % url])
But this says it is invalid.

As JBernardo mentioned in a comment, separate the "-bufsize 4096" argument into two, "-bufsize", "4096". Each argument needs to be separated when subprocess.call is used with shell=False (the default). You can also specify shell=True and give the whole command as a single string, but this is not recommended due to potential security vulnerabilities.
You should not need to use string formatting where you have "%s" % url. If url is a string, pass it directly, otherwise call str(url) to get a string representation.

This is the way to go:
url = 'http://www.whatever.com'
cmd = 'ffplay -vn -nodisp -bufsize 4096 '.split()
subprocess.call(cmd + [str(url)], shell=False)

While using shlex.split() is overkill for your use case, many of the comments seem to be asking about the use of spaces in parameters in cases where a CLI allows you to pass in quoted strings containing spaces (i.e. git commit -m "Commit message here").
Here is a quick python function that can be used to run commands including parameters with spaces:
import shlex, subprocess
def run_command( command ):
subprocess.call(shlex.split(command))

How to escape os.system() calls?

When using os.system() it's often necessary to escape filenames and other arguments passed as parameters to commands. How can I do this? Preferably something that would work on multiple operating systems/shells but in particular for bash.
I'm currently doing the following, but am sure there must be a library function for this, or at least a more elegant/robust/efficient option:
def sh_escape(s):
return s.replace("(","\\(").replace(")","\\)").replace(" ","\\ ")
os.system("cat %s | grep something | sort > %s"
% (sh_escape(in_filename),
sh_escape(out_filename)))
Edit: I've accepted the simple answer of using quotes, don't know why I didn't think of that; I guess because I came from Windows where ' and " behave a little differently.
Regarding security, I understand the concern, but, in this case, I'm interested in a quick and easy solution which os.system() provides, and the source of the strings is either not user-generated or at least entered by a trusted user (me).

shlex.quote() does what you want since python 3.
(Use pipes.quote to support both python 2 and python 3,
though note that pipes has been deprecated since 3.10
and slated for removal in 3.13)

This is what I use:
def shellquote(s):
return "'" + s.replace("'", "'\\''") + "'"
The shell will always accept a quoted filename and remove the surrounding quotes before passing it to the program in question. Notably, this avoids problems with filenames that contain spaces or any other kind of nasty shell metacharacter.
Update: If you are using Python 3.3 or later, use shlex.quote instead of rolling your own.

Perhaps you have a specific reason for using os.system(). But if not you should probably be using the subprocess module. You can specify the pipes directly and avoid using the shell.
The following is from PEP324:
Replacing shell pipe line
-------------------------
output=`dmesg | grep hda`
==>
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

Maybe subprocess.list2cmdline is a better shot?

Note that pipes.quote is actually broken in Python 2.5 and Python 3.1 and not safe to use--It doesn't handle zero-length arguments.
>>> from pipes import quote
>>> args = ['arg1', '', 'arg3']
>>> print 'mycommand %s' % (' '.join(quote(arg) for arg in args))
mycommand arg1 arg3
See Python issue 7476; it has been fixed in Python 2.6 and 3.2 and newer.

I believe that os.system just invokes whatever command shell is configured for the user, so I don't think you can do it in a platform independent way. My command shell could be anything from bash, emacs, ruby, or even quake3. Some of these programs aren't expecting the kind of arguments you are passing to them and even if they did there is no guarantee they do their escaping the same way.

Notice: This is an answer for Python 2.7.x.
According to the source, pipes.quote() is a way to "Reliably quote a string as a single argument for /bin/sh". (Although it is deprecated since version 2.7 and finally exposed publicly in Python 3.3 as the shlex.quote() function.)
On the other hand, subprocess.list2cmdline() is a way to "Translate a sequence of arguments into a command line string, using the same rules as the MS C runtime".
Here we are, the platform independent way of quoting strings for command lines.
import sys
mswindows = (sys.platform == "win32")
if mswindows:
from subprocess import list2cmdline
quote_args = list2cmdline
else:
# POSIX
from pipes import quote
def quote_args(seq):
return ' '.join(quote(arg) for arg in seq)
Usage:
# Quote a single argument
print quote_args(['my argument'])
# Quote multiple arguments
my_args = ['This', 'is', 'my arguments']
print quote_args(my_args)

The function I use is:
def quote_argument(argument):
return '"%s"' % (
argument
.replace('\\', '\\\\')
.replace('"', '\\"')
.replace('$', '\\$')
.replace('`', '\\`')
)
that is: I always enclose the argument in double quotes, and then backslash-quote the only characters special inside double quotes.

On UNIX shells like Bash, you can use shlex.quote in Python 3 to escape special characters that the shell might interpret, like whitespace and the * character:
import os
import shlex
os.system("rm " + shlex.quote(filename))
However, this is not enough for security purposes! You still need to be careful that the command argument is not interpreted in unintended ways. For example, what if the filename is actually a path like ../../etc/passwd? Running os.system("rm " + shlex.quote(filename)) might delete /etc/passwd when you only expected it to delete filenames found in the current directory! The issue here isn't with the shell interpreting special characters, it's that the filename argument isn't interpreted by the rm as a simple filename, it's actually interpreted as a path.
Or what if the valid filename starts with a dash, for example, -f? It's not enough to merely pass the escaped filename, you need to disable options using -- or you need to pass a path that doesn't begin with a dash like ./-f. The issue here isn't with the shell interpreting special characters, it's that the rm command interprets the argument as a filename or a path or an option if it begins with a dash.
Here is a safer implementation:
if os.sep in filename:
raise Exception("Did not expect to find file path separator in file name")
os.system("rm -- " + shlex.quote(filename))

I think these answers are a bad idea for escaping command-line arguments on Windows. Based on the results: people are trying to apply a black-list approach to filtering 'bad' characters, assuming (and hoping) they got them all. Windows is very complex and there could be all manner of characters found in the future that might allow an attacker to hijack command line arguments.
I've already seen some answers neglect to filter basic meta-characters in Windows (like the semi-colon.) The approach I take is far simpler:
Make a list of allowed ASCII characters.
Remove all chars that aren't in that list.
Escape slashes and double-quotes.
Surround entire command with double quotes so the command argument cannot be maliciously broken and commandeered with spaces.
A basic example:
def win_arg_escape(arg, allow_vars=0):
allowed_list = """'"/\\abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-. """
if allow_vars:
allowed_list += "~%$"
# Filter out anything that isn't a
# standard character.
buf = ""
for ch in arg:
if ch in allowed_list:
buf += ch
# Escape all slashes.
buf = buf.replace("\\", "\\\\")
# Escape double quotes.
buf = buf.replace('"', '""')
# Surround entire arg with quotes.
# This avoids spaces breaking a command.
buf = '"%s"' % (buf)
return buf
The function has an option to enable use of environmental variables and other shell variables. Enabling this poses more risk so its disabled by default.