I'm using argparse (Python 3.2). A parameter mode is defined simply as:
p.add_argument('--mode', dest='mode')
I want to call the program from the command line in such a way that parameter mode is set to an empty string; or at least to a single space ' ' (I can strip whitespace later).
I tried to use (under Windows) program_name --mode "" and program_name --mode " ", but neither worked.
This seems to work for me under OS-X:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--mode')
p = parser.parse_args()
print(p)
and then:
python test.py --mode=
I don't have a windows machine, so I don't know anything about how those operate...
Of course, the other things you mentioned above would work as well on OS-X. The advantage here is that it shouldn't rely on how the shell splits arguments enclosed in quotations which should make it work in more places I would think.
Related
I have command line arguments in a string and I need to split it to feed to argparse.ArgumentParser.parse_args.
I see that the documentation uses string.split() plentifully. However in complex cases, this does not work, such as
--foo "spaces in brakets" --bar escaped\ spaces
Is there a functionality to do that in python?
(A similar question for java was asked here).
This is what shlex.split was created for.
If you're parsing a windows-style command line, then shlex.split doesn't work correctly - calling subprocess functions on the result will not have the same behavior as passing the string directly to the shell.
In that case, the most reliable way to split a string like the command-line arguments to python is... to pass command line arguments to python:
import sys
import subprocess
import shlex
import json # json is an easy way to send arbitrary ascii-safe lists of strings out of python
def shell_split(cmd):
"""
Like `shlex.split`, but uses the Windows splitting syntax when run on Windows.
On windows, this is the inverse of subprocess.list2cmdline
"""
if os.name == 'posix':
return shlex.split(cmd)
else:
# TODO: write a version of this that doesn't invoke a subprocess
if not cmd:
return []
full_cmd = '{} {}'.format(
subprocess.list2cmdline([
sys.executable, '-c',
'import sys, json; print(json.dumps(sys.argv[1:]))'
]), cmd
)
ret = subprocess.check_output(full_cmd).decode()
return json.loads(ret)
One example of how these differ:
# windows does not treat all backslashes as escapes
>>> shell_split(r'C:\Users\me\some_file.txt "file with spaces"', 'file with spaces')
['C:\\Users\\me\\some_file.txt', 'file with spaces']
# posix does
>>> shlex.split(r'C:\Users\me\some_file.txt "file with spaces"')
['C:Usersmesome_file.txt', 'file with spaces']
# non-posix does not mean Windows - this produces extra quotes
>>> shlex.split(r'C:\Users\me\some_file.txt "file with spaces"', posix=False)
['C:\\Users\\me\\some_file.txt', '"file with spaces"']
You could use the split_arg_string helper function from the click package:
import re
def split_arg_string(string):
"""Given an argument string this attempts to split it into small parts."""
rv = []
for match in re.finditer(r"('([^'\\]*(?:\\.[^'\\]*)*)'"
r'|"([^"\\]*(?:\\.[^"\\]*)*)"'
r'|\S+)\s*', string, re.S):
arg = match.group().strip()
if arg[:1] == arg[-1:] and arg[:1] in '"\'':
arg = arg[1:-1].encode('ascii', 'backslashreplace') \
.decode('unicode-escape')
try:
arg = type(string)(arg)
except UnicodeError:
pass
rv.append(arg)
return rv
For example:
>>> print split_arg_string('"this is a test" 1 2 "1 \\" 2"')
['this is a test', '1', '2', '1 " 2']
The click package is starting to dominate for command-arguments parsing, but I don't think it supports parsing arguments from string (only from argv). The helper function above is used only for bash completion.
Edit: I can nothing but recommend to use the shlex.split() as suggested in the answer by #ShadowRanger. The only reason I'm not deleting this answer is because it provides a little bit faster splitting then the full-blown pure-python tokenizer used in shlex (around 3.5x faster for the example above, 5.9us vs 20.5us). However, this shouldn't be a reason to prefer it over shlex.
I want to execute a Linux command through Python. This works in the terminal:
/usr/bin/myprogram --path "/home/myuser"
I've tried this:
path = "/home/myuser"
args = ['/usr/bin/myprogram', '--path ' + path]
proc = subprocess.Popen(args)
And this:
path = "/home/myuser"
args = ['/usr/bin/myprogram', '--path "' + path + '"']
proc = subprocess.Popen(args)
But myprogram does not accept the path formatting. I know that paths behave differently when not executing as shell but I can't get it working. I've also tried single quoting the path instead of double quoting it. Bonus points for a solution that also works on Windows (with a different program path, obviously).
EDIT: Sorry, was writing this out from memory and used backslashes instead of forward slashes. The actual code did use the (correct) forward slashes.
Here's something to try:
import subprocess
import shlex
p = subprocess.Popen(shlex.split("/usr/bin/myprogram --path /home/myuser")
Mind the forward slashes ("/"). From what I read, Python doesn't like backslashes ("\") even when running on Windows (I've never used it on Windows myself).
The problem comes from your string literal, '\usr\bin\myprogram'. According to escaping rules, \b is replaced by \x08, so your executable is not found.
Pun an r in front of your string literals (i.e. r'\usr\bin\myprogram'), or use \\ to represent a backslash (i.e. '\\usr\\bin\\myprogram').
I am trying to develop a script to read pcap file and extract some field from it but using tshark as a subprocess. However i am getting syntax error regarding cmd. Can anyone help me out on this?
def srcDestDport (filename):
cmd = r"tshark -o column.format:"Source","%s", "Destination","%d", "dstport"," %uD"' -r %s"%(filename)
subcmd = cmd.split(' ')
lines = subprocess.Popen(subcmd,stdout=subprocess.PIPE)
return lines
As far as Python is concerned, you appear to be missing some commas in your cmd definition:
cmd = r"tshark -o column.format:"Source","%s", "Destination","%d", "dstport"," %uD"' -r %s"%(filename)
# -- no comma here -^ ----^ ----^ --^
because the first string ends when the first " is encountered at "Source"; a raw string does not preclude you from escaping embedded quotes.
If you wanted to produce a list of arguments, just make it a list directly, saves you interpolating the filename too:
cmd = ["tshark", "-o",
'column.format:"Source","%s","Destination","%d","dstport"," %uD"',
"-r", filename]
Note the single quotes around the 3rd argument to preserve the quotes in the command line argument.
This eliminates the need to split as well and preserves whitespace in the filename.
I'm writing a small script, that should be able to handle multiple files. So I've added that files can be passed comma seperated, and do a arg.split(',') and then handle each one.
Now I've wanted to add asterisk as input possibility like
python myPythonScript.py -i folder/*
If I print the the argument to option -i right when I access it the first time I get
folder/firstFileInFolder.txt
But if I call my script with
python myPythonScript.py -i someFolder/someFile,folder/*
it works just fine. Does anyone have an idea, why python might behave that way?
Try to run this script
import sys
for arg in sys.argv:
print arg
python script.py *
your shell expands the asterisk before python sees it.
As mentioned in the comments, your shell is expanding the asterisk for the non-comma separated case. If you know that the user may specify an asterisk as part of a file name as in your second example, you can have Python do the path expansion by using the glob module.
from glob import glob
glob('*')
code which would allow either the shell or Python to do asterisk expansion may look something like this:
import glob
file_list = []
for pattern in sys.argv[1:]:
file_list.extend(glob.glob(pattern))
In your case, using a comma as a separator would then prevent you from using a comma as part of a filename.
My python program is invoking a script on Windows using check_call:
import subprocess
subprocess.check_call(['my_script.bat', 'foo=bar'])
I expect my_script.bat to receive a single command line argument (foo=bar), but is instead receiving two (foo and bar). In other words, it seems that the equal sign is being converted to whitespace. Everything works as expected on Linux.
What is the correct way to format this string on Windows such that my_script sees a single argument containing the equal sign?
my_script.bat:
echo %1
In my actual application, my_script.bat is scons.bat.
This case tests just fine for me under windows 7, using python2.7
my_script.py
import sys
print sys.argv
python shell
>>> import subprocess
>>> subprocess.check_call('python', 'my_script.py', 'arg1', 'foo=bar')
['myscript.py', 'arg1', 'foo=bar']
0
You may want to verify how your "my_script" is handling the incoming args.
Update
Since you have updated your question to reflect that you are specifically working with windows batch files, then here is the solution: http://geoffair.net/ms/batch.htm
Note: If a semicolon, ';' or an equal sign (=) is used as a command
line argument to a batch file, it is treated as a blank space. For
example, in the following test.bat batch file -
Equal sign is a special character that will be replaced with whitespace. It needs to be quoted:
subprocess.check_call(['my_script.bat', '"foo=bar"'])