For some reason, I have non standard command line options for my program. My program takes in a long option also with a single '-'. For example, a valid command line option would be '-f' / '-foo'. Both short and long options need to have an argument separated by space or an '='.
I am trying to parse this using the optparse, but I understand that optparse does not support non GNU-standard options. Is there a way to tweak optparse to do this?
Here's a mildly hackish way to do what you need.
Subclass Option and OptionParser and patch some of the methods:
from optparse import Option, OptionError, OptionParser
class MyOption(Option):
def _set_opt_strings(self, opts):
for opt in opts:
if len(opt) < 2:
raise OptionError(
"invalid option string %r: "
"must be at least two characters long" % opt, self)
elif len(opt) == 2:
self._short_opts.append(opt)
else:
self._long_opts.append(opt)
class MyOptionParser(OptionParser):
def _process_args(self, largs, rargs, values):
while rargs:
arg = rargs[0]
if arg == "--":
del rargs[0]
return
elif arg[0:2] == "--":
self._process_long_opt(rargs, values)
elif arg[:1] == "-" and len(arg) > 1:
if len(arg) > 2:
self._process_long_opt(rargs, values)
else:
self._process_short_opts(rargs, values)
elif self.allow_interspersed_args:
largs.append(arg)
del rargs[0]
else:
return
Now you can do
parser = MyOptionParser()
parser.add_option(MyOption("-f", "-file", dest="filename",
help="write report to FILE", metavar="FILE"))
parser.add_option(MyOption("-q", "-quiet",
action="store_false", dest="verbose", default=True,
help="don't print status messages to stdout"))
With this, parser will accept -file as an option (and will not accept e.g. -fq).
From the optparse documentation
option:
an argument used to supply extra information to guide or customize the execution of a program. There are many different syntaxes for options; the traditional Unix syntax is a hyphen (“-”) followed by a single letter, e.g. -x or -F. Also, traditional Unix syntax allows multiple options to be merged into a single argument, e.g. -x -F is equivalent to -xF. The GNU project introduced -- followed by a series of hyphen-separated words, e.g. --file or --dry-run. These are the only two option syntaxes provided by optparse.
(emphasis added)
So no, you cannot specify other ways of handling arguments with optparse. You can, however, parse the arguments yourself by using argv from the sys module.
This is going to be more work, but it might look something like:
from sys import argv
for arg in argv:
if arg.startswith("-") or arg.startswith("--"):
# Parse the argument
I don't think there's any way to tweak optparse (though I don't know for certain), but getopt is your alternative that will handle C style command-line options.
Related
I want to emulate the behavior of most command-line utilities, where optional arguments can be put anywhere in the command line, including between positional arguments, such as in this mkdir example:
mkdir before --mode 077 after
In this case, we know that --mode takes exactly 1 argument, so before and after are both considered positional arguments. The optional part, --mode 077, can really be put anywhere in the command line.
However, with argparse, the following code does not work with this example:
# mkdir.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--mode', nargs=1)
parser.add_argument('dirs', nargs='*')
args = parser.parse_args()
Running ./mkdir.py before --mode 077 after results in:
mkdir.py: error: unrecognized arguments: after
How can I get argparse to accept an optional argument (with a fixed, known number of items) between positional ones?
Starting from Python 3.7, it seems argparse now supports this kind of Unix-style parsing:
Intermixed parsing
ArgumentParser.parse_intermixed_args(args=None, namespace=None)
A number of Unix commands allow the user to intermix optional arguments with positional arguments. The parse_intermixed_args() and parse_known_intermixed_args() methods support this parsing style.
There is a caveat, but for "simple" options, it does not affect them:
These parsers do not support all the argparse features, and will raise exceptions if unsupported features are used. In particular, subparsers, argparse.REMAINDER, and mutually exclusive groups that include both optionals and positionals are not supported.
(I posted this FAQ-style question after spending 1 hour trying to understand why the examples in the Python argparse documentation didn't seem to include it, and only by chance found a somewhat unrelated question which contained the mention to this "intermixed" function in a comment, which I am unable to find again to cite it properly.)
I'm not familiar with argparse so I would write my own code to handle arguments.
import sys
#the first argument is the name of the program, so we skip that
args = sys.argv[1:]
#just for debugging purposes:
argsLen = len(args)
print(args, argsLen)
#Create a list that will contain all of the indeces that you will have parsed through
completedIndeces = []
i = 0
#add the index to the completed indeces list.
def goToNextIndex(j):
global i
completedIndeces.append(j)
i += 1
def main():
global i
###core logic example
#Go through each item in args and decide what to do based on the arugments passed in
for argu in args:
if i in completedIndeces:
print("breaking out")
#increment i and break out of the current loop
i += 1
# If the indeces list has the index value then nothing else is done.
pass
elif argu == "joe":
print("did work with joe")
goToNextIndex(i)
elif argu == "sam":
print("did work with sam")
goToNextIndex(i)
elif argu == "school":
print("going to school")
goToNextIndex(i)
# If the argument has other arguments after it that are associated with it
# then add those indeces also to the completed indeces list.
#take in the argument following school
nextArg = i
#Do some work with the next argument
schoolName = args[nextArg]
print(f"You're going to the school called {schoolName}")
#make sure to skip the next argument as it has already been handled
completedIndeces.append(nextArg)
else:
print(f"Error the following argument is invalid: {argu}")
goToNextIndex(i)
print(f"Value of i: {i}")
print(f"completed indeces List: {completedIndeces}")
main()
I currently have a python file that utilizes sys.argv[1] to accept a string at the command line. It then performs operations on that string and then returns the modified string to the command line.
I would like to implement a batch mode option in which I can provide a file of strings (one per line, fwiw) and have it return to the command line so that I can redirect the output doing something like
$ python script.py -someflag file.txt > modified.txt
while still retaining the current capabilities.
I am only running 2.6, so argparse is not an option. The tutorials I have seen either use argparse, getopt, or delve into examples that are too complex/don't apply.
What is the best way to check the input and act appropriately?
argparse is still an option, it's just not built into 2.6. You can still install it like any 3rd party package (for example, using easy_install argparse).
An example of code for this would be:
import sys
import argparse
p = argparse.ArgumentParser(description="script.py")
p.add_argument("-s", dest="string")
p.add_argument("-f", dest="infile")
args = p.parse_args()
if args.infile == None and args.string == None:
print "Must be given either a string or a file"
sys.exit(1)
if args.infile != None and args.string != None:
print "Must be given either a string or a file, not both"
sys.exit(1)
if args.infile:
# process the input file one string at a time
if args.string:
# process the single string
See my answer here: What's the best way to grab/parse command line arguments passed to a Python script?
As a shortcut, here's some sample code:
import optparse
parser = optparse.OptionParser()
parser.add_option('-q', '--query',
action="store", dest="query",
help="query string", default="spam")
options, args = parser.parse_args()
print 'Query string:', options.query
Python optparse normally allows the user to specify an option more than once and silently ignores all occurrences of the option but the last one. For example, if the action of option --foo is store and the action of option --flag is store_const, store_true or store_false, the following commands will be equivalent:
my-command --foo=bar --foo=another --flag --foo=last --flag
my-command --flag --foo=last
(Update: argparse does just the same thing by default.)
Now, I have a lot of options, and specifying any of them more than once doesn't make sense. If a user specifies the same option more than once I'd like to warn them about the possible error.
What is the most elegant way to detect options that were specified multiple times? Note that the same option can have a short form, a long form and abbreviated long forms (so that -f, --foobar, --foob and --foo are all the same option). It would be even better if it was possible to detect the case when multiple options that have the same destination were specified simultaneously, so that a warning can be given if a user specifies both --quiet and --verbose while both options store a value into the same destination and effectively override each other.
Update: To be more user-friendly, the warning should refer to the exact option names as used on the command line. Using append actions instead of store is possible, but when we detect a conflict, we cannot say which options caused it (was it -q and --verbose or --quiet --quiet?).
Unfortunately I'm stuck with optparse and cannot use argparse because I have to support Python 2.6.
P. S. If you know of a solution that works only with argparse, please post it, too. While I try to minimize the number of external dependencies, using argparse under Python 2.6 is still an option.
I think the correct way would be to "define your action" in some way.
For example, you could use the action callback and implement a function that implement your desired behaviour.
You could write a function that first checks if the destination was already filled, if it is filled then it stores the overlapping options into a list.
When the parsing is finished you should check if these lists are empty, and if they are not raise the appropriate exception.
Another way of doing this could be to define your own action. You can have a look here
A small example that uses the callback:
import sys
import functools
from optparse import OptionParser
bad_option = 'BAD OPTION'
def store(option, opt, value, parser, dest, val):
"""Set option's destination *dest* to *val* if there are no conflicting options."""
list_name = dest + '_options_list'
try:
# if this option is a conflict, save its name and set the value to bad_option
getattr(parser.values, list_name).append(opt)
setattr(parser.values, dest, bad_option)
except AttributeError:
# no conflicts, set the option value and add the options list
setattr(parser.values, dest, val)
setattr(parser.values, list_name, [opt])
store_true = functools.partial(store, val=True)
store_false = functools.partial(store, val=False)
parser = OptionParser()
parser.add_option('-v', '--verbose',
action='callback', callback=store_true,
help='Increase output verbosity',
callback_kwargs={'dest': 'verbose'})
parser.add_option('-q', '--quiet',
action='callback', callback=store_false,
help='Decrease output verbosity',
callback_kwargs={'dest': 'verbose'})
opts, args = parser.parse_args()
# detects all conflicting options for all destinations
found = False
for dest in ('verbose',):
if getattr(opts, dest) == bad_option:
conflicting_opts = ', '.join(getattr(opts, dest + '_options_list'))
print('Conflicting options %s for destination %s'
% (conflicting_opts, dest))
found = True
if found:
parser.print_usage()
sys.exit(2)
And the output:
$ python testing_optparse.py -v -q
Conflicting options -v, -q for destination verbose
Usage: prova_optparse.py [options]
Probably it would be better to raise an OptionValueError when detecting conflicts, even though this would allow to get only couple of conflicting options. If you want to get all conflicting options you have to parse the remaining arguments( in parser.rargs).
You can use action="append" (optparse) and then check the number of appended elements. See http://docs.python.org/library/optparse.html#other-actions
Using argparse, is it possible to stop parsing arguments at the first unknown argument?
I've found 2 almost solutions;
parse_known_args, but this allows for known parameters to be detected after the first unknown argument.
nargs=argparse.REMAINDER, but this won't stop parsing until the first non-option argument. Any options preceding this that aren't recognised generate an error.
Have I overlooked something? Should I be using argparse at all?
I haven't used argparse myself (need to keep my code 2.6-compatible), but looking through the docs, I don't think you've missed anything.
So I have to wonder why you want argparse to stop parsing arguments, and why the -- pseudo-argument won't do the job. From the docs:
If you have positional arguments that must begin with '-' and don’t look like negative numbers, you can insert the pseudo-argument '--' which tells parse_args() that everything after that is a positional argument:
>>> parser.parse_args(['--', '-f'])
Namespace(foo='-f', one=None)
One way to do it, although it may not be perfect in all situations, is to use getopt instead.
for example:
import sys
import os
from getopt import getopt
flags, args = getopt(sys.argv[1:], 'hk', ['help', 'key='])
for flag, v in flags:
if flag in ['-h', '--help']:
print(USAGE, file=sys.stderr)
os.exit()
elif flag in ['-k', '--key']:
key = v
Once getopt encounters a non-option argument it will stop processing arguments.
With Perl's Getopt::Long you can easily define command-line options that take a variable number of arguments:
foo.pl --files a.txt --verbose
foo.pl --files a.txt b.txt c.txt --verbose
Is there a way to do this directly with Python's optparse module? As far as I can tell, the nargs option attribute can be used to specify a fixed number of option arguments, and I have not seen other alternatives in the documentation.
This took me a little while to figure out, but you can use the callback action to your options to get this done. Checkout how I grab an arbitrary number of args to the "--file" flag in this example.
from optparse import OptionParser,
def cb(option, opt_str, value, parser):
args=[]
for arg in parser.rargs:
if arg[0] != "-":
args.append(arg)
else:
del parser.rargs[:len(args)]
break
if getattr(parser.values, option.dest):
args.extend(getattr(parser.values, option.dest))
setattr(parser.values, option.dest, args)
parser=OptionParser()
parser.add_option("-q", "--quiet",
action="store_false", dest="verbose",
help="be vewwy quiet (I'm hunting wabbits)")
parser.add_option("-f", "--filename",
action="callback", callback=cb, dest="file")
(options, args) = parser.parse_args()
print options.file
print args
Only side effect is that you get your args in a list instead of tuple. But that could be easily fixed, for my particular use case a list is desirable.
My mistake: just found this Callback Example 6.
I believe optparse does not support what you require (not directly -- as you noticed, you can do it if you're willing to do all the extra work of a callback!-). You could also do it most simply with the third-party extension argparse, which does support variable numbers of arguments (and also adds several other handy bits of functionality).
This URL documents argparse's add_argument -- passing nargs='*' lets the option take zero or more arguments, '+' lets it take one or more arguments, etc.
Wouldn't you be better off with this?
foo.pl --files a.txt,b.txt,c.txt --verbose
I recently has this issue myself: I was on Python 2.6 and needed an option to take a variable number of arguments. I tried to use Dave's solution but found that it wouldn't work without also explicitly setting nargs to 0.
def arg_list(option, opt_str, value, parser):
args = set()
for arg in parser.rargs:
if arg[0] == '-':
break
args.add(arg)
parser.rargs.pop(0)
setattr(parser.values, option.dest, args)
parser=OptionParser()
parser.disable_interspersed_args()
parser.add_option("-f", "--filename", action="callback", callback=arg_list,
dest="file", nargs=0)
(options, args) = parser.parse_args()
The problem was that, by default, a new option added by add_options is assumed to have nargs = 1 and when nargs > 0 OptionParser will pop items off rargs and assign them to value before any callbacks are called. Thus, for options that do not specify nargs, rargs will always be off by one by the time your callback is called.
This callback is can be used for any number of options, just have callback_args be the function to be called instead of setattr.
Here's one way: Take the fileLst generating string in as a string and then use http://docs.python.org/2/library/glob.html to do the expansion ugh this might not work without escaping the *
Actually, got a better way:
python myprog.py -V -l 1000 /home/dominic/radar/*.json <- If this is your command line
parser, opts = parse_args()
inFileLst = parser.largs