This simplified script is enough to cause the issue... just checking if the '-d' argument is a valid directory, supplying a default if it's not provided...
#!/usr/bin/python
import os
import argparse
def valid(dir):
subdir = dir + '/Desktop'
if not os.path.exists(subdir):
raise argparse.ArgumentTypeError("%s is not a valid directory" % subdir)
return dir
parser = argparse.ArgumentParser(description="blah blah blah")
parser.add_argument('-d', '--directory', help='directory to check', default=os.getcwd(), type=valid)
args = parser.parse_args()
And it doesn't matter what the default argument is, when I run the script it uses the default, no matter what I enter on the command line, and throws an uncaught exception as follows:
Traceback (most recent call last):
File "./parsertest.py", line 15, in <module>
args = parser.parse_args()
File "/usr/lib/python2.7/argparse.py", line 1688, in parse_args
args, argv = self.parse_known_args(args, namespace)
File "/usr/lib/python2.7/argparse.py", line 1710, in parse_known_args
default = self._get_value(action, default)
File "/usr/lib/python2.7/argparse.py", line 2239, in _get_value
raise ArgumentError(action, msg)
argparse.ArgumentError: argument -d/--directory: /home/users/jrice/Desktop/Desktop is not a valid directory
Runs fine, and by fine I mean, handles the ArgumentTypeError as and when it should, just printing the msg when if I do the following:
Remove the 'default=' argument
Do not append '/Desktop' to dir, so subdir = dir, or just check dir itself
Run the script from my home directory!?!?
Elaboration: If I do any of the above, even if '-d' isn't valid, everything is fine. This is the output, which is what I want.
>./Desktop/parsertest.py -d blah
usage: parsertest.py [-h] [-d DIRECTORY]
parsertest.py: error: argument -d/--directory: blah/Desktop is not a valid directory
why should os.getcwd() + '/Desktop' be any different?
I believe your "type checking" is too aggressive. You treat a non-existing directory as an invalid type, which is not the way argparse has been thought. In your case, the default value might not be a "valid type" which confuses argparse. Check out the following code and its output:
#!/usr/bin/python
import os
import argparse
def valid(dir):
print "Checking " + dir
subdir = dir + '/Desktop'
#if not os.path.exists(subdir):
# raise argparse.ArgumentTypeError("%s is not a valid directory" % subdir)
return dir
parser = argparse.ArgumentParser(description="blah blah blah")
parser.add_argument('-d', '--directory', help='directory to check', type=valid, default=os.getcwd())
args = parser.parse_args()
Executing it from /home/user/Desktop with -d /home/user gives:
Checking /home/user/Desktop
Checking /home/user
As you can see, argparse first converts the default value and only then the command-line given value.
To solve the above issue, either make sure that the default value is always a "valid type" or that you check the directory after argparse is done.
Argparse attempts to convert the default argument to whatever type was given to it.
import argparse
parser = argparse.ArgumentParser(description="blah blah blah")
parser.add_argument('-i',default="1",type=int)
args = parser.parse_args([])
print args # Namespace(i=1)
print type(args.i) # <type 'int'>
The reason for this design choice is a little weird to me, but it is probably so that you can pass strings to default just as it would get them on the commandline and then the help will be formatted properly.
Note I don't really like passing validation code to the type keyword argument even though they do it in the documentation. That argument is to convert the input string into some other type. If you really want to do the validation as you parse, you should consider using a custom Action, but for this example, it's probably just easiest to do:
#...snip...
parser.add_argument('-d', '--directory', help='directory to check')
args = parser.parse_args()
args.directory = valid(args.directory if args.directory is not None else os.getcwd())
#the following should work too.
#args.directory = valid(args.directory if args.directory else os.getcwd())
Related
This is my first post, so please excuse any missing info or stupid questions.So what I am trying to do is run a command line python script (I'm in a Windows machine), pass it 2 arguments, check if the arguments were passed, if not, display an error message specific to that argument. I am getting a missing argument error, but it isnt displaying the message I would like (defined in the function). I'm not sure if I didn't create the function properly, or if I am missing something. I also want to check if the arguments passed are correct strings, not int, but it seems that the arguments are defaulting to strings, so checking if they are int is not working either.
Can someone give me a hint or point me in the right direction? I've been searching online but havent found anything that has answered this question. I am brand new to python, so still learning how to read documentation properly (seems like there are too few examples for me to understand the docs). Here is the code I am using:
import argparse
#parser to create and grab args from CLI
parser = argparse.ArgumentParser()
parser.add_argument("-i", help="File path of original folder")
parser.add_argument("-o", help="File path of output folder")
args = parser.parse_args()
#check if args have been passed and display error if missing
def check_args(input_path,output_path):
if input_path == None:
print("Please input a valid folder path for original folder destination")
elif output_path == None:
print("Please input a valid folder path for output folder destination")
else:
return args
#grab first and sec argument
input_path = args.i
output_path = args.o
check_args(input_path, output_path)
Output:
Program>python JPGtoPNG_conv.py -i 3 -o
usage: JPGtoPNG_conv.py [-h] [-i I] [-o O]
JPGtoPNG_conv.py: error: argument -o: expected one argument
Thanks so much for any help!
The code below should do the job for you
import argparse
import sys
class MyParser(argparse.ArgumentParser):
def error(self, message):
sys.stderr.write('error: %s\n' % message)
self.print_help()
sys.exit(2)
parser = MyParser()
parser.add_argument("-i", help="File path of original folder", required=True)
parser.add_argument("-o", help="File path of output folder", required=True)
args = parser.parse_args()
# grab first and sec argument
input_path = args.i
output_path = args.o
I have a function inside a module that creates an argparse:
def get_options(prog_version='1.0', prog_usage='', misc_opts=None):
options = [] if misc_opts is None else misc_opts
parser = ArgumentParser(usage=prog_usage) if prog_usage else ArgumentParser()
parser.add_argument('-v', '--version', action='version', version='%(prog)s {}'.format(prog_version))
parser.add_argument('-c', '--config', dest='config', required=True, help='the path to the configuration file')
for option in options:
if 'option' in option and 'destination' in option:
parser.add_argument(option['option'],
dest=option.get('destination', ''),
default=option.get('default', ''),
help=option.get('description', ''),
action=option.get('action', 'store'))
return parser.parse_args()
A sample myapp.py would be:
my_options = [
{
"option": "-s",
"destination": "remote_host",
"default": "127.0.0.1",
"description": "The remote server name or IP address",
"action": "store"
},
]
# Get Command Line Options
options = get_options(misc_opts=my_options)
print options.config
print options.remote_host
and this will be called as:
$> python myapp.py -c config.yaml
$> config.yaml
127.0.0.1
Now, I am trying to create a unit test for this function but my problem is that I can't pass command line parameters via test code.
# mytest.py
import unittest
from mymodule import get_options
class argParseTestCase(unittest.TestCase):
def test_parser(self):
options = get_options()
# ...pass the command line arguments...
self.assertEquals('config.yaml', options.config) # ofcourse this fails because I don't know how I will pass the command line arguments
My problem is that I need to pass the command line arguments to get_options() but I don't know how to do it properly.
Expected proper call: python mytest.py (-c config.yaml should be passed inside the test code somehow.)
What is "working"/not working right now:
python mytest.py -c config.yaml is also not working. Returns AttributeError: 'module' object has no attribute 'config' since it expects me to call argParseTestCase instead. In other words, python mytest.py -c argParseTestCase "works" but would ofcourse be an return AssertionError: 'config.yaml' != 'argParseTestCase'
python mytest.py -v to run the unit test in verbose mode also fails. It returns:
test_parser (main.argParseTestCase) ... mytest.py 1.0 ERROR
ERROR: test_parser (main.argParseTestCase)
Traceback (most recent call last):
File "tests/unit_tests/mytest.py", line 376, in test_parser options = get_options()
File "/root/test/lib/python2.7/site-packages/mymodule.py", line 61, in get_options return parser.parse_args()
File "/usr/local/lib/python2.7/argparse.py", line 1701, in parse_args args, argv = self.parse_known_args(args, namespace)
File "/usr/local/lib/python2.7/argparse.py", line 1733, in parse_known_args namespace, args = self._parse_known_args(args, namespace)
File "/usr/local/lib/python2.7/argparse.py", line 1939, in _parse_known_args start_index = consume_optional(start_index)
File "/usr/local/lib/python2.7/argparse.py", line 1879, in consume_optional take_action(action, args, option_string)
File "/usr/local/lib/python2.7/argparse.py", line 1807, in take_action action(self, namespace, argument_values, option_string)
File "/usr/local/lib/python2.7/argparse.py", line 1022, in call parser.exit(message=formatter.format_help())
File "/usr/local/lib/python2.7/argparse.py", line 2362, in exit _sys.exit(status)
SystemExit: 0
I prefer explicitly passing arguments instead of relying on globally available attributes such as sys.argv (which parser.parse_args() does internally). Thus I usually use argparse by passing the list of arguments myself (to main() and subsequently get_options() and wherever you need them):
def get_options(args, prog_version='1.0', prog_usage='', misc_opts=None):
# ...
return parser.parse_args(args)
and then pass in the arguments
def main(args):
get_options(args)
if __name__ == "__main__":
main(sys.argv[1:])
that way I can replace and test any list of arguments I like
options = get_options(['-c','config.yaml'])
self.assertEquals('config.yaml', options.config)
Your error message stack is hard to read because it is in quoted form rather than code. But I think the -v argument is producing a sys.exit. version is like help - it's supposed to display a message and then exit. The -v is used by unittest, but is also read by your parser.
There is an argparse unittest module, test/test_argparse.py. You may need a development Python installation to see that. Some tests are straightforward, others use specialized testing structure. Some of that special code creates arguments in the same way you do with options.
The are two special issues:
generating the input. parse_args uses sys.argv[1:] unless its argv parameter is not None. So you can test a parser by either modifying the sys.argv list (unittest has already used your commandline values), or by passing a argv=None keyword argument into your function and on to parse_args. Trying to make a commandline meant for the unittest code to work with get_options is too complicated.
trapping the output, especially the sys.exit generated by errors. One option is to subclass ArgumentParser and give it a different error and/or exit method. Another is to wrap the function call in a try block.
unittest takes -c argument, but with a different syntax and meaning
-c, --catch Catch control-C and display results
and -v is verbose, not version.
=============
This tests the config argument (in a self contained one file form)
import unittest
import sys
#from mymodule import get_options
def get_options(argv=None, prog_version='1.0', prog_usage='', misc_opts=None):
# argv is optional test list; uses sys.argv[1:] is not provided
from argparse import ArgumentParser
options = [] if misc_opts is None else misc_opts
parser = ArgumentParser(usage=prog_usage) if prog_usage else ArgumentParser()
parser.add_argument('-v', '--version', action='version', version='%(prog)s {}'.format(prog_version))
parser.add_argument('-c', '--config', dest='config', help='the path to the configuration file')
for option in options:
if 'option' in option and 'destination' in option:
parser.add_argument(option['option'],
dest=option.get('destination', ''),
default=option.get('default', ''),
help=option.get('description', ''),
action=option.get('action', 'store'))
args = parser.parse_args(argv)
print('args',args)
return args
class argParseTestCase(unittest.TestCase):
def test_config(self):
sys.argv[1:]=['-c','config.yaml']
options = get_options()
self.assertEquals('config.yaml', options.config)
def test_version(self):
sys.argv[1:]=['-v']
with self.assertRaises(SystemExit):
get_options()
# testing version message requires redirecting stdout
# similarly for a misc_opts test
if __name__=='__main__':
unittest.main()
I am using optparse to get command line input.
Lets say that I am running a script demo.py and it creates some output. But unless I specify the command line input, the output is not written to a file.
I am trying to do the following:
python demo.py in command line should run the script, but not write the output anywhere.
python demo.py -o in command line should write the output to my default file name output.txt.
python demo.py -o demooutput.txt in command line should write the output to file demooutput.txt.
PS: I would not prefer to switch to argparse from optparse.
You can use optparse-callbacks to achieve this.
Here is how it wiill work for your use case.
parser.add_option("-o", action="callback", dest="output", callback=my_callback)
def my_callback(option, opt, value, parser):
if len(parser.rargs) > 0:
next_arg = parser.rargs[0]
if not next_arg.startswith("-"):
# Next argument is not another option
del parser.rargs[0]
setattr(parser.values, option.dest, next_arg)
return
# If not processed, set the default value
setattr(parser.values, option.dest, "output.txt")
I don't think there is unfortunately - the only way I can think of is hacking around the problem by adding your own logic statements. The following code should do the trick.
import re, sys
import optparse from OptionParser
usage = "usage: %prog [options] arg"
parser = OptionParser(usage)
if '-f' in argv:
a = argv.index('-f')
if (a != len(argv)-1) and re.search('[.]txt', argv[a+1]):
parser.add_option("-f", "--foo", dest="foo")
else:
parser.add_option("-f", dest="foo", action="store_true")
This doesn't answer the direct question, 'how to define an Action...', but it handles the inputs in a simple way.
Set '-o' to be 'store_true'. If True check the 'args' variable for a file name.
(options, args) = parser.parse_args()
if options.o:
if args:
dest = args[0]
else:
dest = 'output.txt'
else:
dest = ''
(In argparse the equivalent would be to define a positional argument with nargs='?'.)
If these are the only arguments, you could also get by with checking for the filename without requiring the `-o'.
Another possibility - 'store_const', with the positional 'filename' having priority:
parser = optparse.OptionParser()
parser.add_option('-o',dest='dest',action='store_const', const='output.txt', default='')
(options, args) = parser.parse_args()
if args:
options.dest = args[0]
print options
can argparse be used to validate filename extensions for a filename cmd line parameter?
e.g. if i have a python script i run from the cmd line:
$ script.py file.csv
$ script.py file.tab
$ script.py file.txt
i would like argparse to accept the 1st two filename cmd line options but reject the 3rd
i know you can do something like this:
parser = argparse.ArgumentParser()
parser.add_argument("fn", choices=["csv","tab"])
args = parser.parse_args()
to specify two valid choices for a cmd line option
what i'd like is this:
parser.add_argument("fn", choices=["*.csv","*.tab"])
to specify two valid file extensions for the cmd line option. Unfortunately this doesn't work - is there a way to achieve this using argparse?
Sure -- you just need to specify an appropriate function as the type.
import argparse
import os.path
parser = argparse.ArgumentParser()
def file_choices(choices,fname):
ext = os.path.splitext(fname)[1][1:]
if ext not in choices:
parser.error("file doesn't end with one of {}".format(choices))
return fname
parser.add_argument('fn',type=lambda s:file_choices(("csv","tab"),s))
parser.parse_args()
demo:
temp $ python test.py test.csv
temp $ python test.py test.foo
usage: test.py [-h] fn
test.py: error: file doesn't end with one of ('csv', 'tab')
Here's a possibly more clean/general way to do it:
import argparse
import os.path
def CheckExt(choices):
class Act(argparse.Action):
def __call__(self,parser,namespace,fname,option_string=None):
ext = os.path.splitext(fname)[1][1:]
if ext not in choices:
option_string = '({})'.format(option_string) if option_string else ''
parser.error("file doesn't end with one of {}{}".format(choices,option_string))
else:
setattr(namespace,self.dest,fname)
return Act
parser = argparse.ArgumentParser()
parser.add_argument('fn',action=CheckExt({'csv','txt'}))
print parser.parse_args()
The downside here is that the code is getting a bit more complicated in some ways -- The upshot is that the interface gets a good bit cleaner when you actually go to format your arguments.
Define a custom function which takes the name as a string - split the extension off for comparison and just return the string if it's okay, otherwise raise the exception that argparse expects:
def valid_file(param):
base, ext = os.path.splitext(param)
if ext.lower() not in ('.csv', '.tab'):
raise argparse.ArgumentTypeError('File must have a csv or tab extension')
return param
And then use that function, such as:
parser = argparse.ArgumentParser()
parser.add_argument('filename', type=valid_file)
No. You can provide a container object to choices argument, or anything that supports the "in" operator. You can read more at pydocs
You can always check it yourself and provide feedback to the user though.
Can anyone spot why the following script is not printing the passed arguments?
import sys, getopt
def usage():
print 'Unknown arguments'
def main(argv):
try:
opts, args = getopt.getopt(argv,'fdmse:d',['files=','data-source=','mode=','start','end'])
except getopt.GetoptError:
usage()
sys.exit(999)
for opt, arg in opts:
# print opt,arg
if opt in('-f','--files'):
print 'files: ', arg #
if __name__ == "__main__":
main(sys.argv[1:])
When I run the script at the command line and pass the arguments -f=dummy.csv, usage() seems to be invoked instead - WHY?
BTW, I find the logic of the program flow a bit weird (I copied it from here). Normally, I would have thought that the logic will be implemented in the try branch, and then AFTER that comes the exception handler.
Is this (as pasted in the code above) the 'Pythonic' way to write try/catch blocks?
Did you get your answers?
One way to debug python exceptions is to move (or copy temporarily for debugging) the code out of the try block. You'll get a full trace.
And of course another way is to reduce the test case. Here I've reduced the problems to three lines, and tried the solution hinted at by #s.lott (using 'f:' in the getopts call), and also show at the end how calling with some different test data behaves:
$ cat x1.py
import sys, getopt
opts, args = getopt.getopt(sys.argv[1:],'fdmse:d',['files=','data-source=','mode=','start','end'])
print "opts=", opts, "args=", args
$ python x1.py -f=dummy.csv argblah
Traceback (most recent call last):
File "x1.py", line 2, in <module>
opts, args = getopt.getopt(sys.argv[1:],'fdmse:d',['files=','data-source=','mode=','start','end'])
File "/usr/lib/python2.6/getopt.py", line 91, in getopt
opts, args = do_shorts(opts, args[0][1:], shortopts, args[1:])
File "/usr/lib/python2.6/getopt.py", line 191, in do_shorts
if short_has_arg(opt, shortopts):
File "/usr/lib/python2.6/getopt.py", line 207, in short_has_arg
raise GetoptError('option -%s not recognized' % opt, opt)
getopt.GetoptError: option -= not recognized
$ sed 's/fdm/f:dm/' <x1.py >x2.py
$ diff x1.py x2.py
2c2
< opts, args = getopt.getopt(sys.argv[1:],'fdmse:d',['files=','data-source=','mode=','start','end'])
---
> opts, args = getopt.getopt(sys.argv[1:],'f:dmse:d',['files=','data-source=','mode=','start','end'])
$ python x2.py -f=dummy.csv argblah
opts= [('-f', '=dummy.csv')] args= ['argblah']
$ python x1.py -f dummy.csv argblah
opts= [('-f', '')] args= ['dummy.csv', 'argblah']
Normally, I would have thought that the logic will be implemented in the try branch
"Normally"? What does normally mean?
What is the program supposed to do? What exceptions make sense? What does the program do in response to the exceptions.
There's no "normally". Any more than there's a normal assignment statement or a normal function definition.
Your program does what makes sense to achieve the required end-state. There's no "normally".
Import and use argparse instead of getopt. It's much easier to use and has almost all you need for running from the command line, built into it.
An example,
parser = argparse.ArgumentParser(
description='Description of what the module does when run.')
parser.add_argument("-o", "--output", help='Path of log file.')
args = parser.parse_args()
As easy as that. You need to import argparse at the top of your file for this to work, of course.