I want to use the argparser module with multiple command-line and some of them should have no arguments.
Example:
parser.add_argument('-website', type=str, nargs='*')
parser.add_argument('-auth', type=str, nargs='*')
parser.add_argument('-dothis', action='store_true')
So I want to call in a command line following commands:
- python script.py -website www.website.com www.website2.com -dothis
In this case the -dothis command should only be use for the second website not for the first one, but it is used for both websites.
Another example:
-python script.py -website www.website1.com www.website2.com -auth username/password
In this case the second website has a authentication not the first one.
So what I want is:
- python script.py www.webstie1.com -dothis www.website2.com -auth u:p -dothis www.website3.com -auth u:p www.website4.com
or:
- python script.py -site www.website1.com -auth u:p -site www.website2.com -site www.website3.com -auth u2:p2
so how my script knows which auth is for which website?
How can I fix it?
Your examples are inconsistent. One has '-website' with 2 values, an other has '-site' repeated (append action?), and the third repeated positionals.
Your goal runs into several fundamental issues with argparse:
optionals are parsed in an order independent manner
arguments normally don't interact
usage does not have provision for displaying interactions
I can write a set of custom Action classes that perform the kind of interaction that you want. But are you really interested?
The basic parsing strategy of argparse is to handle the 'positionals' in order, but 'optionals' (flagged) arguments can occur in any order. So the resulting args namespace does not have information about the order of the commands strings. Old parsers like getopt and optparse aren't much better, since they focus on collecting the optionals, and returning the rest as a undifferentiate list.
An alterantive to custom Action classes is to parse sys.argv yourself. Just iterate through the strings. A string without '--' is a site; any '--' flags the follow apply to that site, etc.
A twist on your desired API is to define '--website' as an append that takes multiple arguments.
parser.add_argument('--website','-w','--site',dest='site',nargs='+',action='append',
help='website plus flags; may repeat')
possible commandlines and Namespaces:
--site www.webwsite.com dothig --site www.website2.com
NS(site=[['www.website.con','dothis'],['website2']])
--site site1 username/pass --site site2
NS(site=[['site1','username/pass'],['site2']])
So for each element in args.site, you can interpret the 1st item as the site name, a parse the rest to find the dothis flag and the auth name.
Related
I am new to argparser in python . I am trying to create argparser for file which contains two functions download and upload file on/from box. It will do only one functionality at once according according to that i am trying to create parser for that file as follows but it's not working for me:
parser = argparse.ArgumentParser(description='Download or Upload file on box.')
parser.add_argument('-df', '--download', required=True,
help='download file box')
parser.add_argument('-uf', '--upload', nargs='+', required=True,
help='upload file of box')
parser.add_argument('-fp', '--filepath', required=True,
help='file path to upload(which file to upload) or download(where to download file)')
parser.add_argument('-fn', '--filename', required=True,
help='what should be the name of file on box')
parser.add_argument('-fi', '--fileid', required=True,
help='file id of file to download from box')
args = vars(parser.parse_args())
NOTE :- every time only -df or -uf options will be there, -fp is mandatory for both and if it is -df then -fi is only option and it is mandatory and if -uf then -fn is only option and it's mandatory.
How do I achieve this, following are example how will i pass argument to file
pyhton abc.py -df -fp 'Download/boxfile/xyz.txt' -fi 123
python abc.py -uf -fp 'Download/boxfile/xyz.txt' -fn 'qwe.txt'
As written all 5 of the arguments are required - you made that explicit. If that's what you really want, all the rest of the question is irrelevant. You'll have to provide all 5 regardless.
But the comments indicate that you want to use either -df or -uf, but probably not both (though that bit's unclear).
While there is a mutually_exclusive_mechanism in argparse, there isn't a inclusive equivalent - something that says if -f is present, then -g must also be given.
But subparsers mechanism can be used that way. You could define an download subparser, with a required -fi argument. And an upload with its own argument.
Another option is to set -df to take 2 arguments (nargs=2), both its box and its file.
If -df and -uf are mutually exclusive, why not use the same argument for the name of the file? That is, replace -fn and -fi with one file argument.
Another option is to make all (or most) of the arguments not-required, and check for the correct combinations after parsing. It's easier to implement complicated logic in your own code than to force argparse to do it for you.
e.g.
if args.download is not None: # not default
<check for `args.filename`> etc
It can also be a good idea to provide defaults for optional arguments. That way the code can run even if the user doesn't provide all items.
On a matter of style. Short option flags, with on - are usually a single character. -d, -u, etc. The ability to combine several into one string only works in that case, e.g. -df foo. In this case it probably doesn't matter since none of your arguments are store_true.
I'm not incredibly familiar with argparse, however from reading the documentation I don't believe there's a way to use mutually_exclusive_group and required to force this. It seems conditional statements or equivalent are necessary to confirm that valid parameter combinations were passed.
See: Python Argparse conditionally required arguments
I'd suggest that you get rid of most of these arguments, maybe all, and think about standard Unix ways of doing things.
Consider scp, which has a syntax of: scp source destination
For instance: scp Download/boxfile/xyz.txt qwe.txt
If you don't supply a destination, it infers that you want the file to be called the same thing, and land right here, so these two things are equivalent:
scp Download/boxfile/xyz.txt xyz.txt
scp Download/boxfile/xyz.txt
Of course, scp can talk to machines across the internet, so there is a format for that:
scp hostname:Download/boxfile/xyz.txt xyz.txt
And if you are uploading, you simple switch the order:
scp xyz.txt hostname:Download/boxfile/xyz.txt
I have a python script that I'd like to supply with an argument (usually) containing wildcards, referring to a series of files that I'd like to do stuff with. Example here:
#!/usr/bin/env python
import argparse
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-i', action="store", dest="i")
results = parser.parse_args()
print 'argument i is: ', results.i
list_of_matched_files = glob.glob(results.i)
In this case, everything works great if the user adds quotes to the passed argument like so:
./test_script.py -i "foo*.txt"
...but often times the users forget to add quotes to the argument and are stumped when the list only contains the first match because UNIX already expanded the list and argparse only then gets the first list element.
Is there a way (within the script) to prevent UNIX from expanding the list before passing it to python? Or maybe even just to test if the argument doesn't contain quotes and then warn the user?
No. Wildcards are expanded by the shell (Bash, zsh, csh, fish, whatever) before the script even runs, and the script can't do anything about them. Testing whether the argument contains quotes also won't work, as the shell similarly strips the quotes from "foo*.txt" before passing the argument to the script, so all Python sees is foo*.txt.
Its not UNIX that is doing the expansion, it is the shell.
Bash has an option set -o noglob (or -f) which turns off globbing (filename expansion), but that is non-standard.
If you give an end-user access to the command-line then they really should know about quoting. For example, the commonly used find command has a -name parameter which can take glob constructs but they have to be quoted in a similar manner. Your program is no different to any other.
If users can't handle that then maybe you should give them a different interface. You could go to the extreme of writing a GUI or a web/HTML front-end, but that's probably over the top.
Or why not prompt for the filename pattern? You could, for example, use a -p option to indicate prompting, e.g:
import argparse
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-i', action="store", dest="i")
parser.add_argument('-p', action="store_true", default=False)
results = parser.parse_args()
if results.p:
pattern = raw_input("Enter filename pattern: ")
else:
pattern = results.i
list_of_matched_files = glob.glob(pattern)
print list_of_matched_files
(I have assumed Python 2 because of your print statement)
Here the input is not read by the shell but by python, which will not expand glob constructs unless you ask it to.
You can disable the expansion using set -f from the command line. (re-enable with set +f).
As jwodder correctly says though, this happens before the script is run, so the only way I can think of to do this is to wrap it with a shell script that disables expansion temporarily, runs the python script, and re-enables expansion. Preventing UNIX from expanding the list before passing it to python is not possible.
Here is an example for the Bash shell that shows what #Tom Wyllie is talking about:
alias sea='set -f; search_function'
search_function() { perl /home/scripts/search.pl $# ; set +f; }
This defines an alias called "sea" that:
Turns off expansion ("set -f")
Runs the search_function function which is a perl script
Turns expansion back on ("set +f")
The problem with this is that if a user stops execution with ^C or some such then the expansion may not be turned back on leaving the user puzzling why "ls *" is not working. So I'm not necessarily advocating using this. :).
This worked for me:
files = sys.argv[1:]
Even though only one string is on the command line, the shell expands the wildcards and fills sys.argv[] with the list.
Did some research, but couldn't find any working solution. I'm trying to parse the following command line, where 'test' and 'train' are two independent subcommands each having distinct arguments:
./foo.py train -a 1 -b 2
./foo.py test -a 3 -c 4
./foo.py train -a 1 -b 2 test -a 3 -c 4
I've been trying using two subparsers ('test','train') but it seems like only one can be parsed at the time. Also it would be great to have those subparsers parents of the main parser such that, e.g. command '-a' doesn't have to be added both to the subparsers 'train' and 'test'
Any solution?
This has been asked before, though I'm not sure the best way of finding those questions.
The whole subparser mechanism is designed for one such command. There are several things to note:
add_subparsers creates a positional argument; unlike optionals a `positional acts only once.
'add_subparsers' raises an error if you invoke it several times
the parsing is built around only one such call
One work around that we've proposed in the past is 'nested' or 'recursive' subparers. In other words train is setup so it too takes a subparser. But there's the complication as to whether subparsers are required or not.
Or you can detect and call multiple parsers, bypassing the subparser mechanism.
From the sidebar
Multiple invocation of the same subcommand in a single command line
and
Parse multiple subcommands in python simultaneously or other way to group parsed arguments
Using argparse, is there a simple way to specify arguments which are mutually exclusive so that the application asks for one of these arguments have to be provided but only one of them?
Example of fictive use-case:
> myapp.py foo --bar
"Foo(bar) - DONE"
> myapp.py read truc.txt
"Read: truc.txt - DONE"
>myapp.py foo read
Error: use "myapp.py foo [options]" or "myapp.py read [options]" (or something similar).
> myapp.py foo truc.txt
Error: "foo" action don't need additional info.
> myapp.py read --bar
Error: "read" action don't have a "--bar" option.
My goal is to have a "driver" application(1) that would internally apply one action depending on the first command line argument and have arguments depending on the action.
So far I see no obvious ways to do this with argparse without manually processing the arguments myself, but maybe I missed something Pythonic? (I'm not a Python3 expert...yet)
I call it "driver" because it might be implemented by calling another application, like gcc does with different compilers.
What you're trying to do is actually supported quite well in Python.
See Mutual Exclusion
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('foo', dest='foo', nargs=1)
group.add_argument('read', dest='read', nargs=1)
args = parser.parse_args()
return args
Let's say I want to make a hashing script:
### some code here
def hashlib_based(path, htype='md5', block_size=2**16):
hash = eval(htype)
with open(path, 'rb') as f:
for block in iter(lambda: f.read(block_size), ''):
hash().update(block)
f.close()
return hash().hexdigest()
### some code here
As you can see, I have the opportunity to use different flags to allow me to change the hash type or the block size when I call the script from the command line (for example ./myscript.py -sha1 -b 512 some_file.ext). The thing is, I don't have any clue on how should I do this in order to keep my code as clean and readable as possible. How do I deal with sys.argv?
First of all, how do I check if the user uses the correct flags? I need to do that in order to print out a usage message. Do I make a list with all the flags, then I check if the user uses one that is in that list?
Should I do all these things inside main() or should I do place them in a different function?
Should I construct my flags with a hyphen-minus in front of them (like this: -a, -b) or without one? To check if a certain flag is present in sys.argv, do I simply do something like:
if '-v' in sys.argv:
verbose = True
?
Because sys.argv has indexes, what is the best way to ignore the order of the flags - or in other words, should ./myscript.py -a -b be the same as ./myscript.py -b -a? While it certainly makes the job easier for the common user, is it common practice to do so?
I saw something similar but for C#. Is there a similar concept in Python?
The thing is, as simple as these things are, they get out of hands quickly - for me at least. I end up doing a mess. What is your approach to this problem?
For really simple use cases, such as checking the presence of one argument, you can do a check like you're showing, i.e.:
if '-v' in sys.argv: ...
which is the quick'n dirty way of checking arguments. But once your project gets a bit more serious, you definitely need to use an argument parsing library.
And there are a few ones to handle argument parsing: there is the now deprecated getopt (I won't give a link), the most common one is argparse which is included in any python distribution.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', '--a-long', help='a help')
parser.add_argument('-b', '--b-long', help='b help')
args = parser.parse_args()
then you can call your script -a -b or script -b -a which will be equivalent. And for free, you've got script -h for free! :-)
Though, my preference is now over docopt, imho, which is way simpler and more elegant, for the same example:
"""
My script.
usage:
myscript -a | --along
myscript -b | --blong
Options:
-a --along a help
-b --blong b help
"""
from docopt import docopt
arguments = docopt(__doc__, version='myscript 1.0')
print(arguments)
HTH