Say I have a list of strings containing arguments and options, with argparse, I’m able to parse this list using the parse_args function into an object, as follows:
import argparse
extra_params = [‘—sum’, ‘7’, ‘-1’, ‘42’]
parser=argparse.ArgumentParser(description=“argparse docs example”)
parser.add_argument(‘integers’, metavar=‘N’, type=int, nargs=‘+’,
help=‘an integer for the accumulator’)
parser.add_argument(‘—sum’, dest=‘accumulate’, action=‘store_const’,
const=sum, default=max,
help=‘sum the integers (default: find the max)’)
parsed_object=parser.parse_args(extra_params)
Here, argparse has parsed a provided iterable of strings. Can one use click to also parse a provided iterable of strings?
I’ve searched through the API documentation for click and it appears that there’s a parse_args function within the *Command set of classes but don’t see anything in the docs around how I can do this. I’ve tried instantiating BaseCommand as well as Command but not sure how to get parse_args working without a correct context.
For broader context, this question is a result of having built a launcher application that end users use as a scaffold to launch their own applications. Here, the launcher consumes a number of arguments for which click decorators work perfectly. Unknown arguments can be handled as shown in the documentation here. This launcher then calls an end-user provided callable with these unparsed parameters. Click leaves unparsed parameters as a tuple of strings. How would the end-user, in this situation, be able to use Click to parse the argument's they're interested in? Here's a snippet to illustrate the issue:
import click
from typing import Tuple
#click.command(name="TestLauncher", context_settings={
"ignore_unknown_options": True
})
#click.option('--uri', '-u',
help="URI for the server")
#click.argument('unprocessed_args', nargs=-1,
type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
print(f"Was passed a URI of {uri}")
print(f"Additional args are {unprocessed_args}")
child_function(unprocessed_args)
def child_function(unprocessed_args: Tuple[str, ...]) -> None:
# How do I get Click to parse the provided args for me?
pass
if __name__ == "__main__":
# pylint: disable=no-value-for-parameter, unexpected-keyword-arg
main()
Running this from the command line:
python3 so_test.py --uri test.com --prog-arg 10
Was passed a URI of test.com
Additional args are ('--prog-arg', '10')
For the calling function not knowing anything about parameters for the child function, you can try this:
#click.command(name="TestLauncher", context_settings={
"ignore_unknown_options": True
})
#click.option('--uri', '-u',
help="URI for the server")
#click.argument('unprocessed_args', nargs=-1,
type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
print(f"Was passed a URI of {uri}")
print(f"Additional args are {unprocessed_args}")
unprocessed_args = dict([(unprocessed_args[i].replace('--', '').replace('-', '_'), unprocessed_args[i+1]) for i in range(0, len(unprocessed_args), 2)])
click.get_current_context().invoke(child_function, **unprocessed_args)
#click.command(context_settings={"ignore_unknown_options": True})
#click.option('-p', '--prog-arg')
def child_function(prog_arg: str, **kwargs) -> None:
# How do I get Click to parse the provided args for me?
print(f"Child function passed: {prog_arg}")
# all remaining unknown options are in **kwargs
if __name__ == "__main__":
# pylint: disable=no-value-for-parameter, unexpected-keyword-arg
main()
However, note that:
unprocessed_args = dict([(unprocessed_args[i].replace('--', '').replace('-', '_'), unprocessed_args[i+1]) for i in range(0, len(unprocessed_args), 2)])
This assumes you can only have one value per option. The alternative is to call your script by passing in options like below, splitting the string on = and doing whatever pre-formatting you deem necessary.
--prog-arg=<Your-desired-values>
Try something like this:
import click
#click.command()
#click.option('--count', default=1, help='number of greetings')
#click.option('--test', default='test_was_not_provided', help='test option')
#click.argument('name')
def hello(*args, **kwargs):
click.echo(f"Hello World! {kwargs['name']} {kwargs['count']}")
if __name__ == '__main__':
hello()
run with something like: python main.py haha --test this_is_a_test --count=40
Reviewing the comments and my ensuing edit, made me think that simply applying the click decorators to the child function may work. Indeed it seems to but I don't entirely know why.
import click
from typing import Tuple
#click.command(name="TestLauncher", context_settings={
"ignore_unknown_options": True
})
#click.option('--uri', '-u',
help="URI for the server")
#click.argument('unprocessed_args', nargs=-1,
type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
print(f"Was passed a URI of {uri}")
print(f"Additional args are {unprocessed_args}")
child_function(unprocessed_args)
#click.command()
#click.option('--prog-arg')
def child_function(prog_arg: str) -> None:
# How do I get Click to parse the provided args for me?
print(f"Child function passed: {prog_arg}")
if __name__ == "__main__":
# pylint: disable=no-value-for-parameter, unexpected-keyword-arg
main()
python3 so_test.py --uri test.com --prog-arg 10
Was passed a URI of test.com
Additional args are ('--prog-arg', '10')
Child function passed: 10
Related
I am making a basic MAC changer application in Kali Linux that takes argument in the following form:
python mac_changer.py -i <interface> -m <new MAC address>
In PyCharm I have made the following functions that take the arguments and returns it, I can store the returned values that are parsed but I also want to allow program to ask user for interface and new MAC address in case the user does not input the interface and MAC in the command. I can return the parsed arguments in the form of options variable, I can also return the inputted arguments but is there any way to store the inputted arguments?
all the modules are properly imported and the following code is a part of the full program
def get_arguments():
parser = optparse.OptionParser()
parser.add_option("-i", "--interface", dest="interface", help="Interface to change its MAC address")
parser.add_option("-m", "--mac", dest="new_mac", help="New MAC address")
(options, arguments) = parser.parse_args()
if options.interface and options.new_mac:
if not options.interface:
#Code to handle error
parser.error(("[-] Please specify an interface, use --help for more info"))
elif not options.new_mac:
#Code to handle error
parser.error(("[-] Please specify a new MAC, use --help for more info"))
return options
else:
return input("Inteface> "),input("New MAC> ")
options = get_arguments()
printf(options.interface)
printf(options.new_mac)
This outputs the given arguments from the above command line but I also want to store and use the arguments that code gets from the user using the input function alongside the parsed arguments.
Your code basically works, but this is problematic because you have two different return types:
return options # argparse.Namespace object
else:
return input("Inteface> "),input("New MAC> ") # tuple of two strings
options = get_arguments()
printf(options.interface)
printf(options.new_mac)
If you use a type checker on this code and annotate the return type, one or other of these lines will raise an error. If you return options as a tuple and the caller tries to access options.interface, it's going to raise an AttributeError because the tuple doesn't have a field called interface.
The easiest fix would be to just say that the function returns interface and new_mac as a string tuple in either case:
from typing import Tuple
def get_arguments() -> Tuple[str, str]:
"""Returns interface, new_mac args, either from command line or user input"""
...
and then:
return options.interface, options.new_mac
else:
return input("Interface> "),input("New MAC> ")
interface, new_mac = get_arguments()
print(interface)
print(new_mac)
A slightly nicer solution IMO would be to use a NamedTuple, which is basically a tuple that you can give a name to, as well as names of its individual fields:
from typing import NamedTuple
class ChangerOptions(NamedTuple):
interface: str
new_mac: str
def get_arguments() -> ChangerOptions:
...
return ChangerOptions(options.interface, options.new_mac)
else:
return ChangerOptions(input("Interface> "),input("New MAC> "))
options = get_arguments()
print(options.interface)
print(options.new_mac)
If you're forced to call a function that has no type annotations and multiple possible return types at runtime (i.e. you're using someone else's code and they're a terrible person who wants you to suffer), it's possible to deal with it in the calling code by doing something like:
return options # argparse.Namespace object
else:
return input("Inteface> "),input("New MAC> ") # tuple of two strings
options = get_arguments()
try:
# this will raise an AttributeError if no fields with these names
interface, new_mac = options.interface, options.new_mac
except AttributeError:
# whoops -- maybe it's a tuple?
# this will raise a TypeError if that wasn't right either...
interface, new_mac = options
print(interface)
print(new_mac)
I'm trying to access to the "resources" folder with the ArgumentParser.
This code and the "resources" folder are in the same folder...
Just to try to run the code, I've put a print function in the predict function. However this error occurs:
predict.py: error: the following arguments are required: resources_path
How can I fix it?
from argparse import ArgumentParser
def parse_args():
parser = ArgumentParser()
parser.add_argument("resources_path", help='/resources')
return parser.parse_args()
def predict(resources_path):
print(resources_path)
pass
if __name__ == '__main__':
args = parse_args()
predict(args.resources_path)
I am guessing from your error message that you are trying to call your program like this:
python predict.py
The argument parser by default gets the arguments from sys.argv, i.e. the command line. You'll have to pass it yourself like this:
python predict.py resources
It's possible that you want the resources argument to default to ./resources if you don't pass anything. (And I further assume you want ./resources, not /resources.) There's a keyword argument for that:
....
parser.add_argument('resources_path', default='./resources')
...
Background
I am trying to write a python script that contains multiple functions like this:
import sys
def util1(x, y):
assert(x is not None)
assert(y is not None)
#does something
def util2(x, y):
assert(x is not None)
assert(y is not None)
#does something
def util3(x, y):
assert(x is not None)
assert(y is not None)
#does something
I need to be able to call any method command line:
python3 myscript.py util1 arg1 arg2
or
python3 myscript.py util3 arg1 arg2
Problem
I don't know the proper way to grab the command line args and pass them to the methods. I found a way to grab the first arg... but I would like a way to say "pass all arg to function x" if this is possible.
What I've tried So far
So far, I at the bottom of my script, I added the following logic:
if __name__ == '__main__':
globals()[sys.argv[1]]()
and so now, when I try to run my script, I get the following response:
lab-1:/var/www/localhost/htdocs/widgets# python3 myscript.py utils1 1 99999
Traceback (most recent call last):
File "myscript.py", line 62, in <module>
globals()[sys.argv[1]]()
TypeError: util1() missing 2 required positional arguments: 'x' and 'y'
I've also tried the following:
globals()[*sys.argv[1:]]()
globals()[*sys.argv[1]:[2]]()
But that doesn't work. I'm getting errors like "TypeError: unhashable type: 'list'
If you can point me in the right direction, I'd appreciate it.
Thanks.
EDIT 1
Based on the recommendation here to review a similar post, I changed my logic to include the argparse library. So now I have the following:
parser = argparse.ArgumentParser(description='This is the description of my program')
parser.add_argument('-lc','--lower_create', type=int, help='lower range value for util1')
parser.add_argument('-uc','--upper_create', type=int, help='upper range value for util1')
parser.add_argument('-lr','--lower_reserve', type=int, help='lower range value for util3')
parser.add_argument('-ur','--upper_reserve', type=int, help='upper range value for util3')
args = parser.parse_args()
#if __name__ == '__main__':
# globals()[sys.argv[1]](sys.argv[2], sys.argv[3])
What's not clear is how I "link" these arguments with a specific function.
So let's say I need -lc and -uc for util1. How can I make that association?
and then for example associate -lr and -ur with util3?
Thank you
You need to pass the arguments to the function when you call it. The naive way to do this would be like this: globals()[sys.argv[1]](sys.argv[2], sys.argv[3]) although you'll probably want to do some extra checking to make sure the arguments exist, as well as the function being called.
That is a nice question.
Try like this.
import sys
def util1(x, y):
print('This is "util1" with the following arguments: "'+x+'" and "'+y+'"')
#does something
def util2(x, y):
print('This is "util2" with the following arguments: "'+x+'" and "'+y+'"')
#does something
def util3(x, y):
print('This is "util3" with the following arguments: "'+x+'" and "'+y+'"')
#does something
locals()[sys.argv[1]](sys.argv[2] , sys.argv[3])
Then calling it like this, works great for me. Just tried it on my test machine.
python file.py util1 arg1 arg2
You could do this quite neatly with click, e.g.
#click.command()
#click.argument('x')
#click.argument('y')
def util1(x, y):
#does something
You can also use varargs, so you don't have to specify every argument:
#click.command()
#click.argument('args', nargs=-1)
def util2(args):
#does something, args is a list
Click also supports different arguments types, validation, etc.
With the Python CLI library argh I want to write a wrapper tool. This wrapper tool is suppose to read the two options -a and -b and to pass all other options to a function (which then calls the wrapped UNIX tool with the left-over options via subprocess).
I have experimented with dispatch's parameter skip_unknown_args:
def wrapper(a=True, b=False):
print("Enter wrapper")
# 1. process a and b
# 2. call_unix_tool(left-over-args)
if __name__ == '__main__':
parser = argh.ArghParser()
argh.set_default_command(parser, wrapper)
argh.dispatch(parser, skip_unknown_args=True)
However the program still does exit when it encounters unknown options and it does not enter the function wrapper as needed. Additionally I don't know where the unknown/skipped arguments are stored, so that I can pass them to the UNIX tool.
How can I tell argh to go into wrapper with the skipped arguments?
I believe this is a bug.
when skip_unknown_args=True, here namespace_obj is a tuple, with a namespace object and remaining args:
(Pdb) p namespace_obj
(ArghNamespace(_functions_stack=[<function wrapper at 0x105cb5e18>], a=False, b=True), ['-c'])
underlying _get_function_from_namespace_obj expects an unary one:
154 function = _get_function_from_namespace_obj(namespace_obj)
...
191 if isinstance(namespace_obj, ArghNamespace):
I checked its coressponding issue and unittest, no idea what is the legitmate behivour the author expects, have dropped a comment there as well.
why not use argparse directly?
You cannot do this with skip_unknown_args=True, because as #georgexsh pointed out the argh library doesn't seem to behave sensibly with that option. However you can provide your own parser class which injects the unknown arguments into the normal namespace:
class ArghParserWithUnknownArgs(argh.ArghParser):
def parse_args(self, args=None, namespace=None):
namespace = namespace or ArghNamespace()
(namespace_obj, unknown_args) = super(ArghParserWithUnknownArgs, self).parse_known_args(args=args, namespace=namespace)
namespace_obj.__dict__['unknown_args'] = unknown_args
return namespace_obj
Note that this class' parse_args method calls ArgParser's parse_known_args method!
With this class defined you can write the wrapper code following way:
def wrapper(a=True, b=False, unknown_args={}):
print("a = %s, b = %s" % (a,b))
print("unknown_args = %s" % unknown_args)
if __name__ == '__main__':
parser = ArghParserWithUnknownArgs()
argh.set_default_command(parser, wrapper)
argh.dispatch(parser)
In your main function wrapper you can access all unknown arguments via the parameter unknown_args and pass this on to your subprocess command
ps: In order to keep the help message tidy decorate wrapper with
#argh.arg('--unknown_args', help=argparse.SUPPRESS)
Addendum: I created an enhanced version of the parser and compiled it into a ready-to-use module. Find it on Github.
I've just started learning Python 3, and my argparse method isn't working as it should. I already tried different examples from this website but nothing is working like I want it to.
My code looks like this:
import argparse
class CommandlineArguments():
def __init__(self, number, duplicates, databases):
self.number = args.number
self.duplicates = args.duplicates
self.databases = args.databases
def ArgumentPassing(self):
print('reached the variables')
print(self.number * 2)
print(self.duplicates)
print(self.databases)
if __name__ == '__main__':
parser = argparse.ArgumentParser(prog='TEST_LIST')
parser.add_argument('-number')
parser.add_argument('-duplicates')
parser.add_argument('-databases')
args = parser.parse_args()
commands = CommandlineArguments(args.number, args.duplicates, args.databases)
When I tried to run the program with:
python3 argparse_try.py
I was expecting that I would get a message like "too few arguments..." Instead, it just runs without any error.
Does anyone know what I am doing wrong?
If you want an argument to be required, you have to pass required=True.
Here's working code that requires all arguments. (I also changed your single dashes to double dashes, which is more typical, and I set type=int for --number, since I assume that's what you want.)
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(prog='TEST_LIST')
parser.add_argument('--number', required=True, type=int)
parser.add_argument('--duplicates', required=True)
parser.add_argument('--databases', required=True)
args = parser.parse_args()
# Output if you don't pass any command-line options:
# usage: TEST_LIST [-h] --number NUMBER --duplicates DUPLICATES --databases
# DATABASES
# TEST_LIST: error: the following arguments are required: --number, --duplicates, --databases
EDIT
#Fujia makes a good point in their answer. You could make these positional arguments instead of options. This means you would use python3 my_argparser.py 1 foo bar instead of python3 --number 1 --duplicates foo --databases bar. To do that, use the following code:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(prog='TEST_LIST')
parser.add_argument('number', type=int)
parser.add_argument('duplicates')
parser.add_argument('databases')
args = parser.parse_args()
# Output if you don't pass any command-line options:
# usage: TEST_LIST [-h] number duplicates databases
# TEST_LIST: error: the following arguments are required: number, duplicates, databases
EDIT2
Note also that this code is wrong:
def __init__(self, number, duplicates, databases):
self.number = args.number
self.duplicates = args.duplicates
self.databases = args.databases
You presumably meant this:
def __init__(self, number, duplicates, databases):
self.number = number
self.duplicates = duplicates
self.databases = databases
The problem with your programme is that the argument names are prefixed with dash -.
If you just use number, duplicates and databases in add_argument, these three arguments become required. By default prefixing the argument name with dash or double dash will make them optional arguments during parsing. You can change this behavior by explicitly specifying the required parameter.