I am using the Click library but I can't seem to find a behavior similar to dest from argparse.
For example, I have
#click.option('--format', type=click.Choice(['t', 'j']))
def plug(format):
pass
Notice that I am using a flag with --format that gets translated into a built-in Python construct format which is not ideal.
Is there a way to change the argument passed into the click function for options?
While Click doesn't have dest-equivalent of argparse, it has certain argument-naming behavior which can be exploited. Specifically, for parameters with multiple possible names, it will prefer non-dashed to dashed names, and as secondary preference will prioritize longer names over shorter names.
URL: http://click.pocoo.org/dev/parameters/#parameter-names
So if you declare your option as...
#click.option('--format', 'not-format', type=click.Choice(['t', 'j']))
...then Click will prioritize non-dashed variant ('not-format') and call your function with not_format=... argument.
Of course it also means that this alternative spelling can also be used in command line. If that is not desired, then I guess you could add a decorator to rename keyword arguments:
import functools
def rename_kwargs(**replacements):
def actual_decorator(func):
#functools.wraps(func)
def decorated_func(*args, **kwargs):
for internal_arg, external_arg in replacements.iteritems():
if external_arg in kwargs:
kwargs[internal_arg] = kwargs.pop(external_arg)
return func(*args, **kwargs)
return decorated_func
return actual_decorator
Testing code:
if __name__ == '__main__':
#rename_kwargs(different_arg='format')
def tester(different_arg):
print different_arg
tester(format='test value')
Test output:
$ python test_decor.py
test value
In your case, it would look like:
#click.option('--format', type=click.Choice(['t', 'j']))
#replace_kwargs(not_format='format')
def plug(not_format):
pass
Renaming an option to a differently named function argument is possible by decorating the function with
#click.option('--format', '-f', 'format_arg_name')
def plug(format_arg_name):
print(format_arg_name)
then it will remap the option named format and make it available as the format_arg_name parameter.
format_arg_name will not be available as a command line option, but --format and -f are.
Related
I am trying to pass command line arguments to pytest tests.
Example from this question
print ("Displaying name: %s" % name)
In conftest
parser.addoption("--name", action="store", default="default name")
def pytest_generate_tests(metafunc):
# This is called for every test. Only get/set command line arguments
# if the argument is specified in the list of test "fixturenames".
option_value = metafunc.config.option.name
if 'name' in metafunc.fixturenames and option_value is not None:
metafunc.parametrize("name", [option_value])
I want to pass name argument to pytest.mark.parametrize.
Now, I am aware that according to the question I've referenced, the answer mentions that I shouldn't use #pytest.mark.parametrize. However, I am working on a code base that already utilizes it and I also need to pass arguments as part of CI process. Is there a work around or am I doomed to re-write all of the tests?
Edit:
Since I wasn't clear enough- I cannot use the command line argument in mark.parametrize with pytest console line args.
Example- I have a function that returns a list and has input for name and I add it to the mark.parametrize:
#pytest.mark.parametrize("return_list", foo(name)):
def test_name(return_list):
pass
The name parameter, like I mentioned needs to come from the command line, and this doesn't work in any method that I've encountered.
I don't know how to use a list in pytest without mark.parametrize, so it's a problem
Ok, so I've found an answer using pytest-lazy-fixture.
It can be pip installed for any version of pytest higher than 3.2.5 (and included).
The Gist is calling a custom fixture (conftest of the test itself) by its keyword.
Will leave a full answer here for future reference:
Full example:
In conftest
def pytest_addoption(parser):
parser.addoption("--name", action="store", default="default name")
#pytest.fixture(scope = 'module', autouse = True)
def names(request):
""" main code and manipulation on the data """
return request.config.getoption("--name")
In test.py
#pytest.mark.parametrize('arg1',
[pytest.lazy_fixture('names')]
)
def test_something(arg1):
# do something here
pass
This also works for mark.parametrize if the lazy_fixture is fed into another function.
I use click like this:
import click
#click.command(name='foo')
#click.option('--bar', required=True)
def do_something(bar):
print(bar)
So the name of the option and the name of the function parameter is the same. When it is not the same (e.g. when you want --id instead of --bar), I get:
TypeError: do_something() got an unexpected keyword argument 'id'
I don't want the parameter to be called id because that is a Python function. I don't want the CLI parameter to be called different, because it would be more cumbersome / less intuitive. How can I fix it?
You just need to add a non-option (doesn't start with -) argument in the click.option decorator. Click will use this as the name of the parameter to the function. This allows you to use Python keywords as option names.
Here is an example which uses id_ inside the function:
import click
#click.command(name='foo')
#click.option('--id', 'id_', required=True)
def do_something(id_):
print(id_)
There is an official example here in the --from option.
I have written a python module mymod.py that can be used also as a standalone program from command line.
In mymod.py I have defined a few functions (where default values are set using keywords)
and an if __name__=="__main__" block to use the module as a standalone program.
I want the possibility to override some of the default options, therefore in the main program I import argparse and use it to parse the options. I use a dictionary to store
the default values, so that if some day I need to change the default values I can easily
do it modifying its value in one place only.
It works, but I find that the code is not "clean" and thought that probably I am not doing it in the proper pythonic way.
This is a toy example to show what I do:
#!/usr/bin/env python
#mymod.py
__default_options__={
"f1_x":10,
"f2_x":10
}
def f1(x=__default_options__["f1_x"]):
return x**2
def f2(x=__default_options__["f2_x"]):
return x**4
# this function is the "core" function which uses f1 and f2
# to produce the main task of the program
def f(x=__default_options__["f1_x"],y=__default_options__["f2_x"]):
return f1(x)+f2(y)
if __name__=="__main__":
import argparse
parser = argparse.ArgumentParser(description = "A toy application")
parser.add_argument("--f1-x",help="the parameter passed to f1",
default=__default_options__["f1_x"], type = float,dest = "x")
parser.add_argument("--f2-x",help="the parameter passed to f2",
default=__default_options__["f2_x"], type = float, dest = "y")
options= parser.parse_args()
print f(options.x,options.y)
Passing the default values like I do it is a bit cumbersome and probably against the spirit both of Python and argparse.
How can this code be improved to be more pythonic and use argparse at its best?
You can use the `ArgumentParser.set_defaults method, in the following way
default_options={
"x":10,
"y":10
}
def f1(**kwargs):
x=kwargs.get('x', defalut_options['x'])
return x**2
def f2(**kwargs):
y=kwargs.get('y', defalut_options['y'])
return x**4
def f(**kwargs):
x=kwargs.get('x', defalut_options['x'])
y=kwargs.get('y', defalut_options['y'])
return f1(x=x, y=y)
if __name__=="__main__":
import argparse
parser = argparse.ArgumentParser(description = "A toy application", formatter_class=argparse.ArgumentDefaultsHelpFormatter )
parser.add_argument("--f1-x",help="the parameter passed to f1",
type = float,dest = "x")
parser.add_argument("--f2-x",help="the parameter passed to f2",
type = float, dest = "y")
parser.set_defaults(**default_options)
options= parser.parse_args()
print f(options.x,options.y)
It took me a while to make it work, because I didn't notice that you are using dest in add_argument (I never use it). If this keyword is not provided, argparse set the default dest to the long name of the argument (in this case f1_x and f2_x, as it substitutes - with _). To go to the point: if you want to provide a dictionary of defaults, the keys needs to match dest if provided. Besides, take care that parser.set_defaults just add arguments to the parser, so if you have some entry not in your parser, it will be added to the Namespace.
--Edited to add generic kwargs to the functions--
As #Francesco wrote in a comment, your defaults dictionary won't work as you probably intended: The functions will retain the defaults they had while loading the module, regardless of later changes to the dictionary. Here's how to make them track the current value of the dictionary:
_default_options = {
"f1_x":10,
"f2_x":10
}
def f1(x=None):
if x == None:
x = _default_options["f1_x"]
...
You can then modify _default_options via ArgumentParser, or in any other way, and f1() will use it if called with no arguments.
This requires that None could never be a meaningful value for x; if that's not the case, choose a suitable impossible value.
I'm trying to figure out how to pass optional arguments from optparse. The problem I'm having is if an optparse option is not specified, it defaults to a None type, but if I pass the None type into a function, it yells at me instead of using the default (Which is understandable and valid).
conn = psycopg2.connect(database=options.db, hostname=options.hostname, port=options.port)
The question is, how do I use the function's defaults for optional arguments but still pass in user inputs if there is an input without having a huge number of if statements.
Define a function remove_none_values that filters a dictionary for none-valued arguments.
def remove_none_values(d):
return dict((k,v) for (k,v) in d.iteritems() if not v is None)
kwargs = {
'database': options.db,
'hostname': options.hostname,
...
}
conn = psycopg2.connect(**remove_none_values(kwargs))
Or, define a function wrapper that removes none values before passing the data on to the original function.
def ignore_none_valued_kwargs(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
newkwargs = dict((k,v) for (k,v) in d.iteritems() if not v is None)
return f(*args, **kwargs)
return wrapper
my_connect = ignore_none_valued_kwargs(psycopg2)
conn = my_connect(database=options.db, hostname=options.hostname, port=options.port)
The opo module of my thebops package (pip install thebops, https://bitbucket.org/therp/thebops) contains an add_optval_option function.
This uses an additional keyword argument empty which specifies the value to use if the option is used without a value. If one of the option strings is found in the commandline, this value is injected into the argument list.
This is still hackish, but at least it is made a simple-to-use function ...
It works well under the following circumstances:
The argument vector does already exist when the option is created. This is usually true.
All programs I found which sport arguments with optional values require the given value to be attached as --option=value or -ovalue rather than --option value or -o value.
Maybe I'll tweak thebops.optparse to support the empty argument as well; but I'd like to have a test suite first to prevent regressions, preferably the original Optik / optparse tests.
This is the code:
from sys import argv
def add_optval_option(pog, *args, **kwargs):
"""
Add an option which can be specified without a value;
in this case, the value (if given) must be contained
in the same argument as seen by the shell,
i.e.:
--option=VALUE, --option will work;
--option VALUE will *not* work
Arguments:
pog -- parser or group
empty -- the value to use when used without a value
Note:
If you specify a short option string as well, the syntax given by the
help will be wrong; -oVALUE will be supported, -o VALUE will not!
Thus it might be wise to create a separate option for the short
option strings (in a "hidden" group which isn't added to the parser after
being populated) and just mention it in the help string.
"""
if 'empty' in kwargs:
empty_val = kwargs.pop('empty')
# in this case it's a good idea to have a <default> value; this can be
# given by another option with the same <dest>, though
for i in range(1, len(argv)):
a = argv[i]
if a == '--':
break
if a in args:
argv.insert(i+1, empty_val)
break
pog.add_option(*args, **kwargs)
Say,
I have a python function as following:
def ooxx(**kwargs):
doSomething()
for something in cool:
yield something
I would like to provide another function with named arguments for hints as following:
def asdf(arg1, arg2, arg3=1):
frame = inspect.currentframe()
args, _, _, values = inspect.getargvalues(frame)
kwargs = dict((key, values[key]) for key in args) # convert args list into dictionary form
return list(ooxx(**kwargs))
Is it possible to have some sort of methods to generate automatically the function "asdf"? I have lots of dynamic generated ooxx functions and I would like to have corresponding asdf functions with customized named arguments. Not sure if this is the correct requirement or right way to coding :p
Your descriptions doesn't make such sense to me: You wrote a really verbose function that does this:
def asdf(arg1, arg2, arg3=1):
return list(ooxx(**locals()))
but you want to inspect the ooxx and somehow make up appropriate names for asdfs arguments? That is impossible, there is no information about this on ooxx.
If you actually have a signature and want to create a function from it you would have to resort to eval or generate function definitions to a Python file and import it.
There is also the decorator module. You can create a function with it like this:
import decorator
asdf = decorator.FunctionMaker.create(
'asdf(arg1, arg2, arg3)', # signature
'return ooxx(**locals())', # function body
{'ooxx' : ooxx}, # context for the function
('arg3', 1)) # default arguments