Develop Reference

argparse with optional value with quoted "--options --for --other --cmd"

I need to pass arguments directly to a command executed in my python script. The script itself does not need to evaluate or manipulate the arguments to be passed to the command.
./cookbooks.py vendor -o "--delete"
Unfortunately, argparse recognizes the -- in the quoted string as an attempt to provide an optional argument instead of a value and an error is returned, "error: argument -o/--options: expected one argument". It considers no value was provided for -o.
I simply want to append whatever is passed and not hinder the operators ability to use the underlying command. In below snippet, the options variable is supposed to be the quoted value of whatever.
cmd = ' '.join(["berks", "vendor", dir, options])
subprocess.call(cmd, cwd=entry.path, shell=True)
My parse argument is defined as below. I have also tried with type=str.
parser.add_argument("-o", "--options",
help="berks vendor or knife cookbook upload options passed directly to the command")
Is it possible to indicate to argparse to accept quoted values as is?

With your current definition, you can call your script with
./cookbooks.py vendor -o=--delete
Since -- doesn't start an argument, argparse doesn't see it as an option, only as part of a name-value pair to split on =.

Share dest between positional arguments and options

I have this very simple ArgumentParser instance, with an optional positional argument and an option, which writes a constant to the same destination:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', action='store_const', dest='path', const='<all>')
parser.add_argument('path', nargs='?')
# Prints None instead of '<all>'
print(parser.parse_args(['-a']).path)
But no matter what, parsing the command line ['-a'] does not yield a Namespace instance with path set to that constant. Instead, the default from the positional argument is used.
What am I doing wrong?
My use case is that the user should be able to specify a path (actually a list of paths). This list of paths defaults to the current working directory. But instead of using that default, -a can be passed, which should result in some configured root directory to be used. The full code for this part of the argument parser is this:
all_sentinel = object()
parser = argparse.ArgumentParser()
paths_group = parser.add_mutually_exclusive_group()
paths_group.add_argument('-a', action='store_const', dest='paths', const=all_sentinel)
paths_group.add_argument('paths', nargs='*', default=['.'])

A positional with nargs='?' has some special handling of its default (here None).
Normally defaults are assigned to the Namespace at the start of parsing, and overwritten by the actions, such as the optional.
Because an empty list of values satisfies the nargs, that positional is always 'seen'. But rather than assign [] or some other 'blank' to it, the parser assigns the default. So the positional's default overwrites the value set by '-a'.
nargs='*' gets the same kind of special handling.
I suspect that if you had another positional argument before the '-a', that you wouldn't see this effect. The '?*' positional would be processed before the '-a', and not overwrite its value.
Optionals are only processed if the flag occurs. Positionals are always processed, regardless of the nargs. The 'optional' positionals are processed, but with some extra handling of the defaults. But when they are processed relative to flagged arguments can vary.
That's some tricky behavior that I'm aware of simply because I've studied the code in detail, and answered a lot questions here and on the Python bug/issues.
Sharing the dest often does work, but that's more by default than design. It's the result of other design choices. argparse makes no promises in that regard. So if it isn't reliable, don't use it.

Argparse optional required arguments produce error if missing, but the action is still performed

I have the following snippet:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--p', required=True)
parser.add_argument('arg', action=MyAction)
parser.parse_args()
, where MyAction is a simple custom action class.
As you see, I want to enforce the presence of the p argument. However, my action is performed even if the argument is not present, and then an error message is shown that indicates the fact the the argument is missing.
Obviously, I could check in my action class for the presence of the p argument, but this defies the purpose of having a required parameter in the first place. Why is my action being run if the argument is not present?

Parsing is driven by the commandline strings, and tries to be order agnostic. Within those rules, it alternates between parsing an optional and a positional.
For example, with myprog --p one two:
'--p' - pass the 'one' string to the p action (e.g. setattr(namespace, 'p', 'one')
'two' - matches the nargs for 'arg'. Calls your MyAction.__call__ with values='one'.
at the end of parsing it checks if all required actions have been 'seen'. With your setup both '--p' and 'arg' are required.
With myprog two --p one it does the same, except arg is processed first. The namespace may have a default value for p.
With myprog two, arg is processed, and the required test will raise an error. error: the following arguments are required: --p
Since you have written a custom Action, you can easily explore how the namespace contents vary depending on the commandline arguments and their order.
So the --p and the arg will be processed independently, and in either order, depending on the commandline strings. required testing is performed at the end, using a list of seen_actions. And default values are set at the start of parsing. It is difficult to implement reliable inter-action tests within custom Actions. Usually it is better to perform such tests after parsing.
The defined Actions only change the args namespace. So parsing does not change anything beyond what it returns. Unless there's an error and it forces an sys.exit. A custom MyAction class can change that, but at your own risk.

How can I make argparse apply type conversion to default value of positional arguments?

argparse applies type conversion to arguments of options, and to the default values of these if the default values are strings. However, it seems it doesn't do so for positional arguments:
import argparse as ap
p = ap.ArgumentParser()
p.add_argument('file', nargs='*',
metavar='FILE',
default='-',
type=ap.FileType(),
help='Input files. - is stdin. Default: %(default)s')
print(p.parse_args([]))
# Namespace(file='-')
print(p.parse_args(['-']))
# Namespace(file=[<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>])
(Ideone)
If I change file to --file so that these are option arguments, the default value is converted as expected.
Do I have any options better than explicitly passing argparse.FileType()('-') as the default?

The values put into sys.argv are always str objects, because the underlying operating system construct is an array is C character pointers. You need type to convert them to some other Python type.
The value of default, however, is under no such restriction. You can use a value of any type you like for that keyword argument.
p.add_argument('file',
nargs='*',
metavar='FILE',
default=[sys.stdin],
type=ap.FileType(),
help='Input files. - is stdin. Default: standard input')
I modified the help; getting the value of repr(sys.stdin) isn't particular useful, and the user should not be confused by describing the default value rather than specifying an exact Python object.

Ignoring empty arguments for certain flags

I have a command that takes 1 argument and can take several flags.
#click.command()
#click.argument('item')
#click.option('--list-items', help='list items', is_flag=True)
def cli(item, list_items):
if list_items:
click.echo(ITEMS)
return
currently it returns:
Error: Missing argument "item".
How can I make so that I could access the functionality of --list-items even if I don't provide an argument? Just like --help flag does.

You'd have to make item optional:
#click.argument('item', required=False)
then do error handling in the function (e.g. raise a BadParameter() exeception).

You made it mandatory argument, make it optional by either adding required= False or by adding a default value to it