Search code examples
pythonpython-3.xargparse

Argparse: options for subparsers override main if both share parent

I'm using argparse with several subparsers. I want my program to take options for verbosity anywhere in the args, including the subparser.

from argparse import ArgumentParser
p = ArgumentParser()
p.add_argument('--verbose', '-v', action='count')

sub = p.add_subparsers()
a = sub.add_parser('a')

print(p.parse_args())

By default, options for the main parser will throw an error if used for subparsers:

$ python tmp.py -v a
Namespace(verbose=1)

$ python tmp.py a -v
usage: tmp.py [-h] [--verbose] {a} ...
tmp.py: error: unrecognized arguments: -v

I looked into parent parsers, from this answer.

from argparse import ArgumentParser

parent = ArgumentParser(add_help=False)
parent.add_argument('--verbose', '-v', action='count')

main = ArgumentParser(parents=[parent])

sub = main.add_subparsers()
a = sub.add_parser('a', parents=[parent])

print(main.parse_args())

For some reason though, none of the shared flags work on the main parser.

$ python tmp2.py a -vvv
Namespace(verbose=3)
$ python tmp2.py -vvv a
Namespace(verbose=None)

Note that the main parser definitely has the appropriate arguments, because when I change it to main = ArgumentParser() I get error: unrecognized arguments: -v. What am I missing here?

Solution

  • First, a couple of general comments.

    The main parser handles the input upto the subparser invocation, then the subparser is called and given the remaining argv. When it is done, it's namespace is merged back into the the main namespace.

    The parents mechanism copies Actions from the parent by reference. So your main and subparsers share the same verbose Action object. That's been a problem when the subparser tries to set a different default or help. It may not be an issue here, but just keep it in mind.

    Even without the parents mechanism, sharing a dest or options flag between main and subparser can be tricky. Should the default of the subparser Action be used? What if both are used? Does the subparser overwrite the main parser's actions?

    Originally the main namespace was passed to the subparser, which it modified and returned. This was changed a while back (I can find the bug/issue if needed). Now the subparser starts with a default empty namespace, fills it. And these values are then merged into the main.

    So in your linked SO question, be wary of older answers. argparse may have changed since then.

    I think what's happening in your case is that the main and subparser verbose are counting separately. And when you get None it's the subparser's default that you see.

    The __call__ for _Count_Action is:

    def __call__(self, parser, namespace, values, option_string=None):
    new_count = _ensure_value(namespace, self.dest, 0) + 1
    setattr(namespace, self.dest, new_count)

    I suspect that in older argparse when the namespace was shared, the count would have been cumulative, but I can't test it without recreating an older style subparser action class.

    https://bugs.python.org/issue15327 - here the original developer suggests giving the two arguments different dest. That records the inputs from both main and sub. Your own code can then merge the results if needed.

    https://bugs.python.org/issue27859 argparse - subparsers does not retain namespace. Here I suggest a way of recreating the older style.

    https://bugs.python.org/issue9351 argparse set_defaults on subcommands should override top level set_defaults - this is the issue in 2014 that changed the namespace use.