Develop Reference

Using click.command to make a function as a command

I am trying to make the function log into a command using the following code inside simple.py:
import click
#click.command()
#click.option('-v', '--verbose', count=True)
def log(verbose):
click.echo(f"Verbosity: {verbose}")
When I type the following on the command terminal:log -vvv , I get an error as : Command 'log' not found, but there are 16 similar ones.
#click.command should have converted the function log into a command? But, it doesn't work here. Could someone explain,please? Thanks!
I have tried the following commands:
log -vvv
Command 'log' not found, but there are 16 similar ones.
python3 simple.py log
Usage: simple.py [OPTIONS]
Try 'simple.py --help' for help.
Error: Got unexpected extra argument (log)
Could someone please explain what does #click.command() actually do and how's it different from running simple.py. The documentation does not make it very clear to me as well. Thanks!

import click
#click.command()
#click.option('-v', '--verbose', count=True)
def log(verbose):
click.echo(f"Verbosity: {verbose}")
if __name__ == '__main__':
log()
Then calling it like
$ python simple.py
Verbosity: 0
$ python simple.py -v
Verbosity: 1
The way you try to run it, suggest you think about command group, i.e. nesting commands
import click
#click.group()
def cli():
pass
#cli.command('log')
#click.option('-v', '--verbose', count=True)
def log(verbose):
click.echo(f"Verbosity: {verbose}")
#cli.command('greet')
def greet():
click.echo("Hello")
if __name__ == '__main__':
cli()
Then
$ python simple.py greet
Hello
$ python simple.py log -v -v # or just -vv
Verbosity: 2
Next step would be setuptools integration, etc.

How do I make a command string from click usable within a function in Python?

I have a script that runs commands from a command string from the click package.
from click import command
from kubernetes import client, config
from kubernetes.stream import stream
import os
from kubernetes.client.rest import ApiException
import json
import click
namespace = "epic-dev"
config.load_kube_config()
client1 = client.CoreV1Api(
api_client=config.new_client_from_config(context="eap-dev"))
def pod_exec(podname, namespace, command, api_instance):
pod = podname
container = pod.strip("-0")
name = pod
resp = None
exec_command = ["/bin/sh", "-c", command]
for pods in podlst:
command = """# Create roles to create roles 'listDatabase' & 'readChangeStream'
mongo -u admin -p """ + res[pods] + """ localhost:27017/admin <<-EOF
db.getUsers();
EOF"""
pod_exec(pods, namespace, command, client1)
this snippet of code works fine and sets the command as type click command.
However, when I try to put my code in a function such as:
def generate():
for pods in podlst:
command = """# Create roles to create roles 'listDatabase' & 'readChangeStream'
mongo -u admin -p """ + res[pods] + """ localhost:27017/admin <<-EOF
db.getUsers();
EOF"""
pod_exec(pods, namespace, command, client1)
It declares the command variable as a string localized to the function. I have tried doing,
click.command = ... and click.command("...")
but neither works.
I have this snippet placed above my command within another script:
#click.command()
#click.option('--gh-username', help="Github username", required=True)
and it seems to be interfering with my command.
Ultimately I would like to include the above two lines. I would also like my for loop function call to be placed within a function and have the command to be of type command from click.

Click unable to register group command

I am trying to run a click cli through a bash with a different command groups run through a single command collection.
src/preprocessing_extract_data/scripts/main.py
import click
#click.group()
def run_preprocessing_extract_data():
pass
#run_preprocessing_extract_data.command()
#click.option(
"--start_date",
type=click.DateTime(formats=["%Y-%m-%d"]),
required=True,
help="Start date for the pipeline",
)
#click.option(
"--end_date",
type=click.DateTime(formats=["%Y-%m-%d"]),
required=True,
help="End date for the pipeline",
)
def main(start_date, end_date):
...
if __name__ == "__main__":
main()
src/scripts/main.py
from click import CommandCollection
from src.preprocessing_extract_data.scripts.main import run_preprocessing_extract_data
if __name__ == "__main__":
cmds = [
run_preprocessing_extract_data,
# a few more similar command groups
]
cli = CommandCollection(sources=cmds)
cli()
scripts/entrypoint.sh
#!/bin/sh
start_date="$1"
end_date="$2"
python src/scripts/main.py run_preprocessing_extract_data --start_date=$start_date --end_date=$end_date
I run it using ./scripts/entrypoint.sh --start_date="2020-11-01" --end_date="2021-12-01" --today="2021-12-10" but it keeps failing and throws the following error:
Usage: main.py [OPTIONS] COMMAND [ARGS]...
Try 'main.py --help' for help.
Error: No such command 'run_preprocessing_extract_data'.

From the docs:
The default implementation for such a merging system is the CommandCollection class. It accepts a list of other multi commands and makes the commands available on the same level.
Hence, your script now has a command main; you can check this by running your script with --help (or no arguments at all): python src/scripts/main.py --help.
Hence you can do the following:
python src/scripts/main.py main --start_date="$start_date" --end_date="$end_date"
By the way, invoking your shell script should be done without the --start_date: ./scripts/entrypoint.sh "2020-11-01" "2021-12-01".

Python: Paging of argparse help text?

For a Python script that uses argparse and has a very long argument list, is it possible to make argparse page what it prints to the terminal when calling the script with the -h option?

I could not find a quick answer, so I wrote a little something:
# hello.py
import argparse
import os
import shlex
import stat
import subprocess as sb
import tempfile
def get_pager():
"""
Get path to your pager of choice, or less, or more
"""
pagers = (os.getenv('PAGER'), 'less', 'more',)
for path in (os.getenv('PATH') or '').split(os.path.pathsep):
for pager in pagers:
if pager is None:
continue
pager = iter(pager.split(' ', 1))
prog = os.path.join(path, next(pager))
args = next(pager, None) or ''
try:
md = os.stat(prog).st_mode
if md & (stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH):
return '{p} {a}'.format(p=prog, a=args)
except OSError:
continue
class CustomArgParser(argparse.ArgumentParser):
"""
A custom ArgumentParser class that prints help messages
using either your pager, or less or more, if available.
Otherwise, it does what ArgumentParser would do.
Use the PAGER environment variable to force it to use your pager
of choice.
"""
def print_help(self, file=None):
text = self.format_help()
pager = get_pager()
if pager is None:
return super().print_help(file)
fd, fname = tempfile.mkstemp(prefix='simeon_help_', suffix='.txt')
with open(fd, 'w') as fh:
super().print_help(fh)
cmd = shlex.split('{p} {f}'.format(p=pager, f=fname))
with sb.Popen(cmd) as proc:
rc = proc.wait()
if rc != 0:
super().print_help(file)
try:
os.unlink(fname)
except:
pass
if __name__ == '__main__':
parser = CustomArgParser(description='Some little program')
parser.add_argument('--message', '-m', help='Your message', default='hello world')
args = parser.parse_args()
print(args.message)
This snippet does main things. First, it defines a function to get the absolute path to a pager. If you set the environment variable PAGER, it will try and use it to display the help messages. Second, it defines a custom class that inherits pretty much everything from argparse.ArgumentParser. The only method that gets overridden here is print_help. It implements print_help by defaulting to super().print_help() whenever a valid pager is not found. If a valid is found, then it writes the help message to a temporary file and then opens a child process that invokes the pager with the path to the temporary file. When the pager returns, the temporary file is deleted. That's pretty much it.
You are more than welcome to update get_pager to add as many pager programs as you see fit.
Call the script:
python3 hello.py --help ## Uses less
PAGER='nano --view' python3 hello.py --help ## Uses nano
PAGER=more python3 hello.py --help ## Uses more

python click framework - custom multi command implementation in oop's method

I have written some scripts that I'm trying to integrate with click. All the scripts are written in python OOP's.
The issue is that i am trying to build command section in oop's way but couldn't do it properly.
let me show you, what i am trying to do and please note that i am sharing here dummy code it is very similar to the real code.
First thing the directory structure:
-customcmd <dir>
|
|->commands <dir>
| -> abc-command.py
| -> __init__.py
|
|->__init__.py
|->main.py
|->setup.py
1) I have created one file called main.py, which contains following code:
import click
import os
plugin_folder = os.path.join(os.path.dirname(__file__), 'commands')
class MyCLI(click.MultiCommand):
def list_commands(self, ctx):
rv = []
for filename in os.listdir(plugin_folder):
if filename.startswith('__'):
continue
if filename.endswith('.py'):
rv.append(filename[:-3])
rv.sort()
return rv
def get_command(self, ctx, name):
ns = {}
fn = os.path.join(plugin_folder, name + '.py')
with open(fn) as f:
code = compile(f.read(), fn, 'exec')
eval(code, ns, ns)
return ns['cli']
cli = MyCLI()#help='This tool\'s subcommands are loaded from a ''plugin folder dynamically.'
if __name__ == '__main__':
cli()
2) abc-command.py
#click.command()
#click.option("--file-loc", '-fl', type=open, required=True, default=None, help="Path to the file")
def cli(file_loc):
"""
This is test command
"""
print("Path to file {}".format(file_loc))
Output of above code when you call main.py:
(automation) K:\Pythonenv\automation\customcmd>python main.py
Usage: main.py [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
policyresult This is test command
Output of above code when you call sub command:
(automation) K:\Pythonenv\automation\customcmd>python main.py policyresult --help
Usage: main.py policyresult [OPTIONS]
This is test command
Options:
-fl, --file-loc OPEN Path to the file [required]
--help Show this message and exit.
3) This is how I converted the procedural code of abc-command.py code:
class policyresult():
def __init__(self):
pass
#click.command()
#click.option("--file-loc", '-fl', type=open, required=True, default=None, help="Path to the file")
def cli(self,file_loc):
"""
This is test command
"""
print("Path to file {}".format(file_loc))
obj = policyresult()
obj.cli()
Output of above code doesn't match with the previous output when the code was procedural in abc-command.py:
Here i am calling the main.py
(automation) K:\Pythonenv\automation\customcmd>python main.py
Usage: main.py [OPTIONS]
Try "main.py --help" for help.
Error: Missing option "--file-loc" / "-fl".
In the above output you can see it is directly going into the sub-command options things and giving error as well.
As far as i understand list_commands() which is in main.py can't list out the commands, this part i can't understand why it is not working properly.
I tried various things but couldn't find the proper way to implement OOP's in abc-command.py because of that my ouput doesn't match.
I am new to this click framework, please suggest any new changes in my approach if needed.
please look into this, sorry for this weird way to explaining this.

abc-command.py is evaluated before click parses command options because of this line in the file invoking the cli method:
obj.cli()
Also, in the get_command method implemented for the multi-command, commands are supposed to expose a 'cli' name in their namespace.
To fix this error, in abc-command.py update the line invoking the cli command with:
cli = obj.cli
so that a cli name is exposed in abc-command.py module