Develop Reference

Python: Paging of argparse help text?

For a Python script that uses argparse and has a very long argument list, is it possible to make argparse page what it prints to the terminal when calling the script with the -h option?

I could not find a quick answer, so I wrote a little something:
# hello.py
import argparse
import os
import shlex
import stat
import subprocess as sb
import tempfile
def get_pager():
"""
Get path to your pager of choice, or less, or more
"""
pagers = (os.getenv('PAGER'), 'less', 'more',)
for path in (os.getenv('PATH') or '').split(os.path.pathsep):
for pager in pagers:
if pager is None:
continue
pager = iter(pager.split(' ', 1))
prog = os.path.join(path, next(pager))
args = next(pager, None) or ''
try:
md = os.stat(prog).st_mode
if md & (stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH):
return '{p} {a}'.format(p=prog, a=args)
except OSError:
continue
class CustomArgParser(argparse.ArgumentParser):
"""
A custom ArgumentParser class that prints help messages
using either your pager, or less or more, if available.
Otherwise, it does what ArgumentParser would do.
Use the PAGER environment variable to force it to use your pager
of choice.
"""
def print_help(self, file=None):
text = self.format_help()
pager = get_pager()
if pager is None:
return super().print_help(file)
fd, fname = tempfile.mkstemp(prefix='simeon_help_', suffix='.txt')
with open(fd, 'w') as fh:
super().print_help(fh)
cmd = shlex.split('{p} {f}'.format(p=pager, f=fname))
with sb.Popen(cmd) as proc:
rc = proc.wait()
if rc != 0:
super().print_help(file)
try:
os.unlink(fname)
except:
pass
if __name__ == '__main__':
parser = CustomArgParser(description='Some little program')
parser.add_argument('--message', '-m', help='Your message', default='hello world')
args = parser.parse_args()
print(args.message)
This snippet does main things. First, it defines a function to get the absolute path to a pager. If you set the environment variable PAGER, it will try and use it to display the help messages. Second, it defines a custom class that inherits pretty much everything from argparse.ArgumentParser. The only method that gets overridden here is print_help. It implements print_help by defaulting to super().print_help() whenever a valid pager is not found. If a valid is found, then it writes the help message to a temporary file and then opens a child process that invokes the pager with the path to the temporary file. When the pager returns, the temporary file is deleted. That's pretty much it.
You are more than welcome to update get_pager to add as many pager programs as you see fit.
Call the script:
python3 hello.py --help ## Uses less
PAGER='nano --view' python3 hello.py --help ## Uses nano
PAGER=more python3 hello.py --help ## Uses more

cli: how-to initialise a dict and group click functions

I would like to initialise a global variable, in this case a dict called DOC, after passing a number of command line arguments and using the click library.
I have tried the following:
#!/usr/bin/python3
import os
import sys
import yaml
import logging
import click
DOC = {}
#click.group()
def cli():
pass
#click.command()
#click.option("--logger-file", required=True, default='{}/blabla/cfg/logger.{}.yml'.format(os.environ['HOME'],os.path.basename(__file__)), show_default=True, help="YAML logging configuration file")
def cli_logger_file(logger_file):
if os.path.exists(logger_file):
try:
with open(logger_file, "rt") as f:
DOC = yaml.safe_load(f.read())
print( "logger" )
except Exception as e:
print( str(e) )
sys.exit()
else:
sys.exit()
if __name__ == '__main__':
cli_logger_file()
print( "hi!" )
print( DOC )
But when I run it, the output is:
$ python3 etc.py --logger-file=/home/blabla/cfg/logger.src.app.component.yml
logger
{}
Could you please help me understand:
Why I do not see hi! being printed?
Why if I replace #click.command() with #cli.command() it does not recognise the command-line option --logger-file?

A couple of misunderstandings about how click works.
Why I do not see hi! being printed?
Click is a framework for writing cli programs. After the framework calls your handlers, it does not return...
What is #click.group()?
This question:
Why if I replace #click.command() with #cli.command() it does not recognize the command-line option --logger-file ?
is related to what #click.group() does. A group is a special processor intended to implement sub commands. So in your case, using a group click will parse any --flags before the subcommand. But you don't have any subcommands so the --flags will be consumed by the group. Just remove the group as you don't need it.
Code:
#click.command()
#click.option("--logger-file",
default=os.path.join(os.path.expanduser("~"),
'blabla/cfg/logger.{}.yml'.format(
os.path.basename(__file__))),
show_default=True,
help="YAML logging configuration file")
def cli(logger_file):
if os.path.exists(logger_file):
try:
with open(logger_file, "rt") as f:
global DOC
DOC = yaml.safe_load(f.read())
except Exception as e:
click.echo(str(e))
sys.exit()
click.echo('DOC: %s' % DOC)
if __name__ == '__main__':
cli()
Notes:
You had set the --loggerfile to required but also specifying a default.
I used os.path.expanduser() instead of directly using an environment variable.
In setting the variable DOC, you need to tell python it is a global.
But, why a global? After you understand the answer to the first question at the top of this post, you will realize that any functionality that this program implements will need to be called from the the same function that you are parsing the yaml in. So, you likely should just pass it as a variable....

Assigning to a global variable from a function requires a global declaration.
Group commands are invoked by name, so when you use #cli.command you need to write:
$ python3 etc.py cli_logger_file --logger-file=foo.yml

Python multiprocessing without locking the parent process

OS:- Mac OSX
Python
I'm new to Multiprocessing with python. For my application, I want to open a new process from main and run it without locking the main process. for eg. I'm running process A and now i need to open a new application from A, lets call it process B. I want to open B in such a way that it does not blocks process A and still from Process i should be able to stop the process whenever i wish to.
Till now whatever code i have tried are basic and they lock the process A. and hence i'm unable to achieve it. Is there any workaround to do this ?
I read about fork and spawn but couldn't understand how can i use it to open an application. And i have tried threading also. But with no success. Can anyone tell me how can i do that ?
Currently I'm using subprocess.call() to open Mac Applications through Python.
It would be really helpful.
EDIT :-
I tried the accepted answer of this link
but to no avail. Because it would block the terminal and once we close the app manually it exits with output 0.
Also I have tried this solution. It would do the same. I want to make the the calling process not to be blocked by the called process.
while doing the same task in windows with os.system() gives me exactly what i want. But i don't know how can i do this in Mac.
EDIT 2: CODE
module 1:
import subprocess
def openCmd(name):
subprocess.call(["/usr/bin/open", "-W", "-n", "-a", "/Applications/"+name+".app"])
def closeCmd(name):
subprocess.call(['osascript', '-e', 'tell "'+name+'" to quit'])
main module:
import speech_recognition as sr
import pyttsx
import opCl
speech_engine = pyttsx.init('nsss')
speech_engine.setProperty('rate', 150)
OPEN_COGNATES=['open']
CLOSE_COGNATES=['close']
def speak(text):
speech_engine.say(text)
speech_engine.runAndWait()
re = sr.Recognizer()
def listen():
with sr.Microphone() as source:
re.adjust_for_ambient_noise(source)
while True:
speak("Say something!")
print ">>",
audio = re.listen(source)
try:
speak("now to recognise it,")
value=re.recognize_google(audio)
print (value)
speak("I heard you say {}".format(value))
value=value.strip().split()
name=" ".join(value[1:])
if value[0] in OPEN_COGNATES:
speak("opening "+name)
opCl.openCmd(name)
pass
elif value[0] in CLOSE_COGNATES:
speak("opening "+name)
opCl.closeCmd(name)
pass
else:
pass
except sr.UnknownValueError as e:
speak("Could not understand audio")
print ('Could not understand audio')
except sr.RequestError as e:
speak("can't recognise what you said.")
print ("can't recognise what you said")
if __name__=='__main__':
listen()

Comment: it gave a traceback. FileNotFoundError: [Errno 2] No such file or directory: 'leafpad'
As i wrote, I can't us "/usr/bin/open" and osascript, so my example uses 'leafpad'.
Have you tried replacing Popen([name]) with your
Popen(["/usr/bin/open", "-W", "-n", "-a", "/Applications/"+name+".app"])?
You must pass the same command args as you start it from the command line.
Read this: launch-an-app-on-os-x-with-command-line
Reread From Python » 3.6.1 Documentation subprocess.Popen
Note
shlex.split() can be useful when determining the correct tokenization for args, especially in complex cases:
Python » 3.6.1 Documentation:
subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)
Run the command described by args. Wait for command to complete, then return the returncode attribute
Your given "/usr/bin/open" and osascript didn't work for me.
From Python » 3.6.1 Documentation subprocess.Popen
NOWAIT example, for instance:
import subprocess
def openCmd(name):
subprocess.Popen([name])
def closeCmd(name):
subprocess.Popen(['killall', name])
if __name__ == '__main__':
while True:
key = input('input 1=open, 0=cloes, q=quit:')
if key == '1':
openCmd(('leafpad'))
if key == '0':
closeCmd('leafpad')
if key == 'q':
break
Note: Killing a process can lead to data loos and or other problems.
Tested with Python:3.4.2
I'm

Python using argparse with cmd

Is there a way to use the argparse module hooked in as the interpreter for every prompt in an interface inheriting from cmd?
I'd like for my cmd interface to interpret the typical line parameter in the same way one would interpret the options and arguments passed in at runtime on the bash shell, using optional arguments with - as well as positional arguments.

Well, one way to do that is to override cmd's default method and use it to parse the line with argparse, because all commands without do_ method in your cmd.Cmd subclass will fall through to use the default method. Note the additional _ before do_test to avoid it being used as cmd's command.
import argparse
import cmd
import shlex
class TestCLI(cmd.Cmd):
def __init__(self, **kwargs):
cmd.Cmd.__init__(self, **kwargs)
self.parser = argparse.ArgumentParser()
subparsers = self.parser.add_subparsers()
test_parser = subparsers.add_parser("test")
test_parser.add_argument("--foo", default="Hello")
test_parser.add_argument("--bar", default="World")
test_parser.set_defaults(func=self._do_test)
def _do_test(self, args):
print args.foo, args.bar
def default(self, line):
args = self.parser.parse_args(shlex.split(line))
if hasattr(args, 'func'):
args.func(args)
else:
cmd.Cmd.default(self, line)
test = TestCLI()
test.cmdloop()
argparse does a sys.exit if it encounters unknown commands, so you would need to override or monkey patch your ArgumentParser's error method to raise an exception instead of exiting and handle that in the default method, in order to stay in cmd's command loop.
I would suggest you look into cliff which allows you to write commands that can automatically be used both as argparse and cmd commands, which is pretty neat. It also supports loading commands from setuptools entry points, which allows you to distribute commands as plugins to your app. Note however, that cliff uses cmd2, which is cmd's more powerful cousin, but you can replace it cmd as cmd2 was developed as a drop-in replacement for cmd.

The straight forward way would be to create an argparse parser, and parse line.split() within your function, expecting SystemExit in case invalid arguments are supplied (parse_args() calls sys.exit() when it finds invalid arguments).
class TestInterface(cmd.Cmd):
__test1_parser = argparse.ArgumentParser(prog="test1")
__test1_parser.add_argument('--bar', help="bar help")
def help_test1(self): self.__test1_parser.print_help()
def do_test1(self, line):
try:
parsed = self.__test1_parser.parse_args(line.split())
except SystemExit:
return
print("Test1...")
print(parsed)
If invalid arguments are passed, parse_args() will print errors, and the program will return to the interface without exiting.
(Cmd) test1 --unk
usage: test1 [-h] [--bar BAR]
test1: error: unrecognized arguments: --unk
(Cmd)
Everything else should work the same as a regular argparse use case, also maintaining all of cmd's functionality (help messages, function listing, etc.)
Source: https://groups.google.com/forum/#!topic/argparse-users/7QRPlG97cak
Another way, which simplifies the setup above, is using the decorator below:
class ArgparseCmdWrapper:
def __init__(self, parser):
"""Init decorator with an argparse parser to be used in parsing cmd-line options"""
self.parser = parser
self.help_msg = ""
def __call__(self, f):
"""Decorate 'f' to parse 'line' and pass options to decorated function"""
if not self.parser: # If no parser was passed to the decorator, get it from 'f'
self.parser = f(None, None, None, True)
def wrapped_f(*args):
line = args[1].split()
try:
parsed = self.parser.parse_args(line)
except SystemExit:
return
f(*args, parsed=parsed)
wrapped_f.__doc__ = self.__get_help(self.parser)
return wrapped_f
#staticmethod
def __get_help(parser):
"""Get and return help message from 'parser.print_help()'"""
f = tempfile.SpooledTemporaryFile(max_size=2048)
parser.print_help(file=f)
f.seek(0)
return f.read().rstrip()
It makes defining additional commands simpler, where they take an extra parsed parameter that contains the result of a successful parse_args(). If there are any invalid arguments the function is never entered, everything being handled by the decorator.
__test2_parser = argparse.ArgumentParser(prog="test2")
__test2_parser.add_argument('--foo', help="foo help")
#WrapperCmdLineArgParser(parser=__test2_parser)
def do_test2(self, line, parsed):
print("Test2...")
print(parsed)
Everything works as the original example, including argparse generated help messages - without the need to define a help_command() function.
Source: https://codereview.stackexchange.com/questions/134333/using-argparse-module-within-cmd-interface

Simulating CLI Shell with python

I was asked to simulate CLI with Python.
This is what I did
def somefunction(a,b):
//codes here
//consider some other functions too
print "--- StackOverFlow Shell ---"
while True:
user_input = raw_input("#> ")
splitit = user_input.split(" ")
if splitit[0] == "add":
firstNum = splitit[1]
sNum = splitit[2]
result = somefunction(firstNum, sNum)
print result
//consider some other elif blocks with "sub", "div", etc
else:
print "Invalid Command"
I do also check the length of the list, here "splitit" I will allow only 3 argumets, first will be the operation, and second and third are the arguments with which some functions are to be performed, in case the argument is more than 3, for that i do put a check.
Though Somehow I manage to make it work, but is there a better way to achieve the same?

Use python CMD Module:
Check few examples given on the below pages
http://docs.python.org/library/cmd.html # Support for line-oriented command interpreters
http://www.doughellmann.com/PyMOTW/cmd - # Create line-oriented command processors
prompt can be set to a string to be printed each time the user is asked for a new command.
intro is the “welcome” message printed at the start of the program.
eg:
import cmd
class HelloWorld(cmd.Cmd):
"""Simple command processor example."""
prompt = 'prompt: '
intro = "Simple command processor example."

You should check out the VTE lib:
http://earobinson.wordpress.com/2007/09/10/python-vteterminal-example/
It works really well and you can very easily customize its look. This is how easy it is:
# make terminal
terminal = vte.Terminal()
terminal.connect ("child-exited", lambda term: gtk.main_quit())
terminal.fork_command()
# put the terminal in a scrollable window
terminal_window = gtk.ScrolledWindow()
terminal_window.set_policy(gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC)
terminal_window.add(terminal)