Determine indentation level of line currently running Python Code - python

Is it possible to determine the level of indentation of a line in Python while the program is running? I want to be able to organize a log file according to an outline structure of the script that is being run.
In the following example, the 'first message' function would yield 0, 'second message' would be 1, 'third message' 2 and 'fourth, fifth, sixth' message would be 0
logger.debug('First message')
if True:
logger.info('Second message')
if True:
logger.info('Third message')
logger.warning('Fourth message')
logger.error('Fifth message')
logger.critical('Sixth message')
The corresponding log file would look something like this:
First message
Second message
Third message
Fourth message
Fifth message
Sixth message


I was able to determine the indentation level using the inspect.getouterframes() function. This assumes that 4 ' ' characters are used instead of '\t' characters for indentation.
import inspect
def getIndentationLevel():
# get information about the previous stack frame
frame, filename, line_number,function_name, lines, index = inspect.getouterframes(inspect.currentframe())[1]
# count the number of characters in the line
original_length = len(lines[0])
# trim the line of left spaces and record the new length
new_length = len(lines[0].lstrip())
# take the difference between the two values and divide by 4 to get the level number
return int((original_length - new_length) / 4)


First, I retrieve the code context of the caller:
import inspect
context = inspect.getframeinfo(frame.f_back).code_context
This gives me a list of code lines; I ignore all but the first of those lines. Then I use a regular expression to get the whitespace at the start of this line; if you are using tabs, you have to replace them with the appropriate amount of spaces first:
import re
indent = re.compile("^ *")
firstline = context[0]
firstline = firstline.replace("\t", " ")
match = indent.match(firstline)
For testing purposes, I replace the spaces with dots, so I can see what's going on, and construct my prefix:
white = "." * match.span(0)[1] # Change "." to " "!
Now I can use this to modify my message msg I sent to my logger:
do_some_logging(white + msg)
It would be nice if I could simply wrap an existing logger with a decorator that turns this logger into a whitespace-aware logger. Suppose I have, for testing purposes, a really primitive logger that simply prints messages to stdout:
def pseudo_logger(msg):
"""Pseudo-logging function."""
print(msg)
This logger has a first parameter for the logging message, and potentially some other positional parameters and keywords. I want to write a decorater for such a function:
from functools import wraps
def indented(func):
"""Turns the logger func(msg) into an indented logger."""
#wraps(func)
def wrapped(msg, *args, **kwargs):
# ... compute white ...
func(white + msg, *args, **kwargs)
return wrapped
Now I can get a new logger as, for example:
new_logger = intented(pseudo_logger)
Putting it all together gives:
from functools import wraps
import inspect
import re
def indented(func):
"""Turns the logger func(msg) into an indented logger."""
indent = re.compile("^ *")
#wraps(func)
def wrapped(msg, *args, **kwargs):
frame = inspect.currentframe()
context = inspect.getframeinfo(frame.f_back).code_context
firstline = context[0]
match = indent.match(firstline)
white = "." * match.span(0)[1] # Change "." to " "!
func(white + msg, *args, **kwargs)
return wrapped
#indented
def pseudo_logger(msg):
"""Pseudo-logging function."""
print(msg)
def main():
pseudo_logger("This is an indented message!")
if True:
pseudo_logger("Another message, but more indented!")
pseudo_logger("This "
"will ignore the indention of the second "
"or third line.")
if __name__ == "__main__":
main()
I would be hesitant to use this in production code, though. Using code inspection like this is brittle, and depending on where you call this, it might lead to unexpected effects.

Related

Is it possible to get the source code of a (possibly decorated) Python function body, including inline comments? [duplicate]

This question already has answers here:
How can I get the source code of a Python function?
(13 answers)
Closed 1 year ago.
I am trying to figure out how to only get the source code of the body of the function.
Let's say I have:
def simple_function(b = 5):
a = 5
print("here")
return a + b
I would want to get (up to indentation):
"""
a = 5
print("here")
return a + b
"""
While it's easy in the case above, I want it to be agnostic of decorators/function headers, etc. However, still include inline comments. So for example:
#decorator1
#decorator2
def simple_function(b: int = 5):
""" Very sophisticated docs
"""
a = 5
# Comment on top
print("here") # And in line
return a + b
Would result in:
"""
a = 5
# Comment on top
print("here") # And in line
return a + b
"""
I was not able to find any utility and have been trying to play with inspect.getsourcelines for few hours now, but with no luck.
Any help appreciated!
Why is it different from How can I get the source code of a Python function?
This question asks for a whole function source code, which includes both decorators, docs, def, and body itself. I'm interested in only the body of the function.

I wrote a simple regex that does the trick. I tried this script with classes and without. It seemed to work fine either way. It just opens whatever file you designate in the Main call, at the bottom, rewrites the entire document with all function/method bodies doc-stringed and then save it as whatever you designated as the second argument in the Main call.
It's not beautiful, and it could probably have more efficient regex statements. It works though. The regex finds everything from a decorator (if one) to the end of a function/method, grouping tabs and the function/method body. It then uses those groups in finditer to construct a docstring and place it before the entire chunk it found.
import re
FUNC_BODY = re.compile(r'^((([ \t]+)?#.+\n)+)?(?P<tabs>[\t ]+)?def([^\n]+)\n(?P<body>(^([\t ]+)?([^\n]+)\n)+)', re.M)
BLANK_LINES = re.compile(r'^[ \t]+$', re.M)
class Main(object):
def __init__(self, file_in:str, file_out:str) -> None:
#prime in/out strings
in_txt = ''
out_txt = ''
#open resuested file
with open(file_in, 'r') as f:
in_txt = f.read()
#remove all lines that just have space characters on them
#this stops FUNC_BODY from finding the entire file in one shot
in_txt = BLANK_LINES.sub('', in_txt)
last = 0 #to keep track of where we are in the file
#process all matches
for m in FUNC_BODY.finditer(in_txt):
s, e = m.span()
#make sure we catch anything that was between our last match and this one
out_txt = f"{out_txt}{in_txt[last:s]}"
last = e
tabs = m.group('tabs') if not m.group('tabs') is None else ''
#construct the docstring and inject it before the found function/method
out_txt = f"{out_txt}{tabs}'''\n{m.group('body')}{tabs}'''\n{m.group()}"
#save as requested file name
with open(file_out, 'w') as f:
f.write(out_txt)
if __name__ == '__main__':
Main('test.py', 'test_docd.py')
EDIT:
Apparently, I "missed the entire point" so I wrote it again a different way. Now you can get the body while the code is running and decorators don't matter, at all. I left my other answer here because it is also a solution, just not a "real time" one.
import re, inspect
FUNC_BODY = re.compile('^(?P<tabs>[\t ]+)?def (?P<name>[a-zA-Z0-9_]+)([^\n]+)\n(?P<body>(^([\t ]+)?([^\n]+)\n)+)', re.M)
class Source(object):
#staticmethod
def investigate(focus:object, strfocus:str) -> str:
with open(inspect.getsourcefile(focus), 'r') as f:
for m in FUNC_BODY.finditer(f.read()):
if m.group('name') == strfocus:
tabs = m.group('tabs') if not m.group('tabs') is None else ''
return f"{tabs}'''\n{m.group('body')}{tabs}'''"
def decorator(func):
def inner():
print("I'm decorated")
func()
return inner
#decorator
def test():
a = 5
b = 6
return a+b
print(Source.investigate(test, 'test'))

print on same input line Python 3.x [duplicate]

I know I can stop print from writing a newline by adding a comma
print "Hello, world!",
# print("Hello, world!", end='') for Python 3.x
But how do I stop raw_input (or input for Python 3.x) from writing a newline?
print "Hello, ",
name = raw_input()
print ", how do you do?"
Result:
Hello, Tomas
, how do you do?
Result I want:
Hello, Tomas, how do you do?

But how do I stop raw_input from writing a newline?
In short: You can't.
raw_input() will always echo the text entered by the user, including the trailing newline. That means whatever the user is typing will be printed to standard output.
If you want to prevent this, you will have to use a terminal control library such as the curses module. This is not portable, though -- for example, curses in not available on Windows systems.

I see that nobody has given a working solution, so I decided I might give it a go.
As Ferdinand Beyer said, it is impossible to get raw_input() to not print a new line after the user input. However, it is possible to get back to the line you were before.
I made it into an one-liner. You may use:
print '\033[{}C\033[1A'.format(len(x) + y),
where x is an integer of the length of the given user input and y an integer of the length of raw_input()'s string. Though it might not work on all terminals (as I read when I learned about this method), it works fine on mine. I'm using Kubuntu 14.04.
The string '\033[4C' is used to jump 4 indexes to the right, so it would be equivalent to ' ' * 4. In the same way, the string '\033[1A' is used to jump 1 line up. By using the letters A, B, C or D on the last index of the string, you can go up, down, right and left respectively.
Note that going a line up will delete the existing printed character on that spot, if there is one.

This circumvents it, somewhat, but doesn't assign anything to variable name:
print("Hello, {0}, how do you do?".format(raw_input("Enter name here: ")))
It will prompt the user for a name before printing the entire message though.

You can use getpass instead of raw_input if you don't want it to make a new line!
import sys, getpass
def raw_input2(value="",end=""):
sys.stdout.write(value)
data = getpass.getpass("")
sys.stdout.write(data)
sys.stdout.write(end)
return data

An alternative to backtracking the newline is defining your own function that emulates the built-in input function, echoing and appending every keystroke to a response variable except Enter (which will return the response), whilst also handling Backspace, Del, Home, End, arrow keys, line history, KeyboardInterrupt, EOFError, SIGTSTP and pasting from the clipboard. It's very simple.
Note that on Windows, you'll need to install pyreadline if you want to use line history with the arrow keys like in the usual input function, although it's incomplete so the functionality is still not quite right. In addition, if you're not on v1511 or greater of Windows 10, you'll need to install the colorama module (if you're on Linux or macOS, nothing needs to be done).
Also, due to msvcrt.getwch using '\xe0' to indicate special characters, you won't be able to type 'à'. You should be able to paste it though.
Below is code that makes this work on updated Windows 10 systems (at least v1511), Debian-based Linux distros and maybe macOS and other *NIX operating systems. It should also work regardless of whether you have pyreadline installed on Windows, though it'll lack some functionality.
In windows_specific.py:
"""Windows-specific functions and variables for input_no_newline."""
import ctypes
from msvcrt import getwch # pylint: disable=import-error, unused-import
from shared_stuff import ANSI
try:
import colorama # pylint: disable=import-error
except ImportError:
kernel32 = ctypes.windll.kernel32
# Enable ANSI support to move the text cursor
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
else:
colorama.init()
def get_clipboard_data():
"""Return string previously copied from Windows clipboard.
Adapted from <http://stackoverflow.com/a/23285159/6379747>.
"""
CF_TEXT = 1
user32 = ctypes.windll.user32
user32.OpenClipboard(0)
try:
if user32.IsClipboardFormatAvailable(CF_TEXT):
data = user32.GetClipboardData(CF_TEXT)
data_locked = kernel32.GlobalLock(data)
text = ctypes.c_char_p(data_locked)
kernel32.GlobalUnlock(data_locked)
finally:
user32.CloseClipboard()
return text.value
def sigtstp():
"""Raise EOFError from Ctrl-Z since SIGTSTP doesn't exist on Windows."""
raise EOFError
input_code = {
**ANSI,
'CSI': [['\xe0', '\x00'], ''],
'up': 'H',
'down': 'P',
'right': 'M',
'left': 'K',
'end': 'O',
'home': 'G',
'backspace': '\b',
'del': 'S',
}
In unix_specific.py:
"""Functions and variables for Debian-based Linux distros and macOS."""
import sys
import os
import tty
import signal
import termios
from shared_stuff import ANSI
def getwch():
"""Return a single character from user input without echoing.
ActiveState code, adapted from
<http://code.activestate.com/recipes/134892> by Danny Yoo under
the Python Software Foundation license.
"""
file_descriptor = sys.stdin.fileno()
old_settings = termios.tcgetattr(file_descriptor)
try:
tty.setraw(file_descriptor)
char = sys.stdin.read(1)
finally:
termios.tcsetattr(file_descriptor, termios.TCSADRAIN, old_settings)
return char
def get_clipboard_data():
"""Return nothing; *NIX systems automagically change sys.stdin."""
return ''
def sigtstp():
"""Suspend the script."""
os.kill(os.getpid(), signal.SIGTSTP)
input_code = {
**ANSI,
'CSI': ['\x1b', '['],
'backspace': '\x7f',
'del': ['3', '~'],
}
In readline_available.py:
"""Provide functions for up and down arrows if readline is installed.
Basically to prevent duplicate code and make it work on systems without
readline.
"""
try:
import readline
except ImportError:
import pyreadline as readline
from shared_stuff import move_cursor
def init_history_index():
"""Return index for last element of readline.get_history_item."""
# readline.get_history_item is one-based
return readline.get_current_history_length() + 1
def restore_history(history_index, replaced, cursor_position):
"""Replace 'replaced' with history and return the replacement."""
try:
replacement = readline.get_history_item(history_index)
except IndexError:
replacement = None
if replacement is not None:
move_cursor('right', len(replaced) - cursor_position)
print('\b \b' * len(replaced), end='', flush=True)
print(replacement, end='', flush=True)
return replacement
return replaced
def store_and_replace_history(history_index, replacement, old_history):
"""Store history and then replace it."""
old_history[history_index] = readline.get_history_item(history_index)
try:
readline.replace_history_item(history_index - 1, replacement)
except AttributeError:
# pyreadline is incomplete
pass
def handle_prev_history(history_index, replaced, old_history,
input_replaced, history_modified):
"""Handle some up-arrow logic."""
try:
history = readline.get_history_item(history_index - 1)
except IndexError:
history = None
if history is not None:
if history_index > readline.get_current_history_length():
readline.add_history(replaced)
input_replaced = True
else:
store_and_replace_history(
history_index, replaced, old_history)
history_modified = True
history_index -= 1
return (history_index, input_replaced, history_modified)
def handle_next_history(history_index, replaced, old_history,
input_replaced, history_modified):
"""Handle some down-arrow logic."""
try:
history = readline.get_history_item(history_index + 1)
except IndexError:
history = None
if history is not None:
store_and_replace_history(history_index, replaced, old_history)
history_modified = True
history_index += 1
input_replaced = (not history_index
== readline.get_current_history_length())
return (history_index, input_replaced, history_modified)
def finalise_history(history_index, response, old_history,
input_replaced, history_modified):
"""Change history before the response will be returned elsewhere."""
try:
if input_replaced:
readline.remove_history_item(history_index - 1)
elif history_modified:
readline.remove_history_item(history_index - 1)
readline.add_history(old_history[history_index - 1])
except AttributeError:
# pyreadline is also missing remove_history_item
pass
readline.add_history(response)
In readline_unavailable.py:
"""Provide dummy functions for if readline isn't available."""
# pylint: disable-msg=unused-argument
def init_history_index():
"""Return an index of 1 which probably won't ever change."""
return 1
def restore_history(history_index, replaced, cursor_position):
"""Return the replaced thing without replacing it."""
return replaced
def store_and_replace_history(history_index, replacement, old_history):
"""Don't store history."""
pass
def handle_prev_history(history_index, replaced, old_history,
input_replaced, history_modified):
"""Return 'input_replaced' and 'history_modified' without change."""
return (history_index, input_replaced, history_modified)
def handle_next_history(history_index, replaced, old_history,
input_replaced, history_modified):
"""Also return 'input_replaced' and 'history_modified'."""
return (history_index, input_replaced, history_modified)
def finalise_history(history_index, response, old_history,
input_replaced, history_modified):
"""Don't change nonexistent history."""
pass
In shared_stuff.py:
"""Provide platform-independent functions and variables."""
ANSI = {
'CSI': '\x1b[',
'up': 'A',
'down': 'B',
'right': 'C',
'left': 'D',
'end': 'F',
'home': 'H',
'enter': '\r',
'^C': '\x03',
'^D': '\x04',
'^V': '\x16',
'^Z': '\x1a',
}
def move_cursor(direction, count=1):
"""Move the text cursor 'count' times in the specified direction."""
if direction not in ['up', 'down', 'right', 'left']:
raise ValueError("direction should be either 'up', 'down', 'right' "
"or 'left'")
# A 'count' of zero still moves the cursor, so this needs to be
# tested for.
if count != 0:
print(ANSI['CSI'] + str(count) + ANSI[direction], end='', flush=True)
def line_insert(text, extra=''):
"""Insert text between terminal line and reposition cursor."""
if not extra:
# It's not guaranteed that the new line will completely overshadow
# the old one if there is no extra. Maybe something was 'deleted'?
move_cursor('right', len(text) + 1)
print('\b \b' * (len(text)+1), end='', flush=True)
print(extra + text, end='', flush=True)
move_cursor('left', len(text))
And finally, in input_no_newline.py:
#!/usr/bin/python3
"""Provide an input function that doesn't echo a newline."""
try:
from windows_specific import getwch, get_clipboard_data, sigtstp, input_code
except ImportError:
from unix_specific import getwch, get_clipboard_data, sigtstp, input_code
try:
from readline_available import (init_history_index, restore_history,
store_and_replace_history,
handle_prev_history, handle_next_history,
finalise_history)
except ImportError:
from readline_unavailable import (init_history_index, restore_history,
store_and_replace_history,
handle_prev_history, handle_next_history,
finalise_history)
from shared_stuff import ANSI, move_cursor, line_insert
def input_no_newline(prompt=''): # pylint: disable=too-many-branches, too-many-statements
"""Echo and return user input, except for the newline."""
print(prompt, end='', flush=True)
response = ''
position = 0
history_index = init_history_index()
input_replaced = False
history_modified = False
replacements = {}
while True:
char = getwch()
if char in input_code['CSI'][0]:
char = getwch()
# Relevant input codes are made of two to four characters
if char == input_code['CSI'][1]:
# *NIX uses at least three characters, only the third is
# important
char = getwch()
if char == input_code['up']:
(history_index, input_replaced, history_modified) = (
handle_prev_history(
history_index, response, replacements, input_replaced,
history_modified))
response = restore_history(history_index, response, position)
position = len(response)
elif char == input_code['down']:
(history_index, input_replaced, history_modified) = (
handle_next_history(
history_index, response, replacements, input_replaced,
history_modified))
response = restore_history(history_index, response, position)
position = len(response)
elif char == input_code['right'] and position < len(response):
move_cursor('right')
position += 1
elif char == input_code['left'] and position > 0:
move_cursor('left')
position -= 1
elif char == input_code['end']:
move_cursor('right', len(response) - position)
position = len(response)
elif char == input_code['home']:
move_cursor('left', position)
position = 0
elif char == input_code['del'][0]:
if ''.join(input_code['del']) == '3~':
# *NIX uses '\x1b[3~' as its del key code, but only
# '\x1b[3' has currently been read from sys.stdin
getwch()
backlog = response[position+1 :]
response = response[:position] + backlog
line_insert(backlog)
elif char == input_code['backspace']:
if position > 0:
backlog = response[position:]
response = response[: position-1] + backlog
print('\b', end='', flush=True)
position -= 1
line_insert(backlog)
elif char == input_code['^C']:
raise KeyboardInterrupt
elif char == input_code['^D']:
raise EOFError
elif char == input_code['^V']:
paste = get_clipboard_data()
backlog = response[position:]
response = response[:position] + paste + backlog
position += len(paste)
line_insert(backlog, extra=paste)
elif char == input_code['^Z']:
sigtstp()
elif char == input_code['enter']:
finalise_history(history_index, response, replacements,
input_replaced, history_modified)
move_cursor('right', len(response) - position)
return response
else:
backlog = response[position:]
response = response[:position] + char + backlog
position += 1
line_insert(backlog, extra=char)
def main():
"""Called if script isn't imported."""
# "print(text, end='')" is equivalent to "print text,", and 'flush'
# forces the text to appear, even if the line isn't terminated with
# a '\n'
print('Hello, ', end='', flush=True)
name = input_no_newline() # pylint: disable=unused-variable
print(', how do you do?')
if __name__ == '__main__':
main()
As you can see, it's a lot of work for not that much since you need to deal with the different operating systems and basically reimplement a built-in function in Python rather than C. I'd recommend that you just use the simpler TempHistory class I made in another answer, which leaves all the complicated logic-handling to the built-in function.

Like Nick K. said, you'll need to move the text cursor back to before the newline was echoed. The problem is that you can't easily get the length of the previous line in order to move rightward, lest you store every string printed, prompted and inputted in its own variable.
Below is a class (for Python 3) that fixes this by automatically storing the last line from the terminal (provided you use its methods). The benefit of this compared to using a terminal control library is that it'll work in the standard terminal for both the latest version of Windows as well as *NIX operating systems. It'll also print the 'Hello, ' prompt before getting input.
If you're on Windows but not v1511 of Windows 10, then you'll need to install the colorama module or else this won't work, since they brought ANSI cursor movement support in that version.
# For the sys.stdout file-like object
import sys
import platform
if platform.system() == 'Windows':
try:
import colorama
except ImportError:
import ctypes
kernel32 = ctypes.windll.kernel32
# Enable ANSI support on Windows 10 v1511
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
else:
colorama.init()
else:
# Fix Linux arrow key support in Python scripts
import readline
class TempHistory:
"""Record one line from the terminal.
It is necessary to keep track of the last line on the terminal so we
can move the text cursor rightward and upward back into the position
before the newline from the `input` function was echoed.
Note: I use the term 'echo' to refer to when text is
shown on the terminal but might not be written to `sys.stdout`.
"""
def __init__(self):
"""Initialise `line` and save the `print` and `input` functions.
`line` is initially set to '\n' so that the `record` method
doesn't raise an error about the string index being out of range.
"""
self.line = '\n'
self.builtin_print = print
self.builtin_input = input
def _record(self, text):
"""Append to `line` or overwrite it if it has ended."""
if text == '':
# You can't record nothing
return
# Take into account `text` being multiple lines
lines = text.split('\n')
if text[-1] == '\n':
last_line = lines[-2] + '\n'
# If `text` ended with a newline, then `text.split('\n')[-1]`
# would have merely returned the newline, and not the text
# preceding it
else:
last_line = lines[-1]
# Take into account return characters which overwrite the line
last_line = last_line.split('\r')[-1]
# `line` is considered ended if it ends with a newline character
if self.line[-1] == '\n':
self.line = last_line
else:
self.line += last_line
def _undo_newline(self):
"""Move text cursor back to its position before echoing newline.
ANSI escape sequence: `\x1b[{count}{command}`
`\x1b` is the escape code, and commands `A`, `B`, `C` and `D` are
for moving the text cursor up, down, forward and backward {count}
times respectively.
Thus, after having echoed a newline, the final statement tells
the terminal to move the text cursor forward to be inline with
the end of the previous line, and then move up into said line
(making it the current line again).
"""
line_length = len(self.line)
# Take into account (multiple) backspaces which would
# otherwise artificially increase `line_length`
for i, char in enumerate(self.line[1:]):
if char == '\b' and self.line[i-1] != '\b':
line_length -= 2
self.print('\x1b[{}C\x1b[1A'.format(line_length),
end='', flush=True, record=False)
def print(self, *args, sep=' ', end='\n', file=sys.stdout, flush=False,
record=True):
"""Print to `file` and record the printed text.
Other than recording the printed text, it behaves exactly like
the built-in `print` function.
"""
self.builtin_print(*args, sep=sep, end=end, file=file, flush=flush)
if record:
text = sep.join([str(arg) for arg in args]) + end
self._record(text)
def input(self, prompt='', newline=True, record=True):
"""Return one line of user input and record the echoed text.
Other than storing the echoed text and optionally stripping the
echoed newline, it behaves exactly like the built-in `input`
function.
"""
if prompt == '':
# Prevent arrow key overwriting previously printed text by
# ensuring the built-in `input` function's `prompt` argument
# isn't empty
prompt = ' \b'
response = self.builtin_input(prompt)
if record:
self._record(prompt)
self._record(response)
if not newline:
self._undo_newline()
return response
record = TempHistory()
# For convenience
print = record.print
input = record.input
print('Hello, ', end='', flush=True)
name = input(newline=False)
print(', how do you do?)

As already answered, we can't stop input() from writing a newline. Though it may not satisfy your expectation, somehow the following codes satisfy the condition if -
you don't have any issue clearing the screen
import os
name = input("Hello, ")
os.system("cls") # on linux or mac, use "clear"
print(f"Hello, {name}, how do you do?")
or no issue using the gui dialog box, as dialog box disappears after taking user input, you will see exactly what you expected
import easygui
name = easygui.enterbox("Hello, ", "User Name")
print("Hello, "+name+", how do you do?")

I think you can use this:
name = input("Hello , ")

It should be something like this:-
print('this eliminates the ', end=' ')
print('new line')
The output is this:-
this eliminates the new line.

Proper way to wrap and unwrap a Python function?

I'm writing wrappers for the Python print function, but my question is more general - having wrapped a function, what's the proper way to un-wrap it?
This works, but I have two concerns about it:
class Iprint():
def __init__(self, tab=4, level=0):
''' Indented printer class.
tab controls number of spaces per indentation level (equiv. to tabstops)
level is the indentation level (0=none)'''
global print
self.tab = tab
self.level = level
self.old_print = print
print = self.print
def print(self, *args, end="\n", **kwargs):
indent = self.tab * self.level
self.old_print(" "*indent, end="", **kwargs)
self.old_print(*args, end=end, **kwargs)
indent = Iprint()
indent.level = 3
print("this should be indented")
print = indent.old_print
print("this shouldn't be indented")
My two concerns:
What happens if there's a second instantiation of the Iprint() class? This seems awkward and maybe something I ought to prevent - but how?
The 2nd to last line print = indent.old_print "unwraps" the print function, returning it to it's original function. This seems awkward too - what if it's forgotten?
I could do it in an __exit__ method but that would restrict the use of this to a with block - I think. Is there a better way?
What's the Pythonic way to do this?
(I also should mention that I anticipate having nested wrappers, which I thinks makes doing this properly more important...)

What it seems you are really trying to do here is find a way to override the builtin print function in a "pythonic" way.
While there is a way to do this, I do have a word of caution. One of the rules of "pythonic code" is
Explicit is better than implicit.
Overwriting print is inherently an implicit solution, and it would be more "pythonic" to allow for a custom print function to solve your needs.
However, let's assume we are talking about a use case where the best option available is to override print. For example, lets say you want to indent the output from the help() function.
You could override print directly, but you run the risk of causing unexpected changes you can't see.
For example:
def function_that_prints():
log_file = open("log_file.txt", "a")
print("This should be indented")
print("internally logging something", file = log_file)
log_file.close()
indent = Iprint()
indent.level = 3
function_that_prints() # now this internal log_file.txt has been corrupted
print = indent.old_print
This is bad, since presumably you just meant to change the output that is printed on screen, and not internal places where print may or may not be used.
Instead, you should just override the stdout, not print.
Python now includes a utility to do this called contextlib.redirect_stdout() documented here.
An implementation may look like this:
import io
import sys
import contextlib
class StreamIndenter(io.TextIOBase):
# io.TextIOBase provides some base functions, such as writelines()
def __init__(self, tab = 4, level = 1, newline = "\n", stream = sys.stdout):
"""Printer that adds an indent at the start of each line"""
self.tab = tab
self.level = level
self.stream = stream
self.newline = newline
self.linestart = True
def write(self, buf, *args, **kwargs):
if self.closed:
raise ValueError("write to closed file")
if not buf:
# Quietly ignore printing nothing
# prevents an issue with print(end='')
return
indent = " " * (self.tab * self.level)
if self.linestart:
# The previous line has ended. Indent this one
self.stream.write(indent)
# Memorize if this ends with a newline
if buf.endswith(self.newline):
self.linestart = True
# Don't replace the last newline, as the indent would double
buf = buf[:-len(self.newline)]
self.stream.write(buf.replace(self.newline, self.newline + indent))
self.stream.write(self.newline)
else:
# Does not end on a newline
self.linestart = False
self.stream.write(buf.replace(self.newline, self.newline + indent))
# Pass some calls to internal stream
#property
def writable(self):
return self.stream.writable
#property
def encoding(self):
return self.stream.encoding
#property
def name(self):
return self.stream.name
with contextlib.redirect_stdout(StreamIndenter()) as indent:
indent.level = 2
print("this should be indented")
print("this shouldn't be indented")
Overriding print this way both doesn't corrupt other uses of print and allows for proper handling of more complicated usages.
For example:
with contextlib.redirect_stdout(StreamIndenter()) as indent:
indent.level = 2
print("this should be indented")
indent.level = 3
print("more indented")
indent.level = 2
for c in "hello world\n": print(c, end='')
print()
print("\n", end='')
print(end = '')
print("this shouldn't be indented")
Formats correctly as:
this should be indented
more indented
hello world
this shouldn't be indented

I think I've solved this - at least to my own satisfaction. Here I've called the class T (for test):
class T():
old_print = None
def __init__(self, tab=4, level=0):
''' Indented printer class.
tab controls number of spaces per indentation level (equiv. to tabstops)
level is the indentation level (0=none)'''
T.tab = tab
T.level = level
self.__enter__()
def print(self, *args, end="\n", **kwargs):
indent = T.tab * T.level
T.old_print(" "*indent, end="", **kwargs)
T.old_print(*args, end=end, **kwargs)
def close(self):
if T.old_print is not None:
global print
print = T.old_print
T.old_print = None
def __enter__(self):
if T.old_print is None:
global print
T.old_print = print
print = self.print
def __exit__(self, exception_type, exception_value, exception_traceback):
self.close()
print("this should NOT be indented")
i = T(level=1)
print("level 1")
i2 = T(level=2)
print("level 2")
i.close()
print("this should not be indented")
i3 = T(level=3)
print("level 3")
i2.close()
print("not indented")
with i:
print("i")
print("after i")
with T(level=3):
print("T(level=3)")
print("after T(level=3)")
It silently forces a single (functional) instance of the class, regardless of how many times T() is called, as #MichaelButscher suggested (thanks; that was the most helpful comment by far).
It works cleanly with WITH blocks, and you can manually call the close method if not using WITH blocks.
The output is, as expected:
this should NOT be indented
level 1
level 2
this should not be indented
level 3
not indented
i
after i
T(level=3)
after T(level=3)

Using pexpect to detect and handle notifications from Bluetooth LE

I've been writing a python code to read values on Raspberry Pi 3 Model B received by Bluetooth LE.
I can read the correct values with:
child.sendline("char-read-hnd handle")
child.expect("Characteristic value/descripto: ",timeout=5)
What i am trying to do now is to check for notifications at any time, so i have a Thread that searches for the expected pattern "Notification handle=" like this:
def run():
patterns = ['Notification handle=','Indication handle=']
while True:
try:
matched_pattern_index = child.expect(patterns,timeout=1)
if matched_pattern_index in {0,1}:
print("Received Notification")
handleNotification()
except pexpect.TIMEOUT:
pass
Now during my main code, i am always doing some child.sendline to check for new values as well as child.expect . The problem is that my Thread call to pexpect.expect above blocks my others pexpect.expect in my code.
I already tried to do a second child similar to the first one to work inside the Thread but the result is the same.
So anyone has any idea how i can achieve this?
Any help would be appreciated.
Thanks in advance

I was thinking of subclassing pexpect.spawn and providing an expect_before(p,c) method that would save a list of patterns p and callback functions c, then override expect(p2) to prefix the list p onto the list p2 of that call before calling the real spawn.expect function.
If the real function then returns a match index i that is within the size of the list p, we can call function c[i] and loop again. When the index is beyond that list, we adjust it to be an index in list p2 and return from the call. Here's an approximation:
#!/usr/bin/python
# https://stackoverflow.com/q/51622025/5008284
from __future__ import print_function
import sys, pexpect
class Myspawn(pexpect.spawn):
def __init__(self, *args, **kwargs):
pexpect.spawn.__init__(self, *args, **kwargs)
def expect_before(self, patterns, callbacks):
self.eb_pat = patterns
self.eb_cb = callbacks
def expect(self, patlist, *args, **kwargs):
numbefore = len(self.eb_pat)
while True:
rc = pexpect.spawn.expect(self, self.eb_pat+patlist, *args, **kwargs)
if rc>=numbefore:
break
self.eb_cb[rc]() # call callback
return rc-numbefore
def test():
child = Myspawn("sudo bluetoothctl", logfile=sys.stdout)
patterns = ['Device FC:A8:9A:..:..:..', 'Device C8:FD:41:..:..:..']
callbacks = [lambda :handler1(), lambda :handler2()]
child.expect_before(patterns, callbacks)
child.sendline('scan on')
while True:
child.expect(['[bluetooth].*?#'], timeout=5)

How to print function arguments in sys.settrace?

How could I inspect the values of function arguments inside the sys.settrace call? It seems that I have possibility to output pretty much everything (lines, filenames, stacktraces, return values, etc) except arguments. Is there a workaround that would allow me to track function argument values as well?

You can use the combination of Code Objects and Frame Objects.
See for the descriptions of these in the Python Data-Model Reference.
import sys
def fn(frame, msg, arg):
if msg != 'call': return
# Filter as appropriate
if frame.f_code.co_filename.startswith("/usr"): return
print("Called", frame.f_code.co_name)
for i in range(frame.f_code.co_argcount):
name = frame.f_code.co_varnames[i]
print(" Argument", name, "is", frame.f_locals[name])
sys.settrace(fn)
def hai(a, b, c):
print(a, b, c)
hai("Hallo", "Welt", "!")
The crucial thing to realize is that
we can see all local variables in the frame as f_locals.
We can extract the names of the variables in the parameter list from f_code.co_varnames.

I turned Marcs answer into a script which can be used for inspecting other scripts:
print_func_calls.py:
#!/usr/bin/env python
import sys
# opt-out file names which start with one of these prefixes
FILENAME_FILTER = {"/usr", "<"}
# opt-in file names again which match one of these prefixes
FILENAME_UNFILTER = {"/lib/python/some-important-module"}
# opt-out function names
FN_NAME_FILTER = {"<module>", "__new__", "__setattr__", "<lambda>"}
def to_str(exp):
"""Turn an argument value into a string without dying on exceptions"""
try:
return repr(exp)[:100]
except Exception as exc:
return "n/a"
def fn(frame, msg, arg):
if msg != 'call':
return
filename, fn_name = frame.f_code.co_filename, frame.f_code.co_name
if (not all(not filename.startswith(p) for p in FILENAME_FILTER) and
all(not filename.startswith(p) for p in FILENAME_UNFILTER) or
fn_name in FN_NAME_FILTER):
return
argstr = ", ".join("%s=%s" % (
frame.f_code.co_varnames[i], to_str(frame.f_locals[frame.f_code.co_varnames[i]]))
for i in range(frame.f_code.co_argcount))
print(">>> %s::\033[37m%s\033[0m(%s)" % (filename, fn_name, argstr))
sys.settrace(fn)
sys.argv = sys.argv[1:]
exec(open(sys.argv[0]).read())
Use it like this:
print_func_calls.py my-script.py arg1..argN

Categories

Resources