Develop Reference

Python: Is there a way to have a setup of env. before calling a function and after that restore the previous env.?

My scenario:
I have a variable holding a link. e.g. REMOTE_API = "http://<site>/api/a/b/c"
This link stays the same all the time so it can be thought of as a constant.
It is used in many parts of the program.
But there are few parts of the program where the link needs to be changed e.g. REMOTE_API = "http://<site>/api/<user_name>/a/b/c" only if some condition is met. This condition is dictated by a config. file that may change without notice.
Is there a way to change the variable default before running a function and at the end of the function to switch back?
e.g.
#prepare_env(<if condition is met>)
def func():
<...>
call_api(REMOTE_API) # "http://<site>/api/<user_name>/a/b/c"
<...>
if __name__ == "__main__":
call_api_with_default(REMOTE_API) # REMOTE_API = "http://<site>/api/a/b/c"
func() # codition is met REMOTE_API = "http://<site>/api/<user_name>/a/b/c"
an_other_call_with_default(REMOTE_API) # REMOTE_API = "http://<site>/api/a/b/c"

You may write a function which takes a function and calls it, setting and restoring the environment variable as needed. E.g.:
#!/usr/bin/env python3
# content of env.py
import os
def call_with_env_var(f, var_name, var_value):
old_value = os.environ[var_name]
os.environ[var_name] = var_value
ret = f()
os.environ[var_name] = old_value
def print_var(name):
print(f'value of {name}: {os.environ[name]}')
if __name__ == '__main__':
print_var('HOME')
call_with_env_var(
lambda: print_var('HOME'),
'HOME',
'xyz'
)
print_var('HOME')
 
$ ./env.py
value of HOME: /home/etuardu
value of HOME: xyz
value of HOME: /home/etuardu
$ echo $HOME
/home/etuardu

Python: Calling functions from the same module

I am designing a module, say mymodule.py and I write the code for the module as follows:
def charCount(my_string, my_char):
a = my_string.count(my_char)
return a
def aCount(my_string):
a = charCount(my_string, 'a')
return a
Inside Python shell, I use the following command:
import mymodule as mm
and then,
mString = 'ghghghghgaaaaa'
and then
a = mm.aCount(mString)
It is seen that there is an error. Apparently, the function is not able to be called from the same module. How can this be avoided?

You need to put return statement in both the functions and it would work fine.
Try this:
def charCount(my_string, my_char):
a = my_string.count(my_char)
return a
def aCount(my_string):
a = charCount(my_string, 'a')
return a

How do I get the Python line number and file name of the point this function was called from? [duplicate]

In C++, I can print debug output like this:
printf(
"FILE: %s, FUNC: %s, LINE: %d, LOG: %s\n",
__FILE__,
__FUNCTION__,
__LINE__,
logmessage
);
How can I do something similar in Python?

There is a module named inspect which provides these information.
Example usage:
import inspect
def PrintFrame():
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename) # __FILE__ -> Test.py
print(info.function) # __FUNCTION__ -> Main
print(info.lineno) # __LINE__ -> 13
def Main():
PrintFrame() # for this line
Main()
However, please remember that there is an easier way to obtain the name of the currently executing file:
print(__file__)

For example
import inspect
frame = inspect.currentframe()
# __FILE__
fileName = frame.f_code.co_filename
# __LINE__
fileNo = frame.f_lineno
There's more here http://docs.python.org/library/inspect.html

Building on geowar's answer:
class __LINE__(object):
import sys
def __repr__(self):
try:
raise Exception
except:
return str(sys.exc_info()[2].tb_frame.f_back.f_lineno)
__LINE__ = __LINE__()
If you normally want to use __LINE__ in e.g. print (or any other time an implicit str() or repr() is taken), the above will allow you to omit the ()s.
(Obvious extension to add a __call__ left as an exercise to the reader.)

You can refer my answer:
https://stackoverflow.com/a/45973480/1591700
import sys
print sys._getframe().f_lineno
You can also make lambda function

I was also interested in a __LINE__ command in python.
My starting point was https://stackoverflow.com/a/6811020 and I extended it with a metaclass object. With this modification it has the same behavior like in C++.
import inspect
class Meta(type):
def __repr__(self):
# Inspiration: https://stackoverflow.com/a/6811020
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
# print(info.filename) # __FILE__ -> Test.py
# print(info.function) # __FUNCTION__ -> Main
# print(info.lineno) # __LINE__ -> 13
return str(info.lineno)
class __LINE__(metaclass=Meta):
pass
print(__LINE__) # print for example 18

wow, 7 year old question :)
Anyway, taking Tugrul's answer, and writing it as a debug type method, it can look something like:
def debug(message):
import sys
import inspect
callerframerecord = inspect.stack()[1]
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename, 'func=%s' % info.function, 'line=%s:' % info.lineno, message)
def somefunc():
debug('inside some func')
debug('this')
debug('is a')
debug('test message')
somefunc()
Output:
/tmp/test2.py func=<module> line=12: this
/tmp/test2.py func=<module> line=13: is a
/tmp/test2.py func=<module> line=14: test message
/tmp/test2.py func=somefunc line=10: inside some func

import inspect
.
.
.
def __LINE__():
try:
raise Exception
except:
return sys.exc_info()[2].tb_frame.f_back.f_lineno
def __FILE__():
return inspect.currentframe().f_code.co_filename
.
.
.
print "file: '%s', line: %d" % (__FILE__(), __LINE__())

Here is a tool to answer this old yet new question!
I recommend using icecream!
Do you ever use print() or log() to debug your code? Of course, you
do. IceCream, or ic for short, makes print debugging a little sweeter.
ic() is like print(), but better:
It prints both expressions/variable names and their values.
It's 40% faster to type.
Data structures are pretty printed.
Output is syntax highlighted.
It optionally includes program context: filename, line number, and parent function.
For example, I created a module icecream_test.py, and put the following code inside it.
from icecream import ic
ic.configureOutput(includeContext=True)
def foo(i):
return i + 333
ic(foo(123))
Prints
ic| icecream_test.py:6 in <module>- foo(123): 456

To get the line number in Python without importing the whole sys module...
First import the _getframe submodule:
from sys import _getframe
Then call the _getframe function and use its' f_lineno property whenever you want to know the line number:
print(_getframe().f_lineno) # prints the line number
From the interpreter:
>>> from sys import _getframe
... _getframe().f_lineno # 2
Word of caution from the official Python Docs:
CPython implementation detail: This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.
In other words: Only use this code for personal testing / debugging reasons.
See the Official Python Documentation on sys._getframe for more information on the sys module, and the _getframe() function / submodule.
Based on Mohammad Shahid's answer (above).

Preprocessing function text in runtime bofore compilation

I decided to try to preprocess function text before it's compilation into byte-code and following execution. This is merely for training. I hardly imagine situations where it'll be a satisfactory solution to be used. I have faced one problem which I wanted to solve in this way, but eventually a better way was found. So this is just for training and to learn something new, not for real usage.
Assume we have a function, which source code we want to be modified quite a bit before compilation:
def f():
1;a()
print('Some statements 1')
1;a()
print('Some statements 2')
Let, for example, mark some lines of it with 1;, for them to be sometimes commented and sometimes not. I just take it for example, modifications of the function may be different.
To comment these lines I made a decorator. The whole code it bellow:
from __future__ import print_function
def a():
print('a()')
def comment_1(s):
lines = s.split('\n')
return '\n'.join(line.replace(';','#;',1) if line.strip().startswith('1;') else line for line in lines)
def remove_1(f):
import inspect
source = inspect.getsource(f)
new_source = comment_1(source)
with open('temp.py','w') as file:
file.write(new_source)
from temp import f as f_new
return f_new
def f():
1;a()
print('Some statements 1')
1;a()
print('Some statements 2')
f = remove_1(f) #If decorator #remove is used above f(), inspect.getsource includes #remove inside the code.
f()
I used inspect.getsourcelines to retrieve function f code. Then I made some text-processing (in this case commenting lines starting with 1;). After that I saved it to temp.py module, which is then imported. And then a function f is decorated in the main module.
The output, when decorator is applied, is this:
Some statements 1
Some statements 2
when NOT applied is this:
a()
Some statements 1
a()
Some statements 2
What I don't like is that I have to use hard drive to load compiled function. Can it be done without writing it to temporary module temp.py and importing from it?
The second question is about placing decorator above f: #replace. When I do this, inspect.getsourcelines returns f text with this decorator. I could manually be deleted from f's text. but that would be quite dangerous, as there may be more than one decorator applied. So I resorted to the old-style decoration syntax f = remove_1(f), which does the job. But still, is it possible to allow normal decoration technique with #replace?

One can avoid creating a temporary file by invoking the exec statement on the source. (You can also explicitly call compile prior to exec if you want additional control over compilation, but exec will do the compilation for you, so it's not necessary.) Correctly calling exec has the additional benefit that the function will work correctly if it accesses global variables from the namespace of its module.
The problem described in the second question can be resolved by temporarily blocking the decorator while it is running. That way the decorator remains, along all the other ones, but is a no-op.
Here is the updated source.
from __future__ import print_function
import sys
def a():
print('a()')
def comment_1(s):
lines = s.split('\n')
return '\n'.join(line.replace(';','#;',1) if line.strip().startswith('1;') else line for line in lines)
_blocked = False
def remove_1(f):
global _blocked
if _blocked:
return f
import inspect
source = inspect.getsource(f)
new_source = comment_1(source)
env = sys.modules[f.__module__].__dict__
_blocked = True
try:
exec new_source in env
finally:
_blocked = False
return env[f.__name__]
#remove_1
def f():
1;a()
print('Some statements 1')
1;a()
print('Some statements 2')
f()
def remove_1(f):
import inspect
source = inspect.getsource(f)
new_source = comment_1(source)
env = sys.modules[f.__module__].__dict__.copy()
exec new_source in env
return env[f.__name__]

I'll leave a modified version of the solution given in the answer by user4815162342. It uses ast module to delete some parts of f, as was suggested in the comment to the question. To make it I majorly relied on the information in this article.
This implementation deletes all occurrences of a as standalone expression.
from __future__ import print_function
import sys
import ast
import inspect
def a():
print('a() is called')
_blocked = False
def remove_1(f):
global _blocked
if _blocked:
return f
import inspect
source = inspect.getsource(f)
a = ast.parse(source) #get ast tree of f
class Transformer(ast.NodeTransformer):
'''Will delete all expressions containing 'a' functions at the top level'''
def visit_Expr(self, node): #visit all expressions
try:
if node.value.func.id == 'a': #if expression consists of function with name a
return None #delete it
except(ValueError):
pass
return node #return node unchanged
transformer = Transformer()
a_new = transformer.visit(a)
f_new_compiled = compile(a_new,'<string>','exec')
env = sys.modules[f.__module__].__dict__
_blocked = True
try:
exec(f_new_compiled,env)
finally:
_blocked = False
return env[f.__name__]
#remove_1
def f():
a();a()
print('Some statements 1')
a()
print('Some statements 2')
f()
The output is:
Some statements 1
Some statements 2

Python pdb (debugger) disp equivalent?

Is there a pdb equivalent to disp in gdb?
E.g. when I'm debugging C using gdb I can have variables printed on every 'step' through the code by typing:
disp var
When I'm debugging python using pdb I would like similar functionality, but disp does not seem to be there, the python pdb documentation does not seem to offer an alternative - but it seems like an odd omission?

The code bellow uses Python introspection features to add two new commands to the PDB module 0
just put the given function, and its call in a separate module, and import this module before starting debugging - you should have the 'disp' and 'undisp' commands do add and retract watches to variables.
It works by monkeypatching Python's pdb module, which is written in pure python.
# -*- coding: utf-8 -*-
def patch_pdb():
import pdb
def wrap(func):
def new_postcmd(self, *args, **kw):
result = func(self, *args, **kw)
if hasattr(self, "curframe") and self.curframe and hasattr(self, "watch_list"):
for arg in self.watch_list:
try:
print >> self.stdout, "%s: %s"% (arg, self._getval(arg)) + ", ",
except:
pass
self.stdout.write("\n")
return result #func(self, *args, **kw)
return new_postcmd
pdb.Pdb.postcmd = wrap(pdb.Pdb.postcmd)
def do_disp(self, arg):
if not hasattr(self, "watch_list"):
self.watch_list = []
self.watch_list.append(arg)
pdb.Pdb.do_disp = do_disp
def do_undisp(self, arg):
if hasattr(self, "watch_list"):
try:
self.watch_list.remove(arg)
except:
pass
pdb.Pdb.do_undisp = do_undisp
patch_pdb()
if __name__ == "__main__":
# for testing
import pdb; pdb.set_trace()
a = 0
for i in range(10):
print i
a += 2
Unfortunately I could only make it display the state of the variables as they where previously to the execution of the last command. (I tried a little bit, but monkeypatching the bdb module, which is the base for the Pdb did not seem to work as well). You can try and change the methods in either pdb.Pdb, bdb.Bdb or cmd.Cmd that are decorated by wrap to find one that is called after the debugged frame state has changed.

You can set up some aliases that will do this for you:
alias n next;; p var
alias s step;; p var
Printing a whole list of variable names is left as an exercise to the reader. Unfortunately doing it this way means that when you send the debugger an empty line, the "last command" it executes is p var rather than, for example, n. If you want to fix that, then you can use this somewhat hacky set of Pdb commands instead:
!global __stack; from inspect import stack as __stack
!global __Pdb; from pdb import Pdb as __Pdb
!global __pdb; __pdb = [__framerec[0].f_locals.get("pdb") or __framerec[0].f_locals.get("self") for __framerec in __stack() if (__framerec[0].f_locals.get("pdb") or __framerec[0].f_locals.get("self")).__class__ == __Pdb][-1]
alias s step;; p var;; !__pdb.lastcmd = "!__pdb.cmdqueue.append('s')"
alias n next;; p var;; !__pdb.lastcmd = "!__pdb.cmdqueue.append('n')"

During the pdb debugging you can type normal python code, beyond the one letter commands - so just using print var should work for you.