I find myself doing something like this constantly to pull GET args into vars:
some_var = self.request.get('some_var', None)
other_var = self.request.get('other_var', None)
if None in [some_var, other_var]:
logging.error("some arg was missing in " + self.request.path)
exit()
What I would really want to do is:
pull_args('some_var', 'other_var')
And that would somehow pull these variables to be available in current scope, or log an error and exit if not (or return to calling method if possible). Is this possible in Python?
First, a disclaimer: "pulling" variables into the local scope in any way other than var = something is really really really not recommended. It tends to make your code really confusing for someone who isn't intimately familiar with what you're doing (i.e. anyone who isn't you, or who is you 6 months in the future, etc.)
That being said, for educational purposes only, there is a way. Your pull_args function could be implemented like this:
def pull_args(request, *args):
pulled = {}
try:
for a in args:
pulled[a] = request[a]
except AttributeError:
logging.error("some arg was missing in " + self.request.path)
exit()
else:
caller = inspect.stack()[1][0]
caller.f_locals.update(pulled)
At least, something to that effect worked when I came up with it probably about a year ago. I wouldn't necessarily count on it continuing to work in future Python versions. (Yet another reason not to do it) I personally have never found a good reason to use this code snippet.
No it's not and also pointless. Writing to outer namespaces completely destroys the purpose of namespaces, which is having only the things around that you explicitly set. Use lists!
def pull_args(*names):
return [self.request.get(name, None) for name in names]
print None in pull_args('some_var', 'other_var')
Probably this works too, to check if all _var are set:
print all(name in self.request for name in ('some_var', 'other_var'))
Related
This question is similar to others asked on here, but after reading the answers I'm not grasping it and would appreciate further guidance.
While sketching new code I find myself adding a lot of statements like:
print('var=')
pprint(var)
It became tedious always writing that, so I thought I could make it into a function. Since I want to print the variable name on the preceding line, I tried:
def dbp(var):
eval('print(\'{0}=\')'.format(var))
eval('pprint({0})'.format(var))
so then I do do things like:
foo = 'bar'
dbp('foo')
which prints
foo=
'bar'
This is all great, but when I go to use it in a function things get messed up. For example, doing
def f():
a = ['123']
dbp('a')
f()
raises a NameError (NameError: name 'a' is not defined).
My expectation was that dbp() would have read access to anything in f()'s scope, but clearly it doesn't. Can someone explain why?
Also, better ways of printing a variable's name followed by its formatted contents are also appreciated.
You really should look at other ways to doing this.
The logging module is a really good habit to get into, and you can turn off and on debug output.
Python 3.6 has f'' strings so you would simplify this to:
pprint(f'var=\n{var}`)`
However, here's an example (not recommended) using locals():
In []:
def dbp(var, l):
print('{}='.format(var))
pprint(l[var])
def f():
a = 1
dbp('a', locals())
f()
Out[]:
a=
1
first of all, id like to say that eval is a high security risk for whoever is going to be running that code.
However, if you absolutely must, you can do this.
def dbp(var):
env = {'var': var}
# Adding global variables to the enviroment
env.update(globals())
eval("print('{0}=')".format(var))
eval('pprint(var)', env)
def f():
a = ['123']
dbp('a')
you can then do
>>> f()
a=
'a'
I have found some vaguely related questions to this question, but not any clean and specific solution for CPython. And I assume that a "valid" solution is interpreter specific.
First the things I think I understand:
locals() gives a non-modifiable dictionary.
A function may (and indeed does) use some kind of optimization to access its local variables
frame.f_locals gives a locals() like dictionary, but less prone to hackish things through exec. Or at least I have been less able to do hackish undocumented things like the locals()['var'] = value ; exec ""
exec is capable to do weird things to the local variables, but it is not reliable --e.g. I read somewhere that it doesn't work in Python 3. Haven't tested.
So I understand that, given those limitations, it will never be safe to add extra variables to the locals, because it breaks the interpreter structure.
However, it should be possible to change a variable already existing, isn't it?
Things that I considered
In a function f, one can access the f.func_code.co_nlocals and f.func_code.co_varnames.
In a frame, the variables can be accessed / checked / read through the frame.f_locals. This is in the use case of setting a tracer through sys.settrace.
One can easily access the function in which a frame is --cosidering the use case of setting a trace and using it to "do things" in with the local variables given a certain trigger or whatever.
The variables should be somewhere, preferably writeable... but I am not capable of finding it. Even if it is an array (for interpreter efficient access), or I need some extra C-specific wiring, I am ready to commit to it.
How can I achieve that modification of variables from a tracer function or from a decorated wrapped function or something like that?
A full solution will be of course appreciated, but even some pointers will help me greatly, because I'm stuck here with lots of non writeable dictionaries :-/
Edit: Hackish exec is doing things like this or this
It exists an undocumented C-API call for doing things like that:
PyFrame_LocalsToFast
There is some more discussion in this PyDev blog post. The basic idea seems to be:
import ctypes
...
frame.f_locals.update({
'a': 'newvalue',
'b': other_local_value,
})
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame), ctypes.c_int(0))
I have yet to test if this works as expected.
Note that there might be some way to access the Fast directly, to avoid an indirection if the requirements is only modification of existing variable. But, as this seems to be mostly non-documented API, source code is the documentation resource.
Based on the notes from MariusSiuram, I wrote a recipe that show the behavior.
The conclusions are:
we can modify an existing variable
we can delete an existing variable
we can NOT add a new variable.
So, here is the code:
import inspect
import ctypes
def parent():
a = 1
z = 'foo'
print('- Trying to add a new variable ---------------')
hack(case=0) # just try to add a new variable 'b'
print(a)
print(z)
assert a == 1
assert z == 'foo'
try:
print (b)
assert False # never is going to reach this point
except NameError, why:
print("ok, global name 'b' is not defined")
print('- Trying to remove an existing variable ------')
hack(case=1)
print(a)
assert a == 2
try:
print (z)
except NameError, why:
print("ok, we've removed the 'z' var")
print('- Trying to update an existing variable ------')
hack(case=2)
print(a)
assert a == 3
def hack(case=0):
frame = inspect.stack()[1][0]
if case == 0:
frame.f_locals['b'] = "don't work"
elif case == 1:
frame.f_locals.pop('z')
frame.f_locals['a'] += 1
else:
frame.f_locals['a'] += 1
# passing c_int(1) will remove and update variables as well
# passing c_int(0) will only update
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(frame),
ctypes.c_int(1))
if __name__ == '__main__':
parent()
The output would be like:
- Trying to add a new variable ---------------
1
foo
ok, global name 'b' is not defined
- Trying to remove an existing variable ------
2
foo
- Trying to update an existing variable ------
3
[EDIT 00]: I've edited several times the post and now even the title, please read below.
I just learned about the format string method, and its use with dictionaries, like the ones provided by vars(), locals() and globals(), example:
name = 'Ismael'
print 'My name is {name}.'.format(**vars())
But I want to do:
name = 'Ismael'
print 'My name is {name}.' # Similar to ruby
So I came up with this:
def mprint(string='', dictionary=globals()):
print string.format(**dictionary)
You can interact with the code here:
http://labs.codecademy.com/BA0B/3#:workspace
Finally, what I would love to do is to have the function in another file, named my_print.py, so I could do:
from my_print import mprint
name= 'Ismael'
mprint('Hello! My name is {name}.')
But as it is right now, there is a problem with the scopes, how could I get the the main module namespace as a dictionary from inside the imported mprint function. (not the one from my_print.py)
I hope I made myself uderstood, if not, try importing the function from another module. (the traceback is in the link)
It's accessing the globals() dict from my_print.py, but of course the variable name is not defined in that scope, any ideas of how to accomplish this?
The function works if it's defined in the same module, but notice how I must use globals() because if not I would only get a dictionary with the values within mprint() scope.
I have tried using nonlocal and dot notation to access the main module variables, but I still can't figure it out.
[EDIT 01]: I think I've figured out a solution:
In my_print.py:
def mprint(string='',dictionary=None):
if dictionary is None:
import sys
caller = sys._getframe(1)
dictionary = caller.f_locals
print string.format(**dictionary)
In test.py:
from my_print import mprint
name = 'Ismael'
country = 'Mexico'
languages = ['English', 'Spanish']
mprint("Hello! My name is {name}, I'm from {country}\n"
"and I can speak {languages[1]} and {languages[0]}.")
It prints:
Hello! My name is Ismael, I'm from Mexico
and I can speak Spanish and English.
What do you think guys? That was a difficult one for me!
I like it, much more readable for me.
[EDIT 02]: I've made a module with an interpolate function, an Interpolate class and an attempt for a interpolate class method analogous to the function.
It has a small test suite and its documented!
I'm stuck with the method implementation, I don't get it.
Here's the code: http://pastebin.com/N2WubRSB
What do you think guys?
[EDIT 03]: Ok I have settled with just the interpolate() function for now.
In string_interpolation.py:
import sys
def get_scope(scope):
scope = scope.lower()
caller = sys._getframe(2)
options = ['l', 'local', 'g', 'global']
if scope not in options[:2]:
if scope in options[2:]:
return caller.f_globals
else:
raise ValueError('invalid mode: {0}'.format(scope))
return caller.f_locals
def interpolate(format_string=str(),sequence=None,scope='local',returns=False):
if type(sequence) is str:
scope = sequence
sequence = get_scope(scope)
else:
if not sequence:
sequence = get_scope(scope)
format = 'format_string.format(**sequence)'
if returns is False:
print eval(format)
elif returns is True:
return eval(format)
Thanks again guys! Any opinions?
[EDIT 04]:
This is my last version, it has a test, docstrings and describes some limitations I've found:
http://pastebin.com/ssqbbs57
You can quickly test the code here:
http://labs.codecademy.com/BBMF#:workspace
And clone grom git repo here:
https://github.com/Ismael-VC/python_string_interpolation.git
Modules don't share namespaces in python, so globals() for my_print is always going to be the globals() of my_print.py file ; i.e the location where the function was actually defined.
def mprint(string='', dic = None):
dictionary = dic if dic is not None else globals()
print string.format(**dictionary)
You should pass the current module's globals() explicitly to make it work.
Ans don't use mutable objects as default values in python functions, it can result in unexpected results. Use None as default value instead.
A simple example for understanding scopes in modules:
file : my_print.py
x = 10
def func():
global x
x += 1
print x
file : main.py
from my_print import *
x = 50
func() #prints 11 because for func() global scope is still
#the global scope of my_print file
print x #prints 50
Part of your problem - well, the reason its not working - is highlighted in this question.
You can have your function work by passing in globals() as your second argument, mprint('Hello my name is {name}',globals()).
Although it may be convenient in Ruby, I would encourage you not to write Ruby in Python if you want to make the most out of the language.
Language Design Is Not Just Solving Puzzles: ;)
http://www.artima.com/forums/flat.jsp?forum=106&thread=147358
Edit: PEP-0498 solves this issue!
The Template class from the string module, also does what I need (but more similar to the string format method), in the end it also has the readability I seek, it also has the recommended explicitness, it's in the Standard Library and it can also be easily customized and extended.
http://docs.python.org/2/library/string.html?highlight=template#string.Template
from string import Template
name = 'Renata'
place = 'hospital'
job = 'Dr.'
how = 'glad'
header = '\nTo Ms. {name}:'
letter = Template("""
Hello Ms. $name.
I'm glad to inform, you've been
accepted in our $place, and $job Red
will ${how}ly recieve you tomorrow morning.
""")
print header.format(**vars())
print letter.substitute(vars())
The funny thing is that now I'm getting more fond of using {} instead of $ and I still like the string_interpolation module I came up with, because it's less typing than either one in the long run. LOL!
Run the code here:
http://labs.codecademy.com/BE3n/3#:workspace
Consider this example:
def outer():
s_outer = "outer\n"
def inner():
s_inner = "inner\n"
do_something()
inner()
I want the code in do_something to be able to access the variables of the calling functions further up the call stack, in this case s_outer and s_inner. More generally, I want to call it from various other functions, but always execute it in their respective context and access their respective scopes (implement dynamic scoping).
I know that in Python 3.x, the nonlocal keyword allows access to s_outer from within inner. Unfortunately, that only helps with do_something if it's defined within inner. Otherwise, inner isn't a lexically enclosing scope (similarly, neither is outer, unless do_something is defined within outer).
I figured out how to inspect stack frames with the standard library inspect, and made a small accessor that I can call from within do_something() like this:
def reach(name):
for f in inspect.stack():
if name in f[0].f_locals:
return f[0].f_locals[name]
return None
and then
def do_something():
print( reach("s_outer"), reach("s_inner") )
works just fine.
Can reach be implemented more simply? How else can I solve the problem?
There is no and, in my opinion, should be no elegant way of implementing reach since that introduces a new non-standard indirection which is really hard to comprehend, debug, test and maintain. As the Python mantra (try import this) says:
Explicit is better than implicit.
So, just pass the arguments. You-from-the-future will be really grateful to you-from-today.
What I ended up doing was
scope = locals()
and make scope accessible from do_something. That way I don't have to reach, but I can still access the dictionary of local variables of the caller. This is quite similar to building a dictionary myself and passing it on.
We can get naughtier.
This is an answer to the "Is there a more elegant/shortened way to implement the reach() function?" half of the question.
We can give better syntax for the user: instead of reach("foo"), outer.foo.
This is nicer to type, and the language itself immediately tells you if you used a name that can't be a valid variable (attribute names and variable names have the same constraints).
We can raise an error, to properly distinguish "this doesn't exist" from "this was set to None".
If we actually want to smudge those cases together, we can getattr with the default parameter, or try-except AttributeError.
We can optimize: no need to pessimistically build a list big enough for all the frames at once.
In most cases we probably won't need to go all the way to the root of the call stack.
Just because we're inappropriately reaching up stack frames, violating one of the most important rules of programming to not have things far away invisibly effecting behavior, doesn't mean we can't be civilized.
If someone is trying to use this Serious API for Real Work on a Python without stack frame inspection support, we should helpfully let them know.
import inspect
class OuterScopeGetter(object):
def __getattribute__(self, name):
frame = inspect.currentframe()
if frame is None:
raise RuntimeError('cannot inspect stack frames')
sentinel = object()
frame = frame.f_back
while frame is not None:
value = frame.f_locals.get(name, sentinel)
if value is not sentinel:
return value
frame = frame.f_back
raise AttributeError(repr(name) + ' not found in any outer scope')
outer = OuterScopeGetter()
Excellent. Now we can just do:
>>> def f():
... return outer.x
...
>>> f()
Traceback (most recent call last):
...
AttributeError: 'x' not found in any outer scope
>>>
>>> x = 1
>>> f()
1
>>> x = 2
>>> f()
2
>>>
>>> def do_something():
... print(outer.y)
... print(outer.z)
...
>>> def g():
... y = 3
... def h():
... z = 4
... do_something()
... h()
...
>>> g()
3
4
Perversion elegantly achieved.
Is there a better way to solve this problem? (Other than wrapping the respective data into dicts and pass these dicts explicitly to do_something())
Passing the dicts explicitly is a better way.
What you're proposing sounds very unconventional. When code increases in size, you have to break down the code into a modular architecture, with clean APIs between modules. It also has to be something that is easy to comprehend, easy to explain, and easy to hand over to another programmer to modify/improve/debug it. What you're proposing sounds like it is not a clean API, unconventional, with a non-obvious data flow. I suspect it would probably make many programmers grumpy when they saw it. :)
Another option would be to make the functions members of a class, with the data being in the class instance. That could work well if your problem can be modelled as several functions operating on the data object.
I have a basic "best practices" Python question. I see that there are already StackOverflow answers tangentially related to this question but they're mired in complicated examples or involve multiple factors.
Given this code:
#!/usr/bin/python
def test_function():
try:
a = str(5)
raise
b = str(6)
except:
print b
test_function()
what is the best way to avoid the inevitable "UnboundLocalError: local variable 'b' referenced before assignment" that I'm going to get in the exception handler?
Does python have an elegant way to handle this? If not, what about an inelegant way? In a complicated function I'd prefer to avoid testing the existence of every local variable before I, for example, printed debug information about them.
Does python have an elegant way to
handle this?
To avoid exceptions from printing unbound names, the most elegant way is not to print them; the second most elegant is to ensure the names do get bound, e.g. by binding them at the start of the function (the placeholder None is popular for this purpose).
If not, what about an inelegant way?
try: print 'b is', b
except NameError: print 'b is not bound'
In a complicated function I'd prefer
to avoid testing the existence of
every local variable before I, for
example, printed debug information
about them
Keeping your functions simple (i.e., not complicated) is highly recommended, too. As Hoare wrote 30 years ago (in his Turing acceptance lecture "The Emperor's old clothes", reprinted e.g. in this PDF):
There are two ways of constructing a
software design: One way is to make it
so simple that there are obviously no
deficiencies, and the other way is to
make it so complicated that there are
no obvious deficiencies. The first
method is far more difficult.
Achieving and maintaining simplicity is indeed difficult: given that you have to implement a certain total functionality X, it's the most natural temptation in the world to do so via complicated accretion into a few complicated classes and functions of sundry bits and pieces, "clever" hacks, copy-and-paste-and-edit-a-bit episodes of "drive-by coding", etc, etc.
However, it's a worthwhile effort to strive instead to keep your functions "so simple that there are obviously no deficiencies". If a function's hard to completely unit-test, it's too complicated: break it up (i.e., refactor it) into its natural components, even though it will take work to unearth them. (That's actually one of the way in which a strong focus on unit testing helps code quality: by spurring you relentlessly to keep all the code perfectly testable, it's at the same time spurring you to make it simple in its structure).
You can initialize your variables outside of the try block
a = None
b = None
try:
a = str(5)
raise
b = str(6)
except:
print b
You could check to see if the variable is defined in local scope using the built-in method locals()
http://docs.python.org/library/functions.html#locals
#!/usr/bin/python
def test_function():
try:
a = str(5)
raise
b = str(6)
except:
if 'b' in locals(): print b
test_function()
def test_function():
try:
a = str(5)
raise
b = str(6)
except:
print b
b = str(6) is never run; the program exits try block just after raise. If you want to print some variable in the except block, evaluate it before raising an exception and put them into the exception you throw.
class MyException(Exception):
def __init__(self, var):
self.var = var
def test_function():
try:
a = str(5)
b = str(6)
raise MyException(b)
except MyException,e:
print e.var