Develop Reference

Why must a new global variable be created to reference the current class instance in "exec"?

I have a class that contains ~20 methods, and in def __init__(self, ...): I have to call many of these methods (~9) but I didn't want to have to call each individual method one by one.
So I took the easy way out and created two list list comprehensions, that use exec to call each method:
[exec("self.create%s()" % x) for x in "ArticleObjects SeriesObjects ArticleList SearchList".split(" ")]
[exec("self.compile%sPage(self)" % x) for x in "About Screenshots Search Contact Articles".split(" ")]
When I ran this code using python3 filename.py I got an error, that read:
NameError: name 'self' is not defined
Through trial and error I found that; in order to get this code to work I had to create a copy of self called instance and make the new instance variable a global variable and then call the method using ClassName.methodName(instance) instead of self.methodName():
With the working code being:
global instance; instance = self
[exec("ClassNamecreate%s(instance)" % x) for x in "ArticleObjects SeriesObjects ArticleList SearchList".split(" ")]
[exec("ClassName.compile%sPage(instance)" % x) for x in "About Screenshots Search Contact Articles".split(" ")]
Why is this? Why is the self variable undefined in exec despite it being available to the scope that exec is being called in?
Update: I'm using Python 3.6.7

There's lots of good suggestions here for how to avoid the exec statement (which is generally bad), but to answer your question about why this happens, it's got more to do with the list comprehension. List comprehensions create a new scope, and when you call exec without a globals or locals argument, it uses the locals() function:
Note: The default locals act as described for function locals() below
Source
Here you can see what the results of the locals() function look like from within a list comprehension:
class Sample:
def __init__(self):
k = 4
print(locals())
exec("print(locals())")
[print(locals()) for x in range(1)]
[exec("print(locals())") for x in range(1)]
Sample()
output:
{'k': 4, 'self': <__main__.Sample object at 0x00000000030295C0>}
{'k': 4, 'self': <__main__.Sample object at 0x00000000030295C0>}
{'x': 0, '.0': <range_iterator object at 0x00000000030019F0>}
{'x': 0, '.0': <range_iterator object at 0x00000000030019F0>}
So, locals() is the same inside or outside the exec. It's the list comprehension that changes it. Only, when you're outside an exec statement, the interpreter can fall past the locals of the list comprehension and find self in the outer scope. No such luck once you call exec.

Using getattr is simpler (and usually safer) than exec. Try something along these lines:
def __init__(self):
suffixes = ["ArticleObjects", "SeriesObjects", ...]
for suffix in suffixes:
method = getattr(self, "create" + suffix)
method()

I wouldn't use exec for this. While it may be the shortest version, it might also confuse both collaborators and code analysis tools. I'd use something like this instead:
class Test:
def __init__(self):
for f in (self.createA, self.createB, self.createC):
f()
def createA(self):
print("A")
def createB(self):
print("B")
def createC(self):
print("C")

Python - How to call an instance from a given string?

I want to call the function bar() from the instance foo, just like this:
foo.bar()
But both instance and function names come from a given string. I tried getattr but it just let me use a string for the function, not the instance:
strbar = 'bar'
getattr(foo, strbar)()
What I want to do is something like:
strfoo = 'foo'
strbar = 'bar'
getattr(strfoo, strbar)()
But it gives me:
AttributeError: 'str' object has no attribute 'bar'
I know a dictionary could be an option, but that makes me write a really long dictionary.

The best way would be to have a dictionary or a similar structure.
However you could use eval which is evil. Or you can get your instance from the locals() dictionary:
getattr(locals()[instance_name], attribute_name)
I would think about redesigning your code. There must be a better solution as eval or locals ... like a dictionary ...

How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?

Consider this example:
def outer():
s_outer = "outer\n"
def inner():
s_inner = "inner\n"
do_something()
inner()
I want the code in do_something to be able to access the variables of the calling functions further up the call stack, in this case s_outer and s_inner. More generally, I want to call it from various other functions, but always execute it in their respective context and access their respective scopes (implement dynamic scoping).
I know that in Python 3.x, the nonlocal keyword allows access to s_outer from within inner. Unfortunately, that only helps with do_something if it's defined within inner. Otherwise, inner isn't a lexically enclosing scope (similarly, neither is outer, unless do_something is defined within outer).
I figured out how to inspect stack frames with the standard library inspect, and made a small accessor that I can call from within do_something() like this:
def reach(name):
for f in inspect.stack():
if name in f[0].f_locals:
return f[0].f_locals[name]
return None
and then
def do_something():
print( reach("s_outer"), reach("s_inner") )
works just fine.
Can reach be implemented more simply? How else can I solve the problem?

There is no and, in my opinion, should be no elegant way of implementing reach since that introduces a new non-standard indirection which is really hard to comprehend, debug, test and maintain. As the Python mantra (try import this) says:
Explicit is better than implicit.
So, just pass the arguments. You-from-the-future will be really grateful to you-from-today.

What I ended up doing was
scope = locals()
and make scope accessible from do_something. That way I don't have to reach, but I can still access the dictionary of local variables of the caller. This is quite similar to building a dictionary myself and passing it on.

We can get naughtier.
This is an answer to the "Is there a more elegant/shortened way to implement the reach() function?" half of the question.
We can give better syntax for the user: instead of reach("foo"), outer.foo.
This is nicer to type, and the language itself immediately tells you if you used a name that can't be a valid variable (attribute names and variable names have the same constraints).
We can raise an error, to properly distinguish "this doesn't exist" from "this was set to None".
If we actually want to smudge those cases together, we can getattr with the default parameter, or try-except AttributeError.
We can optimize: no need to pessimistically build a list big enough for all the frames at once.
In most cases we probably won't need to go all the way to the root of the call stack.
Just because we're inappropriately reaching up stack frames, violating one of the most important rules of programming to not have things far away invisibly effecting behavior, doesn't mean we can't be civilized.
If someone is trying to use this Serious API for Real Work on a Python without stack frame inspection support, we should helpfully let them know.
import inspect
class OuterScopeGetter(object):
def __getattribute__(self, name):
frame = inspect.currentframe()
if frame is None:
raise RuntimeError('cannot inspect stack frames')
sentinel = object()
frame = frame.f_back
while frame is not None:
value = frame.f_locals.get(name, sentinel)
if value is not sentinel:
return value
frame = frame.f_back
raise AttributeError(repr(name) + ' not found in any outer scope')
outer = OuterScopeGetter()
Excellent. Now we can just do:
>>> def f():
... return outer.x
...
>>> f()
Traceback (most recent call last):
...
AttributeError: 'x' not found in any outer scope
>>>
>>> x = 1
>>> f()
1
>>> x = 2
>>> f()
2
>>>
>>> def do_something():
... print(outer.y)
... print(outer.z)
...
>>> def g():
... y = 3
... def h():
... z = 4
... do_something()
... h()
...
>>> g()
3
4
Perversion elegantly achieved.

Is there a better way to solve this problem? (Other than wrapping the respective data into dicts and pass these dicts explicitly to do_something())
Passing the dicts explicitly is a better way.
What you're proposing sounds very unconventional. When code increases in size, you have to break down the code into a modular architecture, with clean APIs between modules. It also has to be something that is easy to comprehend, easy to explain, and easy to hand over to another programmer to modify/improve/debug it. What you're proposing sounds like it is not a clean API, unconventional, with a non-obvious data flow. I suspect it would probably make many programmers grumpy when they saw it. :)
Another option would be to make the functions members of a class, with the data being in the class instance. That could work well if your problem can be modelled as several functions operating on the data object.

Call python function as if it were inline

I want to have a function in a different module, that when called, has access to all variables that its caller has access to, and functions just as if its body had been pasted into the caller rather than having its own context, basically like a C Macro instead of a normal function. I know I can pass locals() into the function and then it can access the local variables as a dict, but I want to be able to access them normally (eg x.y, not x["y"] and I want all names the caller has access to not just the locals, as well as things that were 'imported' into the caller's file but not into the module that contains the function.
Is this possible to pull off?
Edit 2 Here's the simplest possible example I can come up with of what I'm really trying to do:
def getObj(expression)
ofs = expression.rfind(".")
obj = eval(expression[:ofs])
print "The part of the expression Left of the period is of type ", type(obj),
Problem is that 'expression' requires the imports and local variables of the caller in order to eval without error.In reality theres a lot more than just an eval, so I'm trying to avoid the solution of just passing locals() in and through to the eval() since that won't fix my general case problem.

And another, even uglier way to do it -- please don't do this, even if it's possible --
import sys
def insp():
l = sys._getframe(1).f_locals
expression = l["expression"]
ofs = expression.rfind(".")
expofs = expression[:ofs]
obj = eval(expofs, globals(), l)
print "The part of the expression %r Left of the period (%r) is of type %r" % (expression, expofs, type(obj)),
def foo():
derp = 5
expression = "derp.durr"
insp()
foo()
outputs
The part of the expression 'derp.durr' Left of the period ('derp') is of type (type 'int')

I don't presume this is the answer that you wanted to hear, but trying to access local variables from a caller module's scope is not a good idea. If you normally program in PHP or C, you might be used to this sort of thing?
If you still want to do this, you might consider creating a class and passing an instance of that class in place of locals():
#other_module.py
def some_func(lcls):
print(lcls.x)
Then,
>>> import other_module
>>>
>>>
>>> x = 'Hello World'
>>>
>>> class MyLocals(object):
... def __init__(self, lcls):
... self.lcls = lcls
... def __getattr__(self, name):
... return self.lcls[name]
...
>>> # Call your function with an instance of this instead.
>>> other_module.some_func(MyLocals(locals()))
'Hello World'
Give it a whirl.

Is this possible to pull off?
Yes (sort of, in a very roundabout way) which I would strongly advise against it in general (more on that later).
Consider:
myfile.py
def func_in_caller():
print "in caller"
import otherfile
globals()["imported_func"] = otherfile.remote_func
imported_func(123, globals())
otherfile.py
def remote_func(x1, extra):
for k,v in extra.iteritems():
globals()[k] = v
print x1
func_in_caller()
This yields (as expected):
123
in caller
What we're doing here is trickery: we just copy every item into another namespace in order to make this work. This can (and will) break very easily and/or lead to hard to find bugs.
There's almost certainly a better way of solving your problem / structuring your code (we need more information in general on what you're trying to achieve).

From The Zen of Python:
2) Explicit is better than implicit.
In other words, pass in the parameter and don't try to get really fancy just because you think it would be easier for you. Writing code is not just about you.

How can you print a variable name in python? [duplicate]

This question already has answers here:
How to print original variable's name in Python after it was returned from a function?
(13 answers)
Closed 4 years ago.
Say I have a variable named choice it is equal to 2. How would I access the name of the variable? Something equivalent to
In [53]: namestr(choice)
Out[53]: 'choice'
for use in making a dictionary. There's a good way to do this and I'm just missing it.
EDIT:
The reason to do this is thus. I am running some data analysis stuff where I call the program with multiple parameters that I would like to tweak, or not tweak, at runtime. I read in the parameters I used in the last run from a .config file formated as
filename
no_sig_resonance.dat
mass_peak
700
choice
1,2,3
When prompted for values, the previously used is displayed and an empty string input will use the previously used value.
My question comes about because when it comes to writing the dictionary that these values have been scanned into. If a parameter is needed I run get_param which accesses the file and finds the parameter.
I think I will avoid the problem all together by reading the .config file once and producing a dictionary from that. I avoided that originally for... reasons I no longer remember. Perfect situation to update my code!

If you insist, here is some horrible inspect-based solution.
import inspect, re
def varname(p):
for line in inspect.getframeinfo(inspect.currentframe().f_back)[3]:
m = re.search(r'\bvarname\s*\(\s*([A-Za-z_][A-Za-z0-9_]*)\s*\)', line)
if m:
return m.group(1)
if __name__ == '__main__':
spam = 42
print varname(spam)
I hope it will inspire you to reevaluate the problem you have and look for another approach.

To answer your original question:
def namestr(obj, namespace):
return [name for name in namespace if namespace[name] is obj]
Example:
>>> a = 'some var'
>>> namestr(a, globals())
['a']
As #rbright already pointed out whatever you do there are probably better ways to do it.

If you are trying to do this, it means you are doing something wrong. Consider using a dict instead.
def show_val(vals, name):
print "Name:", name, "val:", vals[name]
vals = {'a': 1, 'b': 2}
show_val(vals, 'b')
Output:
Name: b val: 2

Rather than ask for details to a specific solution, I recommend describing the problem you face; I think you'll get better answers. I say this since there's almost certainly a better way to do whatever it is you're trying to do. Accessing variable names in this way is not commonly needed to solve problems in any language.
That said, all of your variable names are already in dictionaries which are accessible through the built-in functions locals and globals. Use the correct one for the scope you are inspecting.
One of the few common idioms for inspecting these dictionaries is for easy string interpolation:
>>> first = 'John'
>>> last = 'Doe'
>>> print '%(first)s %(last)s' % globals()
John Doe
This sort of thing tends to be a bit more readable than the alternatives even though it requires inspecting variables by name.

You can't, as there are no variables in Python but only names.
For example:
> a = [1,2,3]
> b = a
> a is b
True
Which of those two is now the correct variable? There's no difference between a and b.
There's been a similar question before.

Will something like this work for you?
>>> def namestr(**kwargs):
... for k,v in kwargs.items():
... print "%s = %s" % (k, repr(v))
...
>>> namestr(a=1, b=2)
a = 1
b = 2
And in your example:
>>> choice = {'key': 24; 'data': None}
>>> namestr(choice=choice)
choice = {'data': None, 'key': 24}
>>> printvars(**globals())
__builtins__ = <module '__builtin__' (built-in)>
__name__ = '__main__'
__doc__ = None
namestr = <function namestr at 0xb7d8ec34>
choice = {'data': None, 'key': 24}

With eager evaluation, variables essentially turn into their values any time you look at them (to paraphrase). That said, Python does have built-in namespaces. For example, locals() will return a dictionary mapping a function's variables' names to their values, and globals() does the same for a module. Thus:
for name, value in globals().items():
if value is unknown_variable:
... do something with name
Note that you don't need to import anything to be able to access locals() and globals().
Also, if there are multiple aliases for a value, iterating through a namespace only finds the first one.

For the revised question of how to read in configuration parameters, I'd strongly recommend saving yourself some time and effort and use ConfigParser or (my preferred tool) ConfigObj.
They can do everything you need, they're easy to use, and someone else has already worried about how to get them to work properly!