In Python 3.6, the new Variable Annotations were introduced in the language.
But, when a type does not exist, the two different things can happen:
>>> def test():
... a: something = 0
...
>>> test()
>>>
>>> a: something = 0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'something' is not defined
Why is the non-existing type handling behavior different? Would not it potentially cause one to overlook the undefined types in the functions?
Notes
Tried with both Python 3.6 RC1 and RC2 - same behavior.
PyCharm highlights something as "unresolved reference" in both inside and outside the function.
The behaviour of the local variable (i.e. inside the function) is at least documented in the section Runtime Effects of Type Annotations:
Annotating a local variable will cause the interpreter to treat it as a local, even if it was never assigned to. Annotations for local variables will not be evaluated:
def f():
x: NonexistentName # No error.
And goes on to explain the difference for global variables:
However, if it is at a module or class level, then the type will be evaluated:
x: NonexistentName # Error!
class X:
var: NonexistentName # Error!
The behaviour seems surprising to me, so I can only offer my guess as to the reasoning: if we put the code in a module, then Python wants to store the annotations.
# typething.py
def test():
a: something = 0
test()
something = ...
a: something = 0
Then import it:
>>> import typething
>>> typething.__annotations__
{'a': Ellipsis}
>>> typething.test.__annotations__
{}
Why it's necessary to store it on the module object, but not on the function object - I don't have a good answer yet. Perhaps it is for performance reasons, since the annotations are made by static code analysis and those names might change dynamically:
...the value of having annotations available locally does not offset the cost of having to create and populate the annotations dictionary on every function call. Therefore annotations at function level are not evaluated and not stored.
The most direct answer for this (to complement #wim's answer) comes from the issue tracker on Github where the proposal was discussed:
[..] Finally, locals. Here I think we should not store the types -- the value of having the annotations available locally is just not enough to offset the cost of creating and populating the dictionary on each function call.
In fact, I don't even think that the type expression should be evaluated during the function execution. So for example:
def side_effect():
print("Hello world")
def foo():
a: side_effect()
a = 12
return a
foo()
should not print anything. (A type checker would also complain that side_effect() is not a valid type.)
From the BDFL himself :-) nor a dict created nor evaluation being performed.
Currently, function objects only store annotations as supplied in their definition:
def foo(a: int):
b: int = 0
get_type_hints(foo) # from typing
{'a': <class 'int'>}
Creating another dictionary for the local variable annotations was apparently deemed too costly.
You can go to https://www.python.org/ftp/python/3.6.0/ and download the RC2 version to test annotations but the released version as wim said is not yet released. I did however downloaded and tried your code using the Python3.6 interpreter and no errors showed up.
You can try write like this:
>>>a: 'something' = 0
Related
Is it possible to forward-declare a function in Python? I want to sort a list using my own cmp function before it is declared.
print "\n".join([str(bla) for bla in sorted(mylist, cmp = cmp_configs)])
I've put the definition of cmp_configs method after the invocation. It fails with this error:
NameError: name 'cmp_configs' is not defined
Is there any way to "declare" cmp_configs method before it's used?
Sometimes, it is difficult to reorganize code to avoid this problem. For instance, when implementing some forms of recursion:
def spam():
if end_condition():
return end_result()
else:
return eggs()
def eggs():
if end_condition():
return end_result()
else:
return spam()
Where end_condition and end_result have been previously defined.
Is the only solution to reorganize the code and always put definitions before invocations?
Wrap the invocation into a function of its own so that
foo()
def foo():
print "Hi!"
will break, but
def bar():
foo()
def foo():
print "Hi!"
bar()
will work properly.
The general rule in Python is that a function should be defined before its usage, which does not necessarily mean it needs to be higher in the code.
If you kick-start your script through the following:
if __name__=="__main__":
main()
then you probably do not have to worry about things like "forward declaration". You see, the interpreter would go loading up all your functions and then start your main() function. Of course, make sure you have all the imports correct too ;-)
Come to think of it, I've never heard such a thing as "forward declaration" in python... but then again, I might be wrong ;-)
If you don't want to define a function before it's used, and defining it afterwards is impossible, what about defining it in some other module?
Technically you still define it first, but it's clean.
You could create a recursion like the following:
def foo():
bar()
def bar():
foo()
Python's functions are anonymous just like values are anonymous, yet they can be bound to a name.
In the above code, foo() does not call a function with the name foo, it calls a function that happens to be bound to the name foo at the point the call is made. It is possible to redefine foo somewhere else, and bar would then call the new function.
Your problem cannot be solved because it's like asking to get a variable which has not been declared.
I apologize for reviving this thread, but there was a strategy not discussed here which may be applicable.
Using reflection it is possible to do something akin to forward declaration. For instance lets say you have a section of code that looks like this:
# We want to call a function called 'foo', but it hasn't been defined yet.
function_name = 'foo'
# Calling at this point would produce an error
# Here is the definition
def foo():
bar()
# Note that at this point the function is defined
# Time for some reflection...
globals()[function_name]()
So in this way we have determined what function we want to call before it is actually defined, effectively a forward declaration. In python the statement globals()[function_name]() is the same as foo() if function_name = 'foo' for the reasons discussed above, since python must lookup each function before calling it. If one were to use the timeit module to see how these two statements compare, they have the exact same computational cost.
Of course the example here is very useless, but if one were to have a complex structure which needed to execute a function, but must be declared before (or structurally it makes little sense to have it afterwards), one can just store a string and try to call the function later.
If the call to cmp_configs is inside its own function definition, you should be fine. I'll give an example.
def a():
b() # b() hasn't been defined yet, but that's fine because at this point, we're not
# actually calling it. We're just defining what should happen when a() is called.
a() # This call fails, because b() hasn't been defined yet,
# and thus trying to run a() fails.
def b():
print "hi"
a() # This call succeeds because everything has been defined.
In general, putting your code inside functions (such as main()) will resolve your problem; just call main() at the end of the file.
There is no such thing in python like forward declaration. You just have to make sure that your function is declared before it is needed.
Note that the body of a function isn't interpreted until the function is executed.
Consider the following example:
def a():
b() # won't be resolved until a is invoked.
def b():
print "hello"
a() # here b is already defined so this line won't fail.
You can think that a body of a function is just another script that will be interpreted once you call the function.
Sometimes an algorithm is easiest to understand top-down, starting with the overall structure and drilling down into the details.
You can do so without forward declarations:
def main():
make_omelet()
eat()
def make_omelet():
break_eggs()
whisk()
fry()
def break_eggs():
for egg in carton:
break(egg)
# ...
main()
# declare a fake function (prototype) with no body
def foo(): pass
def bar():
# use the prototype however you see fit
print(foo(), "world!")
# define the actual function (overwriting the prototype)
def foo():
return "Hello,"
bar()
Output:
Hello, world!
No, I don't believe there is any way to forward-declare a function in Python.
Imagine you are the Python interpreter. When you get to the line
print "\n".join([str(bla) for bla in sorted(mylist, cmp = cmp_configs)])
either you know what cmp_configs is or you don't. In order to proceed, you have to
know cmp_configs. It doesn't matter if there is recursion.
You can't forward-declare a function in Python. If you have logic executing before you've defined functions, you've probably got a problem anyways. Put your action in an if __name__ == '__main__' at the end of your script (by executing a function you name "main" if it's non-trivial) and your code will be more modular and you'll be able to use it as a module if you ever need to.
Also, replace that list comprehension with a generator express (i.e., print "\n".join(str(bla) for bla in sorted(mylist, cmp=cmp_configs)))
Also, don't use cmp, which is deprecated. Use key and provide a less-than function.
Import the file itself. Assuming the file is called test.py:
import test
if __name__=='__main__':
test.func()
else:
def func():
print('Func worked')
TL;DR: Python does not need forward declarations. Simply put your function calls inside function def definitions, and you'll be fine.
def foo(count):
print("foo "+str(count))
if(count>0):
bar(count-1)
def bar(count):
print("bar "+str(count))
if(count>0):
foo(count-1)
foo(3)
print("Finished.")
recursive function definitions, perfectly successfully gives:
foo 3
bar 2
foo 1
bar 0
Finished.
However,
bug(13)
def bug(count):
print("bug never runs "+str(count))
print("Does not print this.")
breaks at the top-level invocation of a function that hasn't been defined yet, and gives:
Traceback (most recent call last):
File "./test1.py", line 1, in <module>
bug(13)
NameError: name 'bug' is not defined
Python is an interpreted language, like Lisp. It has no type checking, only run-time function invocations, which succeed if the function name has been bound and fail if it's unbound.
Critically, a function def definition does not execute any of the funcalls inside its lines, it simply declares what the function body is going to consist of. Again, it doesn't even do type checking. So we can do this:
def uncalled():
wild_eyed_undefined_function()
print("I'm not invoked!")
print("Only run this one line.")
and it runs perfectly fine (!), with output
Only run this one line.
The key is the difference between definitions and invocations.
The interpreter executes everything that comes in at the top level, which means it tries to invoke it. If it's not inside a definition.
Your code is running into trouble because you attempted to invoke a function, at the top level in this case, before it was bound.
The solution is to put your non-top-level function invocations inside a function definition, then call that function sometime much later.
The business about "if __ main __" is an idiom based on this principle, but you have to understand why, instead of simply blindly following it.
There are certainly much more advanced topics concerning lambda functions and rebinding function names dynamically, but these are not what the OP was asking for. In addition, they can be solved using these same principles: (1) defs define a function, they do not invoke their lines; (2) you get in trouble when you invoke a function symbol that's unbound.
Python does not support forward declarations, but common workaround for this is use of the the following condition at the end of your script/code:
if __name__ == '__main__': main()
With this it will read entire file first and then evaluate condition and call main() function which will be able to call any forward declared function as it already read the entire file first. This condition leverages special variable __name__ which returns __main__ value whenever we run Python code from current file (when code was imported as a module, then __name__ returns module name).
"just reorganize my code so that I don't have this problem." Correct. Easy to do. Always works.
You can always provide the function prior to it's reference.
"However, there are cases when this is probably unavoidable, for instance when implementing some forms of recursion"
Can't see how that's even remotely possible. Please provide an example of a place where you cannot define the function prior to it's use.
Now wait a minute. When your module reaches the print statement in your example, before cmp_configs has been defined, what exactly is it that you expect it to do?
If your posting of a question using print is really trying to represent something like this:
fn = lambda mylist:"\n".join([str(bla)
for bla in sorted(mylist, cmp = cmp_configs)])
then there is no requirement to define cmp_configs before executing this statement, just define it later in the code and all will be well.
Now if you are trying to reference cmp_configs as a default value of an argument to the lambda, then this is a different story:
fn = lambda mylist,cmp_configs=cmp_configs : \
"\n".join([str(bla) for bla in sorted(mylist, cmp = cmp_configs)])
Now you need a cmp_configs variable defined before you reach this line.
[EDIT - this next part turns out not to be correct, since the default argument value will get assigned when the function is compiled, and that value will be used even if you change the value of cmp_configs later.]
Fortunately, Python being so type-accommodating as it is, does not care what you define as cmp_configs, so you could just preface with this statement:
cmp_configs = None
And the compiler will be happy. Just be sure to declare the real cmp_configs before you ever invoke fn.
Python technically has support for forward declaration.
if you define a function/class then set the body to pass, it will have an empty entry in the global table.
you can then "redefine" the function/class later on to implement the function/class.
unlike c/c++ forward declaration though, this does not work from outside the scope (i.e. another file) as they have their own "global" namespace
example:
def foo(): pass
foo()
def foo(): print("FOOOOO")
foo()
foo is declared both times
however the first time foo is called it does not do anything as the body is just pass
but the second time foo is called. it executes the new body of print("FOOOOO")
but again. note that this does not fix circular dependancies. this is because files have their own name and have their own definitions of functions
example 2:
class bar: pass
print(bar)
this prints <class '__main__.bar'> but if it was declared in another file it would be <class 'otherfile.foo'>
i know this post is old, but i though that this answer would be useful to anyone who keeps finding this post after the many years it has been posted for
One way is to create a handler function. Define the handler early on, and put the handler below all the methods you need to call.
Then when you invoke the handler method to call your functions, they will always be available.
The handler could take an argument nameOfMethodToCall. Then uses a bunch of if statements to call the right method.
This would solve your issue.
def foo():
print("foo")
#take input
nextAction=input('What would you like to do next?:')
return nextAction
def bar():
print("bar")
nextAction=input('What would you like to do next?:')
return nextAction
def handler(action):
if(action=="foo"):
nextAction = foo()
elif(action=="bar"):
nextAction = bar()
else:
print("You entered invalid input, defaulting to bar")
nextAction = "bar"
return nextAction
nextAction=input('What would you like to do next?:')
while 1:
nextAction = handler(nextAction)
I have a function in a sub-module which needs to manipulate the variables from parent/interpreter Globals (assertion, validation) like this:
import mymodule
mymodule.fun_using_main_interpreter_globals1()
If I do this, it works:
mymodule.fun_using_main_interpreter_globals1(explicit_pass= globals() )
But, If I dont pass explictely globals(), how can I get access to interpreter/parent globals() into my sub-module ?
In IPython, it can be put in profile settings.
I never went for real in this territory, but looking at the documentation this should do it:
caller_globals = dict(inspect.getmembers(inspect.stack()[1][0]))["f_globals"]
inspect module allows you to access, among other things, the python interpreter stack. The second element (accessed with [1]) is the caller, the stack frame is the first element of the tuple (accessed with [0]) and it contains as member the current global dictionary for that context (named f_globals).
Note that this returns the globals() of the caller, not the one of the module of which the called function is a sub-module. That in general I think is not possible because the same module can be a sub-module of different modules (a sub-module is just a global in a module and it's possible that different modules share the same sub-module object).
I had this working for local variables and came across this post while looking for how to get a caller context's global variables, too. While this topic had just enough information to help me see what I was missing ( thanks user #6502 ), I couldn't help but feel that if I didn't already have it working, I might have struggled a bit with the information that was here, therefore:
Here is a relatively simple way to access variables in a caller's context ( local or global ) that works in both py2 and py3:
import inspect
def furmat ( text ):
# ... other initialization stuff ...
this_frame = inspect.currentframe() # GET OUR FRAME HERE
caller_frame = this_frame.f_back # GET CALLER'S FRAME HERE
def caller_value ( key ):
val = caller_frame.f_locals.get ( key ) # LOOKUP CALLER-LOCAL
if val is not None:
return val
return caller_frame.f_globals[key] # LOOKUP CALLER-GLOBAL ( raise KeyError )
# ... process text here, call caller_value() to query for variables in the caller's context
del caller_frame # PREVENT MEMORY LEAKS PART 1
del this_frame # AND PART 2
For those wanting to understand possible motivation, I built a format-like function called "furmat" that would automatically resolve any variables by name when surrounded by {} like py3's f"" strings, but that would work in py2. An example invocation looks like this:
from furmat import furmat
a = "AAA"
def test():
b = "BBB"
return furmat ( "{a}{b}" )
assert test() == "AAABBB"
Notice it resolves both the local variable {b} and the global variable {a}
I'm not posting the source code of furmat here only because it's 80 lines long and depends on a half-dozen other files in my private library, so it's a bit out of scope, but the snippet at the top of this post is the exact means that I'm using to lookup the values of any names provided within the format string ( minus most error handling to make the example more readable ).
In addition to the reason given above, I also wanted to build furmat because:
1) py2's unicode.format() throws a UnicodeError if you disable auto-conversion between bytes and unicode then try to take a {!r} because repr()'s output is always a bytes ( py2 str ) object so you can't avoid the automatic conversion,
2) I don't like how {!r} ambiguously only puts prints the b'' or u'' string prefix for non-native strings which confuses me when I frequently switch between py2 and py3 during mypy and unit testing, and
3) it was fun.
I know that it is a bad idea to name a variable that is the same name as a Python built-in function. But say if a person doesn't know all the "taboo" variable names to avoid (e.g. list, set, etc.), is there a way to make Python at least to stop you (e.g. via error messages) from corrupting built-in functions?
For example, command line 4 below allows me to overwrite / corrupt the built-in function set() without stopping me / producing errors. (This error was left un-noticed until it gets to command line 6 below when set() is called.). Ideally I would like Python to stop me at command line 4 (instead of waiting till command line 6).
Note: following executions are performed in Python 2.7 (iPython) console. (Anaconda Spyder IDE).
In [1]: myset = set([1,2])
In [2]: print(myset)
set([1, 2])
In [3]: myset
Out[3]: {1, 2}
In [4]: set = set([3,4])
In [5]: print(set)
set([3, 4])
In [6]: set
Out[6]: {3, 4}
In [7]: myset2 = set([5,6])
Traceback (most recent call last):
File "<ipython-input-7-6f49577a7a45>", line 1, in <module>
myset2 = set([5,6])
TypeError: 'set' object is not callable
Background: I was following the tutorial at this HackerRank Python Set Challenge. The tutorial involves creating a variable valled set (which has the same name as the Python built-in function). I tried out the tutorial line-by-line exactly and got the "set object is not callable" error. The above test is driven by this exercise. (Update: I contacted HackerRank Support and they have confirmed they might have made a mistake creating a variable with built-in name.)
As others have said, in Python the philosophy is to allow users to "misuse" things rather than trying to imagine and prevent misuses, so nothing like this is built-in. But, by being so open to being messed around with, Python allows you to implement something like what you're talking about, in a limited way*. You can replace certain variable namespace dictionaries with objects that will prevent your favorite variables from being overwritten. (Of course, if this breaks any of your code in unexpected ways, you get both pieces.)
For this, you need to use use something like eval(), exec, execfile(), or code.interact(), or override __import__(). These allow you to provide objects, that should act like dictionaries, which will be used for storing variables. We can create a "safer" replacement dictionary by subclassing dict:
class SafeGlobals(dict):
def __setitem__(self, name, value):
if hasattr(__builtins__, name) or name == '__builtins__':
raise SyntaxError('nope')
return super(SafeGlobals, self).__setitem__(name, value)
my_globals = SafeGlobals(__builtins__=__builtins)
With my_globals set as the current namespace, setting a variable like this:
x = 3
Will translate to the following:
my_globals['x'] = 3
The following code will execute a Python file, using our safer dictionary for the top-level namespace:
execfile('safetyfirst.py', SafeGlobals(__builtins__=__builtins__))
An example with code.interact():
>>> code.interact(local=SafeGlobals(__builtins__=__builtins__))
Python 2.7.9 (default, Mar 1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> x = 2
>>> x
2
>>> dict(y=5)
{'y': 5}
>>> dict = "hi"
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "<stdin>", line 4, in __setitem__
SyntaxError: nope
*Unfortunately, this approach is very limited. It will only prevent overriding built-ins in the top-level namespace. You're free to override built-ins in other namespaces:
>>> def f():
... set = 1
... return set
...
>>> f()
1
This is an interesting idea; unfortunately, Python is not very restrictive and does not offer out-of-the-box solutions for such intentions. Overriding lower-level identifiers in deeper nested scopes is part of Python's philosophy and is wanted and often used, in fact. If you disabled this feature somehow, I guess a lot of library code would be broken at once.
Nevertheless you could create a check function which tests if anything in the current stack has been overridden. For this you would step through all the nested frames you are in and check if their locals also exist in their parent. This is very introspective work and probably not what you want to do but I think it could be done. With such a tool you could use the trace facility of Python to check after each executed line whether the state is still clean; that's the same functionality a debugger uses for step-by-step debugging, so this is again probably not what you want.
It could be done, but it would be like nailing glasses to a wall to make sure you never forget where they are.
Some more practical approach:
For builtins like the ones you mentioned you always can access them by writing explicitly __builtins__.set etc. For things imported, import the module and call the things by their module name (e. g. sys.exit() instead of exit()). And normally one knows when one is going to use an identifier, so just do not override it, e. g. do not create a variable named set if you are going to create a set object.
Let's say I've got the following functions:
def xplusy(x, y):
return x+y
def xplus1(x):
xplusy = xplusy(x, 1)
return xplusy
Now if I call a = xplus1(4) it throws the following error:
UnboundLocalError: local variable 'xplusy' referenced before assignment
The error is because of the naming conflict, if I redefine xplus1 as follows:
def xplus1(x):
s = xplusy(x, 1)
return s
it works fine.
Why is it so: can't compiler properly distinguish between a variable and a function call?
Any ways around it?
In Python, functions are data, and typing is dynamic. This means that the following lines are valid Python:
def func(x):
return x + 3
func = 3
func is now an int. The original function func is no longer referenced. The fact that func was originally a function doesn't have any bearing on what types of data can be assigned to it in the future. (This is what "dynamic typing" means.)
Therefore, since there's no static typing, and "function" is a valid data type, it wouldn't make sense for the Python interpreter to distinguish between a function and a piece of data referenced by the same name. Therefore, within a given scope, there's no way to use the same unqualified variable name to mean two different things.
In your particular case, if the code in your xplus1 function meant anything, it would mean "compute the value of xplusy(x,1) and assign that value to the variable xplusy -- thereby losing the reference to the function xplusy." However, within the scope of a function, the interpreter won't let you make an assignment to a variable outside of that scope, so it assumes that by writing an assignment statement, you're introducing a new local variable xplusy. The local variable, however, hasn't been defined yet, so your attempt to call it, xplusy(x,1), fails. The globally defined function is not called as a fall-back because, again, you can't have two unqualified names be identical and point to different data in the same scope.
Another example demonstrating the "no duplication of variable names within one scope" rule (which I actually only just discovered while playing around with the prompt in my attempt to construct this answer):
>>> def f1():
... a = xplusy(3,4)
... xplusy = 5
... print xplusy
...
>>> f1()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in f1
UnboundLocalError: local variable 'xplusy' referenced before assignment
>>> def f1():
... a = xplusy(3,4)
... print a
...
>>> f1()
7
This demonstrates that it really is the scope, not the statement that requires unique names.
EDIT: This is a really cool post that explains this and other scoping-related behavior: http://me.veekun.com/blog/2011/04/24/gotcha-python-scoping-closures/
In python function is first class object which mean it is as any other object.
More information about What are “first class” objects?
The reason this occurs is because xplusy exists in your scope as a global variable which cannot be changed unless you explicitly say xplusy is global.
def xplusy(x,y):
return x+y
def xplus1(x):
global xplusy
xplusy = xplusy(x,1)
return xplusy
however, this will make xplusy refer to an int, float or whatever xplusy returned the first time, meaning that after the first time it will throw TypeErrors.
The more pythonic way to do this would most likely be
def xplus1(x):
return xplusy(x,1)
or using the functools module:
from functools import partial
xplus1 = partial(xplusy,y=1) #since you wanted to override y with 1
if you don't care about which argument is overridden, you could simply do
xplus1 = partial(xplusy,1)
There are cases, for example with callbacks, that you refer to functions by their static name (funcName instead of funcName()). Thus in Python that variable-name is reserved for the function. A lot like how you would use LAMBDA functions that you store in a variable.
In Python, xplusy can reference anything. You can do this:
def xplusy(x, y):
return x+y
def otherfunction(x, y):
return x*y
def xplus1(x):
xplusy = otherfunction
return xplusy(x, 1)
And xplusy variable will reference to otherfunction in xplus1 scope.
xplus1(2)
>>> 2
Result is 2 * 1 instead 2 + 1. Be careful with assignments.Use as many variables as you need.
class Some(object):
tokens = [ ... list of strings ... ]
untokenized = [tokens.index(a) for a in [... some other list of strings ...]]
... etc ...
some = Some()
This works fine with Python2.7. However python3 says:
Traceback (most recent call last):
File "./test.py", line 17, in <module>
class Some(object):
File "./test.py", line 42, in Some
untokenized = [tokens.index(a) for a in [... some other list of strings ...]]
File "./test.py", line 42, in <listcomp>
untokenized = [tokens.index(a) for a in [... some other list of strings ...]]
NameError: global name 'tokens' is not defined
Though I can work around the problem, I really would like to know what's difference here between Python2 and Python3. I've read python 2->3 changes documents, but I was not able to identify any description which is related to my problem. Also 2to3 tool does not complain anything in my code.
By the way, though I can't recall the situation now, but I had something similar with python2 only too (I haven't even tried this with 3), I thought this should work (within a class):
def some_method(self):
return {a: eval("self." + a) for a in dir(self) if not a.startswith("_")}
However it causes python2 saying: NameError: name 'self' is not defined
I haven't tried this with python3 yet, but for example this works:
[eval("self." + a) for a in dir(self) if not a.startswith("_")]
If I change the relevant part of the previous example to this one (ok the example itself is a bit stupid, but it shows my problem at least). Now I am very curious, why self seems not to be defined for this first example but it is for the second? It seems with dicts, I have similar problem that my original question is about, but with list generator expression it works, but not in python3. Hmmm ...
After my python2 -> 3 problem I mentioned this, since all of these seems to be about the problem that something is not defined according to python interpreter (and maybe the second part of my question is unrelated?). I feel quite confused now. Please enlighten me about my mistake (since I am sure I missed something of course).
As Wooble says, the issue is that classes don't have a lexical scope (actually, in either Python 2 or Python 3). Instead, they have a local namespace that does not constitute a scope. This means that expressions within the class definition have access to the content of the namespace:
class C:
a = 2
b = a + 2 # b = 4
but scopes introduced within the body of the class do not have access to its namespace:
class C:
a = 2
def foo(self):
return a # NameError: name 'a' is not defined, use return self.__class__.a
The difference between Python 2 and Python 3 is that in Python 2 list comprehensions do not introduce a new scope:
[a for a in range(3)]
print a # prints 2
whereas in Python 3 they do:
[a for a in range(3)]
print(a) # NameError: name 'a' is not defined
This was changed in Python 3 for a couple of reasons, including to make list comprehensions behave the same way as generator-expressions (genexps); (a for a in range(3)) has its own scope in both Python 2 and Python 3.
So, within the body of a class, a Python 2 genexp or a Python 3 listcomp or genexp introduces a new scope and therefore does not have access to the class-definition local namespace.
The way to give the genexp/listcomp access to names from the class-definition namespace is to introduce a new scope, using a function or a lambda:
class C:
a = 2
b = (lambda a=a: [a + i for i in range(3)])()
The eval issue
The issue with your eval example is that eval by default evaluates its argument in the local scope; because Python 2 list comprehensions have the above behaviour of sharing the enclosing scope the eval can access the method scope, but a genexp or Python 3 listcomp local scope only has whatever the compiler can tell is required from the enclosing scope (since a genexp/listcomp scope is a closure):
def bar(x):
return list(eval('x') + x for i in range(3))
bar(5) # returns [10, 10, 10]
def baz(x):
return list(eval('x') for i in range(3))
baz(5) # NameError: name 'x' is not defined
As Martijn says, instead of eval you should use getattr.