This question already has answers here:
Why isn't the 'global' keyword needed to access a global variable?
(11 answers)
Closed 2 years ago.
I wonder why I can change global dictionary without global keyword? Why it's mandatory for other types? Is there any logic behind this?
E.g. code:
#!/usr/bin/env python3
stringvar = "mod"
dictvar = {'key1': 1,
'key2': 2}
def foo():
dictvar['key1'] += 1
def bar():
stringvar = "bar"
print(stringvar)
print(dictvar)
foo()
print(dictvar)
print(stringvar)
bar()
print(stringvar)
Gives following results:
me#pc:~/$ ./globalDict.py
{'key2': 2, 'key1': 1}
{'key2': 2, 'key1': 2} # Dictionary value has been changed
mod
bar
mod
where I would expect:
me#pc:~/$ ./globalDict.py
{'key2': 2, 'key1': 1}
{'key2': 2, 'key1': 1} # I didn't use global, so dictionary remains the same
mod
bar
mod
The reason is that the line
stringvar = "bar"
is ambiguous, it could be referring to a global variable, or it could be creating a new local variable called stringvar. In this case, Python defaults to assuming it is a local variable unless the global keyword has already been used.
However, the line
dictvar['key1'] += 1
Is entirely unambiguous. It can be referring only to the global variable dictvar, since dictvar must already exist for the statement not to throw an error.
This is not specific to dictionaries- the same is true for lists:
listvar = ["hello", "world"]
def listfoo():
listvar[0] = "goodbye"
or other kinds of objects:
class MyClass:
foo = 1
myclassvar = MyClass()
def myclassfoo():
myclassvar.foo = 2
It's true whenever a mutating operation is used rather than a rebinding one.
You can modify any mutable object without using global keyword.
This is possible in Python because global is used when you want to reassign new objects to variable names already used in global scope or to define new global variables.
But in case of mutable objects you're not re-assigning anything, you're just modifying them in-place, therefore Python simply loads them from global scope and modifies them.
As docs say:
It would be impossible to assign to a global variable without global.
In [101]: dic = {}
In [102]: lis = []
In [103]: def func():
dic['a'] = 'foo'
lis.append('foo') # but fails for lis += ['something']
.....:
In [104]: func()
In [105]: dic, lis
Out[105]: ({'a': 'foo'}, ['foo'])
dis.dis:
In [121]: dis.dis(func)
2 0 LOAD_CONST 1 ('foo')
3 LOAD_GLOBAL 0 (dic) # the global object dic is loaded
6 LOAD_CONST 2 ('a')
9 STORE_SUBSCR # modify the same object
3 10 LOAD_GLOBAL 1 (lis) # the global object lis is loaded
13 LOAD_ATTR 2 (append)
16 LOAD_CONST 1 ('foo')
19 CALL_FUNCTION 1
22 POP_TOP
23 LOAD_CONST 0 (None)
26 RETURN_VALUE
Related
This question already has answers here:
Why can a function modify some arguments as perceived by the caller, but not others?
(13 answers)
How do I pass a variable by reference?
(39 answers)
Closed 1 year ago.
I have a list which I give to a function as argument.
Inside function, I'm altering the list, but I don't return the list using return.
I'm running the function and print the list afterwards.
Excpeted result: []
Actual result: [5]
How is this possible?
def foo(bar):
for i in range(2):
if bar == []:
bar.append(4)
else:
bar[0] += 1
myvar = []
foo(myvar)
print (myvar)
One thing to know about lists is that they contain references (pointers) to list objects rather than the list itself. This can be seen by running the following code:
>>> list1 = [0, 1, 2, 3]
>>> list2 = list1
>>> list1[0] = 5
>>> list2
[5, 1, 2, 3]
Here, list1 stores a reference to the list [0, 1, 2, 3], and list2 is given that same reference. When the list is modified through the reference in list1, the list referenced by list2 is also modified.
>>> def foo(bar):
for i in range(2):
if bar == []:
bar.append(4)
else:
bar[0] += 1
>>> myvar = []
>>> foo(myvar)
>>> myvar
[5]
Passing the list myvar to the function foo() as the argument bar assigns a pointer to the list [] to the parameter bar. Since myvar and bar reference the same list, when the list is modified through bar, retrieving the list through myvar reflects the same change.
Hopefully this clarifies any confusion!
Because empty list created first in the memory location and second time it refers same list instead of creating new one.
first bar =[] is empty.
for loop checks if bar is empty, add 4 to it.
Now bar = [4] so again it checks for bar == [] but now it's value is 4. So it enters into else part and add 1.
Now value of bar becomes 5.
During this process it bar refers to same memory location.
Below example will help you to understand it better way,
def foo (bar = []):
bar.append('baz')
return bar
foo()
['baz']
foo()
['baz', 'baz']
foo()
['baz', 'baz', 'baz']
It keep appending default value 'baz' to the existing list each time foo() was called.
This is because, the default value for a function argument is only evaluated once, at the time of function is defined.
Thus the bar argument is initialized to its default(i.e. an empty list) only when foo() is first defines, but then call to foo() will continue to use the same list to which bar was originally initialized.
I'm a bit confused about modifying tuple members. The following doesn't work:
>>> thing = (['a'],)
>>> thing[0] = ['b']
TypeError: 'tuple' object does not support item assignment
>>> thing
(['a'],)
But this does work:
>>> thing[0][0] = 'b'
>>> thing
(['b'],)
Also works:
>>> thing[0].append('c')
>>> thing
(['b', 'c'],)
Doesn't work, and works (huh?!):
>>> thing[0] += 'd'
TypeError: 'tuple' object does not support item assignment
>>> thing
(['b', 'c', 'd'],)
Seemingly equivalent to previous, but works:
>>> e = thing[0]
>>> e += 'e'
>>> thing
(['b', 'c', 'd', 'e'],)
So what exactly are the rules of the game, when you can and can't modify something inside a tuple? It seems to be more like prohibition of using the assignment operator for tuple members, but the last two cases are confusing me.
You can always modify a mutable value inside a tuple. The puzzling behavior you see with
>>> thing[0] += 'd'
is caused by +=. The += operator does in-place addition but also an assignment — the in-place addition works just file, but the assignment fails since the tuple is immutable. Thinking of it like
>>> thing[0] = thing[0] + 'd'
explains this better. We can use the dis module from the standard library to look at the bytecode generated from both expressions. With += we get an INPLACE_ADD bytecode:
>>> def f(some_list):
... some_list += ["foo"]
...
>>> dis.dis(f)
2 0 LOAD_FAST 0 (some_list)
3 LOAD_CONST 1 ('foo')
6 BUILD_LIST 1
9 INPLACE_ADD
10 STORE_FAST 0 (some_list)
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
With + we get a BINARY_ADD:
>>> def g(some_list):
... some_list = some_list + ["foo"]
>>> dis.dis(g)
2 0 LOAD_FAST 0 (some_list)
3 LOAD_CONST 1 ('foo')
6 BUILD_LIST 1
9 BINARY_ADD
10 STORE_FAST 0 (some_list)
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
Notice that we get a STORE_FAST in both places. This is the bytecode that fails when you try to store back into a tuple — the INPLACE_ADD that comes just before works fine.
This explains why the "Doesn't work, and works" case leaves the modified list behind: the tuple already has a reference to the list:
>>> id(thing[0])
3074072428L
The list is then modified by the INPLACE_ADD and the STORE_FAST fails:
>>> thing[0] += 'd'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
So the tuple still has a reference to the same list, but the list has been modified in-place:
>>> id(thing[0])
3074072428L
>>> thing[0]
['b', 'c', 'd']
You can't modify the tuple, but you can modify the contents of things contained within the tuple. Lists (along with sets, dicts, and objects) are a reference type and thus the "thing" in the tuple is just a reference - the actual list is a mutable object which is pointed to by that reference and can be modified without changing the reference itself.
( + ,) <--- your tuple (this can't be changed)
|
|
v
['a'] <--- the list object your tuple references (this can be changed)
After thing[0][0] = 'b':
( + ,) <--- notice how the contents of this are still the same
|
|
v
['b'] <--- but the contents of this have changed
After thing[0].append('c'):
( + ,) <--- notice how this is still the same
|
|
v
['b','c'] <--- but this has changed again
The reason why += errors is that it's not completely equivalent to .append() - it actually does an addition and then an assignment (and the assignment fails), rather than merely appending in-place.
You cannot replace an element of a tuple, but you can replace the entire contents of the element. This will work:
thing[0][:] = ['b']
>>> salaries = '{"Alfred" : 300, "Jane" : 400 }'
>>> sal = json.loads(salaries)["Hritik"]=0
>>> sal
0
>>> sal = json.loads(salaries)
>>> sal["Hritik"]=0
>>> sal
{'Alfred': 300, 'Jane': 400, 'Hritik': 0}
>>> type(json.loads(salaries))
<class 'dict'>
>>> type(sal)
<class 'dict'>
Why can't I append to the dict returned by json.loads inline as I can do with the dict sal ?
Doesn't json.loads returns just a dict and that should be same as any other dict ?
The assignment statement x = y = z implies that both x and y will take on the value of z.
As an example, look at the byte code for the assignment a = b = 2:
In [45]: import dis; dis.dis(compile('a = b = 2', '', 'exec'))
1 0 LOAD_CONST 0 (2)
3 DUP_TOP
4 STORE_NAME 0 (a)
7 STORE_NAME 1 (b)
10 LOAD_CONST 1 (None)
13 RETURN_VALUE
With 4 STORE_NAME, a is assigned first to 2, followed by 7 STORE_NAME where b is then assigned to the same value, 2.
So, with
sal = json.loads(salaries)["Hritik"] = 0
sal receives the value 0. Also, a temporary variable is created when you call json.loads and that is modified, following which the reference is lost.
In order to get this to work, you'll need to break this up into 2 parts, as you have already done.
sal = json.loads(salaries)
sal['Hritik'] = 0
Why can't I append to the dict returned by json.loads inline as I can do with the dict sal ?
You can, and you do, but then you just discard that dict. It doesn't have any effect on the salaries variable, and you didn't assign the dict to sal. You assigned 0 to sal.
When you assign sal = json.loads(salaries), that makes a new dict, unrelated to the first dict, and then you actually assign the new dict to sal. Modifications to this new dict are still visible when you view the dict through sal.
This question already has answers here:
Immutable vs Mutable types
(18 answers)
Closed 7 years ago.
I wonder why reference variable(dict, list) is accessible from nested function while simple object is not.
In [1]: a = 1
In [2]: b = [1]
In [3]: c = {"c" : 1}
In [4]:
In [4]: def f():
...: a = 2
...: b[0] = 2
...: c["c"] = 2
...:
In [5]: f()
In [6]:
In [6]: print a
1
In [7]: print b
[2]
In [8]: print c
{'c': 2}
Integer is immutable, that means when python execute
a=2 inside f()
It create a new local object. While list and dictionary is mutable, so it can be modified in place.
>>> a = 0
>>> id(a)
31367908
>>> ls = [1,2]
>>> id(ls)
50082760
>>> def f():
a = 2
print id(a)
ls[0] = 5
print id(ls)
>>> f()
31367884
50082760
With statement a = 2 you are creating new local variable with value 2. However when you modify b[0] or c['c'] you are mutating global variable. If you have to deal with mutable types in function (work with them as with local variables) you have to initialize appropriate variable before mutating it:
def f():
c = {}
b = []
c['c'] = 2
b.append(1)
See Variables and scope chapter with great description of how scope works in python and get more info about mutable and immutable types.
So let's say I wanna make a dictionary. We'll call it d. But there are multiple ways to initialize a dictionary in Python! For example, I could do this:
d = {'hash': 'bang', 'slash': 'dot'}
Or I could do this:
d = dict(hash='bang', slash='dot')
Or this, curiously:
d = dict({'hash': 'bang', 'slash': 'dot'})
Or this:
d = dict([['hash', 'bang'], ['slash', 'dot']])
And a whole other multitude of ways with the dict() function. So obviously one of the things dict() provides is flexibility in syntax and initialization. But that's not what I'm asking about.
Say I were to make d just an empty dictionary. What goes on behind the scenes of the Python interpreter when I do d = {} versus d = dict()? Is it simply two ways to do the same thing? Does using {} have the additional call of dict()? Does one have (even negligible) more overhead than the other? While the question is really completely unimportant, it's a curiosity I would love to have answered.
>>> def f():
... return {'a' : 1, 'b' : 2}
...
>>> def g():
... return dict(a=1, b=2)
...
>>> g()
{'a': 1, 'b': 2}
>>> f()
{'a': 1, 'b': 2}
>>> import dis
>>> dis.dis(f)
2 0 BUILD_MAP 0
3 DUP_TOP
4 LOAD_CONST 1 ('a')
7 LOAD_CONST 2 (1)
10 ROT_THREE
11 STORE_SUBSCR
12 DUP_TOP
13 LOAD_CONST 3 ('b')
16 LOAD_CONST 4 (2)
19 ROT_THREE
20 STORE_SUBSCR
21 RETURN_VALUE
>>> dis.dis(g)
2 0 LOAD_GLOBAL 0 (dict)
3 LOAD_CONST 1 ('a')
6 LOAD_CONST 2 (1)
9 LOAD_CONST 3 ('b')
12 LOAD_CONST 4 (2)
15 CALL_FUNCTION 512
18 RETURN_VALUE
dict() is apparently some C built-in. A really smart or dedicated person (not me) could look at the interpreter source and tell you more. I just wanted to show off dis.dis. :)
As far as performance goes:
>>> from timeit import timeit
>>> timeit("a = {'a': 1, 'b': 2}")
0.424...
>>> timeit("a = dict(a = 1, b = 2)")
0.889...
#Jacob: There is a difference in how the objects are allocated, but they are not copy-on-write. Python allocates a fixed-size "free list" where it can quickly allocate dictionary objects (until it fills). Dictionaries allocated via the {} syntax (or a C call to PyDict_New) can come from this free list. When the dictionary is no longer referenced it gets returned to the free list and that memory block can be reused (though the fields are reset first).
This first dictionary gets immediately returned to the free list, and the next will reuse its memory space:
>>> id({})
340160
>>> id({1: 2})
340160
If you keep a reference, the next dictionary will come from the next free slot:
>>> x = {}
>>> id(x)
340160
>>> id({})
340016
But we can delete the reference to that dictionary and free its slot again:
>>> del x
>>> id({})
340160
Since the {} syntax is handled in byte-code it can use this optimization mentioned above. On the other hand dict() is handled like a regular class constructor and Python uses the generic memory allocator, which does not follow an easily predictable pattern like the free list above.
Also, looking at compile.c from Python 2.6, with the {} syntax it seems to pre-size the hashtable based on the number of items it's storing which is known at parse time.
Basically, {} is syntax and is handled on a language and bytecode level. dict() is just another builtin with a more flexible initialization syntax. Note that dict() was only added in the middle of 2.x series.
Update: thanks for the responses. Removed speculation about copy-on-write.
One other difference between {} and dict is that dict always allocates a new dictionary (even if the contents are static) whereas {} doesn't always do so (see mgood's answer for when and why):
def dict1():
return {'a':'b'}
def dict2():
return dict(a='b')
print id(dict1()), id(dict1())
print id(dict2()), id(dict2())
produces:
$ ./mumble.py
11642752 11642752
11867168 11867456
I'm not suggesting you try to take advantage of this or not, it depends on the particular situation, just pointing it out. (It's also probably evident from the disassembly if you understand the opcodes).
dict() is used when you want to create a dictionary from an iterable, like :
dict( generator which yields (key,value) pairs )
dict( list of (key,value) pairs )
Funny usage:
def func(**kwargs):
for e in kwargs:
print(e)
a = 'I want to be printed'
kwargs={a:True}
func(**kwargs)
a = 'I dont want to be printed'
kwargs=dict(a=True)
func(**kwargs)
output:
I want to be printed
a
In order to create an empty set we should use the keyword set before it
i.e set() this creates an empty set where as in dicts only the flower brackets can create an empty dict
Lets go with an example
print isinstance({},dict)
True
print isinstance({},set)
False
print isinstance(set(),set)
True