This question already has answers here:
Scope of python variable in for loop
(10 answers)
Closed 5 months ago.
Is it guaranteed that the iteration variable will always follow the original sequence no matter how you modify the variable itself?
The doc doesn't mention anything about this.
I've seen similar questions elsewhere but none of them gives authoritative answers. I've tested 100 times and it worked 100 times but I'm still not sure whether this is guaranteed. So please give some references.
Yes. It's totally safe to assign to the loop variable(s) in a for loop.
From The for statement docs:
The for-loop makes assignments to the variables(s) in the target list.
This overwrites all previous assignments to those variables including
those made in the suite of the for-loop:
for i in range(10):
print(i)
i = 5 # this will not affect the for-loop
# because i will be overwritten with the next
# index in the range
This information is also available via help('for')
I agree with PM 2Ring's answer, but you ask if it is guaranteed the loop will "follow the original sequence". For an iterator defined by a generator function, this cannot be guaranteed, because the sequence may be altered in between iterations; and it is possible to alter the way a generator works by altering a mutable element in the sequence.
For example:
def strange_iter():
x = {'value': 1}
while x['value'] < 30:
x = {'value': x['value'] + 1}
yield x
for d in strange_iter():
print(d['value'])
if d['value'] == 10:
d['value'] = 15 # loop will skip from 10 to 16
I'm not sure if such generators would have any use in practice or are best considered a bad practice to be avoided.
The loop will use the original sequence, for example:
a = [1, 2,3 ]
for index,number in enumerate(a):
print(a[index])
index = 0
number = 0
Output:
1
2
3
I think the syntax
for item in stuff:
...
creates an iterator by calling iter(stuff), then values are extracted by calling next() on that iterator.
Short answer, It depends....
It depends on what you meant by safe. Technically there is nothing wrong modifying the variable used for iteration. When you iterate over a list or something, you're actually iterating over an iterator. It returns the value contained in the container (list, dict whatever)...
If you modify the variable returned by the iterator, you have to keep in mind about the mutability of the object you're trying to modify.
If you iterate over integers or strings, you're not actually modifying the variable but affecting a new value to the variable name.. Thus, you're not modifying the value contained in the container.
But if you're iterating over a container with mutable objects (let say dicts).. Modifying the variable by changing the content, will have an effect on the value contained in the container since they are the same values.
But doing something like this would have no effect at all on the value in the container as you're not modifying the value pointed by the variable name but changing to which value the variable name points to:
a = [{}, {}, {}]
for x in a:
x = {'val': x}
While this will:
a = [{}, {}, {}]
for x in a:
x['v'] = 1
Related
2 part question. Given the following code, my understanding is that i is a variable created within the loop function. As such, the changes have local scope and should not carry over ‘outside’ the function.
list = [3,4,5,6,7]
for element in list:
i = element * 100
print(i)
print(i)
printing i within the loop will produce the correct mathematical change. However, the second print(i) outside the loop returns 700. This technically refers to the correct reassignment of element 7 from the original list. So if changes in the loop only exist within the loop, why is it that this last one carried over ‘outside the loop’?
Furthermore, why is it that print(i) outside the loop returns the change to the last element? Why not the first element? Why not all of them? Is there some function I can call outside the loop to see the changes applied to elements 3,4,5,6?
Part 2 of my question - I know for a change to apply outside a loop, you should target the element via its index itself. Eg use for ‘element’ in range(len(list)). But can one also do this with enumerate? If so, how?
It seems that enumerate returns an object in the form of a tuple (it adds an ‘index counter’ as the first element, and keeps list as the 2nd element). And since tuples are immutable it would seem there is no way to effect a change on a global scope, is that correct?
For example, when I run the following code:
my_list = [1,2,100]
for xyz in enumerate(my_list):
xyz = 2 * xyz
print(xyz)
All it does it return to me the final element in my_list, with its index counter, concatenated to itself (‘doubled’). Eg (2,100) has become (2,100, 2,100). So is there no way to use enumerate to change elements within the original list?
To your first question, let's go through your misunderstandings one by one:
i is a variable created within the loop function
No. There is no "loop function" here. (User #0x5453 already mentioned this briefly in a comment.) The for-loop is just a language construct to facilitate iteration. Thus, i is just a variable that happens to be created, assigned, and re-assigned multiple times throughout the iteration of the for-loop.
the changes have local scope and should not carry over ‘outside’
See above. There is no other scope here. All those lines of code, starting with the list assignment and ending with that print(i) after the loop are all in the same scope. This should answer the following of your questions:
why is it that this last [change] carried over ‘outside the loop’?
why is it that print(i) outside the loop returns the change to the last element?
The variable i was re-assigned multiple times. In the last iteration of the for-loop, the value assigned to i was 7 * 100, i.e. 700. Then the loop ends, and nothing else happens to i, so it still holds the value 700.
To the second question:
for a change to apply outside a loop, you should target the element via its index itself
If you refer to the your list, then the loop has nothing to do with that at all. If you want to change an element of the list, yes, re-assignment only works by targeting the index of that element in the list:
my_list = [1, 2, 100]
my_list[2] = 420
print(my_list) # output: [1, 2, 420]
can one also do this with enumerate?
No, enumerate is something else entirely. Calling enumerate on my_list returns an iterator, which can then be used in a for-loop just like the original list can, but each element that iterator produces is (in its simplest form) a 2-tuple, where the second element is an element from my_list and the first element is the index of that element.
Try this:
for tup in enumerate(my_list):
print(tup[0], tup[1])
This will give you:
0 1
1 2
2 420
Since tuples are iterables, they support unpacking, which is why it is common to go for more readability and instead do this:
for idx, element in enumerate(my_list):
print(idx, element)
The output is the same.
Nothing about this changes my_list. All these operations do, is iterate over it in some way, which just produces elements from it one by one.
Now, you do this:
for xyz in enumerate(my_list):
xyz = 2 * xyz
Since xyz is just that index-element-tuple produced by enumerate, multiplying it by 2 just concatenates it with itself, as you correctly noted, creating a new tuple and re-assigning it to xyz. In the next iteration it gets overwritten again by what enumerate produces, then re-assigned again by you and so on. Again, none of that changes my_list, it just changes that xyz variable.
If I understood you correctly, you want to be able to mutate your list within a loop over its elements. While this can quickly lead to dangerous territory, a simple working example would be this:
for idx, element in enumerate(my_list):
my_list[idx] = 2 * element
print(my_list)
The output:
[2, 4, 840]
Now we actually change/re-assign specific elements in that list. As before, assignment only works with an index. But the index is what we get as the first element of the 2-tuple provided by enumerate in this case.
However, there are sometimes (arguably) more elegant ways to accomplish the same thing. For instance, we could use list comprehension to achieve the same result:
my_list = [element * 2 for element in my_list]
Here we overwrite my_list with a new one that we created through list comprehension iterating over the original list.
Hope this helps clear up some misconceptions.
This question already has answers here:
How to modify list entries during for loop?
(10 answers)
Closed 5 months ago.
When I try this code:
bar = [1,2,3]
print(bar)
for foo in bar:
print(id(foo))
foo = 0
print(id(foo))
print(bar)
I get this result:
[1, 2, 3]
5169664
5169676
5169652
5169676
5169640
5169676
[1, 2, 3]
I expected the end result to be [0,0,0] and that id would return identical values for each iteration. Why does it behave like this? How can I elegantly assign back to the elements of the list, without using enumerate or range(len(bar))?
See also: How to change variables fed into a for loop in list form
First of all, you cannot reassign a loop variable—well, you can, but that won’t change the list you are iterating over. So setting foo = 0 will not change the list, but only the local variable foo (which happens to contain the value for the iteration at the begin of each iteration).
Next thing, small numbers, like 0 and 1 are internally kept in a pool of small integer objects (This is a CPython implementation detail, doesn’t have to be the case!) That’s why the ID is the same for foo after you assign 0 to it. The id is basically the id of that integer object 0 in the pool.
If you want to change your list while iterating over it, you will unfortunately have to access the elements by index. So if you want to keep the output the same, but have [0, 0, 0] at the end, you will have to iterate over the indexes:
for i in range(len(bar)):
print id(bar[i])
bar[i] = 0
print id(bar[i])
print bar
Otherwise, it’s not really possible, because as soon as you store a list’s element in a variable, you have a separate reference to it that is unlinked to the one stored in the list. And as most of those objects are immutable and you create a new object when assigning a new value to a variable, you won’t get the list’s reference to update.
Yes, the output you got is the ordinary Python behavior. Assigning a new value to foo will change foo's id, and not change the values stored in bar.
If you just want a list of zeroes, you can do:
bar = [0] * len(bar)
If you want to do some more complicated logic, where the new assignment depends on the old value, you can use a list comprehension:
bar = [x * 2 for x in bar]
Or you can use map:
def double(x):
return x * 2
bar = map(double, bar)
you actually didnt change the list at all.
the first thing for loop did was to assign bar[0] to foo(equivalent to foo = bar[0]). foo is just an reference to 1. Then you assign another onject 0 to foo. This changed the reference of foo to 0. But you didnt change bar[0]. Remember, foo as a variable, references bar[0], but assign another value/object to foo doesn't affect bar[0] at all.
bar = [0 for x in bar]
Long answer : foo is just a local name, rebinding does not impact the list. Python variables are really just key:value pairs, not symbolic names for memory locations.
I am a beginner in python and I find this about mutabilty quite confusing and non intuitive. Given a list:
lst = [[1, 2, 3], [4, 5, 6]]
And trying to change the list within a for-loop.
for i in lst:
i = "test"
I understand that this does not change the list. But:
for i in lst:
i[1] = "test"
I was surprised that referring to the sublist led to following outcome:
[[1, 'test', 3], [4, 'test', 6]]
I tried to understand with a visualizer but I don't get it. Would anybody please explain this in plain words? Thank you.
In the first case, you simply have a copied reference to the element.
i ---> lst[n]
In the latter case, you are dereferencing the reference, and changing data (not in a copy):
i[i] ---> lst[n][i]
Therefore assigning to i[n] will point to the actual mutable object, not a copy of it.
Assignment (which is what the = operator does) assigns a value to a name.
i is a variable name in the local namespace. At the time it runs inside the for loop, it refers to a list. By assigning a string to it, you cause the name to no longer refer to a list, but instead refer to the string. The list is unaffected -- the only thing that changes is the value associated with the name.
i[1] is a name that specifies a specific location inside one of the two lists, depending on what i is set to at the time. By assigning a string to it, you cause the name to no longer refer to the object that previously occupied that space inside the list (an integer, 2 or 5 depending on the list) and instead refer to the string. The integer is unaffected -- the only thing that changes is the value associated with the name.
So in each case, assignment is doing the same thing -- it's making a name refer to a new thing instead of the old thing it referred to. The difference is that the second case is a special name in that it refers to a property of a mutable object.
for does not make copies of each element it yields. As such, the yielded object retains all the properties of the original (since it is the original), including mutability.
since for loop in case of string is called in different way
like
for i in lst:
it means if lst is list or array of string then i is referenced to the value of x[i] directly that is why in first case result was "test " two times because length of "lst" was just 2
while in second case i[1] means
i ------lst[i]
i[1]--------`-lst[i][1] means
first value equal lst[0][1]---[1,'test',3]
second value equal lst[1][1]--[4,'test',6]
This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 9 months ago.
My code :
locs = [ [1], [2] ]
for loc in locs:
loc = []
print locs
# prints => [ [1], [2] ]
Why is loc not reference of elements of locs ?
Python : Everything is passed as reference unless explicitly copied [ Is this not True ? ]
Please explain.. how does python decides referencing and copying ?
Update :
How to do ?
def compute(ob):
if isinstance(ob,list): return process_list(ob)
if isinstance(ob,dict): return process_dict(ob)
for loc in locs:
loc = compute(loc) # What to change here to make loc a reference of actual locs iteration ?
locs must contain the final processed response !
I don't want to use enumerate, is it possible without it ?
Effbot (aka Fredrik Lundh) has described Python's variable passing style as call-by-object: http://effbot.org/zone/call-by-object.htm
Objects are allocated on the heap and pointers to them can be passed around anywhere.
When you make an assignment such as x = 1000, a dictionary entry is created that maps the string "x" in the current namespace to a pointer to the integer object containing one thousand.
When you update "x" with x = 2000, a new integer object is created and the dictionary is updated to point at the new object. The old one thousand object is unchanged (and may or may not be alive depending on whether anything else refers to the object).
When you do a new assignment such as y = x, a new dictionary entry "y" is created that points to the same object as the entry for "x".
Objects like strings and integers are immutable. This simply means that there are no methods that can change the object after it has been created. For example, once the integer object one-thousand is created, it will never change. Math is done by creating new integer objects.
Objects like lists are mutable. This means that the contents of the object can be changed by anything pointing to the object. For example, x = []; y = x; x.append(10); print y will print [10]. The empty list was created. Both "x" and "y" point to the same list. The append method mutates (updates) the list object (like adding a record to a database) and the result is visible to both "x" and "y" (just as a database update would be visible to every connection to that database).
Hope that clarifies the issue for you.
Everything in Python is passed and assigned by value, in the same way that everything is passed and assigned by value in Java. Every value in Python is a reference (pointer) to an object. Objects cannot be values. Assignment always copies the value (which is a pointer); two such pointers can thus point to the same object. Objects are never copied unless you're doing something explicit to copy them.
For your case, every iteration of the loop assigns an element of the list into the variable loc. You then assign something else to the variable loc. All these values are pointers; you're assigning pointers; but you do not affect any objects in any way.
It doesn't help in Python to think in terms of references or values. Neither is correct.
In Python, variables are just names. In your for loop, loc is just a name that points to the current element in the list. Doing loc = [] simply rebinds the name loc to a different list, leaving the original version alone.
But since in your example, each element is a list, you could actually mutate that element, and that would be reflected in the original list:
for loc in locs:
loc[0] = loc[0] * 2
When you say
loc = []
you are rebinding the loc variable to a newly created empty list
Perhaps you want
loc[:] = []
Which assigns a slice (which happens to be the whole list) of loc to the empty list
Everything is passed by object. Rebinding and mutating are different operations.
locs = [ [1], [2] ]
for loc in locs:
del loc[:]
print locs
Why is loc not reference of elements of locs ?
It is. Or at least, it is in the same sense that every other variable in Python is. Python variables are names, not storage. loc is a name that is used to refer to elements of [[1,2], [3,4]], while locs is a name that refers to the entire structure.
loc = []
This does not mean "look at the thing that loc names, and cause it to turn into []". It cannot mean that, because Python objects are not capable of such a thing.
Instead, it means "cause loc to stop being a name for the thing that it's currently a name for, and start instead being a name for []". (Of course, it means the specific [] that's provided there, since in general there may be several objects in memory that are the same.)
Naturally, the contents of locs are unchanged as a result.
How do I remove a character from an element in a list?
Example:
mylist = ['12:01', '12:02']
I want to remove the colon from the time stamps in a file, so I can more easily convert them to a 24hour time. Right now I am trying to loop over the elements in the list and search for the one's containing a colon and doing a substitute.
for num in mylist:
re.sub(':', '', num)
But that doesn't seem to work.
Help!
The list comprehension solution is the most Pythonic one, but, there's an important twist:
mylist[:] = [s.replace(':', '') for s in mylist]
If you assign to mylist, the barename, as in the other answer, rather than to mylist[:], the "whole-list slice", as I recommend, you're really doing something very different than "replacing entries in the list": you're making a new list and just rebinding the barename that you were previously using to refer to the old list.
If that old list is being referred to by multiple names (including entries in containers), this rebinding doesn't affect any of those: for example, if you have a function which takes mylist as an argument, the barename assignment has any effect only locally to the function, and doesn't alter what the caller sees as the list's contents.
Assigning to the whole-list slice, mylist[:] = ..., alters the list object rather than mucking around with switching barenames' bindings -- now that list is truly altered and, no matter how it's referred to, the new value is what's seen. For example, if you have a function which takes mylist as an argument, the whole-list slice assignment alters what the caller sees as the list's contents.
The key thing is knowing exactly what effect you're after -- most commonly you'll want to alter the list object, so, if one has to guess, whole-list slice assignment is usually the best guess to take;-). Performance-wise, it makes no difference either way (except that the barename assignment, if it keeps both old and new list objects around, will take up more memory for whatever lapse of time both objects are still around, of course).
Use a list comprehension to generate a new list:
>>> mylist = ['12:01', '12:02']
>>> mylist = [s.replace(':', '') for s in mylist]
>>> print mylist
['1201', '1202']
The reason that your solution doesn't work is that re.sub returns a new string -- strings are immutable in Python, so re.sub can't modify your existing strings.
for i, num in enumerate(mylist):
mylist[i] = num.replace(':','')
You have to insert the return of re.sub back in the list. Below is for a new list. But you can do that for mylist as well.
mylist = ['12:01', '12:02']
tolist = []
for num in mylist:
a = re.sub(':', '', num)
tolist.append(a)
print tolist
Strings in python are immutable, meaning no function can change the contents of an existing string, only provide a new string. Here's why.
See here for a discussion on string types that can be changed. In practice though, it's better to adjust to the immutability of strings.
Instead of list comprehension, you can also use a map call:
mylist = ['12:01', '12:02']
map(lambda f: f.replace(':', ''), mylist)
Returns:
['1201', '1202']