import numpy as np
a = np.arange(0,60,5)
a = a.reshape(3,4)
for x in np.nditer(a, op_flags = ['readwrite']):
x[...] = 2*x
print 'Modified array is:'
print a
In the above code, why can't we simply write x=2*x instead of x[...]=2*x?
No matter what kind of object we were iterating over or how that object was implemented, it would be almost impossible for x = 2*x to do anything useful to that object. x = 2*x is an assignment to the variable x; even if the previous contents of the x variable were obtained by iterating over some object, a new assignment to x would not affect the object we're iterating over.
In this specific case, iterating over a NumPy array with np.nditer(a, op_flags = ['readwrite']), each iteration of the loop sets x to a zero-dimensional array that's a writeable view of a cell of a. x[...] = 2*x writes to the contents of the zero-dimensional array, rather than rebinding the x variable. Since the array is a view of a cell of a, this assignment writes to the corresponding cell of a.
This is very similar to the difference between l = [] and l[:] = [] with ordinary lists, where l[:] = [] will clear an existing list and l = [] will replace the list with a new, empty list without modifying the original. Lists don't support views or zero-dimensional lists, though.
Related
I want to iterate over a numpy array and do some calculations on the values. However, things are not as expected. To show what I mean, I simply wrote this code to read values from a numpy array and move them to another list.
a = array([1,2,1]).reshape(-1, 1)
u = []
for i in np.nditer(a):
print(i)
u.append(i)
print(u)
According to tutorial, nditer points to elements and as print(i) shows, i is the value. However, when I append that i to an array, the array doesn't store the value. The expected output is u = [1, 2, 1] but the output of the code is
1
2
1
[array(1), array(2), array(1)]
What does array(1) mean exactly and how can I fix that?
P.S: I know that with .tolist() I can convert a numpy array to a standard array. However, in that code, I want to iterate over numpy elements.
As already explained in your previous question, numpy.nditer yields numpy arrays. What is shown by print is only the representation of the object, not the content or type of the object (e.g., 1 and '1' have the same representation, not the same type).
import numpy as np
a = np.array([1,2,1]).reshape(-1, 1)
type(next(np.nditer(a)))
# numpy.ndarray
You just have a zero-dimensional array:
np.array(1).shape
# ()
There is no need to use numpy.nditer here. If you really want to iterate over the rows of your array with single column (and not use tolist), use:
u = []
for i in a[:,0]:
u.append(i)
u
# [1, 2, 1]
numpy.nditer actually returns a numpy array. If you want the actual value of this item, you can use the built in item() function:
a = array([1,2,1]).reshape(-1, 1)
u = []
for i in np.nditer(a):
u.append(i.item())
print(u)
A pure python equivalent of what's happening with your append is:
In [75]: alist = []
...: x = [0]
...: for i in range(3):
...: x[0] = i
...: print(x)
...: alist.append(x)
[0]
[1]
[2]
In [76]: alist
Out[76]: [[2], [2], [2]]
In [77]: x
Out[77]: [2]
x is modified in each loop, but only a reference is saved. The result is that all elements of the list are the same object, and display its last value.
I would like to update (prepend each one with additional elements) many numpy arrays in a loop, without having to repeat the code for each one.
I tried creating a list of all the arrays and looping through the items in that list and updating each one, but that doesn't change the original array.
import numpy as np
arr01 = [1,2,3]
arr02 = [4,5,6]
arr99 = [7,8,9]
print('initial arr01', arr01)
arraylist = [arr01, arr02, arr99]
for array in arraylist:
array = np.concatenate((np.zeros(3, dtype=int), array))
print('array being modified inside the loop', array)
print('final arr01', arr01)
In the sample code, I expected arr01, arr02, arr03 to all be modified with the prepended zeros.
array = np.concatenate((np.zeros(3, dtype=int), array)) does not change the current array but creates a new one and stores it inside the variable array. So for the solution you have to change the values of the array itself, which can be done with array[:].
That means the only change you would have to make is replacing this one line
array[:] = np.concatenate((np.zeros(3, dtype=int), array))
So your correct solution would be
import numpy as np
arr01 = [1,2,3]
arr02 = [4,5,6]
arr99 = [7,8,9]
print('initial arr01', arr01)
arraylist = [arr01, arr02, arr99]
for array in arraylist:
array[:] = np.concatenate((np.zeros(3, dtype=int), array))
print('array being modified inside the loop', array)
print('final arr01', arr01)
I am wondering why when I delete the original array it affects the copied array:
arr = [1,2,3]
arr1 = arr
del arr[:]
print(arr1) #this prints []
but when I modify the elements of the original array there is no effect on the copied array:
arr = [1,2,3]
arr1 = arr
arr = [4,5,6]
print(arr1) #this prints [1,2,3]
If someone can explain this issue I appreciate your help, thanks in advance.
You did not modify the elements of the original array, but rather re-assigned a new list to the arr variable. Your intuition of thinking changes to elements would be reflected in arr1 if you properly accessed its elements is indeed true as lists are mutable in Python. For instance,
arr = [1,2,3]
arr1 = arr
arr[1] = 4
print(arr1) #this prints [1,4,3]
Objects in python would be considered to be passed by reference. It's a bit different than that, however.
arr = [1, 2, 3]
This statement does two things. First it creates a list object in memory; second it points the "arr" label to this object.
arr1 = arr
This statement creates a new label "arr1" and points it to the same list object pointed to by arr.
Now in your original code you did this:
del arr[:]
This deleted the elements of the list object and now any label pointing to it will point to an empty list. In your second batch of code you did this:
arr = [4, 5, 6]
This created a new list object in memory, and pointed the "arr" label to it. Now you have two list objects in memory, each being pointed to by two different labels. I just checked on my console, and arr points to [4,5,6] and arr1 to [1,2,3].
Here is a good post about it: http://robertheaton.com/2014/02/09/pythons-pass-by-object-reference-as-explained-by-philip-k-dick/
There are several ways to copy an array, but
arr1 = arr
is not one of them (in C-speak, that is just an aliased pointer). Try one of
arr1 = arr[:] # slice that includes all elements, i.e. a shallow copy
arr1 = copy.copy(arr) # from the copy module, does a shallow copy
arr1 = copy.deepcopy(arr) # you guessed it.. ;-)
after any of those you will see that changes to arr1 and arr are independent (of course if you're using a shallow copy then the items in the list will be shared).
You did not do a full copy of the list, instead you just re-assigned the variable name.
In order to make a deep-copy of the list do the following:
arr1 = [x for x in arr]
I'm new to python. I'm trying to learn data extracting from an Excel file. I encountered the following statement:
sheet_data = [[sheet.cell_value(r, col) for col in range(sheet.ncols)] for r in range(sheet.nrows)]
I understand regular for loops, but not the below version:
x for y in range()
What does it mean when you have a variable x before the for y in range()?
The for statement is used for looping over a list. This is referred to as an iterator. When it is encompassed by [..], this is referred to as a list comprehension.
List comprehensions allow you to transform one list into another. This is sometimes referred to as a mapping i.e. mapping from X -> Y where a function transforms the value of X into the returned value of Y
So, for example, in
[y + 2 for y in range(...)]
the for is iterating over all values in the list produced by the range(). Each list element has 2 added to each value of y, so the final result is a list where each element is 2 greater than the corresponding element in the source list. Thus, range(3) would produce [0, 1, 2] which then transforms into [2, 3, 4].
So [y for y in range(..)] wouldn't actually accomplish much.
I see that in the example you have provided there are two iterators, which complicates things a bit. But essentially, they are providing two reference variables: r and col, from which the final result is derived using these two variables.
List comprehensions are a very powerful tool in Python. Definitely worth knowing.
These are called list comprehensions in python. If you have a function do_something then the following two blocks are equivalent:
result = [do_something(y) for y in range(10)]
...
result = []
for y in range(10):
result.append(do_something(y))
Where range(10) could be any iterable.
Think of them as quick ways to create lists. They work for dictionaries too as of python 2.7. This tutorial may be helpful.
The "x" is an arbitrary variable name that holds the values of the sequence object. Using it in a list comprehension or in a generator expression will return the items in the iterable object that is being stepped through.
I am loading a file "data.imputation" which is 2 dimensional in variable 'x'. Variable 'y' is a copy of 'x'. I pop the first array from 'y' (y is 2D). Why is the change reflected on x? (The first array from 'x' is also popped)
ip = open('data.meanimputation','r')
x = pickle.load(ip)
y = x
y.pop(0)
At the start, len(x) == len(y). Even after y.pop(0), len(x) == len(y). Why is that? And how can I avoid it?
use y = x[:] instead of y = x. y = x means both y and x are now pointing to the same object.
Take a look at this example:
>>> x=[1,2,3,4]
>>> y=x
>>> y is x
True # it means both y and x are just references to a same object [1,2,3,4], so changing either of y or x will affect [1,2,3,4]
>>> y=x[:] # this makes a copy of x and assigns that copy to y,
>>> y is x # y & x now point to different object, so changing one will not affect the other.
False
If x is a list is list of list then [:] is of no use:
>>> x= [[1,2],[4,5]]
>>> y=x[:] #it makes a shallow copy,i.e if the objects inside it are mutable then it just copies their reference to the y
>>> y is x
False # now though y and x are not same object but the object contained in them are same
>>> y[0].append(99)
>>> x
[[1, 2, 99], [4, 5]]
>>> y
[[1, 2, 99], [4, 5]]
>>> y[0] is x[0]
True #see both point to the same object
in such case you should use copy module's deepcopy() function , it makes non-shallow copies of object.
y = x does not copy anything. It binds the name y to the same object already referred to by x. Assignment to a bare name never copies anything in Python.
If you want to copy an object, you need to copy it explicitly, using whatever methods are available for the object you're trying to copy. You don't say what kind of object x is, so there's no way to say how you might be able to copy it, but the copy module provides some functions that work for many types. See also this answer.