I'm new to python. I'm trying to learn data extracting from an Excel file. I encountered the following statement:
sheet_data = [[sheet.cell_value(r, col) for col in range(sheet.ncols)] for r in range(sheet.nrows)]
I understand regular for loops, but not the below version:
x for y in range()
What does it mean when you have a variable x before the for y in range()?
The for statement is used for looping over a list. This is referred to as an iterator. When it is encompassed by [..], this is referred to as a list comprehension.
List comprehensions allow you to transform one list into another. This is sometimes referred to as a mapping i.e. mapping from X -> Y where a function transforms the value of X into the returned value of Y
So, for example, in
[y + 2 for y in range(...)]
the for is iterating over all values in the list produced by the range(). Each list element has 2 added to each value of y, so the final result is a list where each element is 2 greater than the corresponding element in the source list. Thus, range(3) would produce [0, 1, 2] which then transforms into [2, 3, 4].
So [y for y in range(..)] wouldn't actually accomplish much.
I see that in the example you have provided there are two iterators, which complicates things a bit. But essentially, they are providing two reference variables: r and col, from which the final result is derived using these two variables.
List comprehensions are a very powerful tool in Python. Definitely worth knowing.
These are called list comprehensions in python. If you have a function do_something then the following two blocks are equivalent:
result = [do_something(y) for y in range(10)]
...
result = []
for y in range(10):
result.append(do_something(y))
Where range(10) could be any iterable.
Think of them as quick ways to create lists. They work for dictionaries too as of python 2.7. This tutorial may be helpful.
The "x" is an arbitrary variable name that holds the values of the sequence object. Using it in a list comprehension or in a generator expression will return the items in the iterable object that is being stepped through.
Related
I'm doing my programming coursework, and I've come across an issue
gamecentre1 = [winnerscore, winner]
organise = []
organise.extend([gamecentre1])
from operator import attrgetter, itemgetter
sorted(organise, key= itemgetter(0))
print(organise)
f = open("gameresults.txt","w")
f.write("Here are the winners of the games: \n")
f.write(str(organise))
f.write("\n")
f.close()
I'm trying to add two variables to a list, and add that list to another list. Then, I want to organise that larger list based off of the integer variable of the sublist (the integer is the winnerscore). But, the problem is that I haven't been able to organise them properly, and I worry that since I have to append the same list with the same variables to the larger list without overwriting the existing list in it, the larger list will just have the same values over and over again.
This is because I want to store the variables every time the program runs, without getting rid of the values of the variable from the previous game.
How do I do this?
I'm trying to add two variables to a list, and add that list to another list. Then, I want to organise that larger list based off of the integer variable of the sublist
It sounds like what you want is a nested list - a big list of small lists, such that each small list is independent of the others. The key problem in your code right now that's blocking you from accomplishing this is extend() - this is essentially list concatenation, which isn't what you want. For example,
x = [1, 2]
x.extend([3, 4]) # x == [1, 2, 3, 4]
Instead, try using append(), which adds its argument as the next value in the list. For example:
x = []
x.append([3, 4]) # [[3, 4]]
x.append([1, 2]) # [[3, 4], [1, 2]]
Now in the above example, we have a list x that's two elements long, and each of those elements is itself a list of two elements. Now we can plug that into sort:
y = sorted(x, key=lambda i: i[0])
print(y)
# [[1, 2], [3, 4]]
(the lambda i: i[0] is really just a more elegant way of what you're doing with itemgettr(0) - it's a lambda, a small inline function, that takes one argument i and returns the 0th element of i. In this case, the i that gets passed in is one of the smaller lists).
Now you have a sorted list of smaller lists, and you can do whatever you need with them.
So I know that to get a single column, I'd have to write
a = list(zip(*f)[0])
and the resulting a will be a list containing the first element in the lists in f.
How do I do this to get more than one element per list? I tried
a = list(zip(*f)[1:19])
But it just returned a list of lists where the inner list is the composed of the ith element in every list.
The easy way is not to use zip(). Instead, use a list comprehension:
a = [sub[1:19] for sub in f]
If it is actually the second half that you are looking for:
a = [sub[len(sub) // 2:] for sub in f]
That will include the 3 in [1, 2, 3, 4, 5]. If you don't want to include it:
a = [sub[(len(sub) + 1) // 2:] for sub in f]
You should definitely prefer #zondo's solution for both performance and readability. However, a zip based solution is possible and would look as follows (in Python 2):
zip(*zip(*f)[1:19])
You should not consider this cycle of unpacking, zipping, slicing, unpacking and re-zipping in any serious code though ;)
In Python 3, you would have to cast both zip results to list, making this even less sexy.
This question already has answers here:
Why does this iterative list-growing code give IndexError: list assignment index out of range? How can I repeatedly add (append) elements to a list?
(9 answers)
Closed 4 months ago.
This is such a simple issue that I don't know what I'm doing wrong. Basically I want to iterate through the items in an empty list and increase each one according to some criteria. This is an example of what I'm trying to do:
list1 = []
for i in range(5):
list1[i] = list1[i] + 2*i
This fails with an list index out of range error and I'm stuck. The expected result (what I'm aiming at) would be a list with values:
[0, 2, 4, 6, 8]
Just to be more clear: I'm not after producing that particular list. The question is about how can I modify items of an empty list in a recursive way. As gnibbler showed below, initializing the list was the answer. Cheers.
Ruby (for example) lets you assign items beyond the end of the list. Python doesn't - you would have to initialise list1 like this
list1 = [0] * 5
So when doing this you are actually using i so you can just do your math to i and just set it to do that. there is no need to try and do the math to what is going to be in the list when you already have i. So just do list comprehension:
list1 = [2*i for i in range(5)]
Since you say that it is more complex, just don't use list comprehension, edit your for loop as such:
for i in range(5):
x = 2*i
list1[i] = x
This way you can keep doing things until you finally have the outcome you want, store it in a variable, and set it accordingly! You could also do list1.append(x), which I actually prefer because it will work with any list even if it's not in order like a list made with range
Edit: Since you want to be able to manipulate the array like you do, I would suggest using numpy! There is this great thing called vectorize so you can actually apply a function to a 1D array:
import numpy as np
list1 = range(5)
def my_func(x):
y = x * 2
vfunc = np.vectorize(my_func)
vfunc(list1)
>>> array([0, 2, 4, 6, 8])
I would advise only using this for more complex functions, because you can use numpy broadcasting for easy things like multiplying by two.
Your list is empty, so when you try to read an element of the list (right hand side of this line)
list1[i] = list1[i] + 2*i
it doesn't exist, so you get the error message.
You may also wish to consider using numpy. The multiplication operation is overloaded to be performed on each element of the array. Depending on the size of your list and the operations you plan to perform on it, using numpy very well may be the most efficient approach.
Example:
>>> import numpy
>>> 2 * numpy.arange(5)
array([0, 2, 4, 6, 8])
I would instead write
for i in range(5):
list1.append(2*i)
Yet another way to do this is to use the append method on your list. The reason you're getting an out of range error is because you're saying:
list1 = []
list1.__getitem__(0)
and then manipulate this item, BUT that item does not exist since your made an empty list.
Proof of concept:
list1 = []
list1[1]
IndexError: list index out of range
We can, however, append new stuff to this list like so:
list1 = []
for i in range(5):
list1.append(i * 2)
Is the following safe?
x = [1, 2, 3, 4]
x = [y+5 for y in x]
Does the list comprehension evaluate first, creating a new list, and then assign that new list to x?
I was told once that changing a list while iterating over it is an unsafe operation.
You aren't changing the list while iterating over it, you are creating a completely new list and then once it has been evaluated, you are binding it to the name x so everything is safe.
Yep, it's safe.
As you hinted, the right-hand side is evaluated first, and then its result (an entirely new list) is assigned to the name x. You're right that it's unsafe to change a list while iterating over it, but that's not happening in your code sample, so no worries.
Why do these two operations (append() resp. +) give different results?
>>> c = [1, 2, 3]
>>> c
[1, 2, 3]
>>> c += c
>>> c
[1, 2, 3, 1, 2, 3]
>>> c = [1, 2, 3]
>>> c.append(c)
>>> c
[1, 2, 3, [...]]
>>>
In the last case there's actually an infinite recursion. c[-1] and c are the same. Why is it different with the + operation?
To explain "why":
The + operation adds the array elements to the original array. The array.append operation inserts the array (or any object) into the end of the original array, which results in a reference to self in that spot (hence the infinite recursion in your case with lists, though with arrays, you'd receive a type error).
The difference here is that the + operation acts specific when you add an array (it's overloaded like others, see this chapter on sequences) by concatenating the element. The append-method however does literally what you ask: append the object on the right-hand side that you give it (the array or any other object), instead of taking its elements.
An alternative
Use extend() if you want to use a function that acts similar to the + operator (as others have shown here as well). It's not wise to do the opposite: to try to mimic append with the + operator for lists (see my earlier link on why). More on lists below:
Lists
[edit] Several commenters have suggested that the question is about lists and not about arrays. The question has changed, though I should've included this earlier.
Most of the above about arrays also applies to lists:
The + operator concatenates two lists together. The operator will return a new list object.
List.append does not append one list with another, but appends a single object (which here is a list) at the end of your current list. Adding c to itself, therefore, leads to infinite recursion.
As with arrays, you can use List.extend to add extend a list with another list (or iterable). This will change your current list in situ, as opposed to +, which returns a new list.
Little history
For fun, a little history: the birth of the array module in Python in February 1993. it might surprise you, but arrays were added way after sequences and lists came into existence.
The concatenation operator + is a binary infix operator which, when applied to lists, returns a new list containing all the elements of each of its two operands. The list.append() method is a mutator on list which appends its single object argument (in your specific example the list c) to the subject list. In your example this results in c appending a reference to itself (hence the infinite recursion).
An alternative to '+' concatenation
The list.extend() method is also a mutator method which concatenates its sequence argument with the subject list. Specifically, it appends each of the elements of sequence in iteration order.
An aside
Being an operator, + returns the result of the expression as a new value. Being a non-chaining mutator method, list.extend() modifies the subject list in-place and returns nothing.
Arrays
I've added this due to the potential confusion which the Abel's answer above may cause by mixing the discussion of lists, sequences and arrays.
Arrays were added to Python after sequences and lists, as a more efficient way of storing arrays of integral data types. Do not confuse arrays with lists. They are not the same.
From the array docs:
Arrays are sequence types and behave very much like lists, except that the type of objects stored in them is constrained. The type is specified at object creation time by using a type code, which is a single character.
append is appending an element to a list. if you want to extend the list with the new list you need to use extend.
>>> c = [1, 2, 3]
>>> c.extend(c)
>>> c
[1, 2, 3, 1, 2, 3]
Python lists are heterogeneous that is the elements in the same list can be any type of object. The expression: c.append(c) appends the object c what ever it may be to the list. In the case it makes the list itself a member of the list.
The expression c += c adds two lists together and assigns the result to the variable c. The overloaded + operator is defined on lists to create a new list whose contents are the elements in the first list and the elements in the second list.
So these are really just different expressions used to do different things by design.
The method you're looking for is extend(). From the Python documentation:
list.append(x)
Add an item to the end of the list; equivalent to a[len(a):] = [x].
list.extend(L)
Extend the list by appending all the items in the given list; equivalent to a[len(a):] = L.
list.insert(i, x)
Insert an item at a given position. The first argument is the index of the element before which to insert, so a.insert(0, x) inserts at the front of the list, and a.insert(len(a), x) is equivalent to a.append(x).
you should use extend()
>>> c=[1,2,3]
>>> c.extend(c)
>>> c
[1, 2, 3, 1, 2, 3]
other info: append vs. extend
See the documentation:
list.append(x)
Add an item to the end of the list; equivalent to a[len(a):] = [x].
list.extend(L)
- Extend the list by appending all the items in the given list;
equivalent to a[len(a):] = L.
c.append(c) "appends" c to itself as an element. Since a list is a reference type, this creates a recursive data structure.
c += c is equivalent to extend(c), which appends the elements of c to c.