Develop Reference

List comprehensions in Python "freezing" Jupyter notebook

When I run the code below in Jupyter notebook, my "kernel" freezes (I get an asterisk between the square brackets like so: [*]):
x = [1,2,3,4]
for num in x:
x.append(num**2)
Why doesn't this code "append" the exponentiated numbers to the end of x?
The code below works and I understand why, but why doesn't the code above work:
x = [1,2,3,4]
out = []
for num in x:
out.append(num**2)
print(out)

You are iterating over a list, and in every iteration you append a new element to the list, so the iteration is never going to end.
To see what's happening, change your code to this:
import time
x = [1,2,3,4]
for num in x:
x.append(num**2)
time.sleep(0.5)
print(x)
If you know what you are doing, you can avoid this kind of "dynamic iteration" with the [:] trick:
x = [1,2,3,4]
for num in x[:]:
x.append(num**2)
This way you are iterating over x[:], not the growing x, in effect it's like you are iterating over a snapshot of x.

More concisely, you could use a list comprehension as follows:
x = [1,2,3,4]
x_squared = [elem**2 for elem in x]
x.extend(x_squared)

What do [] brackets in a for loop in python mean?

I'm parsing JSON objects and found this sample line of code which I kind of understand but would appreciate a more detailed explanation of:
for record in [x for x in records.split("\n") if x.strip() != '']:
I know it is spliting records to get individual records by the new line character however I was wondering why it looks so complicated? is it a case that we can't have something like this:
for record in records.split("\n") if x.strip() != '']:
So what do the brackets do []? and why do we have x twice in x for x in records.split....
Thanks

The "brackets" in your example constructs a new list from an old one, this is called list comprehension.
The basic idea with [f(x) for x in xs if condition] is:
def list_comprehension(xs):
result = []
for x in xs:
if condition:
result.append(f(x))
return result
The f(x) can be any expression, containing x or not.

That's a list comprehension, a neat way of creating lists with certain conditions on the fly.
You can make it a short form of this:
a = []
for record in records.split("\n"):
if record.strip() != '':
a.append(record)
for record in a:
# do something

The square brackets ( [] ) usually signal a list in Python.

Why is my program showing a TypeError?

Every time I run my code I get this error
TypeError: sequence item 0: expected str instance, int found
The error shows up even when I have converted each element in the list to a string. Please help me fix this. The code is
def add_list(list1):
m=0
for x in list1:
m=m+x
return m
def summarize(list2):
list1 = list2
for x in list2:
x = "{}".format(x)
return "The sum of {} is {}.".format("".join(list2), add_list(list1))
summarize([1,2,3])

When you run a for loop over a list of immutable objects like strings or numbers it's actually making what's effectively a copy. It doesn't take the actual element itself out of the list. So that means
for x in list2:
x = "{}".format(x)
Changes nothing, because what happens could be written verbosely like this:
for x in list2:
>>> x = list[0]
x = "{}".format(x)
>>> x = list[1]
x = "{}".format(x)
>>> x = list[2]
x = "{}".format(x)
You constantly change x, but that doesn't do anything to the elements of the list. If you want to loop over the elements of a list you need to do this
for i,x in enumerate(list2):
list[i] = "{}".format(x)
In that format, 'i' will be set to the index of the element that x currently is so you can refer back to the actual position in the list and edit it there, which WILL result in the list itself changing.
Also you can just use str(x) to turn something into a string, it's much cleaner. It also works on any datatype so you can always print something
>>> str([12,1,"abba"])
"[12,1,'abba']"
>>> str(open("I'm a file.txt",'r'))
"<open file 'I'm a file.txt', mode 'r' at 0x0000000002ADC420>"
>>> str(None)
"None"
However I recommend trying some of the other solutions for how to print this, this isn't ideal. I just thought it was valuable for you to understand the error.

I think this is clearer
return "The sum of %s is %s." % (list2, add_list(list1))

How do I write conditional for loops in one line in Python?

For example, how could I condense
In [1]: for x in xrange(1,11):
...: if x%2==0:
...: print x
into one line?
Edit: Thanks guys! That was exactly what I was looking for.
To make this a little more challenging though, is there a way to add elif & else and still have it be on one line?
To use the previous example,
for x in xrange(1,11):
if x%2==0:
print x
else
print "odd"

For your specific example:
for x in xrange(2, 11, 2): print x
More generally, in terms of whether you can nest blocks on one line, the answer is no. Paraphrasing the documentation on compound statements, a "suite" may not contain nested compound statements if it is in the one-line form. A "suite" is the group of statements controlled by a clause (like a conditional block or the body of a loop).

Maybe something like this:
from __future__ import print_function
map(print, [x for x in xrange(1,11) if x % 2 == 0])

This isn't quite the same and isn't "one line", but consider removing the side-effect and using a list filter/comprehension.
evens = [x for x in xrange(1,11) if x % 2 == 0]
print "\n".join(evens)
# or (now a saner "one line", the evens-expr could even be moved in-place)
for x in evens: print x

for x in [y for y in xrange(1,11) if y%2==0]:
print x

For loops (novice)

I recently started learning Python, and the concept of for loops is still a little confusing for me. I understand that it generally follows the format for x in y, where y is just some list.
The for-each loop for (int n: someArray)
becomes for n in someArray,
And the for loop for (i = 0; i < 9; i-=2) can be represented by for i in range(0, 9, -2)
Suppose instead of a constant increment, I wanted i*=2, or even i*=i. Is this possible, or would I have to use a while loop instead?

As you say, a for loop iterates through the elements of a list. The list can contain anything you like, so you can construct a list beforehand that contains each step.
A for loop can also iterate over a "generator", which is a small piece of code instead of an actual list. In Python, range() is actually a generator (in Python 2.x though, range() returned a list while xrange() was the generator).
For example:
def doubler(x):
while True:
yield x
x *= 2
for i in doubler(1):
print i
The above for loop will print
1
2
4
8
and so on, until you press Ctrl+C.

You can use a generator expression to do this efficiently and with little excess code:
for i in (2**x for x in range(10)): #In Python 2.x, use `xrange()`.
...
Generator expressions work just like defining a manual generator (as in Greg Hewgill's answer), with a syntax similar to a list comprehension. They are evaluated lazily - meaning that they don't generate a list at the start of the operation, which can cause much better performance on large iterables.
So this generator works by waiting until it is asked for a value, then asking range(10) for a value, doubling that value, and passing it back to the for loop. It does this repeatedly until the range() generator yields no more values.

Bear in mind that the 'list' part of the Python can be any iterable sequence.
Examples:
A string:
for c in 'abcdefg':
# deal with the string on a character by character basis...
A file:
with open('somefile','r') as f:
for line in f:
# deal with the file line by line
A dictionary:
d={1:'one',2:'two',3:'three'}
for key, value in d.items():
# deal with the key:value pairs from a dict
A slice of a list:
l=range(100)
for e in l[10:20:2]:
# ever other element between 10 and 20 in l
etc etc etc etc
So it really is a lot deeper than 'just some list'
As others have stated, just set the iterable to be what you want it to be for your example questions:
for e in (i*i for i in range(10)):
# the squares of the sequence 0-9...
l=[1,5,10,15]
for i in (i*2 for i in l):
# the list l as a sequence * 2...

You will want to use list comprehensions for this
print [x**2 for x in xrange(10)] # X to the 2nd power.
and
print [x**x for x in xrange(10)] # X to the Xth power.
The list comprehension syntax is a follows:
[EXPRESSION for VARIABLE in ITERABLE if CONDITION]
Under the hood, it acts similar to the map and filter function:
def f(VARIABLE): return EXPRESSION
def c(VARIABLE): return CONDITION
filter(c, map(f, ITERABLE))
Example given:
def square(x): return x**2
print map(square, xrange(10))
and
def hypercube(x): return x**x
print map(hypercube, xrange(10))
Which can be used as alternative approach if you don't like list comprehensions.
You could as well use a for loop, but that would step away from being Python idiomatic...

Just for an alternative, how about generalizing the iterate/increment operation to a lambda function so you can do something like this:
for i in seq(1, 9, lambda x: x*2):
print i
...
1
2
4
8
Where seq is defined below:
#!/bin/python
from timeit import timeit
def seq(a, b, f):
x = a;
while x < b:
yield x
x = f(x)
def testSeq():
l = tuple(seq(1, 100000000, lambda x: x*2))
#print l
def testGen():
l = tuple((2**x for x in range(27)))
#print l
testSeq();
testGen();
print "seq", timeit('testSeq()', 'from __main__ import testSeq', number = 1000000)
print "gen", timeit('testGen()', 'from __main__ import testGen', number = 1000000)
The difference in performance isn't that much:
seq 7.98655080795
gen 6.19856786728
[EDIT]
To support reverse iteration and with a default argument...
def seq(a, b, f = None):
x = a;
if b > a:
if f == None:
f = lambda x: x+1
while x < b:
yield x
x = f(x)
else:
if f == None:
f = lambda x: x-1
while x > b:
yield x
x = f(x)
for i in seq(8, 0, lambda x: x/2):
print i
Note: This behaves differently to range/xrange in which the direction </> test is chosen by the iterator sign, rather than the difference between start and end values.