Develop Reference

Create a list with a value in it using only pure functions - python

Why this problem has no trivial solution is because I needs to be solved using only pure functions.
Using only pure functions from Python's functional programming page (https://docs.python.org/3/howto/functional.html#), how can one create a list with a value in it? If we'd like to create a list with a value in it, we'd (in code) just do
x = [1]
I do not consider [] to be to be a part of the functions we're looking at here, since it has no signature and is not callable like any other function.
Using only functions to do this is not so trivial. One thought I had was to create a new list using list() and then append values to it. But list().append is mutable and does not return a new, or the, list with the item in it.
What I really want to do is to turn ["a","b","c"] into [["a"],["b"],["c"]], with above constraints.
Other proposals has been made like creating my own (pure) function doing what I want:
def create_list(value) -> list:
return [value]
and then just do map(create_list, ["a","b","c"]) to get solution.
But this is a custom made function and is not from any of the python package functions (within https://docs.python.org/3/howto/functional.html, as mentioned)

lst=[1,2,3];
#this will print [[1],[2],[3]]
print(list(map(lambda x: [x],lst)));

Single element:
def to_list(elem):
return list(range(elem, elem+1)))
To convert [1,2,3] into [[1], [2], [3]] with list comprehesion (it can be easily changed to map):
return [to_list(el) for el in input_list]
And without (ugly, but works ^^)
import itertools
def make_gen(elem):
yield elem
def to_list(elem):
return list(make_gen(elem))
def helper(elem, l):
return list(itertools.chain(to_list(to_list(elem)), l))
def convert(l):
if not l:
return []
return helper(l[0], convert(l[1:]))
print(convert([1, 2, 3]))

To ensure non-mutability, you probably want to use tuples instead of lists (or be very disciplined with your lists).
Using a list comprehension would be a valid functional approach:
A = [1,2,3]
B = [ [i] for i in A ] # [[1], [2], [3]]
or with tuples:
A = (1,2,3)
B = tuple( (i,) for i in A ) # ((1,), (2,), (3,))
If you must use functions, then map() is probably a good solution to this:
A = [1,2,3]
B = list(map(lambda i:[i],A))
If even [i] is proscribed (but why would it be), you can use a a function to make a list directly from its arguments:
def makeList(*v): return list(*v)
A = makeList(1,2,3)
B = makeList(*map(makeList,A))
# combined
makeList(*map(makeList,makeList(1,2,3)))
BTW functional programming is not about "only using functions", it is more about non-mutability of results (and avoidance of side effects). You may want to question whoever is sending you on this wild goose chase.

Using only pure functions from Python's functional programming page
(https://docs.python.org/3/howto/functional.html#), how can one create
a list with a value in it? If we'd like to create a list with number 1
in it
You might exploit generator as generator are described therein as follows
def justone():
yield 1
lst = list(justone())
print(lst)
output
[1]
justone is function (which might be checked using inspect.isfunction) and is pure (as it does not alter anything outside)

In the documentation you link, there are references to Iterators and Generators, which are powerful constructs present in Python (and other languages). You can consider a function to build a list as follows:
def list_from_args(*args):
return [*args]
This is a (superfluous) wrapper around Iterator functionality. You can leverage the Iterator pattern in Python to accomplish a lot, whether that be creating/consuming objects (e.g. lists, tuples, dictionaries), or for processing data (e.g. reading/writing to a file line-by-line, paginating an API or DB Query, etc.)
The code above does the following, for example:
>>> example = list_from_args(1, 'a', 'ham', 'eggs', 44)
>>> example
[1, 'a', 'ham', 'eggs', 44]
The reason I labeled the above function as superfluous: Oftentimes, if you need to create a list on the fly, you can use list comprehensions.

This does it using only functions from https://docs.python.org/3/library/functional.html
import functools
import itertools
map(
list,
map(
functools.partial(
itertools.repeat,
times=1,
),
[1,2,3]
)
)
functools.partial creates a new function of itertools.repeat with "times" parameter set to 1. Each value in the list is then repeated once and turned into a new list using list function.
>>> [[1], [2], [3]]

A list of a lists into a single string

Suppose I have a list of lists say
A = [[1,1,1,1],[2,2,2,2]]
and I want to create two strings from that to be
'1111'
'2222'
How would we do this in python?

Maybe list comprehension:
>>> A = [[1,1,1,1],[2,2,2,2]]
>>> l=[''.join(map(str,i)) for i in A]
>>> l
['1111', '2222']
>>>
Now you've got it.

This is pretty easily done using join and a list comprehension.
A = [[1,1,1,1],[2,2,2,2]]
a_strings = [''.join(map(str, sub_list)) for sublist in A]
See, join() takes a list of strings and makes a string concatenating all the substrings and the list comprehension I used just loops through them all. Above I combined the 2 together.
On a second thought
map() is actually deemed more efficient (when not using lambda.. etc) and for SOME more readable. I'll just add an approach using map instead of a comprehension.
a_strings = map(''.join(), map(str, A))
This first takes the inner map and makes all the ints > strs then joins all the strs together for every sub-list.
Hopefully this makes things a bit more chewable for ya, each method is close to equivalent such that for this case you could consider them style choices.

How does the list comprehension to flatten a python list work? [duplicate]

This question already has answers here:
How can I use list comprehensions to process a nested list?
(13 answers)
Closed 7 months ago.
I recently looked for a way to flatten a nested python list, like this: [[1,2,3],[4,5,6]], into this: [1,2,3,4,5,6].
Stackoverflow was helpful as ever and I found a post with this ingenious list comprehension:
l = [[1,2,3],[4,5,6]]
flattened_l = [item for sublist in l for item in sublist]
I thought I understood how list comprehensions work, but apparently I haven't got the faintest idea. What puzzles me most is that besides the comprehension above, this also runs (although it doesn't give the same result):
exactly_the_same_as_l = [item for item in sublist for sublist in l]
Can someone explain how python interprets these things? Based on the second comprension, I would expect that python interprets it back to front, but apparently that is not always the case. If it were, the first comprehension should throw an error, because 'sublist' does not exist. My mind is completely warped, help!

Let's take a look at your list comprehension then, but first let's start with list comprehension at it's easiest.
l = [1,2,3,4,5]
print [x for x in l] # prints [1, 2, 3, 4, 5]
You can look at this the same as a for loop structured like so:
for x in l:
print x
Now let's look at another one:
l = [1,2,3,4,5]
a = [x for x in l if x % 2 == 0]
print a # prints [2,4]
That is the exact same as this:
a = []
l = [1,2,3,4,5]
for x in l:
if x % 2 == 0:
a.append(x)
print a # prints [2,4]
Now let's take a look at the examples you provided.
l = [[1,2,3],[4,5,6]]
flattened_l = [item for sublist in l for item in sublist]
print flattened_l # prints [1,2,3,4,5,6]
For list comprehension start at the farthest to the left for loop and work your way in. The variable, item, in this case, is what will be added. It will produce this equivalent:
l = [[1,2,3],[4,5,6]]
flattened_l = []
for sublist in l:
for item in sublist:
flattened_l.append(item)
Now for the last one
exactly_the_same_as_l = [item for item in sublist for sublist in l]
Using the same knowledge we can create a for loop and see how it would behave:
for item in sublist:
for sublist in l:
exactly_the_same_as_l.append(item)
Now the only reason the above one works is because when flattened_l was created, it also created sublist. It is a scoping reason to why that did not throw an error. If you ran that without defining the flattened_l first, you would get a NameError

The for loops are evaluated from left to right. Any list comprehension can be re-written as a for loop, as follows:
l = [[1,2,3],[4,5,6]]
flattened_l = []
for sublist in l:
for item in sublist:
flattened_l.append(item)
The above is the correct code for flattening a list, whether you choose to write it concisely as a list comprehension, or in this extended version.
The second list comprehension you wrote will raise a NameError, as 'sublist' has not yet been defined. You can see this by writing the list comprehension as a for loop:
l = [[1,2,3],[4,5,6]]
flattened_l = []
for item in sublist:
for sublist in l:
flattened_l.append(item)
The only reason you didn't see the error when you ran your code was because you had previously defined sublist when implementing your first list comprehension.
For more information, you may want to check out Guido's tutorial on list comprehensions.

For the lazy dev that wants a quick answer:
>>> a = [[1,2], [3,4]]
>>> [i for g in a for i in g]
[1, 2, 3, 4]

While this approach definitely works for flattening lists, I wouldn't recommend it unless your sublists are known to be very small (1 or 2 elements each).
I've done a bit of profiling with timeit and found that this takes roughly 2-3 times longer than using a single loop and calling extend…
def flatten(l):
flattened = []
for sublist in l:
flattened.extend(sublist)
return flattened
While it's not as pretty, the speedup is significant. I suppose this works so well because extend can more efficiently copy the whole sublist at once instead of copying each element, one at a time. I would recommend using extend if you know your sublists are medium-to-large in size. The larger the sublist, the bigger the speedup.
One final caveat: obviously, this only holds true if you need to eagerly form this flattened list. Perhaps you'll be sorting it later, for example. If you're ultimately going to just loop through the list as-is, this will not be any better than using the nested loops approach outlined by others. But for that use case, you want to return a generator instead of a list for the added benefit of laziness…
def flatten(l):
return (item for sublist in l for item in sublist) # note the parens

Note, of course, that the sort of comprehension will only "flatten" a list of lists (or list of other iterables). Also if you pass it a list of strings you'll "flatten" it into a list of characters.
To generalize this in a meaningful way you first want to be able to cleanly distinguish between strings (or bytearrays) and other types of sequences (or other Iterables). So let's start with a simple function:
import collections
def non_str_seq(p):
'''p is putatively a sequence and not a string nor bytearray'''
return isinstance(p, collections.Iterable) and not (isinstance(p, str) or isinstance(p, bytearray))
Using that we can then build a recursive function to flatten any
def flatten(s):
'''Recursively flatten any sequence of objects
'''
results = list()
if non_str_seq(s):
for each in s:
results.extend(flatten(each))
else:
results.append(s)
return results
There are probably more elegant ways to do this. But this works for all the Python built-in types that I know of. Simple objects (numbers, strings, instances of None, True, False are all returned wrapped in list. Dictionaries are returned as lists of keys (in hash order).

One-line & multi-line loops and vectorization in Python

Can one-line loops in Python only be used to build lists? (i.e. list comprehensions), or can they use for more general computing?
For example, I am aware that list comprehensions (~single-line loop) in Python, e.g.
my_list = [ 2*i for i in range(10)]
can also be built with a multi-line loop:
my_list = []
for i in range(10):
my_list.append(2*i)
But can we always transform general multi-line loops into one-line loops?
For example, say we have the following multi-line for loop:
my_array = np.ones(10*10)
for x in range(10):
my_array[x,:] = 0
can we convert it into a single-line loop? More generally:
Q1. Are the two forms functionally equivalent? (i.e. they support the same set of manipulations/operations)
Q2. I think I have read before that one-line loops in Python are vectorized. Is this true? And does this mean that they can iterate faster than multi-line loops?

But can we we always transform general multi-line loops in Python into one-line loops?
The short answer is no.
List comprehensions are good for projections (mapping) and/or filtering.
For example, if you have code like this:
result = []
for x in seq:
if bar(x):
result.append(foo(x))
Then, as you point out, it can benefit from being rewritten as a list comprehension:
result = [foo(x) for f in seq if bar(x)]
However list comprehensions are generally not so good for operations that don't fit into this projection or filtering pattern.
For example if you need to mutate the elements but don't need the result then a list comprehension wouldn't be suitable. The following code would be inconvenient to write as a list comprehension (assuming that both methods return None):
for x in seq:
x.foo() # Modify x
x.bar() # Modify x again
Some operations are not allowed inside a comprehension. An example is breaking out of the loop early if a condition is met:
for x in seq:
if x.foo():
break
else:
x.bar()

One thing I'll point out is that it's not just lists, you can use comprehension to create sets and even dictionaries.
>>> {i**2 for i in range(5)}
set([0, 1, 4, 16, 9])
>>> {i : str(i) for i in range(5)}
{0: '0', 1: '1', 2: '2', 3: '3', 4: '4'}
Also, list comprehension is generally faster than using append numerous times (like your example) because the comprehension is done by underlying C code, as opposed to append, which has the extra Python-layer.
Of course, comprehension has limitations like anything else. If you wanted to perform a larger set of operations on each element of a list/set, then a normal loop might be more appropriate.

append/extend list in loop

I would like to extend a list while looping over it:
for idx in xrange(len(a_list)):
item = a_list[idx]
a_list.extend(fun(item))
(fun is a function that returns a list.)
Question:
Is this already the best way to do it, or is something nicer and more compact possible?
Remarks:
from matplotlib.cbook import flatten
a_list.extend(flatten(fun(item) for item in a_list))
should work but I do not want my code to depend on matplotlib.
for item in a_list:
a_list.extend(fun(item))
would be nice enough for my taste but seems to cause an infinite loop.
Context:
I have have a large number of nodes (in a dict) and some of them are special because they are on the boundary.
'a_list' contains the keys of these special/boundary nodes. Sometimes nodes are added and then every new node that is on the boundary needs to be added to 'a_list'. The new boundary nodes can be determined by the old boundary nodes (expresses here by 'fun') and every boundary node can add several new nodes.

Have you tried list comprehensions? This would work by creating a separate list in memory, then assigning it to your original list once the comprehension is complete. Basically its the same as your second example, but instead of importing a flattening function, it flattens it through stacked list comprehensions. [edit Matthias: changed + to +=]
a_list += [x for lst in [fun(item) for item in a_list] for x in lst]
EDIT: To explain what going on.
So the first thing that will happen is this part in the middle of the above code:
[fun(item) for item in a_list]
This will apply fun to every item in a_list and add it to a new list. Problem is, because fun(item) returns a list, now we have a list of lists. So we run a second (stacked) list comprehension to loop through all the lists in our new list that we just created in the original comprehension:
for lst in [fun(item) for item in a_list]
This will allow us to loop through all the lists in order. So then:
[x for lst in [fun(item) for item in a_list] for x in lst]
This means take every x (that is, every item) in every lst (all the lists we created in our original comprehension) and add it to a new list.
Hope this is clearer. If not, I'm always willing to elaborate further.

Using itertools, it can be written as:
import itertools
a_list += itertools.chain(* itertools.imap(fun, a_list))
or, if you're aiming for code golf:
a_list += sum(map(fun, a_list), [])
Alternatively, just write it out:
new_elements = map(fun, a_list) # itertools.imap in Python 2.x
for ne in new_elements:
a_list.extend(ne)

As you want to extend the list, but loop only over the original list, you can loop over a copy instead of the original:
for item in a_list[:]:
a_list.extend(fun(item))

Using generator
original_list = [1, 2]
original_list.extend((x for x in original_list[:]))
# [1, 2, 1, 2]