Suppose I have a list of sets and I want to get the union over all sets in that list. Is there any way to do this using a generator expression? In other words, how can I create the union over all sets in that list directly as a frozenset?
Just use the .union() method.
>>> l = [set([1,2,3]), set([4,5,6]), set([1,4,9])]
>>> frozenset().union(*l)
frozenset([1, 2, 3, 4, 5, 6, 9])
This works for any iterable of iterables.
I assume that what you're trying to avoid is the intermediate creations of frozenset objects as you're building up the union?
Here's one way to do it. NOTE: this originally used itertools.chain() but, as Kenny's comment notes, the version below is slightly better:
import itertools
def mkunion(*args):
return frozenset(itertools.chain.from_iterable(args))
Invoke like this:
a = set(['a','b','c'])
b = set(['a','e','f'])
c = mkunion(a,b) # => frozenset(['a', 'c', 'b', 'e', 'f'])
Nested generator expression. But I think they are a bit cryptic, so the way KennyTM suggested may be clearer.
frozenset(some_item for some_set in some_sets for some_item in some_set)
Related
Why this problem has no trivial solution is because I needs to be solved using only pure functions.
Using only pure functions from Python's functional programming page (https://docs.python.org/3/howto/functional.html#), how can one create a list with a value in it? If we'd like to create a list with a value in it, we'd (in code) just do
x = [1]
I do not consider [] to be to be a part of the functions we're looking at here, since it has no signature and is not callable like any other function.
Using only functions to do this is not so trivial. One thought I had was to create a new list using list() and then append values to it. But list().append is mutable and does not return a new, or the, list with the item in it.
What I really want to do is to turn ["a","b","c"] into [["a"],["b"],["c"]], with above constraints.
Other proposals has been made like creating my own (pure) function doing what I want:
def create_list(value) -> list:
return [value]
and then just do map(create_list, ["a","b","c"]) to get solution.
But this is a custom made function and is not from any of the python package functions (within https://docs.python.org/3/howto/functional.html, as mentioned)
lst=[1,2,3];
#this will print [[1],[2],[3]]
print(list(map(lambda x: [x],lst)));
Single element:
def to_list(elem):
return list(range(elem, elem+1)))
To convert [1,2,3] into [[1], [2], [3]] with list comprehesion (it can be easily changed to map):
return [to_list(el) for el in input_list]
And without (ugly, but works ^^)
import itertools
def make_gen(elem):
yield elem
def to_list(elem):
return list(make_gen(elem))
def helper(elem, l):
return list(itertools.chain(to_list(to_list(elem)), l))
def convert(l):
if not l:
return []
return helper(l[0], convert(l[1:]))
print(convert([1, 2, 3]))
To ensure non-mutability, you probably want to use tuples instead of lists (or be very disciplined with your lists).
Using a list comprehension would be a valid functional approach:
A = [1,2,3]
B = [ [i] for i in A ] # [[1], [2], [3]]
or with tuples:
A = (1,2,3)
B = tuple( (i,) for i in A ) # ((1,), (2,), (3,))
If you must use functions, then map() is probably a good solution to this:
A = [1,2,3]
B = list(map(lambda i:[i],A))
If even [i] is proscribed (but why would it be), you can use a a function to make a list directly from its arguments:
def makeList(*v): return list(*v)
A = makeList(1,2,3)
B = makeList(*map(makeList,A))
# combined
makeList(*map(makeList,makeList(1,2,3)))
BTW functional programming is not about "only using functions", it is more about non-mutability of results (and avoidance of side effects). You may want to question whoever is sending you on this wild goose chase.
Using only pure functions from Python's functional programming page
(https://docs.python.org/3/howto/functional.html#), how can one create
a list with a value in it? If we'd like to create a list with number 1
in it
You might exploit generator as generator are described therein as follows
def justone():
yield 1
lst = list(justone())
print(lst)
output
[1]
justone is function (which might be checked using inspect.isfunction) and is pure (as it does not alter anything outside)
In the documentation you link, there are references to Iterators and Generators, which are powerful constructs present in Python (and other languages). You can consider a function to build a list as follows:
def list_from_args(*args):
return [*args]
This is a (superfluous) wrapper around Iterator functionality. You can leverage the Iterator pattern in Python to accomplish a lot, whether that be creating/consuming objects (e.g. lists, tuples, dictionaries), or for processing data (e.g. reading/writing to a file line-by-line, paginating an API or DB Query, etc.)
The code above does the following, for example:
>>> example = list_from_args(1, 'a', 'ham', 'eggs', 44)
>>> example
[1, 'a', 'ham', 'eggs', 44]
The reason I labeled the above function as superfluous: Oftentimes, if you need to create a list on the fly, you can use list comprehensions.
This does it using only functions from https://docs.python.org/3/library/functional.html
import functools
import itertools
map(
list,
map(
functools.partial(
itertools.repeat,
times=1,
),
[1,2,3]
)
)
functools.partial creates a new function of itertools.repeat with "times" parameter set to 1. Each value in the list is then repeated once and turned into a new list using list function.
>>> [[1], [2], [3]]
I thought about a interesting question and I hope somebody can help me solve this!
I want to use multiprocessing, so I choose to use pool.starmap(myfunction,zip([1,2,3,4,5],['a','b','c','d','e'])) in order to pass multi arguments. I want to combine the entire list [1,2,3,4,5] with the every single element in the second list such as
([1,2,3,4,5],'a'),([1,2,3,4,5],'b').....
instead of only combining the single element in the lists such as
(1,'a'),(2,'b')
I know how to do it in a stupid way which is multiply the list by 5
new_list=[1,2,3,4,5]*5
and then zip the new_list with the second list
I'm now wondering if there is a better way to do this?
After reading your comment I assume you are looking for itertools.repeat:
import itertools
import multiprocessing
def combine(val, char):
return f'{val}{char}'
vals = [1, 2, 3, 4, 5]
chars = ['a', 'b', 'c', 'd', 'e']
pool = multiprocessing.Pool(3)
combs = pool.starmap(combine, zip(itertools.repeat(vals, 5), chars))
print(combs)
This has a smaller memory footprint than the naive approach, which is simply
combs = pool.starmap(combine, zip([vals]*5, chars))
If you instead want to generate all combinations of the valsand the chars elements, you could use itertools.product (which is what I first assumed you wanted):
combs = pool.starmap(combine, itertools.product(vals, chars))
As a sidenote; itertools also contain a starmap function that works more or less the same as the multiprocessing one, except for executing all calls in one process, in order. This, however, can not take advantage of multiple cores.
I have a even length tuple having elements like ('a','b','c','d','e','f') which I want to convert to dictionary having elements like ['a':'b', 'c':'d', 'e':'f'].
I tried using dict(tuple) but that wasn't helping. I have just started learning Python and any help will be highly appreciable.
It looks like you're trying to group the tuple into pairs, and then make a dict out of those pairs. There are two ways to do this.
The first is zipping slices:
zip(t[::2], t[1::2])
This is called an "extended slice", which is of the form start:stop:step. The first one is ::2, so it has the default start (the beginning of the tuple) and stop (the end of the tuple), and a step of 2, so it gets elements 0, 2, and 4. The second one is 1::2, so it's the same, but it starts at 1 instead of the default, so it gets elements 1, 3, and 5.
See the tutorial section on Lists for more details. (Of course you're using a tuple, not a list, but they both slice the same way.)
The second is zipping an iterator with itself:
i = iter(t)
zip(i, i)
Since the two references to i are both the same iterator, whenever you advance one, it advances both. So, the first one gets #0, then the second gets #1, then the first gets #2, the second #3, and so on.
See the tutorial section on Iterators for more details. Also see How grouper works, which explains a more general version of this idea (or at least tries to).
Either way, you get ('a', 'b'), then ('c', 'd'), then ('e', 'f'), so you can just pass that to dict:
dict(zip(t[::2], t[1::2]))
So, which one is better?
Slicing is probably easier to understand. It's also usually faster.
However, slicing doesn't work on arbitrary iterables, just sequences, it wastes memory on big inputs (you're essentially making a complete extra copy of the sequence), and it's a little harder to generalize.
You should learn how both of them work so you can choose appropriately.
You can use a dict comprehension:
t = ('a','b','c','d','e','f')
d = {t[i]:t[i+1] for i in range(0,len(t),2)}
Note that the part
range(0,len(t),2)
will generate a list of the form
[0, 2, 4]
Try this:
t = ('a','b','c','d','e','f')
dict(t[i:i+2] for i in xrange(0, len(t), 2))
=> {'a': 'b', 'c': 'd', 'e': 'f'}
>>> tup = ('a','b','c','d','e','f')
>>> dct = dict(zip(tup[::2], tup[1::2]))
{'a': 'b', 'c': 'd', 'e', 'f'}
This should do the trick
def tup2dict():
tup = ('a','b','c','d','e','f')
print ({i:j for (i,j) in zip(tup,tup[1:])[::2]})
Thanks to iterating-over-every-two-elements-in-a-list and python-dictionary-comprehensionn.
In trying to use a list comprehension to make a list given a conditional, I see the following:
In [1]: mydicts = [{'foo':'val1'},{'foo':''}]
In [2]: mylist = [d for d in mydicts if d['foo']]
In [3]: mylist
Out[3]: [{'foo': 'val1'}]
In [4]: mydicts[1]['foo'] = 'val2'
In [5]: mydicts
Out[5]: [{'foo': 'val1'}, {'foo': 'val2'}]
In [6]: mylist
Out[6]: [{'foo': 'val1'}]
I've been reading the docs to try and understand this but have come up with nothing so far, so I'll ask my question here: why is it that mylist never includes {'foo': 'val2'} even though the reference in the list comprehension points to mydict, which by In [6] contains {'foo': 'val2'}? Is this because Python eagerly evaluates list comprehensions? Or is the lazy/eager dichotomy totally irrelevant to this?
There's no lazy evaluation of lists in Python. List comprehensions simply create a new list. If you want "lazy" evaluation, use a generator expression instead.
my_generator_expression = (d for d in mydicts if d['foo']) # note parentheses
mydicts[1]['foo'] = 'val2'
print(my_generator_expression) # >>> <generator object <genexpr> at 0x00000000>
for d in my_generator_expression:
print(d) # >>> {'foo': 'val1'}
# >>> {'foo': 'val2'}
Note that generators differ from lists in several important ways. Perhaps the most notable is that once you iterate over them, they are exhausted, so they're best to use if you only need the data they contain once.
I think you're a bit confused about what list comprehensions do.
When you do this:
[d for d in mydicts if d['foo']]
That evaluates to a new list. So, when you do this:
mylist = [d for d in mydicts if d['foo']]
You're assigning that list as the value of mylist. You can see this very easily:
assert type(mylist) == list
You're not assigning "a list comprehension" that gets reevaluated every time to mylist. There are no magic values in Python that get reevaluated every time. (You can fake them by, e.g., creating a class with a #property, but that's not really an exception; it's the expression myobj.myprop that's being reevaluated, not myprop itself.)
In fact, mylist = [d for d in mydicts if d['foo']] is basically the same mylist = [1, 2, 3].* In both cases, you're creating a new list, and assigning it to mylist. You wouldn't expect the second one to re-evaluate [1, 2, 3] each time (otherwise, doing mylist[0] = 0 wouldn't do much good, because as soon as you try to view mylist you'd be getting a new, pristine list!). The same is true here.
* In Python 3.x, they aren't just basically the same; they're both just different types of list displays. In 2.x, it's a bit more murky, and they just happen to both evaluate to new list objects.
mylist contains the result of a previous list comprehension evaluation, it won't magically updated just because you update a variable that was used for its computation.
Say I have a working list:
['a','b','c']
and an index list
[2,1,0]
which will change the working list to:
['c','b','a']
Is there any python method to do this easily (the working list may also be a numpy array, and so a more adaptable method is greatly preferred)? Thanks!
ordinary sequence:
L = [L[i] for i in ndx]
numpy.array:
L = L[ndx]
Example:
>>> L = "abc"
>>> [L[i] for i in [2,1,0]]
['c', 'b', 'a']
Converting my comment-answer from when this question was closed:
As you mentioned numpy, here is an answer for that case:
For numerical data, you can do this directly with numpy arrays Details here: http://www.scipy.org/Cookbook/Indexing#head-a8f6b64c733ea004fd95b47191c1ca54e9a579b5
the syntax is then
myarray[myindexlist]
For non-numerical data, the most efficient way in the case of long arrays which you read out only once is most likely this:
(myarray[i] for i in myindex)
note the () for generator expressions instead of [] for list comprehension.
Read this:
Generator Expressions vs. List Comprehension
This is very easy to do with numpy if you don't mind converting to numpy arrays:
>>> import numpy
>>> vals = numpy.array(['a','b','c'])
>>> idx = numpy.array([2,1,0])
>>> vals[idx]
array(['c', 'b', 'a'],
dtype='|S1')
To get back to a list, you can do:
>>> vals[idx].tolist()
['c', 'b', 'a']