From an array of indices how to create Python ranges? - python

With numpy's where() function I'm getting the indices of an array by applying a mask. For example, an index list, where (1, 2, 3, 10, 11, 12, 13) where True:
[1, 2, 3, 10, 11, 12, 13]
Is there a way to transform this (or directly get it from a similar function) in separate ranges, for my example:
[ range(1, 4), range(10, 14) ]


Assuming all the ranges include unit differences only, it's worth to start with a search of indexes where to split:
split_idx = np.flatnonzero(np.diff(ls, prepend = ls[0], append=ls[-1])!=1)
Note that it is improved slightly in order to insert starting and ending indexes as well. A further work could be done like so:
bounding_points = np.transpose([split_idx[:-1], split_idx[1:]])
out = [range(ls[n], ls[m-1]+1) for n, m in bounding_points.tolist()]
Sample run:
>>> ls = [1, 2, 3, 10, 11, 12, 13, 16, 18, 19]
>>> split_idx
array([ 0, 3, 7, 8, 10], dtype=int32)
>>> bounding_points
array([[ 0, 3],
[ 3, 7],
[ 7, 8],
[ 8, 10]], dtype=int32)
>>> out
[range(1, 4), range(10, 14), range(16, 17), range(18, 20)]


reading the link I sent you I got this answer:
from operator import itemgetter
from itertools import groupby
data = [1, 2, 3, 10, 11, 12, 13]
ranges = []
for key, group in groupby(enumerate(data), lambda x: x[0] - x[1]):
group = list(map(itemgetter(1), group))
if len(group) > 1:
ranges.append(range(group[0], group[-1]+1))
else:
ranges.append(group[0])
print(ranges)
the output was
[range(1, 4), range(10, 14)]

Related

Python: summing elements in a sequence in increasing order and vice versa

I have the following sequence: (4, 6, 5, 11)
I want to have the outcome: (4, 10, 15, 26)
Can someone help me?
q = list()
for i in enumerate(seq):
q = [x[i] + x[i-1] for x in seq]
What if I would like to do the reverse? from (4, 10, 15, 26) to (4, 6, 5, 11)

Try this:-
i = [4, 6, 5, 11]
j = [sum(i[:k+1]) for k in range(len(i))]
print(j)

For Python3, use itertools.accumulate:
import itertools
s = (4, 6, 5, 11)
final_list = list(itertools.accumulate(s))
Output:
[4, 10, 15, 26]
For Python2, you can use a generator:
def accumulate(s):
val = 0
for i in s:
val += i
yield val
print(list(accumulate((4, 6, 5, 11))))
Output:
[4, 10, 15, 26]
Inverse:
def inverse(s):
first = s[0]
second = 0
yield first
for i in s[1:]:
yield i-first
first = i
print(list(inverse([4, 10, 15, 26])))
Output:
[4, 6, 5, 11]

Not very efficient but you can do:
x= [4, 6, 5, 11]
y = [sum(x[0:i+1] for i in range(len(x))]

Adding and creating chunks of a list in python

This is my list
list1 = [1,2,3,4,5,6,7,8,9,10,11,12]
i tried to do this
it = iter(list1)
chunked_lists = zip(it,it,it)
chunked_lists is now = [(1,2,3),(4,5,6),(7,8,9),(10,11,12)]
Now i'd like to add a 'data' to every chunk so that it would look like
[(1,2,3,'data'),(4,5,6,'data')....etc]
but tuples are immutable and i'd have to destroy them, append and create them again.
is there a better way to accomplish this in python?

it = iter(list1)
from itertools import repeat
chunked_lists = zip(it,it,it,repeat("data"))
[(1, 2, 3, 'data'), (4, 5, 6, 'data'), (7, 8, 9, 'data'), (10, 11, 12, 'data')]

You can use itertools.repeat to create a new iterator contain the data and then apply the zip function on your prior iterators and new repeated iterator :
>>> it = iter(list1)
>>> it2=repeat('data',len(list1))
>>> chunked_lists = zip(it,it,it,it2)
>>> chunked_lists
[(1, 2, 3, 'data'), (4, 5, 6, 'data'), (7, 8, 9, 'data'), (10, 11, 12, 'data')]
>>>
Note that as #Padraic Cunningham mentioned on his answer there is no need to call the len function in itertools.repeat.So you can simply use repeat('data')
If you want to modify the list of tuples after creation you can use a list comprehension to modify your tuples by adding the new elements to the tuples.
Example :
>>> import numpy as np
>>> l2=np.random.randint(0,100,len(list1)/3)
>>> it = iter(list1)
>>> chunked_lists = zip(it,it,it)
>>> chunked_lists = [i+(j,) for i,j in zip(chunked_lists,l2)]
>>> l2
array([35, 22, 35, 95])
>>> chunked_lists
[(1, 2, 3, 35), (4, 5, 6, 22), (7, 8, 9, 35), (10, 11, 12, 95)]
>>>

This should be a linear time way to do it:
list1 = [1,2,3,4,5,6,7,8,9,10,11,12]
result = [tuple(list1[x:x+3]+["data"]) for x in range(0, len(list1), 3)]
If I am not ignoring some Python intricacy, since tuple() has complexity O(n), the slice accessor has complexity O(k) and the overhead caused by the +["data"] isn't greater than simply appending a node at the end of a list (which is O(1)), it should be O((len(list1) / k) * k ^ 2) aka O(n k), where k is fixed to 3 in your case.
result = [tuple(list1[x:x+3]+["data"]) for x in range(0, len(list1), 3)]
means:
[list1[x:x+3]+["data"] for x in (0, 3, ... 12)]
aka:
[
tuple(list1[0:3]+["data"]),
tuple(list1[4:6]+["data"]),
...
tuple(list1[9:12]+["data"])
]
It is also well behaved with odd lists:
>>> list1 = [1,2,3,4,5,6,7,8,9,10,11]
>>> print [tuple(list1[x:x+3]+["data"]) for x in range(0, len(list1), 3)]
[(1, 2, 3, 'data'), (4, 5, 6, 'data'), (7, 8, 9, 'data'), (10, 11, 'data')]

Since tuples are immutable, if you're adding and deleting from them often, you're perhaps better off using lists of lists:
list1 = [1,2,3,4,5,6,7,8,9,10,11,12]
it = iter(list1)
chunked_lists = [list(a) for a in zip(it,it,it)]
# => [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]
for l in chunked_lists:
l.append('data')
# => [[1, 2, 3, 'data'], [4, 5, 6, 'data'], [7, 8, 9, 'data'], [10, 11, 12, 'data']]

How about adding them to the zip function when you are creating the chunked_lists itself -
>>> list1 = [1,2,3,4,5,6,7,8,9,10,11,12]
>>> it = iter(list1)
>>> chunked_lists = zip(it,it,it,['data']*(len(list1)/3))
>>> chunked_lists
[(1, 2, 3, 'data'), (4, 5, 6, 'data'), (7, 8, 9, 'data'), (10, 11, 12, 'data')]
For Python 3.x , you would need to divide the len(list1) using // operator (for it to return int) . (And most probably wrap zip in list() , since zip in Python 3.x , returns an iterator.)

How to skip over current element in a python list

How I can loop over a python list, add each item to another list except the current item:
list = [1,2,8,20,11]
for i,j in enumerate(list):
print j[i+1:]
#this will only give [2,8,20,11] then [8,20,11],[20,11], [11]
#but I want something like [2,8,20,11], [1,8,20,11],[1,2,20,11]... etc.
#then 1,8,20,11
#then

Looks like you're after combinations, eg:
>>> from itertools import combinations
>>> data = [1,2,8,20,11]
>>> list(combinations(data, 4))
[(1, 2, 8, 20), (1, 2, 8, 11), (1, 2, 20, 11), (1, 8, 20, 11), (2, 8, 20, 11)]

you may use list slicing as:
lst = [1,2,8,20,11]
for i in xrange(len(lst)):
print lst[:i]+lst[i+1:]
>>> [2, 8, 20, 11]
[1, 8, 20, 11]
[1, 2, 20, 11]
[1, 2, 8, 11]
[1, 2, 8, 20]

zipping a python dict of lists

I have a python dictionary of type defaultdict(list)
This dictionary is something like this:
a = {1:[1,2,3,4],2:[5,6,7,8]....n:[some 4 elements]}
So basically it has n keys which has a list as values and all the list are of same lenght.
Now, i want to build a list which has something like this.
[[1,5,...first element of all the list], [2,6.. second element of all the list]... and so on]
Soo basically how do i get the kth value from all the keys.. Is there a pythonic way to do this.. ??
THanks

>>> a = {1:[1,2,3,4],2:[5,6,7,8], 3:[9, 10, 11, 12]}
>>>
>>> zip(*(a[k] for k in sorted(a)))
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
(Okay, this produces tuples, not lists, but hopefully that's not a problem.)
Update: I like the above more than this, but the following is a few keystrokes shorter:
>>> zip(*map(a.get, sorted(a)))
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]

How about this solution:
zip(*a.values())
For e.g.
>>> a = {1:[1,2,3,4],2:[5,6,7,8], 3:[9, 10, 11, 12]}
>>> zip(*a.values())
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
Update: to preserve order use DSM's answer.

An alternative way to do this using numpy:
>>> import numpy
>>> a = {1:[1,2,3,4],2:[5,6,7,8], 3:[9, 10, 11, 12]}
>>> x = numpy.zeros((len(a),4), dtype=int)
>>> x[[i-1 for i in a.keys()]] = a.values()
>>> x.T
array([[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11],
[ 4, 8, 12]])

List comprehension makes this easy. To get a list of the kth items:
k = 1
[a[key][k] for key in sorted(a.keys())]
The to build a list of lists:
[ [a[key][k] for key in sorted(a.keys())] for k in range(len(a[1]))]

if you want list of lists (assuming each list is of length=4):
>>> a = {1:[1, 2, 3, 4], 2:[5, 6, 7, 8], 3:[9, 10, 11, 12]}
>>> [[a[key][x] for key in sorted(a.keys())] for x in xrange(4)]
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]

Given a bunch of ranges of numbers, get all the numbers within those ranges?

In Python, you can get the numbers in a range by calling range(x,y). But given two ranges, say 5-15, and 10-20 how can you get all the numbers 5-20 without duplicates? The ranges may also be disjoint.
I could concat all the results and then uniquify the list, but is that the fastest solution?

>>> a = range(5, 15)
>>> b = range(10, 20)
>>> print sorted(set(a + b))
[5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Or if you want a more general expansion of the lists to their elements for inclusion in the set:
>>> list_of_lists = [a, b]
>>> print sorted(set(elem for l in list_of_lists for elem in l))
[5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
And I found a way to compose it all in one line:
>>> list_of_lists = [a, b]
>>> print set.union(*map(set, list_of_lists))
Is sorting necessary? It's just expository, but then I don't see that sets necessarily output in sorted order:
>>> x = set(range(3))
>>> x
set([0, 1, 2])
>>> x.add(-1)
>>> x
set([0, 1, 2, -1])

Or you can join overlapping ranges:
>>> def join_overlapping_ranges(ranges):
... list_of_ranges = []
... # ranges are sorted on first element
... for r in sorted(ranges):
... # ranges are stored as [start, end]
... if list_of_ranges and list_of_ranges[-1][1] >= r[0]:
... list_of_ranges[-1][1] = r[1]
... else:
... list_of_ranges.append(r)
... return list_of_ranges
...
>>> ranges = [[3,4], [5,7], [1,2], [4,6], [5,5]]
>>> print sorted(ranges)
[[1, 2], [3, 4], [4, 6], [5, 5], [5, 7]]
>>> print join_overlapping_ranges(ranges)
[[1, 2], [3, 7]]

Sort the ranges (x, y) by increasing x values. Now, for each range, if it overlaps the previous range, set your current "big range"'s y value to the current range's y value. If it doesn't, start a new "big range": this one won't overlap any of the previous ones. If the current range is completely included in the current big one, ignore it.

For the quantity you need I would just keep it simple
>>> a = range(5, 15)
>>> b = range(10, 20)
>>> from itertools import chain
>>> sorted(set(chain.from_iterable(list_of_lists)))
[5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

Categories

Resources