Develop Reference

Categorization the list of tuples in Python

help me please I'm trying to find out the fastest and logical way to categorize tuple list by values of the first tuple element.
for example I have a list with tuples like
a = [(378, 123), (100, 12), (112, 23), (145, 14), (165, 34), (178, 45), (227, 32), (234, 12), (356, 15)] # and more and more
How I can dynamically categorize it into a groups like
100to150 = [(100, 12), (112, 23), (145, 14)]
150to200 = [(165, 34), (178, 45)]
200to250 = [(227, 32), (234, 12)]
350to400 = [(378, 123), (356, 15)]
In this way I used step 50, but I want to have an ability to change it of course. It doesn't matter what will be in output, maybe list in list for example
[[(100, 112), (124, 145)], [(165, 12), (178, 12)], [(234, 14)], [(356, 65)]] (random data) or maybe a list with a tuple, it doesn't matter. I just want to have an ability to get the length of the category and print category out. Thank you much.

You can try something like this. This will give of course give you back a categorized dictionary though, not separate variables.
a = [(378, 123), (100, 12), (112, 23), (145, 14), (165, 34), (178, 45), (227, 32), (234, 12), (356, 15)] # and more and more
def categorize(array, step=50):
d = dict()
for e in array:
from_n = e[0]//step*step
s = f'{from_n}to{from_n+step}'
if s not in d:
d[s] = []
d[s].append(e)
return d
print(categorize(a))
Output:
{'350to400': [(378, 123), (356, 15)], '100to150': [(100, 12), (112, 23), (145, 14)], '150to200': [(165, 34), (178, 45)], '200to250': [(227, 32), (234, 12)]}

l = [x for x in a if 100<x[0]<150]
I should say this is the minimal you should need to get going. If you want the full solution, you could imagine putting this into some type of function where your low and high (100, 150 in this example) are arguments. You could even have a list of highs/lows and then loop through them all and collect all the out put as a list of lists of tuples.

You can see something like this:
Using a dictionary to store grouped values, to instantly get them later.
def categorize_by_first(pairs, step=50):
d = {}
for pair in pairs:
range_start = (pair[0] // step) * step
dict_key_name = f"{range_start}_{range_start + step}"
if not d.get(dict_key_name):
d[dict_key_name] = []
d[dict_key_name].append(pair)
return d
Output:
{'350_400': [(378, 123), (356, 15)],
'100_150': [(100, 12), (112, 23), (145, 14)],
'150_200': [(165, 34), (178, 45)],
'200_250': [(227, 32), (234, 12)]}
Time complexity of grouping is O(n) (we only once iterate over the input list).
Time complexity of getting element from a dictionary is O(1)
So that should be efficient.

Returning a value from a python for loop function

I have edited my question. The code contains an is_prime formula which indicates whether a number is prime> I'm trying to extract all the prime values in the range 3 to 65
a = []
b = []
c = []
d = []
def ll_prime(n_start, n_end):
for number in range(n_start, n_end):
if is_prime(number) ==True:
a.append(number)
b.append(1)
else:
c.append(number)
d.append(0)
return (list(zip(a,b)))
The above code runs fine but when I call the function ll_prime(3,65) it gives me the following error:
TypeError Traceback (most recent call last)
<ipython-input-498-1a1a58988fa7> in <module>()
----> 1 ll_prime(3,65)
2 #type(tyl)
3 #list_values = [ v for v in tyl.values()]
<ipython-input-497-d99272d4b655> in ll_prime(n_start, n_end)
11 c.append(number)
12 d.append(0)
---> 13 return (list(zip(a,b)))
TypeError: 'list' object is not callable
Can anyone guide me as why I'm getting this error? I have searched previous question on stackoverflow but none were helpful in my case.
I want result as : [(3,1),(5,1),(7,1)] etc

You could use a list comprehension:
def l1_prime():
return [(i, 1) for i in mb]

You can use repeat from itertools and zip this with your list.
>>> from itertools import repeat
>>> mb = [3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61]
>>> zip(mb, repeat(1))
[(3, 1), (5, 1), (7, 1), (11, 1), (13, 1), (17, 1), (19, 1), (23, 1), (29, 1), (31, 1), (37, 1), (41, 1), (43, 1), (47, 1), (53, 1), (59, 1), (61, 1)]
Or you can use a list comprehension like this:
>>> [(x, 1) for x in mb]
[(3, 1), (5, 1), (7, 1), (11, 1), (13, 1), (17, 1), (19, 1), (23, 1), (29, 1), (31, 1), (37, 1), (41, 1), (43, 1), (47, 1), (53, 1), (59, 1), (61, 1)]
To your solution: In your solution you return your result after the first loop iteration. So it doesn't have the right values yet. Try to move the return outside of your loop.

One issue is that your return command is inside your for loop, so it will only execute once. That could be why you aren't getting what you want. When I ran your code, it returned (3,1), which was only the first set of items. That makes sense if it is only running through the for loop once and then returning. Try this:
mb = [3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61]
list1 = []
list2 = []
def prime():
for i in mb:
list1.append(i)
list2.append(1)
print(str(len(list1)))
print(str(len(list2)))
return (list(zip(list1,list2)))
When I run that, I get the correct answer

Removing overlapping tuple values using Python

I have a list of tuples (let's name it yz_list) that contains N tuples, which have the start and end values like: (start, end), represented by the example below:
yz_list = [(0, 6), (1, 7), (2, 8), (3, 9), (4, 10), (5, 11), (6, 12), (18, 24)]
And I would like to remove all values which are overlapped by the interval of a previous saved tuple. The output that represents this case on the sequences showed above is:
result = [(0,6), (6,12), (18,24)]
How could I achieve this result using Python?
Edit #1
The below code is the code that I'm generating this tuples:
for i, a in enumerate(seq):
if seq[i:i+multiplier] == "x"*multiplier:
to_replace.append((i, i+multiplier))
for i, j in enumerate(to_replace):
print(i,j)
if i == 0:
def_to_replace.append(j)
else:
ind = def_to_replace[i-1]
print(j[0]+1, "\n", ind)
if j[0]+1 not in range(ind[0], ind[1]):
def_to_replace.append(j)
# print(i, j)
print(def_to_replace)
for item in def_to_replace:
frag = replacer(frame_calc(seq[:item[0]]), rep0, rep1, rep2)
for k, v in enumerate(seq_dup[item[0]:item[1]]):
seq_dup[int(item[0]) + int(k)] = list(frag)[k]
return "".join(seq_dup)
As I'm developing with TDD, I'm making a step-by-step progress on the development and now I'm thinking on how to implement the removal of overlaping tuples. I don't really know if it's a good idea to use them as sets, and see the overlapping items.
The pseudocode for generating the result list is:
for item in yz_list:
if is not yz_list first item:
gets item first value
see if the value is betwen any of the values from tuples added on the result list

This may work. No fancy stuff, just manually process each tuple to see if either value is within the range of the saved tuple's set bounds:
yz_list = [(0, 6), (1, 7), (2, 8), (3, 9), (4, 10), (5, 11), (6, 12), (18, 24)]
result = [yz_list[0]]
bounds = yz_list[0][0], yz_list[0][1]
for tup in yz_list[1:]:
if tup[0] in range(bounds[0], bounds[1]) or tup[1] in range(bounds[0], bounds[1]):
pass
else:
result.append(tup)
print result # [(0, 6), (6, 12), (18, 24)]

Here is a class that calculates the overlaps using efficient binary search, and code showing its use to solve your problem. Run with python3.
import bisect
import sys
class Overlap():
def __init__(self):
self._intervals = []
def intervals(self):
return self._intervals
def put(self, interval):
istart, iend = interval
# Ignoring intervals that start after the window.
i = bisect.bisect_right(self._intervals, (iend, sys.maxsize))
# Look at remaining intervals to find overlap.
for start, end in self._intervals[:i]:
if end > istart:
return False
bisect.insort(self._intervals, interval)
return True
yz_list = [(0, 6), (1, 7), (2, 8), (3, 9), (4, 10), (5, 11), (6, 12), (18, 24)]
ov = Overlap()
for i in yz_list:
ov.put(i)
print('Original:', yz_list)
print('Result:', ov.intervals())
OUTPUT:
Original: [(0, 6), (1, 7), (2, 8), (3, 9), (4, 10), (5, 11), (6, 12), (18, 24)]
Result: [(0, 6), (6, 12), (18, 24)]

yz_list = [(0, 6), (1, 7), (2, 8), (3, 9), (4, 10), (5, 11), (6, 12), (18, 24)]
result = []
for start, stop in yz_list:
for low, high in result:
if (low < start < high) or (low < stop < high):
break
else:
result.append((start, stop))
This gives the desired output, and it's pretty easy to see how it works. The else clause basically just means "run this if we didn't break out of the loop".

how to construct a dictionary from join? [duplicate]

This question already has answers here:
Creating a Dictionary from a List of 2-Tuples
(2 answers)
Closed 5 years ago.
I am trying to build a dictionary for the below tuple list:
lst=[('ldb', 25), ('baseB', 4), ('code', 112),
('cache-6', 55), ('Xauthority', 1), ('baseA', 4),
('npmrc', 1), ('apmrc', 1),('gz', 190),
('dbf', 1), ('lst', 2), ('markdown', 10),
('sqlite-shm', 2), ('vsixmanifest', 4), ('ttf', 109),
('pkl', 35), ('gitignore', 8), ('xml', 46)]
By using join like this:
op= {','.join( '\'%s\':%d'%i for i in lst)}
But the output op will be of type set as below!!
set(["'ldb':25,'baseB':4,'code':112,'cache-6':55, 'Xauthority':1,'baseA':4,'npmrc':1,'apmrc':1,
'gz':190,'dbf':1,'lst':2,'markdown':10,'sqlite-shm':2,'vsixmanifest':4,'ttf':109,'pkl':35,'gitignore':8,'xml':46"])
Some one correct me in getting dictionary instead of set
Thanks in advance.

Currently, you are creating a set, not a dictionary. Try this:
lst=[('ldb', 25), ('baseB', 4), ('code', 112), ('cache-6', 55), ('Xauthority', 1), ('baseA', 4), ('npmrc', 1), ('apmrc', 1), ('gz', 190), ('dbf', 1), ('lst', 2), ('markdown', 10), ('sqlite-shm', 2), ('vsixmanifest', 4), ('ttf', 109), ('pkl', 35), ('gitignore', 8), ('xml', 46)]
new_data = {a:b for a, b in lst}
Or, better yet:
new_data = dict(lst)

Try it:
lst = [('ldb', 25), ('baseB', 4), ('code', 112), ('cache-6', 55), ('Xauthority', 1), ('baseA', 4), ('npmrc', 1),
('apmrc', 1), ('gz', 190), ('dbf', 1), ('lst', 2), ('markdown', 10), ('sqlite-shm', 2), ('vsixmanifest', 4),
('ttf', 109), ('pkl', 35), ('gitignore', 8), ('xml', 46)]
d = dict()
for i in lst:
d[i[0]] = i[1]
print(d)

More memory efficient way of making a dictionary?

VERY sorry for the vagueness, but I don't actually know what part of what I'm doing is inefficient.
I've made a program that takes a list of positive integers (example*):
[1, 1, 3, 5, 16, 2, 4, 6, 6, 8, 9, 24, 200,]
*the real lists can be up to 2000 in length and the elements between 0 and 100,000 exclusive
And creates a dictionary where each number tupled with its index (like so: (number, index)) is a key and the value for each key is a list of every number (and that number's index) in the input that it goes evenly into.
So the entry for the 3 would be: (3, 2): [(16, 4), (6, 7), (6, 8), (9, 10), (24, 11)]
My code is this:
num_dict = {}
sorted_list = sorted(beginning_list)
for a2, a in enumerate(sorted_list):
num_dict[(a, a2)] = []
for x2, x in enumerate(sorted_list):
for y2, y in enumerate(sorted_list[x2 + 1:]):
if y % x == 0:
pair = (y, y2 + x2 + 1)
num_dict[(x, x2)].append(pair)
But, when I run this script, I hit a MemoryError.
I understand that this means that I am running out of memory but in the situation I'm in, adding more ram or updating to a 64-bit version of python is not an option.
I am certain that the problem is not coming from the list sorting or the first for loop. It has to be the second for loop. I just included the other lines for context.
The full output for the list above would be (sorry for the unsortedness, that's just how dictionaries do):
(200, 12): []
(6, 7): [(24, 11)]
(16, 10): []
(6, 6): [(6, 7), (24, 11)]
(5, 5): [(200, 12)]
(4, 4): [(8, 8), (16, 10), (24, 11), (200, 12)]
(9, 9): []
(8, 8): [(16, 10), (24, 11), (200, 12)]
(2, 2): [(4, 4), (6, 6), (6, 7), (8, 8), (16, 10), (24, 11), (200, 12)]
(24, 11): []
(1, 0): [(1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (6, 7), (8, 8), (9, 9), (16, 10), (24, 11), (200, 12)]
(1, 1): [(2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (6, 7), (8, 8), (9, 9), (16, 10), (24, 11), (200, 12)]
(3, 3): [(6, 6), (6, 7), (9, 9), (24, 11)]
Is there a better way of going about this?
EDIT:
This dictionary will then be fed into this:
ans_set = set()
for x in num_dict:
for y in num_dict[x]:
for z in num_dict[y]:
ans_set.add((x[0], y[0], z[0]))
return len(ans_set)
to find all unique possible triplets in which the 3rd value can be evenly divided by the 2nd value which can be evenly divided by the 1st.
If you think you know of a better way of doing the entire thing, I'm open to redoing the whole of it.
Final Edit
I've found the best way to find the number of triples by reevaluating what I needed it to do. This method doesn't actually find the triples, it just counts them.
def foo(l):
llen = len(l)
total = 0
cache = {}
for i in range(llen):
cache[i] = 0
for x in range(llen):
for y in range(x + 1, llen):
if l[y] % l[x] == 0:
cache[y] += 1
total += cache[x]
return total
And here's a version of the function that explains the thought process as it goes (not good for huge lists though because of spam prints):
def bar(l):
list_length = len(l)
total_triples = 0
cache = {}
for i in range(list_length):
cache[i] = 0
for x in range(list_length):
print("\n\nfor index[{}]: {}".format(x, l[x]))
for y in range(x + 1, list_length):
print("\n\ttry index[{}]: {}".format(y, l[y]))
if l[y] % l[x] == 0:
print("\n\t\t{} can be evenly diveded by {}".format(l[y], l[x]))
cache[y] += 1
total_triples += cache[x]
print("\t\tcache[{0}] is now {1}".format(y, cache[y]))
print("\t\tcount is now {}".format(total_triples))
print("\t\t(+{} from cache[{}])".format(cache[x], x))
else:
print("\n\t\tfalse")
print("\ntotal number of triples:", total_triples)

Well, you could start by not unnecessarily duplicating information.
Storing full tuples (number and index) for each multiple is inefficient when you already have that information available.
For example, rather than:
(3, 2): [(16, 4), (6, 7), (6, 8), (9, 10), (24, 11)]
(the 16 appears to be wrong there as it's not a multiple of 3 so I'm guessing you meant 15) you could instead opt for:
(3, 2): [15, 6, 9, 24]
(6, 7): ...
That pretty much halves your storage needs since you can go from the 6 in the list and find all its indexes by searching the tuples. That will, of course, be extra processing effort to traverse the list but it's probably better to have a slower working solution than a faster non-working one :-)
You could reduce the storage even more by not storing the multiples at all, instead running through the tuple list using % to see if you have a multiple.
But, of course, this all depends on your actual requirements which would be better off stating the intent of what your trying to achieve rather than pre-supposing a solution.

You rebuild tuples in places like pair = (y, y2 + x2 + 1) and num_dict[(x, x2)].append(pair) when you could build a canonical set of tuples early on and then just put references in the containers. I cobbled up a 2000 item test my machine that works. I have python 3.4 64 bit with a relatively modest 3.5 GIG of RAM...
import random
# a test list that should generate longish lists
l = list(random.randint(0, 2000) for _ in range(2000))
# setup canonical index and sort ascending
sorted_index = sorted((v,i) for i,v in enumerate(l))
num_dict = {}
for idx, vi in enumerate(sorted_index):
v = vi[0]
num_dict[vi] = [vi2 for vi2 in sorted_index[idx+1:] if not vi2[0] % v]
for item in num_dict.items():
print(item)