Develop Reference

Best suitable approach to find nearest neighbour to (x, y, z) from list of triplets

I am trying to obtain a triplet from list of triplets that is closest to my required triplet incase if it was not found.
For example:
# V_s,V_g,V_r
triplets = [(500, 12, 5),
(400, 15, 2.5),
(400, 15, 3),
(450, 12, 3),
... ,
(350, 14, 3)]
The triple that I am looking for is
req_triplet = (450, 15, 2) #(Vreq_s, Vreq_g, Vreq_r)
How can I achieve this in python, a best suitable strategy to achieve is what I am in need for.
As of now I am thinking to filter the list by finding nearest parameter V_s. From the resulting list filter further by finding nearest to V_g and finally by V_r.

You can compute Euclidean distance with numPy or you can use
numpy.linalg.norm.
Try this:
>>> import numpy as np
>>> def dist(x,y):
... return np.sqrt(np.sum((x-y)**2))
>>> triplets = [(500, 12, 5), (400, 15, 2.5), (400, 15, 3),(450, 12, 3)(350, 14, 3)]
>>> req_triplet = (450, 15, 2)
>>> arr_dst = [np.linalg.norm(np.array(tr) - np.array(req_triplet)) for tr in triplets]
>>> arr_dst = [dist(np.array(tr), np.array(req_triplet)) for tr in triplets]
>>> arr_dst
[50.17967716117751, 50.002499937503124, 50.00999900019995, 3.1622776601683795, 100.00999950005]
>>> idx = np.argmin(arr_dst)
>>> idx
3
>>> triplets[idx]
(450, 12, 3)

You have to define a metric ||.||, then the triplet T that is close to a fixed one F is the one that minimize ||T - F||. You can use a classic Euclidean distance:
import numpy as np
def dist(u, v):
return np.sqrt(np.sum((np.array(u)-np.array(v))**2))

The general strategy would be to Loop through the list, for each element calculate the distance and check if it is the minimum, otherwise keep going on.
In python this would look something like this-
from math import abs
triplets = [(500, 12, 5),
(400, 15, 2.5),
(400, 15, 3),
(450, 12, 3),
... ,
(350, 14, 3)]
req_triplet = (450, 15, 2)
def calc_dist(a,b):
return sum((abs(a[i]-b[i]) for i in range(3))
def find_closest_triple(req_triplet,triplets):
min_ind = None
min_dist = -1
for i,triplet in enumerate(triplets):
if e == req_triplet:
return i
dist = calc_dist(req_triplet,triplet)
if dist < min_dist:
min_ind = i
return min_ind

Change some dataframe values to 0

from a list of lists of coordinates such as:
coordinates = [((3,54), (6,23)...)((1,10)(2,98)]
I'm getting a dataframe with this code:
all_coordinates = [item for img in coordinates for item in img]
c = Counter(all_coordinates)
max_x = max([item[0] for item in all_coordinates])
max_y = max([item[1] for item in all_coordinates])
coordinates_dict = dict()
for i in range(max_x + 1):
for j in range(max_y + 1):
coordinates_dict.update({(i,j): c[(i,j)]})
df = pd.DataFrame(coordinates_dict.items(), columns=['Coordinates','Value'])
pd.set_option("display.max_rows", 20, "display.max_columns", 20)
df[["x", "y"]] = pd.DataFrame(df.Coordinates.to_list())
That gives me a dataframe like:
I would like to know How can I turn into 0 all values from a selected list of coordiantes:
All coordinates from the following list, turn in the dataframe into 0:
coords_0 = [(29, 131), (29, 132), (29, 133), (30, 129), (30, 130), (30, 133), (31, 127)...]
Thanks

df['Value'] = df.apply(lambda x: 0 if x['Coordinates'] in coords_0 else x['Value'], axis=1)

Recursively trying to find the maximum w/o loops

So I'm given a tuple of ordered pairs in this format:
(x,y) where
x represents the physical weight of the objects, y represents the cost/value of the object.
((5, 20), (10, 70), (40, 200), (20, 80), (10, 100))
Objects may only be used once, but there may be multiples of those objects in the original tuple of ordered pairs.
z is the max weight that can be shipped. It's an integer. z could be 50 or something like that.
Goal: Find the maximum value possible that you can send given the limit Z.
The difficulty is that we can ONLY use recursion and we cannot use loops nor can we use python built-in functions.
I've tried to work out the max value in a list of integers, which I did separately to try to get some sort of idea. I have also tried giving the objects a 'mass' and doing value/weight, but that didn't work very well either.
def maximum_val(objects: ((int,int),) , max_weight : int) -> int:
if max_weight == 0:
return 0
else:
return objects[0][1] + maximum_val(objects[1:], max_weight - gifts[0][0])
((5, 20), (10, 70), (40, 200), (20, 80), (10, 100))
Example: Given the tuple above and the limit Z=40, the best possible value that could be obtained is 250 -> (10, 70), (10, 100), (20, 80)

This is known as knapsack and you are looking for a recursive variant.
At every step, check what is best. Include the first object or skip the first object:
objects = ((5, 20), (10, 70), (40, 200), (20, 80), (10, 100))
def recursive_knapsack(objects, limit ):
if not objects:
return 0
if objects[0][0] > limit:
#first object cant fit
return recursive_knapsack(objects[1:],limit)
include = objects[0][1] + recursive_knapsack(objects[1:], limit-objects[0][0])
exclude = recursive_knapsack(objects[1:],limit)
if include < exclude:
return exclude
else:
return include

Building a set of tile coordinates

I have an image which I want to divide into tiles of specific size (and cropping tiles that don't fit).
The output of this operation should be a list of coordinates in tuples [(x, y, width, height),...]. For example, dividing a 50x50 image in tiles of size 20 would give: [(0,0,20,20),(20,0,20,20),(40,0,10,20),(0,20,20,20),(20,20,20,20),(40,20,10,20),...] etc.
Given a height, width and tile_size, it seems like I should be able to do this in a single list comprehension, but I can't wrap my head around it. Any help would be appreciated. Thanks!

Got it with:
output = [(x,y,w,h) for x,w in zip(range(width)[::tile_size],[tile_size]*(w_tiles-1) + [w_padding]) for y,h in zip(range(height)[::tile_size],[tile_size]*(h_tiles-1) + [h_padding])]

import itertools
def tiles(h, w, ts):
# here is the one list comprehension for list of tuples
return [tuple(list(ele) + [ts if w-ele[0] > 20 else w-ele[0], ts if h-ele[1] > 20 else h-ele[1]]) for ele in itertools.product(*[filter(lambda x: x % ts == 0, range(w)), filter(lambda x: x % ts == 0, range(h))])]
print tiles(50, 50, 20)
[(0, 0, 20, 20), (0, 20, 20, 20), (0, 40, 20, 10), (20, 0, 20, 20), (20, 20, 20, 20), (20, 40, 20, 1
0), (40, 0, 10, 20), (40, 20, 10, 20), (40, 40, 10, 10)]

Is there a standard Python data structure that keeps things in sorted order?

I have a set of ranges that might look something like this:
[(0, 100), (150, 220), (500, 1000)]
I would then add a range, say (250, 400) and the list would look like this:
[(0, 100), (150, 220), (250, 400), (500, 1000)]
I would then try to add the range (399, 450), and it would error out because that overlapped (250, 400).
When I add a new range, I need to search to make sure the new range does not overlap an existing range. And no range will ever be in the list that overlaps another range in the list.
To this end, I would like a data structure that cheaply maintained its elements in sorted order, and quickly allowed me to find the element before or after a given element.
Is there a better way to solve this problem? Is there a data structure like that available in Python?
I know the bisect module exists, and that's likely what I will use. But I was hoping there was something better.
EDIT: I solved this using the bisect module. I had a link to the code since it was a bit longish. Unfortunately, paste.list.org turned out to be a bad place to put it because it's not there anymore.

It looks like you want something like bisect's insort_right/insort_left. The bisect module works with lists and tuples.
import bisect
l = [(0, 100), (150, 300), (500, 1000)]
bisect.insort_right(l, (250, 400))
print l # [(0, 100), (150, 300), (250, 400), (500, 1000)]
bisect.insort_right(l, (399, 450))
print l # [(0, 100), (150, 300), (250, 400), (399, 450), (500, 1000)]
You can write your own overlaps function, which you can use to check before using insort.
I assume you made a mistake with your numbers as (250, 400) overlaps (150, 300).
overlaps() can be written like so:
def overlaps(inlist, inrange):
for min, max in inlist:
if min < inrange[0] < max and max < inrange[1]:
return True
return False

Use SortedDict from the SortedCollection.
A SortedDict provides the same methods as a dict. Additionally, a SortedDict efficiently maintains its keys in sorted order. Consequently, the keys method will return the keys in sorted order, the popitem method will remove the item with the highest key, etc.
I've used it - it works. Unfortunately I don't have the time now to do a proper performance comparison, but subjectively it seems to have become faster than the bisect module.

Cheap searching and cheap insertion tend to be at odds. You could use a linked list for the data structure. Then searching to find the insertion point for a new element is O(n), and the subsequent insertion of the new element in the correct location is O(1).
But you're probably better off just using a straightforward Python list. Random access (i.e. finding your spot) takes constant time. Insertion in the correct location to maintain the sort is theoretically more expensive, but that depends on how the dynamic array is implemented. You don't really pay the big price for insertions until reallocation of the underlying array takes place.
Regarding checking for date range overlaps, I happen to have had the same problem in the past. Here's the code I use. I originally found it in a blog post, linked from an SO answer, but that site no longer appears to exist. I actually use datetimes in my ranges, but it will work equally well with your numeric values.
def dt_windows_intersect(dt1start, dt1end, dt2start, dt2end):
'''Returns true if two ranges intersect. Note that if two
ranges are adjacent, they do not intersect.
Code based on:
http://beautifulisbetterthanugly.com/posts/2009/oct/7/datetime-intersection-python/
http://stackoverflow.com/questions/143552/comparing-date-ranges
'''
if dt2end <= dt1start or dt2start >= dt1end:
return False
return dt1start <= dt2end and dt1end >= dt2start
Here are the unit tests to prove it works:
from nose.tools import eq_, assert_equal, raises
class test_dt_windows_intersect():
"""
test_dt_windows_intersect
Code based on:
http://beautifulisbetterthanugly.com/posts/2009/oct/7/datetime-intersection-python/
http://stackoverflow.com/questions/143552/comparing-date-ranges
|-------------------| compare to this one
1 |---------| contained within
2 |----------| contained within, equal start
3 |-----------| contained within, equal end
4 |-------------------| contained within, equal start+end
5 |------------| overlaps start but not end
6 |-----------| overlaps end but not start
7 |------------------------| overlaps start, but equal end
8 |-----------------------| overlaps end, but equal start
9 |------------------------------| overlaps entire range
10 |---| not overlap, less than
11 |-------| not overlap, end equal
12 |---| not overlap, bigger than
13 |---| not overlap, start equal
"""
def test_contained_within(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,6,30), datetime(2009,10,1,6,40),
)
def test_contained_within_equal_start(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,6,0), datetime(2009,10,1,6,30),
)
def test_contained_within_equal_end(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,6,30), datetime(2009,10,1,7,0),
)
def test_contained_within_equal_start_and_end(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
)
def test_overlaps_start_but_not_end(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,5,30), datetime(2009,10,1,6,30),
)
def test_overlaps_end_but_not_start(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,6,30), datetime(2009,10,1,7,30),
)
def test_overlaps_start_equal_end(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,5,30), datetime(2009,10,1,7,0),
)
def test_equal_start_overlaps_end(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,6,0), datetime(2009,10,1,7,30),
)
def test_overlaps_entire_range(self):
assert dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,5,0), datetime(2009,10,1,8,0),
)
def test_not_overlap_less_than(self):
assert not dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,5,0), datetime(2009,10,1,5,30),
)
def test_not_overlap_end_equal(self):
assert not dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,5,0), datetime(2009,10,1,6,0),
)
def test_not_overlap_greater_than(self):
assert not dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,7,30), datetime(2009,10,1,8,0),
)
def test_not_overlap_start_equal(self):
assert not dt_windows_intersect(
datetime(2009,10,1,6,0), datetime(2009,10,1,7,0),
datetime(2009,10,1,7,0), datetime(2009,10,1,8,0),
)

Maybe the module bisect could be better than the simple following function ? :
li = [(0, 100), (150, 220), (250, 400), (500, 1000)]
def verified_insertion(x,L):
u,v = x
if v<L[0][0]:
return [x] + L
elif u>L[-1][0]:
return L + [x]
else:
for i,(a,b) in enumerate(L[0:-1]):
if a<u and v<L[i+1][0]:
return L[0:i+1] + [x] + L[i+1:]
return L
lo = verified_insertion((-10,-2),li)
lu = verified_insertion((102,140),li)
le = verified_insertion((222,230),li)
lee = verified_insertion((234,236),le) # <== le
la = verified_insertion((408,450),li)
ly = verified_insertion((2000,3000),li)
for w in (lo,lu,le,lee,la,ly):
print li,'\n',w,'\n'
The function returns a list without modifying the list passed as argument.
result
[(0, 100), (150, 220), (250, 400), (500, 1000)]
[(-10, -2), (0, 100), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (102, 140), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (222, 230), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (222, 230), (234, 236), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (408, 450), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (500, 1000)]
[(0, 100), (150, 220), (250, 400), (500, 1000), (2000, 3000)]

To answer your question:
Is there a data structure like that available in Python?
No there is not. But you can easily build one yourself using a list as the basic structure and code from the bisect module to keep the list in order and check for overlaps.
class RangeList(list):
"""Maintain ordered list of non-overlapping ranges"""
def add(self, range):
"""Add a range if no overlap else reject it"""
lo = 0; hi = len(self)
while lo < hi:
mid = (lo + hi)//2
if range < self[mid]: hi = mid
else: lo = mid + 1
if overlaps(range, self[lo]):
print("range overlap, not added")
else:
self.insert(lo, range)
I leave the overlaps function as an exercise.
(This code is untested and may need some tweeking)