Develop Reference

Find pairs of elements in order in terms of distance

Suppose we have data a₁, ..., aₙ, where n is an even integer and each aᵢ ∈ ℝ. Also define the distance between two pairs of elements dis(aᵢ, aⱼ) = | aᵢ − aⱼ |. Now the program should output a list of pairs of elements sorted by the distance in an ascending order. Also the program should pack the input data into pairs, therefore each element aᵢ would only appear once in the output.
For example, given the input [1, 0.4, 3, 1.1] the output should be [(1, 1.1), (0.4, 3)].
A naive brute-force method is to calculate all C(n,2) pair and sorted the distance of each pair.
def not_in_list_of_pair(i, ls):
return not i in [p[0] for p in ls] + [p[1] for p in ls]
def calc(ls):
ls = sorted(ls)
d ={}
for idx1, i in enumerate(ls[:-1]):
for idx2, j in enumerate(ls[idx1+1:], idx1 + 1):
d[(i,j)] = j - i
# 2nd part
res = []
for pair in sorted(d, key = lambda k: d[k]):
i, j = pair
if not_in_list_of_pair(i, res) and not_in_list_of_pair(j, res):
res.append(pair)
return res
# another example
ls = [1, 0.1, 2, 2.4, 3, 4, 1.5]
assert calc(ls) == [(2, 2.4), (1, 1.5), (3, 4)]
But this naive method only works in O(n²), and the 2nd part (extracting min distance) is also slow. Therefore I am looking for a more effective method to solve this problem. Thanks!

I have to say that your descrption of the problem is not clear and the complexity in the description is not correct, i.e., you have to calculate the distance of all the pairs of integers (which is O(n^2)) and after that you sort all the distance (which is O(n^2 * log(n^2))).
For this problem, you are basically finding two integers with smallest distance, pick these two integers out, and repeat the same process on the remaining integers.
One naive solution is, supposed the integers are sorted, and we only find one pair of integers with smallest distance, then we just need to calculate the distance of each two adjacent integers (e.g., dist between ls[0] and ls[1], between ls[1] and ls[2], ..., between ls[n - 2] and ls[n - 1]) and find out which pair is the smallest. After we find one, remove the two selected integers, the remaining integers are still sorted. If we want to find the next pair of integers with smallest distance, the problem remains the same.
The naive solution is still expensive in two aspsects: (1) we need to calculate the distance of each two adjacent integers each time; (2) we need to remove two integers from a sorted array and keep the array sorted.
To solve (1), in fact, we don't have to calculate the all the distances each time. E.g., suppose we have 6 integers where we calculated dist(0, 1), dist(1, 2), dist(2, 3), dist(3, 4), dist(4, 5). We find that the 2nd and the 3rd integers are the closet ones, so we output and remove the 2nd and the 3rd integers. For the next round, we need to calculate dist(0, 1), dist(1, 4), dist(4, 5). We can see that we only need to remove dist(1, 2) and dist(3, 4) as they're useless, but we need to add a new distance dist(1, 4) while dist(0, 1) and dist(4, 5) are not changed. We can maintain a btree to achieve the purpose.
To solve (2), the best data structure where we can remove items from the middle is double linked list with complexity O(1). But we are using array now and we may not want to change array to linked list. One way is that we use index array to mimic a double linked list.
Here is an example.
Update 1: I found OrderedDict does not pop the minimal item each time. I don't find any data structure in python that works as btree. I have to use a heap where I cannot delete those useless distance but I can identiy and ignore them. Sorry for the mistake.
Update 2: Add a else branch in the while loop, i.e., we should not change the double linked list when we see a useless item.
Update 3: Just realize that the heap will have no more than n items in each iteration in the while loop. So the complexity is roughly O(n log n), with n being the number of integers.
from heapq import *
def calc(ls):
ls = sorted(ls) # O(nlogn)
n = len(ls)
# mimic a double linked list
left = [i - 1 for i in range(n)]
right = [i + 1 for i in range(n)]
appeared = [False for i in range(n)]
btree = []
for i in range(0, n - 1):
# distance of adjacent integers, and their indices
heappush(btree, (ls[i + 1] - ls[i], i, i + 1))
# roughly O(n log n), because the heap will have at most `n` items in each iteration
result = []
while len(btree) != 0:
minimal = heappop(btree)
a, b = minimal[1:3]
# skip if either a or b appeared
if not appeared[a] and not appeared[b]:
result.append((ls[a], ls[b]))
appeared[a] = True
appeared[b] = True
else:
continue # this is important
#print result
if left[a] != -1:
right[left[a]] = right[b]
if right[b] != n:
left[right[b]] = left[a]
if left[a] != -1 and right[b] != n:
heappush(btree, (ls[right[b]] - ls[left[a]], left[a], right[b]))
return result
ls = [1, 0.1, 2, 2.4, 3, 4, 1.5]
print calc(ls)
With the following output:
[(2, 2.4), (1, 1.5), (3, 4)]
Note: The number of input integers is 7, which is NOT even.
Show one more image to present what is going on:
I am not very familiar with Python, so I may not be using the best data structure in the above code snippet.

Find ways to get to a number of levels

I got this problem on CoderByte. The requirement was to find a number of ways. I found solutions for that in StackOverflow and other sites. But moving ahead, I need all possible ways as well to reach the Nth step.
Problem description: There is a staircase of N steps and you can climb either 1 or 2 steps at a time. You need to count and return the total number of unique ways to climb the staircase. The order of steps taken matters.
For Example,
Input: N = 3
Output: 3
Explanation: There are 3 unique ways of climbing a staircase of 3 steps :{1,1,1}, {2,1} and {1,2}
Note: There might be another case that a person can take 2 or 3 or 4 steps at a time (I know that's realistically not possible but trying to add scalability to the input steps in the code)
I'm unable to find the right logic to get all the ways possible. It's useful if I get the solution in Python, but it's not a strict requirement though.

Here's a minimal solution using itertools library:
from itertools import permutations, chain
solve = lambda n: [(1,)*n] + list(set(chain(*[permutations((2,)*i + (1,)*(n-2*i)) for i in range(1, n//2+1)])))
For your example input:
> solve(3)
[(1, 1, 1), (1, 2), (2, 1)]
How it works?
It's easier to see what's happening if we take a step backwards:
def solve(n):
combinations = [(1,)*n]
for i in range(1, n//2+1):
combinations.extend(permutations((2,)*i + (1,)*(n-2*i)))
return list(set(combinations))
The most trivial case is the one where you take one step at a time, so n steps: (1,)*n. Then we can look for how many double steps could we take at most, and that's the floor of n divided by 2: n//2. Then we iterate over the possible double steps: try to add a double step each iteration (2,)*i, filling the remaining space with single steps (1,)*(n-2*i).
The function permutations from itertools will generate all the possible permutations of single and double steps for that iteration. With an input of (1,1,2), it will generate (1,1,2), (1,2,1) and (2,1,1). At the end we use the trick of converting the result to a set in order to remove duplicates, then converting it back into a list.
Generalization for any amount and length of steps (not optimal!)
One liner:
from itertools import permutations, chain, combinations_with_replacement
solve = lambda n, steps: list(set(chain(*[permutations(sequence) for sequence in chain(*[combinations_with_replacement(steps, r) for r in range(n//min(steps)+1)]) if sum(sequence) == n])))
Example output:
> solve(8, [2,3])
[(3, 2, 3), (2, 3, 3), (2, 2, 2, 2), (3, 3, 2)]
Easier to read version:
def solve(n, steps):
result = []
for sequence_length in range(n//min(steps)+1):
sequences = combinations_with_replacement(steps, sequence_length)
for sequence in sequences:
if sum(sequence) == n:
result.extend(permutations(sequence))
return list(set(result))

def solve(n) :
if (n == 0):
return [[]]
else:
left_results = []
right_results = []
if (n > 0):
left_results = solve(n - 1)
for res in left_results: # Add the current step to every result
res.append(1)
if (n > 1):
right_results = solve(n - 2)
for res in right_results: # Same above
res.append(2)
return left_results + right_results
I think there is a better way to do this using dynamic programming but I don't know how to do that. Hope it helps anyway.

Efficient enumeration of non-negative integer composition

I would like to write a function my_func(n,l) that, for some positive integer n, efficiently enumerates the ordered non-negative integer composition* of length l (where l is greater than n). For example, I want my_func(2,3) to return [[0,0,2],[0,2,0],[2,0,0],[1,1,0],[1,0,1],[0,1,1]].
My initial idea was to use existing code for positive integer partitions (e.g. accel_asc() from this post), extend the positive integer partitions by a couple zeros and return all permutations.
def my_func(n, l):
for ip in accel_asc(n):
nic = numpy.zeros(l, dtype=int)
nic[:len(ip)] = ip
for p in itertools.permutations(nic):
yield p
The output of this function is wrong, because every non-negative integer composition in which a number appears twice (or multiple times) appears several times in the output of my_func. For example, list(my_func(2,3)) returns [(1, 1, 0), (1, 0, 1), (1, 1, 0), (1, 0, 1), (0, 1, 1), (0, 1, 1), (2, 0, 0), (2, 0, 0), (0, 2, 0), (0, 0, 2), (0, 2, 0), (0, 0, 2)].
I could correct this by generating a list of all non-negative integer compositions, removing repeated entries, and then returning a remaining list (instead of a generator). But this seems incredibly inefficient and will likely run into memory issues. What is a better way to fix this?
EDIT
I did a quick comparison of the solutions offered in answers to this post and to another post that cglacet has pointed out in the comments.
On the left, we have the l=2*n and on the right we have l=n+1. In these two cases, user2357112's second solutions is faster than the others, when n<=5. For n>5, solutions proposed by user2357112, Nathan Verzemnieks, and AndyP are more or less tied. But the conclusions could be different when considering other relationships between l and n.
..........
*I originally asked for non-negative integer partitions. Joseph Wood correctly pointed out that I am in fact looking for integer compositions, because the order of numbers in a sequence matters to me.

Use the stars and bars concept: pick positions to place l-1 bars between n stars, and count how many stars end up in each section:
import itertools
def diff(seq):
return [seq[i+1] - seq[i] for i in range(len(seq)-1)]
def generator(n, l):
for combination in itertools.combinations_with_replacement(range(n+1), l-1):
yield [combination[0]] + diff(combination) + [n-combination[-1]]
I've used combinations_with_replacement instead of combinations here, so the index handling is a bit different from what you'd need with combinations. The code with combinations would more closely match a standard treatment of stars and bars.
Alternatively, a different way to use combinations_with_replacement: start with a list of l zeros, pick n positions with replacement from l possible positions, and add 1 to each of the chosen positions to produce an output:
def generator2(n, l):
for combination in itertools.combinations_with_replacement(range(l), n):
output = [0]*l
for i in combination:
output[i] += 1
yield output

Starting from a simple recursive solution, which has the same problem as yours:
def nn_partitions(n, l):
if n == 0:
yield [0] * l
else:
for part in nn_partitions(n - 1, l):
for i in range(l):
new = list(part)
new[i] += 1
yield new
That is, for each partition for the next lower number, for each place in that partition, add 1 to the element in that place. It yields the same duplicates yours does. I remembered a trick for a similar problem, though: when you alter a partition p for n into one for n+1, fix all the elements of p to the left of the element you increase. That is, keep track of where p was modified, and never modify any of p's "descendants" to the left of that. Here's the code for that:
def _nn_partitions(n, l):
if n == 0:
yield [0] * l, 0
else:
for part, start in _nn_partitions(n - 1, l):
for i in range(start, l):
new = list(part)
new[i] += 1
yield new, i
def nn_partitions(n, l):
for part, _ in _nn_partitions(n, l):
yield part
It's very similar - there's just the extra parameter passed along at each step, so I added wrapper to remove that for the caller.
I haven't tested it extensively, but this appears to be reasonably fast - about 35 microseconds for nn_partitions(3, 5) and about 18s for nn_partitions(10, 20) (which yields just over 20 million partitions). (The very elegant solution from user2357112 takes about twice as long for the smaller case and about four times as long for the larger one. Edit: this refers to the first solution from that answer; the second one is faster than mine under some circumstances and slower under others.)

Finding two integers that multiply to 20. Can I make this code more "pythonic"?

I did this code that finds two integers in a said list (in this case [2,4,5,1,6,40,-1]) that multiply to twenty. I got a little stuck in the beginning, but adding a function to it solved my problems. I showed this code to a friend of mine who's a programmer and he said I could make this code more "pythonic", but I have no clue how.
Here's the code:
num_list = [2,4,5,1,6,40,-1]
def get_mult_num(given_list):
for i in given_list:
for j in range(i+1, len(given_list)): #for j not to be == i and to be in the list
mult_two_numbers = i * j
if mult_two_numbers == 20:
return i,j
print(get_mult_num(num_list))

I don't necessarily think it is 'unpythonic', you are using standard Python idioms to loop over your data and produce a single result or None. The term Pythonic is nebulous, a subject marred in "I know it when I see it" parameters.
Not that you produced a correct implementation. While i loops over given_numbers, j loops over an integer from i + 2 through to len(given_numbers), mixing values from given_list with indices? For your sample input, you are taking j from the half-open ranges [4, 7), [6, 7), [7, 7) (empty), [3, 7), [8, 7) (empty), [42, 7) (empty) and [1, 7), respectively. That it produces the correct answer at all is luck, not due to correctness; if you give your function the list [2, 10], it'll not find a solution! You want to loop over given_numbers again, limited with slicing, or generate indices starting at the current index of i, but then your outer loop needs to add a enumerate() call too:
for ii, i in enumerate(given_numbers):
for j in given_numbers[ii + 1:]:
# ...
or
for ii, i in enumerate(given_numbers):
for jj in range(ii + 1, len(given_numbers)):
j = given_numbers[jj]
# ...
All this is not nearly as efficient as it can be; the Python standard library offers you the tools to generate your i, j pairs without a nested for loop or slicing or other forms of filtering.
Your double loop should generate combinations of the integer inputs, so use the itertools.combinations() object to generate unique i, j pairs:
from itertools import combinations
def get_mult_num(given_list):
return [(i, j) for i, j in combinations(given_list, 2) if i * j == 20]
This assumes there can be zero or more such solutions, not just a single solution.
If you only ever need the first result or None, you can use the next() function:
def get_mult_num(given_list):
multiplies_to_20 = (
(i, j) for i, j in combinations(given_list, 2)
if i * j == 20)
return next(multiplies_to_20, None)
Next, rather than produce all possible combinations, you may want to invert the problem. If you turn given_list into a set, you can trivially check if the target number 20 can be divided cleanly without remainder by any of your given numbers and where the result of the division is larger and is also an integer in the set of numbers. That gives you an answer in linear time.
You can further limit the search by dividing with numbers smaller than the square root of the target value, because you won't find a larger value to match in your input numbers (given a number n and it's square root s, by definition s * (s + 1) is going to be larger than n).
If we add an argument for the target number to the function and make it a generator function, then you get:
def gen_factors_for(target, numbers):
possible_j = set(numbers)
limit = abs(target) ** 0.5
for i in numbers:
if abs(i) < limit and target % i == 0:
j = target // i
if j in possible_j and abs(j) > abs(i):
yield i, j
This approach is a lot faster than testing all permutations, especially if you need to find all possible factors. Note that I made both functions generators here to even out the comparisons:
>>> import random, operator
>>> from timeit import Timer
>>> def gen_factors_for_division(target, numbers):
... possible_j = set(numbers)
... limit = abs(target) ** 0.5
... for i in numbers:
... if abs(i) < limit and target % i == 0:
... j = target // i
... if j in possible_j and abs(j) > abs(i):
... yield i, j
...
>>> def gen_factors_for_combinations(target, given_list):
... return ((i, j) for i, j in combinations(given_list, 2) if i * j == target)
...
>>> numbers = [random.randint(-10000, 10000) for _ in range(100)]
>>> targets = [operator.mul(*random.sample(set(numbers), 2)) for _ in range(5)]
>>> targets += [t + random.randint(1, 100) for t in targets] # add likely-to-be-unsolvable numbers
>>> for (label, t) in (('first match:', 'next({}, None)'), ('all matches:', 'list({})')):
... print(label)
... for f in (gen_factors_for_division, gen_factors_for_combinations):
... test = t.format('f(t, n)')
... timer = Timer(
... f"[{test} for t in ts]",
... 'from __main__ import targets as ts, numbers as n, f')
... count, total = timer.autorange()
... print(f"{f.__name__:>30}: {total / count * 1000:8.3f}ms")
...
first match:
gen_factors_for_division: 0.219ms
gen_factors_for_combinations: 4.664ms
all matches:
gen_factors_for_division: 0.259ms
gen_factors_for_combinations: 3.326ms
Note that I generate 10 different random targets, to try to avoid a lucky best-case-scenario hit for either approach.

[(i,j) for i in num_list for j in num_list if i<j and i*j==20]

This is my take on it, which uses enumerate:
def get_mult_num(given_list):
return [
item1, item2
for i, item1 in enumerate(given_list)
for item2 in given_list[:i]
if item1*item2 == 20
]
I think your friend may be hinting towards using comprehensions when it makes the code cleaner (sometimes it doesn't).

I can think of using list-comprehension. This also helps to find multiple such-pairs if they exist in the given list.
num_list = [2,4,5,1,6,40,-1]
mult_num = [(num_list[i],num_list[j]) for i in range(len(num_list)) for j in range(i+1, len(num_list)) if num_list[i]*num_list[j] == 20]
print mult_num
Output:
[(4, 5)]

I came up with this. It reverses the approach a little bit, in that it searches in num_list for the required pair partner that the iteration value val would multiply to 20 with. This makes the code easier and needs no imports, even if it's not the most efficient way.
for val in num_list:
if 20 / val in num_list:
print(val, int(20/val))

You could make it more pythonic by using itertools.combinations, instead of nested loops, to find all pairs of numbers. Not always, but often iterating over indices as in for i in range(len(L)): is less pythonic than directly iterating over values as in for v in L:.
Python also allows you to make your function into a generator via the yield keyword so that instead of just returning the first pair that multiplies to 20, you get every pair that does by iterating over the function call.
import itertools
def factors(x, numbers):
""" Generate all pairs in list of numbers that multiply to x.
"""
for a, b in itertools.combinations(numbers, 2):
if a * b == x:
yield (a, b)
numbers = [2, 4, 5, 1, 6, 40, -1]
for pair in factors(20, numbers):
print(pair)

Code challenge: finding the divisible in a list

I am playing a code challenge. Simply speaking, the problem is:
Given a list L (max length is of the order of 1000) containing positive integers.
Find the number of "Lucky Triples", which is L[i] divides L[j], and L[j] divides L[k].
for example, [1,2,3,4,5,6] should give the answer 3 because [1,2,4], [1,2,6],[1,3,6]
My attempt:
Sort the list. (let say there are n elements)
3 For loops: i, j, k (i from 1 to n-2), (j from i+1 to n-1), (k from j+1 to n)
only if L[j] % L[i] == 0, the k for loop will be executed
The algorithm seems to give the correct answer. But the challenge said that my code exceeded the time limit. I tried on my computer for the list [1,2,3,...,2000], count = 40888(I guess it is correct). The time is around 5 second.
Is there any faster way to do that?
This is the code I have written in python.
def answer(l):
l.sort()
cnt = 0
if len(l) == 2:
return cnt
for i in range(len(l)-2):
for j in range(1,len(l)-1-i):
if (l[i+j]%l[i] == 0):
for k in range(1,len(l)-j-i):
if (l[i+j+k]%l[i+j] == 0):
cnt += 1
return cnt

You can use additional space to help yourself. After you sort the input list you should make a map/dict where the key is each element in the list and value is a list of elements which are divisible by that in the list so you would have something like this
assume sorted list is list = [1,2,3,4,5,6] your map would be
1 -> [2,3,4,5,6]
2-> [4,6]
3->[6]
4->[]
5->[]
6->[]
now for every key in the map you find what it can divide and then you find what that divides, for example you know that
1 divides 2 and 2 divides 4 and 6, similarly 1 divides 3 and 3 divides 6
the complexity of sorting should be O(nlogn) and that of constructing the list should be better than O(n^2) (but I am not sure about this part) and then I am not sure about the complexity of when you are actually checking for multiples but I think this should be much much faster than a brute force O(n^3)
If someone could help me figure out the time complexity of this I would really appreciate it
EDIT :
You can make the map creation part faster by incrementing by X (and not 1) where X is the number in the list you are currently on since it is sorted.

Thank you guys for all your suggestions. They are brilliant. But it seems that I still can't pass the speed test or I cannot handle with duplicated elements.
After discussing with my friend, I have just come up with another solution. It should be O(n^2) and I passed the speed test. Thanks all!!
def answer(lst):
lst.sort()
count = 0
if len(lst) == 2:
return count
#for each middle element, count the divisors at the front and the multiples at the back. Then multiply them.
for i, middle in enumerate(lst[1:len(lst)-1], start = 1):
countfirst = 0
countthird = 0
for first in (lst[0:i]):
if middle % first == 0:
countfirst += 1
for third in (lst[i+1:]):
if third % middle == 0:
countthird += 1
count += countfirst*countthird
return count

I guess sorting the list is pretty inefficient. I would rather try to iteratively reduce the number of candidates. You could do that in two steps.
At first filter all numbers that do not have a divisor.
from itertools import combinations
candidates = [max(pair) for pair in combinations(l, 2) if max(pair)%min(pair) == 0]
After that, count the number of remaining candidates, that do have a divisor.
result = sum(max(pair)%min(pair) == 0 for pair in combinations(candidates, 2))

Your original code, for reference.
def answer(l):
l.sort()
cnt = 0
if len(l) == 2:
return cnt
for i in range(len(l)-2):
for j in range(1,len(l)-1-i):
if (l[i+j]%l[i] == 0):
for k in range(1,len(l)-j-i):
if (l[i+j+k]%l[i+j] == 0):
cnt += 1
return cnt
There are a number of misimplementations here, and with just a few tweaks we can probably get this running much faster. Let's start:
def answer(lst): # I prefer not to use `l` because it looks like `1`
lst.sort()
count = 0 # use whole words here. No reason not to.
if len(lst) == 2:
return count
for i, first in enumerate(lst):
# using `enumerate` here means you can avoid ugly ranges and
# saves you from a look up on the list afterwards. Not really a
# performance hit, but definitely looks and feels nicer.
for j, second in enumerate(lst[i+1:], start=i+1):
# this is the big savings. You know since you sorted the list that
# lst[1] can't divide lst[n] if n>1, but your code still starts
# searching from lst[1] every time! Enumerating over `l[i+1:]`
# cuts out a lot of unnecessary burden.
if second % first == 0:
# see how using enumerate makes that look nicer?
for third in lst[j+1:]:
if third % second == 0:
count += 1
return count
I bet that on its own will pass your speed test, but if not, you can check for membership instead. In fact, using a set here is probably a great idea!
def answer2(lst):
s = set(lst)
limit = max(s) # we'll never have a valid product higher than this
multiples = {} # accumulator for our mapping
for n in sorted(s):
max_prod = limit // n # n * (max_prod+1) > limit
multiples[n] = [n*k for k in range(2, max_prod+1) if n*k in s]
# in [1,2,3,4,5,6]:
# multiples = {1: [2, 3, 4, 5, 6],
# 2: [4, 6],
# 3: [6],
# 4: [],
# 5: [],
# 6: []}
# multiples is now a mapping you can use a Depth- or Breadth-first-search on
triples = sum(1 for j in multiples
for k in multiples.get(j, [])
for l in multiples.get(k, []))
# This basically just looks up each starting value as j, then grabs
# each valid multiple and assigns it to k, then grabs each valid
# multiple of k and assigns it to l. For every possible combination there,
# it adds 1 more to the result of `triples`
return triples

I'll give you just an idea, the implementation should be up to you:
Initialize the global counter to zero.
Sort the list, starting with smallest number.
Create a list of integers (one entry per number with same index).
Iterate through each number (index i), and do the following:
Check for dividers at positions 0 to i-1.
Store the number of dividers in the list at the position i.
Fetch the number of dividers from the list for each divider, and add each number to the global counter.
Unless you finished, go to 3rd.
Your result should be in the global counter.