Develop Reference

Can I turn this for loop into recursion or dynamic programming?

I am trying to make the code that I have into either recursion or dynamic programming.
import numpy as np
index_list = [1, 2, 0]
weights = [0.3, 0.8]
A_matrix = np.asarray([[0, 1, 2], [0, 1, 2], [0, 1, 2]])
initial_best_vector = A_matrix[:, 1]
# set best_vector_combinations to initial_best_vector
best_vector_combinations = initial_best_vector
for index, _ in enumerate(index_list[1:]):
best_vector_combinations = (
1 - weights[index]
) * best_vector_combinations + (
weights[index] * A_matrix[:, index_list[index + 1]]
)
Is it possible to do so? What I am doing is a nested linear combination of vectors, with the initial base being the initial_best_vector, which corresponds to the index_list.
In other words, let c_i be the columns of the matrix A, I want:
((1-0.3) * c_1 + 0.3 * c_2) * (1-0.8) + 0.8 * c_0
I hope to make this case more general to hold for any length of numbers.
Edit:
The code:
def calculate(vectors, weights):
if not (vectors or weights):
return 0
if not weights:
return vectors[0]
return vectors[0]*(1-weights[0]) + weights[0] * (calculate(vectors[1:], weights[1:]))
vectors = [1,2,3]
weights = [0.2, 0.3]
calculate(vectors, weights) = 1.26
but expected answer is 1.74 where i would expect first case to be 0.8 * 1 + 0.2 * 2 = 1.2, then second to be 1.2 * 0.7 + 3 * 0.3 = 1.74. Note I replaced your typo result to calculate but still unable to recover 1.74.

If you want a recursive implementation, if would be helpful to start with a simpler example and figure out the recurrence relation.
Let vectors = [8,5,2,1] (1D array for simplicity) and let weights = [0.5, 0.8, 0.1, 0.2].
First step of computation: (8 * 0.5) + (1-0.5)*(result of second step).
Second step: 5 * 0.8 + (1-0.8)*(result of third step).
You can work this out further, but the basic relation is
result(vectors, weights) =
(
vectors[0]*weights[0]) +
(1-weights[0]) * (result(vectors[1:], weights[1:]))
) if (vectors and weights) else 0
Implementation:
def calculate(vectors, weights):
if not (vectors or weights):
return 0
if not weights:
return vectors[0]
return vectors[0]*weights[0] + (1-weights[0]) * (calculate(vectors[1:], weights[1:]))
print(calculate([1,2,3], [0.2,0.3])) #left to right processing, 1.26
print(calculate([1,2,3][::-1], [0.2,0.3][::-1])) #right to left processing, 1.74

scipy fsolve fails for few values in input, and how to improve solver convergence

I am using scipy.optimize fsolve for finding roots of two equations. fsolve works well for some range of b values (0.1 to 0.6), but fails for values like 0.9 or 0.99.
I have tried moving to least_squares or minimize, but get a tuple error whilst providing initial conditions.
Including edits from below findings be self:
from scipy.optimize import fsolve
import scipy.stats as st
from numpy import *
import numpy as np
def rod(var1, var2, mu, sigma):
return (st.lognorm.ppf(var1, s = sigma, scale = np.exp(mu), loc = sigma))/(st.lognorm.ppf(var2, s = sigma, scale = np.exp(mu), loc = sigma))
def fs_spfs(var1, mu, sigma):
return (st.lognorm.ppf(var1, s = sigma, scale = np.exp(mu), loc = sigma))
a = 44.0
b = 0.5 #fsolve works for 0.5, 0.9, 0.99 but not for 0.95, incidentally works for 0.950001
c = 1.26
def f(x):
y = np.zeros(2)
y[0] = ((fs_spfs((1-b), x[0], x[1]) - a))
y[1] = (((fs_spfs(0.9, x[0], x[1])/fs_spfs(0.1, x[0], x[1])) - c))
print(y)
return y
x0 = np.array([1., 0.01])
solution = fsolve(f, x0)
print( "(x, y) = (" + str(solution[0]) + ", " + str(solution[1]) + ")")
Results with b = 0.5
b = 0.5
(x, y) = (3.7821340072441982, 0.09035467410258388)
fs_spfs((1-b), solution[0], solution[1]) # expected answer = 44.
43.99999999999982
rod(0.9, 0.1, solution[0], solution[1]) # exptected answer = 1.26
1.2599999999999958
Results with b = 0.9
b = 0.9
(x, y) = (3.8979025451494755, 0.09033430819655046)
fs_spfs((1-b), solution[0], solution[1]) # expected answer = 44.
43.999999999989164
rod(0.9, 0.1, solution[0], solution[1]) # exptected answer = 1.26
1.2600000000001814
Works for b = 0.99 as well, but fails for b = 0.95. Incidentally works for b = 0.950001

Following initial condition seems to be working for majority of common cases:
x0 = np.array([0.000001, 0.0000001])
Works for values till 0.999, still fails for 0.9999.

Propagate calculation result

I'd like to initialize an array b similar to this, but faster:
a = [0.53, 0.66, 0.064, 0.94, 0.44]
b = [0.0]*5
x = 14.3
for i in range(5):
x = b[i] = a[i]*0.1 + x*0.9
Is there something in numpy for that purpose?

An ugly but vectorized numpy solution:
import numpy as np
a = np.array([0.53, 0.66, 0.064, 0.94, 0.44])
x = 14.3
idx = np.arange(a.size)
0.9 ** idx * (0.1 * (a * 0.9 ** (-idx)).cumsum() + x * 0.9)
# array([12.923 , 11.6967 , 10.53343 , 9.574087 , 8.6606783])
Result from for loop:
a = [0.53, 0.66, 0.064, 0.94, 0.44]
b = [0.0]*5
x = 14.3
for i in range(5):
x = b[i] = a[i]*0.1 + x*0.9
b
#[12.923000000000002,
# 11.696700000000003,
# 10.533430000000003,
# 9.574087000000002,
# 8.660678300000002]
This is because:
And components in the result can be vectorized correspondingly, note the sum of a terms in the inner parenthesis is vectorized as cumsum.

Maybe we can break down and find the correlation between each steps
***x0=x0***
***x1=a[0]*0.1 + x0*0.9***
***x2=a[1]*0.1 + x1*0.9=a[1]*0.1 + (a[0]*0.1 + x0*0.9)*0.9***
So xn=an*0.1+an-1*0.1*0.9+...+a0*0.1*0.9**n-1+x0*0.9**n
n=np.array(0.9)**np.arange(len(a))
sum(a[::-1]*n)*0.1+x*(0.9**(len(a)))
Out[338]: 8.6606783
Update output array
np.diag(np.fliplr(((a*n[::-1])*0.1).cumsum()/n[:,None]+x*(0.9**(np.arange(1,len(a)+1)))))[::-1]
Out[472]: array([12.923 , 11.6967 , 10.53343 , 9.574087 , 8.6606783])

Let's rewrite your loop to analyze it more closely:
for i in range(1, 5):
b[i] = a[i]*0.1 + b[i-1]*0.9
This makes it clear that the calculation is recursive. That is, the value of b[i] depends on the value of b[i-1]. This means that you cannot vectorize the calculation. Vectorizing requires that each element of the result vector is independent of all other elements.

Applying if-else one-liner on lists in Python

I have an array y that takes a value of either 0 or 1. Then I have an array yp that takes values between 0 and 1. The two arrays have the same length.
If an entry in y is 1, then I want to append the corresponding yp to a list, otherwise I want to append 1-yp:
y = [1,1,1,0,0]
yp = [0.1, 0.2, 0.3, 0.4, 0.5]
x = []
for idx, i in enumerate(y):
if i:
x.append(yp[idx])
else:
x.append(1-yp[idx])
Is there a shorter way to write this in Python, perhaps without a for-loop?

You can use a list comprehension with zip to iterate over both lists simultaneously:
>>> y = [1,1,1,0,0]
>>> yp = [0.1, 0.2, 0.3, 0.4, 0.5]
>>> [b if a else 1 - b for a, b in zip(y, yp)]
[0.1, 0.2, 0.3, 0.6, 0.5]

Perhaps you are looking for something like this if I understand you correctly?
for idx, i in enumerate(y):
x.append(yp[idx]) if i else x.append(1-yp[idx])
More like a ternary operation approach?
Reference: http://book.pythontips.com/en/latest/ternary_operators.html

If your lists are very long, there's a more efficient way using numpy:
y, yp = map(np.asarray, (y, yp) # convert to an array
x = y * yp + (1 - y) * (1 - yp)
The code above, explained:
a = y * yp: results in an array with the same length as y (and yp) with 1 * yp where y = 1 and 0 where y = 0.
b = (1 - y) * (1 - yp): results in an array with 1-yp where y=0 and 0 where y = 1.
a + b is the pairwise sum and yields your expected result.

You are asking for list comprehension (what you call "one-liner" ??) such as :
y = [1,1,1,0,0]
yp = [0.1, 0.2, 0.3, 0.4, 0.5]
l=[yp[i] if y[i]==1 else 1-yp[i] for i in range(len(y))]
whill give you :
>>> l
[0.1, 0.2, 0.3, 0.6, 0.5]

Python round a float to nearest 0.05 or to multiple of another float

I want to emulate this function. I want to round a floating point number down to the nearest multiple of 0.05 (or generally to the nearest multiple of anything).
I want this:
>>> round_nearest(1.29, 0.05)
1.25
>>> round_nearest(1.30, 0.05)
1.30
I can do this:
import math
def round_nearest(n, r):
return n - math.fmod(n, r)
>>> round_nearest(1.27, 0.05)
1.25 # Correct!
>>> round_nearest(1.30, 0.05)
1.25 # Incorrect! Correct would be 1.30.
The incorrect answer above is presumably due to floating point rounding. I could put some special case check to see if the n is "close enough" to a multiple of r and not do the subtraction, and that would probably work, but is there a better way? Or is this strategy the best option?

You can round down to the nearest multiple of a like this:
def round_down(x, a):
return math.floor(x / a) * a
You can round to the nearest multiple of a like this:
def round_nearest(x, a):
return round(x / a) * a

As Paul wrote:
You can round to the nearest multiple of a like this:
def round_nearest(x, a):
return round(x / a) * a
Works nearly perfectly, but round_nearest(1.39, 0.05) gives 1.4000000000000001.
To avoid it I'll recommend to do:
import math
def round_nearest2(x, a):
return round(round(x / a) * a, -int(math.floor(math.log10(a))))
Which rounds to precision a, and then to number of significant digits, that has your precision a
EDIT
As #Asclepius shown this code has limitation to the first digit in precision (meaning that e.g. if you put 4.3 then rounding is done to closest integer, if you put 0.25 then number is rounded to first decimal digit after all. This can be easily fix by finding how many digits actually precision contains, and rounding to this number after all:
def round_nearest(x, a):
max_frac_digits = 100
for i in range(max_frac_digits):
if round(a, -int(math.floor(math.log10(a))) + i) == a:
frac_digits = -int(math.floor(math.log10(a))) + i
break
return round(round(x / a) * a, frac_digits)
frac_digits is rounded log10 of your precision (nearest number), so it basically shows how many fractional digits should be taken into account (or in case of bigger number - integer digits). So if your precision is 0.25 then frac_digits will be equal to 2, because of 2 fractional digits. If your precision is 40 then frac_digits will be equal to -1, because you need to 'go back' one digit from decimal separator.

The previous answer by Paul fails the test round_down(4.6, 0.2) == 4.6.
This answer has two types of solutions, inexact and exact. They pass all previous tests and more, also with negative numbers. Each approach provides solutions for round_nearest, round_down, and round_up.
As a disclaimer, these solutions require a lot more testing. Where math.isclose is used, its default tolerances apply.
Can you find a failing example?
To devise additional exact solutions, consider this reference.
Using round (inexact)
import math
def round_nearest(num: float, to: float) -> float:
return round(num / to) * to # Credited to Paul H.
def round_down(num: float, to: float) -> float:
nearest = round_nearest(num, to)
if math.isclose(num, nearest): return num
return nearest if nearest < num else nearest - to
def round_up(num: float, to: float) -> float:
nearest = round_nearest(num, to)
if math.isclose(num, nearest): return num
return nearest if nearest > num else nearest + to
# Tests:
rn, rd, ru = round_nearest, round_down, round_up
> rn(1.27, 0.05), rn(1.29, 0.05), rn(1.30, 0.05), rn(1.39, 0.05)
(1.25, 1.3, 1.3, 1.4000000000000001)
> rn(-1.27, 0.05), rn(-1.29, 0.05), rn(-1.30, 0.05), rn(-1.39, 0.05)
(-1.25, -1.3, -1.3, -1.4000000000000001)
> rn(4.4, 0.2), rn(4.5, 0.2), rn(4.6, 0.2)
(4.4, 4.4, 4.6000000000000005)
> rn(-4.4, 0.2), rn(-4.5, 0.2), rn(-4.6, 0.2)
(-4.4, -4.4, -4.6000000000000005)
> rn(82, 4.3)
81.7
> rd(1.27, 0.05), rd(1.29, 0.05), rd(1.30, 0.05)
(1.25, 1.25, 1.3)
> rd(-1.27, 0.05), rd(-1.29, 0.05), rd(-1.30, 0.05)
(-1.3, -1.3, -1.3)
> rd(4.4, 0.2), rd(4.5, 0.2), rd(4.6, 0.2)
(4.4, 4.4, 4.6)
> rd(-4.4, 0.2), rd(-4.5, 0.2), rd(-4.6, 0.2)
(-4.4, -4.6000000000000005, -4.6)
> ru(1.27, 0.05), ru(1.29, 0.05), ru(1.30, 0.05)
(1.3, 1.3, 1.3)
> ru(-1.27, 0.05), ru(-1.29, 0.05), ru(-1.30, 0.05)
(-1.25, -1.25, -1.3)
> ru(4.4, 0.2), ru(4.5, 0.2), ru(4.6, 0.2)
(4.4, 4.6000000000000005, 4.6)
> ru(-4.4, 0.2), ru(-4.5, 0.2), ru(-4.6, 0.2)
(-4.4, -4.4, -4.6)
Using math.fmod (inexact)
import math
def round_down(num: float, to: float) -> float:
if num < 0: return -round_up(-num, to)
mod = math.fmod(num, to)
return num if math.isclose(mod, to) else num - mod
def round_up(num: float, to: float) -> float:
if num < 0: return -round_down(-num, to)
down = round_down(num, to)
return num if num == down else down + to
def round_nearest(num: float, to: float) -> float:
down, up = round_down(num, to), round_up(num, to)
return down if ((num - down) < (up - num)) else up
# Tests:
rd, ru, rn = round_down, round_up, round_nearest
> rd(1.27, 0.05), rd(1.29, 0.05), rd(1.30, 0.05)
(1.25, 1.25, 1.3)
> rd(-1.27, 0.05), rd(-1.29, 0.05), rd(-1.30, 0.05)
(-1.3, -1.3, -1.3)
> rd(4.4, 0.2), rd(4.5, 0.2), rd(4.6, 0.2)
(4.4, 4.4, 4.6)
> rd(-4.4, 0.2), rd(-4.5, 0.2), rd(-4.6, 0.2)
(-4.4, -4.6000000000000005, -4.6)
> ru(1.27, 0.05), ru(1.29, 0.05), ru(1.30, 0.05)
(1.3, 1.3, 1.3)
> ru(-1.27, 0.05), ru(-1.29, 0.05), ru(-1.30, 0.05)
(-1.25, -1.25, -1.3)
> ru(4.4, 0.2), ru(4.5, 0.2), ru(4.6, 0.2)
(4.4, 4.6000000000000005, 4.6)
> ru(-4.4, 0.2), ru(-4.5, 0.2), ru(-4.6, 0.2)
(-4.4, -4.4, -4.6)
> rn(1.27, 0.05), rn(1.29, 0.05), rn(1.30, 0.05), rn(1.39, 0.05)
(1.25, 1.3, 1.3, 1.4000000000000001)
> rn(-1.27, 0.05), rn(-1.29, 0.05), rn(-1.30, 0.05), rn(-1.39, 0.05)
(-1.25, -1.3, -1.3, -1.4000000000000001)
> rn(4.4, 0.2), rn(4.5, 0.2), rn(4.6, 0.2)
(4.4, 4.4, 4.6)
> rn(-4.4, 0.2), rn(-4.5, 0.2), rn(-4.6, 0.2)
(-4.4, -4.4, -4.6)
> rn(82, 4.3)
81.7
Using math.remainder (inexact)
This section implements only round_nearest. For round_down and round_up, use the same exact logic as in the "Using round" section.
def round_nearest(num: float, to: float) -> float:
return num - math.remainder(num, to)
# Tests:
rn = round_nearest
> rn(1.27, 0.05), rn(1.29, 0.05), rn(1.30, 0.05), rn(1.39, 0.05)
(1.25, 1.3, 1.3, 1.4000000000000001)
> rn(-1.27, 0.05), rn(-1.29, 0.05), rn(-1.30, 0.05), rn(-1.39, 0.05)
(-1.25, -1.3, -1.3, -1.4000000000000001)
> rn(4.4, 0.2), rn(4.5, 0.2), rn(4.6, 0.2)
(4.4, 4.4, 4.6000000000000005)
> rn(-4.4, 0.2), rn(-4.5, 0.2), rn(-4.6, 0.2)
(-4.4, -4.4, -4.6000000000000005)
> rn(82, 4.3)
81.7
Using decimal.Decimal (exact)
Note that this is an inefficient solution because it uses str.
from decimal import Decimal
import math
def round_nearest(num: float, to: float) -> float:
num, to = Decimal(str(num)), Decimal(str(to))
return float(round(num / to) * to)
def round_down(num: float, to: float) -> float:
num, to = Decimal(str(num)), Decimal(str(to))
return float(math.floor(num / to) * to)
def round_up(num: float, to: float) -> float:
num, to = Decimal(str(num)), Decimal(str(to))
return float(math.ceil(num / to) * to)
# Tests:
rn, rd, ru = round_nearest, round_down, round_up
> rn(1.27, 0.05), rn(1.29, 0.05), rn(1.30, 0.05), rn(1.39, 0.05)
(1.25, 1.3, 1.3, 1.4)
> rn(-1.27, 0.05), rn(-1.29, 0.05), rn(-1.30, 0.05), rn(-1.39, 0.05)
(-1.25, -1.3, -1.3, -1.4)
> rn(4.4, 0.2), rn(4.5, 0.2), rn(4.6, 0.2)
(4.4, 4.4, 4.6)
> rn(-4.4, 0.2), rn(-4.5, 0.2), rn(-4.6, 0.2)
(-4.4, -4.4, -4.6)
> rn(82, 4.3)
81.7
> rd(1.27, 0.05), rd(1.29, 0.05), rd(1.30, 0.05)
(1.25, 1.25, 1.3)
> rd(-1.27, 0.05), rd(-1.29, 0.05), rd(-1.30, 0.05)
(-1.3, -1.3, -1.3)
> rd(4.4, 0.2), rd(4.5, 0.2), rd(4.6, 0.2)
(4.4, 4.4, 4.6)
> rd(-4.4, 0.2), rd(-4.5, 0.2), rd(-4.6, 0.2)
(-4.4, -4.6, -4.6)
> ru(1.27, 0.05), ru(1.29, 0.05), ru(1.30, 0.05)
(1.3, 1.3, 1.3)
> ru(-1.27, 0.05), ru(-1.29, 0.05), ru(-1.30, 0.05)
(-1.25, -1.25, -1.3)
> ru(4.4, 0.2), ru(4.5, 0.2), ru(4.6, 0.2)
(4.4, 4.6, 4.6)
> ru(-4.4, 0.2), ru(-4.5, 0.2), ru(-4.6, 0.2)
(-4.4, -4.4, -4.6)
Using fractions.Fraction (exact)
Note that this is an inefficient solution because it uses str. Its test results are identical to those in the "Using decimal.Decimal" section. In my benchmarks, approaches using Fraction were much slower than those using Decimal.
from fractions import Fraction
import math
def round_nearest(num: float, to: float) -> float:
num, to = Fraction(str(num)), Fraction(str(to))
return float(round(num / to) * to)
def round_down(num: float, to: float) -> float:
num, to = Fraction(str(num)), Fraction(str(to))
return float(math.floor(num / to) * to)
def round_up(num: float, to: float) -> float:
num, to = Fraction(str(num)), Fraction(str(to))
return float(math.ceil(num / to) * to)

def round_nearest(x, a):
return round(round(x / a) * a, 2)
It is a slightly different variation.