I have a application written in python for calculating minimal return value from a function. Am using scipy.optimize.mminimize with SLSQP as the optimization method.
It runs in a loop and for saving time and kip it from just finding local minima I need it to use the x0 that I provide it.
The problem seems to be that it dos not care what x0 i give it. It just starts optimizing at random values. What do I do wrong?
I have written a smal test application to test x0 on the minimizer:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize
global log
log = []
counter = 0
def callback(x):
global counter
counter += 1
log.append(x)
print('u_guessx',x)
return True
def objectivefunction(x, *arg):
SUM = 2*x[0]**3 + 3*(3-x[0])**2 - 5*x[2]**1 + 50
return SUM
# Defining Initial Conditions
u_guess = np.array([0 for u in range(3)])
#u_guess = np.zeros(4)
print("u shape: ",u_guess.shape)
print("u_init: ",u_guess)
#Simulation loop:
bounds_u = [(0,20) for i in u_guess]
# Run Optimizer
solution_guess = minimize(objectivefunction,
u_guess,
method = 'SLSQP',
callback = callback,
bounds=bounds_u,
options={'ftol': 1e-9, 'disp': True},
)
u_guess = solution_guess.x
u_opt = u_guess.item(0)
print("type(solution_guess.x): ",type(solution_guess.x))
print("u_opt: ",u_opt)
print("solution_guess.x: ",solution_guess.x)
#print("log: ",log)
print("counter: ",counter )
First of all, isn't your objectivefunction() wrong ? I suspect it should be
def objectivefunction(x, *arg):
SUM = 2*x[0]**3 + 3*(3-x[1])**2 - 5*x[2]**1 + 50
return SUM
In the original function, x[1] is not used and therefore the algorithm is insensitive to x[1].
Secondly, the function callback() is called after each iteration. Therefore the first output is not the initial condition, but the first guess for the minima based on your initial conditions. If I run the corrected program and change the initial condition, it output different guesses. But for different runs with the same initial conditions, it always output the same guesses. There is no randomness (assuming I use corrected version of objectivefunction).
Related
I do have a piece of code that compute partitions of a set of (potentialy duplicated) integers. But i am interested in the set of possible partition and there multiplicity.
You can for exemple launch the follwoing code :
import numpy as np
from collections import Counter
import pandas as pd
def _B(i):
# for a given multiindex i, we defined _B(i) as the set of integers containg i_j times the number j:
if len(i) != 1:
B = []
for j in range(len(i)):
B.extend(i[j]*[j])
else:
B = i*[0]
return B
def _partition(collection):
# from here: https://stackoverflow.com/a/62532969/8425270
if len(collection) == 1:
yield (collection,)
return
first = collection[0]
for smaller in _partition(collection[1:]):
# insert `first` in each of the subpartition's subsets
for n, subset in enumerate(smaller):
yield smaller[:n] + ((first,) + subset,) + smaller[n + 1 :]
# put `first` in its own subset
yield ((first,),) + smaller
def to_list(tpl):
# the final hierarchy is
return list(list(i) if isinstance(i, tuple) else i for i in tpl)
def _Pi(inst_B):
# inst_B must be a tuple
if type(inst_B) != tuple :
inst_B = tuple(inst_B)
pp = [tuple(sorted(p)) for p in _partition(inst_B)]
c = Counter(pp)
Pi = c.keys()
N = list()
for pi in Pi:
N.append(c[pi])
Pi = [to_list(pi) for pi in Pi]
return Pi, N
if __name__ == "__main__":
import cProfile
pr = cProfile.Profile()
pr.enable()
sh = (3, 3, 3)
rez = list()
rez_sorted= list()
rez_ref = list()
for idx in np.ndindex(sh):
if sum(idx) > 0:
print(idx)
Pi, N = _Pi(_B(idx))
print(pd.DataFrame({'Pi': Pi, 'N': N * np.array([np.math.factorial(len(pi) - 1) for pi in Pi])}))
pr.disable()
# after your program ends
pr.print_stats(sort="tottime")
This code computes, for several examples of tuples of integer numbers (generated by np.ndindex) the partitions and counts i need. Everything happens in the _partition and the _Pi functions, this is were you should look at.
If you look closely at how these two functions are working, you'll see that they comput eevery potential partition and THEN count up how many times they appeared. For small problems, this is fine, but if the size of the prolbme increase, this starts to take a looooot of time. Try setting sh = (5,5,5), you'll see what i mean;
So the problem is the following :
Is there a way to compute directly the partitions and there number of occurences instead ?
Edit: I cross-posted on mathoverflow there, and they propose a solution in this article, in corrolary 2.10 (page 10 of the pdf). The problem could be solved by implmenting the sets p(v,r) in this corrolary.
I was hoping, as in the univariate case, that those sets would have a nice recursive expression but i ould not find one yet.
More Edit : This problem is equivalent to finding all (multiset)-partitions of a multiset. If the solution for finding (set)-partitions of a set is given by Bell partial polynomials, here we need multivariate version of these polynomials.
I wrote a code that works great for solving equations numerically, but there is I a specific equation that when I get in there and try running the code - it will run and no output will ever come out!
Equation I got an output for: x^3−3*x+2−a*(np.sin(x))
Equation I didn't get an output for: (x-1)(x-2)(x-3)-a*(np.cos(x))
I also tried writing the second equation without brackets, like this: x^3-6*x^2+11*x-6-a*np.cos(x)
and it didn't help. where is the problem?!
this is my code:
import math
import numpy as np
h =1e-5
eps =1e-8
#function of the equation
def nf(x,a,c):
c=math.cos(x)
solu=(x-1)*(x-2)*(x-3)-a*c
return(solu)
#numerical method
def sl(a,x):
c=math.cos(x)
f = nf(x,a,c)
while abs(f)>eps:
x = x - h*f/(nf(x+h,a,c)-f)
f = nf(x,a,c)
return(x)
N = 101
mya = np.linspace(0.0,1.0,N)
myb = np.zeros(mya.shape)
myc = np.zeros(mya.shape)
myd = np.zeros(mya.shape)
for i in range(0,N):
myb[i] = sl(mya[i],1.0)
myc[i] = sl(mya[i],2.0)
myd[i] = sl(mya[i],3.0)
print(myb[i])
print(myc[i])
print(myd[i])
The problem is that for some input to sl, abs(f)>eps might not ever become False, creating an infinite loop. I havn't investigated the mathematical problem of yours, so I can't solve this problem "for real". What I can provide is automatic detection of when this happens, so that the code returns without a result rather than looping forever.
def sl(a,x):
c=math.cos(x)
f = nf(x,a,c)
count, maxcount = 0, 1000
while abs(f)>eps:
x = x - h*f/(nf(x+h,a,c)-f)
f = nf(x,a,c)
count += 1
if count > maxcount:
return
return(x)
Here, a maximum of 1000 iterations is allowed before a solution is deemed unreachable. In such a case, sl returns None, which when inserted into your NumPy float arrays becomes np.nan.
Upon investigating the output, only myc[60] fails in this way.
Your nf function was a little weird. You were passing c = math.cos(x) into nf() but whithin nf() you tried to assign c to math.cos(x) again. Just use the value c what you passed. Commenting it out fixes your code. As for the mathematical correctness, I cannot determine that unless you provide a better explanation what youre trying to do.
import math
import numpy as np
h =1e-5
eps =1e-8
#function of the equation
def nf(x,a,c):
# this line is not needed. Commenting allows your code to run
# c=math.cos(x)
solu=(x-1)*(x-2)*(x-3)-a*c
return(solu)
#numerical method
def sl(a,x):
c = math.cos(x)
f = nf(x,a,c)
while abs(f)>eps:
x = x - h*f/(nf(x+h,a,c)-f)
f = nf(x,a,c)
return(x)
N = 101
mya = np.linspace(0.0,1.0,N)
myb = np.zeros(mya.shape)
myc = np.zeros(mya.shape)
myd = np.zeros(mya.shape)
for i in range(0,N):
myb[i] = sl(mya[i],1.0)
myc[i] = sl(mya[i],2.0)
myd[i] = sl(mya[i],3.0)
print(myb[i])
print(myc[i])
print(myd[i])
Output:
3.2036907284
0.835006605064
0.677633820877
I am currently running test_matrix_speed() to see how fast my search_and_book_availability function is. Using the PyCharm profiler I can see that each search_and_book_availability function call averages a speed of 0.001ms. Having the Numba #jit(nopython=True) decorator makes no difference to the performance of this function. Is this because there are no improvements to be had and Numpy is operating as fast as possible here? (I don't care about the speed of the generate_searches function)
Here's the code I'm running
import random
import numpy as np
from numba import jit
def generate_searches(number, sim_start, sim_end):
searches = []
for i in range(number):
start_slot = random.randint(sim_start, sim_end - 1)
end_slot = random.randint(start_slot + 1, sim_end)
searches.append((start_slot, end_slot))
return searches
#jit(nopython=True)
def search_and_book_availability(matrix, search_start, search_end):
search_slice = matrix[:, search_start:search_end]
output = np.where(np.sum(search_slice, axis=1) == 0)[0]
number_of_bookable_vecs = output.size
if number_of_bookable_vecs > 0:
if number_of_bookable_vecs == 1:
id_to_book = output[0]
else:
id_to_book = np.random.choice(output)
matrix[id_to_book, search_start:search_end] = 1
return True
else:
return False
def test_matrix_speed():
shape = (10, 1440)
matrix = np.zeros(shape)
sim_start = 0
sim_end = 1440
searches = generate_searches(1000000, sim_start, sim_end)
for i in searches:
search_start = i[0]
search_end = i[1]
availability = search_and_book_availability(matrix, search_start, search_end)
Using your function and the following code to profile the speed
import time
shape = (10, 1440)
matrix = np.zeros(shape)
sim_start = 0
sim_end = 1440
searches = generate_searches(1000000, sim_start, sim_end)
def reset():
matrix[:] = 0
def test_matrix_speed():
for i in searches:
search_start = i[0]
search_end = i[1]
availability = search_and_book_availability(matrix, search_start, search_end)
def timeit(func):
# warmup
reset()
func()
reset()
start = time.time()
func()
end = time.time()
return end - start
print(timeit(test_matrix_speed))
I find on the order of 11.5s for jited version and 7.5s without jit. I am no expert on numba, but what it is made for is optimizing numerical code written in non-vectorized way, in particular explicit for loops. In your code there is none, you only use vectorized operations. Therefore I expected jit to not outperform baseline solution, though I must admit that I am surprised to see it that much worse. If you're looking to optimize your solution, you can cut the execution time (at least on my PC) with the following code:
def search_and_book_availability_opt(matrix, search_start, search_end):
search_slice = matrix[:, search_start:search_end]
# we don't need to sum in order to check if all elements are 0.
# ndarray.any() can use short-circuiting and is therefore faster.
# Also, we don't need the selected values from np.where, only the
# indexes, so np.nonzero is faster
bookable, = np.nonzero(~search_slice.any(axis=1))
# short circuit
if bookable.size == 0:
return False
# we can perform random choice even if size is 1
id_to_book = np.random.choice(bookable)
matrix[id_to_book, search_start:search_end] = 1
return True
and by initializing matrix as np.zeros(shape, dtype=np.bool), instead of the default float64. I am able to get execution times of around 3.8s, a ~50% improvement over your unjited solution and ~70% improvement over the jited version. Hope that helps.
I have a weird problem with iterators, which I can't figure out. I have a complicated numerical routine returning a generator object (or after some changes to the code an islice). Afterwards I check, the results as I know that the results must have a negative imaginary part:
import numpy as np
threshold = 1e-8 # just check up to some numerical accuracy
results = result_generator(**inputs)
is_valid = [np.all(_result.imag < threshold) for _result in results]
print("Number of valid results: ", is_valid.count(True))
(Sorry for not giving an executable code, but I can't come up with a simple code at the moment.)
The problem is now, that this returns one valid solution. If I change the code to
import numpy as np
threshold = 1e-8 # just check up to some numerical accuracy
results = list(result_generator(**inputs))
is_valid = [np.all(_result.imag < threshold) for _result in results]
print("Number of valid results: ", is_valid.count(True))
using a list instead of a generator, I get zero valid solution. I can however not wrap my head around what is different and thus have no idea how to debug the problem.
If I go through the debugger and print out the result with the corresponding index the results are even different, the one of the generator is correct, the one of the list is wrong.
Here the numerical function:
def result_generator(z, iw, coeff, n_min, n_max):
assert n_min >= 1
assert n_min < n_max
if n_min % 2:
# index must be even
n_min += 1
id1 = np.ones_like(z, dtype=complex)
A0, A1 = 0.*id1, coeff[0]*id1
A2 = coeff[0] * id1
B2 = 1. * id1
multiplier = np.subtract.outer(z, iw[:-1])*coeff[1:]
multiplier = np.moveaxis(multiplier, -1, 0).copy()
def _iteration(multiplier_im):
multiplier_im = multiplier_im/B2
A2[:] = A1 + multiplier_im*A0
B2[:] = 1. + multiplier_im
A0[:] = A1
A1[:] = A2 / B2
return A1
complete_iterations = (_iteration(multiplier_im) for multiplier_im in multiplier)
return islice(complete_iterations, n_min, n_max, 2)
You're yielding the same array over and over instead of making new arrays. When you call list, you get a list of references to the same array, and that array is in its final state. When you don't call list, you examine the array in the state the generator yields it, each time it's yielded.
Stop reusing the same array over and over.
I'm new to Python and looking to find an optimized solution with a number of constraints, where those constraints are based on functions of the outputs.
First two constraints are straightforward:
1) output1 +output2 + output3 = 1
2) output1, output2, and output3 must all be >= 0
Last constraint needs functions of the outputs to be EQUAL:
3) f(output1) == f(output2) == f(output3)
In this case the function array is produced a matrix multiplication of the outputs:
F = cov.dot(array([output1,output2,output3]))*array([output1,output2,output3])
f(output1) = F[0], f(output2) = F[1], f(output3) = F [2]
Hopefully I've described the problem clearly... Eventually I want to extend this to more outputs than 3.
What I have below gives me output values that don't appear to follow the constraints at all (gives me a negative value). I assume I'm entering the constraints wrong... or perhaps there is an easier way to do this with np.linalg.solve?
import numpy as np
from scipy.optimize import fsolve
cov=np.array([0.04,0.0015,0.03,0.0015,0.0025,0.000625,0.03,0.000625,0.0625]).reshape(3,3)
weights=np.array([0.3,0.2,0.5])
def RC(w):
return cov.dot(w)*w
riskcont = RC(weights)
def PV(riskcont):
return np.sqrt(riskcont.sum())
portvol = PV(riskcont)
def ERC(z):
w1=z[0]
w2=z[1]
w3=z[2]
#1) weights sum to 100%
out=w1 +w2 +w3 -1
#2) weights above zero
out.append((w1*w2*w3)>0)
#3) riskcont must all be equal
out.append([riskcont[0] == riskcont[1] == riskcont[2]] #== riskcont(w4)))
return out
z= fsolve(ERC,[1/3,1/3,1/3])