Develop Reference

Problem while looping variables for Scipy Optimization (SLSQP, COBYLA)

I have a constrained optimization problem where I am trying to minimize an objective function of 100+ variables which is of the form
Min F(x) = f(x1) + f(x2) + ... + f(xn)
Subject to functional constraint
(g(x1) + g(x2) + ... + g(xn))/(f(x1) + f(x2) + ... + f(xn)) - constant >= 0
I also have individual bounds for each variable x1, x2, x3...xn
a <= x1 <= b
c <= x2 <= d
...
For this, I wrote a python script, using the scipy.optimize.minimize implementation with constraints and bounds, but I am unable to fulfill my bounds and constraints in the solutions. These are all cases where optimization could converge to a solution (message: success)
Here is a sample of my code:
df is my pandas dataset
B(x) is LogNorm transform based on x and other constants
Values U, c, lb, ub are pre-calculated constant dictionaries for each index in df
import scipy
df = pd.DataFrame(..)
k = set(df.index.values) ## list of indexes to iterate on
val = 0.25 ## Arbitrary
def obj(x):
fn = 0
for n,i in enumerate(k):
x0 = x[n]
fn1 = (U[i]) * B(x0) * (x0)
fn += fn1
return fn
def cons(x):
cn = 1
c1 = 0
c2 = 0
for n,i in enumerate(k):
x0 = x[n]
c1 += (U[i]) * (B(x0) * (x0 - c[i])
c2 += (U[i]) * (B(x0) * (x0)
cn = c1/(c2)
return cn - val
const = [{'type':'ineq', 'fun':cons}]
bnds = tuple((lb[i], ub[i]) for i in k) ## Lower, Upper for each element ((lb1, ub1), (lb2, ub2)...)
x_init = [lb[i] for i in k] ## for eg. starting from lower bound
## Solution
sol = scipy.optimize.minimize(obj, x_init, method = 'COBYLA', bounds = bnds, constraints = const)
I have more pointed questions if that helps:
Is there a way to construct the same equation concisely/ without the use of loops (given the number of variables could depend on input data and I have no control over it)?
Is there any noticeable issue in my application of bounds? I can't seem to get the final values of all variables follow individual bounds.
Similarly, is there a visible flaw in the construction on constraint equation? My results often DO NOT follow the constraints is repeated runs with different inputs.
Any help with either of the questions can help me progress further at work.
I have also looked into a Lagrangian solution of the same but so far I am unable to solve it for undefined number of (n) variables.
Thanks!

cvxpy - How to obtain the variable value after each iteration?

I am using cvxpy to solve a second order cone program. I have used the boilerplate code as mentioned in the cvxpy website - cvxpy SOCP webpage. I do not know how to obtain the variable value after each iteration...
Code from the link:
# Import packages.
import cvxpy as cp
import numpy as np
# Generate a random feasible SOCP.
m = 3
n = 10
p = 5
n_i = 5
f = np.random.randn(n)
A = []
b = []
c = []
d = []
x0 = np.random.randn(n)
for i in range(m):
A.append(np.random.randn(n_i, n))
b.append(np.random.randn(n_i))
c.append(np.random.randn(n))
d.append(np.linalg.norm(A[i] # x0 + b, 2) - c[i].T # x0)
# Define and solve the CVXPY problem.
x = cp.Variable(n)
# We use cp.SOC(t, x) to create the SOC constraint ||x||_2 <= t.
soc_constraints = [
cp.SOC(c[i].T # x + d[i], A[i] # x + b[i]) for i in range(m)
]
prob = cp.Problem(cp.Minimize(f.T#x), soc_constraints)
prob.solve()
# Print result.
print("The optimal value is", prob.value)
print("A solution x is")
print(x.value)
x.value here only gives the variable value after all the iterations are done. I want x.value after each iteration.

CVXPY numerical instability

I noticed that if you add to your target something multiplied by zero it will affect the solution significantly. I found this bug when using cvxpy on my dataset which I cant upload of course. But here is an example from their official resource: https://www.cvxpy.org/examples/basic/sdp.html
import cvxpy as cp
import numpy as np
# Generate a random SDP.
n = 3
p = 3
np.random.seed(1)
C = np.random.randn(n, n)
A = []
b = []
for i in range(p):
A.append(np.random.randn(n, n))
b.append(np.random.randn())
# Define and solve the CVXPY problem.
# Create a symmetric matrix variable.
X = cp.Variable((n,n), symmetric=True)
# The operator >> denotes matrix inequality.
constraints = [X >> 0]
constraints += [
cp.trace(A[i] # X) == b[i] for i in range(p)]
prob = cp.Problem(cp.Minimize(cp.trace(C # X)),
constraints)
prob.solve()
# Print result.
print("The optimal value is", prob.value)
print("A solution X is")
print(X.value)
Now do the same but change
prob = cp.Problem(cp.Minimize(cp.trace(C # X) + 0*cp.trace(C # X)**9),
constraints)
You will see that value of X will change.

multi variable optimization with cvxpy

X is T by m matrix (Given Matrix)
B is T by n matrix (Variable 1)
A is n by m matrix (Variable 2)
I want to minimize ||X-A*B|| forbinious norm and find A and B for that using Python / cvxpy
I did this on Matlab and works fine
Discriminative disaggregation sparse coding for energy disaggregation algorithm was successfully implemented on Matlab but difficult to use for large sample sets, So need to implement it on python
import cvxpy as cp
import numpy as np
n = 5
m = 4
T = 3
np.random.seed(1)
A = cp.Variable((n, m))
B = cp.Variable((T, n))
x = np.random.rand(T, m)
constraints = [A >= 0,
B >= 0]
obj = cp.Minimize(cp.norm(x - cp.matmul(B,A),"fro"))
prob = cp.Problem(obj,constraints)
prob.solve()
Need to use cvxpy or any other tool on python to minimize multi objective function

From CVX to CVXPY or CVXOPT

I've been trying to pass some code from Matlab to Python. I have the same convex optimization problem working on Matlab but I'm having problems passing it to either CVXPY or CVXOPT.
n = 1000;
i = 20;
y = rand(n,1);
A = rand(n,i);
cvx_begin
variable x(n);
variable lambda(i);
minimize(sum_square(x-y));
subject to
x == A*lambda;
lambda >= zeros(i,1);
lambda'*ones(i,1) == 1;
cvx_end
This is what I tried with Python and CVXPY.
import numpy as np
from cvxpy import *
# Problem data.
n = 100
i = 20
np.random.seed(1)
y = np.random.randn(n)
A = np.random.randn(n, i)
# Construct the problem.
x = Variable(n)
lmbd = Variable(i)
objective = Minimize(sum_squares(x - y))
constraints = [x == np.dot(A, lmbd),
lmbd <= np.zeros(itr),
np.sum(lmbd) == 1]
prob = Problem(objective, constraints)
print("status:", prob.status)
print("optimal value", prob.value)
Nonetheless, it's not working. Does any of you have any idea how to make it work? I'm pretty sure my problem is in the constraints. And also it would be nice to have it with CVXOPT.

I think I got it, I had one of the constraints wrong =), I added a random seed number in order to compare the results and check that are in fact the same in both languages. I leave the data here so maybe this is useful for somebody someday ;)
Matlab
rand('twister', 0);
n = 100;
i = 20;
y = rand(n,1);
A = rand(n,i);
cvx_begin
variable x(n);
variable lmbd(i);
minimize(sum_square(x-y));
subject to
x == A*lmbd;
lmbd >= zeros(i,1);
lmbd'*ones(i,1) == 1;
cvx_end
CVXPY
import numpy as np
import cvxpy as cp
# random seed
np.random.seed(0)
# Problem data.
n = 100
i = 20
y = np.random.rand(n)
# A = np.random.rand(n, i) # normal
A = np.random.rand(i, n).T # in this order to test random numbers
# Construct the problem.
x = cp.Variable(n)
lmbd = cp.Variable(i)
objective = cp.Minimize(cp.sum_squares(x - y))
constraints = [x == A*lmbd,
lmbd >= np.zeros(i),
cp.sum(lmbd) == 1]
prob = cp.Problem(objective, constraints)
result = prob.solve(verbose=True)
CVXOPT is pending.....