Develop Reference

Pyomo: using a list for parametrising a constraint

I am trying to declare a constraint in Pyomo where one parameter would be a list instead of a scalar and then build the constraint by providing a set of the right dimension but it seems that Pyomo does the Cartesian product of every set and deduces the number of inputs (which are then considered as scalars).
Below a dummy example of what I want to achieve:
model.inputs = RangeSet(0,1)
model.x = model.inputs*model.inputs
model.p = Var()
def constraint_rule(model,x,i):
return x[i] > model.p
model.constraint = Constraint(model.x,model.inputs,rule=constraint_rule)
To be more precise on what I want to achieve. My constraint is of the form:
f(x_0,x_1,i) > p
And I'd like to input x as a vector instead of inputting separately x_0 and x_1 (or more if I have more x_i). So I want to input a list of lists for the first parameter and a iterator as the second parameter which could specify which element of the list I want.
I can of course decompose the list x of lenght n with n scalars x[i] but because I want to keep changing the size of the inputs, I wanted to only change model.x and hope it would automatically scale.
Below the full mathematical problem (I don't have enough reputation to put an image, sorry about that):
Tr(MixiNx) > p
Tr(Nx) = 1
Mi0 + Mi1 = Id
Mi0, Mi1 and Nx SDP
Here the Ms and Ns are 2x2 matrices.
We have 4 N for bitstrings of lenght 2 and 2 matrices M per value of i (then xi is either 0 or 1).
Also x specifies the bitstring (here we have only x_0 and x_1) and i specifies which bit (for instance for i=0, we want the value x_0 ie the first bit of x). But i could be larger than the number of the number of bits in x (for instance we could set that for i=2, we want the value x_0 xor x_1 ie the parity of the bits). So I wanted to encode the 1sr constraint such it receives a bitstring x and a value i which could specify which information I want about that bitstring.
I hope that's clearer.

Coding an iterated sum of sums in python

For alpha and k fixed integers with i < k also fixed, I am trying to encode a sum of the form
where all the x and y variables are known beforehand. (this is essentially the alpha coordinate of a big iterated matrix-vector multiplication)
For a normal sum varying over one index I usually create a 1d array A and set A[i] equal to the i indexed entry of the sum then use sum(A), but in the above instance the entries of the innermost sum depend on the indices in the previous sum, which in turn depend on the indices in the sum before that, all the way back out to the first sum which prevents me using this tact in a straightforward manner.
I tried making a 2D array B of appropriate length and width and setting the 0 row to be the entries in the innermost sum, then the 1 row as the entries in the next sum times sum(np.transpose(B),0) and so on, but the value of the first sum (of row 0) needs to vary with each entry in row 1 since that sum still has indices dependent on our position in row 1, so on and so forth all the way up to sum k-i.
A sum which allows for a 'variable' filled in by each position of the array it's summing through would thusly do the trick, but I can't find anything along these lines in numpy and my attempts to hack one together have thus far failed -- my intuition says there is a solution that involves summing along the axes of a k-i dimensional array, but I haven't been able to make this precise yet. Any assistance is greatly appreciated.

One simple attempt to hard-code something like this would be:
for j0 in range(0,n0):
for j1 in range(0,n1):
....
Edit: (a vectorized version)
You could do something like this: (I didn't test it)
temp = np.ones(n[k-i])
for j in range(0,k-i):
temp = x[:n[k-i-1-j],:n[k-i-j]].T#(y[:n[k-i-j]]*temp)
result = x[alpha,:n[0]]#(y[:n[0]]*temp)
The basic idea is that you try to press it into a matrix-vector form. (note that this is python3 syntax)
Edit: You should note that you need to change the "k-1" to where the innermost sum is (I just did it for all sums up to index k-i)

This is 95% identical to #sehigle's answer, but includes a generic N vector:
def nested_sum(XX, Y, N, alpha):
intermediate = np.ones(N[-1], dtype=XX.dtype)
for n1, n2 in zip(N[-2::-1], N[:0:-1]):
intermediate = np.sum(XX[:n1, :n2] * Y[:n2] * intermediate, axis=1)
return np.sum(XX[alpha, :N[0]] * Y[:N[0]] * intermediate)
Similarly, I have no knowledge of the expression, so I'm not sure how to build appropriate tests. But it runs :\

Python | Eigenvector components corresponding to their original rows and columns

I hope this isn't a bad question, but I am computing eigenvectors and eigenvalues of a symmetric matrix, lets call it A.
A is say 10x10.
I want to get the eigenvector corresponding to its 2nd largest eigenvalue. I do
[D, V] = scipy.sparse.linalg.eigs(L, 2)
s = V[:, 0] # this is the 2nd eigenvecotr
Now, I want to do some calculations based on comparing certain elements in 's' to some other number. For example:
for i in range(0, len(s)):
if s[0] > some number:
do something with the first column/row (if this was a graph, the first node) in A
My questions are:
Does it matter if I use scipy.sparse.linalg.eigs, or scipy.sparse.linalg.eigsh? I saw documentation online for both, but can't discern the difference.
Do the elements in the eigenvector correspond to their original rows/columns? For example s[0] corresponds to the eigenvector of first column/row of A?

The docs are pretty clear on the row/column question:
The column v[:, i] is the eigenvector corresponding to the eigenvalue w[i]
The difference between eigs and eigsh is that the latter assumes a symmetric/hermitian matrix.
Finally, for a 10x10 matrix you don't need anything sparse, you're better off just using functions from scipy.linalg.

Python fsolve ValueError

Why does the following code return a ValueError?
from scipy.optimize import fsolve
import numpy as np
def f(p,a=0):
x,y = p
return (np.dot(x,y)-a,np.outer(x,y)-np.ones((3,3)),x+y-np.array([1,2,3]))
x,y = fsolve(f,(np.ones(3),np.ones(3)),9)
ValueError: setting an array element with a sequence.

The basic problem here is that your function f does not satisfy the criteria required for fsolve to work. These criteria are described in the documentation - although arguably not very clearly.
The particular things that you need to be aware of are:
the input to the function that will be solved for must be an n-dimensional vector (referred to in the docs as ndarray), such that the value of x you want is the solution to f(x, *args) = 0.
the output of f must be the same shape as the x input to f.
Currently, your function takes a 2 member tuple of 1x3-arrays (in p) and a fixed scalar offset (in a). It returns a 3 member tuple of types (scalar,3x3 array, 1x3 array)
As you can see, neither condition 1 nor 2 is met.
It is hard to advise you on exactly how to fix this without being exactly sure of the equation you are trying to solve. It seems you are trying to solve some particular equation f(x,y,a) = 0 for x and y with x0 = (1,1,1) and y0 = (1,1,1) and a = 9 as a fixed value. You might be able to do this by passing in x and y concatenated (e.g. pass in p0 = (1,1,1,1,1,1) and in the function use x=p[:3] and y = p[3:] but then you must modify your function to output x and y concatenated into a 6-dimensional vector similarly. This depends on the exact function your are solving for and I can't work this out from the output of your existing f (i.e based on a dot product, outer product and sum based tuple).
Note that arguments that you don't pass in the vector (e.g. a in your case) will be treated as fixed values and won't be varied as part of the optimisation or returned as part of any solution.
Note for those who like the full story...
As the docs say:
fsolve is a wrapper around MINPACK’s hybrd and hybrj algorithms.
If we look at the MINPACK hybrd documentation, the conditions for the input and output vectors are more clearly stated. See the relevant bits below (I've cut some stuff out for clarity - indicated with ... - and added the comment to show that the input and output must be the same shape - indicated with <--)
1 Purpose.
The purpose of HYBRD is to find a zero of a system of N non-
linear functions in N variables by a modification of the Powell
hybrid method. The user must provide a subroutine which calcu-
lates the functions. The Jacobian is then calculated by a for-
ward-difference approximation.
2 Subroutine and type statements.
SUBROUTINE HYBRD(FCN,N,X, ...
...
FCN is the name of the user-supplied subroutine which calculates
the functions. FCN must be declared in an EXTERNAL statement
in the user calling program, and should be written as follows.
SUBROUTINE FCN(N,X,FVEC,IFLAG)
INTEGER N,IFLAG
DOUBLE PRECISION X(N),FVEC(N) <-- input X is an array length N, so is output FVEC
----------
CALCULATE THE FUNCTIONS AT X AND
RETURN THIS VECTOR IN FVEC.
----------
RETURN
END
N is a positive integer input variable set to the number of
functions and variables.
X is an array of length N. On input X must contain an initial
estimate of the solution vector. On output X contains the
final estimate of the solution vector.

Scipy sparse triangular matrix?

I am using Scipy to construct a large, sparse (250k X 250k) co-occurrence matrix using scipy.sparse.lil_matrix. Co-occurrence matrices are triangular; that is, M[i,j] == M[j,i]. Since it would be highly inefficient (and in my case, impossible) to store all the data twice, I'm currently storing data at the coordinate (i,j) where i is always smaller than j. So in other words, I have a value stored at (2,3) and no value stored at (3,2), even though (3,2) in my model should be equal to (2,3). (See the matrix below for an example)
My problem is that I need to be able to randomly extract the data corresponding to a given index, but, at least the way, I'm currently doing it, half the data is in the row and half is in the column, like so:
M =
[1 2 3 4
0 5 6 7
0 0 8 9
0 0 0 10]
So, given the above matrix, I want to be able to do a query like M[1], and get back [2,5,6,7]. I have two questions:
1) Is there a more efficient (preferably built-in) way to do this than first querying the row, and then the column, and then concatenating the two? This is bad because whether I use CSC (column-based) or CSR (row-based) internal representation, one of the two queries is highly inefficient.
2) Am I even using the right part of Scipy? I have seen a few functions in the Scipy library that mention triangular matrices, but they seem to revolve around getting triangular matrices from a full matrix. In my case, (I think) I already have a triangular matrix, and want to manipulate it.
Many thanks.

I would say that you can't have the cake and eat it too: if you want efficient storage, you cannot store full rows (as you say); if you want efficient row access, I'd say that you have to store full rows.
While real performances depend on your application, you could check whether the following approach works for you:
You use Scipy's sparse matrices for efficient storage.
You automatically symmetrize your matrix (there is a small recipe on StackOverflow, that works at least on regular matrices).
You can then access its rows (or columns); whether this is efficient depends on the implementation of sparse matrices…