Related
If I have a (real) hermitian matrix, for instance
H = matrix([[-2. , 0.5, 0.5, 0. ],
[ 0.5, 2. , 0. , 0.5],
[ 0.5, 0. , 0. , 0.5],
[ 0. , 0.5, 0.5, 0. ]])
(This matrix is hermitian; it is the Hamiltonian of a 2-spin Ising chain with coupling to an external field.)
Then there exists a special orthogonal transformation O (preserves length of column and row vectors of a matrix) s.t.
H = O.transpose() # D # O
where D is diagonal. For the matrix exponential this leads to
T = expm(1j * H) = O.transpose() # expm(1j * D) # O
so all column/row vectors of T must have length 1.
If I use scipy.linalg.expm this property is violated:
In [1]: import numpy as np
In [2]: from numpy import matrix
In [3]: from scipy.linalg import expm
In [4]: H = matrix([[-2. , 0.5, 0.5, 0. ],
...: [ 0.5, 2. , 0. , 0.5],
...: [ 0.5, 0. , 0. , 0.5],
...: [ 0. , 0.5, 0.5, 0. ]])
In [5]: T = expm(1j * H)
In [6]: np.sum(np.abs(T[0]))
Out[6]: 1.6099093263121051
In [7]: np.sum(np.abs(T[1]))
Out[7]: 1.609909326312105
In [8]: np.sum(np.abs(T[2]))
Out[8]: 1.7770244703003222
In [9]: np.sum(np.abs(T[3]))
Out[9]: 1.7770244703003222
Is this a bug in expm or am I making a mistake here?
you're using the wrong norm. Use
np.sqrt( np.sum( np.abs(T[0])**2 ) )
Or even in a shorter way
np.linalg.norm( T[0] )
I have an unknown number n of variables that can range from 0 to 1 with some known step s, with the condition that they sum up to 1. I want to create a matrix of all combinations. For example, if n=3 and s=0.33333 then the grid will be (The order is not important):
0.00, 0.00, 1.00
0.00, 0.33, 0.67
0.00, 0.67, 0.33
0.00, 1.00, 0.00
0.33, 0.00, 0.67
0.33, 0.33, 0.33
0.33, 0.67, 0.00
0.67, 0.00, 0.33
0.67, 0.33, 0.00
1.00, 0.00, 0.00
How can I do that for an arbitrary n?
Here is a direct method using itertools.combinations:
>>> import itertools as it
>>> import numpy as np
>>>
>>> # k is 1/s
>>> n, k = 3, 3
>>>
>>> combs = np.array((*it.combinations(range(n+k-1), n-1),), int)
>>> (np.diff(np.c_[np.full((len(combs),), -1), combs, np.full((len(combs),), n+k-1)]) - 1) / k
array([[0. , 0. , 1. ],
[0. , 0.33333333, 0.66666667],
[0. , 0.66666667, 0.33333333],
[0. , 1. , 0. ],
[0.33333333, 0. , 0.66666667],
[0.33333333, 0.33333333, 0.33333333],
[0.33333333, 0.66666667, 0. ],
[0.66666667, 0. , 0.33333333],
[0.66666667, 0.33333333, 0. ],
[1. , 0. , 0. ]])
If speed is a concern, itertools.combinations can be replaced by a numpy implementation.
EDIT
Here is a better solution. It basically partitions the number of steps into the amount of variables to generate all the valid combinations:
def partitions(n, k):
if n < 0:
return -partitions(-n, k)
if k <= 0:
raise ValueError('Number of partitions must be positive')
if k == 1:
return np.array([[n]])
ranges = np.array([np.arange(i + 1) for i in range(n + 1)])
parts = ranges[-1].reshape((-1, 1))
s = ranges[-1]
for _ in range(1, k - 1):
d = n - s
new_col = np.concatenate(ranges[d])
parts = np.repeat(parts, d + 1, axis=0)
s = np.repeat(s, d + 1) + new_col
parts = np.append(parts, new_col.reshape((-1, 1)), axis=1)
return np.append(parts, (n - s).reshape((-1, 1)), axis=1)
def make_grid_part(n, step):
num_steps = round(1.0 / step)
return partitions(num_steps, n) / float(num_steps)
print(make_grid_part(3, 0.33333))
Output:
array([[ 0. , 0. , 1. ],
[ 0. , 0.33333333, 0.66666667],
[ 0. , 0.66666667, 0.33333333],
[ 0. , 1. , 0. ],
[ 0.33333333, 0. , 0.66666667],
[ 0.33333333, 0.33333333, 0.33333333],
[ 0.33333333, 0.66666667, 0. ],
[ 0.66666667, 0. , 0.33333333],
[ 0.66666667, 0.33333333, 0. ],
[ 1. , 0. , 0. ]])
For comparison:
%timeit make_grid_part(5, .1)
>>> 338 µs ± 2.25 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit make_grid_simple(5, .1)
>>> 26.4 ms ± 806 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
make_grid_simple actually runs out of memory if you push it just a bit further.
Here is one simple way:
def make_grid_simple(n, step):
num_steps = round(1.0 / step)
vs = np.meshgrid(*([np.linspace(0, 1, num_steps + 1)] * n))
all_combs = np.stack([v.flatten() for v in vs], axis=1)
return all_combs[np.isclose(all_combs.sum(axis=1), 1)]
print(make_grid_simple(3, 0.33333))
Output:
[[ 0. 0. 1. ]
[ 0.33333333 0. 0.66666667]
[ 0.66666667 0. 0.33333333]
[ 1. 0. 0. ]
[ 0. 0.33333333 0.66666667]
[ 0.33333333 0.33333333 0.33333333]
[ 0.66666667 0.33333333 0. ]
[ 0. 0.66666667 0.33333333]
[ 0.33333333 0.66666667 0. ]
[ 0. 1. 0. ]]
However, this is not the most efficient way to do it, since it is simply making all the possible combinations and then just picking the ones that add up to 1, instead of generating only the right ones in the first place. For small step sizes, it may incur in too high memory cost.
Assuming that they always add up to 1, as you said:
import itertools
def make_grid(n):
# setup all possible values in one position
p = [(float(1)/n)*i for i in range(n+1)]
# combine values, filter by sum()==1
return [x for x in itertools.product(p, repeat=n) if sum(x) == 1]
print(make_grid(n=3))
#[(0.0, 0.0, 1.0),
# (0.0, 0.3333333333333333, 0.6666666666666666),
# (0.0, 0.6666666666666666, 0.3333333333333333),
# (0.0, 1.0, 0.0),
# (0.3333333333333333, 0.0, 0.6666666666666666),
# (0.3333333333333333, 0.3333333333333333, 0.3333333333333333),
# (0.3333333333333333, 0.6666666666666666, 0.0),
# (0.6666666666666666, 0.0, 0.3333333333333333),
# (0.6666666666666666, 0.3333333333333333, 0.0),
# (1.0, 0.0, 0.0)]
We can think of this as a problem of dividing some fixed number of things (1/s in this case and represented using sum_left parameter) between some given number of bins (n in this case). The most efficient way I can think of doing this is using a recursion:
In [31]: arr = []
In [32]: def fun(n, sum_left, arr_till_now):
...: if n==1:
...: n_arr = list(arr_till_now)
...: n_arr.append(sum_left)
...: arr.append(n_arr)
...: else:
...: for i in range(sum_left+1):
...: n_arr = list(arr_till_now)
...: n_arr.append(i)
...: fun(n-1, sum_left-i, n_arr)
This would give an output like:
In [36]: fun(n, n, [])
In [37]: arr
Out[37]:
[[0, 0, 3],
[0, 1, 2],
[0, 2, 1],
[0, 3, 0],
[1, 0, 2],
[1, 1, 1],
[1, 2, 0],
[2, 0, 1],
[2, 1, 0],
[3, 0, 0]]
And now I can convert it to a numpy array to do an elementwise multiplication:
In [39]: s = 0.33
In [40]: arr_np = np.array(arr)
In [41]: arr_np * s
Out[41]:
array([[ 0. , 0. , 0.99999999],
[ 0. , 0.33333333, 0.66666666],
[ 0. , 0.66666666, 0.33333333],
[ 0. , 0.99999999, 0. ],
[ 0.33333333, 0. , 0.66666666],
[ 0.33333333, 0.33333333, 0.33333333],
[ 0.33333333, 0.66666666, 0. ],
[ 0.66666666, 0. , 0.33333333],
[ 0.66666666, 0.33333333, 0. ],
[ 0.99999999, 0. , 0. ]])
This method will also work for an arbitrary sum (total):
import numpy as np
import itertools as it
import scipy.special
n = 3
s = 1/3.
total = 1.00
interval = int(total/s)
n_combs = scipy.special.comb(n+interval-1, interval, exact=True)
counts = np.zeros((n_combs, n), dtype=int)
def count_elements(elements, n):
count = np.zeros(n, dtype=int)
for elem in elements:
count[elem] += 1
return count
for i, comb in enumerate(it.combinations_with_replacement(range(n), interval)):
counts[i] = count_elements(comb, n)
ratios = counts*s
print(ratios)
I have an unknown number n of variables that can range from 0 to 1 with some known step s, with the condition that they sum up to 1. I want to create a matrix of all combinations. For example, if n=3 and s=0.33333 then the grid will be (The order is not important):
0.00, 0.00, 1.00
0.00, 0.33, 0.67
0.00, 0.67, 0.33
0.00, 1.00, 0.00
0.33, 0.00, 0.67
0.33, 0.33, 0.33
0.33, 0.67, 0.00
0.67, 0.00, 0.33
0.67, 0.33, 0.00
1.00, 0.00, 0.00
How can I do that for an arbitrary n?
Here is a direct method using itertools.combinations:
>>> import itertools as it
>>> import numpy as np
>>>
>>> # k is 1/s
>>> n, k = 3, 3
>>>
>>> combs = np.array((*it.combinations(range(n+k-1), n-1),), int)
>>> (np.diff(np.c_[np.full((len(combs),), -1), combs, np.full((len(combs),), n+k-1)]) - 1) / k
array([[0. , 0. , 1. ],
[0. , 0.33333333, 0.66666667],
[0. , 0.66666667, 0.33333333],
[0. , 1. , 0. ],
[0.33333333, 0. , 0.66666667],
[0.33333333, 0.33333333, 0.33333333],
[0.33333333, 0.66666667, 0. ],
[0.66666667, 0. , 0.33333333],
[0.66666667, 0.33333333, 0. ],
[1. , 0. , 0. ]])
If speed is a concern, itertools.combinations can be replaced by a numpy implementation.
EDIT
Here is a better solution. It basically partitions the number of steps into the amount of variables to generate all the valid combinations:
def partitions(n, k):
if n < 0:
return -partitions(-n, k)
if k <= 0:
raise ValueError('Number of partitions must be positive')
if k == 1:
return np.array([[n]])
ranges = np.array([np.arange(i + 1) for i in range(n + 1)])
parts = ranges[-1].reshape((-1, 1))
s = ranges[-1]
for _ in range(1, k - 1):
d = n - s
new_col = np.concatenate(ranges[d])
parts = np.repeat(parts, d + 1, axis=0)
s = np.repeat(s, d + 1) + new_col
parts = np.append(parts, new_col.reshape((-1, 1)), axis=1)
return np.append(parts, (n - s).reshape((-1, 1)), axis=1)
def make_grid_part(n, step):
num_steps = round(1.0 / step)
return partitions(num_steps, n) / float(num_steps)
print(make_grid_part(3, 0.33333))
Output:
array([[ 0. , 0. , 1. ],
[ 0. , 0.33333333, 0.66666667],
[ 0. , 0.66666667, 0.33333333],
[ 0. , 1. , 0. ],
[ 0.33333333, 0. , 0.66666667],
[ 0.33333333, 0.33333333, 0.33333333],
[ 0.33333333, 0.66666667, 0. ],
[ 0.66666667, 0. , 0.33333333],
[ 0.66666667, 0.33333333, 0. ],
[ 1. , 0. , 0. ]])
For comparison:
%timeit make_grid_part(5, .1)
>>> 338 µs ± 2.25 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit make_grid_simple(5, .1)
>>> 26.4 ms ± 806 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
make_grid_simple actually runs out of memory if you push it just a bit further.
Here is one simple way:
def make_grid_simple(n, step):
num_steps = round(1.0 / step)
vs = np.meshgrid(*([np.linspace(0, 1, num_steps + 1)] * n))
all_combs = np.stack([v.flatten() for v in vs], axis=1)
return all_combs[np.isclose(all_combs.sum(axis=1), 1)]
print(make_grid_simple(3, 0.33333))
Output:
[[ 0. 0. 1. ]
[ 0.33333333 0. 0.66666667]
[ 0.66666667 0. 0.33333333]
[ 1. 0. 0. ]
[ 0. 0.33333333 0.66666667]
[ 0.33333333 0.33333333 0.33333333]
[ 0.66666667 0.33333333 0. ]
[ 0. 0.66666667 0.33333333]
[ 0.33333333 0.66666667 0. ]
[ 0. 1. 0. ]]
However, this is not the most efficient way to do it, since it is simply making all the possible combinations and then just picking the ones that add up to 1, instead of generating only the right ones in the first place. For small step sizes, it may incur in too high memory cost.
Assuming that they always add up to 1, as you said:
import itertools
def make_grid(n):
# setup all possible values in one position
p = [(float(1)/n)*i for i in range(n+1)]
# combine values, filter by sum()==1
return [x for x in itertools.product(p, repeat=n) if sum(x) == 1]
print(make_grid(n=3))
#[(0.0, 0.0, 1.0),
# (0.0, 0.3333333333333333, 0.6666666666666666),
# (0.0, 0.6666666666666666, 0.3333333333333333),
# (0.0, 1.0, 0.0),
# (0.3333333333333333, 0.0, 0.6666666666666666),
# (0.3333333333333333, 0.3333333333333333, 0.3333333333333333),
# (0.3333333333333333, 0.6666666666666666, 0.0),
# (0.6666666666666666, 0.0, 0.3333333333333333),
# (0.6666666666666666, 0.3333333333333333, 0.0),
# (1.0, 0.0, 0.0)]
We can think of this as a problem of dividing some fixed number of things (1/s in this case and represented using sum_left parameter) between some given number of bins (n in this case). The most efficient way I can think of doing this is using a recursion:
In [31]: arr = []
In [32]: def fun(n, sum_left, arr_till_now):
...: if n==1:
...: n_arr = list(arr_till_now)
...: n_arr.append(sum_left)
...: arr.append(n_arr)
...: else:
...: for i in range(sum_left+1):
...: n_arr = list(arr_till_now)
...: n_arr.append(i)
...: fun(n-1, sum_left-i, n_arr)
This would give an output like:
In [36]: fun(n, n, [])
In [37]: arr
Out[37]:
[[0, 0, 3],
[0, 1, 2],
[0, 2, 1],
[0, 3, 0],
[1, 0, 2],
[1, 1, 1],
[1, 2, 0],
[2, 0, 1],
[2, 1, 0],
[3, 0, 0]]
And now I can convert it to a numpy array to do an elementwise multiplication:
In [39]: s = 0.33
In [40]: arr_np = np.array(arr)
In [41]: arr_np * s
Out[41]:
array([[ 0. , 0. , 0.99999999],
[ 0. , 0.33333333, 0.66666666],
[ 0. , 0.66666666, 0.33333333],
[ 0. , 0.99999999, 0. ],
[ 0.33333333, 0. , 0.66666666],
[ 0.33333333, 0.33333333, 0.33333333],
[ 0.33333333, 0.66666666, 0. ],
[ 0.66666666, 0. , 0.33333333],
[ 0.66666666, 0.33333333, 0. ],
[ 0.99999999, 0. , 0. ]])
This method will also work for an arbitrary sum (total):
import numpy as np
import itertools as it
import scipy.special
n = 3
s = 1/3.
total = 1.00
interval = int(total/s)
n_combs = scipy.special.comb(n+interval-1, interval, exact=True)
counts = np.zeros((n_combs, n), dtype=int)
def count_elements(elements, n):
count = np.zeros(n, dtype=int)
for elem in elements:
count[elem] += 1
return count
for i, comb in enumerate(it.combinations_with_replacement(range(n), interval)):
counts[i] = count_elements(comb, n)
ratios = counts*s
print(ratios)
I would like calculate the sum of two in two column in a matrix(the sum between the columns 0 and 1, between 2 and 3...).
So I tried to do nested "for" loops but at every time I haven't the good results.
For example:
c = np.array([[0,0,0.25,0.5],[0,0.5,0.25,0],[0.5,0,0,0]],float)
freq=np.zeros(6,float).reshape((3, 2))
#I calculate the sum between the first and second column, and between the fird and the fourth column
for i in range(0,4,2):
for j in range(1,4,2):
for p in range(0,2):
freq[:,p]=(c[:,i]+c[:,j])
But the result is:
print freq
array([[ 0.75, 0.75],
[ 0.25, 0.25],
[ 0. , 0. ]])
Normaly the good result must be (0., 0.5,0.5) and (0.75,0.25,0). So I think the problem is in the nested "for" loops.
Is there a person who know how I can calculate the sum every two columns, because I have a matrix with 400 columns?
You can simply reshape to split the last dimension into two dimensions, with the last dimension of length 2 and then sum along it, like so -
freq = c.reshape(c.shape[0],-1,2).sum(2).T
Reshaping only creates a view into the array, so effectively, we are just using the summing operation here and as such must be efficient.
Sample run -
In [17]: c
Out[17]:
array([[ 0. , 0. , 0.25, 0.5 ],
[ 0. , 0.5 , 0.25, 0. ],
[ 0.5 , 0. , 0. , 0. ]])
In [18]: c.reshape(c.shape[0],-1,2).sum(2).T
Out[18]:
array([[ 0. , 0.5 , 0.5 ],
[ 0.75, 0.25, 0. ]])
Add the slices c[:, ::2] and c[:, 1::2]:
In [62]: c
Out[62]:
array([[ 0. , 0. , 0.25, 0.5 ],
[ 0. , 0.5 , 0.25, 0. ],
[ 0.5 , 0. , 0. , 0. ]])
In [63]: c[:, ::2] + c[:, 1::2]
Out[63]:
array([[ 0. , 0.75],
[ 0.5 , 0.25],
[ 0.5 , 0. ]])
Here is one way using np.split():
In [36]: np.array(np.split(c, np.arange(2, c.shape[1], 2), axis=1)).sum(axis=-1)
Out[36]:
array([[ 0. , 0.5 , 0.5 ],
[ 0.75, 0.25, 0. ]])
Or as a more general way even for odd length arrays:
In [87]: def vertical_adder(array):
return np.column_stack([np.sum(arr, axis=1) for arr in np.array_split(array, np.arange(2, array.shape[1], 2), axis=1)])
....:
In [88]: vertical_adder(c)
Out[88]:
array([[ 0. , 0.75],
[ 0.5 , 0.25],
[ 0.5 , 0. ]])
In [94]: a
Out[94]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]])
In [95]: vertical_adder(a)
Out[95]:
array([[ 1, 5, 4],
[11, 15, 9],
[21, 25, 14]])
I have a matrix that should have ones on the diagonal but the columns are mixed up.
But I don't know how, without the obvious for loop, to efficiently interchange rows to get unity on the diagonals. I'm not even sure what key I would pass to sort on.
Any suggestions?
You can use numpy's argmax to determine the goal column ordering and reorder your matrix using the argmax results as column indices:
>>> z = numpy.array([[ 0.1 , 0.1 , 1. ],
... [ 1. , 0.1 , 0.09],
... [ 0.1 , 1. , 0.2 ]])
numpy.argmax(z, axis=1)
>>> array([2, 0, 1]) #Goal column indices
z[:,numpy.argmax(z, axis=1)]
>>> array([[ 1. , 0.1 , 0.1 ],
... [ 0.09, 1. , 0.1 ],
... [ 0.2 , 0.1 , 1. ]])
>>> import numpy as np
>>> a = np.array([[ 1. , 0.5, 0.5, 0. ],
... [ 0.5, 0.5, 1. , 0. ],
... [ 0. , 1. , 0. , 0.5],
... [ 0. , 0.5, 0.5, 1. ]])
>>> np.array(sorted(a, cmp=lambda x, y: list(x).index(1) - list(y).index(1)))
array([[ 1. , 0.5, 0.5, 0. ],
[ 0. , 1. , 0. , 0.5],
[ 0.5, 0.5, 1. , 0. ],
[ 0. , 0.5, 0.5, 1. ]])
It actually sorts by rows, not columns (but the result is the same). It works by sorting by the index of the column the 1 is in.