Develop Reference

Initialise a numpy array of a specific shape

I want to initialise a numpy array of a specific shape such that when I append numbers to it it will 'fill up' in that shape.
The length of the array will vary - and that is fine I do not mind how long it is - but I want it to have 4 columns. Ideally somthing similar to the following:
array = np.array([:, 4])
print(array)
array = [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
Again the actual length of the array would not be defines. That way if I was to append a different array it would work as follows
test_array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
array = np.append(array, test_array)
print(array)
array = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
Is there any way to do this?

If I understand well your issue, I think you do not need to initialize your array.
You sould check first that your array size divides by 4.
import numpy as np
l = test_array.shape[0]
cols = 4
rows = l / cols
my_array = np.reshape(test_array, (rows, cols))

The kind of behavior that you seek is unusual. You should explain why you need it. If you want something readily grows, use Python list. numpy arrays have a fixed size. Values can be assigned to an array in various ways, but to grow it, you need to create a new array with some version of concatenate. (Yes, there is a resize function/method, but that's not commonly used.)
I'll illustrate the value assignment options:
Initial an array with a known size. In your case the 5 could be larger than anticipated, and the 4 is the desired number of 'columns'.
In [1]: arr = np.zeros((5,4), dtype=int)
In [2]: arr
Out[2]:
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
Assign 4 values to one row:
In [3]: arr[0] = [1,2,3,4]
Assign 3 values starting at a given point in a flat view of the array:
In [4]: arr.flat[4:7] = [1,2,3]
In [5]: arr
Out[5]:
array([[1, 2, 3, 4],
[1, 2, 3, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
This array, while defined as (5,4) shape, can be viewed as (20,) 1d array. I had to choose the appropriate slice values in the flat view.
More commonly we assign values to a block of rows (or a variety of other indexed areas). arr[2:, :] is a (3,4) portion of arr. So we need to assign (3,4) array to it (or an equivalent list structure). To get full benefit of this sort of assignment you need to read up on broadcasting.
In [6]: arr[2:,:] = np.reshape(list(range(10,22)),(3,4))
In [7]: arr
Out[7]:
array([[ 1, 2, 3, 4],
[ 1, 2, 3, 0],
[10, 11, 12, 13],
[14, 15, 16, 17],
[18, 19, 20, 21]])
In [8]: arr.ravel()
Out[8]:
array([ 1, 2, 3, 4, 1, 2, 3, 0, 10, 11, 12, 13, 14, 15, 16, 17, 18,
19, 20, 21])

Python3 Numpy multiply : Could not broadcast together with shapes (10, 10000) (10000, 10)

I'm having an issue using numpy in python3 at this instruction:
res = ( np.multiply(error, v_sigmop ))
I'm trying to multiply element-wise but I'm having this weird error :
res = ( np.multiply(error, v_sigmop ))
ValueError: operands could not be broadcast together with shapes (10,10000) (10000,10)
This operation isn't illegal since the amount of columns matches the amount of rows of the second array...
Any idea?

I think you are maybe trying to multiple 2 matrixes with shape (r1,c) and (c,r2)
You can use A.dot(B) for your problem, that will multiple 2 matrixes.
This is example:
>>> a = np.arange(12).reshape((3,4))
>>> b = np.arange(8).reshape((4,2))
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> b
array([[0, 1],
[2, 3],
[4, 5],
[6, 7]])
>>> a.dot(b)
array([[ 28, 34],
[ 76, 98],
[124, 162]])
Hope it will help you!
Edit
Because you don't want multiple 2 matrixes, you want to multiple as scalar, but multiple scalar is not your operation, that mean you can't multiple 2 matrixes such as:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
multiple with
array([[0, 1],
[2, 3],
[4, 5]])
That is not valid operation for multiple 2 scalar.
You only have operations:
>>> a = np.arange(12).reshape((3,4))
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
# Multiple all elements with a scalar
>>> np.multiply(a,0)
array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
# Multiple each column with a column
>>> b = np.arange(3).reshape((3,1))
>>> b
array([[0],
[1],
[2]])
>>> np.multiply(a,b)
array([[ 0, 0, 0, 0],
[ 4, 5, 6, 7],
[16, 18, 20, 22]])
# Multiple each row with a row
>>> b = np.arange(4).reshape((1,4))
>>> b
array([[0, 1, 2, 3]])
>>> np.multiply(a,b)
array([[ 0, 1, 4, 9],
[ 0, 5, 12, 21],
[ 0, 9, 20, 33]])
# Multiple each element with the same shape
>>> b = np.arange(12).reshape((3,4))
>>> b
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> np.multiply(a,b)
array([[ 0, 1, 4, 9],
[ 16, 25, 36, 49],
[ 64, 81, 100, 121]])

Creating sympy Matrices from columns

I'm trying to create a sympy Matrix by choosing columns from an existing Matrix (for calculating principal minors). At the moment I'm doing it like this:
>>> A = Matrix(3,5,[2,3,4,1,34,23,12,54,5,0,0,0,3,4,5])
>>> l = [A[:,i].T for i in [2,3,0]]
>>> M = Matrix(l).T
>>> M
Matrix([
[ 4, 1, 2],
[54, 5, 23],
[ 3, 4, 0]])
But this seems wasteful to me (especially the need to transpose twice. I don't know if this is time consuming). Is there a better way? Would there be a better way if i only need the determinant?

You can use [2, 3, 0] as index.
>>> A = Matrix(3, 5, [2,3,4,1,34,23,12,54,5,0,0,0,3,4,5])
>>> A[:, [2,3,0]]
Matrix([
[ 4, 1, 2],
[54, 5, 23],
[ 3, 4, 0]])
For lower version that does not support using list as a index, you can use Matrix.hstack:
>>> Matrix.hstack(*(A.col(i) for i in [2,3,0]))
Matrix([
[ 4, 1, 2],
[54, 5, 23],
[ 3, 4, 0]])
or Matrix.row_join:
>>> # from functools import reduce # For Python 3.x
>>> reduce(Matrix.row_join, (A.col(i) for i in [2,3,0]), Matrix(3,0,[]))
Matrix([
[ 4, 1, 2],
[54, 5, 23],
[ 3, 4, 0]])

Is there a way to generate a list of indices using numpy

Can I use numpy to generate repeating patterns of indices for example.
0, 1, 2, 3, 4, 5, 0, 6, 7, 8, 9, 10, 0, 11, 12, 13, 14, 15
or
0,1,2,1,2,3,4,5,6,5,6,7
Is there a method in numpy i can use to generate these lists between a range ?
currently I am doing this using lists in python but I was curious if I could use numpy to speed things up.
I am not sure what methods to even look into other than numpy.arange.
Just to further clarify I am generating indices to triangles in opengl in various patterns.
so for traingles in a circle I have some code like this.
for fan_set in range(0, len(self.vertices) / vertex_length, triangle_count):
for i in range(fan_set + 1, fan_set + 8):
self.indices.append(fan_set)
self.indices.append(i)
self.indices.append(i + 1)

Your first example can be produced via numpy methods as:
In [860]: np.concatenate((np.zeros((3,1),int),np.arange(1,16).reshape(3,5)),axis=1).ravel()
Out[860]:
array([ 0, 1, 2, 3, 4, 5, 0, 6, 7, 8, 9, 10, 0, 11, 12, 13, 14,
15])
That's because I see this 2d repeated pattern
array([[ 0, 1, 2, 3, 4, 5],
[ 0, 6, 7, 8, 9, 10],
[ 0, 11, 12, 13, 14, 15]])
The second pattern can be produced by ravel of this 2d array (produced by broadcasting 2 arrays):
In [863]: np.array([0,1,4,5])[:,None]+np.arange(3)
Out[863]:
array([[0, 1, 2],
[1, 2, 3],
[4, 5, 6],
[5, 6, 7]])
I can produce the 1st pattern with a variation on the 2nd (the initial column of 0s disrupts the pattern)
I=np.array([0,5,10])[:,None]+np.arange(0,6)
I[:,0]=0
I think your double loop can be expressed as a list comprehension as
In [872]: np.array([ [k,i,i+1] for k in range(0,1,1) for i in range(k+1,k+8)]).ravel()
Out[872]: array([0, 1, 2, 0, 2, 3, 0, 3, 4, 0, 4, 5, 0, 5, 6, 0, 6, 7, 0, 7, 8])
or without the ravel:
array([[0, 1, 2],
[0, 2, 3],
[0, 3, 4],
[0, 4, 5],
[0, 5, 6],
[0, 6, 7],
[0, 7, 8]])
though I don't know what parameters produce your examples.

I'm not sure I understand exactly what you mean, but the following is what I use to generate unique indices for 3D points;
def indexate(points):
"""
Convert a numpy array of points into a list of indices and an array of
unique points.
Arguments:
points: A numpy array of shape (N, 3).
Returns:
An array of indices and an (M, 3) array of unique points.
"""
pd = {}
indices = [pd.setdefault(tuple(p), len(pd)) for p in points]
pt = sorted([(v, k) for k, v in pd.items()], key=lambda x: x[0])
unique = np.array([i[1] for i in pt])
return np.array(indices, np.uint16), unique
You can find this code in my stltools package on github.
It works like this;
In [1]: import numpy as np
In [2]: points = np.array([[1,0,0], [0,0,1], [1,0,0], [0,1,0]])
In [3]: pd = {}
In [4]: indices = [pd.setdefault(tuple(p), len(pd)) for p in points]
In [5]: indices
Out[5]: [0, 1, 0, 2]
In [6]: pt = sorted([(v, k) for k, v in pd.items()], key=lambda x: x[0])
In [7]: pt
Out[7]: [(0, (1, 0, 0)), (1, (0, 0, 1)), (2, (0, 1, 0))]
In [8]: unique = np.array([i[1] for i in pt])
In [9]: unique
Out[9]:
array([[1, 0, 0],
[0, 0, 1],
[0, 1, 0]])
The key point (if you'll pardon the pun) is to use a tuple of the point (because a tuple is immutable and thus hashable) as the key in a dictionary with the setdefault method, while the length of the dict is the value. In effect, the value is the first time this exact point was seen.

I am not 100% certain this is what you're after, I think you can achieve this using pair of range values and increment n times 3 (the gap between each group), then use numpy.concatenate to concatenate the final array, like this:
import numpy as np
def gen_list(n):
return np.concatenate([np.array(range(i, i+3) + range(i+1, i+4)) + i*3
for i in xrange(n)])
Usage:
gen_list(2)
Out[16]: array([0, 1, 2, 1, 2, 3, 4, 5, 6, 5, 6, 7])
gen_list(3)
Out[17]:
array([ 0, 1, 2, 1, 2, 3, 4, 5, 6, 5, 6, 7, 8, 9, 10, 9, 10,
11])
list(gen_list(2))
Out[18]: [0, 1, 2, 1, 2, 3, 4, 5, 6, 5, 6, 7]
In my sample I only use n as how many groups you want to generate, you may change this to suit your triangle-ish requirements.

How to pad multiple lists with trailing zeros?

Suppose I have two lists containing the same number of elements which are lists of integers. For instance:
a = [[1, 7, 3, 10, 4], [1, 3, 8], ..., [2, 5, 10, 91, 54, 0]]
b = [[5, 4, 23], [1, 2, 0, 4], ..., [5, 15, 11]]
For each index, I want to pad the shorter list with trailing zeros. The example above should look like:
a = [[1, 7, 3, 10, 4], [1, 3, 8, 0], ..., [2, 5, 10, 91, 54, 0]]
b = [[5, 4, 23, 0, 0], [1, 2, 0, 4], ..., [51, 15, 11, 0, 0, 0]]
Is there an elegant way to perform this comparison and padding build into Python lists or perhaps numpy? I am aware that numpy.pad can perform the padding, but its the iteration and comparison over the lists that has got me stuck.

I'm sure there's an elegant Python one-liner for this sort of thing, but sometimes a straightforward imperative solution will get the job done:
for i in xrange(0, len(a)):
x = len(a[i])
y = len(b[i])
diff = max(x, y)
a[i].extend([0] * (diff - x))
b[i].extend([0] * (diff - y))
print a, b
Be careful with "elegant" solutions too, because they can be very difficult to comprehend (I can't count the number of times I've come back to a piece of code I wrote using reduce() and had to struggle to figure out how it worked).

One line? Yes. Elegant? No.
In [2]: from itertools import izip_longest
In [3]: A, B = map(list, zip(*[map(list, zip(*izip_longest(l1,l2, fillvalue=0)))
for l1,l2 in zip(a,b)]))
In [4]: A
Out[4]: [[1, 7, 3, 10, 4], [1, 3, 8, 0], [2, 5, 10, 91, 54, 0]]
In [5]: B
Out[5]: [[5, 4, 23, 0, 0], [1, 2, 0, 4], [5, 15, 11, 0, 0, 0]]

Note: Creates 2 new lists. Preserves the old lists.
from itertools import repeat
>>> b = [[5, 4, 23], [1, 2, 0, 4],[5, 15, 11]]
>>> a = [[1, 7, 3, 10, 4], [1, 3, 8],[2, 5, 10, 91, 54, 0]]
>>> [y+list(repeat(0, len(x)-len(y))) for x,y in zip(a,b)]
[[5, 4, 23, 0, 0], [1, 2, 0, 4], [5, 15, 11, 0, 0, 0]]
>>> [x+list(repeat(0, len(y)-len(x))) for x,y in zip(a,b)]
[[1, 7, 3, 10, 4], [1, 3, 8, 0], [2, 5, 10, 91, 54, 0]]

a = [[1, 7, 3, 10, 4], [1, 3, 8], [2, 5, 10, 91, 54, 0]]
b = [[5, 4, 23], [1, 2, 0, 4], [5, 15, 11]]
for idx in range(len(a)):
size_diff = len(a[idx]) - len(b[idx])
if size_diff < 0:
a[idx].extend( [0] * abs(size_diff) )
elif size_diff > 0:
b[idx].extend( [0] * size_diff )