i am trying to write a code that needs itertools.product to generate the output from a multiple number of lists , or even variable numbers , for example my inputs will be read from a list as :
a=[[[0,1],[2,3]],[[4,5],[6,7]]]
for this example it would be easy to write down :
itertools.product(a[0],a[1])
but as the list may increase in size and code may get pretty large and messy i can not use a code like this :
itertools.product(a[0],a[1],a[2],...)
what is a clean and efficient alternative way to do this?
i know it is not a function specific question , but a more general question about python itself and functions but i couldn't find any help about the existing functions, if anyone would point me in the right direction i'll be grateful.
thanks in advance
Use * to unpack the input list
>>> list(itertools.product(*a))
[([0, 1], [4, 5]), ([0, 1], [6, 7]), ([2, 3], [4, 5]), ([2, 3], [6, 7])]
Related
Let's consider an array [1,2,3] say, what I would like to generate is the list containing the pairs [[1,2], [1,3], [2,3]]. This can be done using itertools. However, I would like to produce them using pure numpy operations, and no loops or branching is allowed.
A close solution is provided here, but it generates all the possible pairs, instead of a particular fashion as in my case.
Can you please suggest a way to do that? The array will always be 1D.
Also, this is my first question on SE. If it requires any edit, please let me know.
Here is a one-liner using np.triu_indices:
>>> a = np.array([1, 2, 3])
>>> a[np.transpose(np.triu_indices(len(a), 1))]
array([[1, 2],
[1, 3],
[2, 3]])
I'm new to Python and I need a dynamic matrix that I can manipulate adding more columns and rows to it. I read about numpy.matrix, but I can't find a method in there that does what I mentioned above. It occurred to me to use lists but I want to know if there is a simpler way to do it or a better implementation.
Example of what I look for:
matrix.addrow ()
matrix.addcolumn ()
matrix.changeValue (0, 0, "$200")
Am I asking for too much? If so, any ideas of how to implement something like that? Thanks!
You can do all of that in numpy (np.concatenate for example) or native python (my_list.append()). Which one is more efficient will depend on what else your program will do: numpy will be probably less efficient if all you are doing is adding / changing values one at a time, or do a lot of column 'adding' or 'removing'. However if you do matrix or column operations, the overhead of adding new columns to a numpy array maybe offset by the vectorized computation speed offered by numpy. So pick which ever you prefer, and if speed is an issue, then you need to experiment yourself with both approaches...
There are several ways to represent matrices in Python. You can use List of lists or numpy arrays. For example if you were to use numpy arrays
>>> import numpy as np
>>> a = np.array([[1,2,3], [2,3,4]])
>>> a
array([[1, 2, 3],
[2, 3, 4]])
To add a row
>>> np.vstack([a, [7,8,9]])
array([[1, 2, 3],
[2, 3, 4],
[7, 8, 9]])
To add a column
>>> np.hstack((a, [[7],[8]]))
array([[1, 2, 3, 7],
[2, 3, 4, 8]])
Coming from Matlab I am unable to even think of singular datapoints / variables. Anything I deal with is a matrix / array. After one week of searching and insuccesful trial and error I realise, that I ABSOLUTELY do NOT get the concept of dealing with matrices in (plain) Python.
I created
In[]: A = [[1,2,3], [9,8,7], [5,5,5]]
In[]: A
Out[]: [[1, 2, 3], [9, 8, 7], [5, 5, 5]]
Trying to extract the vectors in the matrix along the two dimensions:
In[]: A[:][1]
Out[]: [9, 8, 7]
In[]: A[1][:]
Out[]: [9, 8, 7]
'surprisingly' gives the same! No way to get a specific column (of course, except with one by one iteration).
Consequently, I am unable to manage merging matrix A with another vector, i.e. extending A with another column. Matlab style approach obviously is odd:
In[]: B = A, [4,6,8]
In[]: B
Out[]: ([[1, 2, 3], [9, 8, 7], [5, 5, 5]], [4, 6, 8])
Results in something nested, not an extension of A.
Same for
B = [A, [4,6,8]]
Ok, more Python-like:
A.append([11,12,13])
This easily adds a row. But is there a similar way to add a column??
(The frustrating thing is that Python doc gives all kinds of fancy examples but apparently these focus on demonstrating 'pythonic' solutions for one-dimensional lists.)
Coming from MATLAB myself, I understand your point.
The problem is that Python lists are not designed to serve as matrices. When indexing a list, you always work on the top level list elements, e.g. A[:][1] returns all the ([:]) three list elements, namely [1, 2, 3], [9, 8, 7] and [5, 5, 5]. Then you select the second ([1]) element from those, i.e. [9, 8, 7]. A[1][:] does the same, just the other way round.
This being said, you can still use nested lists for simple indexing tasks, as A[1][1] gives the expected result (8). However, if you are planing to migrate your whole MATLAB code to Python or work on non-trivial matrix problems, you should definitely consider using NumPy. There is even a NumPy guide for former MATLAB users.
Okay, so basically lets say i have a matrix:
matrix([[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]])
Is it possible to get the area below the diagonal easily when working with numpy matrixs? I looked around and could not find anything. I can do the standard, for loop way however wouldnt that somehow invalidate the performance gave by numpy?
I am working on calculating statististics ofcomparing model output results with actual results. The data i currently have been given, results in around a 10,000 x 10,000 matrix. I am mainly trying to just sum those elements.
Is there an easy way to do this?
You could use tril and triu. See them here:
http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html
def tri_flat(array):
R = array.shape[0]
mask = np.asarray(np.invert(np.tri(R,R,dtype=bool)),dtype=float)
x,y = mask.nonzero()
return array[x,y]
I was looking for a convenience function myself, but this'll have to do... not sure it gets much easier. But if it does, I'd be interested to hear it. Every time you avoid a for-loop, an angel gets its wings.
-ejh
Quick NB: This avoids the diagonal... if you want it, and your matrix is symmetric, just omit the inversion (elem-wise NOT). Otherwise, you'll need a transpose in there.
Good day to all.
Help me please to understand theory of function scipy.ndimage.convolve for 1D arrays. I know the formula from http://lagrange.univ-lyon1.fr/docs/scipy/0.17.1/generated/scipy.ndimage.convolve.html
C_i = \sum_j{I_{i+j-k} W_j},
but i can't understand, how can I get results manually.
For example: test_1 = scipy.ndimage.convolve([1, 2, 3], [1, 2, 3, 4, 5])
result is [24 24 30]
Or test_2 = scipy.ndimage.convolve([1, 2, 3], [3, 4, 5])
result is [15 22 31]
If I write here all attempts that I have made, it will take a lot of space.
Give me please step by step instructions on what to do with these examples manually.
Two tricky things going on here
1) the ndimage has this flag called "mode" which is set to "reflect" by default
2) two is that convolutions internally reverse one of the inputs
try comparing this piece of code
scipy.ndimage.convolve([1, 2, 3][::-1], [1, 2, 3, 4, 5],mode='constant')
to your by hand solution. (get rid of the "[::-1]" if you've already accounted for the reversal)