Develop Reference

Python: whats does comma mean in in [ : ,1 ]

guys I am new to python and learning machine learning.I have trouble understanding what does this line of code mean:-
y_scores_knn = y_probas_knn[:,1]
I didnt get what ,1 mean inside an array box [] and what it does

Say for example your list is somewhat like this:
a = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]
if you do this print(a[:,0])
it will return [1, 2, 3]
So basically ':' means that you want to iterate through every item in the x co-ordinate

That is how you access the column with index 1. This functionality is only if y_scores_knn is part of certain classes that deal with 2-d arrays. One class I'm most familiar with is a numpy array, which allows the accessing of columns, along with many other helpful methods.

if x is a two dimensional array (a matrix) then x[:,1] is the second column (all elements where the second index is 1). This notation is quite common for things like octave, matlab and numpy which have a concept of a two dimensional array. While it does work for numpy, it will not work for a python list of lists. A list of lists that happen to have the same number of elements is just another one dimensional list as far as python is concerned. If you want to do work with matrices and you want to use python, use numpy.

randomly choose different sets in numpy?

I am trying to randomly select a set of integers in numpy and am encountering a strange error. If I define a numpy array with two sets of different sizes, np.random.choice chooses between them without issue:
Set1 = np.array([[1, 2, 3], [2, 4]])
In: np.random.choice(Set1)
Out: [4, 5]
However, once the numpy array are sets of the same size, I get a value error:
Set2 = np.array([[1, 3, 5], [2, 4, 6]])
In: np.random.choice(Set2)
ValueError: a must be 1-dimensional
Could be user error, but I've checked several times and the only difference is the size of the sets. I realize I can do something like:
Chosen = np.random.choice(N, k)
Selection = Set[Chosen]
Where N is the number of sets and k is the number of samples, but I'm just wondering if there was a better way and specifically what I am doing wrong to raise a value error when the sets are the same size.
Printout of Set1 and Set2 for reference:
In: Set1
Out: array([list([1, 3, 5]), list([2, 4])], dtype=object)
In: type(Set1)
Out: numpy.ndarray
In: Set2
Out:
array([[1, 3, 5],
[2, 4, 6]])
In: type(Set2)
Out: numpy.ndarray

Your issue is caused by a misunderstanding of how numpy arrays work. The first example can not "really" be turned into an array because numpy does not support ragged arrays. You end up with an array of object references that points to two python lists. The second example is a proper 2xN numerical array. I can think of two types of solutions here.
The obvious approach (which would work in both cases, by the way), would be to choose the index instead of the sublist. Since you are sampling with replacement, you can just generate the index and use it directly:
Set[np.random.randint(N, size=k)]
This is the same as
Set[np.random.choice(N, k)]
If you want to choose without replacement, your best bet is to use np.random.choice, with replace=False. This is similar to, but less efficient than shuffling. In either case, you can write a one-liner for the index:
Set[np.random.choice(N, k, replace=False)]
Or:
index = np.arange(Set.shape[0])
np.random.shuffle(index)
Set[index[:k]]
The nice thing about np.random.shuffle, though, is that you can apply it to Set directly, whether it is a one- or many-dimensional array. Shuffling will always happen along the first axis, so you can just take the top k elements afterwards:
np.random.shuffle(Set)
Set[:k]
The shuffling operation works only in-place, so you have to write it out the long way. It's also less efficient for large arrays, since you have to create the entire range up front, no matter how small k is.
The other solution is to turn the second example into an array of list objects like the first one. I do not recommend this solution unless the only reason you are using numpy is for the choice function. In fact I wouldn't recommend it at all, since you can, and probably should, use pythons standard random module at this point. Disclaimers aside, you can coerce the datatype of the second array to be object. It will remove any benefits of using numpy, and can't be done directly. Simply setting dtype=object will still create a 2D array, but will store references to python int objects instead of primitives in it. You have to do something like this:
Set = np.zeros(N, dtype=object)
Set[:] = [[1, 2, 3], [2, 4]]
You will now get an object essentially equivalent to the one in the first example, and can therefore apply np.random.choice directly.
Note
I show the legacy np.random methods here because of personal inertia if nothing else. The correct way, as suggested in the documentation I link to, is to use the new Generator API. This is especially true for the choice method, which is much more efficient in the new implementation. The usage is not any more difficult:
Set[np.random.default_rng().choice(N, k, replace=False)]
There are additional advantages, like the fact that you can now choose directly, even from a multidimensional array:
np.random.default_rng().choice(Set2, k, replace=False)
The same goes for shuffle, which, like choice, now allows you to select the axis you want to rearrange:
np.random.default_rng().shuffle(Set)
Set[:k]

How to declare a 2 dimensional array with different row lengths using np.array?

For example, I want a 2 row matrix, with a first row of length 1, and second row of length 2. I could do,
list1 = np.array([1])
list2 = np.array([2,3])
matrix = []
matrix.append(list1)
matrix.append(list2)
matrix = np.array(matrix)
I wonder if I could declare a matrix of this shape directly in the beginning of a program without going through the above procedure?

A matrix is by definition a rectangular array of numbers. NumPy does not support arrays that do not have a rectangular shape. Currently, what your code produces is an array, containing a list (matrix), containing two more arrays.
array([array([1]), array([2, 3])], dtype=object)
I don't really see what the purpose of this shape could be, and would advise you simply use nested lists for whatever you are doing with this shape. Should you have found some use for this structure with NumPy however, you can produce it much more idiomatically like this:
>>> np.array([list1,list2])
array([array([1]), array([2, 3])], dtype=object)

Extend numpy array in a way compatible with builtin arrays

I am trying to write code that would not depend on whether the user uses np.array or a builtin array. I am trying to avoid checking object types, etc. The only problem that I have is in extending arrays. For example, if I have two Python arrays, a and b, to extend the first by the second, I can either do a = a + b, or a.extend(b). On the other hand, if a is a numpy array, I need to use np.append or something else.
Is there a quick way to extend arrays independently of whether they are np arrays or Python arrays?

NumPy's append() works on lists too!
>>> np.append([1,2,3], [1,2,3])
array([1, 2, 3, 1, 2, 3])
If you want to automatically make the result be the same type as the input, try this:
mytype = type(a)
arr = np.append(a, b)
result = mytype(arr)

Even if your function is flexible on input, your output should be of specific type. So I would just convert to desired output type.
For example, if my functions is working with numpy.array and returns a numpy.array, but I want to allow lists to be input as well, the first thing I would do is convert lists to numpy.arrays.
Like this:
def my_func(a, b):
a = np.asarray(a)
b = np.asarray(b)
# do my stuff here

slicing python array elements with a vector similar to matlab/R

I'm new to python and wanted to do something I normally do in matlab/R all the time, but couldn't figure it out from the docs.
I'd like to slice an array not as 0:3 which includes elements 0,1,2 but as an explicit vector of indices such as 0,3
For example, say I had this data structure
a = [1, 2, 3, 4, 5]
I'd like the second and third element
so I thought something like this would work
a[list(1,3)]
but that gives me this error
TypeError: list indices must be
integers
This happens for most other data types as well such as numpy arrays
In matlab, you could even say a[list(2,1)] which would return this second and then the first element.
There is an alternative implementation I am considering, but I think it would be slow for large arrays. At least it would be damn slow in matlab. I'm primarily using numpy arrays.
[ a[i] for i in [1,3] ]
What's the python way oh wise ones?
Thanks!!

NumPy allows you to use lists as indices:
import numpy
a = numpy.array([1, 2, 3, 4, 5])
a[[1, 3]]
Note that this makes a copy instead of a view.

I believe you want numpy.take:
newA = numpy.take(a, [1,3])