I'm wondering if there's an easy way to do train test splitting (mainly interested in crossvalidation) in python such that I don't end up with data points from the same patient in both train and test? That is, I'd like to first split the patients into train and test then the observations accordingly.
Is there a functionality for this kind of scenario or do I have to code it manually?
Sklearn GroupKFold should solve this task. K-fold iterator variant with non-overlapping groups. The same group will not appear in two different folds:
from sklearn.model_selection import GroupKFold
X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
y = np.array([1, 2, 3, 4])
groups = np.array([0, 0, 2, 2])
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)
Related
I give TSNE a list of vectors, some of these vectors are exactly the same. But the output of fit() function can be different for each!
IS this expected behavior? How can i assure each input vector will be mapped to same output vector?
Exclamation,
I cannot tell for sure, but I even noticed that the first entry in the input list of vectors always gets different unexpected value.
Consider the following simple example.
Notice that the first three vectors are the same, the random_state is static but the first three 2D vectors in the output can be different from each others.
from sklearn import manifold
import numpy as np
X= np.array([ [2, 1, 3, 5],
[2, 1, 3, 5],
[2, 1, 3, 5],
[2, 1, 3, 5],
[12, 1, 3, 5],
[87, 22, 3, 5],
[3, 23, 9, 5],
[43, 87, 3, 5],
[121, 65, 3, 5]])
m = manifold.TSNE(
n_components=2,
perplexity=0.666666,
verbose=0,
random_state=42,
angle=.99,
init='pca',
metric='cosine',
n_iter=1000)
X_emedded = m.fit_transform(X)
# The following might fail
assert( sum(X_emedded[1] - X_emedded[2] ) == 0)
assert( sum(X_emedded[0] - X_emedded[1] ) == 0)
Update....
sklearn.version is '1.2.0'
t-SNE, as presenter by van der Maaten and Hinton 2008 is a technique to "visualizes high-dimensional data by giving each
datapoint a location in a two or three-dimensional map".
There is no guarantee that two identical points are mapped to the same low dimensional point. As a matter of fact it almost never happens as one can see with Algorithm 1 in (Maaten and Hinton 2008). The points in the low dimensional space are obtained with a gradient descent minimizing a cost function after a random initialisation.
I have some code which calculates the nearest neighbors amongst some vectors (values).
However, the values of these vectors are dependent on weights. Each column of the vectors has a different weight at every iteration.
Just for the sake of the example, at the code below I try to find everytime the nearest neighbor of the last vector (vector[3]).
That's a very simplified version of my code:
from sklearn.neighbors import NearestNeighbors
knn = NearestNeighbors(n_neighbors=1)
values = [
[2, 5, 1],
[4, 2, 3],
[1, 5, 2],
[4, 5, 4]
]
weights = [
[1, 3, 1],
[0.5, 2, 1],
[3, 1, 2]
]
# weights set No1
new_values = []
for line in values:
new_values.append([a*b for a,b in zip(line,weights[0])])
knn.fit(new_values)
print(knn.kneighbors(new_values[3]))
# weights set No2
new_values = []
for line in values:
new_values.append([a*b for a,b in zip(line,weights[1])])
knn.fit(new_values)
print(knn.kneighbors(new_values[3]))
# weights set No3
new_values = []
for line in values:
new_values.append([a*b for a,b in zip(line,weights[2])])
knn.fit(new_values)
print(knn.kneighbors(new_values[3]))
(Obviously I could have a for loop for the different weights sets but I just wanted to point the repetition of the matter)
My question is, is there any way that I can avoid using the KNN 3 times but just use it once at the beginning to do the initial similarity ranking/sorting and then just do some re-calculations?
In different words, is there any way to reduce the computation complexity of this code in terms of calling the KNN fewer times?
PS
I know that there are KNN implementations which are much faster than the ScikitLearn one but that's not really the point; the point is more on using KNN just once instead of N=3 times or something like that.
assuming calling the KNN fewer times means the number of times the KNN is fit, yes it's possible. if calling the KNN means the number of times kneighbors is invoked, that might be difficult due to how relative distances aren't preserved under affine transformations.
This solution runs in O(wk log n) time compared to the original O(wn) time with w being the number of weights.
what you're doing is
taking the input points
scaling its dimensions (projecting the input points into a new coordinate space)
building a knn model from the scaled inputs
classifying the target based on the scaled input.
However, consider
taking the input points
building a knn model from the scaled inputs
inverse scaling the target point (projecting the target into the original coordinate space)
classifying the inverse scaled target based on the input
the result of this process would be that steps 1 and 2 could be reused for each target point. weights with value 0 will require special handling.
this would look would be something like:
from sklearn.neighbors import NearestNeighbors
knn = NearestNeighbors(n_neighbors=1, algorithm="kd_tree")
values = [
[2, 5, 1],
[4, 2, 3],
[1, 5, 2],
[4, 5, 4]
]
weights = [
[1, 3, 1],
[0.5, 2, 1],
[3, 1, 2]
]
targets = [
[4, 15, 4], # values[3] * weights[0]
[2.0, 10, 4], # values[3] * weights[1]
[12, 5, 8] # values[3] * weights[2]
]
knn.fit(values)
# weights set No1
print(knn.kneighbors([[a/b for a, b in zip(targets[0], weights[0])]]))
# weights set No2
print(knn.kneighbors([[a/b for a, b in zip(targets[1], weights[1])]]))
# weights set No3
print(knn.kneighbors([[a/b for a, b in zip(targets[2], weights[2])]]))
When doing small tests on lightgbm, I found a case I could not understand.
I create a small dataset with categorical columns:
import pandas as pd
X = pd.DataFrame(
[
[0, 1, 1],
[2, 2, 0],
[3, 3, 2],
[2, 3, 2],
[0, -3, 0],
[0, 0, 1],
[0, 3, 0],
[1, 2.5, 1],
[2, 5, 0],
[3, -1.5, 2],
],
columns=["col1", "col2", "col3"],
)
X["col1"] = X["col1"].astype("category")
X["col3"] = X["col3"].astype("category")
y = pd.Series([0, 1, 1, 0, 0, 0, 0, 1, 0, 1])
and its associated one hot encoded version:
from sklearn.preprocessing import OneHotEncoder
feats_to_encode = ["col1", "col3"]
enc = OneHotEncoder()
enc.fit(X[feats_to_encode])
X_one_hot_encoded = pd.DataFrame(
enc.transform(X[feats_to_encode]).toarray(),
columns=[
feats_to_encode[feat_id] + str(cat)
for feat_id in range(len(enc.categories_))
for cat in enc.categories_[feat_id]
],
)
X_one_hot_encoded["col2"] = X["col2"]
I then trained 2 lightgbm models, with both the previous datasets. I know that lightgbm manages categorical columns with special algorithms. "When number of categories of one feature smaller than or equal to max_cat_to_onehot, one-vs-other split algorithm will be used" (see max_cat_to_onehot). I would then naively expect that, since my categorical columns have less categories (4 and 3) than the parameter max_cat_to_onehot, I would get the same result with both the datasets, unless if the "one-vs-other split algorithm" is not equivalent to one hot encode categorical columns. I assumed this behaviour because of the names of the parameter, "max_cat_to_onehot", and the algorithm, "one-vs-other split algorithm".
from lightgbm import LGBMRegressor
params = {
"n_estimators": 1,
"max_depth": 2,
"min_child_samples": 1,
"importance_type": "gain",
"max_cat_to_onehot": 10,
}
model = LGBMRegressor(**params)
model.fit(X, y)
print(model._Booster.dump_model()["tree_info"][0]["tree_structure"]["split_gain"]) # 0.30476200580596924
model = LGBMRegressor(**params)
model.fit(X_one_hot_encoded, y)
print(model._Booster.dump_model()["tree_info"][0]["tree_structure"]["split_gain"]) # 1.0666699409484863
The first split is different between the 2 models. The second model chose the best possible split, but this is not the case for the first one.
Does someone know the reason of this? I assume that my guess for the behaviour of "one-vs-other split algorithm" is wrong.
I updated the lightgbm version, from 2.3.1 to 3.1.1. With the new version I got the expected results.
I am using the regression slope as follows to calculate the steepness (slope) of the trend.
Scenario 1:
For example, consider I am using sales figures (x-axis: 1, 4, 6, 8, 10, 15) for 6 days (y-axis).
from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
X = [[1], [4], [6], [8], [10], [15]]
y = [1, 2, 3, 4, 5, 6]
regressor.fit(X, y)
print(regressor.coef_)
This gives me 0.37709497
Scenario 2:
When I run the same program for a different sale figure (e.g., 1, 2, 3, 4, 5, 6) I get the results as 1.
However, you can see that sales is much productive in scenario 1, but not in scenario 2. However, the slope I get for scenario 2 is higher than scenario 1.
Therefore, I am not sure if the regression slope captures what I require. Is there any other approach I can use instead to calculate the sleepness of the trend slope.
I am happy to provide more details if needed.
I believe the problem is your variables are switched. If you want to track sales performance over time, you should perform the regression the other way around. You can invert the slopes you've calculated to get the correct values, which will show higher sales performance in case 1.
1 / 0.377 = 2.65
Here is a visualization of your data:
import matplotlib.pyplot as plt
days = [1,2,3,4,5,6]
sales1 = [1,4,6,8,10,15]
sales2 = [1,2,3,4,5,6]
df = pd.DataFrame({'days': days, 'sales1': sales1, 'sales2': sales2})
df = df.set_index('days')
df.plot(marker='o', linestyle='--')
Forgive my terminology, I'm not an ML pro. I might use the wrong terms below.
I'm trying to perform multivariable linear regression. Let's say I'm trying to work out user gender by analysing page views on a web site.
For each user whose gender I know, I have a feature matrix where each row represents a web site section, and the second element whether they visited it, e.g.:
male1 = [
[1, 1], # visited section 1
[2, 0], # didn't visit section 2
[3, 1], # visited section 3, etc
[4, 0]
]
So in scikit, I am building xs and ys. I'm representing a male as 1, and female as 0.
The above would be represented as:
features = male1
gender = 1
Now, I'm obviously not just training a model for a single user, but instead I have tens of thousands of users whose data I'm using for training.
I would have thought I should create my xs and ys as follows:
xs = [
[ # user1
[1, 1],
[2, 0],
[3, 1],
[4, 0]
],
[ # user2
[1, 0],
[2, 1],
[3, 1],
[4, 0]
],
...
]
ys = [1, 0, ...]
scikit doesn't like this:
from sklearn import linear_model
clf = linear_model.LinearRegression()
clf.fit(xs, ys)
It complains:
ValueError: Found array with dim 3. Estimator expected <= 2.
How am I supposed to supply a feature matrix to the linear regression algorithm in scikit-learn?
You need to create xs in a different way. According to the docs:
fit(X, y, sample_weight=None)
Parameters:
X : numpy array or sparse matrix of shape [n_samples, n_features]
Training data
y : numpy array of shape [n_samples, n_targets]
Target values
sample_weight : numpy array of shape [n_samples]
Individual weights for each sample
Hence xs should be a 2D array with as many rows as users and as many columns as web site sections. You defined xs as a 3D array though. In order to reduce the number of dimensions by one you could get rid of the section numbers through a list comprehension:
xs = [[visit for section, visit in user] for user in xs]
If you do so, the data you provided as an example gets transformed into:
xs = [[1, 0, 1, 0], # user1
[0, 1, 1, 0], # user2
...
]
and clf.fit(xs, ys) should work as expected.
A more efficient approach to dimension reduction would be that of slicing a NumPy array:
import numpy as np
xs = np.asarray(xs)[:,:,1]