This seems like a ridiculously easy question... but I'm not seeing the easy answer I was expecting.
So, how do I get the value at an nth row of a given column in Pandas? (I am particularly interested in the first row, but would be interested in a more general practice as well).
For example, let's say I want to pull the 1.2 value in Btime as a variable.
Whats the right way to do this?
>>> df_test
ATime X Y Z Btime C D E
0 1.2 2 15 2 1.2 12 25 12
1 1.4 3 12 1 1.3 13 22 11
2 1.5 1 10 6 1.4 11 20 16
3 1.6 2 9 10 1.7 12 29 12
4 1.9 1 1 9 1.9 11 21 19
5 2.0 0 0 0 2.0 8 10 11
6 2.4 0 0 0 2.4 10 12 15
To select the ith row, use iloc:
In [31]: df_test.iloc[0]
Out[31]:
ATime 1.2
X 2.0
Y 15.0
Z 2.0
Btime 1.2
C 12.0
D 25.0
E 12.0
Name: 0, dtype: float64
To select the ith value in the Btime column you could use:
In [30]: df_test['Btime'].iloc[0]
Out[30]: 1.2
There is a difference between df_test['Btime'].iloc[0] (recommended) and df_test.iloc[0]['Btime']:
DataFrames store data in column-based blocks (where each block has a single
dtype). If you select by column first, a view can be returned (which is
quicker than returning a copy) and the original dtype is preserved. In contrast,
if you select by row first, and if the DataFrame has columns of different
dtypes, then Pandas copies the data into a new Series of object dtype. So
selecting columns is a bit faster than selecting rows. Thus, although
df_test.iloc[0]['Btime'] works, df_test['Btime'].iloc[0] is a little bit
more efficient.
There is a big difference between the two when it comes to assignment.
df_test['Btime'].iloc[0] = x affects df_test, but df_test.iloc[0]['Btime']
may not. See below for an explanation of why. Because a subtle difference in
the order of indexing makes a big difference in behavior, it is better to use single indexing assignment:
df.iloc[0, df.columns.get_loc('Btime')] = x
df.iloc[0, df.columns.get_loc('Btime')] = x (recommended):
The recommended way to assign new values to a
DataFrame is to avoid chained indexing, and instead use the method shown by
andrew,
df.loc[df.index[n], 'Btime'] = x
or
df.iloc[n, df.columns.get_loc('Btime')] = x
The latter method is a bit faster, because df.loc has to convert the row and column labels to
positional indices, so there is a little less conversion necessary if you use
df.iloc instead.
df['Btime'].iloc[0] = x works, but is not recommended:
Although this works, it is taking advantage of the way DataFrames are currently implemented. There is no guarantee that Pandas has to work this way in the future. In particular, it is taking advantage of the fact that (currently) df['Btime'] always returns a
view (not a copy) so df['Btime'].iloc[n] = x can be used to assign a new value
at the nth location of the Btime column of df.
Since Pandas makes no explicit guarantees about when indexers return a view versus a copy, assignments that use chained indexing generally always raise a SettingWithCopyWarning even though in this case the assignment succeeds in modifying df:
In [22]: df = pd.DataFrame({'foo':list('ABC')}, index=[0,2,1])
In [24]: df['bar'] = 100
In [25]: df['bar'].iloc[0] = 99
/home/unutbu/data/binky/bin/ipython:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
self._setitem_with_indexer(indexer, value)
In [26]: df
Out[26]:
foo bar
0 A 99 <-- assignment succeeded
2 B 100
1 C 100
df.iloc[0]['Btime'] = x does not work:
In contrast, assignment with df.iloc[0]['bar'] = 123 does not work because df.iloc[0] is returning a copy:
In [66]: df.iloc[0]['bar'] = 123
/home/unutbu/data/binky/bin/ipython:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
In [67]: df
Out[67]:
foo bar
0 A 99 <-- assignment failed
2 B 100
1 C 100
Warning: I had previously suggested df_test.ix[i, 'Btime']. But this is not guaranteed to give you the ith value since ix tries to index by label before trying to index by position. So if the DataFrame has an integer index which is not in sorted order starting at 0, then using ix[i] will return the row labeled i rather than the ith row. For example,
In [1]: df = pd.DataFrame({'foo':list('ABC')}, index=[0,2,1])
In [2]: df
Out[2]:
foo
0 A
2 B
1 C
In [4]: df.ix[1, 'foo']
Out[4]: 'C'
Note that the answer from #unutbu will be correct until you want to set the value to something new, then it will not work if your dataframe is a view.
In [4]: df = pd.DataFrame({'foo':list('ABC')}, index=[0,2,1])
In [5]: df['bar'] = 100
In [6]: df['bar'].iloc[0] = 99
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas-0.16.0_19_g8d2818e-py2.7-macosx-10.9-x86_64.egg/pandas/core/indexing.py:118: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
self._setitem_with_indexer(indexer, value)
Another approach that will consistently work with both setting and getting is:
In [7]: df.loc[df.index[0], 'foo']
Out[7]: 'A'
In [8]: df.loc[df.index[0], 'bar'] = 99
In [9]: df
Out[9]:
foo bar
0 A 99
2 B 100
1 C 100
Another way to do this:
first_value = df['Btime'].values[0]
This way seems to be faster than using .iloc:
In [1]: %timeit -n 1000 df['Btime'].values[20]
5.82 µs ± 142 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)
In [2]: %timeit -n 1000 df['Btime'].iloc[20]
29.2 µs ± 1.28 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.iloc[0].head(1) - First data set only from entire first row.
df.iloc[0] - Entire First row in column.
In a general way, if you want to pick up the first N rows from the J column from pandas dataframe the best way to do this is:
data = dataframe[0:N][:,J]
To access a single value you can use the method iat that is much faster than iloc:
df['Btime'].iat[0]
You can also use the method take:
df['Btime'].take(0)
.iat and .at are the methods for getting and setting single values and are much faster than .iloc and .loc. Mykola Zotko pointed this out in their answer, but they did not use .iat to its full extent.
When we can use .iat or .at, we should only have to index into the dataframe once.
This is not great:
df['Btime'].iat[0]
It is not ideal because the 'Btime' column was first selected as a series, then .iat was used to index into that series.
These two options are the best:
Using zero-indexed positions:
df.iat[0, 4] # get the value in the zeroth row, and 4th column
Using Labels:
df.at[0, 'Btime'] # get the value where the index label is 0 and the column name is "Btime".
Both methods return the value of 1.2.
To get e.g the value from column 'test' and row 1 it works like
df[['test']].values[0][0]
as only df[['test']].values[0] gives back a array
Another way of getting the first row and preserving the index:
x = df.first('d') # Returns the first day. '3d' gives first three days.
According to pandas docs, at is the fastest way to access a scalar value such as the use case in the OP (already suggested by Alex on this page).
Building upon Alex's answer, because dataframes don't necessarily have a range index it might be more complete to index df.index (since dataframe indexes are built on numpy arrays, you can index them like an array) or call get_loc() on columns to get the integer location of a column.
df.at[df.index[0], 'Btime']
df.iat[0, df.columns.get_loc('Btime')]
One common problem is that if you used a boolean mask to get a single value, but ended up with a value with an index (actually a Series); e.g.:
0 1.2
Name: Btime, dtype: float64
you can use squeeze() to get the scalar value, i.e.
df.loc[df['Btime']<1.3, 'Btime'].squeeze()
Related
I have a very huge DataFrame with millions of rows and about 20-30 columns with various types of data, eg. string, num, dates, etc.
df
index t1 num1 float1 ... str2
0 2014-10-21 3456 0.000 ... ayzkcxtoScUy
1 2014-10-21 2453 0.000 ... jZygJWtxyVnS
... ... ... ... ... ...
n-1 2020-11-06 708735 670.818 ... UWVhmKCfmzVj
n 2020-11-06 70630 670.817 ... EvhreYZotqVS
let's just say it's pretty wild, but I need each and every row with all of its values. I now want to group on certain columns and eliminate groups and rows from the original DataFrame df based on the group size. In particular, I want to eliminate all groups with size 1.
FIRST NAIVE APPROACH
I searched and tried using this answer: How to select rows in Pandas dataframe where value appears more than once
lst = ["t1", "str1", "num1", "str2", "num2"]
df = df.groupby(lst).filter(lambda x: len(x.index) > 1).reset_index(drop=True)
which indeed works as expected. My DataFrame df is now filtered from all rows which occured in groups which had a size of 1. The problem is the time in relation to the dimensions of my DataFrame using the filter method, which takes just too long. To put this into perspective, grouping on these sample columns would yield about ~ 165,000 groups with a 2.5 million rows DataFrame, about a third of these groups have a size of 1. I had to abort the execution of this script, because it would take ages. I further tried using inspiration from this link How do I improve the performance of pandas GroupBy filter operation? but couldn't get it working with map, because I'm grouping on a DataFrame and not on a Series. Using the transform method, performance worsened.
SIDENOTE
Investigating further, I found out that there has been a problem which occured when using filter on a DataFrame which has datetime64[ns, UTC] and/or datetime64[ns] columns. I used Del df[x] to delete all three of these columns, which increased performance of the filter method by about a third. Still not enough, but significant enough to mention it here, especially when I need these columns and can't just delete them.
SECOND 'CLEVER' APPROACH
I then tried to use clever indexing of my data to circumvent the usage of groupby, filter or transform at all by using .value_counts() from link 1.
vc = df[lst].value_counts()
vc_index = vc.index[vc.gt(1)]
df = data[data[lst].isin(vc_index)]
I'm getting the value counts vc to locate all indices with a count of 1 and then create a MultiIndex new_index with only the wanted indices (which are count > 1). After that, I tried filtering my df with .isin() like in link 1, which set all values of df to NaN/NaT. I'm stuck here - I'm not sure what I did wrong here.
df
index t1 num1 float1 ... str2
0 NaT NaN NaN ... NaN
1 NaT NaN NaN ... NaN
... ... ... ... ... ...
n-1 NaT NaN NaN ... NaN
n NaT NaN NaN ... NaN
In another attempt, I tried using the pd.index.difference() method
vc = data[lst].value_counts()
df = data.set_index(keys=lst)
df.index = df.index.difference(other=vc.index[vc.gt(1)])
but this only gave me an TypeError: '<' not supported between instances of 'float' and 'str'.
Honestly, I'm kinda clueless whats best here, sitting on this problem since two days. I even thought about parallelization (maybe with Dask?), but I'm unsure how to use that since I've never worked with it. Help would be much appreciated.
For this specific use case (group with count > 1), duplicated is much faster:
df[df.duplicated(lst, keep=False)]
# 231 ms ± 10.2 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
Another option, not quite as fast, but significantly faster than filter and works in general is groupby().transform('size'):
df[df.groupby(lst)['t1'].transform('size')>1]
# 554 ms ± 108 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
Compared to:
df.groupby(lst).filter(lambda x: len(x) > 1)
# CPU times: user 38.8 s, sys: 482 ms, total: 39.3 s
# Wall time: 39.4 s
The solution from #Quang Hoang is working very good. Some benchmarks I've made with my dataset:
(rs = rows of df, ngrps = df.groupby(lst).ngroups)
method 100k rs/82.488 ngrps 200k rs/164.466 ngrps 400k rs/331.351 ngrps 800k rs/672.905 ngrps 1.600k rs/1.351.525 ngrps
duplicated 0:00:00.031236 0:00:00.078112 0:00:00.181825 0:00:00.331095 0:00:00.683959
transform 0:00:00.062507 0:00:00.109386 0:00:00.261506 0:00:00.528166 0:00:01.029606
filter 0:00:09.039214 0:00:18.422355 0:00:37.372117 0:01:15.531945 0:02:32.075144
It is scaling pretty well using duplicated, but be aware: If there are NaN values in columns which are inside your list (on which you want to group, lst in my example), duplicate will not drop them.
I'm trying to find the sum of price data in a data frame.
My current code looks like this, whitch there must be a better way to do it
i=5
a = 0
while i<10
a = a + df.loc[i]["Price"]
i = i + 1
averg = a/5
print(averg)
First note you should avoid chained indexing. It's ambiguous and explicitly discouraged in the docs, and instead you can use pd.DataFrame.at. In addition, you can use the += operator for incrementing a value. So you can rewrite as:
i = 5
a = 0
while i < 10
a += df.at[i, 'Price']
i += 1
avg = a/5
print(avg)
However, note you can use pd.DataFrame.loc to combine row and index labelling and give a pd.Series object. You can then use pd.Series.mean to calculate the average:
avg = df.loc[5:10, 'Price'].mean()
This way you are also taking advantage of vectorised computations as opposed to using a Python-level loop.
>>> import pandas as pd
>>> df = pd.DataFrame([[1,2,3],[4,5,6],[7,8,9]], columns=["Price", "Weight", "Size"])
>>> df
Price Weight Size
0 1 2 3
1 4 5 6
2 7 8 9
>>> df.mean()
Price 4
Weight 5
Size 6
dtype: int64
>>> df["Price"].mean()
4
Using loops on dataframes is very inefficient. Try using vector calculations whenever possible. Pandas already has a function mean() for the same.
If the index of that column is "Price" then you could just do the following -
df['Price'].mean()
There are many methods for creating new columns in Pandas (I may have missed some in my examples so please let me know if there are others and I will include here) and I wanted to figure out when is the best time to use each method. Obviously some methods are better in certain situations compared to others but I want to evaluate it from a holistic view looking at efficiency, readability, and usefulness.
I'm primarily concerned with the first three but included other ways simply to show it's possible with different approaches. Here's your sample dataframe:
df = pd.DataFrame({'a':[1,2,3],'b':[4,5,6]})
Most commonly known way is to name a new column such as df['c'] and use apply:
df['c'] = df['a'].apply(lambda x: x * 2)
df
a b c
0 1 4 2
1 2 5 4
2 3 6 6
Using assign can accomplish the same thing:
df = df.assign(c = lambda x: x['a'] * 2)
df
a b c
0 1 4 2
1 2 5 4
2 3 6 6
Updated via #roganjosh:
df['c'] = df['a'] * 2
df
a b c
0 1 4 2
1 2 5 4
2 3 6 6
Using map (definitely not as efficient as apply):
df['c'] = df['a'].map(lambda x: x * 2)
df
a b c
0 1 4 2
1 2 5 4
2 3 6 6
Creating a new pd.series and then concat to bring it into the dataframe:
c = pd.Series(df['a'] * 2).rename("c")
df = pd.concat([df,c], axis = 1)
df
a b c
0 1 4 2
1 2 5 4
2 3 6 6
Using join:
df.join(c)
a b c
0 1 4 2
1 2 5 4
2 3 6 6
Short answer: vectorized calls (df['c'] = 2 * df['a']) almost always win on both speed and readability. See this answer regarding what you can use as a "hierarchy" of options when it comes to performance.
In generally, if you have a for i in ... or lambda present somewhere in a Pandas operation, this (sometimes) means that the resulting calculations call Python code rather than the optimized C code that Pandas' Cython library relies on for vectorized operations. (Same goes for operations that rely on NumPy ufuncs for the underlying .values.)
As for .assign(), it is correctly pointed out in the comments that this creates a copy, whereas you can view df['c'] = 2 * df['a'] as the equivalent of setting a dictionary key/value. The former also takes twice as long, although this is perhaps a bit apples-to-orange because one operation is returning a DataFrame while the other is just assigning a column.
>>> %timeit df.assign(c=df['a'] * 2)
498 µs ± 15.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
>>> %timeit -r 7 -n 1000 df['c'] = df['a'] * 2
239 µs ± 22.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
As for .map(): generally you see this when, as the name implies, you want to provide a mapping for a Series (though it can be passed a function, as in your question). That doesn't mean it's not performant, it just tends to be used as a specialized method in cases that I've seen:
>>> df['a'].map(dict(enumerate('xyz', 1)))
0 x
1 y
2 z
Name: a, dtype: object
And as for .apply(): to inject a bit of opinion into the answer, I would argue it's more idiomatic to use vectorization where possible. You can see in the code for the module where .apply() is defined: because you are passing a lambda, not a NumPy ufunc, what ultimately gets called is technically a Cython function, map_infer, but it is still performing whatever function you passed on each individual member of the Series df['a'], one at a time.
A succinct way would be:
df['c'] = 2 * df['a']
No need to compute the new column elementwise.
Why are you using lambda function?
You can easily achieve the above-mentioned task easily by
df['c'] = 2 * df['a']
This will not increase the overhead.
With a numpy ndarray it is possible to write to multiple columns at a time without making a copy first (as long as they are adjacent). If I wanted to write to the first three columns of an array I would write
a[0,0:3] = 1,2,3 # this is very fast ('a' is a numpy ndarray)
I was hoping that in pandas I would similarly be able to select multiple adjacent columns by "label-slicing" like so (assuming the first 3 columns are labeled 'a','b','c')
a.loc[0,'a':'c'] = 1,2,3 # this works but is very slow ('a' is a pandas DataFrame)
or similarly
a.iloc[0,3:6] = 1,2,3 # this is equally as slow
However, this takes several 100s of milliseconds as compared to writing to a numpy array which takes only a few microseconds. I'm unclear on whether pandas is making a copy of the array under the hood. The only way I could find to write to the dataframe in this way that gives good speed is to work on the underlying ndarray directly
a.values[0,0:3] = 1,2,3 # this works fine and is fast
Have I missed something in the Pandas docs or is their no way to do multiple adjacent column indexing on a Pandas dataframe with speed comparable to numpy?
Edit
Here's the actual dataframe I am working with.
>> conn = sqlite3.connect('prath.sqlite')
>> prath = pd.read_sql("select image_id,pixel_index,skin,r,g,b from pixels",conn)
>> prath.shape
(5913307, 6)
>> prath.head()
image_id pixel_index skin r g b
0 21 113764 0 0 0 0
1 13 187789 0 183 149 173
2 17 535758 0 147 32 35
3 31 6255 0 116 1 16
4 15 119272 0 238 229 224
>> prath.dtypes
image_id int64
pixel_index int64
skin int64
r int64
g int64
b int64
dtype: object
Here is some runtime comparisons for the different indexing methods (again, pandas indexing is very slow)
>> %timeit prath.loc[0,'r':'b'] = 4,5,6
1 loops, best of 3: 888 ms per loop
>> %timeit prath.iloc[0,3:6] = 4,5,6
1 loops, best of 3: 894 ms per loop
>> %timeit prath.values[0,3:6] = 4,5,6
100000 loops, best of 3: 4.8 µs per loop
Edit to clarify: I don't believe pandas has a direct analog to setting a view in numpy in terms of both speed and syntax. iloc and loc are probably the most direct analog in terms of syntax and purpose, but are much slower. This is a fairly common situation with numpy and pandas. Pandas does a lot more than numpy (labeled columns/indexes, automatic alignment, etc.), but is slower to varying degrees. When you need speed and can do things in numpy, then do them in numpy.
I think in a nutshell that the tradeoff here is that loc and iloc will be slower but work 100% of the time whereas values will be fast but not always work (to be honest, I didn't even realize it would work in the way you got it to work).
But here's a really simple example where values doesn't work because column 'g' is a float rather than integer.
prath['g'] = 3.33
prath.values[0,3:6] = 4,5,6
prath.head(3)
image_id pixel_index skin r g b
0 21 113764 0 0 3.33 0
1 13 187789 0 183 3.33 173
2 17 535758 0 147 3.33 35
prath.iloc[0,3:6] = 4,5,6
prath.head(3)
image_id pixel_index skin r g b
0 21 113764 0 4 5.00 6
1 13 187789 0 183 3.33 173
2 17 535758 0 147 3.33 35
You can often get numpy-like speed and behavior from pandas when columns are of homogeneous type, you want to be careful about this. Edit to add: As #toes notes in the comment, the documentation does state that you can do this with homogeneous data. However, it's potentially very error prone as the example above shows, and I don't think many people would consider this a good general practice in pandas.
My general recommendation would be to do things in numpy in cases where you need the speed (and have homogeneous data types), and pandas when you don't. The nice thing is that numpy and pandas play well together so it's really not that hard to convert between dataframes and arrays as you go.
Edit to add: The following seems to work (albeit with a warning) even with column 'g' as a float. The speed is in between the values way and loc/iloc ways. I'm not sure if this can be expected to work all the time though. Just putting it out as a possible middle way.
prath[0:1][['r','g','b']] = 4,5,6
We are adding the ability to index directly even in a multi-dtype frame. This is in master now and will be in 0.17.0. You can do this in < 0.17.0, but it requires (more) manipulation of the internals.
In [1]: df = DataFrame({'A' : range(5), 'B' : range(6,11), 'C' : 'foo'})
In [2]: df.dtypes
Out[2]:
A int64
B int64
C object
dtype: object
The copy=False flag is new. This gives you a dict of dtypes->blocks (which are dtype separable)
In [3]: b = df.as_blocks(copy=False)
In [4]: b
Out[4]:
{'int64': A B
0 0 6
1 1 7
2 2 8
3 3 9
4 4 10, 'object': C
0 foo
1 foo
2 foo
3 foo
4 foo}
Here is the underlying numpy array.
In [5]: b['int64'].values
Out[5]:
array([[ 0, 6],
[ 1, 7],
[ 2, 8],
[ 3, 9],
[ 4, 10]])
This is the array in the original data set
In [7]: id(df._data.blocks[0].values)
Out[7]: 4429267232
Here is our view on it. They are the same
In [8]: id(b['int64'].values.base)
Out[8]: 4429267232
Now you can access the frame, and use pandas set operations to modify.
You can also directly access the numpy array via .values, which is now a VIEW into the original.
You will not incur any speed penalty for modifications as copies won't be made as long as you don't change the dtype of the data itself (e.g. don't try to put a string here; it will work but the view will be lost)
In [9]: b['int64'].loc[0,'A'] = -1
In [11]: b['int64'].values[0,1] = -2
Since we have a view, you can then change the underlying data.
In [12]: df
Out[12]:
A B C
0 -1 -2 foo
1 1 7 foo
2 2 8 foo
3 3 9 foo
4 4 10 foo
Note that if you modify the shape of the data (e.g. if you add a column for example) then the views will be lost.
The following table contains some keys and values:
N = 100
tbl = pd.DataFrame({'key':np.random.randint(0, 10, N),
'y':np.random.rand(N), 'z':np.random.rand(N)})
I would like to obtain a DataFrame in which each row contains a key and all the fields that correspond to the minimal value of a specified field.
Since the original table is very large, I'm interested in the most efficient way.
NOTE getting the minimal value of a field is simple:
tbl.groupby('key').agg(pd.Series.min)
But this takes the minimum values of every field, independently, I would like to know what is the minimum value of y and what z value corresponds to it.
Below I post an answer to my question with my naive approach, but I suspect there are better ways
Here is a straightforward approach:
gr = tbl.groupby('key')
def take_min_y(t):
ix = t.y.argmin()
return t.loc[[ix]]
tbl_mins = gr.apply(take_min_y)
Is there a better way?
Based on your updated edit I believe the following is what you want:
In [107]:
tbl.iloc[gr['y'].agg(pd.Series.idxmin)]
Out[107]:
key y z
47 0 0.094841 0.221435
26 1 0.062200 0.748082
45 2 0.032497 0.160199
28 3 0.002242 0.064829
73 4 0.122438 0.723844
75 5 0.128193 0.638933
79 6 0.071833 0.952624
86 7 0.058974 0.113317
36 8 0.068757 0.611111
12 9 0.082604 0.271268
idxmin returns the index of the min value, we can then use this to filter the original dataframe to select these rows.
Timings show this method is approx 7 times faster:
In [108]:
%timeit tbl.iloc[gr['y'].agg(pd.Series.idxmin)]
def take_min_y(t):
ix = t.y.argmin()
return t.loc[[ix]]
%timeit tbl_mins = gr.apply(take_min_y)
1000 loops, best of 3: 1.08 ms per loop
100 loops, best of 3: 7.06 ms per loop