Develop Reference

Pandas dataframe selecting with index and condition on a column

I am trying for a while to solve this problem:
I have a daraframe like this:
import pandas as pd
df=pd.DataFrame(np.array([['A', 2, 3], ['B', 5, 6], ['C', 8, 9]]),columns=['a', 'b', 'c'])
j=[0,2]
But then when i try to select just a part of it filtering by a list of index and a condition on a column I get error...
df[df.loc[j]['a']=='A']
There is somenting wrong, but i don't get what is the problem here. Can you help me?
This is the error message:
IndexingError: Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match).

There is filtered DataFrame compared by original, so indices are different, so error is raised.
You need compare filtered DataFrame:
df1 = df.loc[j]
print (df1)
a b c
0 A 2 3
2 C 8 9
out = df1[df1['a']=='A']
print(out)
a b c
0 A 2 3
Your solution is possible use with convert ndices of filtered mask by original indices by Series.reindex:
out = df[(df.loc[j, 'a']=='A').reindex(df.index, fill_value=False)]
print(out)
a b c
0 A 2 3
Or nicer solution:
out = df[(df['a'] == 'A') & (df.index.isin(j))]
print(out)
a b c
0 A 2 3

A boolean array and the dataframe should be the same length. here your df length is 3 but the boolean array df.loc[j]['a']=='A' length is 2
You should do:
>>> df.loc[j][df.loc[j]['a']=='A']
a b c
0 A 2 3

How to conditionally change pandas DataFrame values into f-strings?

I have a pandas DataFrame whose values I want to conditionally change into strings without looping over every value.
Example input:
In [1]: df = pd.DataFrame(data = [[1,2], [4,5]], columns = ['a', 'b'])
Out[2]:
a b
0 1 2
1 4 5
This is my best attempt which doesn't work properly
df['a'] = np.where(df['a'] < 3, f'string-{df["a"]}', df['a'])
In [1]: df
Out[2]:
a b
0 string0 1\n1 4\nName: a, dtype: int64 2
1 4 5
Desired output:
Out[2]:
A B
0 string-1 2
1 4 5
I am using np.where() since looping is not feasible due to the size of the actual DataFrame. The actual f-string I am using is also more complex and has two variables that include column names, but the problem is the same.
Are there other ways to conditionally change pandas values into f-strings without looping over each value?

You can use .map() together with f-string, as follows:
df['a'] = df['a'].map(lambda x: f'string-{x}' if x < 3 else x)
Alternatively, you can also use .loc together with string concatenation, as follows:
df.loc[df['a'] < 3, 'a'] = 'string-' + df['a'].astype(str)
#OR
df['a']=np.where(df['a'] < 3, 'string-'+df['a'].astype(str), df['a'])
Result:
print(df)
a b
0 string-1 2
1 4 5

Calling a specific range of rows in a python database with specific columns

I'm looking to select a certain range of rows [25:100] and a certain list of indexed columns [1,3,6] from a python pandas dataframe using the subscript option.
So far I am using the following
df[25:100][[1, 3, 6]]

Use the .iloc (“location by integer”) attribute:
df.iloc[25:100, [1, 3, 6]]
Note that 25:100 select zero-based numbered rows from 25 (inclusive) to 100 (exclusive). If you want to select the row 100, too, use 25:101 instead.

The df.loc will do the task. However, for simple copies, there are other ways.
Import pandas
>>> import pandas as pd
Create dataframe
>>> df = pd.DataFrame({"A": [1, 2, 3], "B": ["a", "b", "c"]})
>>> df
A B
0 1 a
1 2 b
2 3 c
Copy rows from one column only
>>> df1 = df["B"][1:]
>>> df1
1 b
2 c
Name: B, dtype: object
Copy rows from more than one row
>>> df2 = df[["A","B"]][1:]
>>> df2
A B
1 2 b
2 3 c
Copy specific rows and columns (df.loc)
>>> df3 = df.loc[[0,2] , ["A", "B"]]
>>> df3
A B
0 1 a
2 3 c
>>>

Python equivalence of R's match() for indexing

So i essentially want to implement the equivalent of R's match() function in Python, using Pandas dataframes - without using a for-loop.
In R match() returns a vector of the positions of (first) matches of its first argument in its second.
Let's say that I have two df A and B, of which both include the column C. Where
A$C = c('a','b')
B$C = c('c','c','b','b','c','b','a','a')
In R we would get
match(A$C,B$C) = c(7,3)
What is an equivalent method in Python for columns in pandas data frames, that doesn't require looping through the values.

Here is a one liner:
B.reset_index().set_index('c').loc[A.c, 'index'].values
This solution returns the results in the same order as the input A, as match does in R, so it is a better equivalent than #jezrael's answer, because
Full example:
A = pd.DataFrame({'c':['a','b']})
B = pd.DataFrame({'c':['c','c','b','b','c','b','a','a']})
B.reset_index().set_index('c').loc[A.c, 'index'].values
Output array([6, 2])

You can use first drop_duplicates and then boolean indexing with isin or merge.
Python counts from 0, so for same output add 1.
A = pd.DataFrame({'c':['a','b']})
B = pd.DataFrame({'c':['c','c','b','b','c','b','a','a']})
B = B.drop_duplicates('c')
print (B)
c
0 c
2 b
6 a
print (B[B.c.isin(A.c)])
c
2 b
6 a
print (B[B.c.isin(A.c)].index)
Int64Index([2, 6], dtype='int64')
print (pd.merge(B.reset_index(), A))
index c
0 2 b
1 6 a
print (pd.merge(B.reset_index(), A)['index'])
0 2
1 6
Name: index, dtype: int64

This gives all the indices that are matched (with python's 0 based indexing):
import pandas as pd
df1 = pd.DataFrame({'C': ['a','b']})
print df1
C
0 a
1 b
df2 = pd.DataFrame({'C': ['c','c','b','b','c','b','a','a']})
print df2
C
0 c
1 c
2 b
3 b
4 c
5 b
6 a
7 a
match = df2['C'].isin(df1['C'])
print [i for i in range(match.shape[0]) if match[i]]
#[2, 3, 5, 6, 7]

Getting the integer index of a Pandas DataFrame row fulfilling a condition?

I have the following DataFrame:
a b c
b
2 1 2 3
5 4 5 6
As you can see, column b is used as an index. I want to get the ordinal number of the row fulfilling ('b' == 5), which in this case would be 1.
The column being tested can be either an index column (as with b in this case) or a regular column, e.g. I may want to find the index of the row fulfilling ('c' == 6).

Use Index.get_loc instead.
Reusing #unutbu's set up code, you'll achieve the same results.
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame(np.arange(1,7).reshape(2,3),
columns = list('abc'),
index=pd.Series([2,5], name='b'))
>>> df
a b c
b
2 1 2 3
5 4 5 6
>>> df.index.get_loc(5)
1

You could use np.where like this:
import pandas as pd
import numpy as np
df = pd.DataFrame(np.arange(1,7).reshape(2,3),
columns = list('abc'),
index=pd.Series([2,5], name='b'))
print(df)
# a b c
# b
# 2 1 2 3
# 5 4 5 6
print(np.where(df.index==5)[0])
# [1]
print(np.where(df['c']==6)[0])
# [1]
The value returned is an array since there could be more than one row with a particular index or value in a column.

With Index.get_loc and general condition:
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame(np.arange(1,7).reshape(2,3),
columns = list('abc'),
index=pd.Series([2,5], name='b'))
>>> df
a b c
b
2 1 2 3
5 4 5 6
>>> df.index.get_loc(df.index[df['b'] == 5][0])
1

The other answers based on Index.get_loc() do not provide a consistent result, because this function will return in integer if the index consists of all unique values, but it will return a boolean mask array if the index does not consist of unique values. A more consistent approach to return a list of integer values every time would be the following, with this example shown for an index with non-unique values:
df = pd.DataFrame([
{"A":1, "B":2}, {"A":2, "B":2},
{"A":3, "B":4}, {"A":1, "B":3}
], index=[1,2,3,1])
If searching based on index value:
[i for i,v in enumerate(df.index == 1) if v]
[0, 3]
If searching based on a column value:
[i for i,v in enumerate(df["B"] == 2) if v]
[0, 1]