this is my code
for i in range(len(df)-1):
df['ot'].iloc[i]=(df['Open'].iloc[i]-df['Open'].iloc[i+1])/df['Open'].iloc[i+1]
print(df['ot'])
Here the ot is new column just created and open is dervied in dataframe, while I try to print ot after assigning that to the formula I get keyerror.
Replace your loop by vectorization:
df['ot'] = df['Open'].diff(-1) / df['Open'].shift(-1)
print(df)
# Output
Open ot
1 2 -0.50 # (2 - 4) / 4 = -0.5
2 4 1.00 # (4 - 2) / 2 = 1
3 2 -0.75 # (2 - 8) / 8 = -0.75
4 8 NaN
It looks like pct_change:
df['ot'] = df['open'].pct_change(-1)
Using #Corralien's example:
Open ot
0 2 -0.50
1 4 1.00
2 2 -0.75
3 8 NaN
This question already has answers here:
Repeating each element of a numpy array 5 times
(2 answers)
Closed last year.
I'm using numpy.
I want to repeat numbers from 0 to 10000 five times.
Let me show you a simple example below.
0
0
0
0
0
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
9999
9999
9999
9999
9999
10000
10000
10000
10000
10000
How can I do this?
Use numpy.repeat with numpy.arange:
a = np.repeat(np.arange(10001), 5)
print (a)
[ 0 0 0 ... 10000 10000 10000]
I want to do some complex calculations in pandas while referencing previous values (basically I'm calculating row by row). However the loops take forever and I wanted to know if there was a faster way. Everybody keeps mentioning using shift but I don't understand how that would even work.
df = pd.DataFrame(index=range(500)
df["A"]= 2
df["B"]= 5
df["A"][0]= 1
for i in range(len(df):
if i != 0: df['A'][i] = (df['A'][i-1] / 3) - df['B'][i-1] + 25
numpy_ext can be used for expanding calculations
pandas-rolling-apply-using-multiple-columns for reference
I have also included a simpler calc to demonstrate behaviour in simpler way
df = pd.DataFrame(index=range(5000))
df["A"]= 2
df["B"]= 5
df["A"][0]= 1
import numpy_ext as npe
# for i in range(len(df):
# if i != 0: df['A'][i] = (df['A'][i-1] / 3) - df['B'][i-1] + 25
# SO example - function of previous values in A and B
def f(A,B):
r = np.sum(A[:-1]/3) - np.sum(B[:-1] + 25) if len(A)>1 else A[0]
return r
# much simpler example, sum of previous values
def g(A):
return np.sum(A[:-1])
df["AB_combo"] = npe.expanding_apply(f, 1, df["A"].values, df["B"].values)
df["A_running"] = npe.expanding_apply(g, 1, df["A"].values)
print(df.head(10).to_markdown())
sample output
A
B
AB_combo
A_running
0
1
5
1
0
1
2
5
-29.6667
1
2
2
5
-59
3
3
2
5
-88.3333
5
4
2
5
-117.667
7
5
2
5
-147
9
6
2
5
-176.333
11
7
2
5
-205.667
13
8
2
5
-235
15
9
2
5
-264.333
17
This question already has answers here:
Conditional Replace Pandas
(7 answers)
Closed 2 years ago.
I want to convert all values < 100 to 0 in column ODOMETER_FW. I have below DF:
When I use pandas:
stupid_values = fuel_GB['ODOMETER_FW'].replace(fuel_GB['ODOMETER_FW']<100,0)
fuel_GB['ODOMETER_FW'] = stupid_values
fuel_GB.head(13)
And the result as you can see, has some error and I really do not know why.
Use lambda function to convert values less than 100 to 0:
df['ODOMETER_FW'] = df['ODOMETER_FW'].apply(lambda x: 0 if x <100 else x)
print(df)
ODOMETER_FW
0 11833
1 0
2 9080
3 8878
4 0
5 14578
6 14351
7 0
8 13456
9 0
10 0
11 0
12 0
Just ask the modification for the relevant lines:
fuel_GB.loc[fuel_GB['ODOMETER_FW'] < 100, 'ODOMETER_FW'] = 0
Use this pandas code:
fuel_GB[fuel_GB['ODOMETER_FW'] < 100] = 0
I need to round prices in a column to different number of decimals in Python. I am using this code to create the dataframe, df_prices:
df_prices = pd.DataFrame({'InstrumentID':['001','002','003','004','005','006'], 'Price':[12.44,6.5673,23.999,56.88,4333.22,27.8901],'RequiredDecimals':[2,0,1,2,0,3]})
The data looks like this:
InstrumentID Price RequiredDecimals
1 12.444 2
2 6.5673 0
3 23.999 1
4 56.88 2
5 4333.22 0
6 27.8901 3
I often get this issue returned:
TypeError: cannot convert the series to
Neither of these statements worked:
df_prices['PriceRnd'] = np.round(df_prices['Price'] , df_prices['RequiredDecimals'])
df_prices['PriceRnd'] = df_prices['Price'].round(decimals = df_prices['RequiredDecimals'] )
This is what the final output should look like:
Instrument# Price RequiredDecimals PriceRnd
1 12.444 2 12.44
2 6.5673 0 7
3 23.999 1 24.0
4 56.88 2 56.88
5 4333.22 0 4333
6 27.8901 3 27.890
Couldn't find a better solution, but this one seems to work
df['Rnd'] = [np.around(x,y) for x,y in zip(df['Price'],df['RequiredDecimals'])]
Although not elegant, you can try this.
import pandas as pd
df_prices = pd.DataFrame({'InstrumentID':['001','002','003','004','005','006'], 'Price':[12.44,6.5673,23.999,56.88,4333.22,27.8901],'RequiredDecimals':[2,0,1,2,0,3]})
print(df_prices)
list1 = []
for i in df_prices.values:
list1.append('{:.{}f}' .format(i[1], i[2]))
print(list1)
df_prices["Rounded Price"] =list1
print(df_prices)
InstrumentID Price RequiredDecimals Rounded Price
0 001 12.4400 2 12.44
1 002 6.5673 0 7
2 003 23.9990 1 24.0
3 004 56.8800 2 56.88
4 005 4333.2200 0 4333
5 006 27.8901 3 27.890
or a 1-liner code
df_prices['Rnd'] = ['{:.{}f}' .format(x, y) for x,y inzip(df_prices['Price'],df_prices['RequiredDecimals'])]
An alternative way would be to adjust the number that you are trying to round with an appropriate factor and then use the fact that the .round()-function always rounds to the nearest integer.
df_prices['factor'] = 10**df_prices['RequiredDecimals']
df_prices['rounded'] = (df_prices['Price'] * df_prices['factor']).round() / df_prices['factor']
After rounding, the number is divided again by the factor.