how to solve IndexError : single positional indexer is out-of-bounds - python

CODE:-
from datetime import date
from datetime import timedelta
from nsepy import get_history
import pandas as pd
import datetime
# import matplotlib.pyplot as mp
end1 = date.today()
start1 = end1 - timedelta(days=365)
stock = [
'RELIANCE','HDFCBANK','INFY','ICICIBANK','HDFC','TCS','KOTAKBANK','LT','SBIN','HINDUNILVR','AXISBANK','ITC','BAJFINANCE','BHARTIARTL','ASIANPAINT','HCLTECH','MARUTI','TITAN','BAJAJFINSV','TATAMOTORS',
'TECHM','SUNPHARMA','TATASTEEL','M&M','WIPRO','ULTRACEMCO','POWERGRID','HINDALCO','NTPC','NESTLEIND','GRASIM','ONGC','JSWSTEEL','HDFCLIFE','INDUSINDBK','SBILIFE','DRREDDY','ADANIPORTS','DIVISLAB','CIPLA',
'BAJAJ-AUTO','TATACONSUM','UPL','BRITANNIA','BPCL','EICHERMOT','HEROMOTOCO','COALINDIA','SHREECEM','IOC','VEDL','ADANIENT', 'APOLLOHOSP', 'TATAPOWER', 'PIDILITIND', 'SRF', 'NAUKRI', 'ICICIGI', 'DABUR',
'GODREJCP', 'HAVELLS', 'PEL', 'VOLTAS', 'AUBANK', 'LTI', 'CHOLAFIN', 'AMBUJACEM', 'MARICO', 'SRTRANSFIN','GAIL', 'MCDOWELL-N', 'MPHASIS', 'MINDTREE', 'PAGEIND', 'ZEEL', 'BEL', 'TRENT', 'CROMPTON', 'JUBLFOOD',
'DLF', 'SBICARD', 'SIEMENS', 'BANDHANBNK', 'IRCTC', 'LAURUSLABS', 'PIIND', 'INDIGO', 'INDUSTOWER','ICICIPRULI', 'MOTHERSON', 'AARTIIND', 'FEDERALBNK', 'BANKBARODA', 'PERSISTENT', 'HINDPETRO', 'ACC',
'AUROPHARMA', 'COLPAL', 'GODREJPROP', 'MFSL', 'LUPIN', 'BIOCON', 'ASHOKLEY', 'BHARATFORG', 'BERGEPAINT','JINDALSTEL', 'ASTRAL', 'IEX', 'NMDC', 'CONCOR', 'INDHOTEL', 'BALKRISIND', 'PETRONET', 'CANBK', 'ALKEM',
'DIXON', 'DEEPAKNTR', 'DALBHARAT', 'TVSMOTOR', 'ATUL', 'HDFCAMC', 'TATACOMM', 'MUTHOOTFIN', 'TATACHEM','SAIL', 'IDFCFIRSTB', 'PFC', 'BOSCHLTD', 'MRF', 'NAVINFLUOR', 'CUMMINSIND', 'IGL', 'IPCALAB', 'COFORGE',
'ESCORTS', 'TORNTPHARM', 'LTTS', 'RECLTD', 'LICHSGFIN', 'BATAINDIA', 'HAL', 'PNB', 'GUJGASLTD', 'UBL','3MINDIA','ABB','AIAENG','APLAPOLLO','AARTIDRUGS','AAVAS','ABBOTINDIA','ADANIGREEN','ATGL','ABCAPITAL',
'ABFRL','ABSLAMC','ADVENZYMES','AEGISCHEM','AFFLE','AJANTPHARM','ALKYLAMINE','ALLCARGO','AMARAJABAT','AMBER','ANGELONE','ANURAS','APTUS','ASAHIINDIA','ASTERDM','ASTRAZEN','AVANTIFEED','DMART','BASF',
'BSE','BAJAJELEC','BAJAJHLDNG','BALAMINES','BALRAMCHIN','BANKINDIA','MAHABANK','BAYERCROP','BDL','BEL','BHEL','BIRLACORPN','BSOFT','BLUEDART','BLUESTARCO','BORORENEW','BOSCHLTD','BRIGADE','BCG','MAPMYINDIA'
]
target_stocks_list = []
target_stocks = pd.DataFrame()
for stock in stock:
vol = get_history(symbol=stock,
start=start1,
end=end1)
d_vol = pd.concat([vol['Deliverable Volume']])
symbol_s = pd.concat([vol['Symbol']])
close = pd.concat([vol['Close']])
df = pd.DataFrame(symbol_s)
df['D_vol'] = d_vol
# print(df)
cond = df['D_vol'].iloc[-1] > max(df['D_vol'].iloc[-91:-1])
if(cond):
target_stocks_list.append(stock)
target_stocks = pd.concat([target_stocks, df])
print(target_stocks_list)
file_name = f'{datetime.datetime.now().day}-{datetime.datetime.now().month}-{datetime.datetime.now().year}.csv'
target_stocks.to_csv(f'D:/HUGE VOLUME SPURTS/first 250/SEP 2022/{file_name}')
pd.set_option('display.max_columns',10)
pd.set_option('display.max_rows',2000)
print(target_stocks)
ERROR:-
C:\python\Python310\python.exe "C:/Users/Yogesh_PC/PycharmProjects/future oi data analysis/trial2.py"
Traceback (most recent call last):
File "C:\Users\Yogesh_PC\PycharmProjects\future oi data analysis\trial2.py", line 64, in <module>
cond = df['D_vol'].iloc[-1] > max(df['D_vol'].iloc[-91:-1])
File "C:\python\Python310\lib\site-packages\pandas\core\indexing.py", line 967, in __getitem__
return self._getitem_axis(maybe_callable, axis=axis)
File "C:\python\Python310\lib\site-packages\pandas\core\indexing.py", line 1520, in _getitem_axis
self._validate_integer(key, axis)
File "C:\python\Python310\lib\site-packages\pandas\core\indexing.py", line 1452, in _validate_integer
raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds
Process finished with exit code 1
Above code gives the historical stock data of Indian stock market. The data is updated on website after market closed around 8:00PM to 9:00PM daily. Then I run my code. For most of the days my code gives output without any error but frequently it throws an error which showed above.
There are around 150-200 stocks in my code. This error occurs because some time exchange do not update the data of one or two stocks from the above list that is why this error comes.
So please post the code which will skip the particular one or two stocks which are not updated and should give the output for rest all stocks.
for example:- stocks = ['DLF', 'SBICARD', 'SIEMENS', 'BANDHANBNK', 'IRCTC', 'LAURUSLABS', 'PIIND',
'INDIGO', 'INDUSTOWER','ICICIPRULI', 'MOTHERSON']
in above stocks suppose exchange didn't update the data of 'IRCTC' and rest all stocks are up to date then due to 'IRCTC' my code throws error and it is not showing data which is updated.
Thank you.


The "out-of-bounds" error indicates you're trying to access a part of the dataframe series that doesn't exist. It's most likely caused by df['D_vol'] being less than 90 items long when you try to do
df['D_vol'].iloc[-91:-1]
Edit:
add a length check before the offending line:
if df['D_vol'].size > 90:
cond = df['D_vol'].iloc[-1] > max(df['D_vol'].iloc[-91:-1])
if(cond):
target_stocks_list.append(stock)
target_stocks = pd.concat([target_stocks, df])

Related

TypeError when fitting Statsmodels OLS with standard errors clustered 2 ways

Context
Building on top of How to run Panel OLS regressions with 3+ fixed-effect and errors clustering? and notably Josef's third comment, I am trying to adapt the OLS Coefficients and Standard Errors Clustered by Firm and Year section of this example notebook below:
cluster_2ways_ols = sm.ols(formula='y ~ x', data=df).fit(cov_type='cluster',
cov_kwds={'groups': np.array(df[['firmid', 'year']])},
use_t=True)
to my own example dataset.
Note that I am able to reproduce this example (and it works). I can also add fixed-effects, by using 'y ~ x + C(firmid) + C(year)' as formula instead.
Problem
However, trying to port the same command to my example dataset (see code below), I'm getting the following error:
>>> model = sm.OLS.from_formula("gdp ~ population + C(year_publication) + C(country)", df)
>>> result = model.fit(
cov_type='cluster',
cov_kwds={'groups': np.array(df[['country', 'year_publication']])},
use_t=True
)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/path/venv/lib64/python3.10/site-packages/statsmodels/regression/linear_model.py", line 343, in fit
lfit = OLSResults(
File "/path/venv/lib64/python3.10/site-packages/statsmodels/regression/linear_model.py", line 1607, in __init__
self.get_robustcov_results(cov_type=cov_type, use_self=True,
File "/path/venv/lib64/python3.10/site-packages/statsmodels/regression/linear_model.py", line 2568, in get_robustcov_results
res.cov_params_default = sw.cov_cluster_2groups(
File "/path/venv/lib64/python3.10/site-packages/statsmodels/stats/sandwich_covariance.py", line 591, in cov_cluster_2groups
combine_indices(group)[0],
File "/path/venv/lib64/python3.10/site-packages/statsmodels/tools/grouputils.py", line 55, in combine_indices
groups_ = groups.view([('', groups.dtype)] * groups.shape[1])
File "/path/venv/lib64/python3.10/site-packages/numpy/core/_internal.py", line 549, in _view_is_safe
raise TypeError("Cannot change data-type for object array.")
TypeError: Cannot change data-type for object array.
I have tried to manually cast the year_publication to string/object using np.array(df[['country', 'year_publication']].astype("str")), but it doesn't solve the issue.
Questions
What is the cause of the TypeError()?
How to adapt the example command to my dataset?
Minimal Working Example
from io import StringIO
import numpy as np
import pandas as pd
import statsmodels.api as sm
DATA = """
"continent","country","source","year_publication","year_data","population","gdp"
"Africa","Angola","OECD",2020,2018,972,52.69
"Africa","Angola","OECD",2020,2019,986,802.7
"Africa","Angola","OECD",2020,2020,641,568.74
"Africa","Angola","OECD",2021,2018,438,168.83
"Africa","Angola","OECD",2021,2019,958,310.57
"Africa","Angola","OECD",2021,2020,270,144.02
"Africa","Angola","OECD",2022,2018,528,359.71
"Africa","Angola","OECD",2022,2019,974,582.98
"Africa","Angola","OECD",2022,2020,835,820.49
"Africa","Angola","IMF",2020,2018,168,148.85
"Africa","Angola","IMF",2020,2019,460,236.21
"Africa","Angola","IMF",2020,2020,360,297.15
"Africa","Angola","IMF",2021,2018,381,249.13
"Africa","Angola","IMF",2021,2019,648,128.05
"Africa","Angola","IMF",2021,2020,206,179.05
"Africa","Angola","IMF",2022,2018,282,150.29
"Africa","Angola","IMF",2022,2019,125,23.42
"Africa","Angola","IMF",2022,2020,410,247.35
"Africa","Angola","WorldBank",2020,2018,553,182.06
"Africa","Angola","WorldBank",2020,2019,847,698.87
"Africa","Angola","WorldBank",2020,2020,844,126.61
"Africa","Angola","WorldBank",2021,2018,307,239.76
"Africa","Angola","WorldBank",2021,2019,659,510.73
"Africa","Angola","WorldBank",2021,2020,548,331.89
"Africa","Angola","WorldBank",2022,2018,448,122.76
"Africa","Angola","WorldBank",2022,2019,768,761.41
"Africa","Angola","WorldBank",2022,2020,324,163.57
"Africa","Benin","OECD",2020,2018,513,196.9
"Africa","Benin","OECD",2020,2019,590,83.7
"Africa","Benin","OECD",2020,2020,791,511.09
"Africa","Benin","OECD",2021,2018,799,474.43
"Africa","Benin","OECD",2021,2019,455,234.21
"Africa","Benin","OECD",2021,2020,549,238.83
"Africa","Benin","OECD",2022,2018,235,229.33
"Africa","Benin","OECD",2022,2019,347,46.51
"Africa","Benin","OECD",2022,2020,532,392.13
"Africa","Benin","IMF",2020,2018,138,137.05
"Africa","Benin","IMF",2020,2019,978,239.82
"Africa","Benin","IMF",2020,2020,821,33.41
"Africa","Benin","IMF",2021,2018,453,291.93
"Africa","Benin","IMF",2021,2019,526,381.88
"Africa","Benin","IMF",2021,2020,467,313.57
"Africa","Benin","IMF",2022,2018,948,555.23
"Africa","Benin","IMF",2022,2019,323,289.91
"Africa","Benin","IMF",2022,2020,421,62.35
"Africa","Benin","WorldBank",2020,2018,983,271.69
"Africa","Benin","WorldBank",2020,2019,138,23.55
"Africa","Benin","WorldBank",2020,2020,636,623.65
"Africa","Benin","WorldBank",2021,2018,653,534.99
"Africa","Benin","WorldBank",2021,2019,564,368.8
"Africa","Benin","WorldBank",2021,2020,741,312.02
"Africa","Benin","WorldBank",2022,2018,328,292.11
"Africa","Benin","WorldBank",2022,2019,653,429.21
"Africa","Benin","WorldBank",2022,2020,951,242.73
"Africa","Chad","OECD",2020,2018,176,95.06
"Africa","Chad","OECD",2020,2019,783,425.34
"Africa","Chad","OECD",2020,2020,885,461.6
"Africa","Chad","OECD",2021,2018,673,15.87
"Africa","Chad","OECD",2021,2019,131,74.46
"Africa","Chad","OECD",2021,2020,430,61.58
"Africa","Chad","OECD",2022,2018,593,211.34
"Africa","Chad","OECD",2022,2019,647,550.37
"Africa","Chad","OECD",2022,2020,154,105.65
"Africa","Chad","IMF",2020,2018,160,32.41
"Africa","Chad","IMF",2020,2019,654,27.84
"Africa","Chad","IMF",2020,2020,616,468.92
"Africa","Chad","IMF",2021,2018,996,22.4
"Africa","Chad","IMF",2021,2019,126,93.18
"Africa","Chad","IMF",2021,2020,879,547.87
"Africa","Chad","IMF",2022,2018,663,520
"Africa","Chad","IMF",2022,2019,681,544.76
"Africa","Chad","IMF",2022,2020,101,55.6
"Africa","Chad","WorldBank",2020,2018,786,757.22
"Africa","Chad","WorldBank",2020,2019,599,593.69
"Africa","Chad","WorldBank",2020,2020,641,529.84
"Africa","Chad","WorldBank",2021,2018,343,287.89
"Africa","Chad","WorldBank",2021,2019,438,340.83
"Africa","Chad","WorldBank",2021,2020,762,594.67
"Africa","Chad","WorldBank",2022,2018,430,128.69
"Africa","Chad","WorldBank",2022,2019,260,242.59
"Africa","Chad","WorldBank",2022,2020,607,216.1
"Europe","Denmark","OECD",2020,2018,114,86.75
"Europe","Denmark","OECD",2020,2019,937,373.29
"Europe","Denmark","OECD",2020,2020,866,392.93
"Europe","Denmark","OECD",2021,2018,296,41.04
"Europe","Denmark","OECD",2021,2019,402,32.67
"Europe","Denmark","OECD",2021,2020,306,7.88
"Europe","Denmark","OECD",2022,2018,540,379.51
"Europe","Denmark","OECD",2022,2019,108,26.72
"Europe","Denmark","OECD",2022,2020,752,307.2
"Europe","Denmark","IMF",2020,2018,157,24.24
"Europe","Denmark","IMF",2020,2019,303,79.04
"Europe","Denmark","IMF",2020,2020,286,122.36
"Europe","Denmark","IMF",2021,2018,569,69.32
"Europe","Denmark","IMF",2021,2019,808,642.67
"Europe","Denmark","IMF",2021,2020,157,5.58
"Europe","Denmark","IMF",2022,2018,147,112.21
"Europe","Denmark","IMF",2022,2019,414,311.16
"Europe","Denmark","IMF",2022,2020,774,230.46
"Europe","Denmark","WorldBank",2020,2018,695,350.03
"Europe","Denmark","WorldBank",2020,2019,511,209.84
"Europe","Denmark","WorldBank",2020,2020,181,29.27
"Europe","Denmark","WorldBank",2021,2018,503,176.89
"Europe","Denmark","WorldBank",2021,2019,710,609.02
"Europe","Denmark","WorldBank",2021,2020,264,165.78
"Europe","Denmark","WorldBank",2022,2018,670,638.99
"Europe","Denmark","WorldBank",2022,2019,651,354.6
"Europe","Denmark","WorldBank",2022,2020,632,623.94
"Europe","Estonia","OECD",2020,2018,838,263.67
"Europe","Estonia","OECD",2020,2019,638,533.95
"Europe","Estonia","OECD",2020,2020,898,638.73
"Europe","Estonia","OECD",2021,2018,262,98.16
"Europe","Estonia","OECD",2021,2019,569,552.54
"Europe","Estonia","OECD",2021,2020,868,252.48
"Europe","Estonia","OECD",2022,2018,927,264.65
"Europe","Estonia","OECD",2022,2019,205,150.6
"Europe","Estonia","OECD",2022,2020,828,752.61
"Europe","Estonia","IMF",2020,2018,841,176.31
"Europe","Estonia","IMF",2020,2019,614,230.55
"Europe","Estonia","IMF",2020,2020,500,41.19
"Europe","Estonia","IMF",2021,2018,510,169.68
"Europe","Estonia","IMF",2021,2019,765,401.85
"Europe","Estonia","IMF",2021,2020,751,319.6
"Europe","Estonia","IMF",2022,2018,314,58.81
"Europe","Estonia","IMF",2022,2019,155,2.24
"Europe","Estonia","IMF",2022,2020,734,187.6
"Europe","Estonia","WorldBank",2020,2018,332,160.17
"Europe","Estonia","WorldBank",2020,2019,466,385.33
"Europe","Estonia","WorldBank",2020,2020,487,435.06
"Europe","Estonia","WorldBank",2021,2018,461,249.19
"Europe","Estonia","WorldBank",2021,2019,932,763.38
"Europe","Estonia","WorldBank",2021,2020,650,463.91
"Europe","Estonia","WorldBank",2022,2018,570,549.97
"Europe","Estonia","WorldBank",2022,2019,909,80.48
"Europe","Estonia","WorldBank",2022,2020,523,242.22
"Europe","Finland","OECD",2020,2018,565,561.64
"Europe","Finland","OECD",2020,2019,646,161.62
"Europe","Finland","OECD",2020,2020,194,133.69
"Europe","Finland","OECD",2021,2018,529,39.76
"Europe","Finland","OECD",2021,2019,800,680.12
"Europe","Finland","OECD",2021,2020,418,399.19
"Europe","Finland","OECD",2022,2018,591,253.12
"Europe","Finland","OECD",2022,2019,457,272.58
"Europe","Finland","OECD",2022,2020,157,105.1
"Europe","Finland","IMF",2020,2018,860,445.03
"Europe","Finland","IMF",2020,2019,108,47.72
"Europe","Finland","IMF",2020,2020,523,500.58
"Europe","Finland","IMF",2021,2018,560,81.47
"Europe","Finland","IMF",2021,2019,830,664.64
"Europe","Finland","IMF",2021,2020,903,762.62
"Europe","Finland","IMF",2022,2018,179,167.73
"Europe","Finland","IMF",2022,2019,137,98.98
"Europe","Finland","IMF",2022,2020,666,524.86
"Europe","Finland","WorldBank",2020,2018,319,146.01
"Europe","Finland","WorldBank",2020,2019,401,219.56
"Europe","Finland","WorldBank",2020,2020,711,45.35
"Europe","Finland","WorldBank",2021,2018,828,20.97
"Europe","Finland","WorldBank",2021,2019,180,66.3
"Europe","Finland","WorldBank",2021,2020,682,92.57
"Europe","Finland","WorldBank",2022,2018,254,81.2
"Europe","Finland","WorldBank",2022,2019,619,159.08
"Europe","Finland","WorldBank",2022,2020,191,184.4
"""
df = pd.read_csv(StringIO(DATA))
model = sm.OLS.from_formula("gdp ~ population + C(year_publication) + C(country)", df)
result = model.fit(
cov_type='cluster',
cov_kwds={'groups': np.array(df[['country', 'year_publication']])},
use_t=True
)
print(result.summary())

I have realized that the groups must be an array of integers rather than of objects/strings.
Thus, label encoding the string column as follows:
df["country"] = df["country"].astype("category")
df["country_id"] = df.country.cat.codes
and using country_id to cluster the standard errors solves the issue:
result = model.fit(
cov_type='cluster',
cov_kwds={'groups': np.array(df[['country_id', 'year_publication']])},
use_t=True
)
Fully working example:
from io import StringIO
import numpy as np
import pandas as pd
import statsmodels.api as sm
DATA = """
"continent","country","source","year_publication","year_data","population","gdp"
"Africa","Angola","OECD",2020,2018,972,52.69
"Africa","Angola","OECD",2020,2019,986,802.7
"Africa","Angola","OECD",2020,2020,641,568.74
"Africa","Angola","OECD",2021,2018,438,168.83
"Africa","Angola","OECD",2021,2019,958,310.57
"Africa","Angola","OECD",2021,2020,270,144.02
"Africa","Angola","OECD",2022,2018,528,359.71
"Africa","Angola","OECD",2022,2019,974,582.98
"Africa","Angola","OECD",2022,2020,835,820.49
"Africa","Angola","IMF",2020,2018,168,148.85
"Africa","Angola","IMF",2020,2019,460,236.21
"Africa","Angola","IMF",2020,2020,360,297.15
"Africa","Angola","IMF",2021,2018,381,249.13
"Africa","Angola","IMF",2021,2019,648,128.05
"Africa","Angola","IMF",2021,2020,206,179.05
"Africa","Angola","IMF",2022,2018,282,150.29
"Africa","Angola","IMF",2022,2019,125,23.42
"Africa","Angola","IMF",2022,2020,410,247.35
"Africa","Angola","WorldBank",2020,2018,553,182.06
"Africa","Angola","WorldBank",2020,2019,847,698.87
"Africa","Angola","WorldBank",2020,2020,844,126.61
"Africa","Angola","WorldBank",2021,2018,307,239.76
"Africa","Angola","WorldBank",2021,2019,659,510.73
"Africa","Angola","WorldBank",2021,2020,548,331.89
"Africa","Angola","WorldBank",2022,2018,448,122.76
"Africa","Angola","WorldBank",2022,2019,768,761.41
"Africa","Angola","WorldBank",2022,2020,324,163.57
"Africa","Benin","OECD",2020,2018,513,196.9
"Africa","Benin","OECD",2020,2019,590,83.7
"Africa","Benin","OECD",2020,2020,791,511.09
"Africa","Benin","OECD",2021,2018,799,474.43
"Africa","Benin","OECD",2021,2019,455,234.21
"Africa","Benin","OECD",2021,2020,549,238.83
"Africa","Benin","OECD",2022,2018,235,229.33
"Africa","Benin","OECD",2022,2019,347,46.51
"Africa","Benin","OECD",2022,2020,532,392.13
"Africa","Benin","IMF",2020,2018,138,137.05
"Africa","Benin","IMF",2020,2019,978,239.82
"Africa","Benin","IMF",2020,2020,821,33.41
"Africa","Benin","IMF",2021,2018,453,291.93
"Africa","Benin","IMF",2021,2019,526,381.88
"Africa","Benin","IMF",2021,2020,467,313.57
"Africa","Benin","IMF",2022,2018,948,555.23
"Africa","Benin","IMF",2022,2019,323,289.91
"Africa","Benin","IMF",2022,2020,421,62.35
"Africa","Benin","WorldBank",2020,2018,983,271.69
"Africa","Benin","WorldBank",2020,2019,138,23.55
"Africa","Benin","WorldBank",2020,2020,636,623.65
"Africa","Benin","WorldBank",2021,2018,653,534.99
"Africa","Benin","WorldBank",2021,2019,564,368.8
"Africa","Benin","WorldBank",2021,2020,741,312.02
"Africa","Benin","WorldBank",2022,2018,328,292.11
"Africa","Benin","WorldBank",2022,2019,653,429.21
"Africa","Benin","WorldBank",2022,2020,951,242.73
"Africa","Chad","OECD",2020,2018,176,95.06
"Africa","Chad","OECD",2020,2019,783,425.34
"Africa","Chad","OECD",2020,2020,885,461.6
"Africa","Chad","OECD",2021,2018,673,15.87
"Africa","Chad","OECD",2021,2019,131,74.46
"Africa","Chad","OECD",2021,2020,430,61.58
"Africa","Chad","OECD",2022,2018,593,211.34
"Africa","Chad","OECD",2022,2019,647,550.37
"Africa","Chad","OECD",2022,2020,154,105.65
"Africa","Chad","IMF",2020,2018,160,32.41
"Africa","Chad","IMF",2020,2019,654,27.84
"Africa","Chad","IMF",2020,2020,616,468.92
"Africa","Chad","IMF",2021,2018,996,22.4
"Africa","Chad","IMF",2021,2019,126,93.18
"Africa","Chad","IMF",2021,2020,879,547.87
"Africa","Chad","IMF",2022,2018,663,520
"Africa","Chad","IMF",2022,2019,681,544.76
"Africa","Chad","IMF",2022,2020,101,55.6
"Africa","Chad","WorldBank",2020,2018,786,757.22
"Africa","Chad","WorldBank",2020,2019,599,593.69
"Africa","Chad","WorldBank",2020,2020,641,529.84
"Africa","Chad","WorldBank",2021,2018,343,287.89
"Africa","Chad","WorldBank",2021,2019,438,340.83
"Africa","Chad","WorldBank",2021,2020,762,594.67
"Africa","Chad","WorldBank",2022,2018,430,128.69
"Africa","Chad","WorldBank",2022,2019,260,242.59
"Africa","Chad","WorldBank",2022,2020,607,216.1
"Europe","Denmark","OECD",2020,2018,114,86.75
"Europe","Denmark","OECD",2020,2019,937,373.29
"Europe","Denmark","OECD",2020,2020,866,392.93
"Europe","Denmark","OECD",2021,2018,296,41.04
"Europe","Denmark","OECD",2021,2019,402,32.67
"Europe","Denmark","OECD",2021,2020,306,7.88
"Europe","Denmark","OECD",2022,2018,540,379.51
"Europe","Denmark","OECD",2022,2019,108,26.72
"Europe","Denmark","OECD",2022,2020,752,307.2
"Europe","Denmark","IMF",2020,2018,157,24.24
"Europe","Denmark","IMF",2020,2019,303,79.04
"Europe","Denmark","IMF",2020,2020,286,122.36
"Europe","Denmark","IMF",2021,2018,569,69.32
"Europe","Denmark","IMF",2021,2019,808,642.67
"Europe","Denmark","IMF",2021,2020,157,5.58
"Europe","Denmark","IMF",2022,2018,147,112.21
"Europe","Denmark","IMF",2022,2019,414,311.16
"Europe","Denmark","IMF",2022,2020,774,230.46
"Europe","Denmark","WorldBank",2020,2018,695,350.03
"Europe","Denmark","WorldBank",2020,2019,511,209.84
"Europe","Denmark","WorldBank",2020,2020,181,29.27
"Europe","Denmark","WorldBank",2021,2018,503,176.89
"Europe","Denmark","WorldBank",2021,2019,710,609.02
"Europe","Denmark","WorldBank",2021,2020,264,165.78
"Europe","Denmark","WorldBank",2022,2018,670,638.99
"Europe","Denmark","WorldBank",2022,2019,651,354.6
"Europe","Denmark","WorldBank",2022,2020,632,623.94
"Europe","Estonia","OECD",2020,2018,838,263.67
"Europe","Estonia","OECD",2020,2019,638,533.95
"Europe","Estonia","OECD",2020,2020,898,638.73
"Europe","Estonia","OECD",2021,2018,262,98.16
"Europe","Estonia","OECD",2021,2019,569,552.54
"Europe","Estonia","OECD",2021,2020,868,252.48
"Europe","Estonia","OECD",2022,2018,927,264.65
"Europe","Estonia","OECD",2022,2019,205,150.6
"Europe","Estonia","OECD",2022,2020,828,752.61
"Europe","Estonia","IMF",2020,2018,841,176.31
"Europe","Estonia","IMF",2020,2019,614,230.55
"Europe","Estonia","IMF",2020,2020,500,41.19
"Europe","Estonia","IMF",2021,2018,510,169.68
"Europe","Estonia","IMF",2021,2019,765,401.85
"Europe","Estonia","IMF",2021,2020,751,319.6
"Europe","Estonia","IMF",2022,2018,314,58.81
"Europe","Estonia","IMF",2022,2019,155,2.24
"Europe","Estonia","IMF",2022,2020,734,187.6
"Europe","Estonia","WorldBank",2020,2018,332,160.17
"Europe","Estonia","WorldBank",2020,2019,466,385.33
"Europe","Estonia","WorldBank",2020,2020,487,435.06
"Europe","Estonia","WorldBank",2021,2018,461,249.19
"Europe","Estonia","WorldBank",2021,2019,932,763.38
"Europe","Estonia","WorldBank",2021,2020,650,463.91
"Europe","Estonia","WorldBank",2022,2018,570,549.97
"Europe","Estonia","WorldBank",2022,2019,909,80.48
"Europe","Estonia","WorldBank",2022,2020,523,242.22
"Europe","Finland","OECD",2020,2018,565,561.64
"Europe","Finland","OECD",2020,2019,646,161.62
"Europe","Finland","OECD",2020,2020,194,133.69
"Europe","Finland","OECD",2021,2018,529,39.76
"Europe","Finland","OECD",2021,2019,800,680.12
"Europe","Finland","OECD",2021,2020,418,399.19
"Europe","Finland","OECD",2022,2018,591,253.12
"Europe","Finland","OECD",2022,2019,457,272.58
"Europe","Finland","OECD",2022,2020,157,105.1
"Europe","Finland","IMF",2020,2018,860,445.03
"Europe","Finland","IMF",2020,2019,108,47.72
"Europe","Finland","IMF",2020,2020,523,500.58
"Europe","Finland","IMF",2021,2018,560,81.47
"Europe","Finland","IMF",2021,2019,830,664.64
"Europe","Finland","IMF",2021,2020,903,762.62
"Europe","Finland","IMF",2022,2018,179,167.73
"Europe","Finland","IMF",2022,2019,137,98.98
"Europe","Finland","IMF",2022,2020,666,524.86
"Europe","Finland","WorldBank",2020,2018,319,146.01
"Europe","Finland","WorldBank",2020,2019,401,219.56
"Europe","Finland","WorldBank",2020,2020,711,45.35
"Europe","Finland","WorldBank",2021,2018,828,20.97
"Europe","Finland","WorldBank",2021,2019,180,66.3
"Europe","Finland","WorldBank",2021,2020,682,92.57
"Europe","Finland","WorldBank",2022,2018,254,81.2
"Europe","Finland","WorldBank",2022,2019,619,159.08
"Europe","Finland","WorldBank",2022,2020,191,184.4
"""
df = pd.read_csv(StringIO(DATA))
df["country"] = df["country"].astype("category")
df["country_id"] = df.country.cat.codes
model = sm.OLS.from_formula("gdp ~ population + C(year_publication) + C(country)", df)
result = model.fit(
cov_type='cluster',
cov_kwds={'groups': np.array(df[['country_id', 'year_publication']])},
use_t=True
)
print(result.summary())

Issues Querying and Downloading Sentinel-3 OLCI Data with Sentinelsat

I am working with Sentinel-3 OLCI Level-2 Data Products with the Sentinelsat API and am having issues querying and exceeding my data download quota. Overall, I would like to write a program that accepts a date range and a specific geographic location, then downloads a dataframe of all values in the "Oa04_radiance"-band within the specified dates for that location. This is what I have so far:
from sentinelsat import SentinelAPI, read_geojson, geojson_to_wkt
from datetime import date
from geojson import Feature, Point, Polygon
api = SentinelAPI('user', 'password', 'https://apihub.copernicus.eu/apihub')
lon = -123.312383
lat = 49.319269
my_point = Point((lon, lat))
footprint = geojson_to_wkt(my_point)
products = api.query(footprint,
date=(date(2021, 1, 1), date(2021, 6, 15)),
platformname='Sentinel-3',
producttype='OL_2_LRR___',
cloudcoverpercentage=(0, 80))
products_df = api.to_dataframe(products)
api.download_all(products_df.index)
Error Output:
Traceback (most recent call last):
File "C:/Users/t7dej/Desktop/Turbid Time Local/SentSat/SenSat_mdl.py", line 48, in <module>
api.download_all(products_df_sorted.index)
File "E:\Software\Anaconda\lib\site-packages\sentinelsat\sentinel.py", line 723, in download_all
is_online = not self.trigger_offline_retrieval(pid)
File "E:\Software\Anaconda\lib\site-packages\sentinelsat\sentinel.py", line 636, in trigger_offline_retrieval
raise LTAError(msg, r)
sentinelsat.exceptions.LTAError: HTTP status 403 Forbidden: User quota exceeded: MediaRegulationException : An exception occured while creating a stream: Maximum number of 4 concurrent flows achieved by the user
Even when I set my api.query(limit=1) I am receiving this error message. The products_df is 173 MB and has a geometry column with a value of :
MULTIPOLYGON (((-146.081 -49.2196, -145.768 -48.2668, -145.201 -46.4727, -144.658 -44.6765, -144.135 -42.8782, -143.63 -41.0787, -143.142 -39.2767, -142.667 -37.4733, -142.204 -35.6694, -141.753 -33.863, -141.312 -32.0559, -140.878 -30.2474, -140.453 -28.4377, -140.033 -26.6268, -139.62 -24.8151, -139.211 -23.0024, -138.806 -21.1887, -138.404 -19.3744, -138.006 -17.5593, -137.609 -15.7434, -137.213 -13.927, -136.819 -12.1101, -136.425 -10.2928, -136.031 -8.47512, -135.636 -6.65724, -135.24 -4.83889, -134.843 -3.02078, -134.443 -1.2025, -134.04 0.61575, -133.634 2.43352, -133.224 4.25148, -132.81 6.06894, -132.391 7.88578, -131.965 9.702120000000001, -131.534 11.5179, -131.095 13.3329, -130.649 15.1468, -130.194 16.9598, -129.729 18.7716, -129.253 20.582, -128.767 22.3916, -128.267 24.1992, -127.753 26.0043, -127.224 27.8086, -126.678 29.6103, -126.113 31.4098, -125.527 33.2067, -124.919 35.0007, -124.286 36.7919, -123.624 38.5795, -122.932 40.3637, -122.205 42.1436, -121.44 43.9188, -120.631 45.6892, -119.773 47.4541, -118.861 49.2124, -117.886 50.9639, -116.841 52.7073, -115.714 54.4414, -114.495 56.1652, -113.168 57.8769, -111.717 59.5747, -110.12 61.256, -108.352 62.9179, -106.382 64.5568, -104.173 66.16800000000001, -101.677 67.7456, -98.8378 69.2821, -95.5855 70.7672, -91.83369999999999 72.1889, -87.4884 73.5312, -82.44029999999999 74.7715, -76.5813 75.8849, -69.83199999999999 76.83880000000001, -62.1773 77.59399999999999, -53.7161 78.1165, -44.6976 78.3725, -35.5051 78.3451, -26.5677 78.03619999999999, -18.2457 77.46639999999999, -10.7612 76.6694, -4.18714 75.6829, 3.64587789990069e-15 74.8448206506109, 1.50843 74.5429, 6.41354 73.28100000000001, 10.6367 71.9226, 14.2845 70.4875, 17.4519 68.99160000000001, 20.2206 67.4469, 21.6077 67.77589999999999, 23.0264 68.09050000000001, 24.4837 68.3925, 25.9603 68.679, 27.4944 68.9546, 29.0662 69.2161, 30.6951 69.46680000000001, 32.3417 69.6985, 34.024 69.9145, 35.7316 70.10760000000001, 37.4774 70.2903, 39.2531 70.4558, 41.0752 70.6046, 42.9054 70.7343, 44.7579 70.8455, 46.6297 70.938, 48.5174 71.01139999999999, 50.4176 71.0655, 52.3019 71.10290000000001, 52.0832 72.8877, 51.8771 74.6721, 51.6876 76.4562, 51.5212 78.2398, 51.3914 80.0231, 51.3212 81.80629999999999, 51.3582 83.58880000000001, 51.57959163346614 85.05115000000001, 3.911836325497215e-15 85.05115000000001, -133.5599156744917 85.05115000000001, -133.35 83.92870000000001, -133.276 82.14530000000001, -133.33 80.3614, -133.453 78.57859999999999, -133.611 76.7949, -133.799 75.0104, -134.002 73.226, -134.218 71.4409, -134.445 69.6553, -134.678 67.8693, -134.918 66.0826, -135.163 64.2954, -135.413 62.5074, -135.666 60.719, -135.923 58.93, -136.183 57.1401, -136.447 55.3495, -136.712 53.5582, -136.982 51.7664, -137.253 49.9738, -137.528 48.1799, -137.805 46.3858, -138.085 44.5911, -138.368 42.7954, -138.653 40.9991, -138.942 39.2019, -139.234 37.4039, -139.528 35.6058, -139.826 33.8069, -140.128 32.0072, -140.433 30.207, -140.741 28.4055, -141.054 26.6057, -141.37 24.8042, -141.691 23.0012, -142.016 21.1992, -142.346 19.3968, -142.68 17.5942, -143.02 15.7915, -143.365 13.9887, -143.716 12.1859, -144.073 10.3832, -144.436 8.58104, -144.806 6.77937, -145.183 4.97715, -145.567 3.17639, -145.96 1.37623, -146.361 -0.423407, -146.77 -2.22189, -147.19 -4.01968, -147.62 -5.81594, -148.06 -7.61083, -148.512 -9.40438, -148.976 -11.1962, -149.454 -12.9862, -149.946 -14.7742, -150.453 -16.5599, -150.975 -18.343, -151.516 -20.1234, -152.076 -21.901, -152.655 -23.6751, -153.257 -25.4461, -153.883 -27.2129, -154.534 -28.9754, -155.214 -30.7335, -155.923 -32.487, -156.667 -34.2331, -157.447 -35.9749, -158.267 -37.7106, -159.131 -39.4374, -160.043 -41.1566, -161.008 -42.867, -162.033 -44.5672, -162.604 -45.4664, -161.805 -45.7119, -160.989 -45.9615, -160.165 -46.2051, -159.335 -46.4427, -158.497 -46.6742, -157.652 -46.8994, -156.79 -47.1211, -155.932 -47.3333, -155.068 -47.539, -154.207 -47.742, -153.329 -47.9343, -152.444 -48.1198, -151.538 -48.3011, -150.642 -48.4725, -149.738 -48.6368, -148.842 -48.7908, -147.928 -48.9408, -147.008 -49.0835, -146.081 -49.2196)))
I have specified a geojson point object in the products query and am wondering why it is returning such a large multipolygon object in the products_df. I am thinking this is why products_df is so large and my quota is exceeded. Does anyone have any recommendations for this? Also, is it possible to query only the specific band 'Oa04_radiance' before downloading since I do not need any of the other bands from the Sentinel-3 OLCI Level-2 data products.

How do I compare the product dates to today's date and create a new column where it indicates whether they have expired or not?

RM
RACK
CROP
Towers
Transplant date
harvest
harvest date
GR1
R1a
LLQ
7
2021-4-03
21
2021-4-24
O: overdue
H: need to clear by today
N: not due yet
Today's date: 2021-04-29
This is my code
import pandas as pd
import time
from datetime import date
df = pd.read_csv("CGA Towers Spatial.csv")
growth = {'LLQ': 21, 'AZK': 24, 'JER': 21, 'MUS': 35, 'CHA': 28, 'SOR': 28, 'ICE': 35, 'TMD': 21}
df['HARVEST'] = df['CROP'].apply(lambda x:growth[x])
df['HARVEST DATE'] = df['TRANSPLANT DATE'] = pd.to_datetime(df['TRANSPLANT DATE'])
df['HARVEST DATE'] = df.apply(lambda x: x['TRANSPLANT DATE'] + pd.offsets.DateOffset(days=x['HARVEST']), 1)
df.to_csv('CGA Towers Spatial.csv',index=False)
df.set_index('RM', inplace=True)
pd.set_option('display.max_rows', 258)
current_date = time.strptime(pd.to_datetime(date.today()).strftime('%d/%m/%Y'), '%d/%m/%Y')
for index in df.index:
date_check = time.strptime(pd.to_datetime(df.loc[index, 'HARVEST DATE']).strftime('%d/%m/%Y'), '%d/%m/%Y')
if current_date >= date_check:
df.loc[index, 'check'] = 'H'
print(df)
This is the error.
Traceback (most recent call last):
File "/Users/kohjiayu/Desktop/PycharmProjects/towers/tower_add_harvest.py", line 21, in
date_check = time.strptime(pd.to_datetime(df.loc[index, 'HARVEST DATE']).strftime('%d/%m/%Y'), '%d/%m/%Y')
File "/Users/kohjiayu/.conda/envs/towers/lib/python3.9/site-packages/pandas/core/generic.py", line 5465, in getattr
return object.getattribute(self, name)
AttributeError: 'Series' object has no attribute 'strftime'
I would like to add a new column which indicates whether my product has expired/needs to be cleared/not yet but i have no idea how to get the csv mod to read these dates and add indications. Ive tried but there is error. Pls help, I'm new to python.

import time
df = pd.read_csv(file)
df.loc[(df['manufacture date'] == time.strftime('%Y-%m-%d')), 'expired/ due today/ not yet'] = 'H'
df.loc[(df['manufacture date'] < time.strftime('%Y-%m-%d')),'expired/ due today/ not yet'] = 'O'
df.loc[(df['manufacture date'] > time.strftime('%Y-%m-%d')),'expired/ due today/ not yet'] = 'N'

'can only join iterable' when fetching data using tia.bbg.datamgr (Python 2.7)

I'm writing a script that fetches data from Bloomberg using the TIA toolkit. I'm trying to place the PX_VALUE from the start date for each equity in stocks in a dictionary called dict1 so that i can manipulate those values later.
Here is my script so far without the calculations:
from __future__ import division
import numpy as np
import pandas as pd
import datetime
import tia
import tia.bbg.datamgr as dm
from operator import itemgetter
start = datetime.date(2017, 1, 3)
end = datetime.date(2017, 7, 25)
diffdays = ((end - start).days)/365
resolution = 0.01
diff2dp = int(np.round(diffdays/resolution))*resolution
diff = 1/diff2dp
dict1 = {}
stocks = ('GOOGL US Equity','MSFT US Equity', 'IBM US Equity')
mgr = dm.BbgDataManager()
eqt = mgr[stocks]
for eq in eqt:
df = eq.get_historical(['PX_LAST'], start, end)
k = df.loc[start]['PX_LAST']
dict1 [stocks] = k
print dict1
And here is the actual Output:
Traceback (most recent call last):
File "C:\Users\bloomberg\Desktop\examples\CAGR by LouisV2 BROKEN.py", line 23, in <module>
for eq in eqt:
File "C:\Python27\lib\site-packages\tia\bbg\datamgr.py", line 94, in __getitem__
return self.get_attributes(flds, **self.overrides)
File "C:\Python27\lib\site-packages\tia\bbg\datamgr.py", line 90, in get_attributes
frame = self.mgr.get_attributes(self.sids, flds, **overrides)
File "C:\Python27\lib\site-packages\tia\bbg\datamgr.py", line 148, in get_attributes
return self.terminal.get_reference_data(sids, flds, **overrides).as_frame()
File "C:\Python27\lib\site-packages\tia\bbg\v3api.py", line 745, in get_reference_data
return self.execute(req)
File "C:\Python27\lib\site-packages\tia\bbg\v3api.py", line 711, in execute
self.logger.info('executing request: %s' % repr(request))
File "C:\Python27\lib\site-packages\tia\bbg\v3api.py", line 432, in __repr__
fields=','.join(self.fields),
TypeError: can only join an iterable
>>>
I have also written a script that works for 1 equity with the calculations:
from __future__ import division
import numpy as np
import pandas as pd
import datetime
import tia
import tia.bbg.datamgr as dm
start = datetime.date(2017, 1, 3)
end = datetime.date(2017, 7, 25)
diffdays = ((end - start).days)/365
resolution = 0.01
diff2dp = int(np.round(diffdays/resolution))*resolution
diff = 1/diff2dp
mgr = dm.BbgDataManager()
eqt = mgr['GOOGL US Equity']
datafetch = eqt.get_historical(['PX_LAST'], start, end)
calc1 = ((datafetch.loc[end]['PX_LAST'])/(datafetch.loc[start]['PX_LAST']))
calc2 = (pow(calc1,diff))-1
calc22dp = int(np.round(calc2/resolution))*resolution
print calc22dp

Your single-security solution does this:
eqt = mgr['GOOGL US Equity']
but your multiple-security solution does (in effect) this:
eqt = mgr[('GOOGL US Equity','MSFT US Equity', 'IBM US Equity')]
Now, I obviously cannot test this without a Bloomberg installation, but it is clear from the error message that your problem is with eqt. Are you 100% sure you can pass a tuple of BBG ids as a key to dm.BbgDataManager()? The results you are getting suggests that you can't.
Follow the line of your working one-security solution, but looping through the stocks of interest:
stocks = ('GOOGL US Equity','MSFT US Equity', 'IBM US Equity')
mgr = dm.BbgDataManager()
for stock in stocks:
eqt = mgr[stock]
datafetch = eqt.get_historical(['PX_LAST'], start, end)
calc1 = ((datafetch.loc[end]['PX_LAST'])/(datafetch.loc[start]['PX_LAST']))
calc2 = (pow(calc1,diff))-1
calc22dp = int(np.round(calc2/resolution))*resolution
print calc22dp

How can get ' USDJPY'(currency rates) with pandas and yahoo finance?

I am learning and using the pandas and python.
Today, I am trying to make a fx rate table,
but I got a trouble with getting the pricess of 'USDJPY'.
When I get a prices of 'EUR/USD', i code like this.
eur = web.DataReader('EURUSD=X','yahoo')['Adj Close']
it works.
But when I wrote
jpy = web.DataReader('USDJPY=X','yahoo')['Adj Close']
the error message comes like this:
--------------------------------------------------------------------------- IOError Traceback (most recent call
last) in ()
----> 1 jpy = web.DataReader('USDJPY=X','yahoo')['Adj Close']
C:\Anaconda\lib\site-packages\pandas\io\data.pyc in DataReader(name,
data_source, start, end, retry_count, pause)
70 return get_data_yahoo(symbols=name, start=start, end=end,
71 adjust_price=False, chunksize=25,
---> 72 retry_count=retry_count, pause=pause)
73 elif data_source == "google":
74 return get_data_google(symbols=name, start=start, end=end,
C:\Anaconda\lib\site-packages\pandas\io\data.pyc in
get_data_yahoo(symbols, start, end, retry_count, pause, adjust_price,
ret_index, chunksize, name)
388 """
389 return _get_data_from(symbols, start, end, retry_count, pause,
--> 390 adjust_price, ret_index, chunksize, 'yahoo', name)
391
392
C:\Anaconda\lib\site-packages\pandas\io\data.pyc in
_get_data_from(symbols, start, end, retry_count, pause, adjust_price, ret_index, chunksize, source, name)
334 # If a single symbol, (e.g., 'GOOG')
335 if isinstance(symbols, (basestring, int)):
--> 336 hist_data = src_fn(symbols, start, end, retry_count, pause)
337 # Or multiple symbols, (e.g., ['GOOG', 'AAPL', 'MSFT'])
338 elif isinstance(symbols, DataFrame):
C:\Anaconda\lib\site-packages\pandas\io\data.pyc in
_get_hist_yahoo(sym, start, end, retry_count, pause)
188 '&g=d' +
189 '&ignore=.csv')
--> 190 return _retry_read_url(url, retry_count, pause, 'Yahoo!')
191
192
C:\Anaconda\lib\site-packages\pandas\io\data.pyc in
_retry_read_url(url, retry_count, pause, name)
167
168 raise IOError("after %d tries, %s did not "
--> 169 "return a 200 for url %r" % (retry_count, name, url))
170
171
IOError: after 3 tries, Yahoo! did not return a 200 for url
'http://ichart.yahoo.com/table.csv?s=USDJPY=X&a=0&b=1&c=2010&d=1&e=1&f=2014&g=d&ignore=.csv'
Other currencies like 'GBPUSD' also have same problem.
Can you solve this problem?
Do you have any idea of getting 'USDJPY' from yahoo or google???

Yahoo Finance doesn't provide historical data on exchange rates (i.e. there's no "Historical Prices" link in the top left of the page like there would be for stocks, indices, etc...)
You can use FRED (Federal Reserve of St. Louis data) to get these exchange rates...
import pandas.io.data as web
jpy = web.DataReader('DEXJPUS', 'fred')
UPDATE: hase moved the pandas-datareader
from pandas_datareader import data
jpy = data.DataReader('DEXJPUS', 'fred')
or the more direct way...
jpy = web.get_data_fred('DEXJPUS')
A list of all of the exchange rate that FRED has daily data for can be found here: http://research.stlouisfed.org/fred2/categories/94

Yahoo Finance doesn't provide historical data on exchange rates
Yes it does but not on cross rates. All vs the USD
List of Yahoo USD Exchange Rates
a = web.DataReader("JPY=X", 'yahoo')

The free and easy way is Yahoo:
# get fx rates
# https://finance.yahoo.com/currencies
# example EUR/USD = EURUSD%3DX?p=EURUSD%3DX
import pandas as pd
import pandas_datareader as dr
# change date range here
start_date = '2021-02-26'
end_date = '2021-03-01'
# retrieve market data of current ticker symbol
print('This is the table with HLOC, Volume, Adj Close prices')
eurusd = dr.data.DataReader('EURUSD%3DX', data_source='yahoo', start=start_date, end=end_date)
print(eurusd)
# just get latest adjusted close for further use
print('This is the Adj Close prices only')
print(eurusd['Adj Close'])
and it also works with other crosses, contrary to the above statements:
# EURCHF%3DX
eurchf = dr.data.DataReader('EURCHF%3DX', data_source='yahoo', start=start_date, end=end_date)
print(eurchf)

Get the historical exchange rates from OANDA
http://pandas-datareader.readthedocs.io/en/latest/remote_data.html
In [1]: from pandas_datareader.oanda import get_oanda_currency_historical_rates
In [2]: start, end = "2016-01-01", "2016-06-01"
In [3]: quote_currency = "USD"
In [4]: base_currency = ["EUR", "GBP", "JPY"]
In [5]: df_rates = get_oanda_currency_historical_rates(
start, end,
quote_currency=quote_currency,
base_currency=base_currency
)
In [6]: print(df_rates)
Update: Oanda started charging for this lately
https://www.oanda.com/fx-for-business/exchange-rates-api

#!pip install yfinance
#!pip install mplfinance
from datetime import datetime
import yfinance as yf
import mplfinance as mpf
#import pandas as pd
#import pandas_datareader as dr
# change date range here
start_date = '2021-02-26'
end_date = '2021-03-01'
#This Does NOT WORK#
# retrieve market data of current ticker symbol
print('This is the table with HLOC, Volume, Adj Close prices')
eurusd = dr.data.DataReader('EURUSD%3DX', data_source='yahoo',
start=start_date, end=end_date)
print(eurusd)
#This Does#
data = yf.download('USDCAD=X', start=start_date, end=end_date)
#If someone can figure out how to get the S5,S30, M1, M3 etc. Please share

I think you can use custom intervals by passing it as an argument to the yf.download() function. For example:
data = yf.download('USDCAD=X', start=start_date, end=end_date, interval='1m')

Categories

Resources