Trying to retrieve data from the Anbima API - python

I'm trying to automate a process in which i have to download some brazilian fund quotes from Anbima (Brazil regulator). I have been able to work around the first steps to retrieve the access token but i don't know how to use the token in order to make requests. Here is the tutorial website https://developers.anbima.com.br/en/como-acessar-nossas-apis/.
I have tried a lot of thing but all i get from the request is 'Could not find a required APP in the request, identified by HEADER client_id.'
If someone could share some light. Thank you in advance.
import requests
import base64
import json
requests.get("https://api.anbima.com.br/feed/fundos/v1/fundos")
ClientID = '2Xy1ey11****'
ClientSecret = 'faStF1Hc****'
codeString = ClientID + ":" + ClientSecret
codeStringBytes = codeString.encode('ascii')
base64CodeBytes = base64.b64encode(codeStringBytes)
base64CodeString = base64CodeBytes.decode('ascii')
url = "https://api.anbima.com.br/oauth/access-token"
headers = {
'content-type': 'application/json'
,'authorization': f'Basic {base64CodeString}'
}
body = {
"grant_type": "client_credentials"
}
r = requests.post(url=url, data=json.dumps(body), headers=headers, allow_redirects=True)
jsonDict = r.json()
##################
urlFundos = "https://api-sandbox.anbima.com.br/feed/precos-indices/v1/titulos-publicos/mercado-secundario-TPF"
token = jsonDict['access_token']
headers2 = {
'content-type': 'application/json'
,'authorization': f'Bearer {token}'
}
r2 = requests.get(url=urlFundos, headers=headers2)
r2.status_code
r2.text

I was having the same problem, but today I could advance. I believe you need to adjust some parameters in the header.
Follows the piece of code I developed.
from bs4 import BeautifulSoup
import requests
PRODUCTION_URL = 'https://api.anbima.com.br'
SANDBOX_URL = 'https://api-sandbox.anbima.com.br'
API_URL = '/feed/fundos/v1/fundos/'
CODIGO_FUNDO = '594733'
PRODUCTION = False
if PRODUCTION:
URL = PRODUCTION_URL
else:
URL = SANDBOX_URL
URL = URL + API_URL + CODIGO_FUNDO
HEADER = {'access_token': 'your token',
'client_id' : 'your client ID'}
html = requests.get(URL, headers=HEADER).content
soup = BeautifulSoup(html, 'html.parser')
print(soup.prettify())
The sandbox API will return a dummy JSON. To access the production API you will need to request access (I'm trying to do this just now).

url = 'https://api.anbima.com.br/oauth/access-token'
http = 'https://api-sandbox.anbima.com.br/feed/precos-indices/v1/titulos-publicos/pu-intradiario'
client_id = "oLRa*******"
client_secret = "6U2nefG*****"
client_credentials = "oLRa*******:6U2nefG*****"
client_credentials = client_credentials.encode('ascii')
senhabytes = base64.b64encode(client_credentials)
senha = base64.b64decode(senhabytes)
print(senhabytes, senha)
body = {
"grant_type": "client_credentials"
}
headers = {
'content-type': 'application/json',
'Authorization': 'Basic b0xSYTJFSUlOMWR*********************'
}
request = requests.post(url, headers=headers, json=body, allow_redirects=True)
informacoes = request.json()
token = informacoes['access_token']
headers2 = {
"content-type": "application/json",
"client_id": f"{client_id}",
"access_token": f"{token}"
}
titulos = requests.get(http, headers=headers2)
titulos = fundos.json()
I used your code as a model, then I've made some changes. I've printed the encode client_id:client_secret and then I've copied and pasted in the headers.
I've changed the data for json.

Related

Pinterst api: not able to get access token - Stuck in Oauth flow

Using v5 of the pinterest api and stuck on the authentication flow: https://developers.pinterest.com/docs/getting-started/authentication/
Completed the first step and got the access code.
However, I get the below error when I try to use this code to get the access token.
{"code":1,"message":"Missing request body"}
Here is my code:
client_id= 'my_client_id'
client_secret = 'my_client_secret'
data_string = f'{client_id}:{client_secret}'
token = base64.b64encode(data_string.encode())
headers = {
'Authorization': 'Basic ' + token.decode('utf-8'),
'Content-Type': 'application/x-www-form-urlencoded',
}
url = "https://api.pinterest.com/v5/oauth/token"
code = "my_code_that_i_got_in_the_first_step"
params = {
'grant_type':'authorization_code',
'code': code,
'redirect_url':'https://my_redirect_uri'
}
r = requests.post(url, headers=headers, params=params)
print(r.json())
Below is the correct way to get the access token:
client_id= 'my_client_id'
client_secret = 'my_client_secret'
data_string = f'{client_id}:{client_secret}'
token = base64.b64encode(data_string.encode())
headers = {
'Authorization': 'Basic ' + token.decode('utf-8'),
'Content-Type': 'application/x-www-form-urlencoded',
}
url = "https://api.pinterest.com/v5/oauth/token"
code = "my_code_that_i_got_in_the_first_step"
data= {
'grant_type':'authorization_code',
'code': code,
'redirect_uri':'https://my_redirect_uri'
}
r = requests.post(url, headers=headers, data=data)
print(r.json())
In my question, I had mistyped redirect_uri as redirect_url. Also, when sending a POST, you should use data instead of params. See the comment by Amos Baker.

How to get user details after login via oauth?

Trying to understand how it works oauth
How to get authorized user details using Blizzard api?
import json
import requests
client_id = ""
client_secret = ""
region = "eu"
data = {
'grant_type': 'client_credentials'
}
access_token_response = requests.post(f"https://{region}.battle.net/oauth/token", data=data, allow_redirects=False, auth=(client_id, client_secret))
access_token = json.loads(access_token_response.text)["access_token"]
api_call_headers = {
'Authorization': 'Bearer ' + access_token
}
api_call_response = requests.get(f"https://{region}.battle.net/oauth/userinfo", headers=api_call_headers)

Python Requests Post within a nested Json - retrieve data with a specific value

I already look in stackoverflow and I could not find an answer to my problem.
I'm accessing an API from the German Government that has a output limit of 10.000 entries. I want all data from a specific city, and since there is more than 10.000 entries in the original database, I need to "do the query" while doing the requests.post.
Here is one entry of Json result, when I simply do request.post to this API:
{
"results":[
{
"_id":"CXPTYYFY807",
"CREATED_AT":"2019-12-17T14:48:17.130Z",
"UPDATED_AT":"2019-12-17T14:48:17.130Z",
"result":{
"id":"CXPTYYFY807",
"title":"Bundesstadt Bonn, SGB-315114, Ortsteilzentrum Brüser Berg, Fliesenarbeiten",
"description":["SGB-315114","Ortsteilzentrum Brüser Berg, Fliesenarbeiten"],
"procedure_type":"Ex ante Veröffentlichung (§ 19 Abs. 5)",
"order_type":"VOB",
"publication_date":"",
"cpv_codes":["45431000-7","45431100-8"],
"buyer":{
"name":"Bundesstadt Bonn, Referat Vergabedienste",
"address":"Berliner Platz 2",
"town":"Bonn",
"postal_code":"53111"},
"seller":{
"name":"",
"town":"",
"country":""
},
"geo":{
"lon":7.0944,
"lat":50.73657
},
"value":"",
"CREATED_AT":"2019-12-17T14:48:17.130Z",
"UPDATED_AT":"2019-12-17T14:48:17.130Z"}
}
],
"aggregations":{},
"pagination":{
"total":47389,
"start":0,
"end":0 }}
What I want is all the data which was bought in "town" : "Bonn"
What I already tryed:
import requests
url = 'https://daten.vergabe.nrw.de/rest/evergabe/aggregation_search'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
data = {"results": [{"result": {"buyer": {"town":"Bonn"}}}]}
#need to put the size limit, otherwise he delivers me less:
params = {'size': 10000}
req = requests.post(url, params=params, headers=headers, json=data)
This returns me the post, but not "filtered" by city.
I also tryed req = requests.post(url, params=params, headers=headers, data=data) , which returns me ERROR 400 .
Another way is to grab all the data with the pagination parameters on the end of the json code within a loop, but again I'm not being able to writwe down the json path to the pagination, for example : start: 0 , end:500
Can anyone help me solving it?
Try:
url = 'https://daten.vergabe.nrw.de/rest/evergabe/aggregation_search'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
query1 = {
"query": {
"match": {
"buyer.town": "Bonn"
}
}
}
req = requests.post(url, headers=headers, json=query1)
# Check the output
req.text
Edit:
This won't work if the filter matches with more than 10.000 results, but it may be a quick workaround to the problem you are facing.
import json
import requests
import math
url = "https://daten.vergabe.nrw.de/rest/vmp_rheinland"
size = 5000
payload = '{"sort":[{"_id":"asc"}],"query":{"match_all":{}},"size":'+str(size)+'}'
headers = {
'accept': "application/json",
'content-type': "application/json"
'cache-control': "no-cache"
}
response = requests.request("POST", url, data=payload, headers=headers)
tenders_array = []
query_data = json.loads(response.text)
tenders_array.extend(query_data['results'])
total_hits = query_data['pagination']['total']
result_size = len(query_data['results'])
last_id = query_data['results'][-1]["_id"]
number_of_loops = ((total_hits - size) // size )
last_loop_size = ((total_hits - size) % size)
for i in range(number_of_loops+1):
if i == number_of_loops:
size=last_loop_size
payload = '{"sort":[{"_id":"asc"}],"query":{"match_all":{}},"size":'+str(size)+',"search_after":["'+last_id+'"]}'
response = requests.request("POST", url, data=payload, headers=headers)
query_data = json.loads(response.text)
result_size = len(query_data['results'])
if result_size > 0:
tenders_array.extend(query_data['results'])
last_id = query_data['results'][-1]["_id"]
else:
break
https://gist.github.com/thiagoalencar/34401e204358499ea3b9aa043a18395f
code in the gist.
Some code to paginate through elasticsearch API. This is an API over the elasticsearch API, and the docs where not so clear. Tried scroll, no sucess. This solutions uses search_after parameter without point in time, because the endpoint is not available. Some times the servers refuses the request and it is necessary to verify with response.status_code==502.
The code is messy and need refactoring. But it works. The final tenders_array contains all objects.

Cannot close trade successfully with Oanda Api

I am using the following code for request:
import requests
import print
import json
api_key = "XXX-XXX"
api_version = 'v3'
accountID = "XXX"
tradeID = "XX"
#Endpoint Parts
api_base_url = f"https://api-fxpractice.oanda.com/"
key_url = f"https://api-fxpractice.oanda.com/v3/accounts"
headers = {
'Authorization': f'bearer {api_key}',
'Content-Type': 'application/json'
}
data = {'units': 'ALL'}
endpoint_path = f"/{api_version}/accounts/{accountID}/trades/{tradeID}/close"
endpoint = f"{api_base_url}{endpoint_path}"
r = requests.put(endpoint, headers=headers, data=data)
print(r.status_code)
pprint.pprint(r.json())
Response I get:
400
{'errorMessage': "Invalid value specified for 'Authorization'"}

Get bugs from bugzilla using python

I am trying to get bugs to a db from bugzilla rest API. My code is given below.
import requests
import json
URL = "https://bugzilla.mozilla.org/rest/"
API_KEY = "key"
headers = {"Content-type": "application/json"}
params = {
"username": "email",
"password": "password",
"apikey": API_KEY,
}
# r = requests.get(URL + 'login/', headers = headers, params = params)
# print(r)
resp = requests.post(URL + "bug/" , headers = headers, params = params)
if resp.status_code != 200:
print('error: ' + str(resp.status_code))
else:
print('Success')
print(resp)
When I try this I get Response 404.
Someone please direct me to the correct path.
After poking aroung https://resttesttest.com/ I found the answer. Bugzilla API can be authenticate just by API-KEY. So I removed username and password from params dict. It seems I have an error in concatenating the URL too. I just used "https://bugzilla.mozilla.org/rest/bug/35" to get the bug report on bug_id 35. Then json.load(resp.text) gave the json object of the bug report. Final code looks like this.
import requests
import json
URL = "https://bugzilla.mozilla.org/rest/bug/35"
API_KEY = "key"
headers = {"Content-type": "application/json"}
params = {
"api_key": API_KEY,
}
resp = requests.get(URL , headers = headers, params = params)
if resp.status_code != 200:
print('error: ' + str(resp.status_code))
else:
print('Success')
print(json.loads(resp.text))

Categories

Resources