Develop Reference

python dynamic value in api payload

Hey i have an api request and i need the value in payload to by dynamic, i tried with f'' but it wont make it dynamic
will appreciate your help.
import requests
url = "https://www.stie.com/api/conversations/KJQ-CZHNR-985/fields"
valuetarget = "123"
payload = {'apikey': 'zbyc88srdi333d3dpq5lye48pgg1tfo1pnjvj65ld',
'customfields': '[{"code": "orderno", "value":"valuetarget"}]'}
files = [
]
headers = {}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)

You just need to load json and dump it to string and send it.
import json
valuetarget = "123"
payload = {
'apikey': 'zbyc88srdi333d3dpq5lye48pgg1tfo1pnjvj65ld',
'customfields':
json.dumps([{"code": "orderno", "value":valuetarget}])
}

You should escape the curly braces in the formatted string, like so:
f'[{{"code": "orderno", "value":"{valuetarget}"}}]'
But why not let requests format the string for you?
import requests
url = "https://www.stie.com/api/conversations/KJQ-CZHNR-985/fields"
valuetarget = "123"
# 'customfields' is now a list of dictionaries
payload = {'apikey': 'zbyc88srdi333d3dpq5lye48pgg1tfo1pnjvj65ld',
'customfields': [{"code": "orderno", "value": valuetarget}]}
files = [
]
headers = {}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)

Python Requests Post within a nested Json - retrieve data with a specific value

I already look in stackoverflow and I could not find an answer to my problem.
I'm accessing an API from the German Government that has a output limit of 10.000 entries. I want all data from a specific city, and since there is more than 10.000 entries in the original database, I need to "do the query" while doing the requests.post.
Here is one entry of Json result, when I simply do request.post to this API:
{
"results":[
{
"_id":"CXPTYYFY807",
"CREATED_AT":"2019-12-17T14:48:17.130Z",
"UPDATED_AT":"2019-12-17T14:48:17.130Z",
"result":{
"id":"CXPTYYFY807",
"title":"Bundesstadt Bonn, SGB-315114, Ortsteilzentrum Brüser Berg, Fliesenarbeiten",
"description":["SGB-315114","Ortsteilzentrum Brüser Berg, Fliesenarbeiten"],
"procedure_type":"Ex ante Veröffentlichung (§ 19 Abs. 5)",
"order_type":"VOB",
"publication_date":"",
"cpv_codes":["45431000-7","45431100-8"],
"buyer":{
"name":"Bundesstadt Bonn, Referat Vergabedienste",
"address":"Berliner Platz 2",
"town":"Bonn",
"postal_code":"53111"},
"seller":{
"name":"",
"town":"",
"country":""
},
"geo":{
"lon":7.0944,
"lat":50.73657
},
"value":"",
"CREATED_AT":"2019-12-17T14:48:17.130Z",
"UPDATED_AT":"2019-12-17T14:48:17.130Z"}
}
],
"aggregations":{},
"pagination":{
"total":47389,
"start":0,
"end":0 }}
What I want is all the data which was bought in "town" : "Bonn"
What I already tryed:
import requests
url = 'https://daten.vergabe.nrw.de/rest/evergabe/aggregation_search'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
data = {"results": [{"result": {"buyer": {"town":"Bonn"}}}]}
#need to put the size limit, otherwise he delivers me less:
params = {'size': 10000}
req = requests.post(url, params=params, headers=headers, json=data)
This returns me the post, but not "filtered" by city.
I also tryed req = requests.post(url, params=params, headers=headers, data=data) , which returns me ERROR 400 .
Another way is to grab all the data with the pagination parameters on the end of the json code within a loop, but again I'm not being able to writwe down the json path to the pagination, for example : start: 0 , end:500
Can anyone help me solving it?

Try:
url = 'https://daten.vergabe.nrw.de/rest/evergabe/aggregation_search'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
query1 = {
"query": {
"match": {
"buyer.town": "Bonn"
}
}
}
req = requests.post(url, headers=headers, json=query1)
# Check the output
req.text
Edit:
This won't work if the filter matches with more than 10.000 results, but it may be a quick workaround to the problem you are facing.

import json
import requests
import math
url = "https://daten.vergabe.nrw.de/rest/vmp_rheinland"
size = 5000
payload = '{"sort":[{"_id":"asc"}],"query":{"match_all":{}},"size":'+str(size)+'}'
headers = {
'accept': "application/json",
'content-type': "application/json"
'cache-control': "no-cache"
}
response = requests.request("POST", url, data=payload, headers=headers)
tenders_array = []
query_data = json.loads(response.text)
tenders_array.extend(query_data['results'])
total_hits = query_data['pagination']['total']
result_size = len(query_data['results'])
last_id = query_data['results'][-1]["_id"]
number_of_loops = ((total_hits - size) // size )
last_loop_size = ((total_hits - size) % size)
for i in range(number_of_loops+1):
if i == number_of_loops:
size=last_loop_size
payload = '{"sort":[{"_id":"asc"}],"query":{"match_all":{}},"size":'+str(size)+',"search_after":["'+last_id+'"]}'
response = requests.request("POST", url, data=payload, headers=headers)
query_data = json.loads(response.text)
result_size = len(query_data['results'])
if result_size > 0:
tenders_array.extend(query_data['results'])
last_id = query_data['results'][-1]["_id"]
else:
break
https://gist.github.com/thiagoalencar/34401e204358499ea3b9aa043a18395f
code in the gist.
Some code to paginate through elasticsearch API. This is an API over the elasticsearch API, and the docs where not so clear. Tried scroll, no sucess. This solutions uses search_after parameter without point in time, because the endpoint is not available. Some times the servers refuses the request and it is necessary to verify with response.status_code==502.
The code is messy and need refactoring. But it works. The final tenders_array contains all objects.

python requests not recognizing params

I am requesting to mindbodyapi to get token with the following code using requests library
def get_staff_token(request):
URL = "https://api.mindbodyonline.com/public/v6/usertoken/issue"
payload = {
'Api-Key': API_KEY,
'SiteId': "1111111",
'Username': 'user#xyz.com',
'Password': 'xxxxxxxx',
}
r = requests.post(url=URL, params=payload)
print(r.text)
return HttpResponse('Done')
gives a response as follows
{"Error":{"Message":"Missing API key","Code":"DeniedAccess"}}
But if I request the following way it works, anybody could tell me, what I am doing wrong on the above code.
conn = http.client.HTTPSConnection("api.mindbodyonline.com")
payload = "{\r\n\t\"Username\": \"username\",\r\n\t\"Password\": \"xxxxx\"\r\n}"
headers = {
'Content-Type': "application/json",
'Api-Key': API_KEY,
'SiteId': site_id,
}
conn.request("POST", "/public/v6/usertoken/issue", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))

In the second one, you are passing the API Key in headers and the credentials in the body of the request. In the first, you are sending both the API Key and credentials together in the query string, not the request body. Refer to requests.request() docs
Just use two dictionaries like in your second code and the correct keywords, I think it should work:
def get_staff_token(request):
URL = "https://api.mindbodyonline.com/public/v6/usertoken/issue"
payload = {
'Username': 'user#xyz.com',
'Password': 'xxxxxxxx',
}
headers = {
'Content-Type': "application/json",
'Api-Key': API_KEY,
'SiteId': "1111111",
}
r = requests.post(url=URL, data=payload, headers=headers)
print(r.text)
return HttpResponse('Done')

Trying to retrieve data from the Anbima API

I'm trying to automate a process in which i have to download some brazilian fund quotes from Anbima (Brazil regulator). I have been able to work around the first steps to retrieve the access token but i don't know how to use the token in order to make requests. Here is the tutorial website https://developers.anbima.com.br/en/como-acessar-nossas-apis/.
I have tried a lot of thing but all i get from the request is 'Could not find a required APP in the request, identified by HEADER client_id.'
If someone could share some light. Thank you in advance.
import requests
import base64
import json
requests.get("https://api.anbima.com.br/feed/fundos/v1/fundos")
ClientID = '2Xy1ey11****'
ClientSecret = 'faStF1Hc****'
codeString = ClientID + ":" + ClientSecret
codeStringBytes = codeString.encode('ascii')
base64CodeBytes = base64.b64encode(codeStringBytes)
base64CodeString = base64CodeBytes.decode('ascii')
url = "https://api.anbima.com.br/oauth/access-token"
headers = {
'content-type': 'application/json'
,'authorization': f'Basic {base64CodeString}'
}
body = {
"grant_type": "client_credentials"
}
r = requests.post(url=url, data=json.dumps(body), headers=headers, allow_redirects=True)
jsonDict = r.json()
##################
urlFundos = "https://api-sandbox.anbima.com.br/feed/precos-indices/v1/titulos-publicos/mercado-secundario-TPF"
token = jsonDict['access_token']
headers2 = {
'content-type': 'application/json'
,'authorization': f'Bearer {token}'
}
r2 = requests.get(url=urlFundos, headers=headers2)
r2.status_code
r2.text

I was having the same problem, but today I could advance. I believe you need to adjust some parameters in the header.
Follows the piece of code I developed.
from bs4 import BeautifulSoup
import requests
PRODUCTION_URL = 'https://api.anbima.com.br'
SANDBOX_URL = 'https://api-sandbox.anbima.com.br'
API_URL = '/feed/fundos/v1/fundos/'
CODIGO_FUNDO = '594733'
PRODUCTION = False
if PRODUCTION:
URL = PRODUCTION_URL
else:
URL = SANDBOX_URL
URL = URL + API_URL + CODIGO_FUNDO
HEADER = {'access_token': 'your token',
'client_id' : 'your client ID'}
html = requests.get(URL, headers=HEADER).content
soup = BeautifulSoup(html, 'html.parser')
print(soup.prettify())
The sandbox API will return a dummy JSON. To access the production API you will need to request access (I'm trying to do this just now).

url = 'https://api.anbima.com.br/oauth/access-token'
http = 'https://api-sandbox.anbima.com.br/feed/precos-indices/v1/titulos-publicos/pu-intradiario'
client_id = "oLRa*******"
client_secret = "6U2nefG*****"
client_credentials = "oLRa*******:6U2nefG*****"
client_credentials = client_credentials.encode('ascii')
senhabytes = base64.b64encode(client_credentials)
senha = base64.b64decode(senhabytes)
print(senhabytes, senha)
body = {
"grant_type": "client_credentials"
}
headers = {
'content-type': 'application/json',
'Authorization': 'Basic b0xSYTJFSUlOMWR*********************'
}
request = requests.post(url, headers=headers, json=body, allow_redirects=True)
informacoes = request.json()
token = informacoes['access_token']
headers2 = {
"content-type": "application/json",
"client_id": f"{client_id}",
"access_token": f"{token}"
}
titulos = requests.get(http, headers=headers2)
titulos = fundos.json()
I used your code as a model, then I've made some changes. I've printed the encode client_id:client_secret and then I've copied and pasted in the headers.
I've changed the data for json.

How to convert http request to Python?

I need your help.
I am looking to communicate with a REST API containing sensor data at port number 3.
I have a Json (POST) request that works perfectly executed on a REST client like Insomnia.
My request :
{ "header": { "portNumber": 3 }, "data": { "index": 40 } }
Picture of my request
However I am unable to make it work on Python and to recover data from my sensor.
My Python code :
import requests
import json
url = 'http://192.168.1.100/iolink/sickv1' # Address of the OctoPrint Server
header = {'portNumber': '3', 'Content-Type': 'application/json'} #Basic request's header
data = {'index': 40}
def get_sensor_measure():
r = requests.post(url + '/readPort', headers=header, data=data)
print(r.content)
print(r.status_code)
I get the error:
b'{"header":{"status":1,"message":"Parsing Failed"}}'
Thank you in advance

You should pass 'portNumber': '3' in data not in header:
header = {'Content-Type': 'application/json'}
data = {'header': {'portNumber': '3'}, 'data': {'index': 40}}
And also as Karl stated in his answer you need to changes data to json:
r = requests.post(url + '/readPort', headers=header, json=data)

My guess would be that you are using the wrong field to pass your payload. It isn't really obvious, but the requests package expects JSON-type payload to be sent with the json field, not the data field, i.e.:
r = requests.post(url + '/readPort', headers=header, json=data)

With a few changes (Bold), it works. Thanks
url = 'http://192.168.1.100/iolink/sickv1' # Address of the OctoPrint Server
header = {'Content-Type': 'application/json'} #Basic request's header
**data = {'header': {'portNumber': 3}, 'data': {'index': 40}}**
def get_sensor_measure():
r = requests.post(url + '/readPort', headers=header, json=data)
print(r.content)
print(r.status_code)