I want to have a python request and add pagination, but for some reason it does not work,
this is my request. Does someone know how to add 'paging': '{"page":0,"size":100}' corretly? The following isn't working?
url = self.get_url_for_endpoint(Constants.PATH_STATISTICS_CUSTOMERS)
payload = {}
params = {
'paging': '{"page":0,"size":100}'
}
headers = {
'Authorization': 'Bearer ' + self.access_token,
'Cookie': 'JSESSIONID=AB52DV8260C*****************',
'Content-Type': 'application/json',
}
r = requests.request("GET", url, headers=headers, data=payload, params=params)
This is working though via postman:
url + ?paging=%7B%22page%22:0,%22size%22:400%7D
So the endpoint has pagination!
I wouldn't recommend using an object as your paging value in the params, instead break down your page and size into individual parameters, like ?page=0&size=100.
But if you want to use an object, your params should be like this:
params = {
"paging": {"page":0,"size":100}
}
Related
Using v5 of the pinterest api and stuck on the authentication flow: https://developers.pinterest.com/docs/getting-started/authentication/
Completed the first step and got the access code.
However, I get the below error when I try to use this code to get the access token.
{"code":1,"message":"Missing request body"}
Here is my code:
client_id= 'my_client_id'
client_secret = 'my_client_secret'
data_string = f'{client_id}:{client_secret}'
token = base64.b64encode(data_string.encode())
headers = {
'Authorization': 'Basic ' + token.decode('utf-8'),
'Content-Type': 'application/x-www-form-urlencoded',
}
url = "https://api.pinterest.com/v5/oauth/token"
code = "my_code_that_i_got_in_the_first_step"
params = {
'grant_type':'authorization_code',
'code': code,
'redirect_url':'https://my_redirect_uri'
}
r = requests.post(url, headers=headers, params=params)
print(r.json())
Below is the correct way to get the access token:
client_id= 'my_client_id'
client_secret = 'my_client_secret'
data_string = f'{client_id}:{client_secret}'
token = base64.b64encode(data_string.encode())
headers = {
'Authorization': 'Basic ' + token.decode('utf-8'),
'Content-Type': 'application/x-www-form-urlencoded',
}
url = "https://api.pinterest.com/v5/oauth/token"
code = "my_code_that_i_got_in_the_first_step"
data= {
'grant_type':'authorization_code',
'code': code,
'redirect_uri':'https://my_redirect_uri'
}
r = requests.post(url, headers=headers, data=data)
print(r.json())
In my question, I had mistyped redirect_uri as redirect_url. Also, when sending a POST, you should use data instead of params. See the comment by Amos Baker.
I already look in stackoverflow and I could not find an answer to my problem.
I'm accessing an API from the German Government that has a output limit of 10.000 entries. I want all data from a specific city, and since there is more than 10.000 entries in the original database, I need to "do the query" while doing the requests.post.
Here is one entry of Json result, when I simply do request.post to this API:
{
"results":[
{
"_id":"CXPTYYFY807",
"CREATED_AT":"2019-12-17T14:48:17.130Z",
"UPDATED_AT":"2019-12-17T14:48:17.130Z",
"result":{
"id":"CXPTYYFY807",
"title":"Bundesstadt Bonn, SGB-315114, Ortsteilzentrum Brüser Berg, Fliesenarbeiten",
"description":["SGB-315114","Ortsteilzentrum Brüser Berg, Fliesenarbeiten"],
"procedure_type":"Ex ante Veröffentlichung (§ 19 Abs. 5)",
"order_type":"VOB",
"publication_date":"",
"cpv_codes":["45431000-7","45431100-8"],
"buyer":{
"name":"Bundesstadt Bonn, Referat Vergabedienste",
"address":"Berliner Platz 2",
"town":"Bonn",
"postal_code":"53111"},
"seller":{
"name":"",
"town":"",
"country":""
},
"geo":{
"lon":7.0944,
"lat":50.73657
},
"value":"",
"CREATED_AT":"2019-12-17T14:48:17.130Z",
"UPDATED_AT":"2019-12-17T14:48:17.130Z"}
}
],
"aggregations":{},
"pagination":{
"total":47389,
"start":0,
"end":0 }}
What I want is all the data which was bought in "town" : "Bonn"
What I already tryed:
import requests
url = 'https://daten.vergabe.nrw.de/rest/evergabe/aggregation_search'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
data = {"results": [{"result": {"buyer": {"town":"Bonn"}}}]}
#need to put the size limit, otherwise he delivers me less:
params = {'size': 10000}
req = requests.post(url, params=params, headers=headers, json=data)
This returns me the post, but not "filtered" by city.
I also tryed req = requests.post(url, params=params, headers=headers, data=data) , which returns me ERROR 400 .
Another way is to grab all the data with the pagination parameters on the end of the json code within a loop, but again I'm not being able to writwe down the json path to the pagination, for example : start: 0 , end:500
Can anyone help me solving it?
Try:
url = 'https://daten.vergabe.nrw.de/rest/evergabe/aggregation_search'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
query1 = {
"query": {
"match": {
"buyer.town": "Bonn"
}
}
}
req = requests.post(url, headers=headers, json=query1)
# Check the output
req.text
Edit:
This won't work if the filter matches with more than 10.000 results, but it may be a quick workaround to the problem you are facing.
import json
import requests
import math
url = "https://daten.vergabe.nrw.de/rest/vmp_rheinland"
size = 5000
payload = '{"sort":[{"_id":"asc"}],"query":{"match_all":{}},"size":'+str(size)+'}'
headers = {
'accept': "application/json",
'content-type': "application/json"
'cache-control': "no-cache"
}
response = requests.request("POST", url, data=payload, headers=headers)
tenders_array = []
query_data = json.loads(response.text)
tenders_array.extend(query_data['results'])
total_hits = query_data['pagination']['total']
result_size = len(query_data['results'])
last_id = query_data['results'][-1]["_id"]
number_of_loops = ((total_hits - size) // size )
last_loop_size = ((total_hits - size) % size)
for i in range(number_of_loops+1):
if i == number_of_loops:
size=last_loop_size
payload = '{"sort":[{"_id":"asc"}],"query":{"match_all":{}},"size":'+str(size)+',"search_after":["'+last_id+'"]}'
response = requests.request("POST", url, data=payload, headers=headers)
query_data = json.loads(response.text)
result_size = len(query_data['results'])
if result_size > 0:
tenders_array.extend(query_data['results'])
last_id = query_data['results'][-1]["_id"]
else:
break
https://gist.github.com/thiagoalencar/34401e204358499ea3b9aa043a18395f
code in the gist.
Some code to paginate through elasticsearch API. This is an API over the elasticsearch API, and the docs where not so clear. Tried scroll, no sucess. This solutions uses search_after parameter without point in time, because the endpoint is not available. Some times the servers refuses the request and it is necessary to verify with response.status_code==502.
The code is messy and need refactoring. But it works. The final tenders_array contains all objects.
I am requesting to mindbodyapi to get token with the following code using requests library
def get_staff_token(request):
URL = "https://api.mindbodyonline.com/public/v6/usertoken/issue"
payload = {
'Api-Key': API_KEY,
'SiteId': "1111111",
'Username': 'user#xyz.com',
'Password': 'xxxxxxxx',
}
r = requests.post(url=URL, params=payload)
print(r.text)
return HttpResponse('Done')
gives a response as follows
{"Error":{"Message":"Missing API key","Code":"DeniedAccess"}}
But if I request the following way it works, anybody could tell me, what I am doing wrong on the above code.
conn = http.client.HTTPSConnection("api.mindbodyonline.com")
payload = "{\r\n\t\"Username\": \"username\",\r\n\t\"Password\": \"xxxxx\"\r\n}"
headers = {
'Content-Type': "application/json",
'Api-Key': API_KEY,
'SiteId': site_id,
}
conn.request("POST", "/public/v6/usertoken/issue", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
In the second one, you are passing the API Key in headers and the credentials in the body of the request. In the first, you are sending both the API Key and credentials together in the query string, not the request body. Refer to requests.request() docs
Just use two dictionaries like in your second code and the correct keywords, I think it should work:
def get_staff_token(request):
URL = "https://api.mindbodyonline.com/public/v6/usertoken/issue"
payload = {
'Username': 'user#xyz.com',
'Password': 'xxxxxxxx',
}
headers = {
'Content-Type': "application/json",
'Api-Key': API_KEY,
'SiteId': "1111111",
}
r = requests.post(url=URL, data=payload, headers=headers)
print(r.text)
return HttpResponse('Done')
Can anyone please tell me how to write a cURL to get events (only modified) list with nextSyncToken?
This is my code that's not working:
def get_headers():
headers = {
#'Content-Type': "application/json",
'Authorization': access_token_json
}
return headers
def get_nexttokensync_list_event():
url_get_list_event = "https://www.googleapis.com/calendar/v3/calendars/id#gmail.com/events"
querystring = {"nextSyncToken": "CMCEh************jd4CGAU="}
response = requests.request("GET", url_get_list_event, headers=headers, params=querystring)
json_event_list_formatted = response.text
print(json_event_list_formatted)
Yes, i've done it!
Here is my code:
import requests
url = "https://www.googleapis.com/calendar/v3/calendars/here_calendar_id/events"
querystring = {"syncToken":"here_token"}
headers = {
'Content-Type': "application/json",
}
response = requests.request("GET", url, headers=headers, params=querystring)
print(response.text)
Code one
import requests
url = "http://store.place.com.br/api/oms/pvt/orders"
headers = {
'accept': "application/json",
'content-type': "application/json",
'x-vtex-api-apptoken': "{{VTEX-API-TOKEN}}",
'x-vtex-api-appkey': "{{VTEX-API-KEY}}"
}
response = requests.request("GET", url, headers=headers)
print(response.text)
Code two
url = "http://store.place.com.br/api/oms/pvt/orders/oderId"
headers = {
'accept': "application/json",
'content-type': "application/json",
'x-vtex-api-apptoken': "{{VTEX-API-TOKEN}}",
'x-vtex-api-appkey': "{{VTEX-API-KEY}}"
}
response = requests.request("GET", url, headers=headers)
print(response.text)
My results are:
{"list":[{"orderId":"BWW-Lojas_Americanas-265033423001", ...
{"orderId":"BWW-Lojas_Americanas-265032819901","sequence":"506927","market
How can I split each list in many rows? After that, I want to save them in different files.txt
You should read the response as JSON using requests.request(...).json(). Then, you can process each order in the list.
Here's an example:
response = requests.request("GET", url, headers=headers).json()
for order in response['list']:
# Process each order
# ...