I am working with Google Places API in python to look for anything(given as query) in a city/town/place (given as location). Following is a snippet from my code:
from googleplaces import GooglePlaces, types, lang
YOUR_API_KEY = 'xxxx'
google_places = GooglePlaces(YOUR_API_KEY)
query_result = google_places.text_search(
query='Play Schools', location = 'Mumbai, India')
print(len(query_result.places))
for place in query_result.places:
print (place.name)
print (place.place_id)
print('\n')
if (query_result.has_next_page_token):
query_result_next_page = google_places.text_search(
pagetoken=query_result.next_page_token)
Now the problem is that my code restricts the number of results to 20 and there are more than 20 results returned by Google Maps on the same query. How do I go about retrieving all the results?
EDIT: I added the last 3 lines of code to try and get the results on the next page but I am thrown with INVALID_REQUEST error.
Thanks for reading my question.
By default, each Nearby Search or Text Search returns up to 20
establishment results per query; however, each search can return as
many as 60 results, split across three pages. If your search will
return more than 20, then the search response will include an
additional value — next_page_token. Pass the value of the
next_page_token to the pagetoken parameter of a new search to see the
next set of results. If the next_page_token is null, or is not
returned, then there are no further results. There is a short delay
between when a next_page_token is issued, and when it will become
valid. Requesting the next page before it is available will return an
INVALID_REQUEST response. Retrying the request with the same
next_page_token will return the next page of results.
Check this url: HERE
Add the PagerToken, as an additional parameter with your HTTP request url like this
https://maps.googleapis.com/maps/api/place/nearbysearch/json?pagetoken=CpQCAgEAAFxg8o-eU7_uKn7Yqjana-HQIx1hr5BrT4zBaEko29ANsXtp9mrqN0yrKWhf-y2PUpHRLQb1GT-mtxNcXou8TwkXhi1Jbk-ReY7oulyuvKSQrw1lgJElggGlo0d6indiH1U-tDwquw4tU_UXoQ_sj8OBo8XBUuWjuuFShqmLMP-0W59Vr6CaXdLrF8M3wFR4dUUhSf5UC4QCLaOMVP92lyh0OdtF_m_9Dt7lz-Wniod9zDrHeDsz_by570K3jL1VuDKTl_U1cJ0mzz_zDHGfOUf7VU1kVIs1WnM9SGvnm8YZURLTtMLMWx8-doGUE56Af_VfKjGDYW361OOIj9GmkyCFtaoCmTMIr5kgyeUSnB-IEhDlzujVrV6O9Mt7N4DagR6RGhT3g1viYLS4kO5YindU6dm3GIof1Q&key=YOUR_API_KEY
Copy paste this url on a browser and check it carefully. Also, please read the documentation url carefully. Cheers!
I found out what was going wrong. The system was throwing INVALID_REQUEST because the next_page_token wasn't valid (there is a small delay b/w between when a next_page_token is issued, and when it will become valid). I compiled the first half of the code (barring the last three lines) and then compiled the last three lines and it worked!! So to the best of my knowledge, the delay was causing the error.
So now all I have to do is put a timer of 1-2 seconds before the last if condition.
Thanks everyone who commented and responded to this question. You are the heroes of this community.
Related
I am developing an app in which I am getting the list of ATM's near by the user. For that I am using Google Places API, but every time it returns 20 result only. I want to get more results. In the API doc it is mention that it will return 20 result but I would like to know is there any way I can get more data or I have to create my own database for that?
Also what I found in the API is that it does not have all the ATM's for India. The information is also not accurate for some of the locations. So Please suggest me some idea for that I should use my own database or not?
I know that I can register places in the Google Places but there is the problem that every time it returns only 20 result. If anyhow this problem is resolved then it will be great to use the Places API..
Please kindly help me.....
As per the documentation:
https://developers.google.com/places/web-service/search#PlaceSearchResponses
EDIT: Place API now supports pagination of up to 60 results. See below for details.
The Places API will return up to 20 establishments per query; however, each search can return as many as 60 results, split across three pages. If your search will return more than 20, then the search response will include an additional parameter — next_page_token. Pass the value of next_page_token to the page_token parameter of a new place search to see the next set of 20 results.
There is also strict terms of service that specify you cant Pre-Fetch, Cache, or Store Content unless it is the content identifier or key you are permitted to store i.e. reference token from a places search:
https://cloud.google.com/maps-platform/terms/?_ga=#3-license
In regards to not being able to find ATM's in India. All places are categorized under the type establishment until Google has enough metadata about a place to categorize it under more specific place types like ATM, cafe, bar etc. A work around that may find more results is to use the keyword parameter in your request with something like 'ATM', 'bank' or 'finance' as the value.
As per the documentation:
https://developers.google.com/places/web-service/search#PlaceSearchRequests
The keyword parameter is matched against all available fields, including but not limited to name, type, and address, as well as customer reviews and other third-party content.
Google API fetches the 20 Result in one page suppose you want to use the next page 20 result then we use the next_page_token from google first page xml as a result.
1) https://maps.googleapis.com/maps/api/place/search/xml?location=Enter latitude,Enter Longitude&radius=10000&types=store&hasNextPage=true&nextPage()=true&sensor=false&key=Enter Google_Map_key
in second step you use the first page's next_page_token data
2)https://maps.googleapis.com/maps/api/place/search/xml?location=Enter Latitude,Enter Longitude&radius=10000&types=store&hasNextPage=true&nextPage()=true&sensor=false&key=enter google_map_key &pagetoken="Enter the first page token Value"
You can do this by using Google API JAVA Client - Here is an example using the java client for getting all the 60 results.
public PlacesList search(double latitude, double longitude, double radius, String types)
throws Exception {
try {
HttpRequestFactory httpRequestFactory = createRequestFactory(HTTP_TRANSPORT);
HttpRequest request = httpRequestFactory
.buildGetRequest(new GenericUrl("https://maps.googleapis.com/maps/api/place/search/json?"));
request.getUrl().put("key", YOUR_API_KEY);
request.getUrl().put("location", latitude + "," + longitude);
request.getUrl().put("radius", radius);
request.getUrl().put("sensor", "false");
request.getUrl().put("types", types);
PlacesList list = request.execute().parseAs(PlacesList.class);
if(list.next_page_token!=null || list.next_page_token!=""){
Thread.sleep(4000);
/*Since the token can be used after a short time it has been generated*/
request.getUrl().put("pagetoken",list.next_page_token);
PlacesList temp = request.execute().parseAs(PlacesList.class);
list.results.addAll(temp.results);
if(temp.next_page_token!=null||temp.next_page_token!=""){
Thread.sleep(4000);
request.getUrl().put("pagetoken",temp.next_page_token);
PlacesList tempList = request.execute().parseAs(PlacesList.class);
list.results.addAll(tempList.results);
}
}
return list;
} catch (HttpResponseException e) {
return null;
}
}
Radar Search doesn't work anymore because it no longer exists: https://cloud.google.com/blog/products/maps-platform/announcing-deprecation-of-place-add
see: https://stackoverflow.com/a/48171023/9903
On the Google places page on: https://developers.google.com/places/documentation/search
Under 'Radar Search Requests' it states that using: https://maps.googleapis.com/maps/api/place/radarsearch/output?parameters
you get 200 results using this call, but note that each call uses 5 requests against your quota.
Please try below url with below parameters
URL = "http://www.google.com/maps?q=restaurant&ie=UTF8&hl=en&sll=0.000000,0.000000&sspn=0.000000,0.000000&vps=3&sa=N&start=20"
Where start = No of results say for instance start from 0 or 10 0r 20 –
You can get the more results from next page token:
https://maps.googleapis.com/maps/api/place/textsearch/json?query=[your search key word]&location=latitude,longitude&radius=value&key=[your API key]&next_page_token=next_page_token value
I'm quite a newbie in Python espescially to use Gmaps API to get place details.
I want to search for places with this parameters:
places_result = gmaps.places_nearby(location=' -6.880270,107.60794', radius = 300, type = 'cafe')
But actually i want to get many data as i can in the specific lat/lng and radius. So, I try to get new parameters that google api has provided. That's page_token. This is the detail of documentation:
pagetoken — Returns up to 20 results from a previously run search. Setting a pagetoken parameter will execute a search with the same parameters used previously — all parameters other than pagetoken will be ignored.
https://developers.google.com/places/web-service/search
So i tried to get more data (Next page data) with this function:
places_result = gmaps.places_nearby(location=' -6.880270,107.60794', radius = 300, type = 'cafe')
time.sleep(5)
place_result = gmaps.places_nearby(page_token = places_result['next_page_token'])
And this is my whole output function:
for place in places_result['results']:
my_place_id = place['place_id']
my_fields = ['name','formatted_address','business_status','rating','user_ratings_total','formatted_phone_number']
places_details = gmaps.place(place_id= my_place_id , fields= my_fields)
pprint.pprint(places_details['result'])
But unfortunately when i start to running i only get 20 (Max) data of place details. I don't know whether my function of page token parameter it's true or not, because the output can't get more than 20 data.
I'm very appreciate for anyone who can give me an advice to solve the problem. Thank you very much :)
As stated on the documentation here:
By default, each Nearby Search or Text Search returns up to 20 establishment results per query; however, each search can return as many as 60 results, split across three pages.
So basically, what you are currently experiencing is an intended behavior. There is no way for you to get more than 20 results in a single nearby search query.
If a next_page_token was returned upon sending your first nearby search query, then, this means that a second page with results is available.
To access this second page of results, then just like what you did, you just have to send another nearby search request, but use the pagetoken parameter this time, and set its value with the next_page_token you got from the first response.
And if the next_page_token also exists on the response of your second nearby search query, then this means that the third (and the last) page of the result is also available. You could access the third page of results using the same way you accessed the second page.
Going back to your query, I tried the parameters you've specified but I could only get around 9 results. Is it intended that your radius parameter is only set at 300 meters?
According to the Imgur Docs, the "GET Account Favorites" API call takes optional arguments for pagination, implying that all objects are returned without it.
However, when I use the following code snippet (the application has been registered and OAuth has already performed against my account for testing), I get only the first 30 JSON objects. In the snippet below, I already have an access_token for an authorized user and can retrieve data for that username. But the length of the returned list is always the first 30 items.
username = token['username']
bearer_headers = {
'Authorization': 'Bearer ' + token['access_token']
}
fav_url = 'https://api.imgur.com/3/account/' + username + '/' + 'favorites'
r = requests.get(fav_url, headers=bearer_headers)
r_json = r.json()
favorites=r_json['data']
len(favorites)
print(favorites)
The requests response returns a dictionary with three keys: status (the HTTP status code), success (true or false), and data, of which the value is a list of dictionaries (one per favorited item).
I'm trying to retrieve this without pagination so I can extract specific metadata values into a Pandas dataframe (id, post date, etc).
I originally thought this was a Pandas display problem in Jupyter notebook, but tracked it back to the API only returning the newest 30 list items, despite the docs indicating otherwise. If I place an arbitrary page number at the end (eg, "/favorites/1"), it returns the 30 items appropriate to that page, but there doesn't seem to be an option to get all items or retrieve a count of the total items or number of pages in advance.
What am I missing?
Postscript: It appears that none of the URIs work without pagination, eg, get account images, get gallery submissions, etc. Anything where there is an optional "/{{page}}" parameter, it will default to first page if none is specified. So I guess the larger question is, "does Imgur API even support non-paginated data, and how is that accessed?".
Paginated data is usually used when the possible size of the response can be arbitrarily large. I would be surprised if a major service like Imgur had an API that didn't work this way.
As you have found, the page attribute may be optional, and if you don't provide it, you get the first page as your response.
If you want to get more than the first page, you will need to loop over the page number:
data = []
page = 0
while block := connection.get(page=page):
data.append(block)
page += 1
This assumes Python3.8+ due to the := assignment expression. If you are on an older version you'll need to set block in the loop body, but the same idea applies.
I am using GMail API to mine some informations like flights, receipts, invoices, ecc.
I want to skip a number of pages or skip to the very last one. How do I do that with Python?
You can skip pages by relying on the nextPageToken field as described in the reference of the list call. The nextPageToken field is part of the response payload.
query = 'example query has:attachment'
nextPageToken = ''
# The following while goes through all pages using the nextPageToken key in the response.
while True:
### GET MESSAGES THAT MATCH A QUERY
msg_results = service.users().messages().list(userId='me', maxResults=20, pageToken=nextPageToken, q=query).execute()
# messages = msg_results.get('messages', [])
if('nextPageToken' in msg_results or nextPageToken==''):
nextPageToken = msg_results['nextPageToken']
print('nextPageToken={}'.format(nextPageToken))
else:
break
Once you've done this loop, your msg_results will contain only the messages that belong to the last page. How many messages you get will depend on how maxResults is set.
If you only want to skip a number of pages like 3 or 4 it's easy to change the above snippet to count the loops and quit when it's at the desired page.
To have a fully working code example you can try integrating the above snippet with this code on GitHub or with the nice boilerplate from GMail knowledge base which is also on GitHub.
On the Soundcloud API guide (https://developers.soundcloud.com/docs/api/guide#pagination)
the example given for reading more than 100 piece of data is as follows:
# get first 100 tracks
tracks = client.get('/tracks', order='created_at', limit=page_size)
for track in tracks:
print track.title
# start paging through results, 100 at a time
tracks = client.get('/tracks', order='created_at', limit=page_size,
linked_partitioning=1)
for track in tracks:
print track.title
I'm pretty certain this is wrong as I found that 'tracks.collection' needs referencing rather than just 'tracks'. Based on the GitHub python soundcloud API wiki it should look more like this:
tracks = client.get('/tracks', order='created_at',limit=10,linked_partitioning=1)
while tracks.collection != None:
for track in tracks.collection:
print(track.playback_count)
tracks = tracks.GetNextPartition()
Where I have removed the indent from the last line (I think there is an error on the wiki it is within the for loop which makes no sense to me). This works for the first loop. However, this doesn't work for successive pages because the "GetNextPartition()" function is not found. I've tried the last line as:
tracks = tracks.collection.GetNextPartition()
...but no success.
Maybe I'm getting versions mixed up? But I'm trying to run this with Python 3.4 after downloading the version from here: https://github.com/soundcloud/soundcloud-python
Any help much appreciated!
For anyone that cares, I found this solution on the SoundCloud developer forum. It is slightly modified from the original case (searching for tracks) to list my own followers. The trick is to call the client.get function repeatedly, passing the previously returned "users.next_href" as the request that points to the next page of results. Hooray!
pgsize=200
c=1
me = client.get('/me')
#first call to get a page of followers
users = client.get('/users/%d/followers' % me.id, limit=pgsize, order='id',
linked_partitioning=1)
for user in users.collection:
print(c,user.username)
c=c+1
#linked_partitioning means .next_href exists
while users.next_href != None:
#pass the contents of users.next_href that contains 'cursor=' to
#locate next page of results
users = client.get(users.next_href, limit=pgsize, order='id',
linked_partitioning=1)
for user in users.collection:
print(c,user.username)
c=c+1