Develop Reference

Python Request Library multiple API calls sequentially in for loop

In our project we are reading historical data(past 45 days data) from solr db.We can read maximum 5 days data in single api call so we are calling api sequentially in a for loop for 45 days.But observing some 400 status codes for some api call' in between randomly..like for total 9 api calls some giving 200 and some 400 response codes in between randomly..like if I rerun my job, then api call which gave 400 earlier might give 200 this time.
I checked with API owner, they said it is because you are calling next api call before earlier one is completed.
1.) how to identify if api request is completed in python , so that can call next api only when previous request is completed.This can be answered by API owner or there is any way in python request library
2.) should i think of using sleep statement after each api call.But how to know sleep time and is this efficient way ?
Thanks

How do I have an API pickup where it left off after the request times out

I'm still working on this cocktail app and I ran into another issue. After some time, the api call will drop after an hour; the api will only stay open for an hour. After that hour, I have to make a new api call. But I don't want to start all over when trying to pull data back. The data I'm using has over 300 thousand records in it to match, which on average takes about 10 hours to complete. Sorry, if this question does not make any sense I'm still new to coding and I'm doing my best!
For instance:
Make api call
Use data from csv file 300k records to get data back on
Retrieve the data back.
API connection drops--- Will need to make new api connection.
Make API call
Determine what data has been matched from csv file, then pick up from the next record that has not been matched.
Is there a way to make sure that when I have to get a new API key or when the connection drops, whenever the connection is made again it picks up where it left off?

Sounds like you want your API to have something like "sessions" that the client can disconnect and reconnect to.
Just some sort of session ID the server can use to reconnect you to whatever long running query it is doing.
It probably gets quite complex though. The query may finish when no client is connected and so the results need to be stored somewhere keyed by the session id so that when the API reconnects it can get the results.
PS. 10 hours to match 300K records? That is a crazy amount of time ! I'd be doing some profiling on that match algorithm.

Alphavantage: Requests from different api keys but from the same system

I have written a python script that helps me get the prices from alphavantage using an api key. I have 2 different alphavantage api keys (mine and my brother's) and the python script requests data from alphavantage separately from both keys but from the same laptop. But even though I request it separately from separate keys, I get the maximum api call frequency exceeded error (5 api calls per minute per key).
My best assumption is that alphavantage knows whether the request is coming from the same location or the same system. Is there any workaround this problem? Maybe bounce my signal (sort of found an answer but don't know if that's the problem though!) or pretend like the request is going from different system?

Your IP is limited to 5 requests per minute regardless of how many keys you cycle through, so you would need to cycle through that as well.
I went through the same issue and just ended up buying their premium key. It's well worth it. You can also alternatively add a time delay and just run the script in the background so it doesn't time out.
You can also try polygon API its 60 requests per minute for free
https://polygon.io/pricing

Access HTTP Header Details Through Python Instagram API

I'm using the Instagram API to retrieve all photos from a list of hashtags. Starting today, I've been hitting the API rate limits (429 error, specifically). To debug this, I've been trying to see how to get the number of calls I have left per hour and incorporate that into my script.
The Instagram API says that you can access the number of calls left in the hour through the HTTP header, but I'm not sure how to access that in Python.
The following fields are provided in the header of each response and their values are related to the type of call that was made (authenticated or unauthenticated):
X-Ratelimit-Remaining: the remaining number of calls available to your app within the 1-hour window
X-Ratelimit-Limit: the total number of calls allowed within the 1-hour window
http://instagram.com/developer/limits
How would I access this data in Python?

I assumed it was much a much fancier solution based on a few other answers on SO, but after some researching, I found a really simple solution!
import requests
r = requests.get('URL of the API response here')
r.headers

Python: Routine for handling API request limits

I need to make a lot of HTTP requests (> 1000) against a public service which only allows 500 HTTP requests per day. Hence, I have to count the number of executed requests and stop when I reach the maximum daily amount to continue the next day with the remaining calls. In particular, I iterate over a non-sorted list, so I cannot assume that the elements are in any order. My code looks like this:
from requests import Session, Request
request_parameters = {'api_key': api_key}
for user_id in all_user_ids:
r = requests.get('http://public-api.com/%s'% user_id, request_parameters)
text = r.content
# do some stuff with text
Is there any package or pattern which you can recommend for counting and resuming API calls like this?

I would suggest implementing a simple counter to stop when you have hit your limit for the day along with a local cache of the data you have already received. Then when you run the process again the next day check each record against your local cache first and only go on to call the web service if there is no record in the local cache. That way you will eventually have all of the data unless you are generating more requests per day than the service usage limit.
The format of the cache will depend on what is returned from the web service and how much of that data you need, but it may be as simple as a csv file with a unique identifier to search against and the other fields you will need to retrieve in future.
Another option would be to store the whole response from each call (if you need a lot of it) in a dictionary with the key being a unique identifier and the value being the response. This could be saved as a json file and easily loaded back into memory for checking against on future runs.