Develop Reference

Access data through an API with Python

I am a complete beginner in the use of API in python. I would like to access this data about Covid 19 vaccination : https://datavaccin-covid.ameli.fr/explore/dataset/donnees-de-vaccination-par-commune/api/
I tried some stuff and it's never working can someone please help me.
This the code I wrote :
import requests
import json
import pandas as pd
query = """{
"nhits":183120,
"dataset":"donnees-de-vaccination-par-commune",
"rows":10,
"start":0,
"facet":[
"semaine_injection",
"commune_residence",
"libelle_commune",
"classe_age",
"libelle_classe_age"
],
"format":"json",
"timezone":"UTC"
}"""
url = 'https://datavaccin- covid.ameli.fr/api/records/1.0/search/?dataset=donnees-de-vaccination-par-commune&q=&facet=semaine_injection&facet=commune_residence&facet=libelle_commune&facet=classe_age&facet=libelle_classe_age&refine.commune_residence=13001'
r = requests.post(url, json= query)
print(r.status_code)
print(r.text)
and I get an error 400
The url I used is the link at the end of the website, I am not sure it is the good one to use :
url used
Thank you

It looks like you're using the "POST" method when you need the "GET" method. The POST method lets you send information to their server. The GET method retrieves only.
r = requests.get(url, json=query)
Also, your query doesn't need all the """ outside the curly braces.

Thanks a lot #James for your help. This works
url = 'https://datavaccin- covid.ameli.fr/api/records/1.0/search/'
query = {"dataset":"donnees-de-vaccination-par-commune",
"rows":100,
"start":0,
"facet":[
"semaine_injection",
"commune_residence",
"libelle_commune",
"classe_age",
"libelle_classe_age"
],
"format":"json",
"timezone":"UTC"}
r = requests.get(url, params= query)
print(r.status_code)
print(r.text)
I had to use params, change the url and get rid of the triple quotes to make it work

How do I Authenticate my FTX_Client in Python

I have looked through the FTX api documentation found here: https://docs.ftx.us/#overview
And I've looked at example code found in this repo: https://github.com/ftexchange/ftx/tree/master/rest
I can't 'get' or 'post' anything that requires the Authentication. I am using the api key on my account that has 'full trade permissions', and when I look at: print(request.headers) the headers look like they are in the right format.
I've tried: using google colab instead of vs code, updating all my libraries, generating a new api key, restarting kernel and computer. I can pull something like 'markets' because it doesn't need the Authentication.
Let me know if you need any more information, below is a portion of the code I have that isolates the problem and returns {'success': False, 'error': 'Not logged in'}
import time
import urllib.parse
from typing import Optional, Dict, Any, List
from requests import Request, Session, Response
import hmac
ep = 'https://ftx.us/api/wallet/balances'
ts = int(time.time() * 1000)
s = Session()
request = Request('GET', ep)
prepared = request.prepare()
signature_payload = f'{ts}{prepared.method}{prepared.path_url}'.encode()
if prepared.body:
signature_payload += prepared.body
signature = hmac.new(secret.encode(), signature_payload, 'sha256').hexdigest()
request.headers['FTX-KEY'] = key
request.headers['FTX-SIGN'] = signature
request.headers['FTX-TS'] = str(ts)
response = s.send(prepared)
data = response.json()
print(data)

I've faced with the same problem.
You need to change this part:
prepared.headers['FTX-KEY'] = key
prepared.headers['FTX-SIGN'] = signature
prepared.headers['FTX-TS'] = str(ts)
PS. I believe that the FTX needs to fix their API documentation
PSS. I've checked the a part of https://github.com/ftexchange/ftx/tree/master/rest code. I beleave FTX guys just do a copy-paste into docs this code but originally it belongs to more a sophisticated object oriented solution that will work correctly because they pass into method an already created request and use a prepared variable just to calculate path_url and method

For ftx.us, you need to use different headers:
prepared.headers['FTXUS-KEY'] = key
prepared.headers['FTXUS-TS'] = str(ts)
prepared.headers['FTXUS-SIGN'] = signature

API gives only the headers in Python but not the data

I am trying to access an API from this website. (https://www.eia.gov/opendata/qb.php?category=717234)
I am able to call the API but I am getting only headers. Not sure if I am doing correctly or any additions are needed.
Code:
import urllib
import requests
import urllib.request
locu_api = 'WebAPI'
def locu_search(query):
api_key = locu_api
url = 'https://api.eia.gov/category?api_key=' + api_key
locality = query.replace(' ', '%20')
response = urllib.request.urlopen(url).read()
json_obj = str(response, 'utf-8')
data = json.loads(json_obj)
When I try to print the results to see whats there in data:
data
I am getting only the headers in JSON output. Can any one help me figure out how to do extract the data instead of headers.

Avi!
Look, the data you posted seems to be an application/json response. I tried to reorganize your snippet a little bit so you could reuse it for other purposes later.
import requests
API_KEY = "insert_it_here"
def get_categories_data(api_key, category_id):
"""
Makes a request to gov API and returns its JSON response
as a python dict.
"""
host = "https://api.eia.gov/"
endpoint = "category"
url = f"{host}/{endpoint}"
qry_string_params = {"api_key": api_key, "category_id": category_id}
response = requests.post(url, params=qry_string_params)
return response.json()
print(get_categories_data(api_key=API_KEY, category_id="717234"))
As far as I can tell, the response contains some categories and their names. If that's not what you were expecting, maybe there's another endpoint that you should look for. I'm sure this snippet can help you if that's the case.
Side note: isn't your API key supposed to be private? Not sure if you should share that.
Update:
Thanks to Brad Solomon, I've changed the snippet to pass query string arguments to the requests.post function by using the params parameter which will take care of the URL encoding, if necessary.

You haven't presented all of the data. But what I see here is first a dict that associates category_id (a number) with a variable name. For example category_id 717252 is associated with variable name 'Import quantity'. Next I see a dict that associates category_id with a description, but you haven't presented the whole of that dict so 717252 does not appear. And after that I would expect to see a third dict, here entirely missing, associating a category_id with a value, something like {'category_id': 717252, 'value': 123.456}.
I think you are just unaccustomed to the way some APIs aggressively decompose their data into key/value pairs. Look more closely at the data. Can't help any further without being able to see the data for myself.

Reading website data automatically with POST requests using Python

I am trying to automatically read data from a website where first I need to fill in some fields, submit the form and then read the data that appears. I am new to this but I wrote a code which obviously doesn't work and the result is HTTP Error 500. What am I missing here? or How do I fix this?
Also, I am happy to do this using BS4 as well because I will need to build upon this code.
Website: http://www.mlindex.ml.com/GISPublic/bin/SnapShot.asp
Inputs required: Index Ticker = H0A0 , Base Curr = LOC , Date = 09/22/2017
I checked the source code and went through the js form that submits the POST request and created the code and payload accordingly:
import requests
post_data = {'hdnDate':'1/1/2016', 'hdnAction':'SS', 'hdnSelCurr':'0,LOC', 'hdnCurrDesc':'USD', 'hdnSelTitle':'Hedged', 'txtSSCUSIP':'H0A0'}
# POST some form-encoded data:
post_response = requests.post(url='http://www.mlindex.ml.com/GISPublic/bin/Snapshot.asp', data=post_data)
print post_response

You are missing 'cboSnapCurr': 0, 'cboSSHedge' : 1 from the payload data, as the server that handles the request is expecting those values.
post_data = {'hdnDate':'1/1/2016', 'hdnAction':'SS', 'hdnSelCurr':'0,LOC', 'hdnCurrDesc':'USD', 'hdnSelTitle':'Hedged', 'txtSSCUSIP':'H0A0', 'cboSnapCurr': 0, 'cboSSHedge' : 1}

Issues while inserting data in cloudant DB

I am working a project, where in I am suppose to get some user input through web application, and send that data to cloudant DB. I am using python for the use case. Below is the sample code:
import requests
import json
dict_key ={}
key = frozenset(dict_key.items())
doc={
{
"_ID":"1",
"COORD1":"1,1",
"COORD2":"1,2",
"COORD3":"2,1",
"COORD4":"2,2",
"AREA":"1",
"ONAME":"abc",
"STYPE":"black",
"CROPNAME":"paddy",
"CROPPHASE":"initial",
"CROPSTARTDATE":"01-01-2017",
"CROPTYPE":"temp",
"CROPTITLE":"rice",
"HREADYDATE":"06-03-2017",
"CROPPRICE":"1000",
"WATERRQ":"1000",
"WATERSRC":"borewell"
}
}
auth = ('uid', 'pwd')
headers = {'Content-type': 'application/json'}
post_url = "server_IP".format(auth[0])
req = requests.put(post_url, auth=auth,headers=headers, data=json.dumps(doc))
#req = requests.get(post_url, auth=auth)
print json.dumps(req.json(), indent=1)
When I am running the code, I am getting the below error:
"WATERSRC":"borewell"
TypeError: unhashable type: 'dict'
I searched a bit, and found below stackflow link as a prospective resolution
TypeError: unhashable type: 'dict'
It says that "To use a dict as a key you need to turn it into something that may be hashed first. If the dict you wish to use as key consists of only immutable values, you can create a hashable representation of it like this:
key = frozenset(dict_key.items())"
I have below queries:
1) I have tried using it in my code above,but I am not sure if I have used it correctly.
2) To put the data in the cloudant DB, I am using Python module "requests". In the code, I am using the below line to put the data in the DB:
req = requests.put(post_url, auth=auth,headers=headers, data=json.dumps(doc))
But I am getting below error:
"reason": "Only GET,HEAD,POST allowed"
I searched on that as well, and I found IBM BLuemix document about it as follows
https://console.ng.bluemix.net/docs/services/Cloudant/basics/index.html#cloudant-basics
As I referred the document, I can say that I am using the right option. But may be I am wrong.

If you are adding a document to the database and you know the the _id, then you need to do an HTTP POST. Here's some slightly modified code:
import requests
import json
doc={
"_id":"2",
"COORD1":"1,1",
"COORD2":"1,2",
"COORD3":"2,1",
"COORD4":"2,2",
"AREA":"1",
"ONAME":"abc",
"STYPE":"black",
"CROPNAME":"paddy",
"CROPPHASE":"initial",
"CROPSTARTDATE":"01-01-2017",
"CROPTYPE":"temp",
"CROPTITLE":"rice",
"HREADYDATE":"06-03-2017",
"CROPPRICE":"1000",
"WATERRQ":"1000",
"WATERSRC":"borewell"
}
auth = ('admin', 'admin')
headers = {'Content-type': 'application/json'}
post_url = 'http://localhost:5984/mydb'
req = requests.post(post_url, auth=auth,headers=headers, data=json.dumps(doc))
print json.dumps(req.json(), indent=1)
Notice that
the _id field is supplied in the doc and is lower case
the request call is a POST not a PUT
the post_url contains the name of the database being written to - in this case mydb
N.B in the above example I am writing to local CouchDB, but replacing the URL with your Cloudant URL and adding correct credentials should get this working for you.