Develop Reference

Mapbox API PUT Datasets Feature return "Provide a single Feature to insert"

I am trying to add a feature to the Dataset via Mapbox API using Python. I'm following this instruction https://docs.mapbox.com/api/maps/#update-a-dataset but keep getting this error:
{'message': 'Provide a single Feature to insert'}
The code looks like this:
rs = []
dictionary = {
"id":1,
"type":"Feature",
"properties":{},
"geometry":{"coordinates":[-83.750246, 42.269375],"type":"Point"}}
url = "https://api.mapbox.com/datasets/v1/voratima/"+dataset+"/features/1?access_token="+access_token
rs.append(grequests.put(url, data=dictionary, hooks = {'response' : do_something}))
grequests.map(rs, exception_handler=exception_handler)
I've tried the following but none of them work:
using requests instead of grequests
wrapping the dictionary with json.dumps()
changing the put parameter from data=dictionary to json=dictionary
Making sure the id for both data and URL are set to 1.
Postman of the exact same request does not have the error. What am I missing?

Given a dataset with dataset ID dataset exists, your request body looks ok.
Please add the header
headers = {'Content-type': 'application/json'}
Also can you check if you meet these specs:
This should be one individual GeoJSON feature, not a GeoJSON
FeatureCollection. If the GeoJSON feature has a top-level id property,
it must match the feature_id you use in the URL endpoint.

It turns out I forgot the header. Thanks to Mortiz for pointing that out. After the update I got
<Response [400]> {'message': 'Unexpected token i'}
That's because I need to wrap the dictionary inside json.dumps(). Then the error became
<Response [422]> {'message': 'Request URI does not match feature id'}
That's because the id in the dictionary has to be a string i.e. "id":"1" not "id":1. Here's the code that works:
rs = []
dictionary = {
"id":"1",
"type":"Feature",
"properties":{},
"geometry":{"coordinates":[-83.750246, 42.269375],"type":"Point"}}
headers = {'Content-type': 'application/json'}
url = "https://api.mapbox.com/datasets/v1/voratima/"+dataset+"/features/1?access_token="+access_token
rs.append(grequests.put(url, data=json.dumps(dictionary), headers=headers, hooks = {'response' : do_something}))
grequests.map(rs, exception_handler=exception_handler)

Using API to create a new query on Redash

I managed to import queries into another account. I used the endpoint POST function given by Redash, it sort of just applies to just “modifying/replacing”: https://github.com/getredash/redash/blob/5aa620d1ec7af09c8a1b590fc2a2adf4b6b78faa/redash/handlers/queries.py#L178
So actually, if I want to import a new query what should I do? I want to create a new query that doesn’t exist on my account. I’m looking at https://github.com/getredash/redash/blob/5aa620d1ec7af09c8a1b590fc2a2adf4b6b78faa/redash/handlers/queries.py#L84
Following is the function which I made to create new queries if the query_id doesn’t exist.
url = path, api = user api, f = filename, query_id = query_id of file in local desktop
def new_query(url, api, f, query_id):
headers ={'Authorization': 'Key {}'.format(api), 'Content-Type': 'application/json'}
path = "{}/api/queries".format(url)
query_content = get_query_content(f)
query_info = {'query':query_content}
print(json.dumps(query_info))
response = requests.post(path, headers = headers, data = json.dumps(query_info))
print(response.status_code)
I am getting response.status_code 500. Is there anything wrong with my code? How should I fix it?

For future reference :-) here's a python POST that creates a new query:
payload = {
"query":query, ## the select query
"name":"new query name",
"data_source_id":1, ## can be determined from the /api/data_sources end point
"schedule":None,
"options":{"parameters":[]}
}
res = requests.post(redash_url + '/api/queries',
headers = {'Authorization':'Key YOUR KEY'},
json=payload)
(solution found thanks to an offline discussion with #JohnDenver)

TL;DR:
...
query_info = {'query':query_content,'data_source_id':<find this number>}
...
Verbose:
I had a similar problem. Checked redash source code, it looks for data_source_id. I added the data_source_id to my data payload which worked.
You can find the appropriate data_source_id by looking at the response from a 'get query' call:
import json
def find_data_source_id(url,query_number,api)
path = "{}/api/queries/{}".format(url,query_number)
headers ={'Authorization': 'Key {}'.format(api), 'Content-Type': 'application/json'}
response = requests.get(path, headers = headers)
return json.loads(response.text)['data_source_id']

The Redash official API document is so lame, it doesn't give any examples for the documented "Common Endpoints". I was having no idea how I should use the API key.
Instead check this saviour https://github.com/damienzeng73/redash-api-client .

API gives only the headers in Python but not the data

I am trying to access an API from this website. (https://www.eia.gov/opendata/qb.php?category=717234)
I am able to call the API but I am getting only headers. Not sure if I am doing correctly or any additions are needed.
Code:
import urllib
import requests
import urllib.request
locu_api = 'WebAPI'
def locu_search(query):
api_key = locu_api
url = 'https://api.eia.gov/category?api_key=' + api_key
locality = query.replace(' ', '%20')
response = urllib.request.urlopen(url).read()
json_obj = str(response, 'utf-8')
data = json.loads(json_obj)
When I try to print the results to see whats there in data:
data
I am getting only the headers in JSON output. Can any one help me figure out how to do extract the data instead of headers.

Avi!
Look, the data you posted seems to be an application/json response. I tried to reorganize your snippet a little bit so you could reuse it for other purposes later.
import requests
API_KEY = "insert_it_here"
def get_categories_data(api_key, category_id):
"""
Makes a request to gov API and returns its JSON response
as a python dict.
"""
host = "https://api.eia.gov/"
endpoint = "category"
url = f"{host}/{endpoint}"
qry_string_params = {"api_key": api_key, "category_id": category_id}
response = requests.post(url, params=qry_string_params)
return response.json()
print(get_categories_data(api_key=API_KEY, category_id="717234"))
As far as I can tell, the response contains some categories and their names. If that's not what you were expecting, maybe there's another endpoint that you should look for. I'm sure this snippet can help you if that's the case.
Side note: isn't your API key supposed to be private? Not sure if you should share that.
Update:
Thanks to Brad Solomon, I've changed the snippet to pass query string arguments to the requests.post function by using the params parameter which will take care of the URL encoding, if necessary.

You haven't presented all of the data. But what I see here is first a dict that associates category_id (a number) with a variable name. For example category_id 717252 is associated with variable name 'Import quantity'. Next I see a dict that associates category_id with a description, but you haven't presented the whole of that dict so 717252 does not appear. And after that I would expect to see a third dict, here entirely missing, associating a category_id with a value, something like {'category_id': 717252, 'value': 123.456}.
I think you are just unaccustomed to the way some APIs aggressively decompose their data into key/value pairs. Look more closely at the data. Can't help any further without being able to see the data for myself.

requests.post with Python

I'm connecting to a login protected API with a Python script here below.
import requests
url = 'https://api.json'
header = {'Content-Type': 'application/x-www-form-urlencoded'}
login = ('kjji#snm.com', 'xxxxx')
mnem = 'inputRequests':'{'inputRequests':'[{'function':'GDSP','identifier':'ibm','mnemonic':'IQ_TOTAL_REV'}]}}
r = requests.post(url, auth=login, data=mnem, headers=header)
print(r.json())
The connection is established but I am getting an error from the API because of the format of the data request.The original format is here below. I cannot find a way to enter this in the mnem here above:
inputRequests={inputRequests:
[
{function:"xxx",identifier:"xxx",mnemonic:"xxx"},
]
}
The error given is
C:\Users\xxx\Desktop>pie.py
File "C:\Users\xxx\Desktop\pie.py", line 6
mnem={'inputRequests':'{'inputRequests':'[{'function':'xxx','identifier':'xx','mnemonic':'xxx'}]}}
^
SyntaxError: invalid syntax
I am unsure on how to proceed from here. I cannot find anything in the requests documentation that points to how to insert several variables in the data field.

The requests module in Python receive protogenic Python dict as the JSON data in post request but not a string. Therefore, you may try to define mnem like this:
mnem = {
'inputRequests':[
{'function':'GDSP',
'identifier':'ibm',
'mnemonic':'IQ_TOTAL_REV'
}
]}

the data parameter should be a dictionary.
therefore to pass the three parameters try using:
mnem = {'function':'GDSP','identifier':'ibm','mnemonic':'IQ_TOTAL_REV'}

Creating POST request in python, need to send data as multipart/form-data?

I'm in the process of writing a very simple Python application for a friend that will query a service and get some data in return. I can manage the GET requests easily enough, but I'm having trouble with the POST requests. Just to get my feet wet, I've only slightly modified their example JSON data, but when I send it, I get an error. Here's the code (with identifying information changed):
import urllib.request
import json
def WebServiceClass(Object):
def __init__(self, user, password, host):
self.user = user
self.password = password
self.host = host
self.url = "https://{}/urldata/".format(self.host)
mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
mgr.add_password(None, "https://{}".format(self.host), self.user, self.password)
self.opener = urllib.request.build_opener(urllib.request.HTTPDigestAuthHandler(mgr))
username = "myusername"
password = "mypassword"
service_host = "thisisthehostinfo"
web_service_object = WebServiceClass(username, password, service_host)
user_query = {"searchFields":
{
"First Name": "firstname",
"Last Name": "lastname"
},
"returnFields":["Entity ID","First Name","Last Name"]
}
user_query = json.dumps(user_query)
user_query = user_query.encode("ascii")
the_url = web_service_object.url + "modules/Users/search"
try:
user_data = web_service_object.opener.open(the_url, user_query)
user_data.read()
except urllib.error.HTTPError as e:
print(e.code)
print(e.read())
I got the class data from their API documentation.
As I said, I can do GET requests fine, but this POST request gives me a 500 error with the following text:
Content type 'application/x-www-form-urlencoded;charset=UTF-8' not supported
In researching this error, my assumption has become that the above error means I need to submit the data as multipart/form-data. Whether or not that assumption is correct is something I would like to test, but stock Python doesn't appear to have any easy way to create multipart/form-data - there are modules out there, but all of them appear to take a file and convert it to multipart/form-data, whereas I just want to convert this simple JSON data to test.
This leads me to two questions: does it seem as though I'm correct in my assumption that I need multipart/form-data to get this to work correctly? And if so, do I need to put my JSON data into a text file and use one of those modules out there to turn it into multipart, or is there some way to do it without creating a file?

Maybe you want to try the requests lib, You can pass a files param, then requests will send a multipart/form-data POST instead of an application/x-www-form-urlencoded POST. You are not limited to using actual files in that dictionary, however:
import requests
response = requests.post('http://httpbin.org/post', files=dict(foo='bar'))
print response.status_code
If you want to know more about the requests lib, and specially in sending multipart forms take a look at this:
http://docs.python-requests.org/en/master/
and
http://docs.python-requests.org/en/master/user/advanced/?highlight=Multipart-Encoded%20Files