Develop Reference

Python : JSON Error: Syntax error, malformed JSON

My Python code, which I obtained from POSTMAN, throws an error with the string
"code":"E101","message":"JSON Error: Syntax error, malformed JSON"}"
although in POSTMAN, the request was successful and produced the expected JSON result.
below is my python code
import requests
url = "APIURL"
payload={'data': '{
"authenticate":{
"apikey":"ABCSHF"
},
"services":[
{
"call":"history/API",
"identifier":{
"search":"desc"
}
}
]
}'}
files=[
]
headers = {
}
response = requests.request("POST", url, headers=headers, json=payload, files=files)
print(response.text)
Can anyone please help me on this.

I guess you should get clearness what did you code ;):
requests parse a dict to a JSON internally, see explanation here
import json
from pprint import pprint
# some JSON string:
json_string = '{ "authenticate":{ "apikey":"ABCSHF" }, "services":[ { "call":"history/API", "identifier":{ "search":"desc" } } ] }'
# parse JSON string to dict:
json_as_dict = json.loads(json_string)
pprint(json_as_dict)
# >> {'authenticate': {'apikey': 'ABCSHF'},
# >> 'services': [{'call': 'history/API', 'identifier': {'search': 'desc'}}]}
incorrect_payload_as_dict = {'data': json.dumps(json_as_dict)}
pprint(incorrect_payload_as_dict)
# >> {'data': '{"authenticate": {"apikey": "ABCSHF"}, "services": [{"call": '
# >> '"history/API", "identifier": {"search": "desc"}}]}'}
correct_payload_as_dict = {'data': json_as_dict}
pprint(correct_payload_as_dict)
# >> {'data': {'authenticate': {'apikey': 'ABCSHF'},
# >> 'services': [{'call': 'history/API',
# >> 'identifier': {'search': 'desc'}}]}}

Extracting specific JSON values in python

JSON return from spotify api. Example:
{
"tracks": {
"href": "https://api.spotify.com/v1/search?query=Stero+Hearts&type=track&offset=0&limit=1",
"items": [
{
"album": {
"album_type": "album",
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/4IJczjB0fJ04gs4uvP0Fli"
},
"href": "https://api.spotify.com/v1/artists/4IJczjB0fJ04gs4uvP0Fli",
"id": "4IJczjB0fJ04gs4uvP0Fli",
"name": "Gym Class Heroes",
"type": "artist",
"uri": "spotify:artist:4IJczjB0fJ04gs4uvP0Fli"
}
]
}
}
]
}
}
Broken Code
import requests, json
spotifytrack = input("Name of Song?\\n")
link = "https://api.spotify.com/v1/search?q=" + spotifytrack + "&type=track&limit=1"
token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
header = {
"Authorization": "Bearer {}".format(token),
"Content-Type": "application/json",
"Accept": "application/json",
}
auth_response = requests.get(link, headers=header)
pretty_response = json.dumps(auth_response.json(), indent=4)
data_by_user = {}
for d in auth_response:
data_by_user[d["artist"]] = d
print(data_by_user["uri"])
"""
def find_track_from_json(auth_response, artist):
return [p for p in auth_response if p["artist"] == artist][0]["uri"]
urii = find_track_from_json(auth_response, "uri")
print(urii)
x = load.json(auth_response.json())
print("Here is the data whic we have imported\n")
print(pretty_response)
print(x["name"])
print(x["uri"])
print(x["spotify"])
"""
Errors noticed:
File "spotify.py", line 19, in <module>
data_by_user[d["artist"]] = d
TypeError: byte indices must be integers or slices, not str
The aim is to convert word search to link in a cli application.
I tried load.json which i saw in some website and also tried def.
I expected the program to find out the artist name and uri from the json and print it in the cli interface.

You are iterating over the encoded json string:
auth_response = requests.get(link, headers=header)
for d in auth_response:
Python is complaining that you aren't providing a numerical index, which is correct as auth_response is just a string!
You should call json.loads to decode the string, and then you can iterate over it.
auth_response = requests.get(link, headers=header)
decoded_auth_response = json.loads(auth_response)
data_by_user = {}
for d in decoded_auth_response:
data_by_user[d["artist"]] = d
As you haven't provided the full json output from the API call I'm not sure what data is actually in decoded_auth_response, and you haven't described what your expected output would look like, so you may need to do some more work to find the correct data in each iteration.

The result from requests.get() is a requests.Response object. As far as I can see you want to iterate over the response body which is JSON. The requests.Response object has a .json() method which returns a dict from the response JSON.
Looking at the response you would probably want to iterate over resp_json['tracks']['items'] which is a list.
So to summarize your code should look something like this:
auth_response = requests.get(link, headers=header)
items = auth_response.json()['tracks']['items']
for d in items:
print(d)

How can I filter author Ids out of this json?

I'm specifically trying to get the id out of "author" to write it to a json file, But I have no idea how to filter out the ids and then write them to the file. Im kinda new to this
[
{
"id":"865170257588125696",
"content":"<#862323352869797948> I set your AFK: yes",
"channel_id":"859115914444537876",
"author":{
"id":"155149108183695360",
"username":"Dyno",
"discriminator":"3861",
}
},
{
"id":"865170254677409812",
"content":"?afk yes",
"channel_id":"859115914444537876",
"author":{
"id":"862323352869797948",
"username":"zz2w2wfff2",
"discriminator":"2401"
}
},
{
"id":"865170247349567488",
"content":"Meh",
"channel_id":"859115914444537876",
"author":{
"id":"862323352869797948",
"username":"zz2w2wfff2",
"discriminator":"2401"
}
}
]
This is all I have:
r = requests.get("https://discord.com/api/v9/channels/" + channelid + "/messages?limit=50", headers=headers)
if r.status_code == 200:
authorid = r.json()[0]["author"]["id"]
for authorid in r.text:
print(authorid)
All that did was show gibberish when I ran it.

Based on your code, this should get you going. You don't need to access the r response anymore after you've parsed it using r.json():
r = requests.get("https://discord.com/api/v9/channels/" + channelid + "/messages?limit=50", headers=headers)
r.raise_for_status()
data = r.json()
for message in data:
print(message["author"]["id"])

Getting a specific value of JSON data

I'm getting a JSON data from RESTCONF HTTPS request, using the following code
https_request = 'https://' + host + '/restconf/data/' + operation
headers = {'Content-type': 'application/yang-data+json', 'Accept': 'application/yang-data+json'}
r = requests.get(https_request, auth=(user, password), headers=headers, verify=False)
print r.json()
The data I got is the following:
{
"Cisco-IOS-XE-segment-routing:ipv4": {
"prefixes": [
{
"ipprefix": "1.1.1.1/32",
"index": {
"range-start": 333,
"range": 1
}
}
]
}
}
Basically, I want to return the field's "range-start" value which is 333. I tried the following but it did not work.
for element in r:
id = element['range-start']
print(id)
Is there anyway to get that value?

From Python Console:
>>> import json
... data = json.loads('{"Cisco-IOS-XE-segment-routing:ipv4": {"prefixes": [{"ipprefix": "1.1.1.1/32", "index": {"range-start": 333, "range": 1}}]}}')
... print(data['Cisco-IOS-XE-segment-routing:ipv4']['prefixes'][0]['index']['range-start'])
333
>>>

You need to start at the beginning of the JSON and work your way to the key you want. To do this you need to start at Cisco-IOS-XE-segment-routing:ipv4.
prefixes = r.json()["Cisco-IOS-XE-segment-routing:ipv4"]["prefixes"]
id = prefixes[0]["index"]["range-start"]
If there are multiple prefixes you can loop over them and access each range-start.

Since you are looping over elements, I would suggest this approach using a helper function:
def get_id(element):
prefixes = r.json()["Cisco-IOS-XE-segment-routing:ipv4"]["prefixes"]
id = prefixes[0]["index"]["range-start"]
return id
Then you can do, as in your question:
for element in r:
id = get_id(element)
print(id)

post data using python-requests

I'm trying to post the following data. But I'm getting an error. Can you please take look? Thanks a lot.
I'm posting the same data using Postman. And it works.
def _build_post_data(bike_instance):
"""
data = {
"apikey": "XXX",
"data": {
"created_at": "date_XX",
"Price": "Decimal_XX"
}
}
"""
data = {}
raw_data = serializers.serialize('python', [bike_instance])
actual_data = [d['fields'] for d in raw_data]
data.update(
{
"apikey": XXX,
"data": actual_data[0]
}
)
return data
Posting data
bike = Bike.objects.get(pk=XXX)
data = _build_post_data(bike)
dump_data = json.dumps(data, cls=DjangoJSONEncoder)
requests.post(url, data=dump_data)
error
u'{"error":{"message":"422 Unprocessable Entity","errors":[["The data field is required."],["The apikey field is required."]],"status_code":422}}'
data and apikey already in the dict. then why I'm getting an error? Any idea?
Postman works

With Postman you are sending a multipart/form-data request, with requests you only send JSON (the value of the data field in Postman), and are not including the apikey field.
Use a dictionary with the JSON data as one of the values, and pass that in as the files argument. It probably also works as the data argument (sent as application/x-www-urlencoded):
form_structure = {'apikey': 'XXXX', 'data': dump_data}
requests.post(url, files=form_structure)
# probably works too: requests.post(url, data=form_structure)