Develop Reference

Python : JSON Error: Syntax error, malformed JSON

My Python code, which I obtained from POSTMAN, throws an error with the string
"code":"E101","message":"JSON Error: Syntax error, malformed JSON"}"
although in POSTMAN, the request was successful and produced the expected JSON result.
below is my python code
import requests
url = "APIURL"
payload={'data': '{
"authenticate":{
"apikey":"ABCSHF"
},
"services":[
{
"call":"history/API",
"identifier":{
"search":"desc"
}
}
]
}'}
files=[
]
headers = {
}
response = requests.request("POST", url, headers=headers, json=payload, files=files)
print(response.text)
Can anyone please help me on this.

I guess you should get clearness what did you code ;):
requests parse a dict to a JSON internally, see explanation here
import json
from pprint import pprint
# some JSON string:
json_string = '{ "authenticate":{ "apikey":"ABCSHF" }, "services":[ { "call":"history/API", "identifier":{ "search":"desc" } } ] }'
# parse JSON string to dict:
json_as_dict = json.loads(json_string)
pprint(json_as_dict)
# >> {'authenticate': {'apikey': 'ABCSHF'},
# >> 'services': [{'call': 'history/API', 'identifier': {'search': 'desc'}}]}
incorrect_payload_as_dict = {'data': json.dumps(json_as_dict)}
pprint(incorrect_payload_as_dict)
# >> {'data': '{"authenticate": {"apikey": "ABCSHF"}, "services": [{"call": '
# >> '"history/API", "identifier": {"search": "desc"}}]}'}
correct_payload_as_dict = {'data': json_as_dict}
pprint(correct_payload_as_dict)
# >> {'data': {'authenticate': {'apikey': 'ABCSHF'},
# >> 'services': [{'call': 'history/API',
# >> 'identifier': {'search': 'desc'}}]}}

response 422 after trying to send post request with json file

I have been trying to find an answer that work on my case but had no success.
I am getting a response 422 when running the code below. I have replaced the json in the response variable and also used json.dumps around the data variable but I am still getting this response. Would you be able to find the issue within my json file? Thanks a lot
todaystr = datetime.today().strftime('%Y-%m-%d')
today = datetime.strptime(todaystr, '%Y-%m-%d')
first_day_of_month = today.replace(day=1)
day_count_str = today - first_day_of_month
day_count_int = day_count_str / timedelta(days=1)
day_count_split = str(day_count_int)
sep = '.'
day_count = day_count_split.split(sep, 1)[0]
dt = str(first_day_of_month.timestamp())
data = {"requestId": "numberOfUniqueAccountsAndVisitors", "timeSeries": {"period": "dayRange", "count": day_count, "first": dt}, "source": {"events": None}, "pipeline": [{"reduce": [{"reduce": {"visitors": {"count": "visitorId"}, "accounts": {"count": "accountId"}}}]}]}
headers = {
'x-pendo-integration-key': API_Key,
'content-type': "application/json"
}
response = requests.post(url, json=data, headers=headers)
print(response)
response_dictionary = json.loads(response.content)
print(response_dictionary)

In the payload to the Pendo Aggregation API, the request object should not be an array. And make sure the "key" should be named as "request"(singular), not as "requests"(plural).
Try something like this
{
"response": {
"mimeType": "application/json"
},
"request": {
"name": "accounts-last7days",
"requestId": "accounts-last7days"
}
}

Extracting specific JSON values in python

JSON return from spotify api. Example:
{
"tracks": {
"href": "https://api.spotify.com/v1/search?query=Stero+Hearts&type=track&offset=0&limit=1",
"items": [
{
"album": {
"album_type": "album",
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/4IJczjB0fJ04gs4uvP0Fli"
},
"href": "https://api.spotify.com/v1/artists/4IJczjB0fJ04gs4uvP0Fli",
"id": "4IJczjB0fJ04gs4uvP0Fli",
"name": "Gym Class Heroes",
"type": "artist",
"uri": "spotify:artist:4IJczjB0fJ04gs4uvP0Fli"
}
]
}
}
]
}
}
Broken Code
import requests, json
spotifytrack = input("Name of Song?\\n")
link = "https://api.spotify.com/v1/search?q=" + spotifytrack + "&type=track&limit=1"
token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
header = {
"Authorization": "Bearer {}".format(token),
"Content-Type": "application/json",
"Accept": "application/json",
}
auth_response = requests.get(link, headers=header)
pretty_response = json.dumps(auth_response.json(), indent=4)
data_by_user = {}
for d in auth_response:
data_by_user[d["artist"]] = d
print(data_by_user["uri"])
"""
def find_track_from_json(auth_response, artist):
return [p for p in auth_response if p["artist"] == artist][0]["uri"]
urii = find_track_from_json(auth_response, "uri")
print(urii)
x = load.json(auth_response.json())
print("Here is the data whic we have imported\n")
print(pretty_response)
print(x["name"])
print(x["uri"])
print(x["spotify"])
"""
Errors noticed:
File "spotify.py", line 19, in <module>
data_by_user[d["artist"]] = d
TypeError: byte indices must be integers or slices, not str
The aim is to convert word search to link in a cli application.
I tried load.json which i saw in some website and also tried def.
I expected the program to find out the artist name and uri from the json and print it in the cli interface.

You are iterating over the encoded json string:
auth_response = requests.get(link, headers=header)
for d in auth_response:
Python is complaining that you aren't providing a numerical index, which is correct as auth_response is just a string!
You should call json.loads to decode the string, and then you can iterate over it.
auth_response = requests.get(link, headers=header)
decoded_auth_response = json.loads(auth_response)
data_by_user = {}
for d in decoded_auth_response:
data_by_user[d["artist"]] = d
As you haven't provided the full json output from the API call I'm not sure what data is actually in decoded_auth_response, and you haven't described what your expected output would look like, so you may need to do some more work to find the correct data in each iteration.

The result from requests.get() is a requests.Response object. As far as I can see you want to iterate over the response body which is JSON. The requests.Response object has a .json() method which returns a dict from the response JSON.
Looking at the response you would probably want to iterate over resp_json['tracks']['items'] which is a list.
So to summarize your code should look something like this:
auth_response = requests.get(link, headers=header)
items = auth_response.json()['tracks']['items']
for d in items:
print(d)

Post files with json data using python requests

I have API that accepts multiple files with json data -
curl -X POST \
http://localhost:25965/v1/import \
-H 'Content-Type: application/json' \
-H 'Postman-Token: xxxxxxxx-xxxx-xxxx-xxxx-ba66a9b8d6cb' \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-F 'files=#C:\Users\user\File1.xlsx' \
-F 'files=#C:\Users\user\File2.xlsx' \
-F 'files=#C:\Users\user\File3.xlsx' \
-F 'values=[
{
"field1": "Value11",
"field2": "Value21",
"field3": "File1.xlsx"
},
{
"field1": "Value21",
"field2": "Value22",
"field3": "File2.xlsx"
},
{
"field1": "Value31",
"field2": "Value32",
"field3": "File3.xlsx"
}
]'
I'm trying to convert this request into python.
With requests, I get error too many values to unpack (expected 2)
def PostFiles(self, apiName, values, files):
url = self.ApiUrl + apiName
params = {}
for file in files:
params.update({'files':file})
response = requests.post(url,
files=params,
data=values,
headers={
'Content-Type':'application/json'},
auth=HTTPKerberosAuth(delegate=True))
return response
I also tried requests_toolbelt but doesn't work.
def PostFiles(self, apiName, values, files):
url = self.ApiUrl + apiName
params = {}
for file in files:
params.update({'files':file})
params.update({'values':json.dumps(values)})
multipart_data = MultipartEncoder(params)
headers = { 'Content-Type':multipart_data.content_type}
response = requests.post(url,
data=multipart_data,
headers=headers,
auth=HTTPKerberosAuth(delegate=True))
return response
this is my function call -
files = [open(join(directory, filename), 'rb')]
#files = [('files',(filename, open(join(directory, filename), 'rb'),'application/vnd.ms-excel'))]
values = [{
'field1': 'value11',
'field2' : 'value21',
'field3' : 'File1.xlsx'
}]
response = PostFiles('import', values, files)
print(response)
there are posts related to this topic but I couldn't find anything with multiple files posting with json.

I modified my code as per Post JSON and file in single request and worked as expected.
def PostFiles(self, apiName, values, files):
url = self.ApiUrl + apiName
params = {
'values': (None, json.dumps(values), 'application/json')
}
for file in files:
params.update({'files':file})
response = requests.post(url,
files=params,
auth=HTTPKerberosAuth(delegate=True))
result = response.json()
if response.status_code != 200 :
raise Exception(result)
return json.dumps(result)

JSONDecodeError: Expecting value: line 1 column 1 (char 0) error

I am facing this error while making request to fetch json from api.
I can get json data using the "/v1/articles' path.
conn = http.client.HTTPSConnection("api.xxxx.com.tr")
headers = {
'accept': "application/json",
'apikey': "cd6b6c96799847698d87dec9f9a731d6"
}
filter = "daily"
conn.request("GET", "/v1/articles", headers=headers)
reader = codecs.getreader("utf-8")
res = conn.getresponse()
data = json.load(reader(res))
json.dumps(data)
return data
But i am having JSONDecodeError if i set filter. Code:
conn = http.client.HTTPSConnection("api.xxxx.com.tr")
headers = {
'accept': "application/json",
'apikey': "cd6b6c96799847698d87dec9f9a731d6"
}
conn.request("GET", "/v1/articles?$filter=Path eq '/daily/'", headers=headers)
reader = codecs.getreader("utf-8")
res = conn.getresponse()
data = json.load(reader(res))
json.dumps(data)
return data
I tried same filter using Postman with no error and i can get Json data.
Returned Json data from Postman:
[
{
"Id": "40778196",
"ContentType": "Article",
"CreatedDate": "2018-03-20T08:28:05.385Z",
"Description": "İspanya'da 2016 yılında çalınan lüks otomobil, şasi numarası değiştirilerek Bulgaristan üzerinden getirildiği Türkiye'de bulundu.",
"Files": [
{
"FileUrl": "http://i.xxxx.com/i/xxxx/98/620x0/5ab0c6a9c9de3d18a866eb54.jpg",
"Metadata": {
"Title": "",
"Description": ""
}
}
],
"ModifiedDate": "2018-03-20T08:32:12.001Z",
"Path": "/gundem/",
"StartDate": "2018-03-20T08:32:12.001Z",
"Tags": [
"ispanya",
"Araç",
"Hırsız",
"Dolandırıcı"
],
"Title": "İspanya'da çalınan lüks araç Türkiye'de bulundu!",
"Url": "http://www.xxxx.com.tr/gundem/ispanyada-calinan-luks-arac-turkiyede-bulundu-40778196"
}
]
I can not figure out the problem. It would be great if anyone help me about this issue. Thank you.

I finally figured out the problem! Using the requests library have solved my problem now I can filter the api request.
data = requests.get('https://api.xxxxx.com.tr/v1/articles', headers =
headers, params={"$filter":"Path eq '/xxxxxx/'"}).json()
I am leaving this answer here for anyone else who can need this solution in the future.
Thanks for all your suggestions.

The problem is in the following line
data = json.load(reader(res))
when your response is not a json string, JSONDecodeError occurs. so, add an additional logic to see if the response is None or a json string. First thing, print the reader(res) and see what the return is