How do you parse the JSON response from the musixmatch api?
I'm calling the method
res = artist_api.artist_search_get(format=format, q_artist=artist)
I'm getting a response of
{'message': {'body': {'artist_list': [{'artist': {'artist_alias_list': [],
'artist_comment': '',
'artist_country': '',
'artist_credits': {'artist_list': []},
'artist_edit_url': None,
'artist_id': 26575484.0,
'artist_mbid': None,
'artist_name': 'Illenium',
'artist_name_translation_list': [],
'artist_rating': 55.0, .........
I'm trying to get the artist_id.
I tried getting the artist_id like this:
print(res['message']['body']['artist']['artist_id'])
artist_api = swagger_client.ArtistApi()
format = 'json'
try:
artist_list = ["adele", "lady gaga", "john legend"];
for artist in artist_list:
print(artist)
res = artist_api.artist_search_get(format=format, q_artist=artist)
print(res['message']['body']['artist']['artist_id'])
except ApiException as e:
print "Exception when calling ArtistApi->artist_search_get: %s\n" % e
I'm getting this error message:
Traceback (most recent call last):
File "musixmatch.py", line 39, in <module>
print(res['message']['body']['artist_id'])
TypeError: 'InlineResponse2004' object has no attribute '__getitem__'
Please help I've searched through a bunch of threads but I still can't find the answer to this. I'm new to python so I'm not sure what I'm doing wrong.
You a little messed up with the hierarchy of JSON-document, use this code to get desired values:
[artist['artist']['artist_id'] for artist in res['message']['body']['artist_list']]
As an alternative, can be used JSONPath-expression: $..artist_list..artist_id to get the same result.
Example
from jsonpath_rw import parse
import json
res = json.loads("""{
"message": {
"header": {},
"body": {
"artist_list": [
{
"artist": {
"artist_credits": { },
"artist_country": "string",
"artist_id": 110
}
},
{
"artist": {
"artist_credits": {},
"artist_country": "string",
"artist_id": 220
}
}
]
}
}
}""")
# 1-way: Iterate through JSON-document
print([artist['artist']['artist_id'] for artist in res['message']['body']['artist_list']])
# result: [110, 220]
# 2-way: Use JSONPath-expression
jsonpath_expr = parse('$..artist_list..artist_id')
print([match.value for match in jsonpath_expr.find(res)])
# result: [110, 220]
Related
I am currently writing a script to scrape data from an API into a Python dictionary and then export the result into a JSON file. I am trying to get the file extension from a response by splitting using .rsplit('.', 1)[-1] The only problem is that some keys have 'None" as their value and this throws the AttributeError: 'NoneType' object has no attribute 'rsplit'. Here is my code snippet:
d = requests.get(dataset_url)
output = d.json()
output_dict = {
'data_files': {
# Format to get only extension
'format': output.get('connectionParameters', {}).get('url').rsplit('.', 1)[-1],
'url': output.get('connectionParameters', {}).get('url'),
},
}
An example of JSON response with the required key is as follows:
"connectionParameters": {
"csv_escape_char": "\\",
"protocol": "DwC",
"automation": false,
"strip": false,
"csv_eol": "\\n",
"csv_text_enclosure": "\"",
"csv_delimiter": "\\t",
"incremental": false,
"url": "https://registry.nbnatlas.org/upload/1564481725489/London_churchyards_dwc.txt",
"termsForUniqueKey": [
"occurrenceID"
]
},
Any way to tackle this?
Try:
url = output.get("connectionParameters", {}).get("url") or "-"
f = url.rsplit(".", 1)[-1]
output_dict = {
"data_files": {
"format": f,
"url": url,
},
}
This will print:
{'data_files': {'format': '-', 'url': '-'}}
If the "url" parameter is None.
For a flask based python API project, I have to return the json response like below:
{
"images":
[
{
"transaction":
{
"message": "match found",
"status": "success",
"subjectId": 79,
"confidence": 0.56
}
}
]
}
In above json, images looks like a list and transaction looks like a dict which means that the response is list of dict. Now below is the code I have in order to achieve above format:
#app.route('/api/v1/face/recognize', methods=['POST'])
def recognize():
"""
SOME CODE
"""
images = []
transaction = dict()
transaction['status'] = 'success'
transaction['message'] = "match found"
transaction['subjectId'] = 79
transaction['confidence'] = 0.56
images.append(transaction)
return jsonify(images), 200
But it is giving me below response:
[
{
"confidence": 0.56,
"message": "match found",
"status": "success",
"subject_id": 79
}
]
which doesnt looks like what I wanted. I am unable to understand how can I make the response format look like the above mentioned. Can anyone please help. Thanks
You are very close. use images.append({'transaction': transaction})
Ex:
#app.route('/api/v1/face/recognize', methods=['POST'])
def recognize():
"""
SOME CODE
"""
images = []
transaction = dict()
transaction['status'] = 'success'
transaction['message'] = "match found"
transaction['subjectId'] = 79
transaction['confidence'] = 0.56
images.append({'transaction': transaction})
images = {'images': images}
return jsonify(images), 200
Output:
{'images': [{'transaction': {'confidence': 0.56,
'message': 'match found',
'status': 'success',
'subjectId': 79}}]}
I am doing some thing wrong over here, while comparing two images in different S3 Bucket.
Even though, I am comparing images of male and female it would give 99% confidence
or am i missing something in the declaration yet
Maybe This line is causing a problem
key_target = "targett/" + key
Or my event code is error prone this is where i have mentioned my source bucket ,even though i have mentioned it in lambda function for testing below. What else do i need to correct so that it will return the confidence within the rang specified
from __future__ import print_function
import boto3
from decimal import Decimal
import json
import urllib
print('Loading function')
rekognition = boto3.client('rekognition')
#iot = boto3.client('iot-data')
# --------------- Helper Functions to call Rekognition APIs ------------------
def compare_faces(bucket, key, key_target, threshold=75):
response = rekognition.compare_faces(
SourceImage={
"S3Object": {
"Bucket": 'dacss',
"Name": 'obama.jpg',
}
},
TargetImage={
"S3Object": {
"Bucket": 'targett',
"Name": 'michelle.jpg',
}
},
SimilarityThreshold=threshold,
)
return response['SourceImageFace'], response['FaceMatches']
# --------------- Main handler ------------------
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
bucket = event['Records'][0]['s3']['bucket']['name']
key = urllib.unquote_plus(event['Records'][0]['s3']['object']
['key'].encode('utf8'))
key_target = "targett/" + key
try:
response = compare_faces(bucket, key, key_target)
print(response)
# mypayload = json.dumps(response)
# iotResponse = iot.publish(
# topic="rekognition/result",
# qos=1,
# payload=mypayload)
# print(iotResponse)
# return iotResponse
print(response)
return response
except Exception as e:
print(e)
print("Error processing object {} from bucket {}. ".format(key,
bucket)
+
"Make sure your object and bucket exist and your bucket is in
the
same region as this function.")
raise e
---------------output-----------------
Response:
[
{
"BoundingBox": {
"Width": 0.7813892960548401,
"Top": 0.15193353593349457,
"Left": 0.1047489121556282,
"Height": 0.8365015387535095
},
"Confidence": 99.99993896484375
},
[]
]
i think you are having some misunderstanding here
{
'FaceMatches': [
{
'Face': {
'BoundingBox': {
'Height': 0.33481481671333313,
'Left': 0.31888890266418457,
'Top': 0.4933333396911621,
'Width': 0.25,
},
'Confidence': 99.9991226196289,
},
'Similarity': 100,
},
],
'SourceImageFace': {
'BoundingBox': {
'Height': 0.33481481671333313,
'Left': 0.31888890266418457,
'Top': 0.4933333396911621,
'Width': 0.25,
},
'Confidence': 99.9991226196289,
},
'ResponseMetadata': {
'...': '...',
},
}
Here the confidence score does'nt show weather the face matches or not it shows that it found the face in the image. "Similarty" shows the actual match of the image.
These lines:
TargetImage={
"S3Object": {
"Bucket": targett,
"Name": obama.jpg,
}
should be:
TargetImage={
"S3Object": {
"Bucket": 'targett',
"Name": key_target,
}
I am creating a Python script to parse the JSON response from https://vulners.com/api/v3/search/stats/
I have the following code in my .py:
import json
import requests
response = requests.get('https://vulners.com/api/v3/search/stats/')
vuln_set = json.loads(response.text)
vuln_type = vuln_set['data']['type_results']
vuln_bulletinfamily = vuln_set['data']['type_results'][vuln_type]['bulletinFamily']
vuln_name = vuln_set['data']['type_results'][vuln_type]['displayName']
print("Type: " + vuln_type)
print("Bulletin Family: " + vuln_bulletinfamily)
print("Name: " + vuln_name)
I need to get the vuln_type aswell as the child information (vuln_bulletinfamily & vuln_name)
An excerpt from the JSON response:
"data": {
"type_results": {
"aix": {
"lastUpdated": [],
"bulletinFamily": "unix",
"displayName": "IBM AIX",
"lastrun": "2017-09-14T14:04:56",
"count": 110,
"workTime": "0:00:10.983795"
},
"akamaiblog": {
"lastUpdated": [],
"bulletinFamily": "blog",
"displayName": "Akamai Blog",
"lastrun": "2017-09-14T10:38:52",
"count": 1463,
"workTime": "0:00:00.358691"
},
"amazon": {
"lastUpdated": [],
"bulletinFamily": "unix",
"displayName": "Amazon Linux AMI",
"lastrun": "2017-09-14T14:17:40",
"count": 889,
"workTime": "0:00:01.839594"
},
I am getting an error of TypeError: unhashable type: 'dict'
Traceback:
Traceback (most recent call last):
File "test.py", line 9, in <module>
vuln_bulletinfamily = vuln_set['data']['type_results'][vuln_type]['bulletinFamily']
TypeError: unhashable type: 'dict'
In the traceback line, the next line and the first print line, you are trying to access a dict type_results and vuln_type with a key that is also a dictionary.
You need to loop through the keys, like:-
import json
import requests
response = requests.get('https://vulners.com/api/v3/search/stats/')
vuln_set = json.loads(response.text)
vuln_type = vuln_set['data']['type_results']
for k in vuln_type :
vuln_bulletinfamily = vuln_set['data']['type_results'][k]['bulletinFamily']
vuln_name = vuln_set['data']['type_results'][k]['displayName']
print("Type: " + k)
print("Bulletin Family: " + vuln_bulletinfamily)
print("Name: " + vuln_name)
vuln_set = json.loads(response.text) vs response.json()
I try to get all lattitudes and longtitudes from this json.
Code:
import urllib.parse
import requests
raw_json = 'http://live.ksmobile.net/live/getreplayvideos?userid='
print()
userid = 735890904669618176
#userid = input('UserID: ')
url = raw_json + urllib.parse.urlencode({'userid=': userid}) + '&page_size=1000'
print(url)
json_data = requests.get(url).json()
print()
for coordinates in json_data['data']['video_info']:
print(coordinates['lat'], coordinates['lnt'])
print()
Error:
/usr/bin/python3.6 /media/anon/3D0B8DD536C9574F/PythonProjects/getLocation/getCoordinates
http://live.ksmobile.net/live/getreplayvideos?userid=userid%3D=735890904669618176&page_size=1000
Traceback (most recent call last):
File "/media/anon/3D0B8DD536C9574F/PythonProjects/getLocation/getCoordinates", line 17, in <module>
for coordinates in json_data['data']['video_info']:
TypeError: list indices must be integers or slices, not str
Process finished with exit code 1
Where do I go wrong?
In advance, thanks for your help and time.
I just post some of the json to show what it looks like.
The json looks like this:
{
"status": "200",
"msg": "",
"data": {
"time": "1499275646",
"video_info": [
{
"vid": "14992026438883533757",
"watchnumber": "38",
"topicid": "0",
"topic": "",
"vtime": "1499202678",
"title": "happy 4th of july",
"userid": "735890904669618176",
"online": "0",
"addr": "",
"isaddr": "2",
"lnt": "-80.1282576",
"lat": "26.2810628",
"area": "A_US",
"countryCode": "US",
"chatSystem": "1",
},
Full json: https://pastebin.com/qJywTqa1
Your URL construction is incorrect. The URL you have built (as shown in the output of your script) is:
http://live.ksmobile.net/live/getreplayvideos?userid=userid%3D=735890904669618176&page_size=1000
Where you actually want this:
http://live.ksmobile.net/live/getreplayvideos?userid=735890904669618176&page_size=1000
So your were actually getting this JSON in your response:
{
"status": "200",
"msg": "",
"data": []
}
Which is why you were seeing that error.
Here is the corrected script:
import urllib.parse
import requests
raw_json = 'http://live.ksmobile.net/live/getreplayvideos?'
print()
userid = 735890904669618176
#userid = input('UserID: ')
url = raw_json + urllib.parse.urlencode({'userid': userid}) + '&page_size=1000'
print(url)
json_data = requests.get(url).json()
print()
for coordinates in json_data['data']['video_info']:
print(coordinates['lat'], coordinates['lnt'])
print()
According to your posted json, you have problem in this statement-
print(coordinates['lat'], coordinates['lnt'])
Here coordinates is a list having only one item which is dictionary. So your statement should be-
print(coordinates[0]['lat'], coordinates[0]['lnt'])