Merging nested dictionary comparing nested values - python

I am trying to merge two nested dictionaries that will merge different dictionaries by comparing values, If the key will be same it will be replaced with the recent value.
Im using always_merger.merge() for merging but is appending the different element.It supposed to compare and replace values with recent one.
dictionary_1 = dict([('modifyParameterReq', {
'managedObject': {
'distName': 'ethLink_1',
'operation': 'create_mo',
'parameter': [
{
'parameterName': 'houseName',
'newValue': 'Bhawan',
'prevValue': ''
},
{
'parameterName': 'subscribe',
'newValue': 'TRUE',
'prevValue': ''
},
{
'parameterName': 'remoteMacAddress',
'newValue': 'd6-68-05-5e-06-b9',
'prevValue': ''
}
],
'class': 'EthernetLink'
}
})]
)
dictionary_2 = dict([('modifyParameterReq', {
'managedObject': {
'distName': 'ethLink_1',
'operation': 'create_mo',
'parameter': [
{
'parameterName': 'subscribe',
'newValue': 'FALSE',
'prevValue': ''
},
{
'parameterName': 'remoteMacAddress',
'newValue': 'a1-b1-c3-d4-e5-f6',
'prevValue': ''
},
{
'parameterName': 'yourName',
'newValue': 'Vicky',
'prevValue': ''
}
],
'class': 'EthernetLink'
}
})
]
)
if the element is not present add the element else replace with new value.
Any help should be appreciated.

This should do it. I 've written some comments in between to make it more readable:
# Get all current parameter names from dictionary 1.
currentDictParameterNames = [elem['parameterName'] for elem in dictionary_1['modifyParameterReq']['managedObject']['parameter']]
# Iterate through parameters from dictionary 2.
for index, dictData in enumerate(dictionary_2['modifyParameterReq']['managedObject']['parameter']):
# If current key iterated from dictionary 2 exists in dictionary 1, update.
if dictData['parameterName'] in currentDictParameterNames:
# FInd the location of the parameter name in dictionary 1 and update.
for i in range(len(dictionary_1['modifyParameterReq']['managedObject']['parameter'])):
if dictionary_1['modifyParameterReq']['managedObject']['parameter'][i]['parameterName'] == dictData['parameterName']:
dictionary_1['modifyParameterReq']['managedObject']['parameter'][i]['prevValue'] = dictionary_1['modifyParameterReq']['managedObject']['parameter'][i]['newValue']
dictionary_1['modifyParameterReq']['managedObject']['parameter'][i]['newValue'] = dictData['newValue']
break
# Else create a new record.
else:
dictionary_1['modifyParameterReq']['managedObject']['parameter'].append(dictData)
print(dictionary_1)
The result should be:
{'modifyParameterReq': {'managedObject': {
'distName': 'ethLink_1',
'operation': 'create_mo',
'parameter': [
{'parameterName': 'houseName', 'newValue': 'Bhawan', 'prevValue': ''},
{'parameterName': 'subscribe', 'newValue': 'FALSE', 'prevValue': 'TRUE'},
{'parameterName': 'remoteMacAddress', 'newValue': 'a1-b1-c3-d4-e5-f6', 'prevValue': 'd6-68-05-5e-06-b9'},
{'parameterName': 'yourName', 'newValue': 'Vicky', 'prevValue': ''}],
'class': 'EthernetLink',
}}}

Related

How to get multiple key value pairs from list of JSON dictionaries

I have a JSON file called "hostnames" formatted like below
{
'propertyName': 'www.property1.com',
'propertyVersion': 1,
'etag': 'jbcas6764023nklf78354',
'rules': {
'name': 'default',
'children': [{
'name': 'Route',
'children': [],
'behaviors': [{
'name': 'origin',
'options': {
'originType': 'CUSTOMER',
'hostname': 'www.origin1.com',
and I wanted to get the values of keys "propertyName" and "hostname" and have a new JSON file like below
'properties': [{
'propertyName': 'www.property1.com',
'hostnames': ['www.origin1.com', 'www.origin2.com']
}, {
'propertyName': 'www.property1.com',
'hostnames': ['www.origin1.com', 'www.origin2.com']
}]
my code looks like this
hostnames = result.json()
hostnameslist = [host['hostname'] for host in hostnames['rules']['children']['behaviors']['options']]
print(hostnameslist)
but I'm getting the error
TypeError: list indices must be integers or slices, not str
You are trying to access a list elements with a string index ('behaviors').
Try:
hostnames = result.json()
hostnameslist = []
for child in hostnames['rules']['children']:
for behavior in child['behaviors']:
if behavior['name'] == 'origin':
hostnameslist.append(behavior['options']['hostname'])
properties = [{
'propertyName': hostnames['propertyName'],
'hostnames': hostnameslist
}]
Making an assumption about how the OP's data might be structured.
Recursive navigation of the dictionary to find all/any values associated with a dictionary key of 'hostname' appears to be well-suited here.
Doing it this way obviates the need for knowledge about the depth of the dictionary or indeed any of the dictionary key names except (obviously) 'hostname'.
Of course, there may be other dictionaries within the "master" dictionary that contain a 'hostname' key. If that's the case then this function may return values that are not needed/wanted.
data = {
'propertyName': 'www.property1.com',
'propertyVersion': 1,
'etag': 'jbcas6764023nklf78354',
'rules': {
'name': 'default',
'children': [{
'name': 'Route',
'children': [],
'behaviors': [{
'name': 'origin',
'options': {
'originType': 'CUSTOMER',
'hostname': 'www.origin1.com'
}
},
{
'name': 'origin',
'options': {
'originType': 'CUSTOMER',
'hostname': 'www.origin2.com'
}
}
]
}
]
}
}
def get_hostnames(d):
def _get_hostnames(_d, _l):
if isinstance(_d, dict):
if 'hostname' in _d:
_l.append(_d['hostname'])
else:
for _v in _d.values():
_get_hostnames(_v, _l)
else:
if isinstance(_d, list):
for _v in _d:
_get_hostnames(_v, _l)
return _l
return _get_hostnames(d, [])
result = {'properties': [{'propertyName': data.get('propertyName'), 'hostnames': get_hostnames(data)}]}
print(result)
Output:
{'properties': [{'propertyName': 'www.property1.com', 'hostnames': ['www.origin1.com', 'www.origin2.com']}]}

Extract value from list inside JSON dictionary (Python)

I need to extract a value from a dictionary, inside a list, inside a dictionary.
The value I'm tryng to get is dealId inside affectedDeals. (highlighted with ^)
data = {'date': '2022-11-04T12:36:57.016', 'status': 'OPEN', 'reason': 'SUCCESS', 'dealStatus': 'ACCEPTED', 'epic': 'SILVER', 'dealReference': 'o_0bc30104-8ddf-4d67-9daa-e7d878a8cad9', 'dealId': '006011e7-0055-311e-0000-000080507631', 'affectedDeals': [{'dealId': '006011e7-0055-311e-0000-000080507633', 'status': 'OPENED'}], 'level': 20.138, 'size': 1.0, 'direction': 'BUY', 'guaranteedStop': False, 'trailingStop': False}
^
output = 006011e7-0055-311e-0000-000080507633
May be this could help you to get the output as expected in question
Code:
data = {
"date": "2022-11-04T12:36:57.016",
"status": "OPEN",
"reason": "SUCCESS",
"dealStatus": "ACCEPTED",
"epic": "SILVER",
"dealReference": "o_0bc30104-8ddf-4d67-9daa-e7d878a8cad9",
"dealId": "006011e7-0055-311e-0000-000080507631",
"affectedDeals": [
{"dealId": "006011e7-0055-311e-0000-000080507633", "status": "OPENED"}
],
"level": 20.138,
"size": 1.0,
"direction": "BUY",
"guaranteedStop": False,
"trailingStop": False,
}
[i.get("dealId") for i in data.get("affectedDeals")]
Output :
006011e7-0055-311e-0000-000080507633
Output will be enclosed in list you can use indexing to retrieve the data from list like below
lst = [i.get("dealId") for i in data.get("affectedDeals")]
lst[0]
Try doing this:
data = {'date': '2022-11-04T12:36:57.016',
'status': 'OPEN',
'reason': 'SUCCESS',
'dealStatus': 'ACCEPTED',
'epic': 'SILVER',
'dealReference': 'o_0bc30104-8ddf-4d67-9daa-e7d878a8cad9',
'dealId': '006011e7-0055-311e-0000-000080507631',
'affectedDeals': [
{'dealId': '006011e7-0055-311e-0000-000080507633',
'status': 'OPENED'
}
],
'level': 20.138,
'size': 1.0,
'direction': 'BUY',
'guaranteedStop': False,
'trailingStop': False
}
print(data['affectedDeals'][0]['dealId'])
# Output: 006011e7-0055-311e-0000-000080507633
We basically accessed the list(affectedDeals) and then selected the 0th index, which is again a dictionary, so we accessed the specified key(dealID) and then print it.
Hope that helps

How to iterate over a JSON array and get values for a key which itself is a JSON object

I have been trying to do something simple yet something hard for me to solve it!
I have a json object that looks like:
jsonObject = {
'attributes': {
'192': { <--- This can be changed times to times meaning different number
'id': '192',
'code': 'hello',
'label': 'world',
'options': [
{
'id': '211',
'label': '5'
},
{
'id': '1202',
'label': '8.5'
},
{
'id': '54',
'label': '9'
},
{
'id': '1203',
'label': '9.5'
},
{
'id': '58',
'label': '10'
}
]
}
},
'template': '12345',
'basePrice': '51233',
'oldPrice': '51212',
'productId': 'hello',
}
and what I want to do is to get the values from options (To have both id and label saved into a list)
For now I only managed to do:
for att, value in jsonObject.items():
print(f"{att} - {value}"
How can I get the label and id?
You can try the following code:
attr = jsonObject['attributes']
temp = list(attr.values())[0] # It is same as "temp = attr['192']", but you said '192' can be changed.
options = temp['options']
for option in options:
print(f"id: {option['id']}, label: {option['label']}")

Python nmap: TypeError: list indices must be integers or slices, not str

I'm trying to get the target system port id using nmap module but getting list indices error
import nmap
nm_scan = nmap.PortScanner()
nm_scanner = nm_scan.scan('192.168.0.1', '80', arguments='-O')
print("Portid: " + nm_scanner['scan']['192.168.0.1']['portused']['portid'])
print("The host is: " +
nm_scanner['scan']['192.168.0.1']['portused']['portid'])
TypeError: list indices must be integers or slices, not str
print(nm_scanner)
{
'nmap': {
'command_line': 'nmap -oX - -p 80 -O 192.168.0.1',
'scaninfo': {
'tcp': {
'method': 'syn',
'services': '80'
}
},
'scanstats': {
'timestr': 'Fri Jul 26 11:31:30 2019',
'elapsed': '6.36',
'uphosts': '1',
'downhosts': '0',
'totalhosts': '1'
}
},
'scan': {
'192.168.0.1': {
'hostnames': [
{
'name': '',
'type': ''
}
],
'addresses': {
'ipv4': '192.168.0.1',
'mac': '1C:5F:2B:53:45:4F'
},
'vendor': {
'1C:5F:2B:53:45:4F': 'D-Link International'
},
'status': {
'state': 'up',
'reason': 'arp-response'
},
'uptime': {
'seconds': '87161',
'lastboot': 'Thu Jul 25 11:18:49 2019'
},
'tcp': {
80: {
'state': 'open',
'reason': 'syn-ack',
'name': 'http',
'product': '',
'version': '',
'extrainfo': '',
'conf': '3',
'cpe': ''
}
},
'portused': [
{
'state': 'open',
'proto': 'tcp',
'portid': '80'
},
{
'state': 'closed',
'proto': 'udp',
'portid': '42514'
}
],
'osmatch': [
{
'name': 'Allied Telesyn AT-GS950 or D-Link DES-3226L switch or D-Link DSL-2750U router',
'accuracy': '100',
'line': '2603',
'osclass': [
{
'type': 'switch',
'vendor': 'Allied Telesyn',
'osfamily': 'embedded',
'osgen': None,
'accuracy': '100',
'cpe': [
'cpe:/h:alliedtelesyn:at-gs950'
]
},
{
'type': 'switch',
'vendor': 'D-Link',
'osfamily': 'embedded',
'osgen': None,
'accuracy': '100',
'cpe': [
'cpe:/h:dlink:des-3226l',
'cpe:/h:dlink:dsl-2750u'
]
}
]
}
]
}
}
}
As you can see from the output, there are multiple ports, so you must use:
nm_scanner['scan']['192.168.0.1']['portused'][0]['portid']
for the first one, or traverse all using:
for port in nm_scanner['scan']['192.168.0.1']['portused']:
print(port['portid'])
You are using string indices for nm_scanner and not numbers. Is nm_scanner a dictionary or list? You can know by saying the following.
type(nm_scanner)
If it is a list then you have to use numbers as index (in the square brackets). You can use string for dictionary.
You can help us more by printing the nm_scanner and pasting here.

How would I read this data structure in Perl? Dictionary/Hash with keys containing lists containing lists. Python::Inline giving me errors

I've been struggling for about 3 weeks on this simple issue. I can't understand why and I would give anything to solve it lol.
I am trying to read values from the data structure below. The docs say it's a dictionary with keys containing lists of results of that type.
Example: I get the master query reply using an eval function. I lookup the key "song_hits" to get that structure. Then I lookup the key 'track' and parse it. The problem is getting to the 'track' part.
When I do it from how Perl docs tell me to, I get Can't locate object method "FIRSTKEY" via package "Inline::Python::Object::Data".
So I'm wondering if there's a way to read the value using the eval function to bypass ObjectData's hash key limitation, another way to read it given I know exact keys, or if I'm just doing this entirely wrong.
{
'album_hits': [
{
'album':
{
'albumArtRef': 'http://lh5.ggpht.com/DVIg4GiD6msHfgPs_Vu_2eRxCyAoz0fF...',
'albumArtist': 'J.Cole',
'albumId': 'Bfp2tuhynyqppnp6zennhmf6w3y',
'artist': 'J.Cole',
'artistId': ['Ajgnxme45wcqqv44vykrleifpji'],
'description_attribution':
{
'kind': 'sj#attribution',
'license_title': 'Creative Commons Attribution CC-BY',
'license_url': 'http://creativecommons.org/licenses/by/4.0/legalcode',
'source_title': 'Freebase',
'source_url': ''
},
'explicitType': '1',
'kind': 'sj#album',
'name': 'Work Out',
'year': 2011
},
'type': '3'
}],
'artist_hits': [
{
'artist':
{
'artistArtRef': 'http://lh3.googleusercontent.com/MJe-cDw9uQ-pUagoLlm...',
'artistArtRefs': [
{
'aspectRatio': '2',
'autogen': False,
'kind': 'sj#imageRef',
'url': 'http://lh3.googleusercontent.com/MJe-cDw9uQ-pUagoLlmKX3x_K...'
}],
'artistId': 'Ajgnxme45wcqqv44vykrleifpji',
'artist_bio_attribution':
{
'kind': 'sj#attribution',
'source_title': 'David Jeffries, Rovi'
},
'kind': 'sj#artist',
'name': 'J. Cole'
},
'type': '2'
}],
'playlist_hits': [
{
'playlist':
{
'albumArtRef': [
{
'url': 'http://lh3.googleusercontent.com/KJsAhrg8Jk_5A4xYLA68LFC...'
}],
'description': 'Workout Plan ',
'kind': 'sj#playlist',
'name': 'Workout',
'ownerName': 'Ida Sarver',
'shareToken': 'AMaBXyktyF6Yy_G-8wQy8Rru0tkueIbIFblt2h0BpkvTzHDz-fFj6P...',
'type': 'SHARED'
},
'type': '4'
}],
'situation_hits': [
{
'situation':
{
'description': 'Level up and enter beast mode with some loud, aggressive music.',
'id': 'Nrklpcyfewwrmodvtds5qlfp5ve',
'imageUrl': 'http://lh3.googleusercontent.com/Cd8WRMaG_pDwjTC_dSPIIuf...',
'title': 'Entering Beast Mode',
'wideImageUrl': 'http://lh3.googleusercontent.com/8A9S-nTb5pfJLcpS8P...'
},
'type': '7'
}],
'song_hits': [
{
'track':
{
'album': 'Work Out',
'albumArtRef': [
{
'aspectRatio': '1',
'autogen': False,
'kind': 'sj#imageRef',
'url': 'http://lh5.ggpht.com/DVIg4GiD6msHfgPs_Vu_2eRxCyAoz0fFdxj5w...'
}],
'albumArtist': 'J.Cole',
'albumAvailableForPurchase': True,
'albumId': 'Bfp2tuhynyqppnp6zennhmf6w3y',
'artist': 'J Cole',
'artistId': ['Ajgnxme45wcqqv44vykrleifpji', 'Ampniqsqcwxk7btbgh5ycujij5i'],
'composer': '',
'discNumber': 1,
'durationMillis': '234000',
'estimatedSize': '9368582',
'explicitType': '1',
'genre': 'Pop',
'kind': 'sj#track',
'nid': 'Tq3nsmzeumhilpegkimjcnbr6aq',
'primaryVideo':
{
'id': '6PN78PS_QsM',
'kind': 'sj#video',
'thumbnails': [
{
'height': 180,
'url': 'https://i.ytimg.com/vi/6PN78PS_QsM/mqdefault.jpg',
'width': 320
}]
},
'storeId': 'Tq3nsmzeumhilpegkimjcnbr6aq',
'title': 'Work Out',
'trackAvailableForPurchase': True,
'trackAvailableForSubscription': True,
'trackNumber': 1,
'trackType': '7',
'year': 2011
},
'type': '1'
}],
'station_hits': [
{
'station':
{
'compositeArtRefs': [
{
'aspectRatio': '1',
'kind': 'sj#imageRef',
'url': 'http://lh3.googleusercontent.com/3aD9mFppy6PwjADnjwv_w...'
}],
'contentTypes': ['1'],
'description': 'These riff-tastic metal tracks are perfect for getting the blood pumping.',
'imageUrls': [
{
'aspectRatio': '1',
'autogen': False,
'kind': 'sj#imageRef',
'url': 'http://lh5.ggpht.com/YNGkFdrtk43e8H941fuAHjflrNZ1CJUeqdoys...'
}],
'kind': 'sj#radioStation',
'name': 'Heavy Metal Workout',
'seed':
{
'curatedStationId': 'Lcwg73w3bd64hsrgarnorif52r',
'kind': 'sj#radioSeed',
'seedType': '9'
},
'skipEventHistory': [],
'stationSeeds': [
{
'curatedStationId': 'Lcwg73w3bd64hsrgarnorif52r',
'kind': 'sj#radioSeed',
'seedType': '9'
}]
},
'type': '6'
}],
'video_hits': [
{
'score': 629.6226806640625,
'type': '8',
'youtube_video':
{
'id': '6PN78PS_QsM',
'kind': 'sj#video',
'thumbnails': [
{
'height': 180,
'url': 'https://i.ytimg.com/vi/6PN78PS_QsM/mqdefault.jpg',
'width': 320
}],
'title': 'J. Cole - Work Out'
}
}]
}
Cleaned, but broken code with 3 weeks of different attempts: (I have tried for, foreach, while, but the furthest it would read would be either the entire unicode array, error, or an empty string)
sub search {
my $query = shift;
my $uri = 'googlemusic:search:' . $query;
if (my $result = $cache->get($uri)) {
return $result;
}
my $googleResult;
my $result = {
tracks => [],
albums => [],
artists => [],
};
eval {
$googleResult = $googleapi->search($query, $prefs->get('max_search_items'));
};
if ($#) {
$log->error("Not able to search All Access for \"$query\": $#");
return;
}
#gives not an ARRAY refernce error
for my $hit (#{$googleResult->{song_hits}}) {
push #{$result->{tracks}}, to_slim_track($hit->{track});
}
#works, but gives an error on the next line, 'newlist' object has no attribute 'album'
for my $hit ({$googleResult->{album_hits}}) {
push #{$result->{albums}}, album_to_slim_album($hit->{album});
}
#Perl and others recommended way, but gives Can't locate object method "FIRSTKEY" via package "Inline::Python::Object::Data"
for my $hit (%{$googleResult->{artist_hits}}) {
push #{$result->{artists}}, artist_to_slim_artist($hit->{artist});
}
# Add to the cache
$cache->set($uri, $result, $CACHE_TIME);
return $result;
}
I have tried reading up, but have gotten so many errors including:
'key' does not exist
Can't use string ("track") as a HASH ref while strict refs in use
Type of argument to keys on reference must be unblessed hashref or arrayref
My Full Test File: http://pastebin.com/DMnDc56i
GoogleApi PM (Python GAPI Hook): https://raw.githubusercontent.com/hechtus/squeezebox-googlemusic/master/GoogleMusic/GoogleAPI.pm
Edit: Info, There were a couple of people who wanted unmaintained old code fixed, so I offered to help and got everything working besides this part.
Old Code Git: https://github.com/hechtus/squeezebox-googlemusic
Google Api Python I use: https://github.com/simon-weber/gmusicapi
I take it that the data structure shown is in $googleResult. This is 'almost' JSON and you can process it as such using modules, after a simple cleanup. I will use JSON::XS. The code below takes off after $googleResult has been acquired. (In tests I actually copied data shown in the question into a file and read it in.) I first replace ' by " and lower-case True and False, to get a valid JSON format which the module can decode.
# Other code from the question ...
use JSON::XS;
# For tests I loaded shown data into $googleResult (did not run this eval)
eval {
$googleResult = $googleapi->search($query, $prefs->get('max_search_items'));
};
if ($#) {
$log->error("Not able to search All Access for \"$query\": $#");
return;
}
# The structure shown in the question needs a cleanup
# But this may be a road to madness, if there is more
$googleResult =~ s/'/"/g; # ' turn off wrong editor coloring
$googleResult =~ s/False/false/g;
$googleResult =~ s/True/true/g;
my $coder = JSON::XS->new;
# There are many options for how to set it up. Example:
# JSON::XS->new->ascii->pretty->allow_nonref;
my $data = $coder->decode($googleResult);
# Now this is a normal Perl data structure that we can work with.
# Look at what's under 'album_hits' for example
my $ralbhits = $data->{'album_hits'};
print Dumper($ralbhits);
# We get: VAR1 = [ { 'album' => { albumID => ... } } ]
# Array reference, with nested hash references as the sole element
# Extract the 'artist'
my $artist = $ralbhits->[0]->{'album'}->{'artist'};
print "$artist\n";
This prints J. Cole (after the dump which I omit here). You can for convenience first extract a part of the structure and then query it far more simply. For example
# Get the hashref for album
my $ralbum = $ralbhits->[0]->{'album'};
my $artist = $ralbum->{'artist'};
Now once the data is unpacked you can retrieve what you need, based on what artist_to_slim_artist() needs and does. This is a normal data structure to work with.
Modules for JSON parsing return Perl data structures, see Mapping in JSON::XS. Generally they will be nested, except in very simple cases. For how to work with them see perldsc, a cookbook on complex data structures.
The JSON object given in this example, while invalid, needed very little correction. However, it may get far more complicated. For example, there is a far larger document (~100kB) linked to in a comment, with these problems.
Name-value pairs are enclosed in ' instead of " and the values themselves contain ' (like isn't and other contractions), complicating the matching of ' pairs.
Invalid u' sequences at the beginning of names and values (u need be removed). This can be rolled together with the above, as they come together. There was also u".
Text may contain all kinds of escapes, for example some encoding of accents, which are not valid JSON. (One in that document.) This can be found and fixed (escaped for example).
It took a few minutes to come up with a few regex that corrected the document at the link, at close to 100kB in size, so that it parses cleanly with the above code. But the problem is that it is hard to tell what other trouble may be in the next document. Still, since this may be of interest here is the regex.
Instead of being enclosed in a pair of ", the names and values are in between ', and the leading one also has an extra char, u'. What makes it easier is that the closing ' must be followed by either of , : ] } and I use positive lookahead to assert that. Finally, there are some u" opening quotes and u is removed, first.
$googleResult =~ s/False/false/g;
$googleResult =~ s/True/true/g;
$googleResult =~ s/u"/"/g;
# There are also escaped characters in text, escape that backslash
$googleResult =~ s|(\\)|$1$1|g;
# Correct delimiters from u'...' to "...", see text below
$googleResult =~ s/u'(.*?)' (?= []:},] )/"$1"/gx;
# We are good now, decode it
my $data = $coder->decode($googleResult);
my $alb = $data->[0]{track}{album};
print "$alb\n";
This prints These Things Happen (correctly). Above we capture between u' and the first ' that is followed by either of ]:,} (for which a character class [...] is used). Then u'' is replaced by "". After this decode($googleResult); works and we get the Perl data structure to query.
There are various modules that allow a 'relaxed' approach and will accept many such irregularities. However, by using them we agree to use an invalid JSON, which is meant to be a simple and clear data format, and I wouldn't advise to go down that road. Note that the nearly full specification of the format fits nicely in one clear and genereously illustrated page at the above link. Also see JSON Example, for a handful of examples.
I think that the best bet is to try to clean it up. Run a decoder like in the code above and see the error message. It will pinpoint the problem exactly. Then add a regex to correct that particular violation of format. Then go again. If the various documents you may work with carry more or less the same set of problems (like the ones above for example) it may well work. Or it may turn out that it is too much trouble, if new violations keep coming up, in which case you may need a different approach.
Finally, I don't know how you arrived at this format from the original Python-object problem. Could it be that the format got broken somewhere in translation? I don't see how that would be the case. Is it actually not meant to be JSON? However, it is too close to it for that.
Is it possible to ask for valid JSON to be provided?
OK, this isn't really an answer, but out of the goodness of my heart I cleaned up your data for you. Here is a real Python dict. I don't know if some of the numerical string values should be ints or not, so I didn't mess with them. It'll be up to you to figure out what to do with the truncated URLs.
Another way to go about this would be to change True to true, False to false, and parse the dict as JSON.
{
'album_hits': [
{
'album':
{
'albumArtRef': 'http://lh5.ggpht.com/DVIg4GiD6msHfgPs_Vu_2eRxCyAoz0fF...',
'albumArtist': 'J.Cole',
'albumId': 'Bfp2tuhynyqppnp6zennhmf6w3y',
'artist': 'J.Cole',
'artistId': ['Ajgnxme45wcqqv44vykrleifpji'],
'description_attribution':
{
'kind': 'sj#attribution',
'license_title': 'Creative Commons Attribution CC-BY',
'license_url': 'http://creativecommons.org/licenses/by/4.0/legalcode',
'source_title': 'Freebase',
'source_url': ''
},
'explicitType': '1',
'kind': 'sj#album',
'name': 'Work Out',
'year': 2011
},
'type': '3'
}],
'artist_hits': [
{
'artist':
{
'artistArtRef': 'http://lh3.googleusercontent.com/MJe-cDw9uQ-pUagoLlm...',
'artistArtRefs': [
{
'aspectRatio': '2',
'autogen': False,
'kind': 'sj#imageRef',
'url': 'http://lh3.googleusercontent.com/MJe-cDw9uQ-pUagoLlmKX3x_K...'
}],
'artistId': 'Ajgnxme45wcqqv44vykrleifpji',
'artist_bio_attribution':
{
'kind': 'sj#attribution',
'source_title': 'David Jeffries, Rovi'
},
'kind': 'sj#artist',
'name': 'J. Cole'
},
'type': '2'
}],
'playlist_hits': [
{
'playlist':
{
'albumArtRef': [
{
'url': 'http://lh3.googleusercontent.com/KJsAhrg8Jk_5A4xYLA68LFC...'
}],
'description': 'Workout Plan ',
'kind': 'sj#playlist',
'name': 'Workout',
'ownerName': 'Ida Sarver',
'shareToken': 'AMaBXyktyF6Yy_G-8wQy8Rru0tkueIbIFblt2h0BpkvTzHDz-fFj6P...',
'type': 'SHARED'
},
'type': '4'
}],
'situation_hits': [
{
'situation':
{
'description': 'Level up and enter beast mode with some loud, aggressive music.',
'id': 'Nrklpcyfewwrmodvtds5qlfp5ve',
'imageUrl': 'http://lh3.googleusercontent.com/Cd8WRMaG_pDwjTC_dSPIIuf...',
'title': 'Entering Beast Mode',
'wideImageUrl': 'http://lh3.googleusercontent.com/8A9S-nTb5pfJLcpS8P...'
},
'type': '7'
}],
'song_hits': [
{
'track':
{
'album': 'Work Out',
'albumArtRef': [
{
'aspectRatio': '1',
'autogen': False,
'kind': 'sj#imageRef',
'url': 'http://lh5.ggpht.com/DVIg4GiD6msHfgPs_Vu_2eRxCyAoz0fFdxj5w...'
}],
'albumArtist': 'J.Cole',
'albumAvailableForPurchase': True,
'albumId': 'Bfp2tuhynyqppnp6zennhmf6w3y',
'artist': 'J Cole',
'artistId': ['Ajgnxme45wcqqv44vykrleifpji', 'Ampniqsqcwxk7btbgh5ycujij5i'],
'composer': '',
'discNumber': 1,
'durationMillis': '234000',
'estimatedSize': '9368582',
'explicitType': '1',
'genre': 'Pop',
'kind': 'sj#track',
'nid': 'Tq3nsmzeumhilpegkimjcnbr6aq',
'primaryVideo':
{
'id': '6PN78PS_QsM',
'kind': 'sj#video',
'thumbnails': [
{
'height': 180,
'url': 'https://i.ytimg.com/vi/6PN78PS_QsM/mqdefault.jpg',
'width': 320
}]
},
'storeId': 'Tq3nsmzeumhilpegkimjcnbr6aq',
'title': 'Work Out',
'trackAvailableForPurchase': True,
'trackAvailableForSubscription': True,
'trackNumber': 1,
'trackType': '7',
'year': 2011
},
'type': '1'
}],
'station_hits': [
{
'station':
{
'compositeArtRefs': [
{
'aspectRatio': '1',
'kind': 'sj#imageRef',
'url': 'http://lh3.googleusercontent.com/3aD9mFppy6PwjADnjwv_w...'
}],
'contentTypes': ['1'],
'description': 'These riff-tastic metal tracks are perfect for getting the blood pumping.',
'imageUrls': [
{
'aspectRatio': '1',
'autogen': False,
'kind': 'sj#imageRef',
'url': 'http://lh5.ggpht.com/YNGkFdrtk43e8H941fuAHjflrNZ1CJUeqdoys...'
}],
'kind': 'sj#radioStation',
'name': 'Heavy Metal Workout',
'seed':
{
'curatedStationId': 'Lcwg73w3bd64hsrgarnorif52r',
'kind': 'sj#radioSeed',
'seedType': '9'
},
'skipEventHistory': [],
'stationSeeds': [
{
'curatedStationId': 'Lcwg73w3bd64hsrgarnorif52r',
'kind': 'sj#radioSeed',
'seedType': '9'
}]
},
'type': '6'
}],
'video_hits': [
{
'score': 629.6226806640625,
'type': '8',
'youtube_video':
{
'id': '6PN78PS_QsM',
'kind': 'sj#video',
'thumbnails': [
{
'height': 180,
'url': 'https://i.ytimg.com/vi/6PN78PS_QsM/mqdefault.jpg',
'width': 320
}],
'title': 'J. Cole - Work Out'
}
}]
}
I've worked out a solution using a list comprehension like so:
use Inline::Python qw(py_eval);
my $song_hits = py_eval("[x for x in $googleResult->{song_hits}]", 0);
for my $hit (#$song_hits) {
push #{$result->{tracks}}, to_slim_track($hit->{track});
}
The commit is at:
https://github.com/squeezebox-googlemusic/squeezebox-googlemusic/commit/e6fa62d9da3bc7295023283ef5d25698737e5772

Categories

Resources