Related
hi I'm pretty new at coding and I was trying to create a program in python that reads and save in another file the data inside a json file (not everything, just what I want). I googled how to parse data but there's something I don't understand.
that's a part of the json file:
`
{
"profileRevision": 548789,
"profileId": "campaign",
"profileChangesBaseRevision": 548789,
"profileChanges": [
{
"changeType": "fullProfileUpdate",
"profile": {
"_id": "2da4f079f8984cc48e84fc99dace495d",
"created": "2018-03-29T11:02:15.190Z",
"updated": "2022-10-31T17:34:43.284Z",
"rvn": 548789,
"wipeNumber": 9,
"accountId": "63881e614ef543b2932c70fed1196f34",
"profileId": "campaign",
"version": "refund_teddy_perks_september_2022",
"items": {
"8ec8f13f-6bf6-4933-a7db-43767a055e66": {
"templateId": "Quest:heroquest_loadout_constructor_2",
"attributes": {
"quest_state": "Claimed",
"creation_time": "min",
"last_state_change_time": "2019-05-18T16:09:12.750Z",
"completion_complete_pve03_diff26_loadout_constructor": 300,
"level": -1,
"item_seen": true,
"sent_new_notification": true,
"quest_rarity": "uncommon",
"xp_reward_scalar": 1
},
"quantity": 1
},
"6940c71b-c74b-4581-9f1e-c0a87e246884": {
"templateId": "Worker:workerbasic_sr_t01",
"attributes": {
"gender": "2",
"personality": "Homebase.Worker.Personality.IsDreamer",
"level": 1,
"item_seen": true,
"squad_slot_idx": -1,
"portrait": "WorkerPortrait:IconDef-WorkerPortrait-Dreamer-F02",
"building_slot_used": -1,
"set_bonus": "Homebase.Worker.SetBonus.IsMeleeDamageLow"
}
}
}
]
}
`
I can access profileChanges. I wrote this to create another json file with only the profileChanges things:
`
myjsonfile= open("file.json",'r')
jsondata=myjsonfile.read()
obj=json.loads(jsondata)
ciso=obj['profileChanges']
for i in ciso:
print(i)
with open("file2", "w") as outfile:
json.dump( ciso, outfile, indent=1)
the issue I have is that I can't access "profile" (inside profileChanges) in the same way by parsing the new file and I have no idea on how to do it
Access to JSON or dict element is realized by list indexes, please look at below example:
a = [
{
"friends": [
{
"id": 0,
"name": "Reba May"
}
],
"greeting": "Hello, Doris Gallagher! You have 2 unread messages.",
"favoriteFruit": "strawberry"
},
]
b = a['friends']['id] # b = 0
I've added a couple of closing braces to make your snippet valid json:
s = '''{
"profileRevision": 548789,
"profileId": "campaign",
"profileChangesBaseRevision": 548789,
"profileChanges": [
{
"changeType": "fullProfileUpdate",
"profile": {
"_id": "2da4f079f8984cc48e84fc99dace495d",
"created": "2018-03-29T11:02:15.190Z",
"updated": "2022-10-31T17:34:43.284Z",
"rvn": 548789,
"wipeNumber": 9,
"accountId": "63881e614ef543b2932c70fed1196f34",
"profileId": "campaign",
"version": "refund_teddy_perks_september_2022",
"items": {
"8ec8f13f-6bf6-4933-a7db-43767a055e66": {
"templateId": "Quest:heroquest_loadout_constructor_2",
"attributes": {
"quest_state": "Claimed",
"creation_time": "min",
"last_state_change_time": "2019-05-18T16:09:12.750Z",
"completion_complete_pve03_diff26_loadout_constructor": 300,
"level": -1,
"item_seen": true,
"sent_new_notification": true,
"quest_rarity": "uncommon",
"xp_reward_scalar": 1
},
"quantity": 1
},
"6940c71b-c74b-4581-9f1e-c0a87e246884": {
"templateId": "Worker:workerbasic_sr_t01",
"attributes": {
"gender": "2",
"personality": "Homebase.Worker.Personality.IsDreamer",
"level": 1,
"item_seen": true,
"squad_slot_idx": -1,
"portrait": "WorkerPortrait:IconDef-WorkerPortrait-Dreamer-F02",
"building_slot_used": -1,
"set_bonus": "Homebase.Worker.SetBonus.IsMeleeDamageLow"
}
}
}
}
}
]
}
'''
d = json.loads(s)
print(d['profileChanges'][0]['profile']['version'])
This prints refund_teddy_perks_september_2022
Explanation:
d is a dict
d['profileChanges'] is a list of dicts
d['profileChanges'][0] is the first dict in the list
d['profileChanges'][0]['profile'] is a dict
d['profileChanges'][0]['profile']['version'] is the value of version key in the profile dict in the first entry of the profileChanges list.
I have 2 separate JSON-Lists (with dicts) in it.
My goal is, that I want to iterate over list2 "currentUser", grab the values, search those values in list1, and as output print the value of "firstName"
e.g.
liste2: "currentUser": 123,
liste1: "id": "123", --> "firstName": "Lisa",
list1 = {
"X-API-KEY": "XyZzahZaksksXXXYYYOOO000",
"user": {
"email": "Lisa#BLA.com",
"firstName": "Lisa",
"id": "123",
},
"Flat": {
"city": "Munich",
"country": "2",
"countryCode": "DEU",
"currency": "EUR",
"date": "1587671397",
"flatmates": [
{
"email": "Lisa#BLA.com",
"firstName": "Lisa",
"id": "123",
},
{
"email": "Max#BLA.com",
"firstName": "Max",
"id": "124",
},
{
"email": "Hannah#BLA.com",
"firstName": "Hannah",
"id": "125",
},
{
"email": "Kai#BLA.com",
"firstName": "Kai",
"id": "126",
}
],
"founderId": "123",
"id": "99999",
"image": "",
"name": "ABC",
"postCode": "000000",
}
}
list2 = [
{
"creationDate": 1587671663,
"currentUser": 123,
"id": 1717134,
"title": "Do this",
"users": [
124,
126
]
},
{
"creationDate": 1587671663,
"currentUser": 126,
"id": 1717134,
"title": "Do that",
"users": [
123,
125
]
},
{
"creationDate": 1587671821,
"currentUser": 124,
"id": 1717134,
"title": "Clean this",
"users": [
125,
122
]
},
{
"creationDate": 1587671801,
"currentUser": 123,
"id": 1717134,
"title": "Clean that",
"users": [
124,
126
]
}
]
I am pretty new to python.
There are several mind-issues for me since there is a mix between lists and dictionaries in it and how to match/search for values for 2 separate lists/dicts
What I got so far: Iterate over the "CurrentUser"
for user in liste2:
print(user["currentUser"])
Has anyone some approaches?
In pure python with no other modules.
for user in list2:
for mate in list1['Flat']['flatmates']:
if user['currentUser'] == int(mate['id']):
# You found the person now execute this code...
One thing to note that in your list1 you flatmates id is not a integer but a string. So you have to convert that to an int in order to compare the two.
Assuming i have this dictionary inside a list:
[
{
"Name": "Person",
"Confidence": 97.56156921386719,
"Instances": [
{
"BoundingBox": {
"Width": 0.6137702465057373,
"Height": 0.9175498485565186,
"Left": 0.22297996282577515,
"Top": 0.0739903450012207
},
"Confidence": 94.51961517333984
},
{
"BoundingBox": {
"Width": 0.46570318937301636,
"Height": 0.6405649781227112,
"Left": 0.11447866261005402,
"Top": 0.34079012274742126
},
"Confidence": 82.2153549194336
}
],
"Parents": []
},
{
"Name": "Human",
"Confidence": 97.56156921386719,
"Instances": [],
"Parents": []
},
{
"Name": "Clothing",
"Confidence": 91.08417510986328,
"Instances": [],
"Parents": []
}
}]
How can I retrieve the dictionary with key value "Name":"Clothing" only? is it possible to retrieve it from the dictionary? The Key "Clothing" may appear in any part of the list.
{
"Name": "Clothing",
"Confidence": 91.08417510986328,
"Instances": [],
"Parents": []
}
You can use the following code to get every item that has Name of Clothing:
names = [
{
"Name": "Person",
"Confidence": 97.56156921386719,
"Instances": [
{
"BoundingBox": {
"Width": 0.6137702465057373,
"Height": 0.9175498485565186,
"Left": 0.22297996282577515,
"Top": 0.0739903450012207
},
"Confidence": 94.51961517333984
},
{
"BoundingBox": {
"Width": 0.46570318937301636,
"Height": 0.6405649781227112,
"Left": 0.11447866261005402,
"Top": 0.34079012274742126
},
"Confidence": 82.2153549194336
}
],
"Parents": []
},
{
"Name": "Human",
"Confidence": 97.56156921386719,
"Instances": [],
"Parents": []
},
{
"Name": "Clothing",
"Confidence": 91.08417510986328,
"Instances": [],
"Parents": []
}
]
for name in names:
if name['Name'] == 'Clothing':
print(name)
If you want it to be short you can use the following code:
x = [y for y in names if name['Name'] == 'Clothing']
The .get(Key, defaultValue) can be used here.
.get("Name", "") --> Returns an empty string if the key Name doesn't exist in the dictionary. doc
.get("Name") --> Returns None if Name doesn't exist.
So .get("Name") == "Clothing" is the criteria for selection of a dictionary.
And use filter() to choose only those that satisfy the criteria.
Putting it all together
filter(lambda x: x.get("Name") == "Clothing", mylist)
filter returns an iterator, so you could pass that to list to create a full list.
If you just want to iterate over it, no need to call list.
So,
list(filter(lambda x: x.get("Name") == "Clothing", mylist))
You can use this simple list comprehension:
[print(dictionary) for dictionary in lst if 'Name' in dictionary.keys() and dictionary['Name'] == 'Clothing']
Output:
{'Name': 'Clothing', 'Confidence': 91.08417510986328, 'Instances': [], 'Parents': []}
Here is the full code:
lst = [
{
"Name": "Person",
"Confidence": 97.56156921386719,
"Instances": [
{
"BoundingBox": {
"Width": 0.6137702465057373,
"Height": 0.9175498485565186,
"Left": 0.22297996282577515,
"Top": 0.0739903450012207
},
"Confidence": 94.51961517333984
},
{
"BoundingBox": {
"Width": 0.46570318937301636,
"Height": 0.6405649781227112,
"Left": 0.11447866261005402,
"Top": 0.34079012274742126
},
"Confidence": 82.2153549194336
}
],
"Parents": []
},
{
"Name": "Human",
"Confidence": 97.56156921386719,
"Instances": [],
"Parents": []
},
{
"Name": "Clothing",
"Confidence": 91.08417510986328,
"Instances": [],
"Parents": []
}
]
[print(dictionary) for dictionary in lst if 'Name' in dictionary.keys() and dictionary['Name'] == 'Clothing']
Assuming your list is called my_list, then this should give you what you want:
my_list_new = [x for x in my_list if ("Name", "Clothing") in x.items()]
I am writing some code with Python and Spotipy and I'm relatively new to coding. I have some code that get all the info about a Spotify playlist and prints it out for me:
from spotipy.oauth2 import SpotifyClientCredentials
import spotipy
import json
client_credentials_manager = SpotifyClientCredentials()
sp = spotipy.Spotify(client_credentials_manager=client_credentials_manager)
playlist_id = 'spotify:playlist:76CVeJDw2b90up5PgkZXyU'
results = sp.playlist(playlist_id)
#print(json.dumps(results, indent=4))
print((json.dumps(results, indent=4)))
It works well and gives me all the info. My problem is that I only need specifics from the print:
"collaborative": false,
"description": "",
"external_urls": {
"spotify": "https://open.spotify.com/playlist/76CVeJDw2b90up5PgkZXyU"
},
"followers": {
"href": null,
"total": 0
},
"href": "https://api.spotify.com/v1/playlists/76CVeJDw2b90up5PgkZXyU?additional_types=track",
"id": "76CVeJDw2b90up5PgkZXyU",
"images": [
{
"height": 640,
"url": "https://i.scdn.co/image/ab67616d0000b2734a052b99c042dc15f933145b",
"width": 640
}
],
"name": "TEST",
"owner": {
"display_name": "Name",
"external_urls": {
"spotify": "https://open.spotify.com/user/myname"
},
"href": "https://api.spotify.com/v1/users/myname",
"id": "Myname",
"type": "user",
"uri": "spotify:user:kovizsombor"
},
"primary_color": null,
"public": true,
"snapshot_id": "MixmMGE0MDgxNDQ1ZGVlNmE4MThiMmQwODMwYWU0OTI3YzkyOGJhOWIz",
"tracks": {
"href": "https://api.spotify.com/v1/playlists/76CVeJDw2b90up5PgkZXyU/tracks?offset=0&limit=100&additional_types=track",
"items": [
{
"added_at": "2020-05-17T10:00:11Z",
"added_by": {
"external_urls": {
"spotify": "https://open.spotify.com/user/kovizsombor"
},
"href": "https://api.spotify.com/v1/users/kovizsombor",
"id": "kovizsombor",
"type": "user",
"uri": "spotify:user:kovizsombor"
},
"is_local": false,
"primary_color": null,
"track": {
"album": {
"album_type": "album",
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/0PFtn5NtBbbUNbU9EAmIWF"
},
"href": "https://api.spotify.com/v1/artists/0PFtn5NtBbbUNbU9EAmIWF",
"id": "0PFtn5NtBbbUNbU9EAmIWF",
"name": "TOTO",
"type": "artist",
"uri": "spotify:artist:0PFtn5NtBbbUNbU9EAmIWF"
}
],
],
"external_urls": {
"spotify": "https://open.spotify.com/album/62U7xIHcID94o20Of5ea4D"
},
"href": "https://api.spotify.com/v1/albums/62U7xIHcID94o20Of5ea4D",
"id": "62U7xIHcID94o20Of5ea4D",
"images": [
{
"height": 640,
"url": "https://i.scdn.co/image/ab67616d0000b2734a052b99c042dc15f933145b",
"width": 640
},
{
"height": 300,
"url": "https://i.scdn.co/image/ab67616d00001e024a052b99c042dc15f933145b",
"width": 300
},
{
"height": 64,
"url": "https://i.scdn.co/image/ab67616d000048514a052b99c042dc15f933145b",
"width": 64
}
],
"name": "Toto IV",
"release_date": "1982-04-08",
"release_date_precision": "day",
"total_tracks": 10,
"type": "album",
"uri": "spotify:album:62U7xIHcID94o20Of5ea4D"
},
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/0PFtn5NtBbbUNbU9EAmIWF"
},
"href": "https://api.spotify.com/v1/artists/0PFtn5NtBbbUNbU9EAmIWF",
"id": "0PFtn5NtBbbUNbU9EAmIWF",
"name": "TOTO",
"type": "artist",
"uri": "spotify:artist:0PFtn5NtBbbUNbU9EAmIWF"
}
],
"available_markets": [
],
"disc_number": 1,
"duration_ms": 295893,
"episode": false,
"explicit": false,
"external_ids": {
"isrc": "USSM19801941"
},
"external_urls": {
"spotify": "https://open.spotify.com/track/2374M0fQpWi3dLnB54qaLX"
},
"href": "https://api.spotify.com/v1/tracks/2374M0fQpWi3dLnB54qaLX",
"id": "2374M0fQpWi3dLnB54qaLX",
"is_local": false,
"name": "Africa",
"popularity": 83,
"preview_url": "https://p.scdn.co/mp3-preview/dd78dafe31bb98f230372c038a126b8808f9349b?cid=d568e7073a38465bba48268ea9f10153",
"track": true,
"track_number": 10,
"type": "track",
"uri": "spotify:track:2374M0fQpWi3dLnB54qaLX"
},
"video_thumbnail": {
"url": null
}
}
],
"limit": 100,
"next": null,
"offset": 0,
"previous": null,
"total": 1
},
"type": "playlist",
"uri": "spotify:playlist:76CVeJDw2b90up5PgkZXyU"
}
From this long print I somehow need to extract the Artist and the song title and preferably make it into a variable. Also not sure how this would work if there are multiple songs in a playlist.
It's also a solution if I can print out only the Artist and the title of the song without printing out all the information.
Based on your posted example it appears that you have only 1 song named "Africa" by artist "TOTO". Copying the track to have two songs, I added another track with two artists for testing the arrays.
If that json is loaded into a variable named results then (as #xcmkz said) you have a python dictionary and can process accordingly.
Try working with the following to transverse through your dict appending artists and songs to lists:
song_dict = {}
for track in results['tracks']['items']:
song_name = track["track"]["name"]
a2 = []
for t1 in track['track']['artists']:
a2.append(t1['name'])
song_dict.update({song_name: a2})
print(f'Dictionary of Songs and Artists:')
for k, v in song_dict.items():
print(f'Song --> {k}, by --> {", ".join(v)}')
Results:
Dictionary of Songs and Artists:
Song --> Africa, by --> TOTO
Song --> Just Another Silly Song, by --> Artist 2, Artist 3
sp.playlist returns a dictionary, so you can simply access its values by their keys. For example:
>>> results['name']
'TEST'
JSON is a data serialization format, ie a standardized way of representing objects as pure text and parsing them back from the text. json.dumps therefore converts the dictionary object to a string of text. This is useful if you want to for example save the results to a file and load it back later. You don't need it to access contents from results.
(This is a playlist though—you will need to get information on each song/track to get its artist and name.)
A bit new to avro & python.
I am trying to do a simple conversion to avro using the fastavro library, as the speed of the native apache avro library is just a bit too slow.
I want to:
1.Take a json file
2. Convert the data to avro.
My problem is that it seems like my json isn't in the correct 'record' format to be converted to avro. I even tried putting my json into a string var and making it looks similar to the syntax they have on the site # https://fastavro.readthedocs.io/en/latest/writer.html:
{u'station': u'011990-99999', u'temp': 22, u'time': 1433270389},
{u'station': u'011990-99999', u'temp': -11, u'time': 1433273379},
{u'station': u'012650-99999', u'temp': 111, u'time': 1433275478},
Here is my code:
from fastavro import json_writer, parse_schema, writer
import json
key = "test.json"
schemaFileName = "test_schema.avsc"
with open(r'C:/Path/to/file' + schemaFileName) as sc:
w = json.load(sc)
schema = parse_schema(w)
with open(r'C:/Path/to/file/' + key) as js:
x=json.load(js)
with open('C:/Path/to/file/output.avro', 'wb') as out:
writer(out, schema,x, codec='deflate')
Here is what I get as output:
File "avropython.py", line 26, in <module>
writer(out, schema,x, codec='deflate')
File "fastavro\_write.pyx", line 608, in fastavro._write.writer
ValueError: "records" argument should be an iterable, not dict
My json file and schema, rspectively:
"joined": false,
"toward": {
"selection": "dress",
"near": true,
"shoulder": false,
"fine": -109780201.3804388,
"pet": {
"stood": "saddle",
"live": false,
"leather": false,
"tube": false,
"over": false,
"impossible": true
},
"higher": false
},
"wear": true,
"asleep": "door",
"connected": true,
"stairs": -1195512399.5000324
}
{
"name": "MyClass",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "joined",
"type": "boolean"
},
{
"name": "toward",
"type": {
"name": "toward",
"type": "record",
"fields": [
{
"name": "selection",
"type": "string"
},
{
"name": "near",
"type": "boolean"
},
{
"name": "shoulder",
"type": "boolean"
},
{
"name": "fine",
"type": "float"
},
{
"name": "pet",
"type": {
"name": "pet",
"type": "record",
"fields": [
{
"name": "stood",
"type": "string"
},
{
"name": "live",
"type": "boolean"
},
{
"name": "leather",
"type": "boolean"
},
{
"name": "tube",
"type": "boolean"
},
{
"name": "over",
"type": "boolean"
},
{
"name": "impossible",
"type": "boolean"
}
]
}
},
{
"name": "higher",
"type": "boolean"
}
]
}
},
{
"name": "wear",
"type": "boolean"
},
{
"name": "asleep",
"type": "string"
},
{
"name": "connected",
"type": "boolean"
},
{
"name": "stairs",
"type": "float"
}
]
}
If anyone could help me out, it would be greatly appreciated!!
As mentioned in the error ValueError: "records" argument should be an iterable, not dict, the problem is that when you call writer, the argument for the records needs to be an iterable. One way to solve this is to change your last line to writer(out, schema, [x], codec='deflate')
Alternatively, there is a schemaless_writer that can be used to just write a single record: https://fastavro.readthedocs.io/en/latest/writer.html#fastavro._write_py.schemaless_writer