I've got this discord.py command that makes a leaderboard from a json
cogs/coins.json (the dictionary) looks like this:
{
"781524858026590218": {
"name": "kvbot test platform",
"total_coins": 129,
"data": {
"564050979079585803": {
"name": "Bluesheep33",
"coins": 127
},
"528647474596937733": {
"name": "ACAT_",
"coins": 2
}
}
(The green strings with numbers in the json files are discord guild/member ids)
How do I make the code shorter and clearer?
Thanks for helping in advance, because I really don't know the solution
When it comes to finding (sorting) the first ten items within a dict, then the way is much easier than repeatedly going through the dict and doing different things there.
And little better code, like Dict.get for safety access.
Based on a sample of JSON data.
with open('cogs/coins.json', 'r') as f:
coins_data = json.load(f)
# Get is safefy access to dict
# Dict.items() returns pairs of (Key, Val)
members_coins = list(coins_data.get(str(ctx.guild.id), None)['data'].items())
if members_coins is None: # If data not found
await ctx.send('Not data')
return
# Sort list by Val part of pair, and `coins` key, reverse for descending
members_coins.sort(key=lambda x: x[1]['coins'], reverse=True)
output = ''
# list[:10] for first 10 items (if list is smaller, thats okay, python don't mind)
for member_id, vals in members_coins[:10]:
output += f'{vals["name"]}: {vals["coins"]}'
# output += f'<#{member_id}>: {vals["coins"]}' # If you want "mention" display of user
await ctx.send(output)
Related
I want to create a dictionary with Key value pairs which are filled via an for Loop
The dictionary I want to achive
[
{
"timestamp": 123123123,
"image": "image/test1.png"
},
{
"timestamp": 0384030434,
"image": "image/test2.png"
}
]
My code does not work and I´m new to the datatypes of python.
images_dict = []
for i in images:
time = Image.open(i)._getexif()[36867]
images_dict = {"timestamp": time, "image": i}
What am I missing?
First, you seem to be confusing the definition of a list and a dictionary in python. Dictionaries use curly brackets {} and lists use regular brackets []. So in your first example, you are describing a list with a single element, which is a dictionary.
As for your code example, you are creating an empty list, and then iterating over images which I assume is a list of images, and then redefining the variable images_dict to be a dictionary with two key: value pairs for every iteration.
It seems like what you want is this:
images_dict = []
for image in images:
time = Image.open(1)._getexif()[36867]
images_dict.append({'timestamp': time, 'image': image})
The answer from Tom McLean worked for me, I´m a little bit confused with the dataypes of python
images_dict.append({"timestamp": time, "image": i})
I have a JSON file that looks like this:
{
"returnCode": 200,
"message": "OK",
“people”: [
{
“details: {
"first": “joe”,
“last”: doe,
“id”: 1234567,
},
“otheDetails”: {
“employeeNum”: “0000111222”,
“res”: “USA”,
“address”: “123 main street”,
},
“moreDetails”: {
“family”: “yes”,
“siblings”: “no”,
“home”: “USA”,
},
},
{
“details: {
"first": “jane”,
“last”: doe,
“id”: 987654321,
},
“otheDetails”: {
“employeeNum”: “222333444”,
“res”: “UK”,
“address”: “321 nottingham dr”,
},
“moreDetails”: {
“family”: “yes”,
“siblings”: “yes”,
“home”: “UK,
},
}
This shows two entries, but really there are hundreds or more. I do not know the number of entries at the time the code is run.
My goal is to iterate through each entry and get the 'id' under "details". I load the JSON into a python dict named 'data' and am able to get the first 'id' by:
data['people'][0]['details']['id']
I can then get the second 'id' by incrementing the '0' to '1'. I know I can set i = 0 and then increment i, but since I do not know the number of entries, this does not work. Is there a better way?
Less pythonic then a list comprehension, but a simple for loop will work here.
You can first calculate the number of people in the people list and then loop over the list, pulling out each id at each iteration:
id_list = []
for i in range(len(data['people'])):
id_list.append(data['people'][i]['details']['id'])
You can use dict.get method in a list comprehension to avoid getting a KeyError on id. This way, you can fill dictionaries without ids with None:
ids = [dct['details'].get('id') for dct in data['people']]
If you still get KeyError, then that probably means some dcts in data['people'] don't have details key. In that case, it might be better to wrap this exercise in try/except. You may also want to identify which dcts don't have details key, which can be gathered using error_dct list (which you can uncomment out from below).
ids = []
#error_dct = []
for dct in data['people']:
try:
ids.append(dct['details']['id'])
except KeyError:
ids.append(None)
#error_dct.append(dct)
Output:
1234567
987654321
I'm using this as a reference: Elegant way to remove fields from nested dictionaries
I have a large number of JSON-formatted data here and we've determined a list of unnecessary keys (and all their underlying values) that we can remove.
I'm a bit new to working with JSON and Python specifically (mostly did sysadmin work) and initially thought it was just a plain dictionary of dictionaries. While some of the data looks like that, several more pieces of data consists of dictionaries of lists, which can furthermore contain more lists or dictionaries with no specific pattern.
The idea is to keep the data identical EXCEPT for the specified keys and associated values.
Test Data:
to_be_removed = ['leecher_here']
easy_modo =
{
'hello_wold':'konnichiwa sekai',
'leeching_forbidden':'wanpan kinshi',
'leecher_here':'nushiyowa'
}
lunatic_modo =
{
'hello_wold':
{'
leecher_here':'nushiyowa','goodbye_world':'aokigahara'
},
'leeching_forbidden':'wanpan kinshi',
'leecher_here':'nushiyowa',
'something_inside':
{
'hello_wold':'konnichiwa sekai',
'leeching_forbidden':'wanpan kinshi',
'leecher_here':'nushiyowa'
},
'list_o_dicts':
[
{
'hello_wold':'konnichiwa sekai',
'leeching_forbidden':'wanpan kinshi',
'leecher_here':'nushiyowa'
}
]
}
Obviously, the original question posted there isn't accounting for lists.
My code, modified appropriately to work with my requirements.
from copy import deepcopy
def remove_key(json,trash):
"""
<snip>
"""
keys_set = set(trash)
modified_dict = {}
if isinstance(json,dict):
for key, value in json.items():
if key not in keys_set:
if isinstance(value, dict):
modified_dict[key] = remove_key(value, keys_set)
elif isinstance(value,list):
for ele in value:
modified_dict[key] = remove_key(ele,trash)
else:
modified_dict[key] = deepcopy(value)
return modified_dict
I'm sure something's messing with the structure since it doesn't pass the test I wrote since the expected data is exactly the same, minus the removed keys. The test shows that, yes it's properly removing the data but for the parts where it's supposed to be a list of dictionaries, it's only getting returned as a dictionary instead which will have unfortunate implications down the line.
I'm sure it's because the function returns a dictionary but I don't know to proceed from here in order to maintain the structure.
At this point, I'm needing help on what I could have overlooked.
When you go through your json file, you only need to determine whether it is a list, a dict or neither. Here is a recursive way to modify your input dict in place:
def remove_key(d, trash=None):
if not trash: trash = []
if isinstance(d,dict):
keys = [k for k in d]
for key in keys:
if any(key==s for s in trash):
del d[key]
for value in d.values():
remove_key(value, trash)
elif isinstance(d,list):
for value in d:
remove_key(value, trash)
remove_key(lunatic_modo,to_be_removed)
remove_key(easy_modo,to_be_removed)
Result:
{
"hello_wold": {
"goodbye_world": "aokigahara"
},
"leeching_forbidden": "wanpan kinshi",
"something_inside": {
"hello_wold": "konnichiwa sekai",
"leeching_forbidden": "wanpan kinshi"
},
"list_o_dicts": [
{
"hello_wold": "konnichiwa sekai",
"leeching_forbidden": "wanpan kinshi"
}
]
}
{
"hello_wold": "konnichiwa sekai",
"leeching_forbidden": "wanpan kinshi"
}
First, here is a sample JSON feed that I want to read in Python 2.7 with either simplejson or the built in JSON decoder. I am loading the .json file in Python and then searching for a key like "Apple" or "Orange" and when that key is found, I want to bring in the information for it like the types and quantities.
Right now there is only 3 items, but I want to be able to search one that may have up to 1000 items. Here is the code:
{
"fruits": [
{
"Apple": [
{
"type": "Gala",
"quant": 5
},
{
"type": "Honeycrisp",
"quant": 10
},
{
"type": "Red Delicious",
"quant": 4
}
]
},
{
"Banana": [
{
"type": "Plantain",
"quant": 5
}
]
},
{
"Orange": [
{
"type": "Blood",
"quant": 3
},
{
"type": "Navel",
"quant": 20
}
]
}
]
}
My sample Python code is as follows:
import simplejson as json
# Open file
fjson = open('/home/teg/projects/test/fruits.json', 'rb')
f = json.loads(fjson.read())
fjson.close()
# Search for fruit
if 'Orange' in json.dumps(f):
fruit = f['fruits']['Orange']
print(fruit)
else:
print('Orange does not exist')
But whenever I test it out, it gives me this error:
TypeError: list indices must be integers, not str
Was it wrong to have me do a json.dumps and instead should I have just checked the JSON feed as-is from the standard json.loads? I am getting this TypeError because I am not specifying the list index, but what if I don't know the index of that fruit?
Do I have to first search for a fruit and if it is there, get the index and then reference the index before the fruit like this?
fruit = f['fruits'][2]['Orange']
If so, how would I get the index of that fruit if it is found so I could then pull in the information? If you think the JSON is in the wrong format as well and is causing this issue, then I am up for that suggestion as well. I'm stuck on this and any help you guys have would be great. :-)
Your f type is list, it's a list of dictionary's with sub dictionary.
if 'Orange' in json.dumps(f): Will iterate the list and look at each item for Orange.
The problem is that f['fruits'] is a list so it expects an int number (place)
and not a dictionary key like ['Orange']
I think you should check your structure like #kindall said, if you still want to extract Orange this code will do the trick:
for value in f['fruits']:
if 'Orange' in value:
print value['Orange']
The problem is that the data structure has a list enclosing the dictionaries. If you have any control over the data source, that's the place to fix it. Otherwise, the best course is probably to post-process the data after parsing it to eliminate these extra list structures and merge the dictionaries in each list into a single dictionary. If you use an OrderedDict you can even retain the ordering of the items (which is probably why the list was used).
The square bracket in the line "fruits": [ should tell you that the item associated with fruits is (in Python parlance) a list rather than a dict and so cannot be indexed directly with a string like 'Oranges'. It sounds like you want to create a dict of fruits instead. You could do this by reformatting the input.
Or, if the input format is fixed: each item in your fruits list currently has a very specific format. Each item is a dict with exactly one key, and those keys are not duplicated between items. If those rules can be relied upon, it's pretty easy to write a small search routine—or the following code will convert a list-of-dicts into a dict:
fruits = dict(sum([x.items() for x in f['fruits']], []))
print fruits['Orange']
Trying to create a 4 deep nested JSON from a csv based upon this example:
Region,Company,Department,Expense,Cost
Gondwanaland,Bobs Bits,Operations,nuts,332
Gondwanaland,Bobs Bits,Operations,bolts,254
Gondwanaland,Maureens Melons,Operations,nuts,123
At each level I would like to sum the costs and include it in the outputted JSON at the relevant level.
The structure of the outputted JSON should look something like this:
{
"id": "aUniqueIdentifier",
"name": "usually a nodes name",
"data": [
{
"key": "some key",
"value": "some value"
},
{
"key": "some other key",
"value": "some other value"
}
],
"children": [/* other nodes or empty */ ]
}
(REF: http://blog.thejit.org/2008/04/27/feeding-json-tree-structures-to-the-jit/)
Thinking along the lines of a recursive function in python but have not had much success with this approach so far... any suggestions for a quick and easy solution greatly appreciated?
UPDATE:
Gradually giving up on the idea of the summarised costs because I just can't figure it out :(. I'not much of a python coder yet)! Simply being able to generate the formatted JSON would be good enough and I can plug in the numbers later if I have to.
Have been reading, googling and reading for a solution and on the way have learnt a lot but still no success in creating my nested JSON files from the above CSV strucutre. Must be a simple solution somewhere on the web? Maybe somebody else has had more luck with their search terms????
Here are some hints.
Parse the input to a list of lists with csv.reader:
>>> rows = list(csv.reader(source.splitlines()))
Loop over the list to buildi up your dictionary and summarize the costs. Depending on the structure you're looking to create the build-up might look something like this:
>>> summary = []
>>> for region, company, department, expense, cost in rows[1:]:
summary.setdefault(*region, company, department), []).append((expense, cost))
Write the result out with json.dump:
>>> json.dump(summary, open('dest.json', 'wb'))
Hopefully, the recursive function below will help get you started. It builds a tree from the input. Please be aware of what type you want your leaves to be in, which we label as the "cost". You'll need to elaborate on the function to build-up the exact structure you intend:
import csv, itertools, json
def cluster(rows):
result = []
for key, group in itertools.groupby(rows, key=lambda r: r[0]):
group_rows = [row[1:] for row in group]
if len(group_rows[0]) == 2:
result.append({key: dict(group_rows)})
else:
result.append({key: cluster(group_rows)})
return result
if __name__ == '__main__':
s = '''\
Gondwanaland,Bobs Bits,Operations,nuts,332
Gondwanaland,Bobs Bits,Operations,bolts,254
Gondwanaland,Maureens Melons,Operations,nuts,123
'''
rows = list(csv.reader(s.splitlines()))
r = cluster(rows)
print json.dumps(r, indent=4)