First of all, sorry for the title, I couldn't think of a good one to be honest.
I have a list of dictionaries like this
data=[
{'identifier': 'ID', 'resource': 'resource1' , 'name': 'name1'},
{'identifier': 'ID', 'resource': 'resource2' , 'name': 'name1'},
{'identifier': 'ID', 'resource': 'resource3' , 'name': 'name1'},
{'identifier': 'ID', 'resource': 'resource1' , 'name': 'name2'},
{'identifier': 'ID', 'resource': 'resource2' , 'name': 'name2'},
{'identifier': 'ID', 'resource': 'resource3' , 'name': 'name2'}
]
Basically, I want a dict that contains the name and every resource with that name, something like this
final = [
{
'name': 'name1',
'resources': ['resource1','resource2','resource3']
},
{
'name': 'name2',
'resources': ['resource1','resource2','resource3']
}
]
I have tried some approaches, like iterating over the first list and verifying if the key value pair already exists on the second one, so after that I can append the resource to the key I want but clearly thats not the correct way to do it.
Im sure there's a way to easily do this but I cant wrap my head around on how to achieve it. Any ideas on how can this be done?
Group with a collections.defaultdict, with name as the grouping key and the resources for each name appended to a list:
from collections import defaultdict
data = [
{"identifier": "ID", "resource": "resource1", "name": "name1"},
{"identifier": "ID", "resource": "resource2", "name": "name1"},
{"identifier": "ID", "resource": "resource3", "name": "name1"},
{"identifier": "ID", "resource": "resource1", "name": "name2"},
{"identifier": "ID", "resource": "resource2", "name": "name2"},
{"identifier": "ID", "resource": "resource3", "name": "name2"},
]
d = defaultdict(list)
for x in data:
d[x["name"]].append(x["resource"])
result = [{"name": k, "resources": v} for k, v in d.items()]
print(result)
However, since your names are ordered, we can also get away with using itertools.groupby:
from itertools import groupby
from operator import itemgetter
result = [
{"name": k, "resources": [x["resource"] for x in g]}
for k, g in groupby(data, key=itemgetter("name"))
]
print(result)
If your names are not ordered, then we will need to sort data by name:
result = [
{"name": k, "resources": [x["resource"] for x in g]}
for k, g in groupby(sorted(data, key=itemgetter("name")), key=itemgetter("name"))
]
print(result)
Output:
[{'name': 'name1', 'resources': ['resource1', 'resource2', 'resource3']}, {'name': 'name2', 'resources': ['resource1', 'resource2', 'resource3']}]
Note: I would probably just stick with the first defaultdict solution in most cases, because it doesn't care about order.
Related
I have a json like:
pd = {
"RP": [
{
"Name": "PD",
"Value": "qwe"
},
{
"Name": "qwe",
"Value": "change"
}
],
"RFN": [
"All"
],
"RIT": [
{
"ID": "All",
"IDT": "All"
}
]
}
I am trying to change the value change to changed. This is a dictionary within a list which is within another dictionary. Is there a better/ more efficient/pythonic way to do this than what I did below:
for key, value in pd.items():
ls = pd[key]
for d in ls:
if type(d) == dict:
for k,v in d.items():
if v == 'change':
pd[key][ls.index(d)][k] = "changed"
This seems pretty inefficient due to the amount of times I am parsing through the data.
String replacement could work if you don't want to write depth/breadth-first search.
>>> import json
>>> json.loads(json.dumps(pd).replace('"Value": "change"', '"Value": "changed"'))
{'RP': [{'Name': 'PD', 'Value': 'qwe'}, {'Name': 'qwe', 'Value': 'changed'}],
'RFN': ['All'],
'RIT': [{'ID': 'All', 'IDT': 'All'}]}
I have two arrays. "Categories" and "Products".
I need to get a third array where in each category there will be a list of products to which it belongs.
categories = [
{
'id': 1,
'name': 'first category'
},
{
'id': 2,
'name': 'second category'
},
{
'id': 3,
'name': 'third category'
},
]
products = [
{
'id': 1,
'id_category': 1,
'name': 'first product'
},
{
'id': 2,
'id_category': 2,
'name': 'second product'
},
{
'id': 3,
'id_category': 2,
'name': 'third product'
},
]
I need to combine these two arrays into one, by "id_category", to get the following:
the conditions must be met:
if there are no products in the category, then remove category from the list;
products_by_category = [
{
"id":1,
"name":"first category",
"products":[
{
"id":1,
"name":"first product"
}
]
},
{
"id":2,
"name":"second category",
"products":[
{
"id":2,
"name":"second product"
},
{
"id":3,
"name":"third product"
}
]
}
]
i tried the following code:
for category in list(categories):
category['products'] = []
for product in products:
if category['id'] == product['id_category']:
category['products'].append(product)
if not category['products']:
categories.remove(category)
print(categories)
Here is something which does what you want. Note it does not create a new data structure just updates the categories in place.
catdict = {cat['id']:cat for cat in categories}
for product in products:
pcat = product['id_category']
del product['id_category']
cat = catdict[pcat]
if not "products" in cat:
cat["products"] = []
cat["products"].append(product)
print(categories)
Here is another approach using defaultdict, create a lookup between id_category along with there details. Followed by list comprehension to update back to categories.
from collections import defaultdict
# create a mapping for each product
prod_lookup = defaultdict(list)
for prod in products:
prod_lookup[prod['id_category']].append(
{"id": prod['id'], "name": prod['name']}
)
# update back to categories, based on "id"
(
[{**cat, **{"products": prod_lookup.get(cat['id'], [])}}
for cat in categories if prod_lookup.get(cat['id'])]
)
After parsing a URL parameter for partial responses, e.g. ?fields=name,id,another(name,id),date, I'm getting back an arbitrarily nested list of strings, representing the individual keys of a nested JSON object:
['name', 'id', ['another', ['name', 'id']], 'date']
The goal is to map that parsed 'graph' of keys onto an original, larger dict and just retrieve a partial copy of it, e.g.:
input_dict = {
"name": "foobar",
"id": "1",
"another": {
"name": "spam",
"id": "42",
"but_wait": "there is more!"
},
"even_more": {
"nesting": {
"why": "not?"
}
},
"date": 1584567297
}
should simplyfy to:
output_dict = {
"name": "foobar",
"id": "1",
"another": {
"name": "spam",
"id": "42"
},
"date": 1584567297,
}
Sofar, I've glanced over nested defaultdicts, addict and glom, but the mappings they take as inputs are not compatible with my list (might have missed something, of course), and I end up with garbage.
How can I do this programmatically, and accounting for any nesting that might occur?
you can use:
def rec(d, f):
result = {}
for i in f:
if isinstance(i, list):
result[i[0]] = rec(d[i[0]], i[1])
else:
result[i] = d[i]
return result
f = ['name', 'id', ['another', ['name', 'id']], 'date']
rec(input_dict, f)
output:
{'name': 'foobar',
'id': '1',
'another': {'name': 'spam', 'id': '42'},
'date': 1584567297}
here the assumption is that on a nested list the first element is a valid key from the upper level and the second element contains valid keys from a nested dict which is the value for the first element
I have the following list of string:
diabo_faves = [
{"Description": "Local-Standard-Office"},
{"Description": "Local-Standard-Extended},
{"Description": "Local-Standard-247"},
{"Description": "Direct-Standard-Office"},
{"Description": "Direct-Standard-Extended"},
{"Description": "Direct-Standard-247"},
{"Description": "Premium-Standard-Office"},
{"Description": "Premium-Standard-Extended"},
{"Description": "Premium-Standard-247"},
];
And I want to get the following result from it in JSON (Keeping the order):
Filters: [
[
"Local",
"Direct",
"Premium"
],
[
"Standard"
],
[
"Office",
"Extended",
"247"
]
]
However, when I use this method:
def get_filters(self, favorites, seperator = "-", ignores = []):
favorites = self.prepare(favorites, ignores)
steps = []
for fave in favorites:
filters = fave.split(seperator)
for index, filter in enumerate(filters):
try:
steps[index].add(filter)
except IndexError:
steps.append(set([filter]))
return [list(step) for step in steps]
I get the result in the incorrect order like this:
Filters: [
[
"Premium",
"Local",
"Direct"
],
[
"Standard"
],
[
"247",
"Extended",
"Office"
]
]
How can I change my method to keep the original order that was specified? "Local", "Direct", "Premium" etc..
You can do this using collections.OrderedDict and itertools.izip:
from collections import OrderedDict
from itertools import izip
splitted_values = izip(*(x['Description'].split('-') for x in diabo_faves))
print [list(OrderedDict.fromkeys(x)) for x in splitted_values]
#[['Local', 'Direct', 'Premium'], ['Standard'], ['Office', 'Extended', '247']]
Here splitted_values contains all the column values:
[
('Local', 'Local', 'Local', 'Direct', 'Direct', 'Direct', 'Premium', 'Premium', 'Premium'),
('Standard', 'Standard', 'Standard', 'Standard', 'Standard', 'Standard', 'Standard', 'Standard', 'Standard'),
('Office', 'Extended', '247', 'Office', 'Extended', '247', 'Office', 'Extended', '247')
]
Then we pass each of these to OrderedDict.fromkeys to only get the unique items in the order they were seen first. If OrderedDict is not available to you(introduced in Python 2.7) then you can use the unique_everseen recipe from itertoools.
I am creating dict in python and this dict may includes lists and sub dicts . If I want to check a key exist in any dict first I have reach to that dict and then only I can check wether the key exists or not . But is there any way that I can check key existence using parent dict only means if parent dict have any child dicts then searching goes to every level and let me know that wether the key exist in entire dict or not and then using that key I can insert or delete elments in that dict which includes that key.
e.g., I have this
[
{
'groupName': u'MainMenu_1',
'menuItems': [
{
'action': u'/home\r\n',
'sub_1': [
],
'id': 1L,
'name': u'Home\r\n'
},
{
'action': u'/inventory',
'sub_2': [
],
'id': 2L,
'name': u'Inventory\r\n'
},
{
'action': u'/accounting\r\n',
'sub_3': [
{
'action': u'/gl\r\n',
'name': u'GL\r\n',
'sub_4': [
]
},
{
'action': u'/ap\r\n',
'name': u'AP\r\n',
'sub_5': [
]
},
{
'action': u'/ar\r\n',
'sub_6': [
],
'name': u'AR\r\n'
}
],
'id': 3L,
'name': u'Accounting\r\n'
},
{
'action': u'/crm\r\n',
'sub_8': [
],
'id': 8L,
'name': u'CRM\r\n'
}
]
},
{
'groupName': u'MainMenu_2',
'menuItems': [
{
'action': u'/iv-receive\r\n',
'sub_9': [
],
'id': 9L,
'name': u'Receiving\r\n'
},
{
'action': u'/iv-shipping\r\n',
'sub_10': [
],
'id': 10L,
'name': u'Shipping\r\n'
}
]
}
]
Now if in above example I want to search for any key like sub_1 , sub_3, sub_6 then how I can search for this key
We can search for all qualifying dictionary recursively. The following implementation appends all references of such dictionaries to the list found:
def recursive_search(items, key):
found = []
for item in items:
if isinstance(item, list):
found += recursive_search(item, key)
elif isinstance(item, dict):
if key in item:
found.append(item)
found += recursive_search(item.values(), key)
return found
found = recursive_search(items, 'sub_9')
def flatten(obj):
if isinstance(obj, list):
result = set()
for e in obj:
result = result.union(flatten(e))
return result
elif isinstance(obj, dict):
return set(obj.keys()) | flatten(obj.values())
else:
return {obj}
You can alternatively flatten the list/dictionary into a set, and do as many searches as you want quickly.
print(flatten(lst))
set([1, 2, 3, u'/inventory', 8, 9, 'menuItems', u'/home\r\n', u'/ar\r\n', u'AP\r\n', u'/accounting\r\n', 'id', u'MainMenu_2', u'Receiving\r\n', u'Home\r\n', u'MainMenu_1', u'/iv-shipping\r\n', 'groupName', 10, u'AR\r\n', u'Accounting\r\n', u'/ap\r\n', u'Inventory\r\n', u'CRM\r\n', u'/crm\r\n', u'/iv-receive\r\n', 'sub_10', u'GL\r\n', u'Shipping\r\n', 'name', 'sub_2', 'sub_3', 'sub_1', 'sub_6', 'sub_4', 'sub_5', 'action', 'sub_8', 'sub_9', u'/gl\r\n'])