Develop Reference

Python how to parse a list[dict] in python and convert values from nested dictionaries as keys

Need help in writing clean code , I have a yaml parsed output which looks like this :
yaml_output = [{'name' : 'alex', 'subjects' : {'maths' : ['grade_1', 'grade_2']}},
{'name' : 'rio', 'subjects' : {'maths' : ['grade_3', 'grade_2'], 'science : ['grade_4', 'grade_6']}}]
I want it create a list of dictionaries with subjects as key and name of the students as value in a dictionary where grades are the keys.
desired_output = [{'maths' : {'grade_1' : ['alex'], 'grade_2' : ['alex', 'rio'], 'grade_3' : ['rio']}}, {'science' : {'grade_4' : ['rio'], 'grade_6' : ['rio']}
needs subjects as key and name of the students as value in a dictionary where grades are the keys.
new_dict = []
for dict in yaml_output:
for k,v in dict:
for i,j in dict['subjects']:
if any(i in dict_list for dict_list in new_dict):
dict_list[i].append(v)

You were close. Your for k,v loop is looking at the wrong data. You don't want to look at ALL the keys, you want to unravel the subjects key and reference the "name" specifically.
yaml_output = [{'name' : 'alex', 'subjects' : {'maths' : ['grade_1', 'grade_2']}},
{'name' : 'rio', 'subjects' : {'maths' : ['grade_3', 'grade_2'], 'science': ['grade_4', 'grade_6']}}]
out = dict()
for data in yaml_output:
for k,v in data['subjects'].items():
if k not in out:
out[k] = {}
for g in v:
if g not in out[k]:
out[k][g] = []
out[k][g].append( data['name'] )
print(out)
Output:
{'maths': {'grade_1': ['alex'], 'grade_2': ['alex', 'rio'], 'grade_3': ['rio']}, 'science': {'grade_4': ['rio'], 'grade_6': ['rio']}}

Combine two list into a Dict, Tuple

I am creating a rest-api using the Python+Flask_Restful combo, find it amazing.
Currently, i am allowing user to run a sql-query using the browser and it works fine, but the problem is, headers information is not displayed in the response.
Here is the code that i am using:
class RunCustomeSQL(Resource):
def get(self, enter_db_name, query):
if not os.path.isfile(enter_db_name+'.db'):
raise BadRequest("Database Doesn't Exist. Please select a valid database")
conn = sqlite3.connect(enter_db_name+'.db')
search_out = []
cursor = conn.execute(query)
row = None
for row in cursor:
search_out.append(row)
if not row: #This means empty response
raise BadRequest("No Results Found")
conn.commit()
conn.close()
return search_out
While, this code works great, it doesn't print the header values in the json-response. The current response is :
[
[
"dusiri_bibi",
"11",
"----------",
" srt/None ",
"14.30 MB",
"2017-12-13 23:43:54",
"C:/Test_Software/vc_redist.x64.exe"
],
]
Expected Output :
[
[
"Machine Name" : "dusiri_bibi",
"LABEL" : "11",
"PERMISSIONS" : "----------",
"USER" : " srt/None ",
"SIZE" : "14.30 MB",
"CREATED" : "2017-12-13 23:43:54",
"FILENAME" : "C:/Test_Software/vc_redist.x64.exe"
],
]
All the above text such as "machine name, label etc." are my table headers, I am not sure how to print them along with my output.
What if the user runs select user, size from table_name only
or
What if the user runs select * from table_name
In both scenario's, the output should display the table headers
Thanks
UPDATE #1 (25 April) : I managed to answer my first question and able to display a proper json response if the user selects the SELECT * statement in SQL but still facing issue with the second piece of it
Here is the answer to first part if anyone is looking for it : Using Regex
row = None
if re.search(r'(?<=SELECT)(.*)(?=FROM)',query, re.IGNORECASE).group(1) == ' * ':
for row in cursor:
search_out.append({'NAME' : row[0], 'LABEL_NUMBER' : row[1], 'PERM' : row[2], 'USER' : row[3] , 'SIZE' : row[4], 'DATE' : row[5], 'FILENAME' : row[6]})
if not row: #This means empty response
raise BadRequest("No Results Found")
Part II : Unanswered Query:
For the second piece, i now have two list :
list_1 : [[u'LABEL_NUMBER', u'PERM', u'FILENAME']]
list_2 : [(u'11', u'----------', u'C:/Test_Software/26.avi'), (u'11', u'----------', u'C:/Test_Software/6.avi'), (u'11', u'-rwx------', u'C:/Test_Software/Debug/Current_Frame1.avi'), (u'10', u'-rwxrwx---', u'C:/Windows/WinSxS/boxed-split.avi')]
As you can see, i have two list and i want to combine them into a dict to show the response like this:
[
{
LABEL_NUMBER : '11' ,
PERM : '-----------',
FILENAME : 'C:/Test_Software/26.avi'
},
...
....
......
{
LABEL_NUMBER : '10' ,
PERM : '-rwxrwx---',
FILENAME : 'C:/Windows/WinSxS/boxed-split.avi'
},
]
i am using the following code to do the same :
chunks = [list_2[idx:idx+3] for idx in range(0, len(list_2), 3)]
output = []
for each in chunks:
output.append(dict(zip(list_1, each)))
print(output)
But, this is failing with "TypeError: unhashable type: 'list'", i understand that lists are mutable and which is why i am getting this error but then how can i get the desired dict response? what am i doing wrong here?

You can use a list comprehension combined with zip for this:
list_1 = [[u'LABEL_NUMBER', u'PERM', u'FILENAME']]
list_2 = [(u'11', u'----------', u'C:/Test_Software/26.avi'), (u'11', u'----------', u'C:/Test_Software/6.avi'), (u'11', u'-rwx------', u'C:/Test_Software/Debug/Current_Frame1.avi'), (u'10', u'-rwxrwx---', u'C:/Windows/WinSxS/boxed-split.avi')]
d = [dict(zip(list_1[0], i)) for i in list_2]
Result:
[{'FILENAME': 'C:/Test_Software/26.avi',
'LABEL_NUMBER': '11',
'PERM': '----------'},
{'FILENAME': 'C:/Test_Software/6.avi',
'LABEL_NUMBER': '11',
'PERM': '----------'},
{'FILENAME': 'C:/Test_Software/Debug/Current_Frame1.avi',
'LABEL_NUMBER': '11',
'PERM': '-rwx------'},
{'FILENAME': 'C:/Windows/WinSxS/boxed-split.avi',
'LABEL_NUMBER': '10',
'PERM': '-rwxrwx---'}]

Python - append to dictionary by name with multilevels 1, 1.1, 1.1.1, 1.1.2 (hierarchical)

I use openpyxl to read data from excel files to provide a json file at the end. The problem is that I cannot figure out an algorithm to do a hierarchical organisation of the json (or python dictionary).
The data form is like the following:
The output should be like this:
{
'id' : '1',
'name' : 'first',
'value' : 10,
'children': [ {
'id' : '1.1',
'name' : 'ab',
'value': 25,
'children' : [
{
'id' : '1.1.1',
'name' : 'abc' ,
'value': 16,
'children' : []
}
]
},
{
'id' : '1.2',
...
]
}
Here is what I have come up with, but i can't go beyond '1.1' because '1.1.1' and '1.1.1.1' and so on will be at the same level as 1.1.
from openpyxl import load_workbook
import re
from json import dumps
wb = load_workbook('resources.xlsx')
sheet = wb.get_sheet_by_name(wb.get_sheet_names()[0])
resources = {}
prev_dict = {}
list_rows = [ row for row in sheet.rows ]
for nrow in range(list_rows.__len__()):
id = str(list_rows[nrow][0].value)
val = {
'id' : id,
'name' : list_rows[nrow][1].value ,
'value' : list_rows[nrow][2].value ,
'children' : []
}
if id[:-2] == str(list_rows[nrow-1][0].value):
prev_dict['children'].append(val)
else:
resources[nrow] = val
prev_dict = resources[nrow]
print dumps(resources)

You need to access your data by ID, so first step is to create a dictionary where the IDs are the keys. For easier data manipulation, string "1.2.3" is converted to ("1","2","3") tuple. (Lists are not allowed as dict keys). This makes the computation of a parent key very easy (key[:-1]).
With this preparation, we could simply populate the children list of each item's parent. But before doing that a special ROOT element needs to be added. It is the parent of top-level items.
That's all. The code is below.
Note #1: It expects that every item has a parent. That's why 1.2.2 was added to the test data. If it is not the case, handle the KeyError where noted.
Note #2: The result is a list.
import json
testdata="""
1 first 20
1.1 ab 25
1.1.1 abc 16
1.2 cb 18
1.2.1 cbd 16
1.2.1.1 xyz 19
1.2.2 NEW -1
1.2.2.1 poz 40
1.2.2.2 pos 98
2 second 90
2.1 ezr 99
"""
datalist = [line.split() for line in testdata.split('\n') if line]
datadict = {tuple(item[0].split('.')): {
'id': item[0],
'name': item[1],
'value': item[2],
'children': []}
for item in datalist}
ROOT = ()
datadict[ROOT] = {'children': []}
for key, value in datadict.items():
if key != ROOT:
datadict[key[:-1]]['children'].append(value)
# KeyError = parent does not exist
result = datadict[ROOT]['children']
print(json.dumps(result, indent=4))

how to combine different complex list with python

BIG = { "Brand" : ["Clothes" , "Watch"], "Beauty" : ["Skin", "Hair"] }
SMA = { "Clothes" : ["T-shirts", "pants"], "Watch" : ["gold", "metal"],
"Skin" : ["lotion", "base"] , "Hair" : ["shampoo", "rinse"]}
I want to combine this data
like this...
BIG = {"Brand" : [ {"Clothes" : ["T-shirts", "pants"]}, {"Watch" : ["gold", "metal"]} ],...
Please tell me how to solve this problem.

First off, those are dictionaries and not lists. Also, I do no know your intention behind merging two dictionaries in that representation.
Anyways, if you want the output to be exactly as you mentioned, then this is the way to do it -
BIG = { "Brand" : ["Clothes" , "Watch"], "Beauty" : ["Skin", "Hair"] }
SMA = { "Clothes" : ["T-shirts", "pants"], "Watch" : ["gold", "metal"],"Skin" : ["lotion", "base"] , "Hair" : ["shampoo", "rinse"]}
for key,values in BIG.items(): #change to BIG.iteritems() in python 2.x
newValues = []
for value in values:
if value in SMA:
newValues.append({value:SMA[value]})
else:
newValues.append(value)
BIG[key]=newValues
Also, BIG.update(SMA) will not give you the right results in the way you want them to be.
Here is a test run -
>>> BIG.update(SMA)
>>> BIG
{'Watch': ['gold', 'metal'], 'Brand': ['Clothes', 'Watch'], 'Skin': ['lotion', 'base'], 'Beauty': ['Skin', 'Hair'], 'Clothes': ['T-shirts', 'pants'], 'Hair': ['shampoo', 'rinse']}

Firstly, you need to iterate on first dictionary and search the pair of key in second dictionary.
BIG = { "Brand" : ["Clothes" , "Watch"], "Beauty" : ["Skin", "Hair"] }
SMA = { "Clothes" : ["T-shirts", "pants"], "Watch" : ["gold", "metal"], "Skin" : ["lotion", "base"] , "Hair" : ["shampoo", "rinse"]}
for key_big in BIG:
for key_sma in BIG[key_big]:
if key_sma in SMA:
BIG[key_big][BIG[key_big].index(key_sma)] = {key_sma: SMA.get(key_sma)}
print BIG
The result of code:
>>> {'Brand': [{'Clothes': ['T-shirts', 'pants']}, {'Watch': ['gold', 'metal']}], 'Beauty': [{'Skin': ['lotion', 'base']}, {'Hair': ['shampoo', 'rinse']}]}

how to add items to the list that is in the dictionary

inventory = {
'gold' : 500,
'pouch' : ['flint', 'twine', 'gemstone'], # Assigned a new list to 'pouch' key
'backpack' : ['xylophone','dagger', 'bedroll','bread loaf'],
'pocket': ['seashell','strange berry','lint']
}
how to Add 50 to the number stored under the 'gold' key

Elementary, Watson:
inventory['gold'] += 50