Develop Reference

Trying to follow django docs to create serialized json

Trying to seed a database in django app. I have a csv file that I converted to json and now I need to reformat it to match the django serialization required format found here
This is what the json format needs to look like to be acceptable to django (Which looks an awful lot like a dictionary with 3 keys, the third having a value which is a dictionary itself):
[
{
"pk": "4b678b301dfd8a4e0dad910de3ae245b",
"model": "sessions.session",
"fields": {
"expire_date": "2013-01-16T08:16:59.844Z",
...
}
}
]
My json data looks like this after converting it from csv with pandas:
[{'model': 'homepage.territorymanager', 'pk': 1, 'Name': 'Aaron ##', 'Distributor': 'National Energy', 'State': 'BC', 'Brand': 'Trane', 'Cell': '778-###-####', 'email address': None, 'Notes': None, 'Unnamed: 9': None}, {'model': 'homepage.territorymanager', 'pk': 2, 'Name': 'Aaron Martin ', 'Distributor': 'Pierce ###', 'State': 'PA', 'Brand': 'Bryant/Carrier', 'Cell': '267-###-####', 'email address': None, 'Notes': None, 'Unnamed: 9': None},...]
I am using this function to try and reformat
def re_serialize_reg_json(d, jsonFilePath):
for i in d:
d2 = {'Name': d[i]['Name'], 'Distributor' : d[i]['Distributor'], 'State' : d[i]['State'], 'Brand' : d[i]['Brand'], 'Cell' : d[i]['Cell'], 'EmailAddress' : d[i]['email address'], 'Notes' : d[i]['Notes']}
d[i] = {'pk': d[i]['pk'],'model' : d[i]['model'], 'fields' : d2}
print(d)
and it returns this error which doesn't make any sense because the format that django requires has a dictionary as the value of the third key:
d2 = {'Name': d[i]['Name'], 'Distributor' : d[i]['Distributor'], 'State' : d[i]['State'], 'Brand' : d[i]['Brand'], 'Cell' : d[i]['Cell'], 'EmailAddress' : d[i]['email address'], 'Notes' : d[i]['Notes']}
TypeError: list indices must be integers or slices, not dict
Any help appreciated!
Here is what I did to get d:
df = pandas.read_csv('/Users/justinbenfit/territorymanagerpython/territory managers - Sheet1.csv')
df.to_json('/Users/justinbenfit/territorymanagerpython/territorymanagers.json', orient='records')
jsonFilePath = '/Users/justinbenfit/territorymanagerpython/territorymanagers.json'
def load_file(file_path):
with open(file_path) as f:
d = json.load(f)
return d
d = load_file(jsonFilePath)
print(d)

D is actually a list containing multiple dictionaries, so in order to make it work you want to change that for i in d part to: for i in range(len(d)).

Python group by a object key gives errros

Does anyone know of a (itertools.groupby if possible too) way in Python to group an array of objects by an object key then create a new array of objects based on the grouping? For example, I have an array of car objects:
cars = [
{
'make': 'audi',
'model': 'r8',
'year': '2012'
}, {
'make': 'audi',
'model': 'rs5',
'year': '2013'
}, {
'make': 'ford',
'model': 'mustang',
'year': '2012'
}, {
'make': 'ford',
'model': 'fusion',
'year': '2015'
}, {
'make': 'kia',
'model': 'optima',
'year': '2012'
},
]
I want to make a new array of car objects that's grouped by make:
newCarsList = {
'audi': [
{
'model': 'r8',
'year': '2012'
}, {
'model': 'rs5',
'year': '2013'
},
],
'ford': [
{
'model': 'mustang',
'year': '2012'
}, {
'model': 'fusion',
'year': '2015'
}
],
'kia': [
{
'model': 'optima',
'year': '2012'
}
]
}
I tried with but this groupby:
def key_func(k):
return k['make']
newCarsList = []
for key, value in groupby(cars, key_func):
newCarsList.append({key: value})
print(newCarsList)
but this returns: [{'audi': <itertools._grouper object at 0x7f9b2c2bdd30>}, ...] and can't find how to fix.
Can someone help please?

A simple, mostly-functional solution:
from operator import itemgetter
from itertools import groupby
from functools import partial
def del_ret(d, key):
del d[key]
return d
dict(map(lambda k_v: (k_v[0], tuple(map(partial(del_ret, key="make"), k_v[1]))),
groupby(cars, itemgetter("make"))))
Change tuple to list to get identical output to what you want. But I assume you aren't going to modify those, so always use tuple in those circumstances…

The itertools._grouper is an iterable object. You can extract the values by iterating over it. For example, instead of appending {key: value}, you can pull the elements with a list comprehension: {key: [item for item in value]}.
It seems that your desired output is a dict though, not a list. You can get your pattern with
result = {}
for key, value in groupby(cars, key_func):
result[key] = [item for item in value]
for item in result[key]:
del item['make']
Edit: It's nicer to not add items that we're going to delete anyway. That can be done like this:
result = {}
for key, value in groupby(cars, key_func):
result[key] = [{subkey: subval for (subkey, subval)
in make.items() if subkey != 'make'}
for make in value]

df=pd.DataFrame(cars)
print(df)
for mytuple in df.itertuples():
print(mytuple)

Travers through a nested json object and store values- Python

This is a follow up on this question. Question
Also this question is similar but does not solve my problem Question2
I am trying to parse a nested json to get Check how many children a specific location has, I am trying to check if "children:" = None and increment counter to check how many levels down i need to go in order to get the lowest child, or
A more efficient solution would be:
I need to get all the child values into a list and keep going until "children:" = None.
The Json object can increase in the amount of children so we can have multiple level of children, Which can get messy if I want to nest the list and get the values, How could I do it dynamically?
{
'locationId': 'location1',
'name': 'Name',
'type': 'Ward',
'patientId': None,
'children': [{
'locationId': 'Child_location2',
'name': 'Name',
'type': 'Bed',
'patientId': None,
'children': [{
'locationId': 'Child_Child_location3',
'name': 'Name',
'type': 'HospitalGroup',
'patientId': None,
'children': None
}]
}, {
'locationId': 'location4',
'name': 'Name',
'type': 'Hospital',
'patientId': None,
'children': None
}, {
'locationId': 'location5',
'name': 'Name',
'type': 'Bed',
'patientId': None,
'children': None
}, {
'locationId': 'location6',
'name': 'Name',
'type': 'Bed',
'patientId': None,
'children': None
}, {
'locationId': 'location27',
'name': 'Name',
'type': 'Bed',
'patientId': None,
'children': None
}]
}
I tried to do something like this
import requests
def Get_Child(URL, Name):
headers = {
'accept': 'text/plain',
}
response = requests.get(
URL + Name,
headers=headers)
json_data = response.json()
print (json_data)
list = []
for locationId in json_data['locationId']:
list.append(locationId)
for children in locationId['children']:
list.append(children)
but that give me the following error,
for children in locationId['locationId']: TypeError: string indices must be integers

Your code shows append, but you ask for a count. Here is a recursive way to get the number of children in this JSON if I am understanding you correctly:
def get_children(body, c=1):
if not body.get('children'):
c += 1
elif isinstance(body.get('children'), list):
c += 1
for subchild in body.get('children'):
c += 1
get_children(subchild, c)
return c
counts = get_children(your_json_blob)
print(counts)
>>> 7
Edit: I purposely did not use if/else because I don't know if you can have subchildren that are dict rather than list which would mean you would need extra conditions, but that's up to you if that ends up being the case.

I found a solution fro my problem,
The following code will get all the children and append them to a list
class Children():
def Get_All_Children(self,json_input, lookup_key):
if isinstance(json_input, dict):
for k, v in json_input.items():
if k == lookup_key:
yield v
else:
yield from self.Get_All_Children(v, lookup_key)
elif isinstance(json_input, list):
for item in json_input:
yield from self.Get_All_Children(item, lookup_key)
for locations in self.Get_All_Children(self.json_data, 'locationId'):
self.mylist.append(locations)

KeyError: 'name' Why can't I use 'name'?

I wanna make a dictionary has name's key & data.In views.py I wrote
data_dict ={}
def try_to_int(arg):
try:
return int(arg)
except:
return arg
def main():
book4 = xlrd.open_workbook('./data/excel1.xlsx')
sheet4 = book4.sheet_by_index(0)
data_dict_origin = OrderedDict()
tag_list = sheet4.row_values(0)[1:]
for row_index in range(1, sheet4.nrows):
row = sheet4.row_values(row_index)[1:]
row = list(map(try_to_int, row))
data_dict_origin[row_index] = dict(zip(tag_list, row))
if data_dict_origin['name'] in data_dict:
data_dict[data_dict_origin['name']].update(data_dict_origin)
else:
data_dict[data_dict_origin['name']] = data_dict_origin
main()
When I printed out data_dict,it is
OrderedDict([(1, {'user_id': '100', 'group': 'A', 'name': 'Tom', 'dormitory': 'C'}), (2, {'user_id': '50', 'group': 'B', 'name': 'Blear', 'dormitory': 'E'})])
My ideal dictionary is
dicts = {
Tom: {
'user_id': '100',
'group': 'A',
'name': 'Tom',
'dormitory': 'C'
},
Blear: {
},
}
How should I fix this?What should I write it?

The code is using the wrong key in the dictionary. The keys are 1, 2, and do not have the name key. You can use this code instead:
for value in data_dict.values():
if value['name'] in data_dict:
data_dict[value['name']].update(value)
else:
data_dict[value['name']] = value

Your data_dict_origin has numbers as keys and dicts as values (which technically makes it a sparse array of dicts). The "name" key exists in those dicts, not in your data_dict.

How can I recursively add dictionaries in Python from JSON?

Dear Stackoverflow Members,
I have this JSON array, and it consists of the following items (basically):
{
{
'Name': 'x',
'Id': 'y',
'Unsusedstuff' : 'unused',
'Unsusedstuff2' : 'unused2',
'Children': []
},
{ 'Name' : 'xx',
'Id': 'yy',
'Unsusedstuff' : 'unused',
'Unsusedstuff2' : 'unused2',
'Children': [{
'Name': 'xyx',
'Id' : 'yxy',
'Unsusedstuff' : 'unused',
'Unsusedstuff2' : 'unused2',
'Children: []
}
You get the basic idea. I want to emulate this (and just grab the id and the name and the structure) in a Python-list using the following code:
names = []
def parseNames(col):
for x in col:
if(len(x['Children'])> 0):
names.append({'Name' : x['Name'], 'Id' : x['Id'], 'Children' : parseNames(x['Children'])})
else:
return {'Name' : x['Name'], 'Id' : x['Id']}
But, it only seems to return the first 'root' and the first nested folder, but doesn't loop through them all.
How would I be able to fix this?
Greetings,
Mats

The way I read this, you're trying to convert this tree into a tree of nodes which only have Id, Name and Children. In that case, the way I'd think of it is as cleaning nodes.
To clean a node:
Create a node with the Name and Id of the original node.
Set the new node's Children to be the cleaned versions of the original node's children. (This is the recursive call.)
In code, that would be:
def clean_node(node):
return {
'Name': node['Name'],
'Id': node['Id'],
'Children': map(clean_node, node['Children']),
}
>>> print map(clean_node, data)
[{'Name': 'x', 'Children': [], 'Id': 'y'}, {'Name': 'xx', 'Children': [{'Name': 'xyx', 'Children': [], 'Id': 'yxy'}], 'Id': 'yy'}]
I find it's easier to break recursive problems down like this - trying to use global variables turns simple things very confusing very quickly.

Check this
def parseNames(col):
for x in col:
if(len(x['Children'])> 0):
a = [{
'Name' : x['Name'],
'Id' : x['Id'],
'Children' : x['Children'][0]['Children']
}]
parseNames(a)
names.append({'Name' : x['Name'], 'Id' : x['Id']})
return names
Output I get is
[{'Name': 'x', 'Id': 'y'}, {'Name': 'xx', 'Id': 'yy'}, {'Name': 'xx', 'Id': 'yy'}]

You can parse a Json object with this:
import json
response = json.loads(my_string)
Now response is a dictionary with the keys of every Json object.