Develop Reference

How to parse a list of dictionaries in Python 3

I'm using python(requests) to query an API. The JSON response is list of dictionaries, like below:
locationDescriptions = timeseries.publish.get('/GetLocationDescriptionList')['LocationDescriptions']
print(locationDescriptions[0])
{'Name': 'Test',
'Identifier': '000045',
'UniqueId': '3434jdfsiu3hk34uh8',
'IsExternalLocation': False,
'PrimaryFolder': 'All Locations',
'SecondaryFolders': [],
'LastModified': '2021-02-09T06:01:25.0446910+00:00',}
I'd like to extract 1 field (Identifier) as a list for further analysis (count, min, max, etc.) but I'm having a hard time figuring out how to do this.

Python has a syntax feature called "list comprehensions", and you can do something like:
identifiers = [item['Identifier'] for item in locationDescriptions]
Here is a small article that gives you more details, and also shows an alternate way using map. And here is one of the many resources detailing list comprehensions, should you need it.

You could extract them with a list comprehension:
identifiers = [i['Identifier'] for i in locationDescriptions]
You allude to needing a list of numbers (count, min, max, etc...), in which case:
identifiers = [int(i['Identifier']) for i in locationDescriptions]

You can do
ids = [locationDescription['Identifier'] for locationDescription in locationDescriptions]
You will have a list of identifiers as a string.
Best regards

Extract values from json-file which has no unique markers

A json-file which has unique markers (or [more appropriate] field-names) preceeding the values is (rather) easy to dissect, because you can perform a string search on the unique markers/field-names to find within the string the first and last position of the characters of the value, and with that info you can pinpoint the position of the value, and extract the value.
Have performed that function with various lua-scripts and Python-scripts (also on xml-files).
Now need to extract values from a json-file which does not have unique markers/ field-names, but just a multiple occurrence of "value_type" and "value", preceeding the 'name', respectively the 'value': see below.
{
"software_version": "NRZ-2017-099",
"age":"78",
"sensordatavalues":[
{"value_type":"SDS_P1","value":"4.43"},
{"value_type":"SDS_P2","value":"3.80"},
{"value_type":"temperature","value":"20.10"},
{"value_type":"humidity","value":"44.50"},
{"value_type":"samples","value":"614292"},
{"value_type":"min_micro","value":"233"},
{"value_type":"max_micro","value":"25951"},
{"value_type":"signal","value":"-66"}
]
}
Experience as described above does not provide working solution.
Question: In this json-filelayout, how to directly extract the specific, individual values (preferably by lua-script)?
[Or might XML-parsing provide an easier solution?]

Here is Python to read the JSON file and make it more convenient:
import json
import pprint
with open("/tmp/foo.json") as j:
data = json.load(j)
for sdv in data.pop('sensordatavalues'):
data[sdv['value_type']] = sdv['value']
pprint.pprint(data)
The results:
{'SDS_P1': '4.43',
'SDS_P2': '3.80',
'age': '78',
'humidity': '44.50',
'max_micro': '25951',
'min_micro': '233',
'samples': '614292',
'signal': '-66',
'software_version': 'NRZ-2017-099',
'temperature': '20.10'}

You might want to have a look into filter functions.
E.g. in your example json to get only the dict that contains the value for samples you could go by:
sample_sensordata = list(filter(lambda d: d["value_type"] == "samples", your_json_dict["sensordatavalues"]))
sample_value = sample_sensordata["value"]
To make a dictionary like Ned Batchelder said you could also go with a dict comprehension like this:
sensor_data_dict = {d['value_type']: d['value'] for d in a}
and then get the value you want just by sensor_data_dict['<ValueTypeYouAreLookingFor>']

A little bit late and I'm trying Anvil in which the previous answers didn't work. just for the curious people.
resp = anvil.http.request("http://<ipaddress>/data.json", json=True)
#print(resp) # prints json file
tempdict = resp['sensordatavalues'][2].values()
humiddict = resp['sensordatavalues'][3].values()
temperature = float(list(tempdict)[1])
humidity = float(list(humiddict)[1])
print(temperature)
print(humidity)

How do I turn list values into an array with an index that matches the other dic values?

Hoping someone can help me out. I've spent the past couple hours trying to solve this, and fair warning, I'm still fairly new to python.
This is a repost of a question I recently deleted. I've misinterpreted my code in the last example.The correct example is:
I have a dictionary, with a list that looks similar to:
dic = [
{
'name': 'john',
'items': ['pants_1', 'shirt_2','socks_3']
},
{
'name': 'bob',
items: ['jacket_1', 'hat_1']
}
]
I'm using .append for both 'name', and 'items', which adds the dic values into two new lists:
for x in dic:
dic_name.append(dic['name'])
dic_items.append(dic['items'])
I need to split the item value using '_' as the delimiter, so I've also split the values by doing:
name, items = [i if i is None else i.split('_')[0] for i in dic_name],
[if i is None else i.split('_')[0] for i in chain(*dic_items)])
None is used in case there is no value. This provides me with a new list for name, items, with the delimiter used. Disregard the fact that I used '_' split for names in this example.
When I use this, the index for name, and item no longer match. Do i need to create the listed items in an array to match the name index, and if so, how?
Ideally, I want name[0] (which is john), to also match items[0] (as an array of the items in the list, so pants, shirt, socks). This way when I refer to index 0 for name, it also grabs all the values for items as index 0. The same thing regarding the index used for bob [1], which should match his items with the same index.
#avinash-raj, thanks for your patience, as I've had to update my question to reflect more closely to the code I'm working with.

I'm reading a little bit between the lines but are you trying to just collapse the list and get rid of the field names, e.g.:
>>> dic = [{'name': 'john', 'items':['pants_1','shirt_2','socks_3']},
{'name': 'bob', 'items':['jacket_1','hat_1']}]
>>> data = {d['name']: dict(i.split('_') for i in d['items']) for d in dic}
>>> data
{'bob': {'hat': '1', 'jacket': '1'},
'john': {'pants': '1', 'shirt': '2', 'socks': '3'}}
Now the data is directly related vs. indirectly related via a common index into 2 lists. If you want the dictionary split out you can always
>>> dic_name, dic_items = zip(*data.items())
>>> dic_name
('bob', 'john')
>>> dic_items
({'hat': '1', 'jacket': '1'}, {'pants': '1', 'shirt': '2', 'socks': '3'})

You need a list of dictionaries because the duplicate keys name and items are overwritten:
items = [[i.split('_')[0] for i in d['items']] for d in your_list]
names = [d['name'] for d in your_list] # then grab names from list
Alternatively, you can do this in one line with the built-in zip method and generators, like so:
names, items = zip(*((i['name'], [j.split('_')[0] for j in i['items']]) for i in dic))

From Looping Techniques in the Tutorial.
for name, items in div.items():
names.append(name)
items.append(item)
That will work if your dict is structured
{'name':[item1]}

In the loop body of
for x in dic:
dic_name.append(dic['name'])
dic_items.append(dic['items'])
you'll probably want to access x (to which the items in dic will be assigned in turn) rather than dic.

Python Dictionary - List Nesting

I have a dictionary with a key and a pair of values, the values are stored in a List. But i'm keeping the list empty so i can .append its values ,i cant seem to be able to do this
>>>myDict = {'Numbers':[]}
>>>myDict['Numbers'[1].append(user_inputs)
doesn't seem to work, returns an error . How do i refer to the list in myDict so i can append its values.
Also is it possible to have a dictionary inside a list and also have another list inside? if so? what is its syntax or can you recommend anyother way i can do this
>>>myDict2 = {'Names': [{'first name':[],'Second name':[]}]}
do i change the second nested list to a tuple?? Please lets keep it to PYTHON 2.7

You get an error because your syntax is wrong. The following appends to the list value for the 'Numbers' key:
myDict['Numbers'].append(user_inputs)
You can nest Python objects arbitrarily; your myDict2 syntax is entirely correct. Only the keys need to be immutable (so a tuple vs. a list), but your keys are all strings:
>>> myDict2 = {'Names': [{'first name':[],'Second name':[]}]}
>>> myDict2['Names']
[{'first name': [], 'Second name': []}]
>>> myDict2['Names'][0]
{'first name': [], 'Second name': []}
>>> myDict2['Names'][0]['first name']
[]

You should access the list with myDict['Numbers']:
>>>myDict['Numbers'].append(user_inputs)
You can have dicts inside of a list.
The only catch is that dictionary keys have to be immutable, so you can't have dicts or lists as keys.

You may want to look into the json library, which supports a mix of nested dictionaries and lists.
In addition, you may also be interested in the setdefault method of the dictionary class.
Format is something like:
new_dict = dict()
some_list = ['1', '2', '3', ...]
for idx, val in enumerate(some_list):
something = get_something(idx)
new_dict.setdefault(val, []).append(something)

LDAP Python query and converting the appropriate dictionary

Sorry, I couldn't think of a better title for my question.
So I'm a starter at Python and I really trying to learn how to use it. My current problem deals with creating a simple way to reduce the results received from a python query.
If I understand what I'm dealing with, the LDAP query returns a List of a List of Dictionary where value in the Dictionary is a List. That's a lotta stuff there to traverse through so I figured there has to be a nice magical python way to convert this to a List of a Dictionary where the value of the Dictionary is just a simple string.
Currently, my code just simply to get the List of Dictionary but I still have the Dictionary values as a List themselves
for item in data:
results.append(item[1])
Once again, I'm a beginner so I don't really understand what to do from there. I'm also using Django if that's gonna help anyone understand my plight.
Edit (added data example):
The Structure is kinda like this:
data[index][1] = {'uid': ['restest'], 'mail': [''], 'givenName': ['Research'], 'cn': ['Research Test Account'], 'sn': ['Account']}
I'd like it to be instead of 'givenName': ['Research'] to be 'givenName': 'Reaseach'

Haha I'm not sure you ever want such a complicated data structure around. Consider refactoring your code to make your data structures easier to understand. Read: object-oriented programming, classes, functional programming principles.
Here's the answer if you just want to do it for a single dictionary:
data = {k:v[0] for (k,v) in data.items()}
Here it is in action:
>>> data = {'uid': ['restest'], 'mail': [''], 'givenName': ['Research'], 'cn': ['Research Test Account'], 'sn': ['Account']}
>>> data
{'mail': [''], 'sn': ['Account'], 'givenName': ['Research'], 'uid': ['restest'], 'cn': ['Research Test Account']}
>>> data = {k:v[0] for (k,v) in data.items()}
>>> data
{'mail': '', 'givenName': 'Research', 'cn': 'Research Test Account', 'sn': 'Account', 'uid': 'restest'}
All you're doing is re-mapping your dictionary to the first item of each list. If you want to iterate through all the levels of your structure and do this, just nest the above inside some list comprehensions:
[[[{k:v[0] for (k,v) in change_dict.items()} for change_dict in list_of_dicts]
for list_of_dicts in list_of_lists]
for list_of_lists in mydata]
It's not so bad to nest so many list comprehensions when you're only doing something to a "leaf" element in your data, but this is gonna get really messy if you try to manipulate your data structures at every level. See my comment at the beginning.