Develop Reference

Extracting information of JSON file

I got results through results = requests.get(url).json()
results look like this:
{'type': 'FeatureCollection', 'crs': {'type': 'name', 'properties':
{'name': 'EPSG:4326'}}, 'features': [{'type': 'Feature',
'properties': {'kode': '0101',
'navn': 'København',
'region_kode': '1084.0',
'region_navn': 'Hovedstaden'}, 'bbox': [12.453042062098154,
55.612994971371606,
12.734252598475942,
55.732491190632494]}]}
with results['features'], I am getting this
[{'type': 'Feature', 'properties': {'kode': '0101', 'navn':
'København', 'region_kode': '1084.0', 'region_navn':
'Hovedstaden'}, 'bbox': [12.453042062098154,
55.612994971371606,
12.734252598475942,
55.732491190632494]}]
I want to get the information in navn
and I tried all combination of
results['features']['properties']['navn']
results['features']['navn']
results['features']['properties']
they all show the same error message that: list indices must be integers or slices, not str
apparently, results['features'] is a list with a length of 1.
how can I get to navn information?
I want to make several calls as you can imagine.

The results['features']object is a list, try results['features'][0]['properties']['navn']
Now you select the first element in the list (0), the dictionary, and from that dictionary you select the 'navn' key. The result is the value of 'navn'
Note that python lists are between [] and items are seperated by a comma and python dictionaries are between {} and consists of key, value pairs seperated by a comma.

try this
results['features'][0]['properties']['navn']

You can try code below:
results['features'][0]['properties']['navn']

You should try accessing the first element of the list in result['features'], i.e.:
results['features'][0]['properties']['navn']
Full code:
results = {'type': 'FeatureCollection', 'crs': {'type': 'name', 'properties': {'name': 'EPSG:4326'}}, 'features': [{'type': 'Feature',
'properties': {'kode': '0101', 'navn': 'København', 'region_kode': '1084.0', 'region_navn': 'Hovedstaden'}, 'bbox': [12.453042062098154, 55.612994971371606, 12.734252598475942, 55.732491190632494]}]}
print(results['features'][0]['properties']['navn'])
# København

results = {'type': 'FeatureCollection', 'crs': {'type': 'name', 'properties': {'name': 'EPSG:4326'}},
'features': [{'type': 'Feature',
'properties': {'kode': '0101', 'navn': 'København', 'region_kode': '1084.0',
'region_navn': 'Hovedstaden'},
'bbox': [12.453042062098154, 55.612994971371606, 12.734252598475942, 55.732491190632494]}]}
navn = results['features'][0]['properties']['navn']
print(navn)
You got error because inside features there is one list. So, you can not get list with the help of str index and to get the properties inside the features you need to write [0] and the list will gone and you can get the value.

Converting Dictionary Values to new Dictionary

I am pulling json data from an API and have a number of columns in my dataframe that contain dictionaries. These dictionaries are written so that the id and the value are two separate entries in the dictionary like so:
{'id': 'AnnualUsage', 'value': '13071'}
Some of the rows for these columns contain only one dictionary entry like shown above, but others can contain up to 7:
[{'id': 'AnnualUsage', 'value': '13071'},
{'id': 'TestId', 'value': 'Z13753'},
{'id': 'NumberOfMe', 'value': '3'},
{'id': 'Prem', 'value': '960002'},
{'id': 'ProjectID', 'value': '0039'},
{'id': 'Region', 'value': 'CHR'},
{'id': 'Tariff', 'value': 'Multiple'},
{'id': 'Number', 'value': '06860702'}]
When I attempt to break this dictionary down into separate column attributes
CTG_df2 = pd.concat([CTG_df['id'], CTG_df['applicationUDFs'].apply(pd.Series)], axis=1)
I end up with columns in a dataframe each containing a dictionary of the above entry i.e.
{'id': 'AnnualUsageDE', 'value': '13071'}
Is there a way for me to convert my dictionary values into new key-value pairs? For instance I would like to convert from:
{'id': 'AnnualUsageDE', 'value': '13071'}
to
{'AnnualUsageDE': '13071'}
If this is possible I will then be able to create new columns from these attributes.

You can do a dict comprehension. From your list of dicts, compose a new dict where the key is the id of each element and the value is the value of each element.
original = [{'id': 'AnnualUsage', 'value': '13071'},
{'id': 'TestId', 'value': 'Z13753'},
{'id': 'NumberOfMe', 'value': '3'},
{'id': 'Prem', 'value': '960002'},
{'id': 'ProjectID', 'value': '0039'},
{'id': 'Region', 'value': 'CHR'},
{'id': 'Tariff', 'value': 'Multiple'},
{'id': 'Number', 'value': '06860702'}]
newdict = {subdict['id']: subdict['value'] for subdict in original}
print(newdict)
# {'AnnualUsage': '13071',
# 'Number': '06860702',
# 'NumberOfMe': '3',
# 'Prem': '960002',
# 'ProjectID': '0039',
# 'Region': 'CHR',
# 'Tariff': 'Multiple',
# 'TestId': 'Z13753'}

You can iterate through the values and set each of them to the dictionary value:
newdict = dict()
for x in original:
newdict[x["id"]] = x["value"]

Use list of indices to manipulate a nested dictionary

I'm trying to perform operations on a nested dictionary (data retrieved from a yaml file):
data = {'services': {'web': {'name': 'x'}}, 'networks': {'prod': 'value'}}
I'm trying to modify the above using the inputs like:
{'services.web.name': 'new'}
I converted the above to a list of indices ['services', 'web', 'name']. But I'm not able to/not sure how to perform the below operation in a loop:
data['services']['web']['name'] = new
That way I can modify dict the data. There are other values I plan to change in the above dictionary (it is extensive one) so I need a solution that works in cases where I have to change, EG:
data['services2']['web2']['networks']['local'].
Is there a easy way to do this? Any help is appreciated.

You may iterate over the keys while moving a reference:
data = {'networks': {'prod': 'value'}, 'services': {'web': {'name': 'x'}}}
modification = {'services.web.name': 'new'}
for key, value in modification.items():
keyparts = key.split('.')
to_modify = data
for keypart in keyparts[:-1]:
to_modify = to_modify[keypart]
to_modify[keyparts[-1]] = value
print(data)
Giving:
{'networks': {'prod': 'value'}, 'services': {'web': {'name': 'new'}}}

Python jsonpath Filter Expression

Background:
I have the following example data structure in JSON:
{'sensor' : [
{'assertions_enabled': 'ucr+',
'deassertions_enabled': 'ucr+',
'entity_id': '7.0',
'lower_critical': 'na',
'lower_non_critical': 'na',
'lower_non_recoverable': 'na',
'reading_type': 'analog',
'sensor_id': 'SR5680 TEMP (0x5d)',
'sensor_reading': {'confidence_interval': '0.500',
'units': 'degrees C',
'value': '42'},
'sensor_type': 'Temperature',
'status': 'ok',
'upper_critical': '59.000',
'upper_non_critical': 'na',
'upper_non_recoverable': 'na'}
]}
The sensor list will actually contain many of these dicts containing sensor info.
Problem:
I'm trying to query the list using jsonpath to return me a subset of sensor dicts that have sensor_type=='Temperature' but I'm getting 'False' returned (no match). Here's my jsonpath expression:
results = jsonpath.jsonpath(ipmi_node, "$.sensor[?(#.['sensor_type']=='Temperature')]")
When I remove the filter expression and just use "$.sensor.*" I get a list of all sensors, so I'm sure the problem is in the filter expression.
I've scanned multiple sites/posts for examples and I can't seem to find anything specific to Python (Javascript and PHP seem to be more prominent). Could anyone offer some guidance please?

The following expression does what you need (notice how the attribute is specified):
jsonpath.jsonpath(impi_node, "$.sensor[?(#.sensor_type=='Temperature')]")

I am using jsonpath-ng which seems to be active (as of 23.11.20) and I provide solution based on to Pedro's jsonpath expression:
data = {
'sensor' : [
{'sensor_type': 'Temperature', 'id': '1'},
{'sensor_type': 'Humidity' , 'id': '2'},
{'sensor_type': 'Temperature', 'id': '3'},
{'sensor_type': 'Density' , 'id': '4'}
]}
from jsonpath_ng.ext import parser
for match in parser.parse("$.sensor[?(#.sensor_type=='Temperature')]").find(data):
print(match.value)
Output:
{'sensor_type': 'Temperature', 'id': '1'}
{'sensor_type': 'Temperature', 'id': '3'}
NOTE: besides basic documentation provided on project's homepage I found additional information in tests.

Numeric sort of list of dictionary objects

I am very new to python programming and have yet to buy a textbook on the matter (I am buying one from the store or Amazon today). In the meantime, can you help me with the following problem I have encountered?
I have an list of dictionary objects like this:
stock = [
{ 'date': '2012', 'amount': '1.45', 'type': 'one'},
{ 'date': '2012', 'amount': '1.4', 'type': 'two'},
{ 'date': '2011', 'amount': '1.35', 'type': 'three'},
{ 'date': '2012', 'amount': '1.35', 'type': 'four'}
]
I would like to sort the list by the amount date column and then by the amount column so that the sorted list looks like this:
stock = [
{ 'date': '2011', 'amount': '1.35', 'type': 'three'},
{ 'date': '2012', 'amount': '1.35', 'type': 'four'},
{ 'date': '2012', 'amount': '1.4', 'type': 'two'},
{ 'date': '2012', 'amount': '1.45', 'type': 'one'}
]
I now think I need to use sorted() but as a beginner I am having difficulties understanding to concepts I see.
I tried this:
from operator import itemgetter
all_amounts = itemgetter("amount")
stock.sort(key = all_amounts)
but this resulted in an list that was sorted alphanumerically rather than numerically.
Can someone please tell me how to achieve this seemingly simple sort? Thank-you!

Your sorting condition is too complicated for an operator.itemgetter. You will have to use a lambda function:
stock.sort(key=lambda x: (int(x['date']), float(x['amount'])))
or
all_amounts = lambda x: (int(x['date']), float(x['amount']))
stock.sort(key=all_amounts)

Start by converting your data into a proper format:
stock = [
{ 'date': int(x['date']), 'amount': float(x['amount']), 'type': x['type']}
for x in stock
]
Now stock.sort(key=all_amounts) will return correct results.
As you appear to be new in programming, here's a word of general advice if I may:
Proper data structure is 90 percent of success. Do not try to work around broken data by writing more code. Create a structure adequate to your task and write as less code as possible.

You can also use the fact that python's sort is stable:
stock.sort(key=lambda x: int(x["amount"]))
stock.sort(key=lambda x: int(x["date"]))
Since the items with the same key keep their relative positions when sorting (they're never swapped), you can build up a complicated sort by sorting multiple times.