Python value assigned to incorrect dict key

Python value assigned to incorrect dict key - python

I am iterating through a csv and for each column, determining the longest len of a string, and updating a dict as necessary.
If I do this
def get_max_size(current, cell_value):
if cell_value:
current = max(current, len(cell_value))
return current
def my_function():
headers = ["val1","val2","val3","val4","val5"]
d = {header: {'max_size': 0, 'other': {'test': None}} for header in headers}
csv_file = [
["abc","123","HAMILTON","1950.00","17-SEP-2015"],
["ab","321","GLASGOW","711.00","13-NOV-2015"]
]
for row in csv_file:
for i, header in enumerate(headers):
max_size = get_max_size(d[header]['max_size'], row[i])
d[header]['max_size'] = max_size
print(d)
I get the expected output:
{'val1': {'max_size': 3, 'other': {'test': None}},
'val2': {'max_size': 3, 'other': {'test': None}},
'val3': {'max_size': 8, 'other': {'test': None}},
'val4': {'max_size': 7, 'other': {'test': None}},
'val5': {'max_size': 11, 'other': {'test': None}}}
However if I modify my code as such:
REQUIRED_VALUES = {
'max_size': 0,
'allowed_values': {'digit': None, 'alpha': None, 'whitespace': None, 'symbol': None},
'max_value': None,
'allow_null': None,
}
def my_function():
headers = ["val1","val2","val3","val4","val5"]
# d = {header: {'max_size': 0, 'other': {'test': None}} for header in headers}
d = {header: REQUIRED_VALUES for header in headers}
csv_file = [
["abc","123","HAMILTON","1950.00","17-SEP-2015"],
["ab","321","GLASGOW","711.00","13-NOV-2015"]
]
for row in csv_file:
for i, header in enumerate(headers):
max_size = get_max_size(d[header]['max_size'], row[i])
d[header]['max_size'] = max_size
print(d)
Then the largest len of all keys (val5, the date field, of len == 11), is assigned to all max_length:
{'val1': {'max_size': 11, 'allowed_values': {'digit': None, 'alpha': None, 'whitespace': None, 'symbol': None}, 'max_value': None, 'allow_null': None},
'val2': {'max_size': 11, 'allowed_values': {'digit': None, 'alpha': None, 'whitespace': None, 'symbol': None}, 'max_value': None, 'allow_null': None},
'val3': {'max_size': 11, 'allowed_values': {'digit': None, 'alpha': None, 'whitespace': None, 'symbol': None}, 'max_value': None, 'allow_null': None},
'val4': {'max_size': 11, 'allowed_values': {'digit': None, 'alpha': None, 'whitespace': None, 'symbol': None}, 'max_value': None, 'allow_null': None},
'val5': {'max_size': 11, 'allowed_values': {'digit': None, 'alpha': None, 'whitespace': None, 'symbol': None}, 'max_value': None, 'allow_null': None}}
Is there some difference between the dicts that I'm missing? The dict is the only thing that changes, they both contain nested dictionaries... apart from number of items, I can't really see the difference.

Related

How to load list columns into a dataframe?

I try to load "columns" from a python list object into a dataframe.
This is my list object:
list = type(api_response.results) -> <class 'list'>
These are the values from the list object (I think this is a json structur):
{'results': [{'data': [{'interval': '2022-11-11T10:00:00.000Z/2022-11-11T10:30:00.000Z',
'metrics': [{'metric': 'nError',
'qualifier': None,
'stats': {'count': 4,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}},
{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 113,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None}],
'group': {'mediaType': 'voice'}}]}
I just need this result:
Dataframe:
interval metric count
0 2022-11-11T10:00:00.000Z/2022-11-11T10:30:00.000Z nError 4
1 2022-11-11T10:00:00.000Z/2022-11-11T10:30:00.000Z nOffered 113
How get this result? How is it possibly to call intervals or metrics from the list object?
Thanks for any help

you can use:
def get_metric(x):
check=0
vals=[]
for i in range(0,len(x)):
if len(x)==1:
check=1
for j in range(0,len(x) + check):
print(i,j)
vals.append(x[i]['data'][0]['metrics'][j]['metric'])
return vals
def get_count(x):
vals=[]
for i in range(0,len(x)):
for j in range(0,len(x[0])):
vals.append(x[i]['data'][0]['metrics'][j]['stats']['count'])
return vals
df['interval']=df['results'].apply(lambda x: [x[0]['data'][i]['interval'] for i in range(0,len(x[0]['data']))])
df['metric']= df['results'].apply(lambda x: get_metric(x))
df['count']= df['results'].apply(lambda x: get_count(x))
df=df.drop(['results'],axis=1)
df=df.explode(['metric','count']).explode('interval')
print(df)
'''
interval metric count
0 2022-11-11T10:00:00.000Z/2022-11-11T10:30:00.000Z nError 4
0 2022-11-11T10:00:00.000Z/2022-11-11T10:30:00.000Z nOffered 113
'''

How to iterate over interval in json file and create a dataframe?

I am iterating over json file and creating dataframe with the desirable columns. I already implemented the code but now json file has little bit changed. But I am not able to think where to change the code to get the required output.
Explanation:
previous json result:
queryResult: {'results': [{'data': [{'interval': '2021-10-11T11:46:25.000Z/2021-10-18T11:49:48.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 7,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}},
{'metric': 'nTransferred',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None}],
'group': {'mediaType': 'voice',
'queueId': '73643cff-799b-41ae-9a67-efcf5e593155'}}]}
previous dataframe:
Queue_Id,Interval Start,Interval End,nOffered_count,nOffered_sum,nOffered.denominator,nOffered.numerator,nTransferred_count,nTransferred_sum,nTransferred.denominator,nTransferred.numerator
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-11T11:46:25.000Z,2021-10-18T11:49:48.000Z,7,,,,1.0,,,
new json result:
queryResult: {'results': [{'data': [{'interval': '2021-10-11T11:46:25.000Z/2021-10-12T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-13T11:46:25.000Z/2021-10-14T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 2,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}},
{'metric': 'nTransferred',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-14T11:46:25.000Z/2021-10-15T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 3,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-15T11:46:25.000Z/2021-10-16T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None}],
'group': {'mediaType': 'voice',
'queueId': '73643cff-799b-41ae-9a67-efcf5e593155'}}]}
Now desirable dataframe:
Queue_Id,Interval Start,Interval End,nOffered_count,nOffered_sum,nOffered.denominator,nOffered.numerator,nTransferred_count,nTransferred_sum,nTransferred.denominator,nTransferred.numerator
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-11T11:46:25.000Z,2021-10-12T11:46:25.000Z,1,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-13T11:46:25.000Z,2021-10-14T11:46:25.000Z,2,,,,1,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-14T11:46:25.000Z,2021-10-15T11:46:25.000Z,3,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-15T11:46:25.000Z,2021-10-16T11:46:25.000Z,1,,,,,,,
What are the changes I need to do to in below code to get the new result.
column_names = []
if(query_result.results != None):
for item in query_result.results:
data_lst = []
for lst_data in item.data:
print("####################################")
print(lst_data)
print("####################################")
for met in lst_data.metrics:
metric_name = met.metric
column_names.append('Queue_Id')
column_names.append(metric_name+'_count')
column_names.append(metric_name+'_sum')
column_names.append(metric_name+'.denominator')
column_names.append(metric_name+'.numerator')
column_names.append('Interval Start')
column_names.append('Interval End')
data_lst.append(queue_id)
data_lst.append(met.stats.count)
data_lst.append(met.stats.sum)
data_lst.append(met.stats.denominator)
data_lst.append(met.stats.numerator)
data_lst.append(lst_data.interval.split('/')[0])
data_lst.append(lst_data.interval.split('/')[1])
print(data_lst)
else:
data_lst = []
metric_name = query.metrics[0]
column_names.append('Queue_Id')
column_names.append(metric_name+'_count')
column_names.append(metric_name+'_sum')
column_names.append(metric_name+'.denominator')
column_names.append(metric_name+'.numerator')
column_names.append('Interval Start')
column_names.append('Interval End')
data_lst.append(queue_id)
data_lst.append('')
data_lst.append('')
data_lst.append('')
data_lst.append('')
data_lst.append(query.interval.split('/')[0])
data_lst.append(query.interval.split('/')[1])
print("data_lst", data_lst)
print("column_names", column_names)
return data_lst, column_names

I have modified my code little bit and got the result. The below code is working for me-
lst_of_metrics = ["nOffered", "nTransferred"]
out = defaultdict(list)
if(query_result.results != None):
for item in query_result.results:
#data_lst = []
for lst_data in item.data:
print("####################################")
print(lst_data)
print("####################################")
out['queue_id'].append(queue_id)
for met1, met in itertools.zip_longest(query.metrics, lst_data.metrics):
#for met in lst_data.metrics:
if(met):
if(met.metric == met1):
out[met.metric+"_count"].append(met.stats.count)
out[met.metric+"_sum"].append(met.stats.sum)
out[met.metric+".denominator"].append(met.stats.denominator)
out[met.metric+".numerator"].append(met.stats.numerator)
else:
out[met1+"_count"].append('')
out[met1+"_sum"].append('')
out[met1+".denominator"].append('')
out[met1+".numerator"].append('')
else:
out[met1+"_count"].append('')
out[met1+"_sum"].append('')
out[met1+".denominator"].append('')
out[met1+".numerator"].append('')
interval = lst_data.interval.split('/')
out['Interval Start'].append(interval[0])
out['Interval End'].append(interval[1])
print("out", out)
else:
metric_name = query.metrics[0]
out['queue_id'].append(queue_id)
out[metric_name+"_count"].append('')
out[metric_name+"_sum"].append('')
out[metric_name+".denominator"].append('')
out[metric_name+".numerator"].append('')
interval = query.interval.split('/')
out['Interval Start'].append(interval[0])
out['Interval End'].append(interval[1])
print(out)
df = pd.DataFrame(out)
print (df)

Iterate over json result and get the desirable data in pandas dataframe

I have a json result which I am trying to convert into dataframe but not able to get the correct result. Actually for some cases it is giving correct but for some case it is failing.
Example:
Based on metric API is generating result for specified interval. But this is not certain for that particular interval metric have output or not. And process is running 4 different queue_id.
suppose process is running only for 2 metric. ['nOffered', 'nTransferred']
queue_id = 'a72dba75-0bc6-4a65-b120-8803364f8dc3'
for this queue_id, nOffered is having some values but nTransferred doesn't have. Json result is given below-
queryResult: {'results': [{'data': [{'interval': '2021-10-11T11:46:25.000Z/2021-10-12T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-13T11:46:25.000Z/2021-10-14T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 2,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-14T11:46:25.000Z/2021-10-15T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 3,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-15T11:46:25.000Z/2021-10-16T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None}],
'group': {'mediaType': 'voice',
'queueId': '73643cff-799b-41ae-9a67-efcf5e593155'}}]}
My code is giving below output-
queue_id nOffered_count nOffered_sum interval_start interval_end
0 a72dba75-0bc6-4a65-b120-8803364f8dc3 6 None 2021-10-11T11:46:25.000Z 2021-10-12T11:46:25.000Z
1 a72dba75-0bc6-4a65-b120-8803364f8dc3 1 None 2021-10-12T11:46:25.000Z 2021-10-13T11:46:25.000Z
2 a72dba75-0bc6-4a65-b120-8803364f8dc3 12 None 2021-10-13T11:46:25.000Z 2021-10-14T11:46:25.000Z
3 a72dba75-0bc6-4a65-b120-8803364f8dc3 6 None 2021-10-14T11:46:25.000Z 2021-10-15T11:46:25.000Z
4 a72dba75-0bc6-4a65-b120-8803364f8dc3 6 None 2021-10-15T11:46:25.000Z 2021-10-16T11:46:25.000Z
But when process is running for 2nd queue_id that time it is not working-
queue_id - 73643cff-799b-41ae-9a67-efcf5e593155
json output for this queue_id -
queryResult: {'results': [{'data': [{'interval': '2021-10-11T11:46:25.000Z/2021-10-12T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-13T11:46:25.000Z/2021-10-14T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 2,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}},
{'metric': 'nTransferred',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-14T11:46:25.000Z/2021-10-15T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 3,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None},
{'interval': '2021-10-15T11:46:25.000Z/2021-10-16T11:46:25.000Z',
'metrics': [{'metric': 'nOffered',
'qualifier': None,
'stats': {'count': 1,
'count_negative': None,
'count_positive': None,
'current': None,
'denominator': None,
'max': None,
'min': None,
'numerator': None,
'ratio': None,
'sum': None,
'target': None}}],
'views': None}],
'group': {'mediaType': 'voice',
'queueId': '73643cff-799b-41ae-9a67-efcf5e593155'}}]}
This time both metric having some data. So output would be-
Queue_Id,Interval Start,Interval End,nOffered_count,nOffered_sum,nOffered.denominator,nOffered.numerator,nTransferred_count,nTransferred_sum,nTransferred.denominator,nTransferred.numerator
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-11T11:46:25.000Z,2021-10-12T11:46:25.000Z,1,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-13T11:46:25.000Z,2021-10-14T11:46:25.000Z,2,,,,1,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-14T11:46:25.000Z,2021-10-15T11:46:25.000Z,3,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-15T11:46:25.000Z,2021-10-16T11:46:25.000Z,1,,,,,,,
And in final result, both the result merge and give the output with all columns and data.
Queue_Id,Interval Start,Interval End,nOffered_count,nOffered_sum,nOffered.denominator,nOffered.numerator,nTransferred_count,nTransferred_sum,nTransferred.denominator,nTransferred.numerator
a72dba75-0bc6-4a65-b120-8803364f8dc3,2021-10-11T11:46:25.000Z,2021-10-12T11:46:25.000Z,6,,,,,,,
a72dba75-0bc6-4a65-b120-8803364f8dc3,2021-10-12T11:46:25.000Z,2021-10-13T11:46:25.000Z,1.0,,,,,,,
a72dba75-0bc6-4a65-b120-8803364f8dc3,2021-10-13T11:46:25.000Z,2021-10-14T11:46:25.000Z,12.0,,,,,,,
a72dba75-0bc6-4a65-b120-8803364f8dc3,2021-10-14T11:46:25.000Z,2021-10-15T11:46:25.000Z,6.0,,,,,,,
a72dba75-0bc6-4a65-b120-8803364f8dc3,2021-10-15T11:46:25.000Z,2021-10-16T11:46:25.000Z,6.0,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-11T11:46:25.000Z,2021-10-12T11:46:25.000Z,1,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-13T11:46:25.000Z,2021-10-14T11:46:25.000Z,2,,,,1.0,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-14T11:46:25.000Z,2021-10-15T11:46:25.000Z,3,,,,,,,
73643cff-799b-41ae-9a67-efcf5e593155,2021-10-15T11:46:25.000Z,2021-10-16T11:46:25.000Z,1,,,,,,,
Currently I am running below logic-
out = defaultdict(list)
if(query_result.results != None):
for item in query_result.results:
#data_lst = []
for lst_data in item.data:
print("####################################")
print(lst_data)
print("####################################")
out['queue_id'].append(queue_id)
for met in lst_data.metrics:
out[met.metric+"_count"].append(met.stats.count)
out[met.metric+"_sum"].append(met.stats.sum)
out[met.metric+".denominator"].append(met.stats.denominator)
out[met.metric+".numerator"].append(met.stats.numerator)
interval = lst_data.interval.split('/')
out['Interval Start'].append(interval[0])
out['Interval End'].append(interval[1])
print("out", out)
else:
metric_name = query.metrics[0]
out['queue_id'].append(queue_id)
out[metric_name+"_count"].append('')
out[metric_name+"_sum"].append('')
out[metric_name+".denominator"].append('')
out[metric_name+".numerator"].append('')
interval = query.interval.split('/')
out['Interval Start'].append(interval[0])
out['Interval End'].append(interval[1])
print(out)
df = pd.DataFrame(out)
print (df)
return df

I used below logic to get the desirable result. It is working for me.
lst_of_metrics = ["nOffered", "nTransferred"]
out = defaultdict(list)
if(query_result.results != None):
for item in query_result.results:
#data_lst = []
for lst_data in item.data:
print("####################################")
print(lst_data)
print("####################################")
out['queue_id'].append(queue_id)
for met1, met in itertools.zip_longest(query.metrics, lst_data.metrics):
if(met):
if(met.metric == met1):
out[met.metric+"_count"].append(met.stats.count)
out[met.metric+"_sum"].append(met.stats.sum)
out[met.metric+".denominator"].append(met.stats.denominator)
out[met.metric+".numerator"].append(met.stats.numerator)
else:
out[met1+"_count"].append('')
out[met1+"_sum"].append('')
out[met1+".denominator"].append('')
out[met1+".numerator"].append('')
else:
out[met1+"_count"].append('')
out[met1+"_sum"].append('')
out[met1+".denominator"].append('')
out[met1+".numerator"].append('')
interval = lst_data.interval.split('/')
out['Interval Start'].append(interval[0])
out['Interval End'].append(interval[1])
print("out", out)
else:
metric_name = query.metrics[0]
out['queue_id'].append(queue_id)
out[metric_name+"_count"].append('')
out[metric_name+"_sum"].append('')
out[metric_name+".denominator"].append('')
out[metric_name+".numerator"].append('')
interval = query.interval.split('/')
out['Interval Start'].append(interval[0])
out['Interval End'].append(interval[1])
print(out)
df = pd.DataFrame(out)
print (df)

Parse/print out value from JSON content in Python

I am trying to print out the price of an item from this page/JSON:
https://shopee.sg/api/v2/item/get?itemid=2590867516&shopid=165420215
but I am encountering an error as below:
print([d.get('price_max_before_discount') for d in site_json['item'] if d.get('price_max_before_discount')])
AttributeError: 'str' object has no attribute 'get'
import json
from urllib import request
from bs4 import BeautifulSoup
url = 'https://shopee.sg/api/v2/item/get?itemid=2590867516&shopid=165420215'
html = request.urlopen(url).read()
soup = BeautifulSoup(html, 'html.parser')
site_json = json.loads(soup.text)
print([d.get('price_max_before_discount') for d in site_json['item'] if d.get('price_max_before_discount')])
not sure what I am doing wrong but I will appreciate any advice/solutions!
Thank you.

The API url is returning the json object and you can simply use:
import json
from urllib import request
url = 'https://shopee.sg/api/v2/item/get?itemid=2590867516&shopid=165420215'
data = request.urlopen(url).read()
data_json = json.loads(data)
print(data_json)
Output:
{'item': {'itemid': 2590867516, 'price_max_before_discount': 54900000, 'item_status': 'normal', 'can_use_wholesale': False, 'show_free_shipping': True, 'estimated_days': 2, 'is_hot_sales': None, 'is_slash_price_item': False, 'upcoming_flash_sale': None, 'slash_lowest_price': None, 'is_partial_fulfilled': False, 'condition': 1, 'show_original_guarantee': True, 'add_on_deal_info': None, 'is_non_cc_installment_payment_eligible': False, 'categories': [{'display_name': 'Mobile & Gadgets', 'catid': 8, 'image': None, 'no_sub': False, 'is_default_subcat': False, 'block_buyer_platform': None}, {'display_name': 'Mobile Phones & Tablets', 'catid': 10941, 'image': None, 'no_sub': False, 'is_default_subcat': False, 'block_buyer_platform': None}, {'display_name': 'Samsung', 'catid': 10944, 'image': None, 'no_sub': True, 'is_default_subcat': False, 'block_buyer_platform': None}], 'ctime': 1564663070, 'name': "(New Launch) Samsung Galaxy Tab S6 Lite 10.4' LTE Version with S Pen 64GB", 'show_shopee_verified_label': False, 'size_chart': None, 'is_pre_order': False, 'service_by_shopee_flag': None, 'historical_sold': 186, 'reference_item_id': '', 'recommendation_info': None, 'bundle_deal_info': None, 'price_max': 46500000, 'has_lowest_price_guarantee': False, 'shipping_icon_type': 0, 'images': ['68ce24874257ec3c593d0d118e9cf785'], 'price_before_discount': 54900000, 'cod_flag': 0, 'catid': 8, 'is_official_shop': False, 'coin_earn_label': None, 'hashtag_list': None, 'sold': 62, 'makeup': None, 'item_rating': {'rating_star': 4.976744, 'rating_count': [88, 0, 2, 0, 2, 84], 'rcount_with_image': 40, 'rcount_with_context': 58}, 'show_official_shop_label_in_title': False, 'discount': '15%', 'reason': None, 'label_ids': [1000012, 1000035, 1000088, 1001261], 'has_group_buy_stock': False, 'other_stock': 0, 'deep_discount': None, 'attributes': [{'is_pending_qc': False, 'idx': 0, 'value': 'Samsung', 'id': 10054, 'is_timestamp': False, 'name': 'Brand'}, {'is_pending_qc': False, 'idx': 1, 'value': 'Galaxy Tab S6', 'id': 10093, 'is_timestamp': False, 'name': 'Model'}, {'is_pending_qc': True, 'idx': 2, 'value': '64GB', 'id': 10091, 'is_timestamp': False, 'name': 'Built-in Storage'}, {'is_pending_qc': True, 'idx': 3, 'value': '', 'id': 10094, 'is_timestamp': False, 'name': 'RAM'}, {'is_pending_qc': True, 'idx': 4, 'value': '1 Month', 'id': 10095, 'is_timestamp': False, 'name': 'Warranty Period'}], 'badge_icon_type': 0, 'liked': False, 'cmt_count': 88, 'image': '68ce24874257ec3c593d0d118e9cf785', 'recommendation_algorithm': None, 'is_cc_installment_payment_eligible': True, 'shopid': 165420215, 'normal_stock': 8, 'video_info_list': [], 'installment_plans': [{'banks': [{'bank_name': 'UOB', 'sub_options': [{'disabled_reason': None, 'data': {'bank_name': 'UOB', 'down_payment': 0, 'name': '6x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 1641, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 7750000, 'tenure': 6, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [label_bank_uob #6x]', 'option_info': '1000501-19005014'}, {'disabled_reason': None, 'data': {'bank_name': 'UOB', 'down_payment': 0, 'name': '12x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 1641, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 3875000, 'tenure': 12, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [label_bank_uob #12x]', 'option_info': '1000501-19005015'}], 'bank_id': 1641, 'bank_logo': 'https://mall.shopee.sg/static/images/bank_logo/ic_bank_uob.png'}, {'bank_name': 'DBS/POSB', 'sub_options': [{'disabled_reason': None, 'data': {'bank_name': 'DBS/POSB', 'down_payment': 0, 'name': '6x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 4, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 7750000, 'tenure': 6, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [label_bank_dbs_posb #6x]', 'option_info': '1000501-19005011'}, {'disabled_reason': None, 'data': {'bank_name': 'DBS/POSB', 'down_payment': 0, 'name': '12x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 4, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 3875000, 'tenure': 12, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [label_bank_dbs_posb #12x]', 'option_info': '1000501-19005012'}], 'bank_id': 4, 'bank_logo': 'https://mall.shopee.sg/static/images/bank_logo/ic_bank_dbs.png'}, {'bank_name': 'AMERICAN EXPRESS', 'sub_options': [{'disabled_reason': None, 'data': {'bank_name': 'AMERICAN EXPRESS', 'down_payment': 0, 'name': '6x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 437, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 7750000, 'tenure': 6, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [AMERICAN EXPRESS #6x]', 'option_info': '1000501-19005020'}, {'disabled_reason': None, 'data': {'bank_name': 'AMERICAN EXPRESS', 'down_payment': 0, 'name': '12x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 437, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 3875000, 'tenure': 12, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [AMERICAN EXPRESS #12x]', 'option_info': '1000501-19005021'}], 'bank_id': 437, 'bank_logo': ''}, {'bank_name': 'OCBC', 'sub_options': [{'disabled_reason': None, 'data': {'bank_name': 'OCBC', 'down_payment': 0, 'name': '6x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 3430, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 7750000, 'tenure': 6, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [label_bank_ocbc #6x]', 'option_info': '1000501-19005017'}, {'disabled_reason': None, 'data': {'bank_name': 'OCBC', 'down_payment': 0, 'name': '12x # 0%', 'interest_rate': 0, 'option_id': None, 'bank_id': 3430, 'installment_amount': 46500000, 'channel_id': 1000501, 'monthly_installment': 3875000, 'tenure': 12, 'total_amount': 46500000}, 'name': 'Airpay CC Installment [label_bank_ocbc #12x]', 'option_info': '1000501-19005018'}], 'bank_id': 3430, 'bank_logo': 'https://mall.shopee.sg/static/images/bank_logo/ic_bank_ocbc.png'}], 'channel_name': 'label_cc_installment', 'is_cc': True, 'plans': None, 'channel_ic': 'ic_paymentoption_ccinstallment'}], 'view_count': 9765, 'voucher_info': None, 'current_promotion_has_reserve_stock': False, 'liked_count': 491, 'show_official_shop_label': False, 'price_min_before_discount': 54900000, 'show_discount': 15, 'preview_info': None, 'flag': 524290, 'exclusive_price_info': None, 'current_promotion_reserved_stock': 0, 'wholesale_tier_list': [], 'group_buy_info': None, 'shopee_verified': False, 'item_has_post': False, 'hidden_price_display': None, 'transparent_background_image': '68ce24874257ec3c593d0d118e9cf785', 'welcome_package_info': None, 'discount_stock': 8, 'coin_info': {'spend_cash_unit': 1000, 'coin_earn_items': []}, 'is_adult': False, 'currency': 'SGD', 'raw_discount': 15, 'is_preferred_plus_seller': False, 'is_category_failed': False, 'price_min': 46500000, 'can_use_bundle_deal': False, 'cb_option': 0, 'brand': '', 'stock': 8, 'status': 1, 'bundle_deal_id': 0, 'is_group_buy_item': None, 'description': "Brand New Set\r\n\r\nGalaxy Tab S6 Lite is your super portable note-taking, go-getting companion. It comes with a large 10.4 inch display on a slim and light build, One UI 2 on Android, and S Pen in-box and ready to go. Whether you're drawing, learning or gaming, this is the tablet made to be in the moment.\r\n\r\nGalaxy Tab S6 Lite is slim and lightweight thanks to its seamless, metal unibody. It slips easily into a small tote for true portability and minimalist style. \r\n\r\nKeeping up with lectures is a breeze with S Pen. When taking notes, you can jot first and change colors later. Quickly sort through memos when you save with searchable tags. Even write directly onto PDFs to cut the study clutter. When it's finally crunch time, you'll be organised and ready.\r\n\r\nS Pen is a bundle of writing instruments in one. Its natural grip, low latency and impressive pressure sensitivity will make it your go-to for everything from drawing to editing documents. And don't worry about misplacing the S Pen, thanks to the tablet's magnetic holder.\r\n\r\nDisplay : 10.4 inches\r\nExternal Memory Support Slot : Yes , MicroSD (Up to 1TB)\r\n64GB ROM + 4GB RAM\r\n8MP Rear Camera\r\n5MP Selfie Camera\r\n7040 mAh Big battery\r\n\r\nExport set with local seller warranty", 'flash_sale': None, 'models': [{'itemid': 2590867516, 'status': 1, 'current_promotion_reserved_stock': 0, 'name': 'Pink', 'promotionid': 58388, 'price': 46500000, 'price_stocks': [{'model_id': 5839808211, 'stockout_time': 1613452764, 'region': 'SG', 'rebate': 2000000, 'price': 46500000, 'promotion_type': 1, 'allocated_stock': 0, 'shop_id': 165420215, 'end_time': 1613663940, 'stock_breakdown_by_location': [], 'item_id': 2590867516, 'promotion_id': 58388, 'purchase_limit': 1, 'start_time': 1613452860, 'stock': 0}, {'model_id': 5839808211, 'stockout_time': 1612672967, 'region': 'SG', 'rebate': None, 'price': 54900000, 'promotion_type': 0, 'allocated_stock': None, 'shop_id': 165420215, 'end_time': None, 'stock_breakdown_by_location': [], 'item_id': 2590867516, 'promotion_id': 0, 'purchase_limit': None, 'start_time': None, 'stock': 0}], 'current_promotion_has_reserve_stock': False, 'currency': 'SGD', 'normal_stock': 0, 'extinfo': {'seller_promotion_limit': 1, 'has_shopee_promo': True, 'group_buy_info': None, 'holiday_mode_old_stock': None, 'tier_index': [0], 'seller_promotion_refresh_time': 1613663940}, 'price_before_discount': 54900000, 'modelid': 5839808211, 'sold': 12, 'stock': 0}, {'itemid': 2590867516, 'status': 1, 'current_promotion_reserved_stock': 0, 'name': 'Blue', 'promotionid': 58388, 'price': 46500000, 'price_stocks': [{'model_id': 51655477750, 'stockout_time': 1613452764, 'region': 'SG', 'rebate': 2000000, 'price': 46500000, 'promotion_type': 1, 'allocated_stock': 0, 'shop_id': 165420215, 'end_time': 1613663940, 'stock_breakdown_by_location': [], 'item_id': 2590867516, 'promotion_id': 58388, 'purchase_limit': 1, 'start_time': 1613452860, 'stock': 0}, {'model_id': 51655477750, 'stockout_time': 1612146775, 'region': 'SG', 'rebate': None, 'price': 54900000, 'promotion_type': 0, 'allocated_stock': None, 'shop_id': 165420215, 'end_time': None, 'stock_breakdown_by_location': [], 'item_id': 2590867516, 'promotion_id': 0, 'purchase_limit': None, 'start_time': None, 'stock': 0}], 'current_promotion_has_reserve_stock': False, 'currency': 'SGD', 'normal_stock': 0, 'extinfo': {'seller_promotion_limit': 1, 'has_shopee_promo': True, 'group_buy_info': None, 'holiday_mode_old_stock': None, 'tier_index': [2], 'seller_promotion_refresh_time': 1613663940}, 'price_before_discount': 54900000, 'modelid': 51655477750, 'sold': 65, 'stock': 0}, {'itemid': 2590867516, 'status': 1, 'current_promotion_reserved_stock': 0, 'name': 'Gray', 'promotionid': 58388, 'price': 46500000, 'price_stocks': [{'model_id': 70652051151, 'stockout_time': 1613452764, 'region': 'SG', 'rebate': 2000000, 'price': 46500000, 'promotion_type': 1, 'allocated_stock': 0, 'shop_id': 165420215, 'end_time': 1613663940, 'stock_breakdown_by_location': [], 'item_id': 2590867516, 'promotion_id': 58388, 'purchase_limit': 1, 'start_time': 1613452860, 'stock': 0}, {'model_id': 70652051151, 'stockout_time': 0, 'region': 'SG', 'rebate': None, 'price': 54900000, 'promotion_type': 0, 'allocated_stock': None, 'shop_id': 165420215, 'end_time': None, 'stock_breakdown_by_location': [], 'item_id': 2590867516, 'promotion_id': 0, 'purchase_limit': None, 'start_time': None, 'stock': 8}], 'current_promotion_has_reserve_stock': False, 'currency': 'SGD', 'normal_stock': 8, 'extinfo': {'seller_promotion_limit': 1, 'has_shopee_promo': True, 'group_buy_info': None, 'holiday_mode_old_stock': None, 'tier_index': [1], 'seller_promotion_refresh_time': 1613663940}, 'price_before_discount': 54900000, 'modelid': 70652051151, 'sold': 109, 'stock': 8}], 'has_low_fulfillment_rate': False, 'price': 46500000, 'shop_location': None, 'tier_variations': [{'images': ['dd6712ad6486c287be4f4c4ae82d5bb4', '228c0a8b72a457c99e420f980fbe6647', '028138dce1f06a573864ed58033267aa'], 'properties': [], 'type': 0, 'name': 'Colour', 'options': ['Pink', 'Gray', 'Blue']}], 'makeups': None, 'welcome_package_type': 0, 'show_official_shop_label_in_normal_position': None, 'item_type': 0}, 'version': '84fbe13733fdcb30d4c53c991d682692', 'data': None, 'error_msg': None, 'error': None}

You are iterating over dictionary item. Your iteration is making a dictionary get call on every element of item, that's is where the error is coming.
To get what you want, you don't have to iterate:
site_json['item'].get('price_max_before_discount')

The problem here is that at some point you are trying to access the value "price_max_before_discount" from a string, not a dictionary that includes that key. Thus, this error accurs AttributeError: 'str' object has no attribute 'get'.
This happens because you are iterating through the "item" dictionary. In each iteration you get a key. For example you get "itemid" (the key, not the value) and you try to get "price_max_before_discount" from it. But it's just a string!
The solution:
site_json['item']['price_max_before_discount']
Here you are getting the "item" json and from there getting the value of "price_max_before_discount"

Instantiating a nested dictionary

I'm trying to instantiate a nested dictionary which contains another dictionaries as keys and each of these dictionaries contains another dictionaries. I know what keys and how many keys will be in nested and nested-nested dictionaries, but I don't know how many and what keys will be in the upper dictionary (which will be OrderedDict and keys will be integers but I don't know how many).
The upper dictionary contains integers as keys and dictionaries as values - each of these dictionaries has 3 keys = 'forth','back' and 'price'.
'forth' and 'back' has another dictionaries as their values. Each of these dicts (values) contains these keys
'arr_date','arr_place','dep_date','dep_place'.
So for example 'forth' dict is:
dict.fromkeys(['arr_date','arr_place','dep_date','dep_place'],None)
So the point is that I want to instantiate the dictionary with these keys but the problem is that upper dictionary can has variable integers. It can contains these keys [1,2,3,4] but also can contains [1,2,3,4,5,6,7,8].
This is an example of instantiation of nested and nested-nested. So this would be a value of upper dictionary for each of it's keys (I'm not sure if condition will work).
dict.fromkeys(['forth','back','price'], dict.fromkeys(['arr_date','arr_place','dep_date','dep_place'],None) if key in ['forth','back'] else None)
The whole thing is that I want to tell the code as much as possible default values and keys.
Any advices?
EDIT: The condition 6 lines above does not work so anybody could tell how to do that too.
EDIT II: So the dict should looks like:
{1:{'forth':{'arr_date':'15-8-4','arr_place':'Atlanta','dep_date':'15-8-4','dep_place':'New York'},'back':{'arr_date...},'price':158},2:{....}}

Maybe something like this:
def inner_dict(vals = []):
my_vals = vals + [None]*(4 - len(vals))
my_keys = ['arr_date','arr_place','dep_date','dep_place']
return dict(zip(my_keys,my_vals))
def middle_dict(fvals = [], bvals = [], price = None):
d = {"forth": inner_dict(fvals),"back":inner_dict(bvals), 'price': price}
return d
Typical use:
>>> middle_dict(['5-18-4', 'Atlanta','5-18-4','New York'],
['5-19-4', 'New York','5-19-4','Atlanta'], 134.05)
{'forth': {'arr_date': '5-18-4', 'dep_place': 'New York', 'dep_date': '5-18-4', 'arr_place': 'Atlanta'}, 'price': 134.05, 'back': {'arr_date': '5-19-4', 'dep_place': 'Atlanta', 'dep_date': '5-19-4', 'arr_place': 'New York'}}
>>>
>>> d = {i:middle_dict() for i in range(1,4)}
>>> d
{1: {'forth': {'arr_date': None, 'dep_place': None, 'dep_date': None, 'arr_place': None}, 'price': 0.0, 'back': {'arr_date': None, 'dep_place': None, 'dep_date': None, 'arr_place': None}}, 2: {'forth': {'arr_date': None, 'dep_place': None, 'dep_date': None, 'arr_place': None}, 'price': 0.0, 'back': {'arr_date': None, 'dep_place': None, 'dep_date': None, 'arr_place': None}}, 3: {'forth': {'arr_date': None, 'dep_place': None, 'dep_date': None, 'arr_place': None}, 'price': 0.0, 'back': {'arr_date': None, 'dep_place': None, 'dep_date': None, 'arr_place': None}}}

This should produce the empty ordered dict you're looking for assuming you want to instantiate with None values in your nested dicts:
from collections import OrderedDict
d=OrderedDict()
for x in range(1,6):
d[x]={key:dict.fromkeys(['arr_date','arr_place','dep_date','dep_place'],None) if key in ['forth','back'] else None for key in ['forth','back','price']}
Which gives the following dict:
In[42]: dict(d)
Out[42]: {1: {'price': None, 'forth': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}, 'back': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}}, 2: {'price': None, 'forth': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}, 'back': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}}, 3: {'price': None, 'forth': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}, 'back': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}}, 4: {'price': None, 'forth': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}, 'back': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}}, 5: {'price': None, 'forth': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}, 'back': {'arr_date': None, 'dep_date': None, 'arr_place': None, 'dep_place': None}}}

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.

Python value assigned to incorrect dict key - python

Related

How to load list columns into a dataframe?

How to iterate over interval in json file and create a dataframe?

Iterate over json result and get the desirable data in pandas dataframe

Parse/print out value from JSON content in Python

Instantiating a nested dictionary

Categories

Resources