I use the Python Client for Google Maps Services to get following data from google-maps:
{
'address_components':[
{
'long_name':'20',
'short_name':'20',
'types':[
'street_number'
]
},
{
'long_name':'Oberböhl',
'short_name':'Oberböhl',
'types':[
'route'
]
},
{
'long_name':'Ingelheim am Rhein',
'short_name':'Ingelheim am Rhein',
'types':[
'locality',
'political'
]
},
{
'long_name':'Mainz-Bingen',
'short_name':'Mainz-Bingen',
'types':[
'administrative_area_level_3',
'political'
]
},
{
'long_name':'Rheinland-Pfalz',
'short_name':'RP',
'types':[
'administrative_area_level_1',
'political'
]
},
{
'long_name':'Germany',
'short_name':'DE',
'types':[
'country',
'political'
]
},
{
'long_name':'55218',
'short_name':'55218',
'types':[
'postal_code'
]
}
],
'adr_address':'<span class="street-address">Oberböhl 20</span>, <span class="postal-code">55218</span> <span class="locality">Ingelheim am Rhein</span>, <span class="country-name">Germany</span>',
'formatted_address':'Oberböhl 20, 55218 Ingelheim am Rhein, Germany',
'formatted_phone_number':'06132 5099968',
'geometry':{
'location':{
'lat':49.9810156,
'lng':8.0739617
},
'viewport':{
'northeast':{
'lat':49.9823942302915,
'lng':8.075293780291501
},
'southwest':{
'lat':49.9796962697085,
'lng':8.072595819708498
}
}
},
'icon':'https://maps.gstatic.com/mapfiles/place_api/icons/generic_business-71.png',
'id':'d2b37ffe23fd5e76648a90df2987558b039fcdf7',
'international_phone_number':'+49 6132 5099968',
'name':'Esch Metalltechnik GmbH',
'place_id':'ChIJHaERGJ_svUcRRfqNoGXq3EU',
'plus_code':{
'compound_code':'X3JF+CH Ingelheim am Rhein, Germany',
'global_code':'8FXCX3JF+CH'
},
'reference':'ChIJHaERGJ_svUcRRfqNoGXq3EU',
'scope':'GOOGLE',
'types':[
'general_contractor',
'point_of_interest',
'establishment'
],
'url':'https://maps.google.com/?cid=5034156205699627589',
'utc_offset':60,
'vicinity':'Oberböhl 20, Ingelheim am Rhein',
'website':'http://www.esch-metalltechnik.de/'
}{
'long_name':'55218',
'short_name':'55218',
'types':[
'postal_code'
]
}
Now I want to extract certain variables, like the "street_number". I don't know which format this data is, so I worked with it like a dictionary:
try:
self.hausnr = place_result_2["address_components"][0]["long_name"]
except:
self.hausnr = "NA"
The problem is, that the index "0" isn't always the same position of the data I want, i varies. Is there a way to extract the data in another way? Perhaps I have to use a JSON-parser or something similar?
Thanks a lot.
The answer is: List comprehensions
try:
# make a list of all address components that have type "street number"
comp = [c for c in place_result_2["address_components"] if "street_number" in c["types"]]
# the first one of them (assuming there will never be more than one) is the desired one
self.hausnr = comp[0]["long_name"]
except:
self.hausnr = "NA"
Since this will probably be a common operation, make a function:
def get_address_component(place_result, comp_type, comp_property="long_name", default=None):
""" returns the first address component of a given type """
try:
comp = [c for c in place_result["address_components"] if comp_type in c["types"]]
return comp[0][comp_property]
except KeyError:
return default
# ...
self.hausnr = get_address_component(place_result_2, "street_number", default="NA")
PS, regarding:
Perhaps I have to use a JSON-parser or something similar?
JSON is a data transfer format - it's plain text. The Google API server used it to get the data across the wire. In your program it has already been parsed - by the Google API client library you are using. What you are looking at is not JSON anymore, it's a Python data structure (nested dicts and lists and values). It just happens to look quite similar to JSON when you print it to the console, because Python uses a similar format to represent data.
On other words, no, you don't need to JSON-parse it again.
Related
I am trying to use Python to extract pricePerUnit from JSON. There are many entries, and this is just 2 of them -
{
"terms": {
"OnDemand": {
"7Y9ZZ3FXWPC86CZY": {
"7Y9ZZ3FXWPC86CZY.JRTCKXETXF": {
"offerTermCode": "JRTCKXETXF",
"sku": "7Y9ZZ3FXWPC86CZY",
"effectiveDate": "2020-11-01T00:00:00Z",
"priceDimensions": {
"7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7": {
"rateCode": "7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7",
"description": "Processed translation request in AWS GovCloud (US)",
"beginRange": "0",
"endRange": "Inf",
"unit": "Character",
"pricePerUnit": {
"USD": "0.0000150000"
},
"appliesTo": []
}
},
"termAttributes": {}
}
},
"CQNY8UFVUNQQYYV4": {
"CQNY8UFVUNQQYYV4.JRTCKXETXF": {
"offerTermCode": "JRTCKXETXF",
"sku": "CQNY8UFVUNQQYYV4",
"effectiveDate": "2020-11-01T00:00:00Z",
"priceDimensions": {
"CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7": {
"rateCode": "CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7",
"description": "$0.000015 per Character for TextTranslationJob:TextTranslationJob in EU (London)",
"beginRange": "0",
"endRange": "Inf",
"unit": "Character",
"pricePerUnit": {
"USD": "0.0000150000"
},
"appliesTo": []
}
},
"termAttributes": {}
}
}
}
}
}
The issue I run into is that the keys, which in this sample, are 7Y9ZZ3FXWPC86CZY, CQNY8UFVUNQQYYV4.JRTCKXETXF, and CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7 are a changing string that I cannot just type out as I am parsing the dictionary.
I have python code that works for the first level of these random keys -
with open('index.json') as json_file:
data = json.load(json_file)
json_keys=list(data['terms']['OnDemand'].keys())
#Get the region
for i in json_keys:
print((data['terms']['OnDemand'][i]))
However, this is tedious, as I would need to run the same code three times to get the other keys like 7Y9ZZ3FXWPC86CZY.JRTCKXETXF and 7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7, since the string changes with each JSON entry.
Is there a way that I can just tell python to automatically enter the next level of the JSON object, without having to parse all keys, save them, and then iterate through them? Using JQ in bash I can do this quite easily with jq -r '.terms[][][]'.
If you are really sure, that there is exactly one key-value pair on each level, you can try the following:
def descend(x, depth):
for i in range(depth):
x = next(iter(x.values()))
return x
You can use dict.values() to iterate over the values of a dict. You can also use next(iter(dict.values())) to get a first (only) element of a dict.
for demand in data['terms']['OnDemand'].values():
next_level = next(iter(demand.values()))
print(next_level)
If you expect other number of children than 1 in the second level, you can just nest the fors:
for demand in data['terms']['OnDemand'].values():
for sub_demand in demand.values()
print(sub_demand)
If you are insterested in the keys too, you can use dict.items() method to iterate over dict keys and values at the same time:
for demand_key, demand in data['terms']['OnDemand'].items():
for sub_demand_key, sub_demand in demand.items()
print(demand_key, sub_demand_key, sub_demand)
Appreciate if you could help me for the best way to transform a result into json as below.
We have a result like below, where we are getting an information on the employees and the companies. In the result, somehow, we are getting a enum like T, but not for all the properties.
[ {
"T.id":"Employee_11",
"T.category":"Employee",
"node_id":["11"]
},
{
"T.id":"Company_12",
"T.category":"Company",
"node_id":["12"],
"employeecount":800
},
{
"T.id":"id~Employee_11_to_Company_12",
"T.category":"WorksIn",
},
{
"T.id":"Employee_13",
"T.category":"Employee",
"node_id":["13"]
},
{
"T.id":"Parent_Company_14",
"T.category":"ParentCompany",
"node_id":["14"],
"employeecount":900,
"childcompany":"Company_12"
},
{
"T.id":"id~Employee_13_to_Parent_Company_14",
"T.category":"Contractorin",
}]
We need to transform this result into a different structure and grouping based on the category, if category in Employee, Company and ParentCompany, then it should be under the node_properties object, else, should be in the edge_properties. And also, apart from the common properties(property_id, property_category and node), different properties to be added if the category is company and parent company. There are few more logic also where we have to get the from and to properties of the edge object based on the 'to' . the expected response is,
"node_properties":[
{
"property_id":"Employee_11",
"property_category":"Employee",
"node":{node_id: "11"}
},
{
"property_id":"Company_12",
"property_category":"Company",
"node":{node_id: "12"},
"employeecount":800
},
{
"property_id":"Employee_13",
"property_category":"Employee",
"node":{node_id: "13"}
},
{
"property_id":"Company_14",
"property_category":"ParentCompany",
"node":{node_id: "14"},
"employeecount":900,
"childcompany":"Company_12"
}
],
"edge_properties":[
{
"from":"Employee_11",
"to":"Company_12",
"property_id":"Employee_11_to_Company_12",
},
{
"from":"Employee_13",
"to":"Parent_Company_14",
"property_id":"Employee_13_to_Parent_Company_14",
}
]
In java, we have used the enhanced for loop, switch etc. How we can write the code in the python to get the structure as above from the initial result structure. ( I am new to python), thank you in advance.
Regards
Here is a method that I quickly made, you can adjust it to your requirements. You can use regex or your own function to get the IDs of the edge_properties then assign it to an object like the way I did. I am not so sure of your full requirements but if that list that you gave is all the categories then this will be sufficient.
def transform(input_list):
node_properties = []
edge_properties = []
for input_obj in input_list:
# print(obj)
new_obj = {}
if input_obj['T.category'] == 'Employee' or input_obj['T.category'] == 'Company' or input_obj['T.category'] == 'ParentCompany':
new_obj['property_id'] = input_obj['T.id']
new_obj['property_category'] = input_obj['T.category']
new_obj['node'] = {input_obj['node_id'][0]}
if "employeecount" in input_obj:
new_obj['employeecount'] = input_obj['employeecount']
if "childcompany" in input_obj:
new_obj['childcompany'] = input_obj['childcompany']
node_properties.append(new_obj)
else: # You can do elif == to as well based on your requirements if there are other outliers
# You can use regex or whichever method here to split the string and add the values like above
edge_properties.append(new_obj)
return [node_properties, edge_properties]
I'm trying to use the mediaItems().search() method, using the following body:
body = {
"pageToken": page_token if page_token != "" else "",
"pageSize": 100,
"filters": {
"contentFilter": {
"includedContentCategories": {"LANDSCAPES","CITYSCAPES"}
}
},
"includeArchiveMedia": include_archive
}
but the problem is that the set {"LANDSCAPES","CITYSCAPES"} should actually be a set of enums (as in Java enums), and not strings as ive written. this is specified in the API: (https://developers.google.com/photos/library/reference/rest/v1/albums)
ContentFilter - This filter allows you to return media items based on the content type.
JSON representation
{
"includedContentCategories": [
enum (ContentCategory)
],
"excludedContentCategories": [
enum (ContentCategory)
]
}
is there a proper way of solving this in python?
Modification points:
When albumId and filters are used, an error of The album ID cannot be set if filters are used. occurs. So when you want to use filters, please remove albumId.
The value of includedContentCategories is an array as follows.
"includedContentCategories": ["LANDSCAPES","CITYSCAPES"]
includeArchiveMedia is includeArchivedMedia.
Please include includeArchivedMedia in filters.
When above points are reflected to your script, it becomes as follows.
Modified script:
body = {
# "albumId": album_id, # <--- removed
"pageToken": page_token if page_token != "" else "",
"pageSize": 100,
"filters": {
"contentFilter": {
"includedContentCategories": ["LANDSCAPES", "CITYSCAPES"]
},
"includeArchivedMedia": include_archive
}
}
Reference:
Method: mediaItems.search
TL;DR:
Confused on how to parse following JSON response and get the value of [status of 12345 of dynamicValue_GGG of payload] in this case.
Full question:
I get the following as (sanitized) response upon hitting a REST API via Python code below:
response = requests.request("POST", url, data=payload,
headers=headers).json()
{
"payload": {
"name": "asdasdasdasd",
"dynamicValue_GGG": {
"12345": {
"model": "asad",
"status": "active",
"subModel1": {
"dynamicValue_67890": {
"model": "qwerty",
"status": "active"
},
"subModel2": {
"dynamicValue_33445": {
"model": "gghjjj",
"status": "active"
},
"subModel3": {
"dynamicValue_66778": {
"model": "tyutyu",
"status": "active"
}
}
}
},
"date": "2016-02-04"
},
"design": "asdasdWWWsaasdasQ"
}
If I do a type(response['payload']), it gives me 'dict'.
Now, I'm trying to parse the response above and fetch certain keys and values out of it. The problem is that I'm not able to iterate through using "index" and rather have to specify the "key", but then the response has certain "keys" that are dynamically generated and sent over. For instance, the keys called "dynamicValue_GGG", "dynamicValue_66778" etc are not static unlike the "status" key.
I can successfully parse by mentioning like:
print response['payload']['dynamicValue_GGG']['12345'][status]
in which case I get the expected output = 'active'.
However, since I have no control on 'dynamicValue_GGG', it would work only if I can specify something like this instead:
print response['payload'][0][0][status]
But the above line gives me error: " KeyError: 0 " when the python code is executed.
Is there someway in which I can use the power of both keys as well as index together in this case?
The order of values in a dictionary in Python are random, so you cannot use indexing. You'll have to iterate over all elements, potentially recursive, and test to see if it's the thing you're looking for. For example:
def find_submodels(your_dict):
for item_key, item_values in your_dict.items():
if 'status' in item_values:
print item_key, item_values['status']
if type(item_values) == dict:
find_submodels(item_values)
find_submodels(your_dict)
Which would output:
12345 active
dynamicValue_67890 active
dynamicValue_66778 active
dynamicValue_33445 active
I'm pretty new to Python, so just working my way through understanding the data sets.
I'm having a little trouble producing the JSON output that is required for the API I am working with.
I am using
import json
json.load(data_file)
Working with Python dictionary and then doing
json.dump(dict, json_data)
My data needs to look like the following when it is output.
{
"event":{
"id":10006,
"event_name":"My Event Name",
},
"sub event":[
],
"attendees":[
{
"id":11201,
"first_name":"Jeff",
"last_name":"Smith",
},
{
"id":10002,
"first_name":"Victoria",
"last_name":"Baker",
},
]
}
I have been able to create the arrays in python and dump to json, but I am having difficulty creating the event "object" in the dictionary. I am using the below:
attendees = ['attendees']
attendeesdict = {}
attendeesdict['first_name'] = "Jeff"
attendees.append(attendeesdict.copy())
Can anyone help me add the "event" object properly?
In general, going from JSON to dictionary is almost no work because the two are very similar, if not identical:
attendees = [
{
"first_name": "Jeff"
# Add other fields which you need here
},
{
"first_name": "Victoria"
}
]
In this instance, attendees is a list of dictionaries. For the event:
event = {
"id": 10006,
"event_name": "My Event Name"
}
Putting it all together:
data = {
"event": event,
"sub event": [],
"attendees": attendees
}
Now, you can convert it to a JSON object, ready to send to your API:
json_object = json.dumps(data)
Assuming you have built all the values elsewhere and now you're just putting them together:
result = {'event':event_dict, 'sub event':subevent_list, 'attendees':attendees_list}
If you want just to statically create a nested dict, you can use a single literal. If you paste the JSON above into python code, you would get a valid dict literal.
Construct your dicts and add like below
{
"event":"add your dict"
"sub event":["add your dict"],
"attendees":["add your dict"]
}