I need to export dict to yaml with pyyaml without the "-"
export_dash_dict = {}
export_dash_dict["dashboards"] = []
for dashboard in dashboards_to_export:
single_dashboard = {}
single_dashboard[dashboard.title] = {}
single_dashboard[dashboard.title]["owner"] = dashboard.owner.username
single_dashboard[dashboard.title]["description"] = dashboard.description
export_dash_dict["dashboards"].append(single_dashboard)
final_yaml = yaml.dump(export_dash_dict, default_flow_style=False,default_style=None)
This is how pyyaml export my dict:
dashboards:
- Dashboard title 1:
description: First
owner: username1
- Dashboard title 2:
description: Second
owner: username2
Using an online parsed, this is what I get:
{
"dashboards": [
{
"Dashboard title 1": {
"owner": "username1",
"description": "First"
}
},
{
"Dashboard title 2": {
"owner": "username2",
"description": "Second"
}
}
]
}
But I need something like this:
dashboards:
Dashboard title 1:
description: First
owner: username1
Dashboard title 2:
description: Second
owner: username2
with the online parser:
{
"dashboards": {
"Dashboard title 2": {
"owner": "username2",
"description": "Second"
},
"Dashboard title 1": {
"owner": "username1",
"description": "First"
}
}
}
In this way I can avoid a entire level dept of data when I use yaml.load
Im using: default_flow_style=False parameters but I can't find a way to avoid "-"
Is that a standard of YAML?
The yaml dump is correctly exporting your data structure. You have a dictionary (export_dash_dict) with one key and one value. The one key is dashboards and its value is a list with two elements. The two elements are dictionaries (whose contents we won't go into here, but they have two key-value pairs).
The data structure you say that you would like to export is not a dictionary with one key/value pair where the value is a list, but rather a dictionary with two key-value pairs. The first key is to be Dashboard title 1 and its value is to be a dictionary (with two key-value pairs that we won't detail here). The second key is to be Dashboard title 2 and its value is to be a dictionary (with, again, two key-value pairs that we won't detail).
If you want yaml.dump to write such a data structure, you need to construct that data structure.
Thanks to torek, this is the correct way to build the dict:
export_dash_dict = {}
export_dash_dict["dashboards"] = {}
for dashboard in dashboards_to_export:
export_dash_dict["dashboards"][dashboard.title] = {}
export_dash_dict["dashboards"][dashboard.title]["owner"] = dashboard.owner.username
export_dash_dict["dashboards"][dashboard.title]["description"] = dashboard.description
final_yaml = yaml.dump(export_dash_dict, default_flow_style=False,default_style=None)
Related
I am trying to use Python to extract pricePerUnit from JSON. There are many entries, and this is just 2 of them -
{
"terms": {
"OnDemand": {
"7Y9ZZ3FXWPC86CZY": {
"7Y9ZZ3FXWPC86CZY.JRTCKXETXF": {
"offerTermCode": "JRTCKXETXF",
"sku": "7Y9ZZ3FXWPC86CZY",
"effectiveDate": "2020-11-01T00:00:00Z",
"priceDimensions": {
"7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7": {
"rateCode": "7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7",
"description": "Processed translation request in AWS GovCloud (US)",
"beginRange": "0",
"endRange": "Inf",
"unit": "Character",
"pricePerUnit": {
"USD": "0.0000150000"
},
"appliesTo": []
}
},
"termAttributes": {}
}
},
"CQNY8UFVUNQQYYV4": {
"CQNY8UFVUNQQYYV4.JRTCKXETXF": {
"offerTermCode": "JRTCKXETXF",
"sku": "CQNY8UFVUNQQYYV4",
"effectiveDate": "2020-11-01T00:00:00Z",
"priceDimensions": {
"CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7": {
"rateCode": "CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7",
"description": "$0.000015 per Character for TextTranslationJob:TextTranslationJob in EU (London)",
"beginRange": "0",
"endRange": "Inf",
"unit": "Character",
"pricePerUnit": {
"USD": "0.0000150000"
},
"appliesTo": []
}
},
"termAttributes": {}
}
}
}
}
}
The issue I run into is that the keys, which in this sample, are 7Y9ZZ3FXWPC86CZY, CQNY8UFVUNQQYYV4.JRTCKXETXF, and CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7 are a changing string that I cannot just type out as I am parsing the dictionary.
I have python code that works for the first level of these random keys -
with open('index.json') as json_file:
data = json.load(json_file)
json_keys=list(data['terms']['OnDemand'].keys())
#Get the region
for i in json_keys:
print((data['terms']['OnDemand'][i]))
However, this is tedious, as I would need to run the same code three times to get the other keys like 7Y9ZZ3FXWPC86CZY.JRTCKXETXF and 7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7, since the string changes with each JSON entry.
Is there a way that I can just tell python to automatically enter the next level of the JSON object, without having to parse all keys, save them, and then iterate through them? Using JQ in bash I can do this quite easily with jq -r '.terms[][][]'.
If you are really sure, that there is exactly one key-value pair on each level, you can try the following:
def descend(x, depth):
for i in range(depth):
x = next(iter(x.values()))
return x
You can use dict.values() to iterate over the values of a dict. You can also use next(iter(dict.values())) to get a first (only) element of a dict.
for demand in data['terms']['OnDemand'].values():
next_level = next(iter(demand.values()))
print(next_level)
If you expect other number of children than 1 in the second level, you can just nest the fors:
for demand in data['terms']['OnDemand'].values():
for sub_demand in demand.values()
print(sub_demand)
If you are insterested in the keys too, you can use dict.items() method to iterate over dict keys and values at the same time:
for demand_key, demand in data['terms']['OnDemand'].items():
for sub_demand_key, sub_demand in demand.items()
print(demand_key, sub_demand_key, sub_demand)
Appreciate if you could help me for the best way to transform a result into json as below.
We have a result like below, where we are getting an information on the employees and the companies. In the result, somehow, we are getting a enum like T, but not for all the properties.
[ {
"T.id":"Employee_11",
"T.category":"Employee",
"node_id":["11"]
},
{
"T.id":"Company_12",
"T.category":"Company",
"node_id":["12"],
"employeecount":800
},
{
"T.id":"id~Employee_11_to_Company_12",
"T.category":"WorksIn",
},
{
"T.id":"Employee_13",
"T.category":"Employee",
"node_id":["13"]
},
{
"T.id":"Parent_Company_14",
"T.category":"ParentCompany",
"node_id":["14"],
"employeecount":900,
"childcompany":"Company_12"
},
{
"T.id":"id~Employee_13_to_Parent_Company_14",
"T.category":"Contractorin",
}]
We need to transform this result into a different structure and grouping based on the category, if category in Employee, Company and ParentCompany, then it should be under the node_properties object, else, should be in the edge_properties. And also, apart from the common properties(property_id, property_category and node), different properties to be added if the category is company and parent company. There are few more logic also where we have to get the from and to properties of the edge object based on the 'to' . the expected response is,
"node_properties":[
{
"property_id":"Employee_11",
"property_category":"Employee",
"node":{node_id: "11"}
},
{
"property_id":"Company_12",
"property_category":"Company",
"node":{node_id: "12"},
"employeecount":800
},
{
"property_id":"Employee_13",
"property_category":"Employee",
"node":{node_id: "13"}
},
{
"property_id":"Company_14",
"property_category":"ParentCompany",
"node":{node_id: "14"},
"employeecount":900,
"childcompany":"Company_12"
}
],
"edge_properties":[
{
"from":"Employee_11",
"to":"Company_12",
"property_id":"Employee_11_to_Company_12",
},
{
"from":"Employee_13",
"to":"Parent_Company_14",
"property_id":"Employee_13_to_Parent_Company_14",
}
]
In java, we have used the enhanced for loop, switch etc. How we can write the code in the python to get the structure as above from the initial result structure. ( I am new to python), thank you in advance.
Regards
Here is a method that I quickly made, you can adjust it to your requirements. You can use regex or your own function to get the IDs of the edge_properties then assign it to an object like the way I did. I am not so sure of your full requirements but if that list that you gave is all the categories then this will be sufficient.
def transform(input_list):
node_properties = []
edge_properties = []
for input_obj in input_list:
# print(obj)
new_obj = {}
if input_obj['T.category'] == 'Employee' or input_obj['T.category'] == 'Company' or input_obj['T.category'] == 'ParentCompany':
new_obj['property_id'] = input_obj['T.id']
new_obj['property_category'] = input_obj['T.category']
new_obj['node'] = {input_obj['node_id'][0]}
if "employeecount" in input_obj:
new_obj['employeecount'] = input_obj['employeecount']
if "childcompany" in input_obj:
new_obj['childcompany'] = input_obj['childcompany']
node_properties.append(new_obj)
else: # You can do elif == to as well based on your requirements if there are other outliers
# You can use regex or whichever method here to split the string and add the values like above
edge_properties.append(new_obj)
return [node_properties, edge_properties]
I am using rest with a python script to extract Name and Start Time from a response.
I can get the information but I can't combine data so that the information is on the same line in a CSV. When I go to export them to CSV they all go on new lines.
There is probably a much better way to extract data from a JSON List.
for item in driverDetails['Query']['Results']:
for data_item in item['XValues']:
body.append(data_item)
for key, value in data_item.items():
#driver = {}
#test = {}
#startTime = {}
if key == "Name":
drivers.append(value)
if key == "StartTime":
drivers.append(value)
print (drivers)
Code to write to CSV:
with open(logFileName, 'a') as outcsv:
# configure writer to write standard csv file
writer = csv.writer(outcsv, delimiter=',', quotechar="'",
quoting=csv.QUOTE_MINIMAL, lineterminator='\n',skipinitialspace=True)
for driver in drivers:
writer.writerow(driver)
Here is a sample of the response:
"Query": {
"Results": [
{
"XValues": [
{
"ReportScopeStartTime": "2018-06-18T23:00:00Z"
},
{
"ReportScopeEndTime": "2018-06-25T22:59:59Z"
},
{
"ID": "1400"
},
{
"Name": " John Doe"
},
{
"StartTime": "2018-06-19T07:16:10Z"
},
],
},
"XValues": [
{
"ReportScopeStartTime": "2018-06-18T23:00:00Z"
},
{
"ReportScopeEndTime": "2018-06-25T22:59:59Z"
},
{
"ID": "1401"
},
{
"Name": " Jane Smith"
},
{
"StartTime": "2018-06-19T07:16:10Z"
},
],
},
My ouput in csv:
John Doe
2018-06-19T07:16:10Z
Jane Smith
2018-06-19T07:16:10Z
Desired Outcome:
John Doe, 2018-06-19T07:16:10Z
Jane Smith, 2018-06-19T07:16:10Z
Just use normal dictionary access to get the values:
for item in driverDetails['Query']['Results']:
for data_item in item['XValues']:
body.append(data_item)
if "Name" in data_item:
drivers.append(data_item["Name"])
if "StartTime" in data_item:
drivers.append(data_item["StartTime"])
print (drivers)
If you know the items will already have the required fields then you won't even need the in tests.
writer.writerow() expects a sequence. You are calling it with a single string as a parameter so it will split the string into individual characters. Probably you want to keep the name and start time together so extract them as a tuple:
for item in driverDetails['Query']['Results']:
name, start_time = "", ""
for data_item in item['XValues']:
body.append(data_item)
if "Name" in data_item:
name = data_item["Name"]
if "StartTime" in data_item:
start_time = data_item["StartTime"]
drivers.append((name, start_time))
print (drivers)
Now instead of being a list of strings, drivers is a list of tuples: the name for every item that has a name and the start time but if an input item has a name and no start time that field could be empty. Your code to write the csv file should now do the expected thing.
If you want to get all or most of the values try gathering them together into a single dictionary, then you can pull out the fields you want:
for item in driverDetails['Query']['Results']:
fields = {}
for data_item in item['XValues']:
body.append(data_item)
fields.update(data_item)
drivers.append((fields["ID"], fields["Name"], fields["StartTime"]))
print (drivers)
Once you have the fields in a single dictionary you could even build the tuple with a loop:
drivers.append(tuple(fields[f] for f in ("ID", "Name", "StartTime", "ReportScopeStartTime", "ReportScopeEndTime")))
I think you should list the fields you want explicitly just to ensure that new fields don't surprise you.
I'm pretty new to Python, so just working my way through understanding the data sets.
I'm having a little trouble producing the JSON output that is required for the API I am working with.
I am using
import json
json.load(data_file)
Working with Python dictionary and then doing
json.dump(dict, json_data)
My data needs to look like the following when it is output.
{
"event":{
"id":10006,
"event_name":"My Event Name",
},
"sub event":[
],
"attendees":[
{
"id":11201,
"first_name":"Jeff",
"last_name":"Smith",
},
{
"id":10002,
"first_name":"Victoria",
"last_name":"Baker",
},
]
}
I have been able to create the arrays in python and dump to json, but I am having difficulty creating the event "object" in the dictionary. I am using the below:
attendees = ['attendees']
attendeesdict = {}
attendeesdict['first_name'] = "Jeff"
attendees.append(attendeesdict.copy())
Can anyone help me add the "event" object properly?
In general, going from JSON to dictionary is almost no work because the two are very similar, if not identical:
attendees = [
{
"first_name": "Jeff"
# Add other fields which you need here
},
{
"first_name": "Victoria"
}
]
In this instance, attendees is a list of dictionaries. For the event:
event = {
"id": 10006,
"event_name": "My Event Name"
}
Putting it all together:
data = {
"event": event,
"sub event": [],
"attendees": attendees
}
Now, you can convert it to a JSON object, ready to send to your API:
json_object = json.dumps(data)
Assuming you have built all the values elsewhere and now you're just putting them together:
result = {'event':event_dict, 'sub event':subevent_list, 'attendees':attendees_list}
If you want just to statically create a nested dict, you can use a single literal. If you paste the JSON above into python code, you would get a valid dict literal.
Construct your dicts and add like below
{
"event":"add your dict"
"sub event":["add your dict"],
"attendees":["add your dict"]
}
i use facebook.py from:
https://github.com/pythonforfacebook/facebook-sdk
my problem is:
I don't know to use the next-url from graph.get_object("me/friends")
graph = facebook.GraphAPI(access_token)
friends = graph.get_object("me/friends")
If you type in /me/friends into the Graph API Explorer, you'll see that it returns a JSON file, which is just a combination of dictionaries and lists inside one another.
For example, the output could be:
{
"data": [
{
"name": "Foo",
"id": "1"
},
{
"name": "Bar",
"id": "1"
}
],
"paging": {
"next": "some_link"
}
}
This JSON file is already converted to a Python dictionary/list. In the outer dictionary, the key data maps to a list of dictionaries, which contain information about your friends.
So to print your friends list:
graph = facebook.GraphAPI(access_token)
friends = graph.get_object("me/friends")
for friend in friends['data']:
print "{0} has id {1}".format(friend['name'].encode('utf-8'), friend['id'])
The .encode('utf-8') is to properly print out special characters.
The above answer is mislead, as Facebook has shut down graph users from getting lists of friends UNLESS THE FRIENDS HAVE ALSO INSTALLED THE APP.
See:
graph = facebook.GraphAPI( token )
friends = graph.get_object("me/friends")
if friends['data']:
for friend in friends['data']:
print ("{0} has id {1}".format(friend['name'].encode('utf-8'), friend['id']))
else:
print('NO FRIENDS LIST')