I have two lists in Python something similar.
list1 = [
{"name": "sample1",
"place": "sampleplace1",
"value": "",
"time": "sampletime"
},
{"name": "sample2",
"place": "sampleplace2",
"value": "",
"time": "sampletime2"
}
]
list2 = [
{"name": "sample1",
"value": "10"
},
{"name": "sample2",
"value": "20"
}
]
I need to compare both the lists and whereever the name is matching, I need to update the value property in list1. I did by running a for loop on list1, get the matching list object from list2 for each list1 object and update the value.
I'm just wondering, is there a way to do this without running a for loop (something like Linq in C#)?
Sadly, Python does not have the same abilities as LINQ.
If you don't want to explicitly use a function there is map, but it uses a loop under the hood, as LINQ does.
You need for loops, like in :
list1 = [
{"name": "sample1",
"place": "sampleplace1",
"value": "",
"time": "sampletime"
},
{"name": "sample2",
"place": "sampleplace2",
"value": "",
"time": "sampletime2"
}
]
list2 = [
{"name": "sample1",
"value": "10"
},
{"name": "sample2",
"value": "20"
}
]
for elem1 in list1:
for elem2 in list2:
if elem1["name"] == elem2["name"]:
# match ! we replace the value
elem1["value"] = elem2["value"]
break # and stop searching
else:
print(f"no match in list2 for {elem1['name']=}")
# just for displaying the result
import json
print(json.dumps(list1, indent=2))
[
{
"name": "sample1",
"place": "sampleplace1",
"value": "10",
"time": "sampletime"
},
{
"name": "sample2",
"place": "sampleplace2",
"value": "20",
"time": "sampletime2"
}
]
Related
[{
"name": "hyderabad1",
"status": "one",
"value": [{
"level_name": "hyderabad2",
"status": "two",
"value": [{
"level_name": "hyderabad3",
"status": "three",
}]
"count": zero,
"unit": [
""
]
}]
},
{
"name": "hyderabad4",
"status": "one",
"value": [{
"level_name": "hyderabad5",
"status": "two",
"value": [{
"level_name": "hyderabad6",
"status": "three",
}]
"count": zero,
"unit": [
""
]
}]
}
]
In this if i search name hyderabad2 in the status two of level name in first object, then i just want to return that entire object which consists three levels,if not does not match then it should not return anything.
iam expecting the whole object in mongodb when the name match found.
So, I'm trying to parse this json object into multiple events, as it's the expected input for a ETL tool. I know this is quite straight forward if we do this via loops, if statements and explicitly defining the search fields for given events. This method is not feasible because I have multiple heavily nested JSON objects and I would prefer to let the python recursions handle the heavy lifting. The following is a sample object, which consist of string, list and dict (basically covers most use-cases, from the data I have).
{
"event_name": "restaurants",
"properties": {
"_id": "5a9909384309cf90b5739342",
"name": "Mangal Kebab Turkish Restaurant",
"restaurant_id": "41009112",
"borough": "Queens",
"cuisine": "Turkish",
"address": {
"building": "4620",
"coord": {
"0": -73.9180155,
"1": 40.7427742
},
"street": "Queens Boulevard",
"zipcode": "11104"
},
"grades": [
{
"date": 1414540800000,
"grade": "A",
"score": 12
},
{
"date": 1397692800000,
"grade": "A",
"score": 10
},
{
"date": 1381276800000,
"grade": "A",
"score": 12
}
]
}
}
And I want to convert it to this following list of dictionaries
[
{
"event_name": "restaurants",
"properties": {
"restaurant_id": "41009112",
"name": "Mangal Kebab Turkish Restaurant",
"cuisine": "Turkish",
"_id": "5a9909384309cf90b5739342",
"borough": "Queens"
}
},
{
"event_name": "restaurant_address",
"properties": {
"zipcode": "11104",
"ref_id": "41009112",
"street": "Queens Boulevard",
"building": "4620"
}
},
{
"event_name": "restaurant_address_coord"
"ref_id": "41009112"
"0": -73.9180155,
"1": 40.7427742
},
{
"event_name": "restaurant_grades",
"properties": {
"date": 1414540800000,
"ref_id": "41009112",
"score": 12,
"grade": "A",
"index": "0"
}
},
{
"event_name": "restaurant_grades",
"properties": {
"date": 1397692800000,
"ref_id": "41009112",
"score": 10,
"grade": "A",
"index": "1"
}
},
{
"event_name": "restaurant_grades",
"properties": {
"date": 1381276800000,
"ref_id": "41009112",
"score": 12,
"grade": "A",
"index": "2"
}
}
]
And most importantly these events will be broken up into independent structured tables to conduct joins, we need to create primary keys/ unique identifiers. So the deeply nested dictionaries should have its corresponding parents_id field as ref_id. In this case ref_id = restaurant_id from its parent dictionary.
Most of the example on the internet flatten's the whole object to be normalized and into a dataframe, but to utilise this ETL tool to its full potential it would be ideal to solve this problem via recursions and outputting as list of dictionaries.
This is what one might call a brute force method. Create a translator function to move each item into the correct part of the new structure (like a schema).
# input dict
d = {
"event_name": "demo",
"properties": {
"_id": "5a9909384309cf90b5739342",
"name": "Mangal Kebab Turkish Restaurant",
"restaurant_id": "41009112",
"borough": "Queens",
"cuisine": "Turkish",
"address": {
"building": "4620",
"coord": {
"0": -73.9180155,
"1": 40.7427742
},
"street": "Queens Boulevard",
"zipcode": "11104"
},
"grades": [
{
"date": 1414540800000,
"grade": "A",
"score": 12
},
{
"date": 1397692800000,
"grade": "A",
"score": 10
},
{
"date": 1381276800000,
"grade": "A",
"score": 12
}
]
}
}
def convert_structure(d: dict):
''' function to convert to new structure'''
# the new dict
e = {}
e['event_name'] = d['event_name']
e['properties'] = {}
e['properties']['restaurant_id'] = d['properties']['restaurant_id']
# and so forth...
# keep building the new structure / template
# return a list
return [e]
# run & print
x = convert_structure(d)
print(x)
the reuslt (for the part done) looks like this:
[{'event_name': 'demo', 'properties': {'restaurant_id': '41009112'}}]
If a pattern is identified, then the above could be improved...
How to convert a list of dictionaries to a list?
Here is what I have:
{
"sources": [
{
"ID": "6953",
"VALUE": "https://address-jbr.ofp.ae"
},
{
"ID": "6967",
"VALUE": "https://plots.ae"
},
{
"ID": "6970",
"VALUE": "https://dubai-creek-harbour.ofp.ae"
}]}
Here is what I want it to look like:
({'6953':'https://address-jbr.ofp.ae','6967':'https://plots.ae','6970':'https://dubai-creek-harbour.ofp.ae'})
This is indeed very straightforward:
data = {
"sources": [
{
"ID": "6953",
"VALUE": "https://address-jbr.ofp.ae"
},
{
"ID": "6967",
"VALUE": "https://plots.ae"
},
{
"ID": "6970",
"VALUE": "https://dubai-creek-harbour.ofp.ae"
}]
}
Then:
data_list = [{x["ID"]: x["VALUE"]} for x in data["sources"]]
Which is the same as:
data_list = []
for x in data["sources"]:
data_list.append({
x["ID"]: x["VALUE"]
})
EDIT: You said convert to a "list" in the question and that confused me. Then this is what you want:
data_dict = {x["ID"]: x["VALUE"] for x in data["sources"]}
Which is the same as:
data_dict = {}
for x in data["sources"]:
data_dict[x["ID"]] = x["VALUE"]
P.S. Seems like you're asking for answers to your course assignments or something here, which is not what this place is for.
A solution using pandas
import pandas as pd
data = {
"sources": [
{"ID": "6953", "VALUE": "https://address-jbr.ofp.ae"},
{"ID": "6967", "VALUE": "https://plots.ae"},
{"ID": "6970", "VALUE": "https://dubai-creek-harbour.ofp.ae"},
]
}
a = pd.DataFrame.from_dict(data["sources"])
print(a.set_index("ID").T.to_dict(orient="records"))
outputs to:
[{'6953': 'https://address-jbr.ofp.ae', '6967': 'https://plots.ae', '6970': 'https://dubai-creek-harbour.ofp.ae'}]
this should work.
Dict = {
"sources": [
{
"ID": "6953",
"VALUE": "https://address-jbr.ofp.ae"
},
{
"ID": "6967",
"VALUE": "https://plots.ae"
},
{
"ID": "6970",
"VALUE": "https://dubai-creek-harbour.ofp.ae"
}]}
# Store all the keys here
value_LIST = []
for item_of_list in Dict["sources"]:
for key, value in item_of_list.items():
value_LIST.append(value)
This is an extension from a previous question that is unresolved but another way of looking at the problem HERE
{
"name": "Sarah",
"region": "south west",
"age": 21,
"occupation": "teacher",
"height": 145,
"education": "university",
"favouriteMovie": "matrix",
"gender": "female",
"country": "US",
"level": "medium",
"tags": [],
"data": "abc",
"moreData": "xyz",
"logging" : {
"systemLogging" : [ {
"environment" : "test",
"example" : [ "this", "is", "an", "example", "array" ]
} ]
}
}
If I wanted to see if a value exists for simple key value pairs in a python dictionary I could use the jmespath query to do so
"occupation == `teacher`"
>> True
"occupation == `dancer`"
>> False
But with more complicated data e.g nested dictionaries with lists/arrays the best I can do is filter to obtain the value
"logging.systemLogging[?environment == `test`] | [0].enabled"
>> test
"logging.systemLogging[?environment == `prod`] | [0].enabled"
>> None
How do I get the example above to return True or False rather then the value if it exists and None or empty array [] if it doesn't
Well, you can perfectly do the same:
logging.systemLogging[?environment == `test`] | [0].enabled != null
On
{
"name": "Sarah",
"region": "south west",
"age": 21,
"occupation": "teacher",
"height": 145,
"education": "university",
"favouriteMovie": "matrix",
"gender": "female",
"country": "US",
"level": "medium",
"tags": [],
"data": "abc",
"moreData": "xyz",
"logging": {
"systemLogging": [{
"environment": "test",
"example": ["this", "is", "an", "example", "array"]
}]
}
}
Will result in
false
While the same on this JSON — where logging.systemLogging[0].enabled: false was added:
{
"name": "Sarah",
"region": "south west",
"age": 21,
"occupation": "teacher",
"height": 145,
"education": "university",
"favouriteMovie": "matrix",
"gender": "female",
"country": "US",
"level": "medium",
"tags": [],
"data": "abc",
"moreData": "xyz",
"logging": {
"systemLogging": [{
"enabled": false,
"environment": "test",
"example": ["this", "is", "an", "example", "array"]
}]
}
}
Will result in
true
I have 2 separate JSON-Lists (with dicts) in it.
My goal is, that I want to iterate over list2 "currentUser", grab the values, search those values in list1, and as output print the value of "firstName"
e.g.
liste2: "currentUser": 123,
liste1: "id": "123", --> "firstName": "Lisa",
list1 = {
"X-API-KEY": "XyZzahZaksksXXXYYYOOO000",
"user": {
"email": "Lisa#BLA.com",
"firstName": "Lisa",
"id": "123",
},
"Flat": {
"city": "Munich",
"country": "2",
"countryCode": "DEU",
"currency": "EUR",
"date": "1587671397",
"flatmates": [
{
"email": "Lisa#BLA.com",
"firstName": "Lisa",
"id": "123",
},
{
"email": "Max#BLA.com",
"firstName": "Max",
"id": "124",
},
{
"email": "Hannah#BLA.com",
"firstName": "Hannah",
"id": "125",
},
{
"email": "Kai#BLA.com",
"firstName": "Kai",
"id": "126",
}
],
"founderId": "123",
"id": "99999",
"image": "",
"name": "ABC",
"postCode": "000000",
}
}
list2 = [
{
"creationDate": 1587671663,
"currentUser": 123,
"id": 1717134,
"title": "Do this",
"users": [
124,
126
]
},
{
"creationDate": 1587671663,
"currentUser": 126,
"id": 1717134,
"title": "Do that",
"users": [
123,
125
]
},
{
"creationDate": 1587671821,
"currentUser": 124,
"id": 1717134,
"title": "Clean this",
"users": [
125,
122
]
},
{
"creationDate": 1587671801,
"currentUser": 123,
"id": 1717134,
"title": "Clean that",
"users": [
124,
126
]
}
]
I am pretty new to python.
There are several mind-issues for me since there is a mix between lists and dictionaries in it and how to match/search for values for 2 separate lists/dicts
What I got so far: Iterate over the "CurrentUser"
for user in liste2:
print(user["currentUser"])
Has anyone some approaches?
In pure python with no other modules.
for user in list2:
for mate in list1['Flat']['flatmates']:
if user['currentUser'] == int(mate['id']):
# You found the person now execute this code...
One thing to note that in your list1 you flatmates id is not a integer but a string. So you have to convert that to an int in order to compare the two.