Let's say I have a python class that has instance variables that are dictionaries. I'm new-ish to JSON and looking for the most pythonic way to encode/decode lists of these objects to/from .json files.
I'm curious if there is a clever way to use an object_hook to decode things that contain dictionaries as the typical encoder works inside->out as I understand. I guess I could encode the dictionary into an ugly list of 2-item lists (??) and then handle that with the decoder?
Here's what I've got now in a reproducible example. It works, but I suspect that it could be improved. Curious if there is a way to handle this with custom decoder or object_hook or other pythonic approach
import json
from typing import Dict
class Critic():
def __init__(self, name: str, reviews: Dict[str, int]):
self.name = name
self.reviews = reviews
#staticmethod
def encode(obj):
return {"name": obj.name, "reviews": obj.reviews}
#staticmethod
def decode(obj):
# ??? Can it be done ???
pass
def __repr__(self):
return f'{self.name}, {self.reviews}'
x1 = Critic('Bob', {'Jaws':4, 'Star Wars':5})
x2 = Critic('Jim', {'Dune':3, 'Top Gun 2':4})
data = json.dumps([x1, x2], default=Critic.encode, indent=2)
#print(data)
# load it, then parse the result "manually"
retrieved = json.loads(data)
res = []
for item in retrieved:
res.append(Critic(**item))
print(res)
Yields (as expected):
[Bob, {'Jaws': 4, 'Star Wars': 5}, Jim, {'Dune': 3, 'Top Gun 2': 4}]
Related
food_data is a variable containing JSON data. Using the data, I want to create a list of Food objects, like so
foods = []
for data_row in food_data:
foods.append(Food(data_row))
This is what my Food class looks like as of right now:
class Food(dict):
""" by inheriting from dict, Food objects become automatically serializable for JSON formatting """
def __init__(self, data):
""" create a serialized food object with desired fields """
id = data["id"]
name = data["title"]
image = data["image"]
super().__init__(self, id=id, name=name, image=image)
And here is some example data:
[
{
"id": 738290,
"title": "Pasta with Garlic, Scallions, Cauliflower & Breadcrumbs",
"image": "https://spoonacular.com/recipeImages/716429-312x231.jpg",
},
{
"id": 343245,
"title": "What to make for dinner tonight?? Bruschetta Style Pork & Pasta",
"image": "https://spoonacular.com/recipeImages/715538-312x231.jpg",
}
]
Is there a method I can write for the Food class that will take the data and return a list of different versions of itself?
I would start by not subclassing dict: there is a better way to make an instance of Food serializable.
Next, make Food.__init__ dumb: three arguments, used to set three attributes.
Then, define a class method that is responsible for parsing an arbitrary dict with at least id, title, and image keys to get the values expected by Food.__init__.
Finally, define a method that turns an instance of Food back into a dict (though not necessarily the same dict that from_dict uses; generate one that serializes the way you want).
class Food:
def __init__(self, id, name, image):
self.id = id
self.name = name
self.image = image
#classmethod
def from_dict(cls, d):
return cls(id=d['id'], name=d['title'], image=d['image'])
def to_dict(self):
return dict(id=self.id, name=self.name, image=self.image)
foods = [Food.from_dict(d) for d in food_data]
To make your instance serializable, define a customer encoder that uses your to_dict method,
class FoodEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Food):
return obj.to_dict()
return super().default(obj)
This piggy backs on the default encoder; if the immediate object is a Food, default returns a serializable dict. Otherwise, it defers to its parent to try to serialize it.
Then use that class in the call to json.dumps.
print(json.dumps(foods, cls=FoodEncoder))
I think in my case I may have been over-engineering my code. But I received many responses that did help me out in other aspects so I'm going to offer them here:
#Juanpa
Use a list comprehension
foods = [Food[data] for data in food_data]
#Chepner - unrelated but useful
subclass json.JSONEncoder instead of dict for serializability
#Matthias
Create a staticmethod within the class to return a list of objects
#staticmethod
def create_foods(food_data):
foods = []
for data_row in food_data:
foods.append(Food(data_row))
I am getting Data via a REST-Interface and I want to store those data in a class-object.
my class could looks like this:
class Foo:
firstname = ''
lastname = ''
street = ''
number = ''
and the json may look like this:
[
{
"fname": "Carl",
"lname": "any name",
"address": ['carls street', 12]
}
]
What's the easiest way to map between the json and my class?
My problem is: I want to have a class with a different structure than the json.
I want the names of the attributes to be more meaningful.
Of course I know that I could simply write a to_json method and a from_json method which does what I want.
The thing is: I have a lot of those classes and I am looking for more declarative way to write the code.
e.g. in Java I probably would use mapstruct.
Thanks for your help!
Use a dict for the json input. Use **kwargs in an __init__ method in your class and map the variables accordingly.
I had a similar problem, and I solved it by using #classmethod
import json
class Robot():
def __init__(self, x, y):
self.type = "new-robot"
self.x = x
self.y = y
#classmethod
def create_robot(cls, sdict):
if sdict["type"] == "new-robot":
position = sdict["position"]
return cls(position['x'], position['y'])
else:
raise Exception ("Unable to create a new robot!!!")
if __name__=='__main__':
input_string = '{"type": "new-robot", "position": {"x": 3, "y": 3}}'
cmd = json.loads(input_string)
bot = Robot.create_robot(cmd)
print(bot.type)
Perhaps you could you two classes, one directly aligned with the Json (your source class) and the other having the actual structure you need. Then you could map them using the ObjectMapper class[https://pypi.org/project/object-mapper/]. This is very close to the MapStruct Library for Java.
ObjectMapper is a class for automatic object mapping. It helps you to create objects between project layers (data layer, service layer, view) in a simple, transparent way.
For a custom object I am able to encode into json using JSONEncoder.
class CustomEncoder(JSONEncoder):
def encode(self, custom):
prop_dict = {}
for prop in Custom.all_properties_names():
if custom.__getattribute__(prop) is not None:
if prop is 'created_timestamp':
prop_dict.update({prop: custom.__getattribute__(
prop).isoformat()})
else:
prop_dict.update({prop: custom.__getattribute__(prop)})
return prop_dict
To generate json, I am using json.dumps(custom, cls=CustomEncoder, indent=True)
Now I have a list of Custom class objects. How do convert the list to json?
custom_list = //get custom object list from service
How do I convert the whole list to json? Do I need to iterate and capture json of each custom object and append to a list with comma separated? I feel like there should be something straightforward I am missing here.
The custom encoder is called only when needed. If you have a custom thing that the JSON library thinks it can encode, like a string or dictionary, the custom encoder won't be called. The following example shows that encoding an object, or a list including an object, works with a single custom encoder:
import json
class Custom(object):
pass
class CustomEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, Custom):
return 'TASTY'
return CustomEncoder(self, o)
print json.dumps( Custom(), cls=CustomEncoder )
print json.dumps( [1, [2,'three'], Custom()], cls=CustomEncoder )
Output:
"TASTY"
[1, [2, "three"], "TASTY"]
In my way, I convert object to dict then using json.dumps list of dict:
def custom_to_dict(custom):
return {
'att1': custom.att1,
'att2': custom.att2,
...
}
#custom_list is your list of customs
out = json.dumps([custom_to_dict(custom) for custom in custom_list])
It might be helpful
I'm working on a trivial problem here to deserialize some JSON (I cannot change the format, it's not a service I created) into Python objects. I've managed to do the conversion using lambda's, but I'd like to use an object_hook now, to see if the it's possible to do a conversion using the json.loads method. However, that's where I'm failing right now, and I was wondering if someone could point me in the right direction.
This is the code I currently have:
import json
class Light:
def __init__(self, id, name):
self.id = id
self.name = name
response = '{"1": {"name": "bedroom"}, "2": {"name": "kitchen"}}'
def object_decoder(obj):
return Light(......)
print json.loads(response, object_hook=object_decoder)
As you can see, the response is one document with two keys, named 1 and 2. It would be nice if I can make the the code work in a way that the json.loads would return two Light objects, but at the moment, I'm stuck, and I don't know how to iterate over response to make this work.
object_hook won't help you, since you have id and name on the different levels in the json string:
object_hook, if specified, will be called with the result of every
JSON object decoded and its return value will be used in place of the
given dict.
Let's see why object_hook won't help. If you print objects that are coming to the object_decoder function, you'll see that it is going up from the deep, like this:
{u'name': u'bedroom'}
{u'name': u'kitchen'}
{u'1': None, u'2': None}
None
This means that you cannot join object_decoder calls in order to produce a Light instance.
How about using custom JSONDecoder class instead:
import json
class Light:
def __init__(self, id, name):
self.id = id
self.name = name
response = '{"1": {"name": "bedroom"}, "2": {"name": "kitchen"}}'
class Decoder(json.JSONDecoder):
def decode(self, s):
obj = super(Decoder, self).decode(s)
return [Light(id=k, name=v['name']) for k, v in obj.iteritems()]
lights = json.loads(response, cls=Decoder)
print lights # prints [<__main__.Light instance at 0x9c3b50c>, <__main__.Light instance at 0x9c3b56c>]
print [light.__dict__ for light in lights] # prints [{'id': u'1', 'name': u'bedroom'}, {'id': u'2', 'name': u'kitchen'}]
This is actually the same as making json.loads() and then instantiate classes after.
If you are able to change the format of the string, I suggest you use jsonpickle. I've founded it perfect for this sort of thing.
How can I get it to print out like below in a pythonic like way (I could manually craft it but feel like there is a better pythonic way), was originally thinking I could use dict for this but with my limited python knowledge, I'm guessing their is an easier way. My end goal is to create a json object via json.dumps()
Desired Result:
{'cars': [{'make': 'VW', 'model': 'Jetta'},{'make': 'BMW', 'model': 'X5'}], 'name': 'John Smith'}
Code:
class Person(object):
def __init__(self):
self.name = None
self.cars = []
class Car(object):
def __init__(self, make, model):
self.make = make
self.model = model
>>> p = Person()
>>> p.name = 'John Smith'
>>> p.cars.append(Car('VW', 'Jetta'))
>>> p.cars.append(Car('BMW' ,'X5'))
>>> p.__dict__
{'cars': [<__main__.Car object at 0x7fe5566e9490>, <__main__.Car object at 0x7fe5566e94d0>], 'name': 'John Smith'}
The json.dumps() method has a default keyword that lets you specify a method to convert unknown items to JSON. Make use of that:
def default(obj):
if isinstance(obj, (Car, Person)): # so if this is one of *your* objects
return obj.__dict__ # turn it into a python dict
raise TypeError # Sorry, don't know how to handle this
then use that for creating JSON:
json.dumps(p, default=default)
Result:
>>> json.dumps(p, default=default)
'{"cars": [{"make": "VW", "model": "Jetta"}, {"make": "BMW", "model": "X5"}], "name": "John Smith"}'
Add this to the class Car:
def __repr__(self):
return "{'make': %s, 'model': %s}" % (self.make, self.model)
EDIT:
You can also do:
def __repr__(self):
return str(self.__dict__)
But it will print all the attributes.
EDIT2:
I didn't realise that your goal is to generate a JSON file using json.dumps(). In that case, you should use Martijn Pieters' approach.