For a custom object I am able to encode into json using JSONEncoder.
class CustomEncoder(JSONEncoder):
def encode(self, custom):
prop_dict = {}
for prop in Custom.all_properties_names():
if custom.__getattribute__(prop) is not None:
if prop is 'created_timestamp':
prop_dict.update({prop: custom.__getattribute__(
prop).isoformat()})
else:
prop_dict.update({prop: custom.__getattribute__(prop)})
return prop_dict
To generate json, I am using json.dumps(custom, cls=CustomEncoder, indent=True)
Now I have a list of Custom class objects. How do convert the list to json?
custom_list = //get custom object list from service
How do I convert the whole list to json? Do I need to iterate and capture json of each custom object and append to a list with comma separated? I feel like there should be something straightforward I am missing here.
The custom encoder is called only when needed. If you have a custom thing that the JSON library thinks it can encode, like a string or dictionary, the custom encoder won't be called. The following example shows that encoding an object, or a list including an object, works with a single custom encoder:
import json
class Custom(object):
pass
class CustomEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, Custom):
return 'TASTY'
return CustomEncoder(self, o)
print json.dumps( Custom(), cls=CustomEncoder )
print json.dumps( [1, [2,'three'], Custom()], cls=CustomEncoder )
Output:
"TASTY"
[1, [2, "three"], "TASTY"]
In my way, I convert object to dict then using json.dumps list of dict:
def custom_to_dict(custom):
return {
'att1': custom.att1,
'att2': custom.att2,
...
}
#custom_list is your list of customs
out = json.dumps([custom_to_dict(custom) for custom in custom_list])
It might be helpful
Related
I have the following model:
class Car(models.Model):
data = models.TextField(default="[]")
Also, I have the following serializer:
class CarSerializer(serializers.ModelSerializer):
data = serializers.ListField(child=serializers.CharField())
The REST API gets data and saves it as text field. In my to_dict method of Car, I want to convert self.data into JSON and return the dict:
def to_dict(self):
result = dict()
result['data']= json.loads(self.data)
return result
But it fails with the error:
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
As I understand, the reason is that self.data is:
"['a', 'b', 'c']"
And not:
'["a", "b", "c"]'
I'm familiar with JsonField, but since I'm using SQLite without JSON1 externation, I can't use it. How can I convert self.data to JSON?
You can use python json.dumps() method to convert string into json format and then use json.loads() to convert json into python object.
import json
def to_dict(self):
result = dict()
data = json.dumps(self.data)
result['data'] = json.loads(data)
return result
The simplest way to solve this problem is json.loads(self.data.replace('\'','\"')).
Replace ' to ".
Or you can try eval(self.data)
you can watch a sample here about the usage of eval
I have a pydantic object definition that includes an optional field. I am looking to be able to configure the field to only be serialised if it is not None.
class MyObject(BaseModel):
id: str
msg: Optional[str] = None
pri: Optional[int] = None
MyObject(id="123").json() # ideal output: {"id": "123", "pri": null}
MyObject(id="123", msg="hello").json() # ideal output: {"id": "123", "msg": "hello", "pri": null}
I would like to be able to specify the field precisely, as this object will be nested, and there are other optional fields that should be returned, regardless of whether they are None or not.
The solution to set json option exclude_none to True won't work for this purpose.
you can't do such thing with pydantic and even with more powerfull lib like attrs. The why may be because it is not a good way of returning json object, it is realy confusing for you, the api client and your test suite.
you may get some inspiration from elegant-way-to-remove-fields-from-nested-dictionaries.
you would be able to achieve something (not recommanded at all) by parsing your object jsoned and remove fiels folowing a logic.
exemple of key/value manipulation in nested dict:
import re
def dict_key_convertor(dictionary):
"""
Convert a dictionary from CamelCase to snake_case
:param dictionary: the dictionary given
:return: return a dict
"""
if not isinstance(dictionary, (dict, list)):
return dictionary
if isinstance(dictionary, list):
return [dict_key_convertor(elem) for elem in dictionary]
return {to_snake(key): dict_key_convertor(data) for key, data in dictionary.items()}
def to_snake(word) -> str:
"""
Convert all word from camel to snake case
:param word: the word given to be change from camelCase to snake_case
:return: return word variable in snake_case
"""
return re.sub(r'([A-Z]{2,}(?=[a-z]))', '\\1_', re.sub(r'([a-z])([A-Z]+)', '\\1_\\2', word)).lower()
with a bit of work you may achive something with this:
from typing import List
def dict_key_cleaner(dictionary):
if not isinstance(dictionary, (dict, list)):
return dictionary
if isinstance(dictionary, list):
return [dict_key_cleaner(elem) for elem in dictionary]
# change this return to work with dict
return {poper(key, dictionary): dict_key_cleaner(data) for key, data in dictionary.items()}
def poper(key, dictionary):
special_keys: List[str] = ["field_name","field_name1","field_name2"]
# do some stuff here
for spe_key in special_keys:
if key == spe_key and key.key_value is None:
dictionary.pop(key)
# add return of modified dict
i am trying convert json string to model
then it is easy to get value with .
i have checked another question
but different, my json sting looks like,
{
"id":"123",
"name":"name",
"key":{
"id":"345",
"des":"des"
},
}
i prefer to use 2 class like,
class A:
id = ''
name = ''
key = new B()
class B:
id = ''
des = ''
There are few libraries that might help:
marshmallow is nice
colander from Pylons
schematics
For easier cases you can also use something from standard library like
named tuples and one from collections which is available also in py2
SimpleNamespace
In order to do that you should provide your custom callback as an object_hook argument to the json.loads function.
object_hook is an optional function that will be called with the
result of any object literal decode (a dict). The return value of
object_hook will be used instead of the dict. This feature
can be used to implement custom decoders (e.g. JSON-RPC class hinting).
Consider using collections.namestuple subclasses:
json_str = '''
{
"id":"123",
"name":"name",
"key":{
"id":"345",
"des":"des"
}
}'''
B = collections.namedtuple('B', 'id des')
A = collections.namedtuple('A', 'id name key')
def make_models(o):
if 'key' in o:
return A(o['id'], o['name'], B(id=o['key']['id'], des=o['key']['des']))
else:
return o
result = json.loads(json_str, object_hook=make_models)
print(type(result)) # outputs: <class '__main__.A'>
print(result.id) # outputs: 123
print(result.key.id) # outputs: 345
How do I convert a MappedClass object to JSON?
json.dumps({"data": mappedClassObject}, indent=True)
Above code raises error that MappedClass object is not JSON serializable.
Is there any utility similar to BSON's json_util to convert MappedClass to JSON? or Do I have to write a encoder as mentioned in TypeError: ObjectId('') is not JSON serializable
Am using python 2.7
As of now, I have created a encoder for my use. This is hacky and if there is a better alternate, I will consider it.
# MappedClass from Ming is not JSON serializable and we do not have utilities like json_util (for BSON) to convert
# MappedClass object to JSON. Below encoder is created for this requirement.
# Usage: MappedClassJSONEncoder().encode(data)
class MappedClassJSONEncoder(JSONEncoder):
"""
Returns a MappedClass object JSON representation.
"""
def _get_document_properties(self, klass):
"""
Returns the declared properties of the MappedClass's child class which represents Mongo Document
Includes only the user declared properties such as tenantId, _id etc
:param klass:
:return:
"""
return [k for k in dir(klass) if k not in dir(MappedClass)]
def _get_attr_json_value(self, attr):
if isinstance(attr, bson.objectid.ObjectId):
return str(attr)
elif isinstance(attr, datetime.datetime):
return attr.isoformat()
elif isinstance(attr, dict):
dict_data = {}
for member in attr:
dict_data.update({member: self._get_attr_json_value(attr[member])})
return dict_data
else:
return attr
def default(self, o):
mapped_class_attributes = self._get_document_properties(type(o))
attributes_data = {}
for attr_name in mapped_class_attributes:
attr = o.__getattribute__(attr_name)
attributes_data.update({attr_name: self._get_attr_json_value(attr)})
return attributes_data
How do I serialise a Python Enum member to JSON, so that I can deserialise the resulting JSON back into a Python object?
For example, this code:
from enum import Enum
import json
class Status(Enum):
success = 0
json.dumps(Status.success)
results in the error:
TypeError: <Status.success: 0> is not JSON serializable
How can I avoid that?
I know this is old but I feel this will help people. I just went through this exact problem and discovered if you're using string enums, declaring your enums as a subclass of str works well for almost all situations:
import json
from enum import Enum
class LogLevel(str, Enum):
DEBUG = 'DEBUG'
INFO = 'INFO'
print(LogLevel.DEBUG)
print(json.dumps(LogLevel.DEBUG))
print(json.loads('"DEBUG"'))
print(LogLevel('DEBUG'))
Will output:
LogLevel.DEBUG
"DEBUG"
DEBUG
LogLevel.DEBUG
As you can see, loading the JSON outputs the string DEBUG but it is easily castable back into a LogLevel object. A good option if you don't want to create a custom JSONEncoder.
The correct answer depends on what you intend to do with the serialized version.
If you are going to unserialize back into Python, see Zero's answer.
If your serialized version is going to another language then you probably want to use an IntEnum instead, which is automatically serialized as the corresponding integer:
from enum import IntEnum
import json
class Status(IntEnum):
success = 0
failure = 1
json.dumps(Status.success)
and this returns:
'0'
If you want to encode an arbitrary enum.Enum member to JSON and then decode
it as the same enum member (rather than simply the enum member's value attribute), you can do so by writing a custom JSONEncoder class, and a decoding function to pass as the object_hook argument to json.load() or json.loads():
PUBLIC_ENUMS = {
'Status': Status,
# ...
}
class EnumEncoder(json.JSONEncoder):
def default(self, obj):
if type(obj) in PUBLIC_ENUMS.values():
return {"__enum__": str(obj)}
return json.JSONEncoder.default(self, obj)
def as_enum(d):
if "__enum__" in d:
name, member = d["__enum__"].split(".")
return getattr(PUBLIC_ENUMS[name], member)
else:
return d
The as_enum function relies on the JSON having been encoded using EnumEncoder, or something which behaves identically to it.
The restriction to members of PUBLIC_ENUMS is necessary to avoid a maliciously crafted text being used to, for example, trick calling code into saving private information (e.g. a secret key used by the application) to an unrelated database field, from where it could then be exposed (see https://chat.stackoverflow.com/transcript/message/35999686#35999686).
Example usage:
>>> data = {
... "action": "frobnicate",
... "status": Status.success
... }
>>> text = json.dumps(data, cls=EnumEncoder)
>>> text
'{"status": {"__enum__": "Status.success"}, "action": "frobnicate"}'
>>> json.loads(text, object_hook=as_enum)
{'status': <Status.success: 0>, 'action': 'frobnicate'}
In Python >= 3.7, can just use
json.dumps(enum_obj, default=str)
If you want to use the enum value, you can do
json.dumps(enum_obj, default=lambda x: x.value)
or if you want to use the enum name,
json.dumps(enum_obj, default=lambda x: x.name)
You just need to inherit from str or int class:
from enum import Enum, unique
#unique
class StatusEnum(int, Enum):
pending: int = 11
approved: int = 15
declined: int = 266
That's it, it will be serialised using any JSON encoder.
I liked Zero Piraeus' answer, but modified it slightly for working with the API for Amazon Web Services (AWS) known as Boto.
class EnumEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Enum):
return obj.name
return json.JSONEncoder.default(self, obj)
I then added this method to my data model:
def ToJson(self) -> str:
return json.dumps(self.__dict__, cls=EnumEncoder, indent=1, sort_keys=True)
I hope this helps someone.
If you are using jsonpickle the easiest way should look as below.
from enum import Enum
import jsonpickle
#jsonpickle.handlers.register(Enum, base=True)
class EnumHandler(jsonpickle.handlers.BaseHandler):
def flatten(self, obj, data):
return obj.value # Convert to json friendly format
if __name__ == '__main__':
class Status(Enum):
success = 0
error = 1
class SimpleClass:
pass
simple_class = SimpleClass()
simple_class.status = Status.success
json = jsonpickle.encode(simple_class, unpicklable=False)
print(json)
After Json serialization you will have as expected {"status": 0} instead of
{"status": {"__objclass__": {"py/type": "__main__.Status"}, "_name_": "success", "_value_": 0}}
You can even combine the solutions mentioned above with the automatic value creation for Enums. I use this in combination with Pydantic and FastAPI to provide lower case names for a REST API:
from enum import Enum, auto
import json
class StrEnum(str, Enum):
pass
# this creates nice lowercase and JSON serializable names
# https://docs.python.org/3/library/enum.html#using-automatic-values
class AutoNameLower(StrEnum):
def _generate_next_value_(name, start, count, last_values):
return name.lower()
class AutoNameLowerStrEnum(AutoNameLower):
pass
class MyActualEnum(AutoNameLowerStrEnum):
THIS = auto()
THAT = auto()
FOO = auto()
BAR = auto()
print(MyActualEnum.THIS)
print(json.dumps(MyActualEnum.THIS))
print(list(MyActualEnum))
Console:
>>> MyActualEnum.THIS
>>> "this"
>>> [<MyActualEnum.THIS: 'this'>, <MyActualEnum.THAT: 'that'>, <MyActualEnum.FOO: 'foo'>, <MyActualEnum.BAR: 'bar'>]
This worked for me:
class Status(Enum):
success = 0
def __json__(self):
return self.value
Didn't have to change anything else. Obviously, you'll only get the value out of this and will need to do some other work if you want to convert the serialized value back into the enum later.