Develop Reference

How to get value or name of enum from SQLAlchemy result query?

I want to build an API for my project and return everything as JSON using Flask and SQLAlchemy. Unfortunately, SQLAlchemy did not return the query as JSON Seriazeble, so I'm using data classes to solve that problem. The code working and it returns JSON as I wanted. The problem occurs when I try to implement enum column like gender, because the enum column returns an enum object, so it's not JSON Seriazeble again. This is the code:
ActivityLevel.py
class GenderEnum(Enum):
p = 0
l = 1
#dataclass
class ActivityLevel(db.Model):
__tablename__ = "activity_level"
id: int
name: str
gender: GenderEnum
activity_score: float
date_created: str
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False)
gender = db.Column(db.Enum(GenderEnum), nullable=False)
activity_score = db.Column(
db.Float(precision=3, decimal_return_scale=2), nullable=False
)
date_created = db.Column(db.DateTime, default=datetime.utcnow)
ActivityLevelController.py
from flask import jsonify
from flask_restful import Resource
from models.ActivityLevel import ActivityLevel
class ActivityLevelController(Resource):
def get(self):
try:
activity = ActivityLevel().query.all()
result = {
"activity": activity
}
print(activity)
return jsonify(result)
except Exception as e:
print(e)
return jsonify({"message": "Error again"})
And this is the result of print(activity)
[
ActivityLevel(id=1, name='asdfasdf', gender=<GenderEnum.p: 0>, activity_score=12.0, date_created=datetime.datetime(2022, 8, 12, 10, 54, 58)),
ActivityLevel(id=2, name='qwerqwer', gender=<GenderEnum.l: 1>, activity_score=13.0, date_created=datetime.datetime(2022, 8, 12, 10, 54, 58))
]
As you can see, gender did not return l or p, and it return <GenderEnum.l: 1>. Which is correct as the documentation says https://docs.python.org/3/library/enum.html, when i call GenderEnum.l it will result just like the response.
What I want to ask is something like this:
Is there a way to override the return value of GenderEnum.l to the value or name by doing something in GenderEnum class?
Is there a way I can get the value or name of GenderEnum when I query the data from the database?
Or a way to make the query call the gender value or name as the default instead the enum object?
Thank you very much.

Is there a way to override the return value of GenderEnum.l to the value or name by doing something in GenderEnum class?
That's not really what Enums are meant to do, but you can access the name and value of your enum via the name and value attributes. In fact, SQLAlchemy will store the name of the Enum in the database in most cases.
Is there a way I can get the value or name of GenderEnum when I query the data from the database?
You define your model with an gender as an enum, so when loading, that is what you will get.
Or a way to make the query call the gender value or name as the default instead the enum object?
Same answer as above, you define gender as an enum, so SQLAlchemy gives it to you.
But to fix your problem, I can only recommend you look into dependency inversion.
Have a model which corresponds to your logic and allows you to solve the problem, then this model can be mapped to an entity, which allows you to store the model in whatever persistance layer, and finally a converter/serialiser for your model to load from/dump to your API.
This way, your model is not tied to where you store it or how you recieve it.
Slightly simplified example from your code:
# model.py
from dataclasses import dataclass, field
from enum import Enum, auto
class GenderEnum(Enum):
p = auto()
l = auto()
#dataclass
class ActivityLevel:
id: int = field(init=False) # NOTE add `repr=False` if trying to display instances
name: str
gender: GenderEnum
# orm.py
from sqlalchemy import Column, Enum, ForeignKey, Integer, String, Table
from sqlalchemy.orm import registry
from .model import ActivityLevel
mapper_registry = registry()
activity_level = Table(
"activity_level",
mapper_registry.metadata,
Column("id", Integer, primary_key=True),
Column("name", String(100), nullable=False),
Column("gender", Enum(GenderEnum), nullable=False),
)
def start_mapper():
mapper_registry.map_imperatively(ActivityLevel, activity_level)
# serialiser.py
from json import JSONDecoder, JSONEncoder
from .model import ActivityLevel, GenderEnum
class ActivityLevelJSONEncoder(JSONEncoder):
def default(self, o):
try: # NOTE duck typing and asking for forgiveness not permission
return {
"name": o.name,
"gender": o.gender.name,
}
except AttributeError:
return super().default(self, o)
class ActivityLevelJSONDecoder(JSONDecoder):
def __init__(self, *args, **kwargs):
super().__init__(*args, object_hook=self._object_hook, **kwargs)
#staticmethod
def _object_hook(d: dict) -> ActivityLevel:
return ActivityLevel(
name=d["name"],
gender=GenderEnum[d["gender"]],
)
Then in during application startup, start the mappers, and whenever needed json.dumps your ActivityLevel instances with the kwarg cls=ActivityLevelJSONEncoder.

This is what I found to temporarily fix my problem. I found this post and answer and follow the answer of #Vlad Bezden that uses the f-string method to override the return value of enum object when I call it like this GenderEnum.l from this <GenderEnum.l: 1> to just the value like 0 or 1. This is what I change in the enum class
class GenderEnum(int, Enum):
p = 0
l = 1
I add additional int. From what I noticed in the query behavior, the query just essentially calls enum object like this GenderEnum.l so that's why I got the response like that, just like what ljmc's says in the comment of his/her answer. That code initially overrides the response to the value of the enum object. If you want to return the name of the enum object, I'm terribly sorry, I can't seem to find a way to accomplish that.
After that it change the result to this
{
"activity": [
{
"date_created": "Fri, 12 Aug 2022 17:22:03 GMT",
"gender": 0,
"id": 1,
"name": "asdf",
"activity_score": 12.0
},
],
}
It works and I can return it as JSON. In contrast to that, I really not recommended this way to solve this problem, cause I find a hard way to do some relationship calls like join, and I really recommend anyone to follow Ijmc's answer to solving this problem, that's why I gonna accept Ijmc's answer as the correct one.
Thank you very much for helping me Ijmc for giving me insight, so I can come up with this little solution.
In any case of you guys wondering example how to do a join so i can return it as json, link to this post

DRF queryset to return specific field

I'm creating a django rest framework application with this structure (assuming imports are correct, so I omit them from the code below.
models.py:
class Door(models.Model):
type = models.CharField(max_length=40)
color = models.CharField(max_length=40)
serializers.py:
class DoorSerializer(serializers.ModelSerializer):
class Meta:
model = Door
fields = ['type', 'color']
views.py:
class DoorViewSet(viewsets.ModelViewSet):
serializer_class = DoorSerializer
queryset = Door.objects.all()
def get_queryset(self, *args, **kwargs):
queryset = Door.objects.all()
parameter = self.request.query_params.get('type', '')
if parameter:
return queryset.filter(type=parameter)
else:
return queryset
So far this behaves as intended, when I make an api call to localhost/Doors it lists all the doors. And when I make an api call to localhost/Doors/?type=big it lists all the doors that have the value "big" in their "type" field.
The addition I would like to make is another parameter check which would return a list of all the unique door types that exist in the database. This can be achieved in the manage.py shell by using: Door.objects.all().values('type').distinct()
My attempt was the following modifications to views.py:
...
parameter = self.request.query.params.get('type', '')
unique = self.request.query.params.get('unique', '')
if parameter:
...
elif unique:
return Door.objects.all().values('type').distinct()
...
My assumption was that this would return the same as Door.objects.all().values('type').distinct() when I make a call to localhost/Doors/?unique=whatever
However I am getting the error: "Got KeyError when attempting to get a value for field color on serializer DoorSerializer.\nThe serializer field might be named incorrectly and not match any attribute or key on the dict instance.\nOriginal exception text was: 'color'."
I assume this means that the serializer expects an object or a list of objects that contains all the fields of the corresponding model.
Is there some way I could circumvent this by fixing the view or should I create a different serializer? In either case, since I've gotten pretty confused with DRF / django differences and it is possible I won't be able to follow abstract instructions, could you provide a code solution that addresses the issue? Also, in the very likely case that my assumption is completely off, could you also explain what is causing the problem? Thank you for your time!
Edit for clarifying the desired result:
Assuming my database has 4 doors which are:
{
"id": 1,
"type": "big",
"color": "blue"
},
{
"id": 2,
"type": "big",
"color": "yellow"
},
{
"id": 3,
"type": "small",
"color": "green"
},
{
"id": 4,
"type": "big",
"color": "red"
},
I would like to make a get request to some url, for instance localhost/Doors/?unique=Yes and have the api return to me the list {"big", "small}

WRITING YOUR OWN VIEW: Short view that returns the list of type. You need to set up a new path here. I'd personally go for this option as the response you expect is way different to what the rest of your view does.
from rest_framework.decorators import api_view
from rest_framework.response import Response
#api_view()
def Unique_door_types(request):
types = Door.objects.values_list('type', flat=True).distinct()
return Response({"types": list(types)})
WITHOUT AN ADDITIONAL VIEW:
No need for additional view or serializer. Override the list method. Note that this is closer to a trick than to a good way of programming.
from rest_framework.response import Response
class DoorViewSet(viewsets.ModelViewSet):
serializer_class = DoorSerializer
def get_queryset(self, *args, **kwargs):
queryset = Door.objects.all()
parameter = self.request.query_params.get('type', '')
if parameter:
return queryset.filter(type=parameter)
else:
return queryset
def list(self, request):
unique = self.request.query_params.get('unique', '')
if unique:
types = Door.objects.values_list('type', flat=True).distinct()
return Response({"types": list(types)})
return super().list()

My suggestion would be to create a separate route like /doors/types/. You do this by adding a method to your DoorViewSet class with a #action decorator. See https://www.django-rest-framework.org/api-guide/viewsets/#marking-extra-actions-for-routing for more details about how to do this.

Serialising an Enum member to JSON

How do I serialise a Python Enum member to JSON, so that I can deserialise the resulting JSON back into a Python object?
For example, this code:
from enum import Enum
import json
class Status(Enum):
success = 0
json.dumps(Status.success)
results in the error:
TypeError: <Status.success: 0> is not JSON serializable
How can I avoid that?

I know this is old but I feel this will help people. I just went through this exact problem and discovered if you're using string enums, declaring your enums as a subclass of str works well for almost all situations:
import json
from enum import Enum
class LogLevel(str, Enum):
DEBUG = 'DEBUG'
INFO = 'INFO'
print(LogLevel.DEBUG)
print(json.dumps(LogLevel.DEBUG))
print(json.loads('"DEBUG"'))
print(LogLevel('DEBUG'))
Will output:
LogLevel.DEBUG
"DEBUG"
DEBUG
LogLevel.DEBUG
As you can see, loading the JSON outputs the string DEBUG but it is easily castable back into a LogLevel object. A good option if you don't want to create a custom JSONEncoder.

The correct answer depends on what you intend to do with the serialized version.
If you are going to unserialize back into Python, see Zero's answer.
If your serialized version is going to another language then you probably want to use an IntEnum instead, which is automatically serialized as the corresponding integer:
from enum import IntEnum
import json
class Status(IntEnum):
success = 0
failure = 1
json.dumps(Status.success)
and this returns:
'0'

If you want to encode an arbitrary enum.Enum member to JSON and then decode
it as the same enum member (rather than simply the enum member's value attribute), you can do so by writing a custom JSONEncoder class, and a decoding function to pass as the object_hook argument to json.load() or json.loads():
PUBLIC_ENUMS = {
'Status': Status,
# ...
}
class EnumEncoder(json.JSONEncoder):
def default(self, obj):
if type(obj) in PUBLIC_ENUMS.values():
return {"__enum__": str(obj)}
return json.JSONEncoder.default(self, obj)
def as_enum(d):
if "__enum__" in d:
name, member = d["__enum__"].split(".")
return getattr(PUBLIC_ENUMS[name], member)
else:
return d
The as_enum function relies on the JSON having been encoded using EnumEncoder, or something which behaves identically to it.
The restriction to members of PUBLIC_ENUMS is necessary to avoid a maliciously crafted text being used to, for example, trick calling code into saving private information (e.g. a secret key used by the application) to an unrelated database field, from where it could then be exposed (see https://chat.stackoverflow.com/transcript/message/35999686#35999686).
Example usage:
>>> data = {
... "action": "frobnicate",
... "status": Status.success
... }
>>> text = json.dumps(data, cls=EnumEncoder)
>>> text
'{"status": {"__enum__": "Status.success"}, "action": "frobnicate"}'
>>> json.loads(text, object_hook=as_enum)
{'status': <Status.success: 0>, 'action': 'frobnicate'}

In Python >= 3.7, can just use
json.dumps(enum_obj, default=str)
If you want to use the enum value, you can do
json.dumps(enum_obj, default=lambda x: x.value)
or if you want to use the enum name,
json.dumps(enum_obj, default=lambda x: x.name)

You just need to inherit from str or int class:
from enum import Enum, unique
#unique
class StatusEnum(int, Enum):
pending: int = 11
approved: int = 15
declined: int = 266
That's it, it will be serialised using any JSON encoder.

I liked Zero Piraeus' answer, but modified it slightly for working with the API for Amazon Web Services (AWS) known as Boto.
class EnumEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Enum):
return obj.name
return json.JSONEncoder.default(self, obj)
I then added this method to my data model:
def ToJson(self) -> str:
return json.dumps(self.__dict__, cls=EnumEncoder, indent=1, sort_keys=True)
I hope this helps someone.

If you are using jsonpickle the easiest way should look as below.
from enum import Enum
import jsonpickle
#jsonpickle.handlers.register(Enum, base=True)
class EnumHandler(jsonpickle.handlers.BaseHandler):
def flatten(self, obj, data):
return obj.value # Convert to json friendly format
if __name__ == '__main__':
class Status(Enum):
success = 0
error = 1
class SimpleClass:
pass
simple_class = SimpleClass()
simple_class.status = Status.success
json = jsonpickle.encode(simple_class, unpicklable=False)
print(json)
After Json serialization you will have as expected {"status": 0} instead of
{"status": {"__objclass__": {"py/type": "__main__.Status"}, "_name_": "success", "_value_": 0}}

You can even combine the solutions mentioned above with the automatic value creation for Enums. I use this in combination with Pydantic and FastAPI to provide lower case names for a REST API:
from enum import Enum, auto
import json
class StrEnum(str, Enum):
pass
# this creates nice lowercase and JSON serializable names
# https://docs.python.org/3/library/enum.html#using-automatic-values
class AutoNameLower(StrEnum):
def _generate_next_value_(name, start, count, last_values):
return name.lower()
class AutoNameLowerStrEnum(AutoNameLower):
pass
class MyActualEnum(AutoNameLowerStrEnum):
THIS = auto()
THAT = auto()
FOO = auto()
BAR = auto()
print(MyActualEnum.THIS)
print(json.dumps(MyActualEnum.THIS))
print(list(MyActualEnum))
Console:
>>> MyActualEnum.THIS
>>> "this"
>>> [<MyActualEnum.THIS: 'this'>, <MyActualEnum.THAT: 'that'>, <MyActualEnum.FOO: 'foo'>, <MyActualEnum.BAR: 'bar'>]

This worked for me:
class Status(Enum):
success = 0
def __json__(self):
return self.value
Didn't have to change anything else. Obviously, you'll only get the value out of this and will need to do some other work if you want to convert the serialized value back into the enum later.

Peewee model to JSON

I'm creating an API using peewee as the ORM and I need the ability to convert a peewee model object into a JSON object to send to the user. Does anyone know of a good way to do this?

Peewee has a model_to_dict and dict_to_model helpers in the playhouse.shortcuts extension module.
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#model_to_dict
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#dict_to_model
You could use these as follows:
from playhouse.shortcuts import model_to_dict, dict_to_model
user_obj = User.select().where(User.username == 'charlie').get()
json_data = json.dumps(model_to_dict(user_obj))
Also note that model_to_dict() can recurse through related models, include back-referenced models, and exclude certain fields from being serialized.

when single fetch
user = User.select().where(User.id == 1).get()
model_to_dict(user) #to Dict
when Multiple fetch
users = list(User.select().where(User.name ** 'a%').dicts())

also, you can get model as a dict, and then convert to json with correct field types (bool, int, float, etc.):
import peewee
import json
from bson import json_util
from datetime import datetime
class User(peewee.Model):
email = CharField()
status = BooleanField(default=True)
firstname = CharField()
lastname = CharField()
age = IntegerField()
created = DateTimeField(default=datetime.now())
class Meta:
database = db
user = User.select().dicts().get()
print json.dumps(user, default=json_util.default)

For anybody having issues like TypeError: Object of type date is not JSON serializable, this works for me (tested on Python 3.8.2).
from playhouse.shortcuts import model_to_dict
import json
def print_model(model):
print(json.dumps(model_to_dict(model), indent=4, sort_keys=True, default=str))
def print_models(models):
print(json.dumps(list(models.dicts()), indent=4, sort_keys=True, default=str))
Usage 1 - Single model
for person in Person.select():
print_model(person)
Usage 2 - Many models
print_models(Person.select())

I had this very same problem and ended up defining my own parser extension for JSON types that could not be automatically serialized. I'm fine for now in using strings as data represented (although you could possibly use different datatypes, but beware of approximation using floating points!
In the following example, I put this in a file called json_serialize.py inside a utils folder:
from decimal import Decimal
import datetime
try:
import uuid
_use_uuid = True
except ImportError:
_use_uuid = False
datetime_format = "%Y/%m/%d %H:%M:%S"
date_format = "%Y/%m/%d"
time_format = "%H:%M:%S"
def set_datetime_format(fmt_string):
datetime_format = fmt_string
def set_date_format(fmt_string):
date_format = fmt_string
def set_time_format(fmt_string):
time_format = fmt_string
def more(obj):
if isinstance(obj, Decimal):
return str(obj)
if isinstance(obj, datetime.datetime):
return obj.strftime(datetime_format)
if isinstance(obj, datetime.date):
return obj.strftime(date_format)
if isinstance(obj, datetime.time):
return obj.strftime(time_format)
if _use_uuid and isinstance(obj, uuid.UUID):
return str(obj.db_value())
raise TypeError("%r is not JSON serializable" % obj)
Then, in my app:
import json
from utils import json_serialize
...
json.dumps(model_to_dict(User.get()), default=json_serialize.more)
edit just to add: this is very largely inspired by json_utils.default module found in mongodb but mainly relies on the json module and needs no import of mongodb own bson/json_utils module.
Usually I update it to support new types as soon as my app raises the TypeError for it found a type not able to serialize

I usually implement the model to dict and dict to model functions, for maximum security and understanding of the inner workings of the code. Peewee does a lot of magic and you want to be in control over it.
The most obvious argument for why you should not iterate on the fields but rather explicitly specify them is because of security considerations. Not all fields can be exposed to the user, and I assume you need this functionality to implement some sort of REST API.
So - you should do something like this:
class UserData(db.Model):
user = db.ForeignKeyField(User)
data = db.CharField()
def serialize():
# front end does not need user ID here
return {
'data': self.data
}
#classmethod
def from_json(cls, json_data):
UserData.create(
# we enforce user to be the current user
user=current_user,
data=json_data['data']
)

you can do something like that:
class MyModel(peewee.Model):
def __str__(self):
r = {}
for k in self._data.keys():
try:
r[k] = str(getattr(self, k))
except:
r[k] = json.dumps(getattr(self, k))
return str(r)
class User(MyModel):
email = CharField()
status = CharField(default="enabled")
firstname = CharField()
lastname = CharField()
class Meta:
database = db

How can one customize Django Rest Framework serializers output?

I have a Django model that is like this:
class WindowsMacAddress(models.Model):
address = models.TextField(unique=True)
mapping = models.ForeignKey('imaging.WindowsMapping', related_name='macAddresses')
And two serializers, defined as:
class WindowsFlatMacAddressSerializer(serializers.Serializer):
address = serializers.Field()
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = WindowsFlatMacAddressSerializer(many=True)
clientId = serializers.Field()
When accessing the serializer over a view, I get the following output:
[
{
"id": 1,
"macAddresses": [
{
"address": "aa:aa:aa:aa:aa:aa"
},
{
"address": "bb:bb:bb:bb:bb:bb"
}
],
"clientId": null
}
]
Almost good, except that I'd prefer to have:
[
{
"id": 1,
"macAddresses": [
"aa:aa:aa:aa:aa:aa",
"bb:bb:bb:bb:bb:bb"
],
"clientId": null
}
]
How can I achieve that ?

Create a custom serializer field and implement to_native so that it returns the list you want.
If you use the source="*" technique then something like this might work:
class CustomField(Field):
def to_native(self, obj):
return obj.macAddresses.all()
I hope that helps.
Update for djangorestframework>=3.9.1
According to documentation, now you need override either one or both of the to_representation() and to_internal_value() methods. Example
class CustomField(Field):
def to_representation(self, value)
return {'id': value.id, 'name': value.name}

Carlton's answer will work do the job just fine. There's also a couple of other approaches you could take.
You can also use SlugRelatedField, which represents the relationship, using a given field on the target.
So for example...
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = serializers.SlugRelatedField(slug_field='address', many=True, read_only=True)
clientId = serializers.Field()
Alternatively, if the __str__ of the WindowsMacAddress simply displays the address, then you could simply use RelatedField, which is a basic read-only field that will give you a simple string representation of the relationship target.
# models.py
class WindowsMacAddress(models.Model):
address = models.TextField(unique=True)
mapping = models.ForeignKey('imaging.WindowsMapping', related_name='macAddresses')
def __str__(self):
return self.address
# serializers.py
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = serializers.RelatedField(many=True)
clientId = serializers.Field()
Take a look through the documentation on serializer fields to get a better idea of the various ways you can represent relationships in your API.