Develop Reference

Ordering by computed field in admin panel

I'm learning Django and I got stuck on this problem
I'm basically trying to order by a computed field in an admin panel view, I've read some "tutorials" like this: https://books.agiliq.com/projects/django-admin-cookbook/en/latest/sorting_calculated_fields.html on google but I can't seem to figure out how it all works (between annotations etc)
Here's my classes:
class StockAdmin(admin.ModelAdmin):
list_display = ("ticker_symbol", "security_name", "current_pretax_yield", "current_aftertax_yield", "displayed_price")
search_fields = ("ticker_symbol", "security_name")
def current_pretax_yield(self, obj):
try:
curyield = float(obj.coupon_amount/obj.last_price)
return str(round((curyield*100),3)) + "%"
except:
return "n/a"
def current_aftertax_yield(self, obj):
try:
withholding_tax = 15
at_yield = ((obj.coupon_amount/obj.last_price*100)*(1-(withholding_tax/100))*0.74)
return str(round(at_yield, 2)) + "%"
except:
return "n/a"
def get_queryset(self, request):
queryset = super().get_queryset(request)
queryset = queryset.annotate(
_current_aftertax_yield=self.current_aftertax_yield(),
_current_pretax_yield=self.current_pretax_yield(),
)
current_aftertax_yield.admin_order_field = '_current_aftertax_yield'
current_pretax_yield.admin_order_field = '_current_pretax_yield'
Basically, I want to get "coupon amount" and "last price" fields from the database, perform the calculations you see in the functions, then display those calculated files in the admin panel and be able to "order by" them
The code as I have now errors out with a TypeError: current_aftertax_yield() missing 1 required positional argument: 'obj'
I've tried to follow this: https://books.agiliq.com/projects/django-admin-cookbook/en/latest/sorting_calculated_fields.html but I can't quite figure it out on my own..
Any ideas? Is there an easier way of doing this? I used a lot of computed values in PHP and it was trivial to implement!

You can not use methods to annotate a Queryset. You should specify an expression constructed an expression for the database. You can not use a method for that. You can make use of combinations of F-expressions together with certain aggregates, etc.
Here both the current_pretax_yield and current_aftertax_yield scale with coupon_amount/last_price, so we can make an annotation, and then sort by that annotation:
from django.db.models import F
class StockAdmin(admin.ModelAdmin):
list_display = ("ticker_symbol", "security_name", "current_pretax_yield", "current_aftertax_yield", "displayed_price")
search_fields = ("ticker_symbol", "security_name")
def current_pretax_yield(self, obj):
try:
curyield = float(obj.coupon_amount/obj.last_price)
return str(round((curyield*100),3)) + "%"
except:
return "n/a"
def current_aftertax_yield(self, obj):
try:
withholding_tax = 15
at_yield = ((obj.coupon_amount/obj.last_price*100)*(1-(withholding_tax/100))*0.74)
return str(round(at_yield, 2)) + "%"
except:
return "n/a"
def get_queryset(self, request):
return super().get_queryset(request).annotate(
_yield=F('coupon_amount')/F('last_price')
)
current_aftertax_yield.admin_order_field = '_yield'
current_pretax_yield.admin_order_field = '_yield'

Share data between method in serializer (Django Rest Framework)

I have quite heavy calculation and a lot of database access but reusable in SerializerMethodField of ModelSerializer, how should I result that calculated data across the method.
Example Code
class MySerializer(serializers.ModelSerializer):
a = serializers.SerializerMethodField()
b = serializers.SerializerMethodField()
def get_a(self, obj):
result_x = heavy_calculation(self.context['request'].user, obj)
return result_x + 1
def get_b(self, obj):
result_x = heavy_calculation(self.context['request'].user, obj)
return result_x + 2
heavy_calculation() in both method is the same function so result_x would be the same answer for any requests. I would like to make heavy_calculation() call only once per requests to reduce load work (my real work would like to call 10+ times)
Best Regards,

There're few options for your case:
If you could rewrite your serializer class:
class MySerializer(serializers.ModelSerializer):
a = serializers.IntegerField()
def calc_a(self, obj):
return self._cached_result + 1
def to_representation(self, instance):
data = super().to_representation(instance)
self._cached_result = heavy_calculation(self.context['request'].user, instance)
data['a'] = self.calc_a(instance)
...
return data
Using python3 LRU cache (https://docs.python.org/3/library/functools.html#functools.lru_cache)
#lru_cache
def heavy_calculation(user, obj):
...
There're django-memoize libary that cache your function result (you'll need to setup cache for your Django site)
#memoize(timeout=60)
def heavy_calculation(user, obj):
...

There are several ways to achieve this.
1. You can override init and populate data with object_id and result in a dictionary.
2. You can write list serializer and override data property and populate on your own. Guide of writing list serializer here

Can't add extra argument to Python(Django) function call

I need to serialize data with a serialiser and also I have a file for saving but I can't path the extra varible with file to my serializer
issue_dict = request.data.get('issue')
file = request.data.get('file')
This is working fine:
serializer = WriteIssueSerializer(data=issue_dict, context=self.get_serializer_context())
This is what I'd like to get, but it says "got an unexpected keyword argument 'file'" :
serializer = WriteIssueSerializer(data=issue_dict, file=file, context=self.get_serializer_context())
I understand that inside the serializer I should define variable "file",
so look at this serializer:
class WriteIssueSerializer(serializers.ModelSerializer):
notes = IssueNoteSerializer(many=True)
def create(self, val):
issue_dict = val.get('issue')
# issue_dict['assigned_to'] = issue_dict['assigned_to']['id']
# issue_dict['reported_by'] = issue_dict['reported_by']['id']
assigned_to_id = issue_dict.pop('assigned_to').id
reported_by_id = issue_dict.pop('reported_by').id
notes_info = issue_dict.pop('notes')
# print(validated_data.pop('file'))
issue = Issue.objects.create(assigned_to_id=assigned_to_id, reported_by_id=reported_by_id, **issue_dict)
for note_info in notes_info:
note = IssueNote.objects.create(**note_info)
note.issue = issue
It's obvious that changing from
def create(self, val): to def create(self, val, file): will fix my error, but not, the error is still the same

serializer = WriteIssueSerializer(data=issue_dict, file=file, context=self.get_serializer_context())
This calls the constructor of WriteIssueSerializer (__init__()), not .create(). So you have to create the extra argument in there, or call .create().

Convert list into queryset

To improve performance, In my project most of the model instances are stored in cache as list values. But all generic views in Django Rest Framework expect them to be queryset objects. How can I convert the values I got from list into a queryset like objeccts, such that I can use generic views.
Say, I have a function like
def cache_user_articles(user_id):
key = "articles_{0}".format(user_id)
articles = cache.get(key)
if articles is None:
articles = list(Article.objects.filter(user_id = user_id))
cache.set(key, articles)
return articles
In my views.py,
class ArticleViewSet(viewsets.ModelViewSet):
...
def get_queryset(self, request, *args, **kwargs):
return cache_user_articles(kwargs.get(user_id))
But, this of course this wouldn't work as Django Rest Framework expects the result of get_queryset to be QuerySet object, and on PUT request it would call 'get' method on it. Is there any way, I could make it to work with generic DRF views.

That's the place where Python like dynamic languages really shine due to Duck Typing. You could easily write something that quacks like a QuerySet.
import mongoengine
from bson import ObjectId
class DuckTypedQuerySet(list):
def __init__(self, data, document):
if not hasattr(data, '__iter__') or isinstance(data, mongoengine.Document):
raise TypeError("DuckTypedQuerySet requires iterable data")
super(DuckTypedQuerySet, self).__init__(data)
self._document = document
#property
def objects(self):
return self
def _query_match(self, instance, **kwargs):
is_match = True
for key, value in kwargs.items():
attribute = getattr(instance, key, None)
if isinstance(attribute, ObjectId) and not isinstance(value, ObjectId):
attribute = str(attribute)
if not attribute == value:
is_match = False
break
return is_match
def filter(self, **kwargs):
data = filter(lambda instance: self._query_match(instance, **kwargs), self)
return self.__class__(data, self._document)
def get(self, **kwargs):
results = self.filter(**kwargs)
if len(results) > 1:
raise self._document.MultipleObjectsReturned("{0} items returned, instead of 1".format(len(results)))
if len(results) < 1:
raise self._document.DoesNotExist("{0} matching query does not exist.".format(str(self._document)))
return results[0]
def first(self):
return next(iter(self), None)
def all(self):
return self
def count(self):
return len(self)
def cache_user_articles(user_id):
key = "articles_{0}".format(user_id)
articles = cache.get(key)
if articles is None:
articles = DuckTypedQuerySet(list(Article.objects.filter(user_id = user_id)), document = Article)
cache.set(key, articles)
return articles
Ofcourse, this is not an exhaustive implementation. You might need to add other methods that exists in queryset. But I think these will do for simple use-cases. Now you can make do with generic implementations of Django Rest Framework.

What about this?
(I have mocked the redis cache with a class variable)
class CachedManager(models.Manager):
cache = dict()
def cached(self, user_id):
cached = self.cache.get(user_id, [])
if not cached:
self.cache[user_id] = [article.pk for article in self.filter(user_id=user_id)]
return self.cache[user_id]
class Article(models.Model):
objects = CachedManager()
user_id = models.IntegerField()
# Whatever fields your Article model has
Then in your views or wherever you need it:
you can call Article.objects.cached(<a user id>)

Automatically strip() all values in WTForms?

Is there any way to strip surrounding whitespace from all values in WTForms without adding a filter to every single field?
Currently I'm passing filters=[strip_whitespace] with the function shown below to my fields but having to repeat this for every field is quite ugly.
def strip_whitespace(s):
if isinstance(s, basestring):
s = s.strip()
return s
A solution requiring subclassing of Form would be fine since I'm already doing that in my application.

You can do it in WTForms 2.x by using the bind_field primitive on class Meta. The class Meta paradigm is a way to override WTForms behaviors in contexts such as binding/instantiating fields, rendering fields, and more.
Because anything overridden in class Meta defined on a Form is inherited to any form subclasses, you can use it to set up a base form class with your desired behaviors:
class MyBaseForm(Form):
class Meta:
def bind_field(self, form, unbound_field, options):
filters = unbound_field.kwargs.get('filters', [])
filters.append(my_strip_filter)
return unbound_field.bind(form=form, filters=filters, **options)
def my_strip_filter(value):
if value is not None and hasattr(value, 'strip'):
return value.strip()
return value
Now, just inherit MyBaseForm for all your forms and you're good to go.

Unfortunately, I have no enough reputation to comment first response.
But, there is extremely unpleasant bug in that example:
When you do filters.append(smth) then on each form initialization filters growth by 1 element.
As a result, your code works slower and slower until you restart it
Consider Example:
class MyBaseForm(Form):
class Meta:
def bind_field(self, form, unbound_field, options):
filters = unbound_field.kwargs.get('filters', [])
filters.append(my_strip_filter)
return unbound_field.bind(form=form, filters=filters, **options)
def my_strip_filter(value):
if value is not None and hasattr(value, 'strip'):
return value.strip()
return value
class MyCustomForm(MyBaseForm):
some_field = StringField(filters=[lambda x: x])
for i in range(100):
MyCustomForm(MultiDict({'some_field': 'erer'}))
print(len(MyCustomForm.some_field.kwargs['filters'])) # print: 101
So the fast fix is to check that this filter not in list:
class MyBaseForm(Form):
class Meta:
def bind_field(self, form, unbound_field, options):
filters = unbound_field.kwargs.get('filters', [])
if my_strip_filter not in filters:
filters.append(my_strip_filter)
return unbound_field.bind(form=form, filters=filters, **options)

I wouldn't be surprised if you could do it by subclassing form, but my solution was to just create custom Stripped* fields. I think this is at least better than passing filters every time because it is less error prone:
from wtforms import StringField, PasswordField
class Stripped(object):
def process_formdata(self, valuelist):
if valuelist:
self.data = valuelist[0].strip()
else:
self.data = ''
class StrippedStringField(Stripped, StringField): pass
class StrippedPasswordField(Stripped, PasswordField): pass