How to distinct a specified field in Django? - python

I'm working with django and mysql.
Say there's a table as below:
class OrderInfo(models.Model):
gg_account_id = models.CharField(max_length=45, blank=True, null=True)
order_status = models.IntegerField(blank=True, null=True)
gg_status = models.IntegerField(blank=True, null=True)
uid = models.IntegerField(blank=True, null=True)
class Meta:
managed = False
db_table = 'order_info'
And data saved in it are:
id gg_account_id order_status gg_status uid
1 6270491342 2 0 1
2 12321323 2 0 34
3 12321323 2 0 34
4 55551233 1 0 54
5 55551233 2 0 54
6 55551233 2 0 54
7 55551233 2 0 54
If there are more than one data with same gg_account_id I want to get only one of them. My expected output should be like :
1 6270491342 1
2 12321323 34
5 55551233 54
and here's my trial with orm query:
recharge_account_list = OrderInfo.objects.\
filter(order_status=2, gg_status=0).\
distinct("gg_account_id").\
values_list("gg_account_id", "uid", "id")
print(recharge_account_list)
But I got error always
File "D:\virtual\Envs\smb_middle_server\lib\site-packages\django\db\backends\base\operations.py", line 171, in distinct
_sql
raise NotSupportedError('DISTINCT ON fields is not supported by this database backend')
django.db.utils.NotSupportedError: DISTINCT ON fields is not supported by this database backend
How can I get expected result?
Thanks

Solution 1: use forloop
gg_account_ids = set()
recharge_accounts = []
for i in recharge_account_list:
if i[0] not in gg_account_ids:
gg_account_ids.add(i[0])
recharge_accounts.append(i)
recharge_account_list = recharge_accounts
Solution 2: use raw SQL
recharge_account_list = OrderInfo.objects.raw('SELECT ... FROM Order_info GROUP BY ...')

It can be observed that you are using mysql as a database.
However, MySQL does not support distinct on the field. I mean by distinct('field_name') it only supports generic distinct So you can do only distinct() operation but not on specific fields.
distinct documentation Django
Similar question
Furthermore, you can achieve that by using group by specific fields.
Distinct with groupby

Related

Django-python- TestCase - transactions should never have a timestamp in the Future

I am working on a payment system that is registering transactions and timestamps. I would like to make a test to ensure that transactions are only made on a past date - it should not be possible to have a transaction with a future date.
models.py
20 class Ledger(models.Model):
19 account = models.ForeignKey(Account, on_delete=models.PROTECT)
18 transaction = models.ForeignKey(UID, on_delete=models.PROTECT)
17 amount = models.DecimalField(max_digits=10, decimal_places=2)
16 timestamp = models.DateTimeField(auto_now_add=True, db_index=True)
15 text = models.TextField()
14
13 #classmethod
12 def transfer(cls, amount, debit_account, debit_text, credit_account, credit_text, is_loan=False) -> int:
11 assert amount >= 0, 'Negative amount not allowed for transfer.'
10 with transaction.atomic():
9 if debit_account.balance >= amount or is_loan:
8 uid = UID.uid
7 cls(amount=-amount, transaction=uid, account=debit_account, text=debit_text).save()
6 cls(amount=amount, transaction=uid, account=credit_account, text=credit_text).save()
5 else:
4 raise InsufficientFunds
3 return uid
2
1 def __str__(self):
122 return f'{self.amount} :: {self.transaction} :: {self.timestamp} :: {self.account} :: {self. text}'
I honestly don't even know where to start testing this class because Testing is very new to me. I have tested some things I saw online close to the below, but cannot see anything happening. Does it make any sense? Maybe there is something that makes more sense testing here instead... Suggestions are more than welcome!
tests.py
class LedgerModelTestCase(TestCase):
# def setUp(self):
1 # no data to test yet
2 def transfer_completed(self):
3 time = timezone.now() + datetime.timedelta(days=30)
4 print(time)
5 future_transfer = Ledger(timestamp=time)
6 print(future_transfer)
7 self.assertIs(future_transfer.was_published_recently(), False)
8

Django: How to retrieve a Queryset of the latest, uniquely named records with a MySQL database

I require a query that can search my model for the latest, unique records and return a queryset of the results.
I need distinct on setup_name and latest on updated
models.py
class VehicleSetupModel(models.Model):
inertia = models.IntegerField()
tyre_pressure = models.IntegerField()
class StandardVehicleSetupModel(VehicleSetupModel):
setup_name = models.CharField(max_length=20)
updated = models.DateTimeField(auto_now=True)
vehicle = models.ForeignKey(VehicleModel, on_delete=models.CASCADE)
What I've tried
I tried the following:
StandardVehicleSetupModel.objects.all().order_by('updated').distinct('setup_name')
but MySQL does not support DISTINCT ON querys.
Table Layout (DD-MM-YYYY)
setup_name | updated | vehicle
----------------------------------------------------------
TEH 10-10-2020 1
TEH 11-10-2020 1
TEL 10-10-2020 1
TEL 08-10-2020 1
TEL 01-10-2020 1
TEP 01-10-2020 1
Expected Result (DD-MM-YYYY)
setup_name | updated | vehicle
----------------------------------------------------------
TEH 11-10-2020 1
TEL 10-10-2020 1
TEP 01-10-2020 1
Yes, MySQL doesn't support distinct() with arguments. We can use another way:
from django.db.models import Max
results = (
StandardVehicleSetupModel.objects
.values('setup_name')
.annotate(max_id=Max('id'))
.values_list('max_id', 'setup_name')
.order_by('max_id')
)
Not tested yet, but it should work ^^

Django - How to do complex query with left join and coalesce?

Scenario: Showing all voucher that a user can apply.
I have 2 tables Voucher (with all information of a voucher) and VoucherCustomer (listing number of vouchers that a user has used)
A validation voucher that can show to user on the application should be
Within use-able duration
Do not exceed number of times used within a day
Do not exceed number of times used per user
Do not exceed number of times used for a voucher
Must active
Here is my model:
class Voucher(models.Model):
code = models.ForeignKey('VoucherCustomer', related_name= 'voucher_code', on_delete = models.CASCADE) (1)
start_at = models.DateTimeField() (2)
end_at = models.DateTimeField() (3)
usage_limit_per_customer = models.BigIntegerField() (4)
times_used = models.BigIntegerField() (5)
usage_limit_daily = models.BigIntegerField() (6)
times_used_daily = models.BigIntegerField() (7)
is_global = models.BooleanField(blank=True, null=True) (8)
is_active = models.BooleanField() (9)
class VoucherCustomer(models.Model):
voucher_code = models.CharField(max_length = 255, primary_key=True) (1)
customer_id = models.IntegerField() (2)
times_used = models.BigIntegerField(blank=True, null=True) (3)
created_at = models.DateTimeField(blank=True, null=True) (4)
updated_at = models.DateTimeField(blank=True, null=True) (5)
Here is the sample data:
+++++++ Voucher ++++++++
(1) (2) (3) (4) (5) (6) (7) (8) (9)
TEST01 | 2020-11-30 17:00:00 | 2021-03-01 16:59:59 | 100 | 1124 | 5000 | 6 | true | true
+++++++ VoucherCustomer ++++++++
(1) (2) (3) (4) (5)
TEST01 10878 9 2020-12-03 02:17:32.012722 2020-12-08 10:32:03.877349
TEST01 12577 1 2020-12-02 07:17:34.005964 2020-12-02 07:17:34.005964
TEST01 8324 18 2020-12-02 07:49:37.385682 2021-02-01 14:35:38.096381
TEST01 7638 2 2020-12-02 08:17:46.532566 2020-12-02 08:17:46.532566
TEST01 3589 1 2020-12-02 14:57:01.356616 2020-12-02 14:57:01.356616
My expected query:
SELECT v.*
FROM leadtime.voucher v
LEFT JOIN leadtime.voucher_customer vc ON v.code = vc.voucher_code AND vc.customer_id in ({input_customer_id})
WHERE 1=1
AND v.start_at <= CURRENT_TIMESTAMP
AND v.end_at >= CURRENT_TIMESTAMP
AND v.is_active = TRUE
AND v.times_used < v.usage_limit
AND v.times_used_daily < v.usage_limit_daily
AND (v.customer_id = {input_customer_id} OR v.is_global)
AND COALESCE(vc.times_used,0) < usage_limit_per_customer
ORDER BY created_at
Here is my code on Django:
from django.db.models import Q, F, Value
from django.db.models.functions import Coalesce
now = datetime.now()
customer_input = customer_id = request.GET.get('customer_id')
query = Voucher.objects.filter(
start_at__lte = now,
end_at__gte = now,
is_active = True,
times_used__lt = F('usage_limit'),
times_used_daily__lt = F('usage_limit_daily'),
Q(customer_id = customer_id_input ) | Q(is_global = True),
VoucherCustomer__customer_id = customer_id_input ,
Coalesce(VoucherCustomer__times_used, Value(0)) <= F('usage_limit_per_customer')
).order_by('created_at').values()
Obviously, it does not work.
I got this error:
File "/code/app_test/views.py", line 467
).order_by('created_at').values()
^
SyntaxError: positional argument follows keyword argument
Anyway, since I am just a beginner, if there are parts that I could improve in my code please feel free to tell me.
------------ Updated ----------------
The current code is not working as I received this err
Cannot resolve keyword 'VoucherCustomer' into field. Choices are: airport, arrival_airport, code, code_id, content, created_at, customer_id, delivery_type, departure_airport, description, discount_amount, discount_type, end_at, from_time, id, is_active, is_global, max_discount_amount, start_at, times_used, times_used_daily, tittle, to_time, updated_at, usage_limit, usage_limit_daily, usage_limit_per_customer
I tried to change the model of VoucherCustomer into this one but still not working.
class VoucherCustomer(models.Model):
voucher_code = models.ManyToOneRel(field = "voucher_code", field_name = "voucher_code", to = "")
......................
class Meta:
managed = False
db_table = 'voucher_customer'
The code is not syntetically correct, you can't use <= inside a method signature, ie use that in filter(), also you need to pass the arguments before passing the keyword arguments through the function, ie some_function(a, b, x=y).
But, you can use Coalesce to annotate the value with Voucher queryset then run the filter again, like this:
query = Voucher.objects.filter(
Q(customer_id = customer_id_input ) | Q(is_global = True),
start_at__lte = now,
end_at__gte = now,
is_active = True,
times_used__lt = F('usage_limit'),
times_used_daily__lt = F('usage_limit_daily'),
code__customer_id = customer_id_input
).annotate(
usage_so_far=Coalesce('code__times_used', Value(0))
).filter(
usage_so_far__gte=F('usage_limit_per_customer')
).order_by('created_at').values()

Perform JOIN in tables django

Perform JOIN on in django fetching related date in reverse relationship.
There are three models.
Following is code for models
class Question(models.Model):
title = models.CharField(max_length=255 )
description = models.TextField(max_length=300)
class Quiz(models.Model):
name = models.CharField(max_length=225,blank=False )
quiz_type =models.IntegerField(choices=QUIZ_TYPE,default=0)
questions = models.ManyToManyField( Question, through='QuestionQuiz', related_name="quiz_question")
categories= models.ManyToManyField(Category,through='CategoryQuiz',related_name='quiz_category')
class QuestionQuiz(models.Model):
quiz = models.ForeignKey(Quiz,on_delete=models.CASCADE)
question = models.ForeignKey(Question,on_delete=models.CASCADE)
correct =models.DecimalField(max_digits=4, decimal_places=3)
incorrect= models.DecimalField(max_digits=4, decimal_places=3)
class Meta:
unique_together = ('quiz','question')
In this the questions are added to the quiz using model Question Quiz.
Here , QuizQuestion has foreign key relationship with the question. I need to fetch all from question JOIN and records from QuestionQuiz with a particular quiz_id.
Suppose quiz_id =3
Then I will fetch all questions with correct and incorrect. if that quiz id is added to the question then it will display correct incorrect else these would be blank.
question_id | title|description|correct|incorrect|quesquizid
1 | Q1 |Q1desc |2 | -2 | 1
2 | Q2 |Q2desc | | |
ques_id =1 added to quiz_id=3 .ques_id=2 not added to quiz_id=3.So, correct incorrect are blank.
I tried following but it fetches all question and related quizes scores irrespective of their occurrence in current quiz :
Question.objects.prefetch_related('questionquiz_set').all()
The result should be similar to following query
Select * from question as qs LEFT JOIN questionquiz as qq on (qq.question_id = qs.id AND qq.id=3)
Please check the result of the query:
I think prefetch_related along with Prefetch may get you the desired result.
q = Question.objects.prefetch_related(
Prefetch('questionquiz_set', queryset=QuestionQuiz.objects.filter(quiz=3))
)
print(q)
This may retrieve all related data along with null if exist.

Django ORM: Joining a table to itself

I have a table with cryptocurrency prices:
id | price | pair_id | exchange_id | date
---+--------+---------+-------------+---------------------------
1 | 8232.7 | 1 | 1 | 2018-02-09 09:31:00.160837
2 | 8523.8 | 1 | 2 | 2018-02-09 09:31:01.353998
3 | 240.45 | 2 | 1 | 2018-02-09 09:31:02.524333
I want to get the latest prices of a single pair from different exchanges. In raw SQL, I do it like this:
SELECT b.price, b.date, k.price, k.date, AVG((b.price + k.price) / 2)
FROM converter_price b JOIN converter_price k
WHERE b.exchange_id=1 AND k.exchange_id=2 AND b.pair_id=1 AND k.pair_id=1
ORDER BY b.date DESC, k.date DESC LIMIT 1;
8320.1|2018-02-09 11:23:00.369810|8318.2|2018-02-09 11:23:06.467424|8245.05199328066
How to do such query in the Django ORM?
EDIT: Adding models.py. I understand that it's quite likely that I'll need to update it.
from django.db import models
# Create your models here.
class Pair(models.Model):
identifier = models.CharField('Pair identifier', max_length=6)
def __str__(self):
return self.identifier
class Exchange(models.Model):
name = models.CharField('Exchange name', max_length=20)
def __str__(self):
return self.name
class Price(models.Model):
pair = models.ForeignKey(Pair, on_delete=models.CASCADE)
exchange = models.ForeignKey(Exchange, on_delete=models.CASCADE)
price = models.FloatField(default=0)
date = models.DateTimeField(auto_now=True, db_index=True)
def __str__(self):
return '{} - {}: {} ({})'.format(
self.date, self.pair, self.price, self.exchange)
EDIT2: To clarify what I'm really after.
I want the latest price with pair_id=1 and exchange_id=1, and the latest price with pair_id=1 and exchange_id=2. With a single query, and without any subsequent processing in Python - of course I can get Price.objects.all() and then search for it myself, but that's not the right way to use the ORM.
The way to do this with raw SQL is joining the table to itself. Showing how to join a table to itself using the Django ORM would (probably) answer the question.

Categories

Resources