How to group by date from timestamp field with Django ORM? - python

I have this MySQL specific query
SELECT SUM(trip_amount) as tp , DATE(FROM_UNIXTIME(request_time)) as timestamp
FROM trip_master
WHERE trip_master.date > 1493836200
AND trip_master.date < 1493922600
AND trip_master.id = 6
GROUP BY timestamp
Implemented this query in Django,
Trip.objects.filter(id=userid,date__gt = start,date__lt = end).annotate(trip_amount = Sum('trip_amount')).extra({'timestamp':'DATE(FROM_UNIXTIME(request_time))'})
As defined I have to convert time stamp to date to have grouping from dates. Has reached at the almost level but not getting group by dates.
Any help is much appreciated

Try:
from django.db.models.expressions import RawSQL
Trip.objects.filter(
id=userid,
date__gt=start,
date__lt=end
).annotate(
timestamp=RawSQL('DATE(FROM_UNIXTIME(request_time))', [])
).values(
'timestamp'
).annotate(
trip_amount=Sum('trip_amount')
)
See Group objects by dates

Related

Custom TruncFunc in Django ORM

I have a Django model with the following structure:
class BBPerformance(models.Model):
marketcap_change = models.FloatField(verbose_name="marketcap change", null=True, blank=True)
bb_change = models.FloatField(verbose_name="bestbuy change", null=True, blank=True)
created_at = models.DateTimeField(verbose_name="created at", auto_now_add=True)
updated_at = models.DateTimeField(verbose_name="updated at", auto_now=True)
I would like to have an Avg aggregate function on objects for every 3 days.
for example I write a queryset that do this aggregation for each day or with something like TruncDay function.
queryset = BBPerformance.objects.annotate(day=TruncDay('created_at')).values('day').annotate(marketcap_avg=Avg('marketcap_change'),bb_avg=Avg('bb_change')
How can I have a queryset of the aggregated value with 3-days interval and the index of the second day of that interval?
I guess it's impossible on DB level (and Trunc is DB level function) as only month, days weeks and so on are supported in Postgres and Oracle.
So what I would suggest is to use TruncDay and then add python code to group those by 3 days.
The following should work, although it's slightly ugly.
If you get the difference in days between each row's date and the min date you can then take the Mod of this difference to work out how many days you need to shift to get "middle" date. This middle date can then be grouped on using a values query
import datetime
from django.db.models import F, IntegerField, Avg, Min, DurationField, DateField
from django.db.models.functions import Cast, Mod, Extract
BBPerformance.objects.order_by(
'created_at'
).annotate(
diff=F('created_at__date') - BBPerformance.objects.aggregate(min=Min('created_at__date'))['min']
).annotate(
diff_days=Cast(Extract('diff', 'days'), output_field=IntegerField())
).annotate(
to_shift=Mod('diff_days', 3) - 1
).annotate(
grouped_date=Cast(F('created_at__date') - Cast(F('to_shift') * datetime.timedelta(days=1), output_field=DurationField()), output_field=DateField())
).order_by(
'grouped_date'
).values(
'grouped_date'
).annotate(
marketcap_avg=Avg('marketcap_change'),
bb_avg=Avg('bb_change')
)

Django F Date Expresion generated from static field

I am trying to get a series of ages from a persons list, but the age generated change with each query because is the age in a specific event so i can accomplish this with a simple loop, extracting the timedelta from the diference:
[ (event_date - user.birth_date).days/365.25 for user in User.objects.all() ]
event_date is always a datatime.date object anr user.birth_date too. I consider it a "Static"
field or constant because it is outside the database.
This gives me the correct results, but since I am doing this many times and i have other calculations to do I wanted to generate the ages from the database using the F() expresion.
``from django.db.models import ExpressionWrapper, fields
diference = ExpressionWrapper(
event_date- F('birth_date'),
output_field=fields.DurationField())
qs_ages= self.annotate(age_dist=diference)
this should give me a field named age_dist that will be a timedelta contains the total days between the two dates, so now I should do this and should give me the same result as above.
[ user.age_dist.days/365.25 for user in User.objects.all() ]
But this does not work, the result is a time delta of microseconds
What i am doing wrong? and how should I include the static value of event_date to the expression?
And... going beyond. Is there any way to get the days of the resultant time delta from the ExpressionWrapper?
since you're doing the operation on a datetime object it'll return you a datetime object which is converted to timedelta microseconds when you use DurationField as an the output field.
you can workaround this by doing everything in the database :
age = ExpressionWrapper(
Cast((event_date.date() - F('birth_date')) / 365.25 ,output_field=fields.PositiveIntegerField()),
output_field=fields.PositiveIntegerField()
)

Filter Django DB object with a manipulated keyword

I take a timestamp for my Institution class:
class Institution(models.Model):
timestamp_utc = models.DateTimeField()
If there is an entry in the DB that has the same year, month and date (not time), then I want to update the value of the entry. If not, then I want to create a new entry.
The conditional is as follows:
if Institution.objects.filter(timestamp_utc.strftime("%Y/%m/%d")=b['timestamp_utc'].strftime("%Y/%m/%d")).exists():
I am getting this error:
Exception Value: keyword can't be an expression
Is there a way to filter the DB object with a manipulated keyword?
You can just filter by the date range, i.e. time stamps that are great than or equal to the date, and less that the date + 1 day.
from datetime import relativedelta
date_start = b['timestamp_utc'].strftime("%Y-%m-%d")
date_end = (b['timestamp_utc'] + relativedelta(days=1)).strftime("%Y-%m-%d")
if Institution.objects.filter(
timestamp_utc__gte=date_start, timestamp_utc__lt=date_end
).exists():

With specific dates AND times in a database column, how do I query for all records on a specific day?

Convoluted title. I am using python sqlalchemy with a postgresql database.
I have a database that stores Events, and each event record has a starttime and a endtime column. Both of the aforementioned columns are in this format:
2015-06-22 13:00:00
How can I therefore query the database for all Events from a specific day. Thereby ignoring the "time" part of the starttime and endtime columns?
For example I tried:
query = Event.query.filter(Event.starttime=='2015-06-22', Event.endtime=='2015-06-22').all()
However this came up with 0 results, I am assuming perhaps this is because the absence of hh/mm/ss makes the query 'assume' that I mean 00:00:00.
I also tried
query = Event.query.filter(Event.starttime>='2015-06-22', Event.endtime<='2015-06-22').all()
But this came up with 0 results as well, I assume for the same reason as the previous.
Any help would be appreciated!
This is the query that gets sent to the DB:
SELECT events.id AS events_id, events.summary AS events_summary, events.starttime AS events_starttime, events.endtime AS events_endtime, events.parentcal AS events_parentcal, events.userid AS events_userid
FROM events
WHERE events.starttime >= :starttime_1 AND events.endtime < :endtime_1
Try using actual datetime objects. SQLAlchemy can interpret them.
from datetime import datetime
starttime = datetime.strptime('2015-06-22', '%Y-%m-%d')
endtime = datetime.strptime('2015-06-23', '%Y-%m-%d')
query = Event.query.filter(Event.starttime>=starttime, Event.endtime<endtime).all()
For more information on parsing datetime strings check out the docs

How do I exclude items created on same day in Django

I'm trying an exclude() query to exclude objects created on the same day as a user's profile. The user's user.created field is a DateTimeField, but I want to exclude all objects created on the same date as the user:
I'm trying:
my_objects = MyClass.objects.exclude(created__contains=user.created.date())
but it throws this error: Warning: Incorrect datetime value: '%2013-09-14%' for column 'created' at row 1
Is there a better way or a way to fix this?
Compare year, month, day:
d = user.created
my_objects = MyClass.objects.exclude(
created__year=d.year,
created__month=d.month,
created__day=d.day
)
or compare with the date (inclusive) and the next day (exclusive)
d = user.created.date()
my_objects = MyClass.objects.exclude(
created__gte=d,
created__lt=d + datetime.timedelta(days=1)
)

Categories

Resources