Django MySQL group by day with timezone - python

Lets say I have a sale model:
class Sale(models.Model):
total = models.DecimalField(max_digits=8, decimal_places=2, default=0)
sale_date = models.DateTimeField(auto_now_add=True)
Now, with each sale sale_date get its value saved in UTC, so if I try to group and sum all the sales by day:
report = Sale.objects.extra({'day':"date(sale_date)"}).values('day').annotate(day_total=Sum('total'))
I get all wrong becouse I expect each day in a different timezone (UTC-6).
Is there a way to get the correct sums in a specific tiemezone? Im working with MySQL.

Ah, this was a good challenge. I was able to test from PostGres, and I can confirm it is working. The MySQL code should be pretty close. However, there is a note on the CONVERT_TZ documentation:
To use named time zones such as MET or Europe/Moscow, the time zone tables must be properly set up. See Section 10.6, “MySQL Server Time Zone Support”, for instructions.
MySQL (using CONVERT_TZ(dt, from_tz, to_tz))
from_tz = 'UTC'
to_tz = 'Australia/ACT'
report = Sale.objects.extra(
{
'day': "date(CONVERT_TZ(sale_date, '{from_tz}', '{to_tz}'))".format(
from_tz=from_tz,
to_tz=to_tz
)
}
).values(
'day'
).annotate(
day_total=Sum('total')
)
PostGres: (using AT TIME ZONE)
time_zone = 'Australia/ACT'
report = Sale.objects.extra(
{'day': "date(sale_date) AT TIME ZONE '{0}'".format(time_zone)}
).values(
'day'
).annotate(
day_total=Sum('total')
)

Related

Django ORM Get users who have NOT updated their salary information in the last 1 year (Simlpe History )

I want to bring users who have not updated their salary information in the last 1 year. BUT WITH ORM not For Loop.
from simple_history.models import HistoricalRecords
class User(AbstractUser):
...
salary_expectation = models.IntegerField()
history = HistoricalRecords(cascade_delete_history=True)
################################################################
User.objects.filter(# MAGIC ) # Get users who have NOT updated their salary information in the last year
I can see that this is a package which has its documentation in querying its entries, see below:
https://django-simple-history.readthedocs.io/en/latest/querying_history.html
nevertheless you can do that intuitively following Django's normal behavior and a couple of SQL knowledge, I'd expect that history field's table most likely has a one-to-many relationship with the users table, so what I'd do is first open the database, find the column that shows the date of change, write down its name and then write this ORM query below
sub_query = ~Q(history__history_date__lte= "Replace with end of date", history__history_date__gte= "Replace with beginning of date", salary_expectation__isnull=False)
users = User.objects.filter(sub_query)
dont forget to import Q
from django.db.models import Q
You do not need to check HistoricalRecords class for this information.
Add created_at and updated_at (date_time_fields) fields to your User model
class User(...):
...
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
Queryset Code
from django.db.models.functions import Now, ExtractDay
from django.contrib.auth import get_user_model
User = get_user_model()
users = User.objects.annotate(
# Calculate duration between now and last update date saved
duration=models.ExpressionWrapper(
Now() - models.F("updated_at"),
output_field=models.DurationField()
),
# Extract the amount of days in the duration
days=ExtractDay('duration'),
# Check if the number of days between the 2 fields exceeds 1 year (365.25 Days)
last_update_beyond_a_year=models.Case(
models.When(
models.Q(days__gte=365.25),
then=True
),
default=False,
output_field=models.BooleanField()
)
# Then filter
).filter(last_update_beyond_a_year=True)
and Voila !

Django - Given time zone, month and year get all post created on that date in that time zone

So I have this Post model. I want to be able to retrieve all posts that were created in a month, year under a certain time zone.
My goal is to implement a feature where a user anywhere in the world let's say in PST can get all posts by another person from a certain month in their time zone. So let's say user A is in EST and user B is in PST (3 hours behind EST). User B wants to see all posts that user A created in October of 2021. Since the app will display posts in the time zone the user is currently in (we send date time in UTC then the front-end converts to local time) then the app should only send to user B all posts by user A that were created in October 2021 PST. So for example if user A (the user in EST) made a post at 11pm Oct 31 2021 EST(8pm Oct 31 2021 PST) and a post at 1am Nov 1st 2021 EST (10pm Oct 31st 2021 PST) then user B should on get both posts back, because the 2nd one was made in November in EST, but October in PST.
model.py
class Post(models.Model):
uuid = models.UUIDField(primary_key=True)
created = models.DateTimeField('Created at', auto_now_add=True)
updated_at = models.DateTimeField('Last updated at', auto_now=True, blank=True, null=True)
creator = models.ForeignKey(
User, on_delete=models.CASCADE, related_name="post_creator")
body = models.CharField(max_length=POST_MAX_LEN)
So for example if a user creates 10 posts in November, 2 in December of 2021 in PST. Then I have a view that takes month, year and time_zone and let's say the url looks something like /post/<int:month>/<int:year>/<str:time_zone> and the user pings /post/11/2021/PST then it should return the 10 posts from November. How do I return all posts from a month and year in a time zone given time zone, month and year?
Note: The tricky edge case to take into consideration is if they post on the very last day of a month very late. Depending on the time zone something like 12/31/2021 in UTC could be 01/01/2022. Because Django stores datetime fields in UTC what would need to be done is converted created to the given time_zone then get posts from the specified month and year.
Setup:
Django 3.2.9
Postgresql
Attempted Solutions
The most obvious solution to me is to convert created to the specified time_zone then to do Post.objects.filter(created__in_range=<some range>)
Note
Main issue seems to be Pyzt, which takes in time in a very specific format "Amercian/Los_Angeles" w.e format this is. Rather than the abbreviated time zone format like "PST".
Take the month's first moment (midnight on the 1st) in UTC and the next month's first moment in UTC, adjust them with the timezone you want, do a posted__range=(a, b) query?
This might work (but date math is fiddly...).
This requires python-dateutil to make computing the end time robust.
import datetime
import pytz
from dateutil.relativedelta import relativedelta
from django.utils import timezone
year = 2021
month = 6
tz = pytz.timezone("America/Los_Angeles")
start = datetime.datetime(year, month, 1)
end = start + relativedelta(months=1)
start_as_tz_in_utc = tz.localize(start).astimezone(pytz.utc)
end_as_tz_in_utc = tz.localize(end).astimezone(pytz.utc)
print(start_as_tz_in_utc, end_as_tz_in_utc, sep="\n")
That prints out
2021-06-01 07:00:00+00:00
2021-07-01 07:00:00+00:00
which seems about right.
Then you might make a query such as
posts = Post.objects.filter(created__range=(
start_as_tz_in_utc,
end_as_tz_in_utc,
))
I would try something like this:
Assuming you are passing a given month through as given_month
and given year as given_year
and given timezone as given_timezone
Model_Name.objects.filter(created.split('/',4)[2]=given_month,created.split('/',4)[3]=given_year, created.split('/',4)[4]=given_timezone)
This should get the value from the month and year and timzone section of your post. You might need to play around with what I gave you. Also it might be better to add a relationship from user to post so you can filter the user for posts with my given answer. In most cases this will be a lot more efficient, assuming there are more posts than users. This should get you on the correct track.
Consider saving the record in UTC date time, do not attempt to save any time zone offset. just record the time the data arrived to the server in UTC, and let the browser or your app translate it to their configured timezone. It doesnt matter if your User is in China created a post and then moved to NY the next day and read the previously created post, the created_at datetime will be same for both locations because it is universal. So if the user B wants to recover the data the user A recorded since october it will as easy to querying created_at::date>="2021-10-01"
Sorry, if it is not very on point to what you're expecting, but Django creates a PostgreSQL timestamp with a time zone field for the DateTimeField.
# \d posts_post
Table "public.posts_post"
Column | Type | Collation | Nullable | Default
------------+--------------------------+-----------+----------+---------
uuid | uuid | | not null |
created | timestamp with time zone | | not null |
updated_at | timestamp with time zone | | |
body | character varying(500) | | not null |
Indexes:
"posts_post_pkey" PRIMARY KEY, btree (uuid)
It means you can query them as timestamps with a custom field class from Timestamp fields in django.
from django.db import models
from datetime import datetime
from time import strftime
class UnixTimestampField(models.DateTimeField):
"""UnixTimestampField: creates a DateTimeField that is represented on the
database as a TIMESTAMP field rather than the usual DATETIME field.
"""
def __init__(self, null=False, blank=False, **kwargs):
super(UnixTimestampField, self).__init__(**kwargs)
# default for TIMESTAMP is NOT NULL unlike most fields, so we have to
# cheat a little:
self.blank, self.isnull = blank, null
self.null = True # To prevent the framework from shoving in "not null".
def db_type(self, connection):
typ=['TIMESTAMP']
# See above!
if self.isnull:
typ += ['NULL']
if self.auto_created:
typ += ['default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP']
return ' '.join(typ)
def to_python(self, value):
if isinstance(value, int):
return datetime.fromtimestamp(value)
else:
return models.DateTimeField.to_python(self, value)
def get_db_prep_value(self, value, connection, prepared=False):
if value==None:
return None
# Use '%Y%m%d%H%M%S' for MySQL < 4.1
return strftime('%Y-%m-%d %H:%M:%S',value.timetuple())
In the query, you will pass the timestamp created from the required timezone DateTime. This approach is also should be faster since you don't need to perform 1 additional DateTime conversion.
class Post(models.Model):
uuid = models.UUIDField(primary_key=True)
created = UnixTimestampField(verbose_name='Created at', auto_now_add=True)
updated_at = UnixTimestampField(verbose_name='Last updated at', auto_now=True, blank=True, null=True)
from datetime import datetime, timezone
Post.objects.filter(created__gte=datetime(year=2021, month=11, day=1, tzinfo=timezone.utc).timestamp(), created__lt=datetime(year=2021, month=12, day=1, tzinfo=timezone.utc).timestamp())

first and last methods used excluded items

i used last() to get last item of queryset after exclude some items as below:
holidays = HolidayModel.objects.all().values_list('date', flat=True)
result = BorseExchangeLog.objects.exclude(
hint_time__date__in=holidays
)
# output 1
print(list(result.valuse_list('hint_time__date',flat=True).distinct('hint_time__date')))
#output2
print(result.last().hint_time.date())
but in output2 print item that not exists in output1
i test some other codes as below:
print(list(logs.values_list('hint_time__date',flat=True).distinct('hint_time__date')))
print(list(logs.values_list('hint_time__date', flat=True).distinct('hint_time__date'))[-1])
print(logs.order_by('hint_time__date').last().hint_time.date())
[..., datetime.date(2020, 10, 21), datetime.date(2020, 10, 26)]
2020-10-26
2020-10-25
my holiday model:
class HolidayModel(models.Model):
creator = models.ForeignKey('accounts.Account', on_delete=models.PROTECT, verbose_name=_('Creator'))
reason = models.CharField(default='', max_length=200, verbose_name=_('Reason'))
date = models.DateField(default=timezone.now, verbose_name=_('Date'))
and other model is :
class BorseExchangeLog(models.Model):
create_time = models.DateTimeField(default=timezone.now)
hint_time = models.DateTimeField(default=timezone.now)
i test that by first() and problem was there too
what is problem? my code is wrong or bug from django orm?
using django2.2 and postgresql
Your datetimes are timezone aware but the date() method on datetime objects does not take the timezone into account, __date will take the timezone into account provided your DB supports it. Use the django.utils.timezone.localdate function to get a date taking into account the timezone
from django.utils.timezone import localdate
print(localdate(result.last().hint_time))

How to filter DataTime field by hour in Django?

I have Django 1.8 and Python 2.7, I need to get orders made per hour
the field is DateTimeField
close_date = models.DateTimeField(null=True, blank=True)
I do the simple loop:
for hour in xrange(0, 24):
orders = Order.objects.filter(close_date__hour=hour)
but it does not filter.
You can try:
Order.objects.filter(close_date__regex = 'HH:00') or (close_date__contains = 'HH:00')
As perhaps django 1.8 only support year, month, day, week_day.

Get objects in which today date lies between two dates in table

I have a django model like this
class AthleteSubscription(models.Model):
user = models.ForeignKey(User, related_name="user_subscription", default='')
subscription_start = models.DateField(default=datetime.date.today)
subscription_end = models.DateField(default=datetime.date.today() + timedelta(30))
Where subscription_start is start date of subscription and subscription_end is the end date of subscription. Subscription is of 30 days. I want to get those records in which current date (date today) lies between subscription_start and subscription_end. How can I do this with django ORM.
qv = AthleteSubscription.objects.filter(subscription_start__gte=some_date, subscription_end__lte=some_date)

Categories

Resources