Develop Reference

How to make a Timer exactly 30 seconds from created date Django

I need to make an operation only available in 30 seconds, and then it expires. I Have this model:
(operation)
status = CharField(max_length=10, default="INI")
created_at = DateTimeField(auto_now_add=True)
account = ForeignKey(Account, null=False, on_delete=CASCADE)
metadata = models.JSONField(default=dict)
external_ref = CharField(max_length=128)
expires_at = DateTimeField(default=timezone.now()+timezone.timedelta(seconds=30))
I wanna be able to create in the expires_at field, a timestamp with exactly 30 seconds from the created_at Date, it is like a timeout function, but when I run the test:
def test_timeout_is_30_seconds(self):
print(self.operation.created_at)
timer = self.operation.created_at + timezone.timedelta(seconds=30)
print(timer)
self.assertEqual(self.operation.expires_at, timer)
it fails with this message:
AssertionError: datetime.datetime(2021, 6, 22, 19, 0, 42, 537490, tzinfo=<UTC>) != datetime.datetime(2021, 6, 22, 19, 0, 45, 844588, tzinfo=<UTC>)
I dont know if I need to make an external function or method inside the class or directly in the View, but I would prefer this default behavior in the models so I dont need to worry about setting expiry dates
I would be very grateful if you could help me solve it! :D any tips and information is appreciated

This is a common error. Your expression timezone.now()+timezone.timedelta(seconds=30) is being evaluated once, when the class is defined, and that value is being used as the default for every instance.
What you actually want is for the expiration time to be freshly calculated each time a new instance is created. Which means that you want to set default to a function:
def half_minute_hence():
return timezone.now() + timezone.timedelta(seconds=30)
expires_at = DateTimeField(default=half_minute_hence)

How to check if a Marshmallow schema's Datetime field is less than today's date?

I have an endpoint that accepts POST method. The POST body contains a DateTime field of format - "%Y-%m-%d %H:%MZ". I need to validate if that datetime is less than current Datetime in UTC. I'm using Marshmallow to validate the request body.
run_datetime = fields.DateTime(format="%Y-%m-%d %H:%MZ")
Are there any inbuilt validators for this case to validate DateTime field. Or should I be writing a custom function for this to compare the run_datetime with today's UTC's datetime.

There is no built-in validator that can solve your particular issue at hand, take a look at the available validators here.
Although, defining your own validator is trivially easy, for your particular case: fields.Datetime will accept an argument named validate which can take a function that returns a bool. For example, I quickly defined a lambda here to validate if the datetime is more recent than "now":
from datetime import datetime
from marshmallow import Schema, fields
NOW = datetime(2020, 11, 23, 14, 23, 0, 579974)
class User(Schema):
id = fields.Integer(required=True)
name = fields.String(required=True)
# Define an arbitrary datetime here like NOW or just use datetime.now()
date_created = fields.DateTime(required=True, validate=lambda x: x > NOW)
# This will succeed
User().load(dict(
id=10,
name="Test",
# Note that this date is more recent than previously defined NOW
date_created="2020-11-23T14:24:40.224965",
))
#{'date_created': datetime.datetime(2020, 11, 23, 14, 24, 40, 224965),
# 'id': 10,
# 'name': 'Test'}
# While this will fail
User().load(dict(
id=10,
name="Test",
# Here the date is one month behind than NOW
date_created="2020-10-23T14:24:40.224965",
))
# ValidationError: {'date_created': ['Invalid value.']}

TruncDate timezone parameter is not working in Django

Not able to change TImezone in Trunc functions. It always take timezone from settings.py
import pytz
ind = pytz.timezone('Asia/Calcutta')
Query:
queryset = Order.objects.annotate(date=TruncDate('created_at', tzinfo=ind)).values('date')
While inspecting sql query by queryset.query
SELECT DATE(CONVERT_TZ(`nm_order`.`created_at`, 'UTC', UTC)) AS `date` FROM `nm_order`
Reference: Trunc in Django
But for Extract, it's get working
ORM:
queryset = Order.objects.annotate(date=ExtractDay('created_at',tzinfo=ind)).values('date')
Query:
SELECT EXTRACT(DAY FROM CONVERT_TZ(`nm_order`.`created_at`, 'UTC', Asia/Calcutta)) AS `date` FROM `nm_order`
Am I miss something in Trunc ?
TimeZone Settings in my settings.py
IME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True

You need to use TruncDay() instead of TruncDate. In the usage example just below that section of the documentation you'll see the difference between the two.
TruncDate does not take a timezone option - it uses the current timezone and gives you the date in that timezone.
I think the distinction between the two is that TruncDate returns a DateField which by definition cannot be timezone-aware. TruncDay on the other hand returns a DateTimeField(with time portion set to 00:00:00), which can be timezone aware.

From django 3.2, TruncDate now accepts tzinfo parameter. https://docs.djangoproject.com/en/dev/ref/models/database-functions/#django.db.models.functions.TruncDate

After Inspecting TruncDate Class looks like it doesn't have timezone option
class TruncDate(TruncBase):
kind = 'date'
lookup_name = 'date'
#cached_property
def output_field(self):
return DateField()
def as_sql(self, compiler, connection):
# Cast to date rather than truncate to date.
lhs, lhs_params = compiler.compile(self.lhs)
tzname = timezone.get_current_timezone_name() if settings.USE_TZ else None
sql, tz_params = connection.ops.datetime_cast_date_sql(lhs, tzname)
lhs_params.extend(tz_params)
return sql, lhs_params

How to convert Django's TimeField to Postgres' timestamp with time zone

In my Django app, the program gets the time of an event from the user, say user inputs 10:30 for the time of an event. The program uses modelformset_factory to create a whole bunch of forms:
#forms.py
class EventForm(ModelForm):
class Meta:
model = Event
fields = ['entry', 'time', 'work']
localized_fields = ('time', )
widgets = {
'work': Textarea(attrs = {'cols': 40, 'rows': 8}),
}
#models.py
class Event(models.Model):
'''
Each day there are many events, e.g. at 10 am, the framer orders material, etc.
'''
entry = models.ForeignKey(Entry)
time = models.TimeField()
work = models.TextField()
#views.py:
EventFormSet = modelformset_factory( Event, form = EventForm, exclude = ('entry',),extra = 5)
eventset = self.EventFormSet(request.POST)
all_errors = eventset.errors # All errors returns [{}, {}, {}, {}, {}], so the data is free of error.
try:
eventset.is_valid()
except ValidationError:
return render(request, self.template_name, {'form': self.day_log(initial = self.initial_values), 'eventforms': self.event_formset})
events_instances = eventset.save(commit = False)
for instance in events_instances:
if instance.time:
event_date = datetime.date(the_year, the_month, the_day) # Naive date
event_time = instance.time # Naive time
naive_event_time = timezone.is_naive(event_time) # Checking naivaty
event_datetime = datetime.datetime.combine(event_date, event_time) # Naive datetime object
is_aware_event_datetime = timezone.is_aware(event_datetime) # Checking to see if the datetime object is aware or naive
event_aware_datetime = datetime.datetime(the_year, the_month, the_day, instance.time.hour, instance.time.minute, instance.time.second, tzinfo = pytz.utc) # Making an aware time
is_aware_event_aware_datetime = timezone.is_aware(event_aware_datetime) # Making sure the event_aware_datetime is indeed aware
instance.time = event_aware_datetime
awareness_test = timezone.is_aware(instance.time) # Making sure instance.time is aware
eventset.save() # Here is where the exception is thrown.
The Django app fails to save time data to Postgres. The exception type is DataError with the exception value:
invalid input syntax for type timestamp with time zone:
LINE 1: ...y_event" ("entry_id", "time", "work") VALUES (14, '18:43:04....
with ^ pointing at ' of '18:
Edit:
Here is my database:
CREATE TABLE site_activity_event
(
id serial NOT NULL,
entry_id integer NOT NULL,
"time" time without time zone NOT NULL,
work text NOT NULL,
CONSTRAINT site_activity_event_pkey PRIMARY KEY (id),
CONSTRAINT site_activity_event_entry_id_fkey FOREIGN KEY (entry_id)
REFERENCES site_activity_entry (id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION DEFERRABLE INITIALLY DEFERRED
)
I did sqlall my app to ensure that the data type of model field correspond to that of the app and all is well there. This seems to be a postgres issue, but Django is supposed to take care of all those pesky interface issues with Postgres (https://docs.djangoproject.com/en/1.8/topics/i18n/timezones/#migration-guide), so that makes me think I am missing something. Any help is much appreciated.

Well, is "18:43:04" a valid timestamp with a timezone? Do you think it specifies a unique point in time? If you do then you have uncovered a major bug in PostgreSQL's date/time handling. If not, then you should either supply a valid timestamp or use the correct type in your database - I can't say which without knowing more about these events.

Your Event has a TimeField but it looks like the database was created with a DateTimeField.

serialize a datetime as an integer timestamp

I would like for django rest to not convert my DateTime model field into a string date represtation when serializing it.
response_date = serializers.DateTimeField(source="updated_at")
I would like this to come out as
1411880508
and not
"2014-09-28T05:01:48.123"

You'll want to write a custom serializer field, like so:
class TimestampField(serializers.Field):
def to_native(self, value):
epoch = datetime.datetime(1970,1,1)
return int((value - epoch).total_seconds())
To support write operations you'd want to inherit from WritableField and also implement from_native().
EDIT for DRF 3.x & Python 3.8:
class TimestampField(serializers.Field):
def to_representation(self, value):
return value.timestamp()
If you want a JavaScript style timestamp:
class JsTimestampField(serializers.Field):
def to_representation(self, value):
return round(value.timestamp()*1000)

REST_FRAMEWORK = {
# if you want with milliseconds or
'DATETIME_FORMAT': '%s.%f',
# only with seconds
'DATETIME_FORMAT': '%s',
}
Result in REST will be string
"1517863184.666435"
"1517863249"
If you want float(or integer) value in API, than you can use monkey patching.
Put the file monkey_patching.py in any of your apps and import it in app's __init__.py file. ie:
app/monkey_patching.py
#app/monkey_patching.py
import six
from rest_framework import ISO_8601
from rest_framework.fields import DateTimeField
from rest_framework.settings import api_settings
def to_representation_ext(self, value):
if not value:
return None
output_format = getattr(self, 'format', api_settings.DATETIME_FORMAT)
if output_format is None or isinstance(value, six.string_types):
return value
if output_format.lower() == ISO_8601:
value = self.enforce_timezone(value)
value = value.isoformat()
if value.endswith('+00:00'):
value = value[:-6] + 'Z'
return value
# FOR INTEGER RESULT 'DATETIME_FORMAT': '%s',
# return int(value.strftime(output_format))
# FOR FLOAT RESULT 'DATETIME_FORMAT': '%s.%f',
return float(value.strftime(output_format))
DateTimeField.to_representation = to_representation_ext
app/init.py
#app/__init__.py
import app.monkey_patching
Tested with Django version 2.0.10 and Python 3.5.9

I was not able to get Tom's example to work and it seemed the values were not being modified. However it gave me a starting point and after some reading I found a way to produce the desired result:
[METHOD 1]
serializers.py
import time
class TimestampField(serializers.Field):
def to_representation(self, value):
return int(time.mktime(value.timetuple()))
class MySerializer(serializers.ModelSerializer):
ts = TimestampField(source="my_fieldname") #Source must be a models.DateTimeField
class Meta:
model = myModel
fields = ('id', 'ts')
JSON output:
[{
"id": 1,
"ts": 1475894303
},
{
"id": 2,
"ts": 1475833070
}]
[METHOD 2]
Tom's explanation and the previous mentioned method are definitely more on track with maintaining standards (as the results are actually of type integer).
However a quick and dirty solution is to specify the format parameter for the DateTimeField and set it to show the value in seconds.
Note this probably won't work correctly on Windows machines!
And may result in a ValueError: Invalid format string
To try it out just include the "format" keyword parameter in your serializer field like so:
serializers.py
class MySerializer(serializers.ModelSerializer):
timestamp = serializers.DateTimeField(format="%s")
class Meta:
model = myModel
fields = ('id', 'ts')
JSON output:
[{
"id": 1,
"ts": "1475890361"
},
{
"id": 2,
"ts": "1475833070"
}]
Additionally you may include microseconds:
timestamp = serializers.DateTimeField(format="%s.%f")
If you want to test the functionality in your own interpreter (to verify your OS supports the %s parameter) just copy over these lines:
import datetime
print datetime.datetime.now().strftime('%s') #datetime formatted as seconds for REST
import time #This is just for confirmation
print time.mktime(datetime.datetime.now().timetuple()) #time object result as float
I feel this method is a little inconsistent with the OPs question because the result is not actually of type integer, instead it is a string representation of an integer/float - and REST will surly add quotes around the value.

Although I prefer the answer given by Tom Christie as it is more robust.
I decided to post my solution for the benefit of the potential readers
response_date = serializers.SerializerMethodField('get_timestamp')
def get_timestamp(self, obj):
#times 1000 for javascript.
return time.mktime(obj.updated_at.timetuple()) * 1000

Global Configuration:
REST_FRAMEWORK = {
'DATETIME_FORMAT': '%s.%f',
}

In python, the timestamp is 10 digit. However, in Javascript, it is 13 digit.
Therefore, if you want to return Javascript format timestamp in global configure, just add '000' after '%s':
JS_TIMESTAMP = '%s000'
REST_FRAMEWORK = {
'DATETIME_FORMAT': JS_TIMESTAMP,
'DATE_FORMAT': JS_TIMESTAMP
}
The result will looks like this: 1567413479000

As mentioned before you can set timestamp format for all datetimes globally by:
REST_FRAMEWORK = {
'DATETIME_FORMAT': '%s',
}
But this doesnt work for regular dates, to make it work for dates you also have to set:
REST_FRAMEWORK = {
'DATE_FORMAT': '%s',
}

Thanks to #megajoe for monkey patch solution. I was developing on windows so was getting invalid format string since windows does not support any "%s" format (http://msdn.microsoft.com/en-us/library/fe06s4ak.aspx).
So I used monkey patch like #megajoe and tweaked the solution a little to return value.timestamp() for "%s.%f" and int(value.timestamp()) for "%s".