Intermittent wrong date from time.strftime() - python

I have an Appengine app that still runs on the python2.5 runtime, which means it is single threaded. I have a school class:
class School(db.Model):
...
dateformat=db.StringProperty(default='%a %e %b',indexed=False) # For bookings
timeformat=db.StringProperty(default='%l:%M%P',indexed=False) # For bookings
def date(self,t):
return time.strftime(self.dateformat,time.gmtime(t))
def datetime(self,bt,et=None):
return time.strftime(self.dateformat+' '+self.timeformat,time.gmtime(bt))+\
(time.strftime(' - '+self.timeformat,time.gmtime(et)) if et else '')
def time(self,t):
return time.strftime(self.timeformat,time.gmtime(t))
Then when I want to format a date in the school's chosen format, I do this:
s=School.get_by_id(the_id)
date_string=s.date(the_timestamp)
Very occasionally, I get a date that is exactly a week out. So instead of "Wed 2 May" I get "Wed 9 May". This has been reported three times in the last week, out of probably tens of thousands of cases. All exactly a week out, none with any other time difference.
Nothing's changed in that part of my code for ages so I don't understand why this should suddenly start happening. Because it's single threaded, there shouldn't be issues with strftime, and I can't find any reports of thread issues with it anyway anyway.
Any ideas?

Related

How do I test an AWS Lambda Function that is triggered by EventBridge but needs to be aware of daylight savings?

I have a lambda function that needs to run at a user defined frequency, defined via cron, at 9 AM Eastern time. For example, "weekdays at 9 AM" or "every alternate day at 9 AM".
EventBridge (via CDK) only allows me to define the cron trigger based on UTC. I'm triggering the function at 1 PM as well as 2 PM UTC, to account for daylight savings, and detecting whether daylight savings is in effect in the lambda code. That way, I can skip if it isn't 9 AM Eastern time and run the actual code when it really is 9 AM.
Here's how the code looks like:
def aws_lambda(event, context):
cron_expression = os.getenv('cron_expression') # The cron in CDK also passed as env var
local_time = localized_time_now()
if local_time.hour != 9:
return
window_start = local_time - timedelta(days=1)
window_stop = local_time
iter = croniter(cron_expression, window_start, ret_val=datetime)
prev_invocation = iter.get_prev()
if prev_invocation > window_start:
raise RuntimeError("Cron expression needs to allow for 24 hours to pass between events")
# Business logic here, using window_start and window_stop to query the DB
...
How do I test this function? I can't control the server time in EventBridge or Lambda, so it seems inherently untestable. It seems like the localized_time_now() function is key, but I'll just have to trust that it will always work properly in the context of AWS?
In my opinion localized_time_now() works with the TZ environment variable (look for TZ). So You have to set this varialble to Your timezone first. This is of course possible with CDK.
environment: {
TZ: 'Europe/Berlin',
}
After that datetime.datetime.now() should return the local time.
This is kind of awful that we can only set the event schedule in UTC, but it is how it is and running the lambda every hour and checking if it is the correct one is a valid solution for me here.
The second thing is how to test this. You can of course run this lambda locally with diffrent event timestamps. You can do unit tests or probably better at this point would be to use the lambda-local package. Consider also playng around with test events in the lambda console. There You can play around with the actual runtime.

Django Opening Hours With UTC Support/Timezone Issue

I have a Django application where users can setup stores. I recently added functionality to support opening hours following the advice on this thread - Business Opening hours in Django. My model looks like this:
class LocationHours(SafeDeleteModel):
location = models.ForeignKey(Location, related_name="hours", on_delete=models.CASCADE)
weekday = models.IntegerField(choices=WEEKDAYS, blank=False, null=False)
start_time = models.TimeField(blank=False, null=False, help_text="Opening time (00:00 format)")
end_time = models.TimeField(blank=False, null=False, help_text="Closing time (00:00 format)")
class Meta:
ordering = ('weekday', 'start_time')
unique_together = ('location', 'weekday', 'start_time', 'end_time')
verbose_name_plural = "Location hours"
Process goes like this - these times are entered in a form by the end user and thus assumed to be localtime, most datetimes/times being used in my application are in UTC. I need to compare the two often so originally, I thought I could figure out the timezone of each location object, then whenever I compare something to these OpeningHours, I can just use the given time and the tz (on the associated Location object) to calculate the time in UTC and then it's just a regular datetime comparison.
I wrote the following function to try and fix this:
def is_dt_within_location_hours(location, dt):
# see if time falls within hours
hours = location.hours
if hours.count() > 0:
for hour in hours.all():
dt = dt.astimezone(hour.location.timezone)
if dt.weekday() == hour.weekday:
if hour.start_time < dt.time() < hour.end_time:
return True
return False
else: # this location has no hours
return True
I thought this worked however has some issues.
Primary issue is this - when the Location objects are originally made or edited, I look up the timezone it's in (using the timezonefinder package) and store that in the Location object (using a TimeZoneField) at that time. This is to say, it will not auto update for DST or anything like that as far as I know. I could figure out the timezone everytime I call the above function however I call said function A LOT such that resource wise I'd like to say this is borderline not an option.
I imagine I could find a way to figure out the localtime at the moment they create an OpeningHours object and that way I could just convert to UTC and save it then but I don't know a good way to do that.
I'm thinking now I may need to scrap my entire solution and start from scratch but any advice is really helpful I've been struggling with this for a while.
You're doing it the right way.
You're worried about the timezone offset changing (as with DST) in between the time you record the Location and when you do the computation. But a timezone (represented by a name like "America/Chicago") isn't just an offset, it's a set of rules for computing the local time at any point in history. So it will do the right thing regardless of when you happened to record the timezone name.
A few other notes on the code you posted:
You probably want to make LocationHours unique on just location and weekeday, unless you're purposely trying to allow multiple opening hours for the same location on the same weekday.
Your is_dt_within_location_hours() is fairly inefficient. You don't need to fetch all the LocationHours objects and re-compute the weekday each time. Instead, first compute the local time, then filter location.hours to only include the LocationHours objects on that weekday.

Issue with datetime module

I'm trying to make a simple Python function that prints
two things;
What day of the week it currently is.
How many days it is until Christmas.
I don't get any errors when I run this, however nothing prints either.
In fact nothing at all happens when its run.
I've checked and I've installed the datetime module correctly (I think).
I'm not really sure what I'm doing wrong so any guidance would be helpful.
As you can probably guess, I'm relatively new to Python/stackoverflow.
Thanks.
Edit: I stupidly named the file itself datetime.py because yes I am that stupid.
from datetime import *
def day_of_week():
""" Python function to return a specified day of the week"""
today_date = date.today()
thisYear = today_date.year
xmas = date(thisYear,12,25)
day_of_week =("Mon","Tue","Wed","Thu","Fri","Sat","Sun")[today.weekday]
print("It's a %s." % (day_of_week))
days_to_xmas = (xmas - today_date).days
print("There are %i days to xmas!" & days_to_xmas)
Try calling the function in the script , you defined the function but never executed it.
day_of_week()

Deciphering datetime discrepancy between application output and server

In a Django/Python application where I'm using redis, I do:
my_server = redis.Redis(connection_pool=POOL)
updated_at = time.time()
object_hash = "np:"+str(object_id)
sorted_set = "sn:"+str(user_id)
my_server.zadd(sorted_set, object_hash, updated_at)
This is straight forward. Essentially, I'm maintaining a sorted set that contains objects sorted by time of updating the object.
The problem is if I used redis-cli to get zrange sorted_set 0 -1 WITHSCORES, the score shows time that is precisely 5 hours older than what was originally in updated_at.
e.g. if updated_at was fed 1479646405.21, the redis sorted set score ends up being 1479628405.497179 (as per output from redis-cli). I.e. 5 hours behind. This looks like an issue of timezone - my location's ahead by 5 hours from UTC.
My question is: why does the score jump back 5 hours when updating the redis server? Whenever I print the updated_at variable from within my application, I get the correct number. Is this a Linux issue (the OS my application resides on is Ubuntu 14.04), and if so, can you explain precisely what could be going on? Being a beginner, I'm trying to understand the dynamics at play here. Thanks!

Code resulting in weird datetime result on Google App Engine

I'm running a python program on Google App Engine that uses the datetime function. It's supposed to always return UTC time, but it seems to intermittently give an incorrect time. I'm not sure if there's an error with my code, or whether this is an issue on Google's side.
To get my local time (GMT +8:00), I run this function:
def SGTOffset(obj=datetime.now()):
if isinstance(obj, datetime):
return obj + timedelta(hours=8)
return SGTOffset(datetime.now())
and in my main program:
today_date = commoncode.SGTOffset().date()
logging.debug('Today date: %s | Today datetime: %s' % (today_date.isoformat(), commoncode.SGTOffset().isoformat()))
In the logs, I get the following:
[25/Nov/2015:09:00:02 -0800] "GET ... etc ...
01:00:03.287 Today date: 2015-11-25 | Today datetime: 2015-11-25T15:38:20.804300
So, Google kindly formats the log datetime to my locale (GMT +8), showing that the code is run at 01:00:03.287 (26th Nov, GMT +8). Also, this is confirmed by the timestamp 25/Nov/2015:09:00:02 -0800 provided. So the code was run at 25/Nov/2015 17:00:02 UTC time.
However, my code is outputting the wrong time. The datetime that is being generated in the code 2015-11-25T15:38:20.804300 has the timezone of GMT-9:30 instead of UTC time. (Because SGToffset() adds 8 hours to datetime)
This is quite catastrophic as I use the local datetime in many areas of my program. This is also happening intermittently only, because yesterday, the same code ran and got this log:
[24/Nov/2015:09:00:00 -0800] "GET ... etc ...
01:00:02.237 Today date: 2015-11-25 | Today datetime: 2015-11-25T01:00:01.768140
Which is correct! (Google's log timestamp 01:00:02.237 matches the time generated by SGTOffset() which is 01:00:01)
Could I know what is wrong with my program, or whether this is an issue with Google App Engine?
Thank you spending time to read this question!
The problem lies with the code.
Python stores a default value of the parameter obj for the function SGTOffset() when it is first defined (when the function object is first instantiated), instead of whenever the function is called as I intuitively expected. So, the datetime value will reflect the start time of the instance in GAE.
In order to get the current time whenever I call SGTOffset() without any parameters, I should instead have used:
def SGTOffset(obj=None): # Instead of obj=datetime.now() which is converted to obj='specific datetime' when instantiated
if isinstance(obj, datetime):
return obj + timedelta(hours=8)
return SGTOffset(datetime.now())
In this case, datetime.now() is called dynamically whenever it is required.
I arrived at this solution after viewing a question about old datetime values being returned.
I'm adding a quick answer to give you suggestions to make your code more readable:
obj is not a good variable name because it is not informative
No need for a recursive call to the function
Better to not use isinstance because is None gives you the needed functionality and your code will not work if some other instance type is given anyway.
Here is my suggestion:
def SGTOffset(dt=None):
if dt is None:
dt = datetime.now()
return dt + timedelta(hours=8)
Or if you prefer brevity:
def SGTOffset(dt=None):
return (dt or datetime.now()) + timedelta(hours=8)

Categories

Resources