Develop Reference

How do I test an AWS Lambda Function that is triggered by EventBridge but needs to be aware of daylight savings?

I have a lambda function that needs to run at a user defined frequency, defined via cron, at 9 AM Eastern time. For example, "weekdays at 9 AM" or "every alternate day at 9 AM".
EventBridge (via CDK) only allows me to define the cron trigger based on UTC. I'm triggering the function at 1 PM as well as 2 PM UTC, to account for daylight savings, and detecting whether daylight savings is in effect in the lambda code. That way, I can skip if it isn't 9 AM Eastern time and run the actual code when it really is 9 AM.
Here's how the code looks like:
def aws_lambda(event, context):
cron_expression = os.getenv('cron_expression') # The cron in CDK also passed as env var
local_time = localized_time_now()
if local_time.hour != 9:
return
window_start = local_time - timedelta(days=1)
window_stop = local_time
iter = croniter(cron_expression, window_start, ret_val=datetime)
prev_invocation = iter.get_prev()
if prev_invocation > window_start:
raise RuntimeError("Cron expression needs to allow for 24 hours to pass between events")
# Business logic here, using window_start and window_stop to query the DB
...
How do I test this function? I can't control the server time in EventBridge or Lambda, so it seems inherently untestable. It seems like the localized_time_now() function is key, but I'll just have to trust that it will always work properly in the context of AWS?

In my opinion localized_time_now() works with the TZ environment variable (look for TZ). So You have to set this varialble to Your timezone first. This is of course possible with CDK.
environment: {
TZ: 'Europe/Berlin',
}
After that datetime.datetime.now() should return the local time.
This is kind of awful that we can only set the event schedule in UTC, but it is how it is and running the lambda every hour and checking if it is the correct one is a valid solution for me here.
The second thing is how to test this. You can of course run this lambda locally with diffrent event timestamps. You can do unit tests or probably better at this point would be to use the lambda-local package. Consider also playng around with test events in the lambda console. There You can play around with the actual runtime.

Python convert timestamps with specific timezone to datetime in UTC

I'm trying to convert a timestamp with a specific timezone(Europe/Paris) to a datetime format in UTC.
From my laptop it works with the solution below but when I'm executing my code in a remote server(AWS- Lambda function in Ireland), I've a shift of 1 hour because the local timezone of the server is different from mine.
How can I have a code who can work on my laptop and at the same time in a remote server(dynamically handle local timezone)?
import pytz
import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
utc = pytz.timezone('UTC')
now_in_utc = datetime.datetime.utcnow().replace(tzinfo=utc).astimezone(pytz.UTC)
fr = pytz.timezone('Europe/Paris')
new_date = datetime.datetime.fromtimestamp(timestamp_received)
return fr.localize(new_date, is_dst=None).astimezone(pytz.UTC)
Thanks

I am not sure what timestamp_received is, but I think what you want is utcfromtimestamp()
import pytz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_naive_utc = datetime.utcfromtimestamp(timestamp_received)
return dt_naive_utc.replace(tzinfo=pytz.utc)
For completeness, here is another way to accomplish the same thing by referencing python-dateutil's tzlocal time zone:
from dateutil import tz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_local = datetime.fromtimestamp(timestamp_received, tz.tzlocal())
if tz.datetime_ambiguous(dt_local):
raise AmbiguousTimeError
if tz.datetime_imaginary(dt_local):
raise ImaginaryTimeError
return dt_local.astimezone(tz.tzutc())
class AmbiguousTimeError(ValueError):
pass
class ImaginaryTimeError(ValueError):
pass
(I added in the AmbiguousTimeError and ImaginaryTimeError conditions to mimic the pytz interface.) Note that I'm including this just in case you have a similar problem that needs to make reference to the local time zone for some reason - if you have something that will give you the right answer in UTC, it's best to use that and then use astimezone to get it into whatever local zone you want it in.
How it works
Since you expressed that you were still a bit confused about how this works in the comments, I thought I would clarify why this works. There are two functions that convert timestamps to datetime.datetime objects, datetime.datetime.fromtimestamp(timestamp, tz=None) and datetime.datetime.utcfromtimestamp(timestamp):
utcfromtimestamp(timestamp) will give you a naive datetime that represents the time in UTC. You can then do dt.replace(tzinfo=pytz.utc) (or any other utc implementation - datetime.timezone.utc, dateutil.tz.tzutc(), etc) to get an aware datetime and convert it to whatever time zone you want.
fromtimestamp(timestamp, tz=None), when tz is not None, will give you an aware datetime equivalent to utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(tz). If tz is None, instead of converting too the specified time zone, it converts to your local time (equivalent to dateutil.tz.tzlocal()), and then returns a naive datetime.
Starting in Python 3.6, you can use datetime.datetime.astimezone(tz=None) on naive datetimes, and the time zone will be assumed to be system local time. So if you're developing a Python >= 3.6 application or library, you can use datetime.fromtimestamp(timestamp).astimezone(whatever_timezone) or datetime.utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(whatever_timezone) as equivalents.

Python, get timestamp for specific timezone

I am new to Python and I tried to find the answer from the existing posts, and I did my attempt but I can't find what I want.
I need to validate the time(based of china timezone regardless of where the client at) diff when the client send requests to my server .
From the existing posts I can find, I had tried:
import calendar
import datetime
import pytz
import time
tz = pytz.timezone('Asia/Shanghai') # china timezone
cn_time = datetime.datetime.now(tz) # get datetime for china
print calendar.timegm(cn_time.timetuple())*1000 #try to get the milliseconds
But I find that the result is far away from my java server's answer from Joda Time:
DateTime serverDt = new DateTime(DateTimeZone.forID("Asia/Shanghai"));
long milis = serverDt.getMillis();
One test case is:
python : 1457005502000
java: 1456976702999
seonds diff from int secDiff = Seconds.secondsBetween(dt, serverDt).getSeconds(); is -28799 which is -7 hours
Note: My machine is at china timezone.

Your code tries to find the current Unix time. That value does not depend on a specific timezone (it is the same number on all computers with synchonized (e.g., using ntp) clocks whatever (perhaps different) time zones they use). Just call milis = time.time() * 1000, to get the same value as the java server.
If you need to get the Unix time that corresponds to a given timezone-aware datetime (such as created by datetime.now(tz) at some point) then just call posix_timestamp = cn_time.timestamp().
There is no datetime.timestamp() method on Python 2. You could emulate it easily:
def timestamp(aware_dt, epoch=datetime(1970, 1, 1, tzinfo=pytz.utc)):
return (aware_dt - epoch).total_seconds()
Usage: posix_timestamp = timestamp(cn_time). See more details in this answer.

Code resulting in weird datetime result on Google App Engine

I'm running a python program on Google App Engine that uses the datetime function. It's supposed to always return UTC time, but it seems to intermittently give an incorrect time. I'm not sure if there's an error with my code, or whether this is an issue on Google's side.
To get my local time (GMT +8:00), I run this function:
def SGTOffset(obj=datetime.now()):
if isinstance(obj, datetime):
return obj + timedelta(hours=8)
return SGTOffset(datetime.now())
and in my main program:
today_date = commoncode.SGTOffset().date()
logging.debug('Today date: %s | Today datetime: %s' % (today_date.isoformat(), commoncode.SGTOffset().isoformat()))
In the logs, I get the following:
[25/Nov/2015:09:00:02 -0800] "GET ... etc ...
01:00:03.287 Today date: 2015-11-25 | Today datetime: 2015-11-25T15:38:20.804300
So, Google kindly formats the log datetime to my locale (GMT +8), showing that the code is run at 01:00:03.287 (26th Nov, GMT +8). Also, this is confirmed by the timestamp 25/Nov/2015:09:00:02 -0800 provided. So the code was run at 25/Nov/2015 17:00:02 UTC time.
However, my code is outputting the wrong time. The datetime that is being generated in the code 2015-11-25T15:38:20.804300 has the timezone of GMT-9:30 instead of UTC time. (Because SGToffset() adds 8 hours to datetime)
This is quite catastrophic as I use the local datetime in many areas of my program. This is also happening intermittently only, because yesterday, the same code ran and got this log:
[24/Nov/2015:09:00:00 -0800] "GET ... etc ...
01:00:02.237 Today date: 2015-11-25 | Today datetime: 2015-11-25T01:00:01.768140
Which is correct! (Google's log timestamp 01:00:02.237 matches the time generated by SGTOffset() which is 01:00:01)
Could I know what is wrong with my program, or whether this is an issue with Google App Engine?
Thank you spending time to read this question!

The problem lies with the code.
Python stores a default value of the parameter obj for the function SGTOffset() when it is first defined (when the function object is first instantiated), instead of whenever the function is called as I intuitively expected. So, the datetime value will reflect the start time of the instance in GAE.
In order to get the current time whenever I call SGTOffset() without any parameters, I should instead have used:
def SGTOffset(obj=None): # Instead of obj=datetime.now() which is converted to obj='specific datetime' when instantiated
if isinstance(obj, datetime):
return obj + timedelta(hours=8)
return SGTOffset(datetime.now())
In this case, datetime.now() is called dynamically whenever it is required.
I arrived at this solution after viewing a question about old datetime values being returned.

I'm adding a quick answer to give you suggestions to make your code more readable:
obj is not a good variable name because it is not informative
No need for a recursive call to the function
Better to not use isinstance because is None gives you the needed functionality and your code will not work if some other instance type is given anyway.
Here is my suggestion:
def SGTOffset(dt=None):
if dt is None:
dt = datetime.now()
return dt + timedelta(hours=8)
Or if you prefer brevity:
def SGTOffset(dt=None):
return (dt or datetime.now()) + timedelta(hours=8)

How to test datetime conversion with pytz and tzlocal?

So I have datetime objects that I want to show up for users in their local time.
Using answered questions on here, I've come up with a jinja filter to accomplish this:
from tzlocal import get_localzone
import pytz
def local_datetime(utc_dt):
local_tz = get_localzone()
local_dt = utc_dt.replace(tzinfo=pytz.utc).astimezone(local_tz)
return local_dt.strftime('%m/%d/%y # %I:%M %p')
app.jinja_env.filters['local_dt'] = local_datetime
{{ user.last_login_at|local_dt }} # in my template
My thought was that it would run each time someone views the page (hence the filter) so that it will always show in the user's native timezone.
It shows up right on my development machine, but I'd like to make sure that get_localzone() is actually grabbing the user's local timezone and not always the server's.
My question is: How can I test if this is working correctly?

get_localzone() will always return the local timezone of the server your application is running on.
There is nothing in the HTTP headers of the request which can tell you the user's timezone. Instead, the standard way of approaching this is to ask the user to tell you their preferred timezone.
See Determine a User's Timezone for more discussion about this.