Suddenly ran into issue with this timestamp (which was hidden in JSON object):
-180835200000
Tried to extract standard timestamp with:
dt.strftime(dt.fromtimestamp(json.loads(res.text)['suggestions'][0]['data']['state']['actuality_date']/1000),'%Y-%m-%d')
But failed since the value is negative.
How do I convert this negative TIMESTAMP to human date ?
The given time is a POSIX time (aka Epoch time or Unix time). It represents the number of seconds that have elapsed since the 00:00:00 UTC on 1 January 1970, excluding leap seconds. Unix time
For your application, if the timestamp has a negative sign in front then it represents a time point before January 1, 1970. Make sure the time is in integer format and the following result is tested in Python 3.7.12;
from datetime import datetime as dt
dt.strftime(dt.fromtimestamp(-180835200000/1000),'%Y-%m-%d')
Outputs (human readable)
1964-04-09
If you believe that the negative sign is a mistake in data, you can simply use abs() function which will output 1975-09-25.
Related
Say I have a unix epoch time stamp 1664203669. When generating a UTC date from this time stamp using datetime I get the following date.
import datetime
datetime.datetime.utcfromtimestamp(1664203669).strftime("%Y-%m-%d %H:%M:%S")
Out[25]: '2022-09-26 14:47:49'
Though when using skyfield I get the following date
from skyfield.api import load
load.timescale().utc(1970, 1, 1, 0, 0, 1664203669).utc_strftime()
Out[27]: '2022-09-26 14:47:22 UTC'
These UTC times are 27 seconds apart. Corresponding with the total number of leap seconds since 1970 to the given time stamp. From the skyfield documentation this construction should handle leap seconds, but it doesn't seem to?
Why are these two date different?
You have asked the two libraries two different questions, and they each answered correctly.
You asked Python's datetime module: What does the Unix timestamp 1664203669 correspond to?
To make the math easy, Unix timestamps treat each day as containing exactly 60×60×24 seconds. This means that Unix timestamps can't refer to leap seconds; they skip them as if they don't exist. So datetime can simply divide 1664203669 by the number of seconds in a normal day, and then compute hours and minutes from the remainder.
You asked Skyfield something different: What UTC date and time followed exactly 1,664,203,669 seconds after the beginning of 1970?
Because Skyfield knows all about leap seconds, it understands that 27 of those 1.66-million seconds get eaten up by leap seconds through the years between 1970 and 2022, so it returns a time of day that's earlier.
I'm trying to wrap my head in understanding the implication of using .utcnow vs. .now on Python's DateTime.
Here's the reason for my confusion: I live in France. Right now, we have a +1 hour on the UTC timezone (CET timezone in winter (now) / CEST (+2) timezone in summer).
If I take the following value :
dt = datetime.datetime.utcnow()
dt.strftime('%c') # Thu Dec 9 16:17:38 2021
int(dt.timestamp()) # 1639063064
This is correct as it is, in France right now, 17h17.
So, from my understanding, that timestamp, 1639063064, is the UTC representation of the time since EPOCH.
But if I test this value in the website Epoch Converter, I get
GMT: Thursday 9 December 2021 15:17:44
Your time zone: jeudi 9 décembre 2021 16:17:44 GMT+01:00
It seems that the website ALSO subtracts my timezone to an already "substracted" value, ending in removing twice the timezone and causing an invalid value.
The actual confusion is when I tried to import that UTC timestamp to Luxon on my front app, doing the following doesn't work :
DateTime.fromMillis(parseInt(ts), { zone: 'utc' }).toLocal().setLocale('en')
I'm one hour behind.
How can I "tell" Luxon that the current TS is in the UTC timezone, and calling toLocal will apply the proper user's timezone ?
It seems that the website ALSO substract my timezone t
No, epochconverter.com isn't doing anything. The value 1639063064 really does represent 2021-12-09T15:17:44Z. That's not the value you want.
I'm no Python expert, but I believe the problem is the combination of this utcnow() behavior (emphasis mine):
Return the current UTC date and time, with tzinfo None.
This is like now(), but returns the current UTC date and time, as a naive datetime object.
And this timestamp() behavior:
Naive datetime instances are assumed to represent local time and this method relies on the platform C mktime() function to perform the conversion.
It sounds like you want to follow this advice:
An aware current UTC datetime can be obtained by calling datetime.now(timezone.utc).
So just change your first line to:
dt = datetime.now(timezone.utc)
... and it should be okay.
I'm parsing and manipulating some dates and times which, for reasons of interoperability with other systems, also need to be stored as UNIX (epoch) timestamps. In doing so, I'm seeing some weird behavior from pandas' Timestamp.tz_convert(), and then in its Timestamp.strftime() behavior in casting to epoch time, that makes me doubt my understanding of what should be going on.
The times I'm working with are in the US/Eastern timezone, but of course, epoch time is UTC, so my approach had been to cast to UTC since most conversions to/from UNIX timestamps assume that a tz-naive DateTime is in UTC. Let's leave aside whether doing that conversion is absolutely necessary to get valid timestamps; here's what I'm seeing that's problematic:
1. Using Timestamp.tz_convert() to change the timezone representation of a timestamp (i.e., a universal point in time) also changes the UNIX timestamp when you convert using Timestamp.strftime().
2. The differences in those timestamps don't even correspond to the proper hour differences between US-Eastern and GMT.
Here's some basic interactive-mode python to illustrate:
>>> import pytz
>>> from pytz import timezone
>>> import pandas as pd
>>> dtest = pd.to_datetime("Sunday, July 28, 2018 10:00 AM", infer_datetime_format=True).replace(tzinfo=timezone('America/New_York')) # okay, this should uniquely represent a point in time
>>> dtest
Timestamp('2018-07-28 10:00:00-0400', tz='America/New_York') # yup, that's the time - 10AM at GMT-0400.
>>> dtest2 = dtest.tz_convert('UTC') # convert to UTC
>>> dtest2
Timestamp('2018-07-28 14:00:00+0000', tz='UTC') # yup, same point in time, just different time zone now
>>> dtest.strftime('%s') # let's convert to unix time - this looks right
'1532786400'
>>> dtest2.strftime('%s') # should be the same, but it's not. WTF?
'1532804400'
The timestamps look like they are describing things equivalently: one is 10 AM at GMT-0400, the other is 2 PM at GMT+0000, a difference of 4 hours of clock time, as expected. They're both, of course, timezone-aware. But then converting them to UNIX timestamps yields
(A) different numbers, and even worse,
(B) numbers that differ by 5 hours (18000 seconds = 5 * 60 * 60) rather than 4, so I can't even assume that strftime() is merely ignoring timezone.
I'm using https://www.epochconverter.com/ to validate any timestamps as I sanity-check this, so that's a possible point of being misled. But according to that site,
1532786400 = 2018-07-28T10:00 -0400, and
1532804400 (that last result) = 2018-07-28T15:00 -0400, or 7pm GMT, a difference of 5 hours.
There are lots of questions on the subject of casting pandas Timestamps FROM a UNIX timestamp, but very little on questions casting TO epoch time. I can think of 2 possible explanations:
(1) tz_convert() is pulling some environment variable on my system that says I'm GMT -0500 and using that in the conversion process, in spite of that being irrelevant to converting between timezone-aware timestamps, and in so doing is actually changing the underlying point in time being represented. Or:
(2) Timestamp.strftime() is bugged and either ignoring the timezone parameter of a tz-aware timestamp or doing something truly bizarre when asked for a '%s' formatting parameter.
All advice greatly appreciated.
I have a wired situation here with returning the time in unix format.
I have this function:
def get_today_date():
nowDate = str(datetime.datetime.utcnow().date())
unix_date = time.mktime(datetime.datetime.strptime(nowDate, "%Y-%m-%d").timetuple())
return (unix_date)
it's straightforward when I print the value of "nowDate" -> it gives me the right value I want (ex: 7/29/2018 00:00:00 UTC) which is correct. But when I change the format from Unix timestamp to date format and get only the date without time, It gives me (7/28/2018).
I've took the value in unix and checked it, it gives me (7/28/2018 21:00:00). Why?
My laptop is in a UTC +3 timezone.
Did something go wrong in the conversion? Or its an internal error in my laptop makes the output wrong?
If you're looking for today's date/time, use datetime.datetime.now(), if you need just the date, use datetime.datetime.today() it's automatically converted into your timezone.
If you play with UTC-based functions, you'll get 3 hours offset for your location, which sometimes might result in time being 21:00 of previous day instead of midnight of the next one -- that's quite reasonable =)
you may use the following approach:
utcnow = datetime.datetime.utcnow()
midnight = time.mktime( datetime.datetime( utcnow.year, utcnow.month, utcnow.day).timetuple() )
import arrow
print arrow.utcnow()
print arrow.utcnow().timestamp
print arrow.utcnow().to('Asia/Kolkata')
print arrow.utcnow().to('Asia/Kolkata').timestamp
I need the timestamp (in int) of 'Asia/Kolkata' timezone, which is +5:30 from utc.
arrow.utcnow() and arrow.utcnow().to('Asia/Kolkata') are coming out to be different and the second one is +5:30 the first, as expected.
However, arrow.utcnow().timestamp and arrow.utcnow().to('Asia/Kolkata').timestamp are still coming out to be same.
I am sure I am missing something very basic here, but can anyone explain this?
I think "timestamp", by definition, is always in UTC:
The Unix time (or Unix epoch or POSIX time or Unix timestamp) is a
system for describing points in time, defined as the number of seconds
elapsed since midnight proleptic Coordinated Universal Time (UTC) of
January 1, 1970, not counting leap seconds.
If you take your localized time string, convert it to a UTC date time (that is, 5pm Kolkata time becomes 5pm UTC), then you can get a timestamp that corresponds to the local clock time. Example:
import arrow
print arrow.utcnow()
print arrow.utcnow().timestamp
kolkata = arrow.utcnow().to('Asia/Kolkata')
print kolkata.replace(tzinfo='UTC').timestamp
Timestamps are UTC, this is also described in the Arrow docs
timestamp
Returns a timestamp representation of the Arrow object, in
UTC time.
Arrow will let you convert a non-UTC timestamp into arrow time but it won't let you shoot yourself in the foot by generating non-UTC timestamps for you.
classmethod fromtimestamp(timestamp, tzinfo=None)
Constructs an Arrow
object from a timestamp, converted to the given timezone.
Parameters: timestamp – an int or float timestamp, or a str that
converts to either. tzinfo – (optional) a tzinfo object. Defaults to
local time. Timestamps should always be UTC. If you have a non-UTC
timestamp:
arrow.Arrow.utcfromtimestamp(1367900664).replace(tzinfo='US/Pacific')
<Arrow [2013-05-07T04:24:24-07:00]>
The full docs are here:
http://arrow.readthedocs.io/en/latest/