I am trying to convert string to Datetime- but the conversion adds 5 hours to the original time. How do I convert but keep the time as is?
>>> import pandas as pd
>>> t = pd.to_datetime("2016-09-21 08:56:29-05:00", format='%Y-%m-%d %H:%M:%S')
>>> t
Timestamp('2016-09-21 13:56:29')
The conversion doesn't add 5 hours to the original time. Pandas just detects that your datetime is timezone-aware and converts it to naive UTC. But it's still the same datetime.
If you want a localized Timestamp instance, use Timestamp.tz_localize() to make t a timezone-aware UTC timestamp, and then use the Timestamp.tz_convert() method to convert to UTC-0500:
>>> import pandas as pd
>>> import pytz
>>> t = pd.to_datetime("2016-09-21 08:56:29-05:00", format='%Y-%m-%d %H:%M:%S')
>>> t
Timestamp('2016-09-21 13:56:29')
>>> t.tz_localize(pytz.utc).tz_convert(pytz.timezone('America/Chicago'))
Timestamp('2016-09-21 08:56:29-0500', tz='America/Chicago')
To achieve what you want you can remove the "-5:00" from the end of your time string "2016-09-21 08:56:29-05:00"
However, Erik Cederstrand is correct in explaining that pandas is not modifying the time, it's simply displaying it in a different format.
Related
I have a date column in my dataframe that consists of strings like this...'201512'
I would like to convert it into a datetime object of just year to do some time series analysis.
I tried...
df['Date']= pd.to_datetime(df['Date'])
and something similar to
datetime.strptime(Date, "%Y")
I am not sure how datetime interfaces with pandas dataframes (perhaps somebody will comment if there is special usage), but in general the datetime functions would work like this:
import datetime
date_string = "201512"
date_object = datetime.datetime.strptime(date_string, "%Y%m")
print(date_object)
Getting us:
2015-12-01 00:00:00
Now that the hard part of creating a datetime object is done we simply
print(date_object.year)
Which spits out our desired
2015
More info about the parsing operators (the "%Y%m" bit of my code) is described in the documentation
I would look at the module arrow
https://arrow.readthedocs.io/en/latest/
import arrow
date = arrow.now()
#example of text formatting
fdate = date.format('YYYY')
#example of converting text into datetime
date = arrow.get('201905', 'YYYYMM').datetime
I have a string in the form of 68.830320 Format. this I need to convert to time Format in second.millisecond. it does not contain date or any other values. I cannot use strptime since the Format is not right. tstamp that I'm trying to parse is a list of calues containg values with decimal Point. I cannot round this value. it still gives error. I'm not sure how to proceeed. please help!
tried a lot of threads from here that always take the datetime object. But since my Format is not in the same way, I cannot use that info. I have tries .time dateutil, and everything else available. I still cannot solve this problem
tstamp = child2.get('timestamp').replace(" ", "").replace("\n", "")
print(tstamp)
parser.parser(tstamp)
format_time = datetime.date(tstamp)
print(format_time)
A number of seconds isn't a datetime, it's a timedelta. It isn't a datetime because you can't take the string "68.830320" and set the hands on a wall clock to represent that time.
Convert your string to a timedelta like this:
>>> from datetime import timedelta
>>> mytime = timedelta(seconds=float("68.830320"))
>>> mytime
datetime.timedelta(0, 68, 830320)
You can then add the timedelta to a datetime to get a wall clock time.
I am trying to set time index to pandas DataFrame using pandas.to_datetime function, but the outcome datetime is UTC when converting seconds, and while it is not requested :
import pandas
import datetime,time
datetime1 = '2017-03-30T12-00-00'
d = datetime.datetime.strptime(datetime1, "%Y-%m-%dT%H-%M-%S")
s = time.mktime(d.timetuple())
print pandas.to_datetime(datetime1, format = "%Y-%m-%dT%H-%M-%S")
print pandas.to_datetime(s, unit='s')
Get two different results, although utc option of pandas.to_datetime is not used in both cases.
Any ideas ?
time.mktime() does not return UTC by default, see the docs for time.mktime():
Its argument is the struct_time or full 9-tuple [...] which expresses the time in local time, not UTC.
I am trying to set a variable to equal today's date.
I looked this up and found a related article:
Set today date as default value in the model
However, this didn't particularly answer my question.
I used the suggested:
dt.date.today
But after
import datetime as dt
date = dt.date.today
print date
<built-in method today of type object at 0x000000001E2658B0>
Df['Date'] = date
I didn't get what I actually wanted which as a clean date format of today's date...in Month/Day/Year.
How can I create a variable of today's day in order for me to input that variable in a DataFrame?
You mention you are using Pandas (in your title). If so, there is no need to use an external library, you can just use to_datetime
>>> pandas.to_datetime('today').normalize()
Timestamp('2015-10-14 00:00:00')
This will always return today's date at midnight, irrespective of the actual time, and can be directly used in pandas to do comparisons etc. Pandas always includes 00:00:00 in its datetimes.
Replacing today with now would give you the date in UTC instead of local time; note that in neither case is the tzinfo (timezone) added.
In pandas versions prior to 0.23.x, normalize may not have been necessary to remove the non-midnight timestamp.
If you want a string mm/dd/yyyy instead of the datetime object, you can use strftime (string format time):
>>> dt.datetime.today().strftime("%m/%d/%Y")
# ^ note parentheses
'02/12/2014'
Using pandas: pd.Timestamp("today").strftime("%m/%d/%Y")
pd.datetime.now().strftime("%d/%m/%Y")
this will give output as '11/02/2019'
you can use add time if you want
pd.datetime.now().strftime("%d/%m/%Y %I:%M:%S")
this will give output as '11/02/2019 11:08:26'
strftime formats
You can also look into pandas.Timestamp, which includes methods like .now and .today.
Unlike pandas.to_datetime('now'), pandas.Timestamp.now() won't default to UTC:
import pandas as pd
pd.Timestamp.now() # will return California time
# Timestamp('2018-12-19 09:17:07.693648')
pd.to_datetime('now') # will return UTC time
# Timestamp('2018-12-19 17:17:08')
i got the same problem so tried so many things
but finally this is the solution.
import time
print (time.strftime("%d/%m/%Y"))
simply just use pd.Timestamp.now()
for example:
input: pd.Timestamp.now()
output: Timestamp('2022-01-12 14:43:05.521896')
I know all you want is Timestamp('2022-01-12') you don't anything after
thus we could use replace to remove hour, minutes , second and microsecond
here:
input: pd.Timestamp.now().replace(hour=0, minute=0, second=0, microsecond=0)
output: Timestamp('2022-01-12 00:00:00')
but looks too complicated right, here is a simple way use normalize
input: pd.Timestamp.now().normalize()
output: Timestamp('2022-01-12 00:00:00')
Easy solution in Python3+:
import time
todaysdate = time.strftime("%d/%m/%Y")
#with '.' isntead of '/'
todaysdate = time.strftime("%d.%m.%Y")
import datetime
def today_date():
'''
utils:
get the datetime of today
'''
date=datetime.datetime.now().date()
date=pd.to_datetime(date)
return date
Df['Date'] = today_date()
this could be safely used in pandas dataframes.
There are already quite a few good answers, but to answer the more general question about "any" period:
Use the function for time periods in pandas. For Day, use 'D', for month 'M' etc.:
>pd.Timestamp.now().to_period('D')
Period('2021-03-26', 'D')
>p = pd.Timestamp.now().to_period('D')
>p.to_timestamp().strftime("%Y-%m-%d")
'2021-03-26'
note: If you need to consider UTC, you can use: pd.Timestamp.utcnow().tz_localize(None).to_period('D')...
From your solution that you have you can use:
import pandas as pd
pd.to_datetime(date)
using the date variable that you use
I have a process where I read in a bunch of strings in ISO 8601 format at Zulu or UTC time. For example,
2012-06-20T21:15:00Z
2012-06-20T21:16:00Z
2012-06-20T21:17:00Z
2012-06-20T21:18:00Z
I convert the strings into timezone aware python datetime objects and then save them in a binary format as integers by converting them to Unix Timestamps. For example,
dt_str = '2012-06-20T21:15:00Z'
ts = int(mktime(datetime.strptime(dt_str, '%Y-%m-%dT%H:%M:%SZ').timetuple()))
# ts = 1340241300
When I read these timestamps back into another process I would like to instantiate a numpy.datetime64 object directly from the timestamp. The problem is that datetime64 sets the timezone to my local timezone.
np_dt = np.datetime64(ts,'s')
# np_dt = numpy.datetime64('2012-06-20T21:15:00-0400')
Does anyone know of a way that I could read in my timestamp so that it is UTC time? Would I would like for np_dt to equal is numpy.datetime64('2012-06-20T21:15:00-0000')...I think.
Regards
What about setting the timezone for your code.
import os, time
os.environ['TZ'] = 'GMT'
time.tzset()
# then your code
np_dt = np.datetime64(ts,'s')
You can use the dateutil module to help out. First, create a datetime object from the timestamp integer you saved:
>>> ts = 1340241300
>>> import datetime
>>> from dateutil import tz
>>> dt = datetime.datetime.fromtimestamp(ts).replace(tzinfo=tz.tzutc())
>>> dt
datetime.datetime(2012, 6, 20, 21, 15, tzinfo=tzutc())
Then pass it to numpy, which will convert it to the local timezone (I am in -4):
>>> import numpy as np
>>> np.datetime64(dt.isoformat())
numpy.datetime64('2012-06-20T17:15:00-0400')
Judging by the documentation, the only way to do it is to create it from a string that specifies the time zone directly. Thus you'll need to create a datetime.datetime object first and format it to a string with 'Z' appended, then construct the numpy.datetime64 from that.