Develop Reference

web scraping convert unix timestamp to date format

I'm trying to webscrape a flight data website using beautifulsoup in python but the timestamp is in unix timestamp how can i convert to regular datetime format. There are several such columns to be converted.
#scheduled_departure
result_items[0]['flight']['time']['scheduled']['departure']
and the output is shown as 1655781000. how can I convert it to Tue, Jun 21, 2022 8:40 AM

import time
print(time.strftime("%a, %b %d, %Y %H:%M %p", time.localtime(1655781000)))

There is only one Unix time and it is created by using the UTC/GMT time zone. This means you might have convert time zones to calculate timestamps.
import datetime
from pytz import timezone
local_datetime = datetime.datetime.fromtimestamp(1655781000)
local_time_str = datetime.datetime.strftime(local_datetime, "%a, %d %b %Y %H:%M:%S %p")
print(f'Local time: {local_time_str}')
other_timezone = 'Asia/Kolkata' # Replace your interest timezone here
remote_datetime = local_datetime.astimezone(timezone(other_timezone))
remote_time_str = datetime.datetime.strftime(remote_datetime, "%a, %d %b %Y %H:%M:%S %p")
print(f'Time at {other_timezone }: {remote_time_str}')

strftime formatting - what am I doing wrong?

I've got a string 3:01 AM - 18 Dec 2017
I've written the following pattern strftime('%-I:%M %p - %-d %b %Y') and I can't seem to get it to work, following this
My notes:
%-I hour 12-hour clock as a decimal (no zero padding)
: separation between hour and minute
%M minute as a zero padded decimal
%p AM/PM
- space-dash-space pattern betwen time and date
%-d date of the month as a decimal (no zero padding)
%b Month abbreviated
%Y Year with century as decimal
df['tweet_date'] = pd.to_datetime(df['tweet_date'], errors='coerce').apply(lambda x: x.strftime('%I:%M %p - %d %b %Y')if not pd.isnull(x) else '')
On another dataframe with a similar column this works:
df2['created_at'] = pd.to_datetime(df2['created_at'], errors='coerce').apply(lambda x: x.strftime('%Y-%m-%d %H:%M:%S')if not pd.isnull(x) else '')
df2['created_at'] = df2['created_at'].astype('datetime64[s]')`
where values before formatting look like this for example 2017-10-03T15:48:10.000Z

Your format is fine but some os's can't use the negative formatting for zero-padded units. datetime should be able to parse both padded and non-padded instances of those just fine:
from datetime import datetime as dt
z_time = '06:48 PM - 03 Jun 2021'
nz_time = '6:48 PM - 3 Jun 2021'
dt.strptime(z_time, '%I:%M %p - %d %b %Y')
[out:] datetime.datetime(2021, 6, 3, 18, 48)
dt.strptime(nz_time, '%I:%M %p - %d %b %Y')
[out:] datetime.datetime(2021, 6, 3, 18, 48)
And since you're getting strings from datetimes, you should look whether your os supports certain formatting rules. Here's one for windows.

from datetime import datetime
str="3:01 AM - 18 Dec 2017"
date=datetime.strptime(str,"%I:%M %p - %d %b %Y")

To turn your string into a time, do this:
>>> import time
>>> s = "3:01 AM - 18 Dec 2017"
>>> time.strptime(s,'%I:%M %p - %d %b %Y')
time.struct_time(tm_year=2017, tm_mon=12, tm_mday=18, tm_hour=3, tm_min=1,
tm_sec=0, tm_wday=0, tm_yday=352, tm_isdst=-1)
No hyphens after %. The are not mentioned in the official Python documentation.

I wanted to figure out how to use strftime in a lambda function or better, using the .dt accessor after the column in my dataframe had been converted to a datetime
I couldn't figure this out so I went for the next fastest method
from datetime import datetime
formatted_dates = []
for item in df.tweet_date:
formatted_dates.append(datetime.strptime(item,"%I:%M %p - %d %b %Y"))
df.tweet_date = formatted_dates

How to create date timestamp logs like git log

Using datetime.datetime.now(), I receive some badly formatted timestamps.
Is there an intuitive way of creating a date timestamp in this format?
Wed Aug 7 13:38:59 2019 -0500
This is seen in git log.

You can use datetime.datetime.strftime() to format dates as shown below:
from datetime import datetime
d = '2019-08-07 13:38:59-0500'
d2 = datetime.strptime(d, '%Y-%m-%d %H:%M:%S%z')
d3 = d2.strftime('%a %b %d %H:%M:%S %Y %z')
print(d3)
This returns:
Wed Aug 07 13:38:59 2019 -050000
This website is a great resource for strftime formatting.

You can still use the datetime library. Using strftime, you can rewrite the datetime object into a nicely formatted string.
In your case, you are going for Wed Aug 7 13:38:59 2019 -0500, which translated to strftime formatting is "%a %b %d %H:%M:%S %Y %z".
Overall, it'd be
datetime.datetime.now().strftime("%a %b %d %H:%M:%S %Y %z")
Which will give a string that looks like 'Wed Aug 7 13:38:59 2019 -0500'.

I would do the following:
from time import gmtime, strftime
if __name__ == "__main__":
time = strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())
print(time)
This was found on the documentation page of the time module. There are also a lot of additional features you might be interested in using outlined here: https://docs.python.org/3/library/time.html#time.strftime

Converting from unix time string to python date type, Does not convert when CDT is present

I have a date time string in this format in python
"Wed Mar 20 00:52:54 CDT 2019 20 Mar 2019"
I am trying to convert this to python date time format using below code
datetime.datetime.strptime('Wed Mar 20 00:52:54 CDT 2019', "%a %b %d %H:%M:%S %Y")
But it gives error as CDT is present in the text. How to handle this problem. I am not sure if CDT will be present for all dates. In both cases i want to convert.
My requirement is only to get the difference in days between this date and now

Use CDT in the format, then calculate the difference between now and that date. Finally print delta.days.
from datetime import datetime
date0 = datetime.strptime('Wed Mar 20 00:52:54 CDT 2019', "%a %b %d %H:%M:%S CDT %Y")
date_now = datetime.now()
delta = date_now - date0
print(delta.days)
Output in this case:
0
If you're not sure if there will be CDT, check the string before passing it to strptime. Or use try/except:
try:
date0 = datetime.strptime(string, "%a %b %d %H:%M:%S CDT %Y")
except ValueError:
date0 = datetime.strptime(string, "%a %b %d %H:%M:%S %Y")

Datetime - Strftime and Strptime

Date = datetime.datetime.today().strftime("%d %B %Y")
x = datetime.datetime.strptime(Date , "%d %B %Y")
returns:
2018-05-09 00:00:00
instead of:
9 May 2018, what am I doing wrong?
Essentially I am trying to get the non zero padded version of strftime for which I searched around and I read that the way to go about it is strptime.

A possible solution is to use str.lstrip
Ex:
import datetime
Date = datetime.datetime.today().strftime("%d %B %Y")
print(Date.lstrip("0"))
Output:
9 May 2018