Develop Reference

Getting API Report Results into a pandas dataframe

I am having an issue because of one of my vendors. For some reason whenever I run any report through their statistics API it is always ran using Pacific Standard Time, regardless of the fact that I am in Eastern Standard Time. To account for this, I have to run the report with the start and end date dialed back by three hours, then I need to manually change the time of the "TimeStamp" column forward by three hours. Finally I need all the results input into my MS SQL instance. I have gotten to the point where I can get the results back, but I am stuck on what to do next. My instincts say it's going to probably be a pandas solution, but I am not sure how to get the results into the pandas dataframe. Here is what I have so far (note the vendor I am working with is called Five9, and I found a library for them that helps me connect to the API and get the report results I want):
from five9 import Five9
import datetime
from datetime import datetime, timedelta
import time
from pytz import timezone
import pyodbc
import json
now_utc = datetime.now(timezone('UTC'))
now_eastern = now_utc.astimezone(timezone('US/Eastern'))
#Change days from current time
startreportime = now_eastern - timedelta(days=2)
endreportime = now_eastern - timedelta(days=1)
#Set start and end time for report criteria
starttime = f"{(startreportime):%Y-%m-%d}" + 'T21:00:00.000'
endtime = f"{(endreportime):%Y-%m-%d}" + 'T20:59:00.000'
#connect to API
client = Five9('MyUID','MyPWD')
#Set variables as start and end
start = starttime
end = endtime
#set criteria using variables
criteria = {'time':{'end':end, 'start':start}}
#Get report and seet criteria for report
identifier = client.configuration.runReport(folderName='Five9 Import Data',\
reportName='Agent State Details',criteria=criteria)
#Sleep so report has time to complete
time.sleep(30)
#Get report results
get_results = client.configuration.getReportResult(identifier)
results = get_results['records']
print(results)
Using this I get these kinds of results:
[{
'values': {
'data': [
'Mon, 22 Feb 2021 21:00:00',
'abowling#*****.com',
'Adam',
'Bowling',
'Login',
None,
None,
'TUPSS, Telamon Inbound, Stericycle Environment Inbound, Stericycle ComSol Inbound,
'01:18:05',
'08 - TS'
]
}
If I could get these results into a dataframe I am pretty sure I could manage the rest. I know how to use a timedelta to handle the timestamp issues, and I can handle getting it from a dataframe to sql. I am just having a heck of a time trying to figure out how to get these results into a dataframe.

Not sure if anyone will read this, but I got it to work with the following:
def process_rows(rows):
for row in rows:
date1 = row['values']['data'][0]
date1 = datetime.strptime(date1, '%a, %d %b %Y %H:%M:%S').astimezone(timezone("US/Pacific"))
date2 = date1.astimezone(timezone("US/Eastern"))
date2 = date2.strftime('%Y-%m-%d %H:%M:%S')
cloned_row = [value for value in row['values']['data']]
cloned_row[0] = str(date2)
if cloned_row[8] == '24:00:00':
cloned_row[8] = '00:00:00'
yield cloned_row
args = process_rows(results)
insertSQL = ('''
INSERT INTO [Reporting].[dbo].[AgentState]
(TimeStamp, Agent, FirstName, LastName, State, ReasonCode, Media, Skill, StateTime, [Group])
VALUES (?,?,?,?,?,?,?,?,?,?)
'''
)
cursor.fast_executemany = True
cursor.executemany(insertSQL, args)
conn.commit()

Convert timestamp to time only

I'm getting the calendar results from outlook, fetching only the Start time and the Subject of each calendar item.
import win32com, win32com.client
import datetime, time, pytz
def getCalendarEntries():
Outlook = win32com.client.Dispatch("Outlook.Application")
appointments = Outlook.GetNamespace("MAPI").GetDefaultFolder(9).Items
appointments.Sort("[Start]");
appointments.IncludeRecurrences = "True"
today = datetime.datetime.today().date().strftime("%Y-%d-%m")
tomorrow = (datetime.date.today() + datetime.timedelta(days=1)).strftime("%Y-%d-%m")
appointments = appointments.Restrict("[Start] >= '" +today+"' AND [Start] < '"+tomorrow+"'");
events={'Start':[],'Subject':[]}
for a in appointments:
events['Start' ].append(a.Start );
events['Subject'].append(a.Subject)
return events
calendar = getCalendarEntries();
n=len(calendar['Start']);
i=0;
while( n ):
print(
calendar['Start'][i] ,
calendar['Subject'][i]
);
n-=1;
i+=1;
This is the result, and it is correct:
$ py test_outlook.py
2019-12-06 10:00:00+00:00 test apointment
What I need now is to manipule this data above to get only the time: 10:00, so that I can do calculations and find out how much time there is until the event starts... like if it's 10min away, 1h away, etc.
I really have no idea on how to do it... anyone has any idea?

Uri Goren seems to have answered the question here: https://stackoverflow.com/a/38992623/8678978
You need to use strptime with the datetime format to get a date object, and then you can extract the time portion.
dateString = '2019-12-06 10:00:00+00:00'
dateObject = datetime.datetime.strptime(str[0:19], '%Y-%m-%d %H:%M:%S')
Now you have a date object and can get the time parts using:
dateObject.hour
dateObject.minute
dateObject.second

I am not sure what type getCalendarEntries returns. You can find out by adding an additional temporary line in your program:
print(type(calendar['Start'][i]))
If it is a datetime object, you can simply query the hour attribute:
hours = calendar['Start'][i].hour
If getCalendarEntries returns a POSIX timestamp, you can first convert it to a Python datetime object and then query the hour
dt = datetime.fromtimestamp(calendar['Start'][i])
hours = dt.hour
If it is a string, you can parse it using datetime.fromisoformat:
dt = datetime.datetime.fromisoformat(calendar['Start'][i])
hours = dt.hour

Python3 Timestamp to unix time (with sometime milliseconds)

I'm trying to convert a timestamp to unix time.
Timestamp looks like this:
2018-01-16T20:26:16.35
So now I use this code:
timestamp = '2018-01-16T20:26:16.35'
ts = timestamp.replace("T", " ")
tsUnix = time.mktime(datetime.datetime.strptime(ts, "%Y-%m-%d %H:%M:%S.%f").timetuple())
Sometime the api where get the timestamps from gives me the time without milliseconds.
It will result in a error, 'Does not match the format'. (Duh, %f is not in there)
Do I need to script a workaround or is there a function for this? And is there a better way to strip the "T" from the original timecode?

Here is one way to handle your issue, using try and except:
import time
import datetime
def get_ts_unix(ts):
try:
tsUnix = time.mktime(
datetime.datetime.strptime(ts, "%Y-%m-%dT%H:%M:%S.%f").timetuple()
)
except ValueError:
tsUnix = time.mktime(
datetime.datetime.strptime(ts, "%Y-%m-%dT%H:%M:%S").timetuple()
)
return tsUnix
Example:
timestamps = ['2018-01-16T20:26:16.35', '2018-01-16T20:26:16']
for ts in timestamps:
print("%-24s: %f" % (ts, get_ts_unix(ts)))
Output:
2018-01-16T20:26:16.35 : 1516152376.000000
2018-01-16T20:26:16 : 1516152376.000000

To skip the ts = timestamp.replace("T", " ") add a T in the format string
timestamp = '2018-01-16T20:26:16.35'
time.mktime(datetime.datetime.strptime(timestamp, "%Y-%m-%dT%H:%M:%S.%f").timetuple())
# returns 1516130776.0
To fix the milliseconds you could use the python-dateutil package
from dateutil import parser
raw_timestamp = '2018-01-16T20:26:16.35'
parsed_time = parser.parse(raw_timestamp)
timestamp = parsed_time.timestamp()

Error while calculating time diff

I am running into following error while calculating time diff?how to fix it?
from datetime import datetime, date
import time
print "Start : %s" % time.ctime()
start_time = time.ctime()
#do something
print "How to calculate time diff?"
end_time = time.ctime()
total_elapsed_time = datetime.combine(date.today(), end_time) - datetime.combine(date.today(), start_time)
print "TOTAL ELAPSED TIME" + str(total_elapsed_time)
Error:-
Traceback (most recent call last):
File "time_diff.py", line 8, in <module>
total_elapsed_time = datetime.combine(date.today(), end_time) - datetime.combine(date.today(), start_time)
TypeError: combine() argument 2 must be datetime.time, not str

The error clearly states that argument 2 of combine() must be datetime.time.
You should have an instance of datetime.time which is something like:
d = datetime.now()
timepart = d.time()
datepart = d.date()
datetime.combine(datepart, timepart)

First of all you are mixin 3 data types, so your approach is fundamentally wrong.
There are two (basic) ways, how to get and store time in python (two from std lib):
timestamp - you import time module, then time.time() will return a float - a number of seconds from epoch, then any comparison is just like comparing float numbers, if number is greater - then time that it represents is in the future, any difference between dates is a number too - representing number of seconds between dates - so it is as well simple to convert it minutes, hours etc. by dividing by 60, 3600 etc.
datetime object you import datetime.datetime to get an object which you can operate on, it is much simpler to operate on than timestamp, because it is automatically converted to human readable format, all comparison operators works and any difference between dates (date2-date1) is returned as datetime.timedelta where you can extract number of days, seconds and milliseconds it represents.
so in your code you can use:
timestamps:
import time
date1 = time.time()
print 'Start time: %s' % time.ctime(date1)
...
date2 = time.time()
print 'End time: %s' % time.ctime(date2)
print 'Time it took: %0.2f seconds' % (date2 - date1)
datetime
from datetime import datetime
date1 = datetime.now()
print 'Start time: %s' % date1
...
date2 = datetime.now()
print 'End time: %s' % date2
print 'Time it took: %s' % (date2 - date1)

from datetime import datetime, timedelta
def calc():
s = datetime.now()
e = datetime(day=21, month=7, year=2016, hour=12, minute=0)
diff = e - s
diff is datetime.timedelta instance now. On this instance you have properties such as days, seconds, microseconds and function by name total_seconds.
PS: This is for reference only. I hope it helps to achieve a solution

time.ctime() returns a string, that is the source of your error.
See Python time.ctime
time.time() is probably what you are looking for:
See Python ticks

Your start_time and end_time are strings. Also your datetime.combine should have the 2nd argument as a datetime.time object. Your time.ctime returns the date and year information as well. So first extract only the time information and then convert it to a datetime.time object.
You can store the start and end time both using ctime and then do something like this:
import datetime
import time
#store at start time
a=datetime.datetime.strptime(time.ctime(), "%a %b %d %H:%M:%S %Y")
#store at end time
b=datetime.datetime.strptime(time.ctime(), "%a %b %d %H:%M:%S %Y")
print b-a
As #Jerzyk pointed out, there is no need to parse it unless you want to display it in a particular format. So alternatively, you could do:
a=datetime.datetime.now()
b=datetime.datetime.now()
print b-a
This will return a datetime.timedelta that you can parse in order to display the result in the way you want.

Converting string to datetime then comparing it with datetime.now

I want to take a string
"9:09am Jan 23"
and compare it with datetime.now() to get the amount of difference in hours only.
So if I did something like
now = datetime.datetime.now()
if now - (string in hours) > 24:
return True
else:
return False
I've been messing with it for a little while and cant seem to successfully compare a string (or converted string object) with datetime.now.

You can convert the string to datetime using strptime in conjunction with the relevant format. You'll need to add the current year though:
import datetime
now = datetime.datetime.now()
year = now.year
datestring = "9:09am Jan 23 " + year
dt = datetime.datetime.strptime(datestring, "%H:%M%p %b %d %Y")
Subtracting two datetimes gives you a timedelta object, so you'll need to extract the days:
diff = now - dt
days_diff = diff.days

Here is a function that will grab two instances of current time and return the difference in seconds. This is done mostly with datetime's 'strptime', in which I convert the current time to strings. If you want to ditch the current time, replace the 'time' or 'xvar' variables with a string. I used seconds here, but you can return hours of difference just as easily.
import datetime, sys
def time_passed():
while True:
inp = raw_input("Press [Y] to start, or [X] to exit.: ").lower()
if inp == 'y':
now = datetime.datetime.now()
xvar = now.strftime("%b %d %H:%M:%S")
time = datetime.datetime.strptime(xvar, "%b %d %H:%M:%S")
time = time.replace(year = 2016)
print "The first recording of the time is: {0}".format(time)
print "The second recording of the time is: {0}".format(now)
yvar = (now - time)
print "Between the two times, there is a difference of {0} seconds".format(yvar.total_seconds())
continue
elif inp == 'x':
sys.exit()
break
else:
print "Command Error!"
continue
if __name__ == '__main__':
time_passed()

Thanks for the help guys! What I wanted was how to compare the hours only, so I did a little hacking. What I needed to do was get
diff = now - dt
then to get the hours of that, which is what I was stuck on. The solution was to get the seconds of diff and divide by 3600.
My solution: a function that takes a string time and hour param to determine which messages to delete. (writing a web automation script)
def to_delete(self, date, accepted_range):
""" decides to delete a message based on certain criteria """
now = datetime.datetime.now()
date_str = "%s %s" % (date, now.year)
dt = datetime.datetime.strptime(date_str, "%H:%M%p %b %d %Y")
diff = now - dt
if diff.seconds/3600 > accepted_range:
return True
else:
return False
It gives me the hours only and now I can delete posts if they're over x hours old