I am new to Python and need some help in being able to import done day old logs. Below is the script I have come up with, but not sure if it is working or if there is a better way to do this.
def fileCreation(path):
now = time.time()
oneday_ago = now - (24*60*60) ## seconds in 1 day
if fileCreation < oneday_ago:
print f
getAuditRecords(f)
I have a script that does import the whole database from mid June 2014 but only need to get day old logs.
Here is a sample of the logs I am trying to import
/mnt/hcp1/R1P/R1P_ora_982_2.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_20_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_29962_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_15593_2.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_9946_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_10746_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_6508_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_17340_2.xml.201409070400
/mnt/hcp1/SCC/SCC_ora_18881_2.xml.201407090400
In order to compare the file creation time to one day ago, you need to actually get the file creation time. Your code is using fileCreation, the function; it doesn't mean anything useful to ask whether that function is less than some time.
Unfortunately, "file creation time" is not a portable concept. If you really want that, you need to write different code for different platforms, which I won't explain.
Usually, you're happy with "file modification time". This is set when the file is created, and updated only when you overwrite or append to the file. You can use getmtime to read this. So:
def fileCreation(path):
now = time.time()
oneday_ago = now - (24*60*60) ## seconds in 1 day
mtime = os.path.getmtime(path)
if mtime < oneday_ago:
print f
getAuditRecords(f)
However, it looks like there's a timestamp attached to each filename. If /mnt/hcp1/R1P/R1P_ora_982_2.xml.201409070400 means that the file was created on 7 September 2014 at 04:00 (and if the timezones, etc. aren't an issue), you may want to consider parsing those strings instead of statting the file.
And once you're parsing date strings, you might as well use the simpler and higher-level datetime library instead of the lower-level time. (You could do this with the previous version too, but since getmtime returns a time-style timestamp, you'd have to convert it manually to use it as a datetime, so there's less advantage.)
So:
def fileCreation(path):
now = datetime.datetime.now()
oneday_ago = now - datetime.timedelta(days=1)
fileext = os.path.splitext(path)[1][1:]
filetime = datetime.datetime.strptime(fileext, '%Y%m%d%H%M')
if filetime < oneday_ago:
print f
getAuditRecords(f)
(Also, I'm not sure what that f is. Maybe you meant path?)
Regarding the "two days ago" part, you should use datetime.datetime and datetime.timedelta
E.g.
import datetime
now = datetime.datetime.now()
two_days = datetime.timedelta(days=2)
two_days_ago = now - two_days
Related
On this problem I keep getting stuck when trying several options provided.
In simple words, I'm running a script that has a starting time (several actually, based on different criteria) and in a loop I want to display the running time of that criteria in a JSON and put it in a program (using requests) that is updated every time the loop passes one of the criteria.
I was doing that by simply running:
starting_time = datetime.now() #but just a bit earlier in the script
now = datetime.now()
running_time = now-starting_time
This running_time is then used as a variable in a JSON, but that needs to be in the format of 'HH:MM:SS' else my requests doesn't allow me to put. Which caused the problem for me, because it isn't possible to use strftime on a timedelta.
The timedelta might be based on miliseconds, but those are fine as "00:00:00"... but that caused me problems when trying to convert the timedelta to string first and then convert it back to a regular datetime.
What am I missing?
A possible workaround would be:
starting_time = datetime.now()
now = datetime.now()
running_time = now-starting_time
x = datetime.timedelta(seconds=running_time.seconds)
result = str(x)
if result[1] == ":":
result = "0"+result
print(result)
Here line 4 makes sure that x only has the seconds and ignores the miliseconds of running_time. Then we add a zero at the beginning in case needed.
But also see comment to better understand timedelta.
I recently started using Google's BigQuery service, and their Python API, to query some large databases. I'm new to SQL, and the BigQuery documentation isn't incredibly helpful for what I'm doing.
Currently I'm looking through the reddit_comments database, and there's 'created_utc' tag that I'm trying to filter by. This created_utc field is in terms of Unix timestamps (i.e. November 1st, 12:00 AM is 1541030400)
I'd like to grab comments day by day (or between two Unix timestamps) but in a way that I'm iterating over each day. Something like:
from datetime import datetime, timedelta
start = datetime.fromtimestamp(1538352000)
end = datetime.fromtimestamp(1541030400)
time = start
while time < end:
print(time)
time = time + timedelta(days = 1)
Printing times here yield one like: 2018-09-30 20:00:00
However in order to query, I have to convert back to the Unix timestamp by invoking datetime's timestamp() function like time.timestamp()
The problem is, I'm trying to use the timestamp() function inside the query like so:
SELECT *
FROM 'fh-bigquery.reddit_comments.2018_10'
...
AND (created_utc >= curr_day.timestamp() AND created_utc <= next_day.timestamp())
however, it's throwing a BadRequest: 400 Function not found. Is there a way to use built-in Python functions in the way that I've described above? Or does there need to be some alternative?
Everything so far seems pretty intuitive, but it's weird that I can't find much helpful information on this specifically.
You should use BigQuery's Built-in functions
For example:
To get current timestamp - CURRENT_TIMESTAMP()
To get timestamp of start of current date - TIMESTAMP_TRUNC(CURRENT_TIMESTAMP(), DAY)
To get timestamp of start of next date - TIMESTAMP_TRUNC(TIMESTAMP_ADD(CURRENT_TIMESTAMP() , INTERVAL 1 DAY), DAY)
and so on
Also, to convert created_utc to TIMESTAMP type - you can use TIMESTAMP_SECONDS(created_utc)
You can see more about TIMESTAMP Functions
I am trying to apply current time and date in a Windows application using Python. I used the following code:
current_time = str(time.strftime("%m.%d.%Y %H:%M"))
type(waitForObject("{name='TextBoxSampleName'}"), "Test." + current_time)
but when I run the script I see the following format
27Test.06.03.2016 10 (the hours and minutes are separated for some reason)
I want the final result to be Test.06.03.2016.10:27
Without knowing what waitForObject is, we can still get the output you desire.
First, there is a minor formatting mistake in your current_time = line:
current_time = str(time.strftime("%m.%d.%Y.%H:%M"))
^-- Add this period to get the format you want
Next, the second line can simply be (or you can assign it to a variable for later use):
print("Test."+current_time)
These two changes will print out something like this:
Test.06.03.2016.12:42
Here is my problem, I'm using Icalendar on Python to parse data from 2 ical file and one excel file (xls).
Problem, when I try to get my calendar (using this):
for event in cal.walk('vevent'):
dates = event.get('dtstart')
location = event.get('location')
dates=dates.to_ical()
month=dates[4:6]
day=dates[6:8]
heures2=dates[9:11]
minut=dates[11:13]
sec=dates[13:15]
yearical=dates[0:4]
date2=day+"/"+month+"/"+yearical
summary = event.get('summary')
icaldebheur = heures2+":"+minut+":"+sec
The problem is that my ical file look like this:
Calendar
The only hour that the code give me are the 8H one and 13h30one (observed by doing a print icaldebheur), no matter the day, it never give me the one at 10H (or, if it happen the afternoon, it don't give me one at 15H).
It seem like it only give me 2 event per day.
Do you have an idea?
One thing you have to be aware of when dealing with iCalendar files is timezones.
If the date/time property value ends in Z, that means the time is
formatted in UTC time.
If the date/time property has a TZID
parameter, that means the date/time value is formatted in a specific
timezone.
i am trying to do something very specific without using Pytz or messing with timezones.
horaactual = datetime.datetime.now()
#horaactual - 7 hours.
I would like to get the actual time with the exact second and then edit and change the hours to display 7 less hours, however i didn't seem to find an efficient way to do it and google always suggest timezones, wich is something i'd like to not use since this is a fixed time.
You can subtract a datetime.timedelta:
horaactual = datetime.datetime.now() - datetime.timedelta(hours=7)
Just subtract 7 hours:
now = datetime.datetime.now()
in_the_past = now - datetime.timedelta(hours=7)