I'm trying to build an app where I can an API request, but I want to send a time query where the time is the time 10 minutes ago. How should I go about doing that in Python?
Something like this should do the trick.
datetime.now(timezone.utc)
Alternatively, you could try this as well. For me, I had to add 5 hours because I'm in EST, so do the respective conversion.
curTime = (datetime.datetime.now() + datetime.timedelta(hours=5)).strftime('%Y-%m-%d %H:%M:%S')
Related
I recently started using Google's BigQuery service, and their Python API, to query some large databases. I'm new to SQL, and the BigQuery documentation isn't incredibly helpful for what I'm doing.
Currently I'm looking through the reddit_comments database, and there's 'created_utc' tag that I'm trying to filter by. This created_utc field is in terms of Unix timestamps (i.e. November 1st, 12:00 AM is 1541030400)
I'd like to grab comments day by day (or between two Unix timestamps) but in a way that I'm iterating over each day. Something like:
from datetime import datetime, timedelta
start = datetime.fromtimestamp(1538352000)
end = datetime.fromtimestamp(1541030400)
time = start
while time < end:
print(time)
time = time + timedelta(days = 1)
Printing times here yield one like: 2018-09-30 20:00:00
However in order to query, I have to convert back to the Unix timestamp by invoking datetime's timestamp() function like time.timestamp()
The problem is, I'm trying to use the timestamp() function inside the query like so:
SELECT *
FROM 'fh-bigquery.reddit_comments.2018_10'
...
AND (created_utc >= curr_day.timestamp() AND created_utc <= next_day.timestamp())
however, it's throwing a BadRequest: 400 Function not found. Is there a way to use built-in Python functions in the way that I've described above? Or does there need to be some alternative?
Everything so far seems pretty intuitive, but it's weird that I can't find much helpful information on this specifically.
You should use BigQuery's Built-in functions
For example:
To get current timestamp - CURRENT_TIMESTAMP()
To get timestamp of start of current date - TIMESTAMP_TRUNC(CURRENT_TIMESTAMP(), DAY)
To get timestamp of start of next date - TIMESTAMP_TRUNC(TIMESTAMP_ADD(CURRENT_TIMESTAMP() , INTERVAL 1 DAY), DAY)
and so on
Also, to convert created_utc to TIMESTAMP type - you can use TIMESTAMP_SECONDS(created_utc)
You can see more about TIMESTAMP Functions
I have a wired situation here with returning the time in unix format.
I have this function:
def get_today_date():
nowDate = str(datetime.datetime.utcnow().date())
unix_date = time.mktime(datetime.datetime.strptime(nowDate, "%Y-%m-%d").timetuple())
return (unix_date)
it's straightforward when I print the value of "nowDate" -> it gives me the right value I want (ex: 7/29/2018 00:00:00 UTC) which is correct. But when I change the format from Unix timestamp to date format and get only the date without time, It gives me (7/28/2018).
I've took the value in unix and checked it, it gives me (7/28/2018 21:00:00). Why?
My laptop is in a UTC +3 timezone.
Did something go wrong in the conversion? Or its an internal error in my laptop makes the output wrong?
If you're looking for today's date/time, use datetime.datetime.now(), if you need just the date, use datetime.datetime.today() it's automatically converted into your timezone.
If you play with UTC-based functions, you'll get 3 hours offset for your location, which sometimes might result in time being 21:00 of previous day instead of midnight of the next one -- that's quite reasonable =)
you may use the following approach:
utcnow = datetime.datetime.utcnow()
midnight = time.mktime( datetime.datetime( utcnow.year, utcnow.month, utcnow.day).timetuple() )
I am new to Python and need some help in being able to import done day old logs. Below is the script I have come up with, but not sure if it is working or if there is a better way to do this.
def fileCreation(path):
now = time.time()
oneday_ago = now - (24*60*60) ## seconds in 1 day
if fileCreation < oneday_ago:
print f
getAuditRecords(f)
I have a script that does import the whole database from mid June 2014 but only need to get day old logs.
Here is a sample of the logs I am trying to import
/mnt/hcp1/R1P/R1P_ora_982_2.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_20_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_29962_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_15593_2.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_9946_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_10746_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_6508_1.xml.201409070400
/mnt/hcp1/R1P/R1P_ora_17340_2.xml.201409070400
/mnt/hcp1/SCC/SCC_ora_18881_2.xml.201407090400
In order to compare the file creation time to one day ago, you need to actually get the file creation time. Your code is using fileCreation, the function; it doesn't mean anything useful to ask whether that function is less than some time.
Unfortunately, "file creation time" is not a portable concept. If you really want that, you need to write different code for different platforms, which I won't explain.
Usually, you're happy with "file modification time". This is set when the file is created, and updated only when you overwrite or append to the file. You can use getmtime to read this. So:
def fileCreation(path):
now = time.time()
oneday_ago = now - (24*60*60) ## seconds in 1 day
mtime = os.path.getmtime(path)
if mtime < oneday_ago:
print f
getAuditRecords(f)
However, it looks like there's a timestamp attached to each filename. If /mnt/hcp1/R1P/R1P_ora_982_2.xml.201409070400 means that the file was created on 7 September 2014 at 04:00 (and if the timezones, etc. aren't an issue), you may want to consider parsing those strings instead of statting the file.
And once you're parsing date strings, you might as well use the simpler and higher-level datetime library instead of the lower-level time. (You could do this with the previous version too, but since getmtime returns a time-style timestamp, you'd have to convert it manually to use it as a datetime, so there's less advantage.)
So:
def fileCreation(path):
now = datetime.datetime.now()
oneday_ago = now - datetime.timedelta(days=1)
fileext = os.path.splitext(path)[1][1:]
filetime = datetime.datetime.strptime(fileext, '%Y%m%d%H%M')
if filetime < oneday_ago:
print f
getAuditRecords(f)
(Also, I'm not sure what that f is. Maybe you meant path?)
Regarding the "two days ago" part, you should use datetime.datetime and datetime.timedelta
E.g.
import datetime
now = datetime.datetime.now()
two_days = datetime.timedelta(days=2)
two_days_ago = now - two_days
i am trying to do something very specific without using Pytz or messing with timezones.
horaactual = datetime.datetime.now()
#horaactual - 7 hours.
I would like to get the actual time with the exact second and then edit and change the hours to display 7 less hours, however i didn't seem to find an efficient way to do it and google always suggest timezones, wich is something i'd like to not use since this is a fixed time.
You can subtract a datetime.timedelta:
horaactual = datetime.datetime.now() - datetime.timedelta(hours=7)
Just subtract 7 hours:
now = datetime.datetime.now()
in_the_past = now - datetime.timedelta(hours=7)
Currently I'm creating timestamps with the Python time module. I created a string using this command
timestamp = time.strftime('%l:%M:%S')
However, this prints the format in a 12 hours format. Is there a way I can do this in a 24 hours format so it would show up as 20:52:53 instead of 8:52:53?
Thanks!
Try
timestamp = time.strftime('%H:%M:%S')
Check out this link for info on Python time module
https://docs.python.org/3/library/time.html#time.strftime