Timestamps in QuestDB Python time.time() Not working - python

ı am building a database in QuestDB.
ı set up a table and one column is timestamp.
docs about table = https://questdb.io/docs/guides/working-with-timestamps-timezones/
timestamp column is converting automaticly 1623167145123456 to '2021-06-08T16:45:45.123456Z'.
doc say :
The native timestamp format used by QuestDB is a Unix timestamp in microsecond resolution. QuestDB does not store time zone information alongside timestamp values and therefore it should be assumed that all timestamps are in UTC. The following example shows how a Unix timestamp in microseconds may be passed into a timestamp column directly
when ı try to send this column
time.time()
it's not working.
Python time.time() = 1657105707.8171313
doc input = 1623167145123456
when ı delete this blank in python time.time() and send this to timestamp column
output is :
2495-02-11T11:02:24.069445Z
what should ı do ?

Try this out int(time.time() * 1000000000)
It is working for me in this example to send open data from TFL to QuestDB

Related

How to convert the format all the values in a date-time array?

This is my data :
dates = np.arange("2018-01-01", "2021-12-31", dtype="datetime64[D]")
I now want to convert from :
"2018-01-01" -> "Jan-01-2018" ["Monthname-day-year"] format
How to i do this ?
Is it possible to initialize this in the way we want to convert ?
Can i use something like:
for i in dates:
i = i.replace(i.month,i.strftime("%b"))
You can try this:
from datetime import datetime
dates = np.arange("2018-01-01", "2021-12-31", dtype="datetime64[D]")
result_dates = []
for date in dates.astype(datetime):
result_dates.append(date.strftime("%b-%d-%Y"))
But you will need to convert result dates as shown in the code
I feel compelled to elaborate on Silvio Mayolo's very relevant but ostensibly ignored comment above. Python stores a timestamp as structure (see How does Python store datetime internally? for more information) Hence, the DateTime does not as such have a 'format'. A format only becomes necessary when you want to print the date because you must first convert the timestamp to a string. Thus, you do NOT need to initialise any format. You only need to declare a format when the time comes to print the timestamp.
While you CAN store the date as a string in your dataframe index in a specific format, you CANNOT perform time related functions on it without first converting the string back to a time variable. ie current_time.hour will return an integer with the current hour if current_time is a datetime variable but will crash if it is a string formatted as a timestamp (such as "2023-01-15 17:23").
This is important to understand, because eventually you will need to manipulate the variables and need to understand whether you are working with a time or a string.

InfluxBD timestamp from data now

How to properly convert the current date for example in python timestamp in influxdb? what is the correct formula to do this. I have tried many solutions and I still get the result that timestamp is for example 1970-01-01T00:00:00.15436224Z.

Using BigQuery SQL with Built-in Python Functions

I recently started using Google's BigQuery service, and their Python API, to query some large databases. I'm new to SQL, and the BigQuery documentation isn't incredibly helpful for what I'm doing.
Currently I'm looking through the reddit_comments database, and there's 'created_utc' tag that I'm trying to filter by. This created_utc field is in terms of Unix timestamps (i.e. November 1st, 12:00 AM is 1541030400)
I'd like to grab comments day by day (or between two Unix timestamps) but in a way that I'm iterating over each day. Something like:
from datetime import datetime, timedelta
start = datetime.fromtimestamp(1538352000)
end = datetime.fromtimestamp(1541030400)
time = start
while time < end:
print(time)
time = time + timedelta(days = 1)
Printing times here yield one like: 2018-09-30 20:00:00
However in order to query, I have to convert back to the Unix timestamp by invoking datetime's timestamp() function like time.timestamp()
The problem is, I'm trying to use the timestamp() function inside the query like so:
SELECT *
FROM 'fh-bigquery.reddit_comments.2018_10'
...
AND (created_utc >= curr_day.timestamp() AND created_utc <= next_day.timestamp())
however, it's throwing a BadRequest: 400 Function not found. Is there a way to use built-in Python functions in the way that I've described above? Or does there need to be some alternative?
Everything so far seems pretty intuitive, but it's weird that I can't find much helpful information on this specifically.
You should use BigQuery's Built-in functions
For example:
To get current timestamp - CURRENT_TIMESTAMP()
To get timestamp of start of current date - TIMESTAMP_TRUNC(CURRENT_TIMESTAMP(), DAY)
To get timestamp of start of next date - TIMESTAMP_TRUNC(TIMESTAMP_ADD(CURRENT_TIMESTAMP() , INTERVAL 1 DAY), DAY)
and so on
Also, to convert created_utc to TIMESTAMP type - you can use TIMESTAMP_SECONDS(created_utc)
You can see more about TIMESTAMP Functions

Comparing a python date variable with timestamp from select query

I want to take some action based on comparing two dates. Date 1 is stored in a python variable. Date 2 is retrieved from the database in the select statement. For example I want to retrieve some records from the database where the associated date in the record (in form of the timestamp) is later than the date defined by the python variable. Preferably, I would like the comparison to be in readable date format rather than in timestamps.
I am a beginner with python.
----edit -----
Sorry for being ambiguous. Here's what I am trying to do:
import MySQLdb as mdb
from datetime import datetime
from datetime import date
import time
conn = mdb.connect('localhost','root','root','my_db')
cur = conn.cursor()
right_now = date.today()// python date
this is the part which I want to figure out
The database has a table which has timestamp. I want to compare that timestamp with this date and then retrieve records based on that comparison. For example I want to retrieve all records for which timestamp is above this date
cur.execute("SELECT created from node WHERE timestamp > right_now")
results = cur.fetchall()
for row in results:
print row
first of all, I guess Date 1 (python variable) is a datetime object. http://docs.python.org/2/library/datetime.html
As far as I have used it, MySQLdb gives you results in a (python) datetime object if the sql type was datetime.
So actually you have nothing to do, you can use python datetime comparison methods with date 1 and date 2.
I am a little bit confused about "comparison to be in readable date format rather than in timestamps". I mean the timestamps is readable enough, right?
If Date 1 is timestamps data, then you just simply do comparison. If not, then convert it to timestamps or convert the date in database to date type, both way works.
If you are asking how to write the code to do the comparison, you would use either '_mysql' or sqlalchemy to help you. The detailed syntax can be found at any where.
Anyway, the question itself is not clear enough, so the answer is blur, too.

24 hour format for Python timestamp

Currently I'm creating timestamps with the Python time module. I created a string using this command
timestamp = time.strftime('%l:%M:%S')
However, this prints the format in a 12 hours format. Is there a way I can do this in a 24 hours format so it would show up as 20:52:53 instead of 8:52:53?
Thanks!
Try
timestamp = time.strftime('%H:%M:%S')
Check out this link for info on Python time module
https://docs.python.org/3/library/time.html#time.strftime

Categories

Resources