I have a python script that generates a datetime string using this line of code:
data['timestamp'] = datetime.isoformat(datetime.utcnow())
That generates something like the following:
2017-05-24T04:08:09.530033
How do I convert that to "MYSQL insertable" datetime format in a clean way?
Thanks!
Try to use MySQL's STR_TO_DATE() function to parse the string that you're attempting to insert.
I hope this may help you
You can specify any type of format like this depending on the one you `ve set in mysql
data['timestamp'] =pd.to_datetime(data['timestamp'] , format='%d%b%Y:%H:%M:%S.%f')
First off, it looks like you ran from datetime import * rather than import datetime. That's tempting because it lets you type less when you want to refer to parts of the module, but it can get you into name collision issues later. An alternative with less typing is something like import datetime as dt, that way later you can just use dt.datetime. This will make your code cleaner.
MySQL accepts several date formats, which can be read about in detail here. In particular:
The DATETIME type is used for values that contain both date and time
parts. MySQL retrieves and displays DATETIME values in YYYY-MM-DD HH:MM:SS format.
ISO8601 numbers look just like that! 2017-05-24T04:19:32
So if the only difference is the "T" in the middle instead of a space, just run something like this, assuming you don't change your import statements.
timestamp = str(datetime.isoformat(datetime.utcnow()))
timestamp = timestamp.replace("T", " ")
data['timestamp'] = timestamp
Related
This is my data :
dates = np.arange("2018-01-01", "2021-12-31", dtype="datetime64[D]")
I now want to convert from :
"2018-01-01" -> "Jan-01-2018" ["Monthname-day-year"] format
How to i do this ?
Is it possible to initialize this in the way we want to convert ?
Can i use something like:
for i in dates:
i = i.replace(i.month,i.strftime("%b"))
You can try this:
from datetime import datetime
dates = np.arange("2018-01-01", "2021-12-31", dtype="datetime64[D]")
result_dates = []
for date in dates.astype(datetime):
result_dates.append(date.strftime("%b-%d-%Y"))
But you will need to convert result dates as shown in the code
I feel compelled to elaborate on Silvio Mayolo's very relevant but ostensibly ignored comment above. Python stores a timestamp as structure (see How does Python store datetime internally? for more information) Hence, the DateTime does not as such have a 'format'. A format only becomes necessary when you want to print the date because you must first convert the timestamp to a string. Thus, you do NOT need to initialise any format. You only need to declare a format when the time comes to print the timestamp.
While you CAN store the date as a string in your dataframe index in a specific format, you CANNOT perform time related functions on it without first converting the string back to a time variable. ie current_time.hour will return an integer with the current hour if current_time is a datetime variable but will crash if it is a string formatted as a timestamp (such as "2023-01-15 17:23").
This is important to understand, because eventually you will need to manipulate the variables and need to understand whether you are working with a time or a string.
I'm learning Pandas and I have a problem trying to change a format from Object to Date_time.
When I use 'to_datetime' the date I get in return is like in ISO Format, and I just want DD/MM/YYYY (13/10/1960). And I doing something wrong? Thanks a lot!!
enter image description here
At a glance, it doesn't seem like the code uses the right format.
The to_datetime() with the format argument follows strftime standards per the documentation here, and it's a way to tell the method how the original time was represented (so it can properly format it into a datetime object). An example can be seen from this Stack Overflow question.
Simple example:
datetime_object = pd.to_datetime(format='%d/%m/%Y')
The next problem is how you want that datetime object to be printed out (i.e. DD/MM/YYYY). Just throwing thoughts out there (would comment, but I don't have those privileges yet), if you want to print the string, you can cast that datetime object into the string that you want. Many ways to do this, one of which is to use strftime().
Simple example:
date_as_string = datetime_object.strftime('%d/%m/%Y')
But of course, why would you use a datetime object in that case. So the other option I can think of is to override how the datetime object is printed (redefining __str__ in a new class of datetime).
I am trying to use timestamp information to graph.
However, when I convert the number into hh:mm:ss. It does not work.
I have tried this:
timestamp = [dt.strptime(str(datetime.timedelta(seconds=round(t/1000))),'%H:%M:%S') for t in timestamp1]
Also I tried this
timestamp = [dt.strftime(dt.strptime(str(datetime.timedelta(seconds=round(t / 1000))), '%H:%M:%S'), '%H:%M:%S') for t in timestamp]
However, it is possible to see the list with the new values. However, I have problems with the graphs and these new values.
Can anybody help me?
Try using this and make sure to import the time module!
timestamp = [time.strftime('%H:%M:%S', time.gmtime(t/1000)) for t in timestamp1]
In your code you're using datetime.strptime(date_string, format) function, it's main use is to parse time strings in order to create datetime object.
In order to get a parsed string of a desired format you should use date.strftime(format) function instead.
So basically in your code you can only add it and should get the wanted result:
import datetime
from datetime import datetime as dt
timestamp = [dt.strftime(dt.strptime(str(datetime.timedelta(seconds=round(t / 1000))), '%H:%M:%S'), '%H:%M:%S') for t in timestamp1]
I'm not sure what was your input and desired output, but you could also use date.fromtimestamp(timestamp) for the parsing of timestamps
How to set default datetime format in python because i have multiple tuples to send via template on client side. This is not good approach to set each object's value to specified format. I want to set a datetime format on server side and these converted values will be shown to client. I tried
datetime.strftime("%Y-%m-%d %X")
but it is giving error.
strftime is a method of datetime objects - it doesn't set a default representation, which seems to be what you suggest. For example, you might call it like this:
>>> import datetime
>>> now = datetime.datetime.now()
>>> now.strftime("%Y-%m-%d %X")
'2011-03-17 10:14:12'
If you need to do this a lot, it would be worth creating a method that wraps this conversion of a datetime to a string. The documentation for the datetime module can be found here.
I'm not sure I understand your issue, but this might help
http://docs.djangoproject.com/en/dev/ref/settings/
there is a datetime format section, this sets datetime format globally.
I would like a simple way to find and reformat text of the format 'DD/MM/YYYY' into 'YYYY/MM/DD' to be compatible with MySQL TIMESTAMPs, in a list of text items that may or may not contain a date atall, under python. (I'm thinking RegEx?)
Basically i am looking for a way to inspect a list of items and correct any timestamp formats found.
Great thing about standards is that there are so many to choose from....
You can read the string into a datetime object and then output it back as a string using a different format. For e.g.
>>> from datetime import datetime
>>> datetime.strptime("31/12/2009", "%d/%m/%Y").strftime("%Y/%m/%d")
'2009/12/31'
Basically i am looking for a way to inspect a list of items and correct any timestamp formats found.
If the input format is inconsistent, can vary, then you are better off with dateutil.
>>> from dateutil.parser import parse
>>> parse("31/12/2009").strftime("%Y/%m/%d")
'2009/12/31'
Dateutil can handle a lot of input formats automatically. To operate on a list you can map the a wrapper over the parse function over the list and convert the values appropriately.
If you're using the MySQLdb (also known as "mysql-python") module, for any datetime or timestamp field you can provide a datetime type instead of a string. This is the type that is returned, also and is the preferred way to provide the value.
For Python 2.5 and above, you can do:
from datetime import datetime
value = datetime.strptime(somestring, "%d/%m/%Y")
For older versions of python, it's a bit more verbose, but not really a big issue.
import time
from datetime import datetime
timetuple = time.strptime(somestring, "%d/%m/%Y")
value = datetime(*timetuple[:6])
The various format-strings are taken directly from what's accepted by your C library. Look up man strptime on unix to find other acceptable format values. Not all of the time formats are portable, but most of the basic ones are.
Note datetime values can contain timezones. I do not believe MySQL knows exactly what to do with these, though. The datetimes I make above are usually considered as "naive" datetimes. If timezones are important, consider something like the pytz library.