I've got a mongodb collection with one of the fields like:
u'Date': u'15/03/2016'
Now from my understanding u means that strings are simply unicode which is fine. I'm using Django and pymongo to allow users to select 2 dates and query my DB for stuff between those two dates like that:
number = coll.find({"Date": {'$gt': startDate}, "Date": {'$lt': endDate}}).count()
Where my startDate and endDate are both in formats "DD/MM/YYYY". What I receive back however is some rubbish data. How to correctly query for dates in python-mongo?
P.S From my understanding comma between those two results in 'AND' query right?
If the dates are stored as strings in the DB, then when you use the operator $gt or $lt, the operation is a string comparison, not a date comparison. Which means that for example: "15/03/2016 < 16/02/2016", because 5 comes before 6 in lexical order.
For the string comparison to work, the dates would need to be stored in a format so that a "smaller" date is always represented as a "smaller" string. For example by using YYYY/MM/DD.
So if you don't want to do the comparison in python, you could either change the date format, or store the date as a date in DB. But in both cases, this means changing the DB...
If doing in Python is OK, then you can do it like so:
from datetime import datetime
date_format = "%d/%m/%Y"
start_date = datetime.strptime(startDate, date_format)
end_date = datetime.strptime(endDate, date_format)
items = coll.find({})
def compare(c, item):
item_date = datetime.strptime(item['Date'], date_format)
if start_date < item_date < end_date:
return c+1
else:
return c
count = reduce(compare, items, 0)
Related
I am new to functions and I am trying to write a function that returns the number of days between two dates:
My attempt:
import datetime
from dateutil.parser import parse
def get_x_days_ago (date_from, current_date = None):
td = current_date - parse(date_from)
if current_date is None:
current_date = datetime.datetime.today()
else:
current_date = datetime.datetime.strptime(date_from, "%Y-%m-%d")
return td.days
print(get_x_days_ago(date_from="2021-04-10", current_date="2021-04-11"))
Expected outcome in days:
1
So there seem to be multiple issues, and as I said in the comments, a good idea would be to separate the parsing and the logic.
def get_x_days_ago(date_from, current_date = None):
if current_date is None:
current_date = datetime.datetime.today()
return (current_date - date_from).days
# Some other code, depending on where you are getting the dates from.
# Using the correct data types as the input to the get_x_days_ago (datetime.date in this case) will avoid
# polluting the actual logic with the parsing/formatting.
# If it's a web framework, convert to dates in the View, if it's CLI, convert in the CLI handling code
date_from = parse('April 11th 2020')
date_to = None # or parse('April 10th 2020')
days = get_x_days_ago(date_from, date_to)
print(days)
The error you get is from this line (as you should see in the traceback)
td = current_date - parse(date_from)
Since current_date="2021-04-11" (string), but date_from is parsed parse(date_from), you are trying to subtract date from the str.
P.S. If you have neither web nor cli, you can put this parsing code into def main, or any other point in code where you first get the initial strings representing the dates.
It looks like you're already aware that you can subtract a datetime from a datetime. I think, perhaps, you're really looking for this:
https://stackoverflow.com/a/23581184/2649560
I want to add a blank column of date of format "%Y-%m-%d" to a dataframe. I tried datetime.datetime.strptime('0000-00-00',"%Y-%m-%d")
But I get an error ValueError: time data '0000-00-00' does not match format '%Y-%m-%d'
How can I create a column of blank date of format "%Y-%m-%d"?
In R following works.
df$date =""
class(df$date) = "Date"
How can I achieve this in Python?
Thank you.
I don't think that's possible with datetime module. The oldest you can go to is answered here:
What is the oldest time that can be represented in Python?
datetime.MINYEAR
The smallest year number allowed in a date or datetime object. MINYEAR is 1.
datetime.MAXYEAR
The largest year number allowed in a date or datetime object. MAXYEAR is 9999.
source: datetime documentation
initial_date = request.GET.get('data') or datetime.min # datetime.min is 1
end_date = request.GET.get('data_f') or datetime.max # datetime.max is 9999
I run a sql query that returns a date in the format '2015-03-01T17:09:00.000+0000' I want to subtract this from today's date.
I am getting today's date with the following:
import datetime
now = datetime.datetime.now()
The formats don't seem to line up and I can't figure out a standardize format.
You can use strptime from datetime module to get python compatible date time from your query result using a format string. (You might have to play with the format string a bit to suit your case)
ts = '2015-03-01T17:09:00.000+0000' to a format string like
f = '%Y-%m-%dT%H:%M:%S.%f%z'
date_from_sql = datetime.datetime.strptime(ts, f)
now = datetime.datetime.now()
delta = date_from_sql - now
The .000 is probably microseconds (denoted by %f in the format string) and the +0000 is the utc offset (denoted by %z in the format string). Check this out for more formatting options: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior
Check out this thread for an example: what is the proper way to convert between mysql datetime and python timestamp?
Checkout this for more on strptime https://docs.python.org/2/library/datetime.html#datetime.datetime.strptime
Getting the delta between two datetime objects in Python is really simple, you simply subtract them.
import datetime
d1 = datetime.datetime.now()
d2 = datetime.datetime.now()
delta = d2 - d1
print delta.total_seconds()
d2 - d1 returns a datetime.timedelta object, from which you can get the total second difference between the two dates.
As for formatting the dates, you can read about formatting strings into datetime objects, and datetime objects into string here
You'll read about the strftime() and strptime() functions, and with them you can get yourself two datetime objects which you can subtract from each other.
I'm trying to filter my query by a datetime. This datetime is the datetime for the value range the customer wants to know information for. I'm trying to set it to the first of the month selected by the customer. I pass the month number convert it to the correct string format and then convert to a datetime object because simply looking for the string object was returning no values and Django's documentation says you need to do it like:
pub_date__gte=datetime(2005, 1, 30)
Code:
if 'billing-report' in request.POST:
customer_id = int(post_data['selected_customer'])
This is the code I use to get the selected customer date and turn it into a tupple
if 'billing-report' in request.POST:
customer_id = int(post_data['selected_customer'])
selected_date = int(post_data['month'])
if selected_date < 10:
selected_date = '0'+str(selected_date)
year = datetime.now()
year = year.year
query_date = str(year) + '-' + str(selected_date) + '-01'
query_date_filter = datetime.strptime(query_date, "%Y-%m-%d")
compute_usages = ComputeUsages.objects.filter(customer_id = customer_id).filter(values_date = query_date_filter)
django debug shows: datetime.datetime(2014, 10, 1, 0, 0)
query_date looks like: 2014-07-01 before it is converted
.
No error but no data is returned
I used to use:
compute_usages = ComputeUsages.objects.filter(customer_id = customer_id).filter(values_date = datetime(query_date_filter))
which was causing the error. I'm sorry for changing my question as it evolved that is why I'm re-including what I was doing before so the comments make sense.
Almost all of that code is irrelevant to your question.
I don't understand why you are calling datetime on query_date. That is already a datetime, as you know because you converted it to one with strptime earlier. So there's no need for any more conversion:
ComputeUsages.objects.filter(customer_id=customer_id).filter(values_date=query_date)
Well after spending sometime exploring setting the query filter to datetime(year, month, day) I came to the realization that django doesn't convert it to a neutral datetime format it has to match exactly. Also my data in the database had the year, day, month.
Learning point:
You have to use the datetime() exactly how it is in the database django does not convert to a neutral format and compare. I assumed it was like writing a query and saying to_date or to_timestamp where the db will take your format and convert it to a neutral format to compare against the rest of the db.
Here is the correct way
compute_usages = ComputeUsages.objects.filter(customer_id = customer_id).filter(values_date = datetime(year, day, selected_month))
I have the following simple setup, where fromDate and toDate are strings on the format "YYYY-MM-DD":
class SomeType(Base):
date = Column(DateTime)
def findAll(fromDate, toDate):
return session.query(SomeType).filter(SomeType.date >= fromDate, SomeType.date <= toDate).all()
The problem is that it doesn't find what I want it to find unless I modify the input dates like this:
def findAll(fromDate, toDate):
fromDate = fromDate + " 00:00"
toDate = toDate + " 24:00"
return session.query(SomeType).filter(SomeType.date >= fromDate, SomeType.date <= toDate).all()
But that doesn't look good. Any ideas on how I can do this the right way?
How about using datetime.datetime objects instead of strings for fromDate, toDate?
from datetime import datetime, timedelta
def findAll(fromDate, toDate):
fromDate = datetime.strptime(fromDate, '%Y-%m-%d')
toDate = datetime.strptime(toDate, '%Y-%m-%d') + timedelta(days=1)
return session.query(SomeType).filter(
SomeType.date >= fromDate,
SomeType.date < toDate).all()
The problem is that your SomeType.date column is not simple date, but is datetime column, so it contains also a time component.
This type mismatch is the cause of your problem. If this is the case then following should work:
session.query(SomeType).filter(func.date(SomeType.date) >= fromDate, func.date(SomeType.date) <= toDate).all()
where we basically cast datetime to date using DATE(...) function of MySql.
However, I would probably also prefer working with date(time) data types instead of strings. You are just lucky that most databases implicitly allow parsing of ISO-compliant string representations of DATEs.
I know this is old, but while trying to find my answer, I found datetime.combine
you can do
select(SomeTable)
.filter( SomeTable.datetime_issued >= datetime.combine(start_date, time.min),
SomeTable.datetime_issued <= datetime.combine(end_date, time.max))
datetime.combine will combine date and time into datetime
https://docs.python.org/3/library/datetime.html#datetime.datetime.combine
When combining, you should use time.min, time.max which will give you min and max time
print(combine(date.today(), time.min), combine(date.today(), time.max))
This will print
2022-10-14 00:00:00, 2022-10-14 23:59:59.999999
https://docs.python.org/3/library/datetime.html#datetime.time.max