I'm trying to retrieve data from mongodb via mongoengine within a specified time span. Below is the db model used.
class DeviationReport(db.Document):
meta = {'collection': 'DeviationReport'}
created_at = db.DateTimeField()
date = db.DateTimeField()
author = db.StringField()
read_by = db.ListField(default=[])
prod_line = db.ReferenceField(ProductionLine)
product = db.ReferenceField(Product)
description = db.StringField()
What I've tried is the code below. It does however not return any results. I've used a similar approach when I've needed to build dynamic queries depending on user input.
kwargs = {}
start = datetime.datetime(2018, 12, 11)
end = datetime.datetime(2019, 03, 13)
kwargs['created_at'] = { '$lt': end, '$gt': start }
DeviationReport.objects(**kwargs)
I've obviously made sure that there are objects within the date range, and I've read other similar posts where the query below has been successfully used. How do I get my query to return everything between 'start' and 'end', or how do I rewrite it to do as I wish?
Thanks you.
I worked around/solved the problem by first getting my results sans date filtering using **kwargs and then filtered that using Q. It may not be optimal, but it works for what I need it to do.
reports = DeviationReport.objects(**kwargs)
reports = reports.filter((Q(date__gte=start) & Q(date__lte=end)))
There's a number of ways to achieve your query, adjust the collection and params accordingly using the example below:
date_to = datetime.datetime.utcnow() # The end date
date_from = date_to - datetime.timedelta(days=120) # The start date
query_a = Application.objects(category="rest_api").filter(
date_created__gte=date_from,
date_created__lte=date_to
)
query_b = Application.objects(
date_created__gte=date_from,
date_created__lte=date_to
).filter(category="rest_api")
query = {"category": "rest_api"}
query_c = Application.objects(
date_created__gte=date_from,
date_created__lte=date_to,
**query
)
Querying with Q as suggested above did not work for me, but a raw query did:
raw_query = {'date': {'$gte': start, '$lt': end}}
reports = DeviationReport.objects(__raw__=raw_query)
Related
Hi i have some Django 11 project, my model look like
class Event(models.Model):
name = models.CharField(max_length=100, unique=True)
title = models.CharField(max_length=100)
info = models.CharField(max_length=100)
image = models.ImageField(upload_to='events/%Y/%m/%d')
start_date = models.DateField(default=timezone.now)
start_time = models.TimeField(default=timezone.now)
stop_date = models.DateField(default=timezone.now)
stop_time = models.TimeField(default=timezone.now)
place = models.ForeignKey('places.Place', on_delete=models.CASCADE)
company = models.ForeignKey('companies.Company', on_delete=models.CASCADE)
and my view look like
def place_website(request, place_id):
place_template = get_template('room.html')
place_obj = Place.objects.filter(id=place_id)
# filter for event obejts only for requested place, filtered for now and next events
place_event_now = Event.objects.filter(place=place_id, start_date=timezone.now().date, stop_date__gte=timezone.now().date)
place_events_next = Event.objects.filter(place=place_id, start_date=timezone.now(), stop_date__gte=timezone.now()).order_by('start_time')
place_context = {
'place_obj': place_obj,
'place_event_now': place_event_now,
'place_events_next': place_events_next,
}
return HttpResponse(place_template.render(place_context))
the thing i want to manage is to pass to template the list of filtered Event objects based on time.
Lets pick this line
place_event_now = Event.objects.filter(place=place_id, start_date=timezone.now().date, stop_date__gte=timezone.now().date)
it couse error "expected string or bytes-like object" but when i remove ".date" from "timezone.now()" error disappear (then filter do nothing) but i want to compare date to date and time to time.
How to do this properly ?
This approach to filter objects in view rather than in template is proper?
###### UPDATE ########
Its werid because after correction now i have no error but queryset is not filtered properly, look like only two first parameter is filtering ok and the another two is causing empty queryset.
place_event_now = Event.objects.filter(place=place_id, start_date=timezone.now().strftime('%Y-%m-%d'), start_time__lte=timezone.now().strftime('%H:%M:%S'), stop_time__gte=timezone.now().strftime('%H:%M:%S'))
I my database time is saved in format H:M:S and timezone.now().time() has different format so i modified filter with .strftime this didnt help, what i wont is to limit "place_event_now" queryset to particular object/objects that come true with condition start_time < currenttime < stop_time.
Another case is with place_event_next
place_events_next = Event.objects.filter(place=place_id, start_date=timezone.now().strftime('%Y-%m-%d'), stop_date__gte=timezone.now().strftime('%Y-%m-%d'), start_time__gt=timezone.now().strftime('%H:%M:%S')).order_by('start_time')
Event when i filter objects that start_time is greater than timezone.now() they still are in queryset.
Am I doing something wrong ?
I figured it out that timezone.now() return time not in my current timezone, i change it to timezone.localtime() and everything working perfect !!!
May be you need call date to date()
replace
filter(place=place_id, start_date=timezone.now().date, stop_date__gte=timezone.now().date)
# ^^^ ^^^
to
filter(place=place_id, start_date=timezone.now().date(), stop_date__gte=timezone.now().date())
# ^^^ ^^^
I have two collections ScenarioDrivers and ModelDrivers which has One to Many relationship with each other.
class ScenarioDrivers(Document):
meta = {
'collection': 'ScenarioDrivers'
}
ScenarioId = ReferenceField('ModelScenarios')
DriverId = ReferenceField('ModelDrivers')
DriverCalibrationMethod = StringField()
SegmentName = StringField()
DriverValue = ListField()
CalibrationStatus = StringField()
AdjustedValues = ListField(default=[])
CreateDate = DateTimeField(default=ObjectId().generation_time)
LastUpdateDate = DateTimeField(default=datetime.utcnow())
class ModelDrivers(Document):
meta = {
'collection': 'ModelDrivers'
}
PortfolioModelId = ReferenceField('PortfolioModels')
DriverName = StringField()
CreateDate = DateTimeField(default=ObjectId().generation_time)
LastUpdateDate = DateTimeField(default=datetime.utcnow())
FieldFormat = StringField()
DriverData = ListField()
My query is like this.
class GetCalibratedDrivers(Resource):
def get(self, scenario_id):
scenario_drivers_list = []
scenario_drivers = ScenarioDrivers.objects(ScenarioId=scenario_id).exclude('ScenarioId').select_related(1)
for scenario_driver in scenario_drivers:
scenario_driver_dict = {
'id': str(scenario_driver.id),
'DriverId': str(scenario_driver.DriverId.id),
'SegmentName': scenario_driver.SegmentName,
'CalibrationMethod': scenario_driver.DriverCalibrationMethod,
'CalibratedValues': exchange(scenario_driver.DriverValue),
'AdjustedValues': scenario_driver.AdjustedValues,
'LastUpdateDate': formatted_date(scenario_driver.LastUpdateDate),
'FieldFormat': scenario_driver.DriverId.FieldFormat
}
scenario_drivers_list.append(scenario_driver_dict)
return {
'DriverCalibrations': scenario_drivers_list
}
The Query matches 1140 records and then I construct a dictionary and make it a list.
But this API call takes 30s to process just 1140 records. Where I am missing? Please help. I am using latest version of Pymongo and MongoEngine.
I think the problem is not with your query, it is with you looping over 1140 records. I do not see any use of referenced objects so you should consider removing select_related(1). Once you do that, if you want to convert reference object ids to string, you can use as_pymongo() which will do that by default for you. And finally if you must read some data in specific format like formatted_date or exchange, it is better to save them as part of your document. i.e. save FormattedLastUpdateDate with LastUpdateDate. In MongoDB, you have to think about your read specific logic when you save the document.
I have an API from which I receive a query. This API is in Python.
I call it from a django app (views.py). Then, I want to query my MongoDB collection, using mongoengine:
api_response = requests.get("http://*******", {'query':query}) #We call the API
json_resp = api_response.json()
person = Person.objects(__raw__=json_resp).to_json() #We search for the json_query in the DB (raw) and the output is JSON
It works fine but I have a problem with dates... Indeed, my Person model is as follow:
class Person(DynamicDocument):
# Meta Variables
meta = {
'collection':'personsExample'
}
#Document variables
PersonID = models.CharField(max_length = 6)
FirstName = models.CharField(max_length = 50)
LastName = models.CharField(max_length = 50)
Gender = models.CharField(max_length = 6) #male/female
BirthDate = models.DateField()
CitizenCountryCode = models.CharField(max_length = 2)
My personsExample collection was imported via mongoimport from a CSV file:
mongoimport --host localhost --db persons --collection personsExample --type csv --file reducedPersonsExtract.csv --headerline
As the birth dates were set as string, I have converted them using:
db.personsExample.find().forEach(function(el){el.BirthDate = new ISODate(el.BirthDate); db.personsExample.save(el)})
The problem I have now is that it gives BirthDate field as follow:
"BirthDate" : ISODate("1970-12-21T00:00:00Z")
But in my json query, date is stored as
datetime.datetime(1970,12,21,0,0,0).isoformat()
Which gives:
{
"BirthDate": "1970-12-21T00:00:00"
}
Thus, the query doesn't work, I would need to make my query with
{
"BirthDate": ISODate"1970-12-21T00:00:00Z"
}
(But I can't create such objects (ISODate) with Python... )
Or to find another way to store the date in MongoDB.
Would you happen to know how I could solve my problem please?
I have succeeded to to what I wanted to. In fact, I don't convert dates to ISODate in the DB, I store them as string "YYYY-MM-DD". Then, I format the date in my API (which sends the JSON query to the App which uses MongoDB) :
my_dict['BirthDate'] = datetime.datetime(YYYY,MM,DD).isoformat()
Then, in my App :
json_resp = api_response.json() #Deserialization of the response
json_resp['BirthDate'] = datetime.datetime.strptime(json_resp['BirthDate'], "%Y-%m-%dT%H:%M:%S")
It may not be the best solution but it works.
Have looked through DateField, TimeField, DateTimeField related documents but I cannot seem to find what I need. I simply want to have a selection of month, day, year, hour, minute, (AM/PM) type option. I have tried using 'choice=', but do not get the nice behavior I am looking for.
** TL;DR: I simply want a way of putting in the date and time without having to type it in. I would like a nice drop down menu **
class Event(models.Model):
event_name = models.CharField(max_length = 50)
date_time = models.DateTimeField()
date = models.DateField()
location = models.CharField(max_length = 30)
address = models.CharField(max_length = 30)
city = models.CharField(max_length = 30)
zip_code = models.CharField(max_length = 30)
state = models.CharField(max_length = 30)
description = models.TextField()
def __unicode__(self):
return self.event_name
class EventForm(ModelForm):
class Meta:
model = Event
This is what I currently have. I have removed the choices part and I even tried making my own model object dedicated to date and time, but that did not go well
I tried it using this ...
DATE_CHOICES = (('Jan', "January"),
('Feb', "Feburary"),
('Mar', 'March'),
('Apr','April'),
('May ','May'),
('June','June'),
('July','July'),
('Aug','August'),
('Sept','Septemeber'),
('Oct','October'),
('Nov','November'),
('Dec','December')
)
class DateTime(models.Model):
month = models.CharField(max_length = 5, choices=DATE_CHOICES)
But I am not getting the correct behavior as I want.
You might be interested in using jquery date picker or jquery datetime picker.
http://jqueryui.com/datepicker/
http://trentrichardson.com/examples/timepicker/
On both sites there are exampels so you can see it in action :)
You definitely want to use this snippet. Unless you're a django ninja and want to roll up your own multi widget, which is what you will need to transform a set of select inputs into one datetime value.
This widget is the closest you will get to do it, without using any js plugins.
Your can use datepicker plagin as Erfin mentioned (I recommend it too and also datepicker for bootstrap) for date and masked input for time or just simple selects. Anyway you should send a request with datetime information. What to do in django:
If you use datepicker plugin for date and masked input for time
Let's assume that your request is POST. Your date will be a string with format you specify in javascript. It looks like "31.01.2013". And time will be like "22:30".
def write_datetime(request):
event = Event.objects.get(id=int(request.POST.get('id')))
from datetime.datetime import strptime
date = request.POST.get('date')
time = request.POST.get('time')
date_time = date + " " + time
event.date_time = strptime(date_time, "%d.%m.%y %H:%M"
event.save()
If you use just selects
In this case, it's simplier to make a datetime string from request parameters and repeat the preveous example.
def def write_datetime(request):
year = request.POST.get('year')
month = request.POST.get('month')
# etc
date_time = "%s.%s.%s %s:%s" % (day, month, year, hours, minutes)
# etc
I have a Task model where every task has deadline.
class Tasks(models.Model):
assigned_to = models.ForeignKey('Users', related_name='user_assigned_to')
deadline = models.DateTimeField()
status = models.CharField(max_length=20,null=True)
I need to filter out task that has deadline date earlier than today's date with other filters.So I used this query:
all_task = Tasks.objects.filter(Q(assigned_to = i),Q(deadline<datetime.datetime.today()),~Q(status="Done"))
But this give me error:
NameError: name 'deadline' is not defined
If i changed the query to(just changing < sign to = ) :
all_task = Tasks.objects.filter(Q(assigned_to = i),Q(deadline=datetime.datetime.today()),~Q(status="Done"))
The above query returns me objects , but why it is not returning objects in case I use < or > with datetime?
Because using < is trying to use it as a name instead of a keyword argument.
..., Q(deadline__lt=datetime.datetime.today()), ....
"Field lookups"
have u try this one ??
all_task = Tasks.objects.filter(Q(assigned_to = i),Q(deadline__lt = datetime.datetime.today()),~Q(status="Done"))
I hope it will work.