I'm having an issue with a Django 1.6 project.
I have a view that reports a bunch of web logs. When the view is loaded, it gets all logs from the DB. However, the user has access to two datepickers to limit the from_date and to_date on the query. To accomplish this, I added a check in the get_queryset() to verify if any of those datepickers had been used. This feels really stupid and clunky, and I'm sure there's a better way to do it, but I'm not sure what it is. Here's my view code, let me know if theres anything I'm missing here.
class SiteUsersView(generic.ListView):
"""
EX: localhost/reports/site/facebook
"""
model = Cleanedlog
template_name = "trafficreport/site_chunk_log.html"
def get_context_data(self, **kwargs):
context = super(SiteUsersView, self).get_context_data(**kwargs)
context["url_chunk"] = self.kwargs["url_chunk"]
return context
def get_queryset(self):
from_date = self.request.GET.get("fromDate")
to_date = self.request.GET.get("toDate")
if from_date is not None and to_date is not None:
return self.from_to_date()
elif from_date is not None and to_date is None:
return self.from_date()
elif to_date is not None and to_date is None:
return self.to_date()
else:
return self.all_dates()
def from_date(self, from_date):
return Cleanedlog.objects.filter(dest_url__contains=self.kwargs["url_chunk"],
time_received__gte=from_date).values('user__name').annotate(
count=Sum('size')).order_by('-count')
def to_date(self, to_date):
return Cleanedlog.objects.filter(dest_url__contains=self.kwargs["url_chunk"],
time_received__lte=to_date).values('user__name').annotate(
count=Sum('size')).order_by('-count')
def from_to_date(self, to_date, from_date):
return Cleanedlog.objects.filter(dest_url__contains=self.kwargs["url_chunk"],
time_received__gte=from_date,
time_received__lte=to_date).values('user__name').annotate(
count=Sum('size')).order_by('-count')
def all_dates(self):
return Cleanedlog.objects.filter(dest_url__contains=self.kwargs["url_chunk"]).values('user__name').annotate(
count=Sum('size')).order_by('-count')
What's more is that I have other reports (Users/IPs) that follow the exact same format, just with a different model and marginally different queries. The datepickers and all the data presentation is in a higher level template, and that works well, but it feels really stupid to just duplicate all this code. Am I missing something obvious?
Thanks!
I've solved problems like this in the past by inverting the logic and using exclude, eg:
return Cleanedlog.objects.filter(dest_url__contains=self.kwargs["url_chunk"]) \
.exclude(time_received__lt=from_date).exclude(time_received__gt=to_date)
And then take values and annotate and count as desired.
Comparisons against None always fail, so nothing gets excluded unless the date is provided.
That's probably the simplest way to write it, but you can also take advantage of the ability to chain querysets without evaluating them:
base_qs = Cleanedlog.objects.filter(dest_url__contains=self.kwargs["url_chunk"])
if from_date is not None:
base_qs = base_qs.filter(time_received_gte=from_date)
if to_date is not None:
base_qs = base_qs.filter(time_received_lte=to_date)
return base_qs.values(# as above
That avoids putting the unnecessary None comparisons into the query at all.
Related
I have a model in Django with a DateTimeField attribute. I want to forbid insertion of new data in the database if the duration between the datetime field of the new data and the latest datetime field in the database is less than some duration threshold.
class MyModel(models.Model):
time_stamp = models.DateTimeField(default=timezone.now, null=True)
When I want to insert a datapoint say today, and the latest time stamp in my database is yesterday, and the duration threshold is one month (this operation should not be possible).
You can define this logic in your views like so:
from django.shortcuts import get_object_or_404
import datetime
def CreateNew(request, id):
obj = get_object_or_404(MyModel, id = id) #Get the object from your database
form = YourForm(request.POST or None, instance = obj) #create form instance to be rendered inside template
diff = (timezone.now() - obj.time_stamp).total_seconds()
threshold = datetime.timedelta(days=30).total_seconds()
if diff < threshold: # Compare dates to check condition
return HttpResponse('<h1>Not Allowed</h1>')
elif form.is_valid(): # If condition is passed save form as you normally would
form.instance.time_stamp = timezone.now() # Update time_stamp to current time
form.save()
return HttpResponseRedirect("/")
context = {
'form': form
}
return render(request, "Your_template", context)
If you are determined that this is prevented in a more hard manner than putting protection logic in view(s), then instead of checking in the view you can check in the model's save method.
def save( self, *args, **kwargs):
diff = (timezone.now() - self.time_stamp).total_seconds()
threshold = datetime.timedelta(days=30).total_seconds()
if diff < threshold: # Compare dates to check condition
# not certain ValueError is the best choice of exception
raise ValueError(
f"Can't save because {diff} seconds since the previous save, the minimum is {theshold}"
)
super().save( *args, **kwargs)
This check can still be bypassed, by Django bulk_update for example, and by raw SQL. Some databases may let you put the check into the database itself.
The downside is that fixing mistakes using (for example) the Django Admin may become difficult. In this case you can programmatically bypass the check by resetting the timestamp first.
I'm using Django-filter to allow the user to filter a database based on multiple choices in two fields. The filterset.py looks like this:
class TapFilter(django_filters.FilterSet):
bar__region = django_filters.MultipleChoiceFilter(choices=CHOICES, label="Regions:", widget=forms.CheckboxSelectMultiple,help_text="")
bar = django_filters.ModelMultipleChoiceFilter(queryset=Bar.objects.all(), label="Bars:", widget=forms.CheckboxSelectMultiple,help_text="")
However, this functions as an AND between the two lists. I need OR instead. That is, I need to show anything matching the selection in either category.
I have seen similar questions using normal filters, but I would prefer to keep using django-filter if possible.
The website in question is here: http://bestap.pythonanywhere.com/
Update: I've put this in my filtersets.py, but am clearly not doing things right...
class TapFilter(django_filters.FilterSet):
bar__region = django_filters.MultipleChoiceFilter(choices=CHOICES, label="Regions:", widget=forms.CheckboxSelectMultiple,help_text="")
bar = django_filters.ModelMultipleChoiceFilter(queryset=Bar.objects.all(), label="Bars:", widget=forms.CheckboxSelectMultiple,help_text="")
def qs(self):
base_qs = Bar.objects.all()
qs = Bar.objects.none()
for name, filter_ in six.iteritems(self.filters):
value = self.form.cleaned_data[name]
qs = qs | filter_.filter(base_qs, value)
return qs
This gives me the error 'function' object has no attribute 'count'.
You'll need to override qs on your TapFilter FilterSet subclass.
The base implementation is not that complicated; the essence of it loops over the filters applying them to the queryset.
Simplified:
for name, filter_ in six.iteritems(self.filters):
value = self.form.cleaned_data[name]
qs = filter_.filter(qs, value)
You need the union of the filters' QuerySets, which you can get because QuerySet implements __or__, so (again simplified) you'll need something like:
base_qs = Bar.objects.all()
qs = Bar.objects.none()
for name, filter_ in six.iteritems(self.filters):
value = self.form.cleaned_data[name]
qs = qs | filter_.filter(base_qs, value)
Hopefully that gets you started.
I'm trying to programmatically build a search query, and to do so, I'm joining a table.
class User(db.Model):
id = db.Column(db.Integer(), primary_key=True)
class Tag(db.Model):
id = db.Column(db.Integer(), primary_key=True)
user_id = db.Column(db.Integer(), db.ForeignKey('user.id'))
title = db.Column(db.String(128))
description = db.Column(db.String(128))
This is a bit of a contrived example - I hope it makes sense.
Say my search function looks something like:
def search(title_arg, desc_arg):
query = User.query
if title_arg:
query = query.join(Tag)
query = query.filter(Tag.title.contains(title_arg))
if desc_arg:
query = query.join(Tag)
query = query.filter(Tag.description.contains(desc_arg))
return query
Previously, I’ve kept track of what tables that have already been joined in a list, and if the table is in the list, assume it’s already joined, and just add the filter.
It would be cool if I could look at the query object, see that Tag is already joined, and skip it if so. I have some more complex query building that would really benefit from this.
If there’s a completely different strategy for query building for searches that I’ve missed, that would be great too. Or, if the above code is fine if I join the table twice, that's great info as well. Any help is incredibly appreciated!!!
You can find joined tables in query._join_entities
joined_tables = [mapper.class_ for mapper in query._join_entities]
Since SQLAlchemy 1.4, the earlier proposed solutions including _join_entities don't work anymore.
SQLAlchemy 1.4
I tried to solve this in SQLAlchemy 1.4, but there is a caveat:
This approach includes all entities in the query, so not only joined entities
from sqlalchemy.sql import visitors
from contextlib import suppress
def _has_entity(self, model) -> bool:
for visitor in visitors.iterate(self.statement):
# Checking for `.join(Parent.child)` clauses
if visitor.__visit_name__ == 'binary':
for vis in visitors.iterate(visitor):
# Visitor might not have table attribute
with suppress(AttributeError):
# Verify if already present based on table name
if model.__table__.fullname == vis.table.fullname:
return True
# Checking for `.join(Child)` clauses
if visitor.__visit_name__ == 'table':
# Visitor might be of ColumnCollection or so,
# which cannot be compared to model
with suppress(TypeError):
if model == visitor.entity_namespace:
return True
# Checking for `Model.column` clauses
if visitor.__visit_name__ == "column":
with suppress(AttributeError):
if model.__table__.fullname == visitor.table.fullname:
return True
return False
def unique_join(self, model, *args, **kwargs):
"""Join if given model not yet in query"""
if not self._has_entity(model):
self = self.join(model, *args, **kwargs)
return self
Query._has_entity = _has_entity
Query.unique_join = unique_join
SQLAlchemy <= 1.3
For SQLAlchemy 1.3 and before, #mtoloo and #r-m-n had perfect answers, I've included them for the sake of completeness.
Some where in your initialization of your project, add a unique_join method to the sqlalchemy.orm.Query object like this:
def unique_join(self, *props, **kwargs):
if props[0] in [c.entity for c in self._join_entities]:
return self
return self.join(*props, **kwargs)
Now use query.unique_join instead of query.join:
Query.unique_join = unique_join
According to the r-m-n answer:
Some where in your initialization of your project, add a unique_join method to the sqlalchemy.orm.Query object like this:
def unique_join(self, *props, **kwargs):
if props[0] in [c.entity for c in self._join_entities]:
return self
return self.join(*props, **kwargs)
Query.unique_join = unique_join
Now use query.unique_join instead of query.join:
query = query.unique_join(Tag)
I have a django form and on my view function I do this :
search_packages_form = SearchPackagesForm( data = request.POST )
I would like to overwrite a form field called price which is decleared as such :
price = forms.ChoiceField( choices = PRICE_CHOICES, required = False,widget = forms.RadioSelect )
I would like to overwrite the form field before calling search_packages_form.is_valid()
I thought of doing :
search_packages_form.data['price'] = NEW_PRICE
But it does not work. Any ideas ?
Probably not the Django way but based on https://stackoverflow.com/a/17304350/2730032 I'm guessing the easiest way to change your form value before validation is to do something like the following:
def search_packages_view(request):
if request.method == 'POST'
updated_request = request.POST.copy()
updated_request.update({'price': NEW_PRICE})
search_packages_form = SearchPackagesForm(updated_request)
if search_packages_form.is_valid():
# You're all good
This works but I'd be interested if anyone has another way that seems more in line with Django, or if there isn't: then an explanation about why.
one trick for what you want is to do it like this:
changed_data = dict(request.POST)
changed_data['price'] = NEW_PRICE
search_packages_form = SearchPackagesForm(data = changed_data)
My solution is build on an earlier proposal. It is a working solution, which can be used in several cases.
#Milad Khodabandehloo
had a tricky solution to solve the problem.
changed_data = dict(request.POST)
changed_data['price'] = NEW_PRICE
search_packages_form = SearchPackagesForm(data = changed_data)
as #The EasyLearn Academy commented: it does not allow you to access actual data submitted in form.
This is because the request.POST is immutable.
But there is a solution to the problem - just have to be more tricky.
This solution is only good if a reference to the object is enough for the certain cause. It leaves the object itself the same.
# write object to variable (data)
data = request.POST
# set to mutable
data._mutable = True
# modify the values in data
data[modified_field] = new_value
# set mutable flag back (optional)
data._mutable = False
Hope it's useful!
form.is_valid() runs through your form's (and model's in case of a ModelForm) clean method's, returning True or False
If you plan on changing form data you can do it within the general clean method or at field level, e.g.
class YourForm(DerivingClass):
# regular stuff
def clean_<ATTR>(self):
# here
return self.cleaned_data
def clean(self):
# or here
return super(YourForm, self).clean()
I have the following logic in my application:
provider = request.POST.get('provider', '*')
order_items = OrderItem.objects.filter(provider=provider)
Is there a wildcard I can use in django, such that if no provider is found in the POST request, it will return all objects for it?
In other words, is there a way to accomplish this?
if request.POST.get('provider'):
order_items = OrderItem.objects.filter(provider=provider)
else:
order_items = OrderItem.objects.all()
There is no direct wildcard parameter, so what you have is perfectly acceptable. Code readability also counts, so even though you might end up with more code, it may be more maintainable.
You could chain the queryset like this:
provider = request.POST.get('provider')
order_items = OrderItem.objects.all()
if provider is not None:
order_items = order_items.filter(provider=provider)
Or you could set up an empty dictionary of kwargs for a filter() call like this, but it's less readable in my opinion:
provider = request.POST.get('provider')
kwargs = {}
if provider is not None:
kwargs['provider'] = provider
order_items = OrderItem.objects.filter(**kwargs)
This could be turned into a function call like this:
def all_or_filter_args(request, item):
"""Return dictionary of arguments for filter() if item is specified in request."""
value = request.get(item)
if value is None:
return {}
return { item : value }
and then the following one-liner used for queries:
order_items = OrderItem.objects.filter(**all_or_filter_args(request, 'provider'))
But again, I don't find this as readable as the example you've given.
Another approach is to write a custom manager which allows you to use your own functions for filtering. With a custom manager you could implement code to allow queries like this, where you provide a function for all_or_filtered to apply the correct filtering:
order_items = OrderItem.objects.all_or_filtered('provider', request.POST.get('provider'))
How about this...
parameter = request.POST.get('provider', '%%')
order_items = OrderItem.objects.raw("SELECT * FROM %s WHERE provider LIKE '%s'" % (OrderItem._meta.db_table, parameter))