Develop Reference

Django - Filter GTE LTE for alphanumeric IDs

I am trying to serve up our APIs to allow filtering capabilities using LTE and GTE on our ID look ups. However, the IDs we have are alphanumeric like AB:12345, AB:98765 and so on. I am trying to do the following on the viewset using the Django-Filter:
class MyFilter(django_filters.FilterSet):
item_id = AllLookupsFilter()
class Meta:
model = MyModel
fields = {
'item_id': ['lte', 'gte']
}
But the issue is, if I query as: http://123.1.1.1:7000/my-entities/?item_id__gte=AB:1999 or even http://123.1.1.1:7000/my-entities/?item_id__lte=AB:100 it won't exactly return items with ID's greater than 1999 or less than 100 since the filter will take ID as a string and tries to filter by every character. Any idea how to achieve so I can filter on IDs so I get items exactly greater / less than the numeric ID (ignoring the initial characters) ?

What you'll want to do is write a custom lookup. You can read more about them here: https://docs.djangoproject.com/en/2.0/howto/custom-lookups/
The code sample below has everything you need to define your own except the actual function. For that part of the example check the link.
from django.db.models import Lookup
#Field.register_lookup
class NotEqual(Lookup):
lookup_name = 'ne'
In the lookup, you'll need to split the string and then search based on your own parameters. This will likely require you to do one of the following:
Write custom SQL that you can pass through Django into your query.
Request a large number of results containing the subset you're looking for and filter that through Python, returning only the important bits.
What you're trying to accomplish is usually called Natural Sorting, and tends to be very difficult to do on the SQL side of things. There's a good trick, explained well here: https://www.copterlabs.com/natural-sorting-in-mysql/ However, the highlights are simple in the SQL world:
Sort by Length first
Sort by column value second

Django, how to use filter to check if string field is contained in parameter

Say, I have a model with a text field:
class SomeModel
keyword=models.CharField(null=True, max_length=255)
Now, I know how to check if a parameter string (lets call the variable "querystring" is contained in the field keyword:
results = SomeModel.objects.filter(keyword_icontains=querystring).all()
which I found in the django docs
Question, how do I filter for the objects whose field values are contained in the querystring variable?
Apologies if my question is confusing... maybe an example will clarify...
In django docs, if my keyword field contains,for example, 'python-django', then, for a querystring that contains 'django', I can extract the object that contains that field with an
results=SomeModel.objects.filter(keyword_icontains=querystring).all()
or results=SomeModel.objets.filter(keyword_icontains='django').all()
But say, I want to extract all rows/objects whose keyword field is contained in a querystring? For example, if querystring contains 'In django, how do I create a filter'? then I want results to contain all objects whose keyword fields have the values 'django', 'filter', etc...

You'll have to split your input into words (depending on your definition of "word"), and then iterate over them, concatenating the various resulting querysets (most of which are likely to be empty).
To split, you could use a regex, catching all letters and the apostrophe (but you'll miss a smart-quote, if someone uses that as input instead of the standard ASCII apostrophe):
words = re.split(r"[^A-Za-z']+", querystring)
Then loop and filter:
query = Q() # empty Q object
for word in words:
# 'or' the queries together
query |= Q(keyword_icontains=word)
results = SomeModel.objects.filter(query).all()
The above code is untested, and I got the idea to 'or' the queries and an empty Q() from this answer.
Depending on your next step(s), evaluating the query directly in each loop step, appending to a results list, might work better for you (using e.g. itertools.chain, as per this answer).

Here is a solution that will select SomeModel rows whose keyword is any substring of the querystring, not just complete words:
SomeModel.objects\
.annotate(querystring=Value(querystring, output_field=CharField()))\
.filter(querystring__icontains=F('keyword'))
See docs for info about annotate, Value expressions and F expressions

Have multiple elements in sqlite column

How can i add multiple elements for a single column in a row:
Say i have a column, topic, which can have infinitely many elements inside:
topics = ['Particle Physics,'Karaoke','jazz']
I have a statement in sqlite:
def UpdateElement(new_user,new_topic):
new_topic = new_topic + "; "
querycurs.execute('''UPDATE First_Data SET topic = (?) WHERE user = (?)''', (new_topic, new_user))
However this will allow only one element at a time to exist under the topic column. How can you edit the code so that it can add another given element to the current topic.
If in the table topic = ['Math'], then i could make it into topic = '[Math; Python']. This way i can use simple python .join statement to split it.

With a text field you can store anything. You could store the list as semi-colon delimited string or as a json object string. You could also pickle the list and store it as a base64 string. The problem with all of these solutions is that you lose a level of access to your data. To count how many users have like Jazz topic, you need to read/split the text field...or use some more complicated LIKE statement.
Since you are using SQL, you may want to consider normalizing your data to include a Topic table, a User table, and a cross-walk table with foreign keys to your users and topics to enforce the many-to-many relationship. While its a bit more to setup, but it can be simpler to update when user topics change.

Variable interpolation in python/django, django query filters [duplicate]

Given a class:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=20)
Is it possible, and if so how, to have a QuerySet that filters based on dynamic arguments? For example:
# Instead of:
Person.objects.filter(name__startswith='B')
# ... and:
Person.objects.filter(name__endswith='B')
# ... is there some way, given:
filter_by = '{0}__{1}'.format('name', 'startswith')
filter_value = 'B'
# ... that you can run the equivalent of this?
Person.objects.filter(filter_by=filter_value)
# ... which will throw an exception, since `filter_by` is not
# an attribute of `Person`.

Python's argument expansion may be used to solve this problem:
kwargs = {
'{0}__{1}'.format('name', 'startswith'): 'A',
'{0}__{1}'.format('name', 'endswith'): 'Z'
}
Person.objects.filter(**kwargs)
This is a very common and useful Python idiom.

A simplified example:
In a Django survey app, I wanted an HTML select list showing registered users. But because we have 5000 registered users, I needed a way to filter that list based on query criteria (such as just people who completed a certain workshop). In order for the survey element to be re-usable, I needed for the person creating the survey question to be able to attach those criteria to that question (don't want to hard-code the query into the app).
The solution I came up with isn't 100% user friendly (requires help from a tech person to create the query) but it does solve the problem. When creating the question, the editor can enter a dictionary into a custom field, e.g.:
{'is_staff':True,'last_name__startswith':'A',}
That string is stored in the database. In the view code, it comes back in as self.question.custom_query . The value of that is a string that looks like a dictionary. We turn it back into a real dictionary with eval() and then stuff it into the queryset with **kwargs:
kwargs = eval(self.question.custom_query)
user_list = User.objects.filter(**kwargs).order_by("last_name")

Additionally to extend on previous answer that made some requests for further code elements I am adding some working code that I am using
in my code with Q. Let's say that I in my request it is possible to have or not filter on fields like:
publisher_id
date_from
date_until
Those fields can appear in query but they may also be missed.
This is how I am building filters based on those fields on an aggregated query that cannot be further filtered after the initial queryset execution:
# prepare filters to apply to queryset
filters = {}
if publisher_id:
filters['publisher_id'] = publisher_id
if date_from:
filters['metric_date__gte'] = date_from
if date_until:
filters['metric_date__lte'] = date_until
filter_q = Q(**filters)
queryset = Something.objects.filter(filter_q)...
Hope this helps since I've spent quite some time to dig this up.
Edit:
As an additional benefit, you can use lists too. For previous example, if instead of publisher_id you have a list called publisher_ids, than you could use this piece of code:
if publisher_ids:
filters['publisher_id__in'] = publisher_ids

Django.db.models.Q is exactly what you want in a Django way.

This looks much more understandable to me:
kwargs = {
'name__startswith': 'A',
'name__endswith': 'Z',
***(Add more filters here)***
}
Person.objects.filter(**kwargs)

A really complex search forms usually indicates that a simpler model is trying to dig it's way out.
How, exactly, do you expect to get the values for the column name and operation?
Where do you get the values of 'name' an 'startswith'?
filter_by = '%s__%s' % ('name', 'startswith')
A "search" form? You're going to -- what? -- pick the name from a list of names? Pick the operation from a list of operations? While open-ended, most people find this confusing and hard-to-use.
How many columns have such filters? 6? 12? 18?
A few? A complex pick-list doesn't make sense. A few fields and a few if-statements make sense.
A large number? Your model doesn't sound right. It sounds like the "field" is actually a key to a row in another table, not a column.
Specific filter buttons. Wait... That's the way the Django admin works. Specific filters are turned into buttons. And the same analysis as above applies. A few filters make sense. A large number of filters usually means a kind of first normal form violation.
A lot of similar fields often means there should have been more rows and fewer fields.

Python/Django: Get record from DB based on a dictionary field

I have a database table and one of the fields is a serialized python dictionary (or stringified JSON) in this form:
{"full_name":"first_name=John&last_name=Smith", "id":24354,"username":"hello"}
How can i select a record based on this field and specifically on the dictionary's username, if i'm searching for a specific username? I mean is there any smart/fast way to do it without loops or splitting the process, maybe with just one line of code?
#If the field name is "user_info"
account = theModel.filter(user_info=???)
*I know that this design is not so good but i found it that way...
Thanks!

You could use the regex query syntax for Django.
theModel.objects.get(user_info__regex=r'some regex here')