Develop Reference

How can i change this query to ORM?

Hi i have two models like this,
class Sample(models.Model):
name = models.CharField(max_length=256) ##
processid = models.IntegerField(default=0) #
class Process(models.Model):
sample = models.ForeignKey(Sample, blank=False, null=True, on_delete=models.SET_NULL, related_name="process_set")
end_at = models.DateTimeField(null=True, blank=True)
and I want to join Sample and Process model. Because Sample is related to process and I want to get process information with sample .
SELECT sample.id, sample.name, process.endstat
FROM sample
INNER JOIN process
ON sample.processid = process.id
AND process.endstat = 1;
(i'm using SQLite)
I used
sample_list = sample_list.filter(process_set__endstat=1))
but it returned
SELECT sample.id, sample.name
FROM sample
INNER JOIN process
ON (sample.id = process.sample_id)
AND process.endstat = 1)
This is NOT what I want.
How can i solve the problem?

This should work for you
Process.objects.filter(end_at=1).values('sample__id','sample__name','end_at')
.values() method returns selective table fields.

I'm assuming sample_list = Sample.objects.
When you are filtering a model, only the fields defined in the model are selected. In your example, id and processid. If you want to retrieve values from related models as a single record you need to use values or values_list. To get the desired query you have to do this
sample_list = sample_list.filter(process_set__endstat=1).values('id', 'name', 'process__endstat')
Btw, Django does JOIN on the foreign key field. So, you can't get ON sample.processid = process.id since processid is not a ForeignKey field.
Reference:
https://docs.djangoproject.com/en/4.0/ref/models/querysets/#values

I found JOIN not on foreign key field in django.
sample_list = sample_list.filter(processid__in=Process.objects.filter(endstat=1)
I used the medthod of
Django-queryset join without foreignkey

How to create multiple objects with different values at a time in django?

I need to create two models from a single template. Creating Product model is fine. The Product model has the ManyToOne relation with ProductVariant. But I got problem while creating ProductVariant model.
request.POST.getlist('names') this gives me the result like this ['name1','name2] and the same goes for all.
I want to create ProductVariant object with each values. How can I do this ? Also I think there is a problem while stroing a HStoreField. request.POST.getlist('attributes') gives the value like this ['a:b','x:z'] so I converted it into dictionary(but not sure it works).
UPDATE:
What I want is
attributes, names ... all will have the same number of items in the list.
For example if the name is ['a','b','c'] then weight will also have 3 values in the list [12,15,23] like this.
I want to create ProductVariant object 3 times since every list will have 3 items in the list. The first object will have field values from the list first item which is name=a,weight=12.. and for the second object values will be name=b, weight=15 like this.
How will it be possible? Or I should change the logic ? Any suggestions ?
models
class ProductVariant(models.Model):
name = models.CharField(max_length=255, blank=False, null=False)
product = models.ForeignKey(Product, on_delete=models.CASCADE)
attributes = HStoreField()
price = models.FloatField(blank=False, null=False, default=0.0)
views
product = product_form.save()
attributes = request.POST.getlist('attributes')
names = request.POST.getlist('name')
up = request.POST.getlist('price')
weight = request.POST.getlist('weight')
print(names, 'names')
# converting attributes into the dictionary for the HStore field
for attribute in attributes:
attributes_dict = {}
key, value = attribute.split(':')
attributes_dict[key] = value
ProductVariant.objects.create(name=name,...) # for each value I want to create this.

Answer for update:
names = ['a', 'b', 'c']
weights = [12, 15, 23]
params = zip(names, weights)
products = [ProductVariant(name=param[0], weight=param[1]) for param in params]
ProductVariant.objects.bulk_create(products)

I disagree with this approach, but if you really want to do it this way, ziping would be the way as #forkcs pointed out.
I would use Django to help me as much as possible, before i get there, please make this change. float != money
class ProductVariant(models.Model):
name = models.CharField(max_length=255, blank=False, null=False)
product = models.ForeignKey(Product, on_delete=models.CASCADE)
attributes = HStoreField()
price = models.DecimalField(blank=False, null=False, default=0, max_digits=6, decimal_places=2)
Once thats done, the form should look like this:
class ProductVariantForm(forms.ModelForm):
class Meta:
fields = ('name', 'product', 'attributes', 'price')
model = ProductVariant
ProductVariantFormSet = formset_factory(ProductVariantForm)
Note that I don't have to parse/clean/format attributes? Thats because Django did it for me ;)
And you can use it as follow IF you raname your fields and not use the same name multiple times: (instead of all your fields being called "attributes", you call them "form-X-attributes" where X is the number 0-infinity, example)
product = product_form.save()
formset = ProductVariantFormSet(data=request.POST)
if formset.is_valid():
instances = []
for form in formset:
if form.is_valid(): # this could probably be removed
instances.append(form.save())
For extra credit you can also do: (it shouldn't really matter)
product = product_form.save()
formset = ProductVariantFormSet(data=request.POST)
if formset.is_valid():
instances = []
for form in formset:
if form.is_valid(): # this could probably be removed
instances.append(form.save(save=False))
ProductVariant.objects.bulk_create(instances)
What do you gain? STANDARDS!!! AND compartmentalization! Everyone that knows Django knows what you did. All your clean logic will be placed in the right place (the form), and you'll be less error prone.
Ps. i wrote tests for you. https://gist.github.com/kingbuzzman/937a9d207bd937d1b2bb22249ae6bdb2#file-formset_example-py-L142
If you want more information on my approach, see the docs https://docs.djangoproject.com/en/3.1/topics/forms/formsets/

As for attributes, it could be reduced to one line like this:
attributes_dict = dict(map(lambda x: x.split(':'), attributes))
To create multiple objects you should either iterate and create one object at a time or use bulk_create:
for name in names:
ProductVariant.objects.create(name=name,...)
Or
ProductVariant.objects.bulk_create([ProductVariant(name=name) for name in names])

Best practice for this is using bulk_create method.
product_variants = [ProductVariant(name=name) for name in names]
ProductVariant.objects.bulk_create(product_variants)

Search a generated format in Django Database

In my django app I am creating Barcodes with a combination of str and id of model and id of product.
The barcode is generated but the problem that I am encountering is when I scan the barcode I want to show the information of the product scanned.
I'll be able to understand the problem with code in a better way
Models.py
class GrnItems(models.Model):
item = models.ForeignKey(Product, on_delete=models.CASCADE)
item_quantity = models.IntegerField(default=0)
item_price = models.IntegerField(default=0)
label_name = models.CharField(max_length=25, blank=True, null=True)
class Grn(models.Model):
owner = models.ForeignKey(User, on_delete=models.CASCADE)
reference_no = models.CharField(max_length=500, default=0)
items = models.ManyToManyField(GrnItems)
Views.py
def PrintGRNsv1(request, pk):
grn = Grn.objects.filter(pk=pk)[0]
grn_prod = grn.items.all()
print("grn prod", grn_prod)
items = []
for i in grn_prod:
for j in range(i.item_quantity):
items.append({'bar': "YNT9299" + str(pk) +
str(i.item.pk) + str(j + 1)} )
Now let's suppose I generated a Barcode YNT92991231, Now I have no idea how to get the i.item.pk from this code
How can I do this ?
P.s.
Okay the cherry on top is that I have created the Barcodes for a large number of products and they are already placed on them, so can't really change the barcode format at this point

For now when we can't change anything. Generate all the barcodes for you GRN. Save them in files or in DB. Whenever there is a query that matches those just find them from there. If there is any conflict (when 2 or more product has the same barcode) Django USER can be one way to resolve the conflict.
For new barcodes change the generator function. Use delimiter or fixed-width characters (for pk).
ex: YNT9299GGGGGPPPPPCCCCC where G => GRN pk, P => Product pk, C => Count. Or with delimiters YNT9299G1P23C2.
Yep, the solution is not a foolproof solution is just a workaround for the mess created.

Django full text search using indexes with PostgreSQL

After solving the problem I asked about in this question, I am trying to optimize performance of the FTS using indexes.
I issued on my db the command:
CREATE INDEX my_table_idx ON my_table USING gin(to_tsvector('italian', very_important_field), to_tsvector('italian', also_important_field), to_tsvector('italian', not_so_important_field), to_tsvector('italian', not_important_field), to_tsvector('italian', tags));
Then I edited my model's Meta class as follows:
class MyEntry(models.Model):
very_important_field = models.TextField(blank=True, null=True)
also_important_field = models.TextField(blank=True, null=True)
not_so_important_field = models.TextField(blank=True, null=True)
not_important_field = models.TextField(blank=True, null=True)
tags = models.TextField(blank=True, null=True)
class Meta:
managed = False
db_table = 'my_table'
indexes = [
GinIndex(
fields=['very_important_field', 'also_important_field', 'not_so_important_field', 'not_important_field', 'tags'],
name='my_table_idx'
)
]
But nothing seems to have changed. The lookup takes exactly the same amount of time as before.
This is the lookup script:
from django.contrib.postgres.search import SearchQuery, SearchRank, SearchVector
# other unrelated stuff here
vector = SearchVector("very_important_field", weight="A") + \
SearchVector("tags", weight="A") + \
SearchVector("also_important_field", weight="B") + \
SearchVector("not_so_important_field", weight="C") + \
SearchVector("not_important_field", weight="D")
query = SearchQuery(search_string, config="italian")
rank = SearchRank(vector, query, weights=[0.4, 0.6, 0.8, 1.0]). # D, C, B, A
full_text_search_qs = MyEntry.objects.annotate(rank=rank).filter(rank__gte=0.4).order_by("-rank")
What am I doing wrong?
Edit:
The above lookup is wrapped in a function I use a decorator on to time. The function actually returns a list, like this:
#timeit
def search(search_string):
# the above code here
qs = list(full_text_search_qs)
return qs
Might this be the problem, maybe?

You need to add a SearchVectorField to your MyEntry, update it from your actual text fields and then perform the search on this field. However, the update can only be performed after the record has been saved to the database.
Essentially:
from django.contrib.postgres.indexes import GinIndex
from django.contrib.postgres.search import SearchVector, SearchVectorField
class MyEntry(models.Model):
# The fields that contain the raw data.
very_important_field = models.TextField(blank=True, null=True)
also_important_field = models.TextField(blank=True, null=True)
not_so_important_field = models.TextField(blank=True, null=True)
not_important_field = models.TextField(blank=True, null=True)
tags = models.TextField(blank=True, null=True)
# The field we actually going to search.
# Must be null=True because we cannot set it immediately during create()
search_vector = SearchVectorField(editable=False, null=True)
class Meta:
# The search index pointing to our actual search field.
indexes = [GinIndex(fields=["search_vector"])]
Then you can create the plain instance as usual, for example:
# Does not set MyEntry.search_vector yet.
my_entry = MyEntry.objects.create(
very_important_field="something very important", # Fake Italien text ;-)
also_important_field="something different but equally important"
not_so_important_field="this one matters less"
not_important_field="we don't care are about that one at all"
tags="things, stuff, whatever"
Now that the entry exists in the database, you can update the search_vector field using all kinds of options. For example weight to specify the importance and config to use one of the default language configurations. You can also completely omit fields you don't want to search:
# Update search vector on existing database record.
my_entry.search_vector = (
SearchVector("very_important_field", weight="A", config="italien")
+ SearchVector("also_important_field", weight="A", config="italien")
+ SearchVector("not_so_important_field", weight="C", config="italien")
+ SearchVector("tags", weight="B", config="italien")
)
my_entry.save()
Manually updating the search_vector field every time some of the text fields change can be error prone, so you might consider adding an SQL trigger to do that for you using a Django migration. For an example on how to do that see for instance a blog article on Full-text Search with Django and PostgreSQL.
To actually search in MyEntry using the index you need to filter and rank by your search_vector field. The config for the SearchQuery should match the one of the SearchVector above (to use the same stopword, stemming etc).
For example:
from django.contrib.postgres.search import SearchQuery, SearchRank
from django.core.exceptions import ValidationError
from django.db.models import F, QuerySet
search_query = SearchQuery("important", search_type="websearch", config="italien")
search_rank = SearchRank(F("search_vector"), search_query)
my_entries_found = (
MyEntry.objects.annotate(rank=search_rank)
.filter(search_vector=search_query) # Perform full text search on index.
.order_by("-rank") # Yield most relevant entries first.
)

I'm not sure but according to postgresql documentation (https://www.postgresql.org/docs/9.5/static/textsearch-tables.html#TEXTSEARCH-TABLES-INDEX):
Because the two-argument version of to_tsvector was used in the index
above, only a query reference that uses the 2-argument version of
to_tsvector with the same configuration name will use that index. That
is, WHERE to_tsvector('english', body) ## 'a & b' can use the index,
but WHERE to_tsvector(body) ## 'a & b' cannot. This ensures that an
index will be used only with the same configuration used to create the
index entries.
I don't know what configuration django uses but you can try to remove first argument

Django: query with ManyToManyField count?

In Django, how do I construct a COUNT query for a ManyToManyField?
My models are as follows, and I want to get all the people whose name starts with A and who are the lord or overlord of at least one Place, and order the results by name.
class Manor(models.Model):
lord = models.ManyToManyField(Person, null=True, related_name="lord")
overlord = models.ManyToManyField(Person, null=True, related_name="overlord")
class Person(models.Model):
name = models.CharField(max_length=100)
So my query should look something like this... but how do I construct the third line?
people = Person.objects.filter(
Q(name__istartswith='a'),
Q(lord.count > 0) | Q(overlord.count > 0) # pseudocode
).order_by('name'))

Actually it's not the count you're interested in here, but just whether or not there are any members in that relationship.
Q(lord__isnull=False) | Q(overlord__isnull=False)

In this case, better resort to raw SQL.
for p in Person.objects.raw('SELECT * FROM myapp_person WHERE...'):
print p