Django MultipleChoiceField with large set of objects (around 100000) - python

I have a model with ManyToManyField:
class WordList(models.Model):
words = models.ManyToManyField('Word')
These WordList objects should be created from admin, by choosing words (The size of WordList can be different, but, in general, there are should be around 10-20 words in every WordList) By default, Django admin use MultipleChoiceField to render control for ManyToMany field.
There are two problems with it.
The number of Word objects is around 100000, and when I try to edit the WordList object in admin - it takes about 10 seconds on my dev server to load the page. Obviously, almost all time is taken by SELECT * FROM "app_word". That is bad, I am want to speed up it at least to 1-3 seconds.
This is mostly design problem, but the issue reason connected with such large amount of objects. It is hard to find a Word in a drop-down list. I am tried to use [django-easy-select2][https://github.com/asyncee/django-easy-select2] which uses select2.js to add search to ModelChoiceField, but it works really slow and browser starting to eat all RAM and CPU.
Please help. Thank you.
P.S. I am using PostgreSQL

Please try this widget: https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.filter_horizontal.
According to the doc:
By default, a ManyToManyField is displayed in the admin site with a
select multiple. However, multiple-select boxes can be difficult to
use when selecting many items. Adding a ManyToManyField to this list
will instead use a nifty unobtrusive JavaScript “filter” interface
that allows searching within the options.
In your code you would do this:
class WordListAdmin(admin.ModelAdmin):
filter_horizontal = ('words',)

Related

How to create element in another model with django

I have a small question. That is actually making me scratch my head.
So in my Database, I have the following models:
Activity
Bill
Clients
I think you are all seeing the relationship I am trying to create :
A Bill has one client and one or more activities. Here is the trick to make this whole thing user-friendly I am trying to create Bills (with the url: Bill/new) that can be edited manually. So the user is sent to an HTML page with the basic Bill template and he has a table that can add some rows with the activity the time spent and its cost.
There are three things I am trying to achieve.
Generate automatically the ID of the Bill (it should be pk of Bill) but it seems it's not generated until I have pressed on save.
When I save a Bill I want to save also the activities I have entered manually.
When I save the Bill I would like to save it as a Word or PDF document in the database.
Are these possible?
Thanks all for reading and helping I am banging my head to figure out how to do all of this and I am quite a newbie so any help is welcome.
Thanks in advance.
One way would be for the Bill objects to have a boolean field in_preparation. There would be a sequence of forms involved. The first would create a minimal Bill object with in_preparation=True. Then the related objects could be created and linked to this Bill. The final stage would be to display the entire bill and related objects for checking, with options to go back and edit, or "Confirm and issue to customer". This latter would set in_preparation=False and generate the pdf and word files.
If all the necessary information is already available and you are just asking how to use a ModelForm in this circumstance, the answer is obj = form.save( commit=False). It's then up to you to call obj.save() once you have finished updating it or its related objects More information here
If you are saving a bunch of objects at once and want to be sure it's all-or-nothing (the latter if something throws an exception), you need a Django transaction.
Generate automatically the ID of the Bill (it should be pk of Bill) but it seems it's not generated until I have pressed on save.
For bills and any other "real world"-like documents you'd probably better define some custom number generator to detach number from the auto-id and to be able to generate numbers using more complex patterns (eg AB-12345/321).
But in any case whether you choose to use auto-id or a custom generator the most simple way to garanty existence and uniqueness of that id/number is to save bill instance first. Also this approach has some additional pros.
When I save a Bill I want to save also the activities I have entered manually.
You can use django formsets for this (assuming that your bill and activity models have fk or m2m relation)
https://docs.djangoproject.com/en/3.2/topics/forms/formsets/
When I save the Bill I would like to save it as a Word or PDF document in the database.
I'm not really sure if it is a good idea to store pdf or word files in db somehow.
In my opinion there are better ways
Generate pdf (word, excel etc) docs from data on request
Store as files in filesystem.
In this case files shouldn't be stored in publicly accessible
dirs (like media and static) and should be served instead with
FileResponce with proper access checks in the view

Alternative Widget for Django Admin Choice Field Select with Many Entries?

I have a model with many country fields and each is a choice field wit list of countries.
The issue is that there's a lot of countries! So each choice field adds a lot of size and rendering time to the page to generate a select dropdown in django admin. (E.g. Making 5 of these read only brought down response time from 10s to 4s!)
I want to see if there's any alternatives known for handling choice field inputs. Since we use ISO-2 for countries, I want to avoid operators having to input values directly as they would not know them.
Ideally I was hoping for some sort of search or select pop-up similar to what happens with raw_id_fields. That way a select list is only generated when needed, but so far havent found
You may try django-jet package or one of its forks. It is fully rewritten django-admin template, and it supports typing search for model choice fields. Or try looking for another package in django packages

Why does this Django form rendering taking forever?

I have a ModelForm based on a gigantic 300 fields model. Actually one model with 150 fields inheriting from another one with 150 other.
I was expecting the model to cause some issues at syncdb or save(), but it works fine.
However, calling the form based on this model as_p() method just take forever. I can wait 10 minutes without seen any output. Same for any HTML rendering method for this form. No error, just python hanging there wherever I'm in the shell or in the dev server.
Is there any known limitation or bug for hug forms in Django? Is there a workaround?
I'm suspecting some issue with string concatenation. I'm going to dig in the Django source code in the meantime, but if anybody got some clue that would save my day.
if you have foreign keys to tables with a large number of rows, it's worth remembering that the default widget for each is a dropdown (<select>) with each row ('s unicode) as a choice. these often get enormous and it's worth looking at options (like the admin's raw_id_field)
I found something like this in the code:
INCOME_CHOICES = tuple(zip(range(1, 1000000), range(1, 1000000)))
models.IntegerField(choices=INCOME_CHOICES)
I guess there no more mystery :-)

Limiting options in list filter in Django admin

I'd like to add a "user" column to the list_filters for a Django modeladmin. The model's user column only contains a dozen unique users, however, I have thousands of users, causing the select field to be rendering with thousands of options, making it unusable.
How would I make the select only show the users that are actually used by my model, or at least use some other widget for rendering the select, so the user doesn't have to scroll through thousands of options?
Your question is a little confusing. At first glance, you want to limit what items show up as available filters on the changelist Filters sidebar, but then you go on to talk about selects, which seems to imply you're talking about limiting the options for a field on your change form.
If the latter is the case, #kgr's answer is appropriate, however, if you're asking about the former, see my question and answer regarding a similar thing here on SO

Processing forms that generate many rows in DB

I'm wondering what the best approach to take here is. I've got a form that people use to register for a class and a lot of times the manager of a company will register multiple people for the class at the same time. Presently, they'd have to go through the registration process multiple times and resubmit the form once for every person they want to register.
What I want to do is give the user a form that has a single <input/> for one person to register with, along with all the other fields they'll need to fill out (Email, phone number, etc); if they want to add more people, they'll be able to press a button and a new <input/> will be generated. This part I know how to do, but I'm including it to best describe what I'm aiming to do.
The part I don't know how to approach is processing that data the form submits, I need some way of making a new row in the Registrant table for every <input/> that's added and include the same contact information (phone, email, etc) as the first row with that row. For the record, I'm using the Django framework for my back-end code.
What's the best approach here? Should it just POST the form x times for x people, or is there a less "brute force" way of handling this?
Django includes FormSet for dealing with exactly these challenges. Using a FormSet you can create multiple forms for creating or updating information. There's even possible to generate the FormSets from a Model. http://docs.djangoproject.com/en/dev/topics/forms/formsets/ and http://docs.djangoproject.com/en/dev/topics/forms/modelforms/#id1 are great resources.
Now, for creating more forms on the fly, you need some javascript magic. I've done this on work projects using jQuery which made it a lot simpler. The basic idea is create a new form with the correct inputs and change the hidden metadata in the formset form so it will now how many forms to process. The admin implements this when using multiple inline forms so I suggest looking there for code as it is a bit tricky to get right.

Categories

Resources