Develop Reference

How can I safely override the django.contrib.admin.utils quote() method?

It appears that the quote() and unquote() methods inside django.contrib.admin.utils do not effectively handle underscores in primary keys. Specifically, I have some string-type primary keys that look like cus_C2xVQnht and when I use the django admin interface to edit them via the small pencil icon, the popup window will display an error like Customer with ID "cusÂxVQnht" doesn't exist. Perhaps it was deleted? (it is converting the C2 to the codepoint 00C2, aka Â. This is true for other valid codepoints as well (00C7, 00C6, 001B, etc)
If I manually go to the customers model and find the ID, I can pull it up and edit it just fine, but it seems the URL encoding doesn't work right when the primary key has an underscore in it.
After quite a lot of digging I managed to find these two functions buried deep inside django.contrib.admin.utils:
def quote(s):
"""
Ensure that primary key values do not confuse the admin URLs by escaping
any '/', '_' and ':' and similarly problematic characters.
Similar to urllib.parse.quote(), except that the quoting is slightly
different so that it doesn't get automatically unquoted by the Web browser.
"""
if not isinstance(s, str):
return s
res = list(s)
for i in range(len(res)):
c = res[i]
if c in """:/_#?;#&=+$,"[]<>%\n\\""":
res[i] = '_%02X' % ord(c)
return ''.join(res)
def unquote(s):
"""Undo the effects of quote(). Based heavily on urllib.parse.unquote()."""
mychr = chr
myatoi = int
list = s.split('_')
res = [list[0]]
myappend = res.append
del list[0]
for item in list:
if item[1:2]:
try:
myappend(mychr(myatoi(item[:2], 16)) + item[2:])
except ValueError:
myappend('_' + item)
else:
myappend('_' + item)
return "".join(res)
They appear to be called somewhere in the admin template rendering process, but I couldn't figure out where/how often/all the locations, so I decided to do a quick monkey patch to decide if it was worth pursuing as a solution: I changed all the underscores in quote() and unquote() except for the one in the list of problem characters in quote to dots...for example:
'_%02X' in quote()becomes '.%02X'
split('_') in unquote() becomes split('.')
myappend('_' + item) in unquote() becomes myappend('.' + item)
Upon doing this, the admin works correctly and it appears that the links attached to the edit icons on related fields are to the correct model instances, so I can edit them by clicking the pencil icons and don't get the error message noted above.
All that said, I can't seem to find a way to safely override these two methods. I really would rather not change the primary keys to eliminate the underscores because there are a lot of linked models in my database and it just seems like it will become a huge pain. This fix seems much easier and more reliable, and given that it worked properly on previous versions of Django I don't see how it's a bad idea to implement.
So, how can I override those methods? Or, as a related question, is there something I can do in the __str__ methods of my models to alleviate this problem? I'd do that much sooner than start writing custom classes that override Django admin internals. If there is no other solution, I would need some help in properly restructuring my database to adjust the primary keys, but I can say that these keys work perfectly on the "old" site I'm working on, which runs Django 1.11.6 and Python 2.7.9 (vs the current Django 2.1.1 and Python 3.6.5)
Please let me know if I can provide any more info. Thank you!!

This is fixed in django 2.2. See https://github.com/django/django/commit/e4df8e6dc021fa472fa77f9b835db74810184748

Mako: render with predefined/default items in Context?

I'm looking for an idiom for setting some common item definitions so that any time I call render() on a mako template, those items will be added to the parameters available in the template without having to include them in the parameters to render() each time.
The example at http://docs.makotemplates.org/en/latest/usage.html suggests that the place I should be looking is in some way of either setting some master Context definitions or something, or perhaps a way of merging two contexts:
from mako.template import Template
from mako.runtime import Context
from StringIO import StringIO
mytemplate = Template("hello, ${name}!")
buf = StringIO()
ctx = Context(buf, name="jack")
mytemplate.render_context(ctx)
print(buf.getvalue())
I can't find anything explicitly showing an example of doing this, and poking around in dir(Context) isn't making anything come to mind. Is this a simple thing to do and I'm just overlooking the obvious? Or is there a standard idiom for this?
Thanks in advance!
Update: I suppose one approach is to explicitly append **kwargs to each call to render(), and that's not bad, as it's explicit:
template = self.template_lookup.get_template("index.mako.html")
return template.render(var1=val1, var2=val2, **common_vars)
Still curious if there's a way to "fully predefine" such items.

Using django's `reverse_lazy` and concatenizing it

I want to insert a resolved URL into a string, used as the help_text of a form field:
class ContactForm(forms.Form):
[...]
email_sender = forms.EmailField(
label="Votre email",
widget=forms.EmailInput(attrs={'disabled': 'disabled'}),
help_text="[...], <a href='{}'>[...]</a>.".format(reverse_lazy('account_email'))
)
But inserting the reversed URL into the string is not possible, because the format function (or any concatenation way I tried) is not "lazy", and wants to produce the output immediately.
I get the following error:
django.core.exceptions.ImproperlyConfigured: The included urlconf 'myproject.urls' does not appear to have any patterns in it. If you see valid patterns in the file then the issue is probably caused by a circular import.
For example, using the following code works perfectly, but is not what I want :)
email_sender = forms.EmailField(
help_text=reverse_lazy('account_email')
)
So how do I concatenate a string of "lazy" values?

Note: This was written in the times of Django 2. While the essence is still valid as of Django 4, the helper function described in this post is now provided out-of-the-box as format_lazy.
You cannot concatenate lazy strings in Django. The implementation is very basic, it's not even actual lazy strings, it's lazy function calls, they may return other types. When you do reverse_lazy, you do just get a lazy function call with no special other behavior
So, just play by the rules. If you need a string to be lazily computed, create a lazy function yourself:
from django.utils.functional import lazy
def email_sender_help_text():
return "[...], <a href='{}'>[...]</a>.".format(reverse('account_email'))
email_sender_help_text_lazy = lazy(email_sender_help_text, str)
You can now use it:
email_sender = forms.EmailField(
help_text=email_sender_help_text_lazy()
)
Or, for a more general version:
from django.utils.functional import lazy
def format(string, *args, **kwargs):
return string.format(*args, **kwargs)
lazy_format = lazy(format, str)
 
help_text=lazy_format("<a href='{}'>", reverse_lazy('account_email'))
You may also want to check out django.utils.functional.allow_lazy.
Later edit:
Since Django 3, Django provides exactly that helper function, as format_lazy. So use that instead of recreating it.

In the latest versions of django, there is a util function called format_lazy: https://docs.djangoproject.com/en/3.0/ref/utils/#django.utils.text.format_lazy

Why doesn't Django template language allow something like a[ 0 ][ 1 ]?

When I was trying to access a tuple inside a list in the Django template format, I found out I couldn't access it with a[ 0 ][ 1 ], instead I had to use a.0.1.
Suppose that a is something like
a = [
( 'a', 'apple' ),
( 'b', 'bee' ),
]
Why doesn't Django template language support a[ 0 ][ 1 ]? In normal Python programming, a.0.1 would give you a syntax error.

The Django docs on the template API explain this nicely:
Dots have a special meaning in template rendering. A dot in a variable name signifies a lookup. Specifically, when the template system encounters a dot in a variable name, it tries the following lookups, in this order:
Dictionary lookup. Example: foo["bar"]
Attribute lookup. Example: foo.bar
List-index lookup. Example: foo[bar]
The template system uses the first lookup type that works. It's short-circuit logic. Here are a few examples:
>>> from django.template import Context, Template
>>> t = Template("My name is {{ person.first_name }}.")
>>> d = {"person": {"first_name": "Joe", "last_name": "Johnson"}}
>>> t.render(Context(d))
"My name is Joe."
>>> class PersonClass: pass
>>> p = PersonClass()
>>> p.first_name = "Ron"
>>> p.last_name = "Nasty"
>>> t.render(Context({"person": p}))
"My name is Ron."
>>> t = Template("The first stooge in the list is {{ stooges.0 }}.")
>>> c = Context({"stooges": ["Larry", "Curly", "Moe"]})
>>> t.render(c)
"The first stooge in the list is Larry."
Variable._resolve_lookup in django.templates.base appears to be the function responsible for this, and hasn't changed much since the oldest revision I can find

You can find some information about this in the django book:
The beginning of the chapter should explain why it works this way:
In the previous chapter, you may have noticed something peculiar in how we returned the text in our example views. Namely, the HTML was hard-coded directly in our Python code, like this:
def current_datetime(request):
now = datetime.datetime.now()
html = "<html><body>It is now %s.</body></html>" % now
return HttpResponse(html)
Although this technique was convenient for the purpose of explaining how views work, it’s not a good idea to hard-code HTML directly in your views. Here’s why:
Any change to the design of the page requires a change to the Python code. The design of a site tends to change far more frequently than the underlying Python code, so it would be convenient if the design could change without needing to modify the Python code.
Writing Python code and designing HTML are two different disciplines, and most professional Web development environments split these responsibilities between separate people (or even separate departments). Designers and HTML/CSS coders shouldn’t be required to edit Python code to get their job done.
It’s most efficient if programmers can work on Python code and designers can work on templates at the same time, rather than one person waiting for the other to finish editing a single file that contains both Python and HTML.
For these reasons, it’s much cleaner and more maintainable to separate the design of the page from the Python code itself. We can do this with Django’s template system, which we discuss in this chapter.
...
Dot lookups can be summarized like this: when the template system encounters a dot in a variable name, it tries the following lookups, in this order:
Dictionary lookup (e.g., foo["bar"])
Attribute lookup (e.g., foo.bar) 1
Method call (e.g., foo.bar())
List-index lookup (e.g., foo[2])
The system uses the first lookup type that works. It’s short-circuit logic.

The reason I would say that the django template language doesnt do XYZ way of accessing context data is because generally at that point you are doing too much in the template side vs your view that renders it.
The design decision of their template engine seems lighter than maybe some others which give you more pythonic direct access to data. But ideally you would be formatting th context before passing it in.
You also have the ability to create your own template filters for doing more custom processing of data.
Specific to your question, accesing child members using dot notation is the django template way to try multiple approaches to resolving the member. It tries dictionary keys, attributes, etc. in a certain order. You just use dot notation for everything.

In Django, the HTML code is shown instead of the actual text

& g t ; Welcome
How do I show the actual symbol instead? Is it a template filter?
Thanks.

Bit hard to know without more details. If it's from data that you're passing in from the view, you might want to use mark_safe.
from django.utils.safestring import mark_safe
def your_view(request):
...
foo = '>'
mark_safe(foo)
...
Otherwise, you want the safe filter:
{{ myvar|safe }}
Obviously, be careful with this, make sure the variable actually is safe, otherwise you'll open yourself up to cross-site scripting attacks and the like. There's a reason Django escapes this stuff.