Develop Reference

Can I extend Django model fields without creating mixed types?

I have subclassed built-in model Fields to reduce repetition in similar columns. This triggers exceptions in tests against Django 3.2 (but interestingly does work in the otherwise now irrelevant, unsupported, version 2.2)
django.core.exceptions.FieldError: Expression contains mixed types: DecimalField, DecimalFWB. You must set output_field.
from django.db.models import Model,DecimalField,F
from decimal import Decimal
class DecimalFWB(DecimalField):
#property
def validators(self):
return super().validators + [MinValueValidator(0.1), ]
...
class Repro(Model):
frac = DecimalFWB(max_digits=4, decimal_places=4, default=Decimal("0.2"))
...
# same internal type
assert DecimalFWB().get_internal_type() == DecimalField().get_internal_type()
# 3.2: works
# 2.2: works
Repro.objects.annotate(dec_annotation = -F("frac") + Decimal(1)).first()
# 3.2: django.core.exceptions.FieldError
# 2.2: works
Repro.objects.annotate(dec_annotation = Decimal(1) - F("frac")).first()
I found this entry in the Django 3.2 release notes that could explain the change in behaviour from the earlier version:
[..] resolving an output_field for database functions and combined expressions may now crash with mixed types when using Value(). You will need to explicitly set the output_field in such cases.
That suggestion does not solve my problem. If I were to bloat all annotations with ExpressionWrapper/output_field=, I could just as well bloat the model definition and not use the subclass in the first place.
I am trying to emulate the internal type. I want the combined output_field of DecimalField and DecimalFWB to be DecimalField - regardless of order of super/subclass. How do I express that no mixing is happening here?

Automatically selecting the shared field as the output has been fixed as of Bug #33397, released in Django 4.1 (but not backported). The change does however comes with a warning (emphasis mine):
As a guess, if the output fields of all source fields match then
simply infer the same type here.
This guess is mostly a bad idea, but there is quite a lot of code
(especially 3rd party Func subclasses) that depend on it, we'd need a
deprecation path to fix it.
Meaning this might change again in a future release, but at least then intentionally, and reliably triggering a DeprecationWarning on request.

Python-pptx: “Show legend overlapping the chart”

I am using Office 2007.
I found if I would like to show the legend overlapping the chart in office2007.
The XML should be as the following.
`-<c:legend>
<c:overlay val="1"/>`
But no matter I use the API from python-pptx 'chart.legend.include_in_layout = True' or I leave it as the default. The generated XML would always be as the following.
`-<c:legend>
<c:overlay/>`
Without the val=1, then office2007 won't show the format properly.
What can I do to force the python-pptx to write the val=1? thanks.

Explanation
In short, the True value is not explicitly set (in contrast to False) because True corresponds to the default value of overlay's val attribute.
To explain it in more detail - you can follow the python-pptx hierarchy as follows: overlay is mapped to CT_Boolean (all overlay oxml elements are instantiated from CT_Boolean). The actual val parameter is then mapped via OptionalAttribute and is defined with the default value of True:
class CT_Boolean(BaseOxmlElement):
"""
Common complex type used for elements having a True/False value.
"""
val = OptionalAttribute('val', XsdBoolean, default=True)
Now, when setting the optional attribute to its default value, it is actually skipped/deleted, as you can see here if value == self._default:
class OptionalAttribute(BaseAttribute):
"""
Defines an optional attribute on a custom element class. An optional
attribute returns a default value when not present for reading. When
assigned |None|, the attribute is removed.
"""
#property
def _setter(self):
def set_attr_value(obj, value):
if value == self._default:
if self._clark_name in obj.attrib:
del obj.attrib[self._clark_name]
return
str_value = self._simple_type.to_xml(value)
obj.set(self._clark_name, str_value)
return set_attr_value
Fix - provide custom CT_Boolean class
Add these lines somewhere before you need to use the overlay. It will overwrite python-pptx overlay mapping with the custom CT_Boolean_NoDefault class:
from pptx.oxml import register_element_cls
from pptx.oxml.xmlchemy import BaseOxmlElement, OptionalAttribute
from pptx.oxml.simpletypes import XsdBoolean
class CT_Boolean_NoDefault(BaseOxmlElement):
"""
Common complex type used for elements having a True/False value with no
default value.
"""
val = OptionalAttribute('val', XsdBoolean)
register_element_cls('c:overlay', CT_Boolean_NoDefault)
This worked for me and finally I got:
<c:legend>
<c:overlay val="1"/>
</c:legend>
Fix - modify python-pptx permanently
This is not recommended but you might want to modify python-pptx instead of adding the solution from above for each script you run.
First, add the following to pptx/oxml/chart/shared.py which defines a new bool class without a default value:
class CT_Boolean_NoDefault(BaseOxmlElement):
"""
Common complex type used for elements having a True/False value.
"""
val = OptionalAttribute('val', XsdBoolean)
Second, modify pptx/oxml/__init__.py to add the new bool class:
from .chart.shared import (
CT_Boolean, CT_Double, CT_Layout, CT_LayoutMode, CT_ManualLayout,
CT_NumFmt, CT_Tx, CT_UnsignedInt, CT_Boolean_NoDefault
)
Third, modify pptx/oxml/__init__.py to change the mapping of the overlay element to the new bool class:
register_element_cls('c:overlay', CT_Boolean_NoDefault)
Better solution
In case you have time, please submit a ticket here so it might become a permanent fix. In case #scanny finds some time, he will read this. Perhaps there is some better solution for this, too, and I've completely missed something.

#pansen 's analysis is spot-on. Here's an alternative way to get this working in your case that might be a little lighter weight:
def include_in_layout(legend):
legend_element = legend._element
overlay = legend_element.get_or_add_overlay()
overlay.set('val', '1')
This appears to be a localized non-conformance of that version of PowerPoint with the ISO/IEC 29500 spec. As pansen rightly points out, a missing val attribute is to be interpreted the same as val=1 (True). I'd be interested to discover how extensive this non-conformance goes, i.e. what other elements exhibit this same behavior. The CT_Boolean type is used quite frequently in PowerPoint, for things like bold, italic, varyColors, smooth, and on and on. So a "compensating" fix would need to be applied carefully to avoid reporting incorrect results for other elements.
I think I'll take pansen's cue and use a specialized element class for this element only. It will still report True for an element without the val attribute, which will be inconsistent with the observed behavior on this version of PowerPoint; but assuming other versions behave correctly (according to the spec), the inconsistency will be localized and at least assigning True to that property will make the legend show up the way you want.

How can I specify a default time for a ndb.TimeProperty()?

I find myself stuck on this problem, and repeated Googling, checking SO, and reading numerous docs has not helped me get the right answer, so I hope this isn't a bad question.
One entity I want to create is an event taking place during a convention. I'm giving it the property start_time = ndb.TimeProperty(). I also have a property date = messages.DateProperty(), and I'd like to keep the two discrete (in other words, not using DateTimeProperty).
When a user enters information to create an event, I want to specify defaults for any fields they do not enter at creation and I'd like to set the default time as midnight, but I can't seem to format it correctly so the service accepts it (constant 503 Service Unavailable response when I try it using the API explorer).
Right now I've set things up like this (some unnecessary details removed):
event_defaults = {...
...
"start_time": 0000,
...
}
and then I try looping over my default values to enter them into a dictionary which I'll use to .put() the info on the server.
data = {field.name: getattr(request, field.name) for field in request.all_fields()
for default in event_defaults:
if data[default] in (None, []):
data[default] = event_defaults[default]
setattr(request, default, event_defaults[default])
In the logs, I see the error Encountered unexpected error from ProtoRPC method implementation: BadValueError (Expected time, got 0). I have also tried using the time and datetime modules, but I must be using them incorrectly, because I still receive errors.
I suppose I could work around this problem by using ndb.StringProperty() instead, and just deal with strings, but then I'd feel like I would be missing out on a chance to learn more about how GAE and NDB work (all of this is for a project on udacity.com, so learning is certainly the point).
So, how can I structure my default time properly for midnight? Sorry for the wall of text.
Link to code on github. The conference.py file contains the code I'm having the trouble with, and models.py contains my definitions for the entities I'm working with.
Update: I'm a dummy. I had my model class using a TimeProperty() and the corresponding message class using a StringField(), but I was never making the proper conversion between expected types. That's why I could never seem to give it the right thing, but it expected two different things at different points in the code. Issue resolved.

TimeProperty expects a datetime.time value
import datetime
event_defaults = {...
...
"start_time": datetime.time(),
...
}
More in the docs: https://cloud.google.com/appengine/docs/python/ndb/entity-property-reference#Date_and_Time

Use the datetime() library to convert it into a valid ndb time property value
if data['time']:
data['time'] = datetime.strptime(data['time'][:10], "%H:%M").time()
else:
data['time'] = datetime.datetime.now().time()
ps: Don't forget to change data['time'] with your field name

Is it a good practise to use underscore to ignore a value that is being passed to a function, which we don't need?

I am working on Django rest framework which specifies a set format for function prototype for detail_route in ModelViewSet. Some background: -
The function takes in request object and lookup_field which can be the primary key for a particular table.
#detail_route(methods=["get"], url_path="get-some-data")
def get_some_data(self, request, id=None):
return Response(get_some_data(id))
Now as you can see, I do not need request object here, So should I keep it like this? or change it to
#detail_route(methods=["get"], url_path="get-some-data")
def get_some_data(self, _, id=None):
return Response(get_some_data(id))
Here I changed request to _ to indicate that I do not need this value.
which approach should be followed? Should I let it remain as a request, or change it to an underscore?

For the method arguments I would always use the proper variable name so that in future whether I work on it or my peers if I provide this code to someone else they don't have to struggle to understand what it is.
For now you might think to ignore it but since it is a method argument it would be better to have a name it stands for.
Or, let's say you are adding a docstring where you are including and defining which parameter is what. You would yourself appreciate it if some one had:
#param request: HTTP request object
instead of:
#param _: HTTP request object

If you leave the parameter exist, then give it a meaningful name always do good, even you do not use it.
In addition, _ has special use in python, check it in the following url.
What is the purpose of the single underscore "_" variable in Python?

I'd leave it with a descriptive name. Changing it to underscore or any other non-descriptive name is not beneficial.

Why is REPEATED incompatible with REQUIRED? [duplicate]

In old google appengine datastore API "required" and "default" could be used together for property definitions. Using ndb I get a
ValueError: repeated, required and default are mutally exclusive.
Sample code:
from google.appengine.ext import ndb
from google.appengine.ext import db
class NdbCounter(ndb.Model):
# raises ValueError
count = ndb.IntegerProperty(required=True, default=1)
class DbCounter(db.Model):
# Doesn't raise ValueError
count = db.IntegerProperty(required=True, default=1)
I want to instantiate a Counter without having to specify a value. I also want to avoid someone to override that value to None. The example above is constructed. I could probably live without a required attribute and instead add an increment() method. Still I don't see the reason why required and default are mutually exclusive.
Is it a bug or a feature?

I think you are right. Perhaps I was confused when I write that part of the code. It makes sense that "required=True" means "do not allow writing the value None" so it should be possible to combine this with a default value. Please file a feature request in the NDB tracker: http://code.google.com/p/appengine-ndb-experiment/issues/list
Note that for repeated properties things are more complicated, to repeated will probably continue to be incompatible with either required or default, even if the above feature is implemented.

Im not sure what was intended, heres is the "explanation" from appengine.ext.ndb.model.py:
The repeated, required and default options are mutually exclusive: a
repeated property cannot be required nor can it specify a default
value (the default is always an empty list and an empty list is always
an allowed value), and a required property cannot have a default.
Beware that ndb has some other really annoying behaviour ( Text>500 Bytes not possible without monkey-patching the expando-model, filtering by .IN( [] ) raises exception, ..).
So unless you need the speed-improvements by its caching you should might consider staying with ext.db atm.