At least in Python 3, float has attributes real and imag, and a method conjugate(). Since issubclass(float, complex) evaluates to False, what is the reason for these?
It is obviously a design choice and it is very well rooted in Python numeric types (i.e. bool, int, float, complex), as clear from the source code (e.g. for float).
This has been discussed in PEP 3141, which resulted in the numbers module for Numeric abstract base classes module.
As you can see, .real, .imag and .conjugate() are part of the generic Number abstraction.
From a practical perspective, this means that any numeric algorithm can be safely written for complex and it will gracefully work for any Number subtype.
Related
Please excuse my confusion - I'm new to using typing and trying to use it along with mypy for checking.
It looks like the problem/question I have seems to happen to people starting to use typing and Mypy quite a lot.
Problem
I'm trying to define an abstract composition of dataclasses, that will be subclassed into concrete classes to add additional data.
So in a simplified form I'm trying to do the following:
from dataclasses import dataclass
from typing import List
#dataclass
class TestResultImage:
base_var_a: int
#dataclass
class TestSeries:
imgs: List[TestResultImage]
# --- concrete instances -------
#dataclass
class SpecificImageType1(TestResultImage):
specific_var_b: float
specific_var_c: int
#dataclass
class SpecificSeries(TestSeries):
imgs: List[SpecificImageType1]
Mypy fails on the above with the error\
error: Incompatible types in assignment (expression has type "List[SpecificImageType1]", base class "TestSeries" defined the type as "List[TestResultImage]")
note: "List" is invariant -- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance
note: Consider using "Sequence" instead, which is covariant
Fix
Changing {List} to {Sequence} solves the problem - as noted in the error.
Question
I have seen quite a few SO and Mypy git issues related to this issue and the confusion of people.
So I then went and attempted to read as many of the Mypy docs as possible.
But it's still - IMHO - pretty confusing as to why List is problematic when you're subclassing. ...or perhaps confused why 'List is invariant, but Sequence is covariant'.
So I'm asking, perhaps on behalf of others like me trying to really use typing, and so Mypy, for more than trivial examples - is there any good explanations of the reason List is problematic, and some examples anywhere?
Suppose we add the following to your original code:
def check_specific_images(imgs: List[SpecificImageType1]) -> None:
for img in imgs:
print(img.specific_var_b)
def modify_series(series: TestSeries) -> None:
series.append(TestResultImage(1))
specific = SpecificTestSeries(imgs=[
SpecificImageType1(1, 2.0, 3),
SpecificImageType1(4, 5.0, 6),
])
modify_series(specific)
check_specific_images(specific.imgs)
This program on the surface ought to type check: specific is an instance of TestSeries so it's legal to do modify_series(specific). Similarly, specific.imgs is of type List[SpecificImageType1] so doing check_specific_images(specific.imgs) is also legal.
However, if we actually try running this program, we'll get a runtime error when we call check_specific_images! The modify_series added a TestResultImage object to our List[SpecificImageType1] causing the subsequent call to check_specific_images crash at runtime.
This problem is fundamentally why mypy (or pretty much any other sane type system) will not let List[SpecificImageType1] be treated as a subtype of List[TestResultImage]. In order for one type to be a valid subtype of another, it should be possible to safely use the subtype in any location that expects the parent type. This is simply not true for lists.
Why? Because lists support write operations. It should always be safe to insert a TestResultImage (or any subtype of TestResultImage) into a List[TestResultImage], and this is not true for List[SpecificImageType1].
So if the problem is that lists are mutable, what if we instead use a type that's immutable instead -- supports only read operations? This would let us side-step the problem entirely.
This is exactly what Sequence is: it's a type that contains all of the read-only methods that lists support (and is a supertype of List).
More broadly, let's suppose we have some sort of generic type Wrapper[T] along with two classes Parent and Child where Child is a subtype of Parent.
This then raises the question: how does Wrapper[Parent] relate to Wrapper[Child]?
There are four possible answers to this:
Wrapper is covariant: Wrapper[Child] is a subtype of Wrapper[Parent].
Wrapper is contravariant: Wrapper[Parent] is a subtype of Wrapper[Child].
Wrapper is invariant: Wrapper[Parent] and Wrapper[Child] are unrelated to each other and neither is a subtype of the other.
Wrapper is bivariant: Wrapper[Parent] is a subtype of Wrapper[Child] and Wrapper[Child] is a subtype of Wrapper[Parent].
When you're defining Wrapper[T], mypy will let you pick whether you want that type to be covariant, contravariant, or invariant. Once you've made your choice, mypy will then enforce the following rules:
If a class is covariant, it can only support read operations against T. In practice, this means you're disallowed from defining methods that accept anything of type T.
If a class is contravariant, it can only support write operations against T. In practice, this means you're disallowed from defining methods that return anything of type T.
If a class is invariant, it can support both read and write operations against T. There are no restrictions on what types of methods you can define.
Mypy doesn't let you create bivariant types: the only time when such a type would be safe is if it supported neither read nor write operations against T -- which would be pretty pointless.
You usually only see bivariant types in programming languages/type systems that intentionally want to make generics as simple as possible, even if it means letting the user introduce bugs like the one shown above into their program.
The high-level intuition here is that supporting either read operations or write operations against T will place constraints on how Wrapper[Parent] is related to Wrapper[Child] -- and if you support both kinds of operations, the combined constraints will end up making the two types be simply unrelated.
Searching for this topic I came across the following: How to represent integer infinity?
I agree with Martijn Peeters that adding a separate special infinity value for int may not be the best of ideas.
However, this makes type hinting difficult. Assume the following code:
myvar = 10 # type: int
myvar = math.inf # <-- raises a typing error because math.inf is a float
However, the code behaves everywhere just the way as it should. And my type hinting is correct everywhere else.
If I write the following instead:
myvar = 10 # type: Union[int, float]
I can assign math.inf without a hitch. But now any other float is accepted as well.
Is there a way to properly constrain the type-hint? Or am I forced to use type: ignore each time I assign infinity?
The super lazy (and probably incorrect) solution:
Rather than adding a specific value, the int class can be extended via subclassing. This approach is not without a number of pitfalls and challenges, such as the requirement to handle the infinity value for the various __dunder__ methods (i.e. __add__, __mul__, __eq__ and the like, and all of these should be tested). This would be an unacceptable amount of overhead in the use cases where a specific value is required. In such a case, wrapping the desired value with typing.cast would be able to better indicate to the type hinting system the specific value (i.e. inf = cast(int, math.inf)) be acceptable for assignment.
The reason why this approach is incorrect is simply this: since the value assigned looks/feels exactly like some number, some other users of your API may end up inadvertently use this as an int and then the program may explode on them badly when math.inf (or variations of such) be provided.
An analogy is this: given that lists have items that are indexed by positive integers, we would expect that any function that return an index to some item be some positive integer so we may use it directly (I know this is not the case in Python given there are semantics that allow negative index values be used, but pretend we are working with say C for the moment). Say this function return the first occurrence of the matched item, but if there are any errors it return some negative number, which clearly exceed the range of valid values for an index to some item. This lack of guarding against naive usage of the returned value will inevitably result in problems that a type system is supposed to solve.
In essence, creating surrogate values and marking that as an int will offer zero value, and inevitably allow unexpected and broken API/behavior to be exhibited by the program due to incorrect usage be automatically allowed.
Not to mention the fact that infinity is not a number, thus no int value can properly represent that (given that int represent some finite number by its very nature).
As an aside, check out str.index vs str.find. One of these have a return value that definitely violate user expectations (i.e. exceed the boundaries of the type positive integer; won't be told that the return value may be invalid for the context which it may be used at during compile time, results in potential failure randomly at runtime).
Framing the question/answer in more correct terms:
Given the problem is really about the assignment of some integer when a rate exist, and if none exist some other token that represent unboundedness for the particular use case should be done (it could be some built-in value such as NotImplemented or None). However as those tokens would also not be int values, it means myvar would actually need a type that encompasses those, and with a way to apply operation that would do the right thing.
This unfortunately isn't directly available in Python in a very nice way, however in strongly static typed languages like Haskell, the more accepted solution is to use a Maybe type to define a number type that can accept infinity. Note that while floating point infinity is also available there, it inherits all the problems of floating point numbers that makes that an untenable solution (again, don't use inf for this).
Back to Python: depending on the property of the assignment you actually want, it could be as simple as creating a class with a constructor that can either accept an int or None (or NotImplemented), and then provide a method which the users of the class may make use of the actual value. Python unfortunately do not provide the advanced constructs to make this elegant so you will inevitably end up with code managing this be splattered all over the place, or have to write a number of methods that handle whatever input as expected and produce the required output in the specific ways your program actual needs.
Unfortunately, type-hinting is really only scratching the surface and simply grazing over of what more advanced languages have provided and solved at a more fundamental level. I supposed if one must program in Python, it is better than not having it.
Facing the same problem, I "solved" as follow.
from typing import Union
import math
Ordinal = Union[int, float] # int or infinity
def fun(x:Ordinal)->Ordinal:
if x > 0:
return x
return math.inf
Formally, it does exactly what you did not wanted to. But now the intend is clearer. When the user sees Ordinal, he knows that it is expected to be int or math.inf.
and the linter is happy.
I was looking through the quicktions fractions library and I found this cython syntax I've never seen before:
an, ad = (<Fraction>a)._numerator, (<Fraction>a)._denominator
What does (<Fractions>a) represent? It seems like it's some sort of memory allocation. But, I'm not sure.
It's a type cast.
It assures Cython that the object really is a Fraction so that it can access the _numerator and _denominator attributes of the cdef type. Without the cast it can only use the generic Python lookup mechanisms to find attributes which doesn't allow you to access any non-public attributes of cdef types.
It doesn't do any checks that it is actually the correct type, so if you're not 100% sure that the object is actually a fraction then you should do <Fraction?> instead which does check.
That is just the Cython syntax for type casting. In this case, a is being casted to a Fraction type. The additional parentheses are necessary to signify that you want to cast a and the get the _numerator property of the casted value, not cast a._numerator.
PEP 3141 defines a numerical hierarchy with Complex.__add__ but no Number.__add__. This seems to be a weird choice, since the other numeric type Decimal that (virtually) derives from Number also implements an add method.
So why is it this way? If I want to add type annotations or assertions to my code, should I use x:(Complex, Decimal)? Or x:Number and ignore the fact that this declaration is practically meaningless?
I believe the answer can be found in the Rejected Alternatives:
The initial version of this PEP defined an algebraic hierarchy
inspired by a Haskell Numeric Prelude [3] including MonoidUnderPlus,
AdditiveGroup, Ring, and Field, and mentioned several other possible
algebraic types before getting to the numbers. We had expected this to
be useful to people using vectors and matrices, but the NumPy
community really wasn't interested ...
There are more complicated number systems where addition is clearly not supported. They could have went in much more detail with their class hierarchy (and originally intended to), but there is a lack of interest in the community. Hence, it is easier just to leave Numbers unspecified for anyone who wants to get more complicated.
Note that Monoids are an example where only one binary operation is defined.
In numbers.py. There is note on Decimal and Real.
24 ## Notes on Decimal
25 ## ----------------
26 ## Decimal has all of the methods specified by the Real abc, but it should
27 ## not be registered as a Real because decimals do not interoperate with
28 ## binary floats (i.e. Decimal('3.14') + 2.71828 is undefined). But,
29 ## abstract reals are expected to interoperate (i.e. R1 + R2 should be
30 ## expected to work if R1 and R2 are both Reals).
And also put some related links here. Really a good question, drive me dig the hole around. :P
A related github issue
PEP 3119 Which all about
ABC(Abstract Base Class) and PEP3141 defines Number part.
cpython/Lib/numbers.py
For a group of objects that are number like (called an ordered field) you only need the following things:
Addition
Multiplication
Negation
Reciprocal
LessThanEqual
And the rest (like subtraction and equality) follow. Obvioulsy, I will also need to add things like __init__ and __str__, but what type of object can I inherit from that will supply the other operators? Some other operators that I wish would be inferred from the above include:
Subtraction
Division
Absolute Value
All other comparison operators
Etc...
Take a look at the numbers module. It has abstract base classes for numeric types.
Also take a look at the list of magic methods related to numerical types:
http://www.rafekettler.com/magicmethods.html#numeric
While not a complete answer, for comparisons, there is functools.total_ordering.
You need to override operators for that.
The complete method is well documented here: http://docs.python.org/2/reference/datamodel.html