Suppose I have a function which takes a dictionary as a parameter:
def f(d: dict) -> None:
x = d["x"]
print(x)
Can I specify that this dictionary must have the key "x" to mypy? I'm looking for something similar to interface from typescript, without changing d to a class.
The reason I don't want to change d to a class, is because I am modifying a large existing codebase to add mypy type checking and this dictionary is used in many places. I would have to modify a lot of code if I had to change all instances of d["x"] to d.x.
As of Python 3.8 you can use typing.TypedDict, added as per PEP 589. For older Python versions you can use the typing-extensions package
Note that the PEP does acknowledge that the better option is that you use dataclasses for this use-case, however:
Dataclasses are a more recent alternative to solve this use case, but there is still a lot of existing code that was written before dataclasses became available, especially in large existing codebases where type hinting and checking has proven to be helpful.
So the better answer is to consider a different data structure, such as a named tuple or a dataclass, where you can specify the attribute a type has. This is what the typescript declaration does, really:
The printLabel function has a single parameter that requires that the object passed in has a property called label of type string.
Python attributes are the moral equivalent of Typescript object properties. That Typescript object notation and Python dictionaries have a lot in common perhaps confuses matters, but you should not look upon Typescript object declarations as anything but classes, when trying to map concepts to Python.
That could look like this:
from dataclasses import dataclass
#dataclass
class SomeClass:
x: str
def f(sc: SomeClass) -> None:
x = sc.x
print(x)
That said, you can use typing.TypedDict here:
from typing import TypedDict
class SomeDict(TypedDict):
x: str
def f(d: SomeDict) -> None:
x = d['x']
print(x)
Keys in a TypeDict declaration are either all required, or all optional (when you set total=False on the declaration); you'd have to use inheritance to produce a type with some keys optional, see the documentation linked. Note that TypedDict currently has issues with a mix of optional and required keys; you may want to use the typing-extensions package to get the Python 3.9 version (which fixes this) as a backport even when using Python 3.8. Just use from typing_extensions import TypedDict instead of the above from typing ... import, the typing-extensions package falls back to the standard library version when appropriate.
Mypy extends PEP 484 by providing a TypedDict type. This allows specifying specific attributes of a dict type. In your case you can do the following:
from mypy_extensions import TypedDict
# you can also do HasX = TypedDict('HasX', {'x': str})
class HasX(TypedDict):
x: str
def f(x: HasX) -> None:
reveal_type(d["x"]) # error: Revealed type is 'builtins.str'
Related
Using Python 3's function annotations, it is possible to specify the type of items contained within a homogeneous list (or other collection) for the purpose of type hinting in PyCharm and other IDEs?
A pseudo-python code example for a list of int:
def my_func(l:list<int>):
pass
I know it's possible using Docstring...
def my_func(l):
"""
:type l: list[int]
"""
pass
... but I prefer the annotation style if it's possible.
Answering my own question; the TLDR answer is No Yes.
Update 2
In September 2015, Python 3.5 was released with support for Type Hints and includes a new typing module. This allows for the specification of types contained within collections. As of November 2015, JetBrains PyCharm 5.0 fully supports Python 3.5 to include Type Hints as illustrated below.
Update 1
As of May 2015, PEP0484 (Type Hints) has been formally accepted. The draft implementation is also available at github under ambv/typehinting.
Original Answer
As of Aug 2014, I have confirmed that it is not possible to use Python 3 type annotations to specify types within collections (ex: a list of strings).
The use of formatted docstrings such as reStructuredText or Sphinx are viable alternatives and supported by various IDEs.
It also appears that Guido is mulling over the idea of extending type annotations in the spirit of mypy: http://mail.python.org/pipermail/python-ideas/2014-August/028618.html
Update: please see other answers, this one is outdated.
Original Answer (2015):
Now that Python 3.5 is officially out, there is the Type Hints supporting module - typing and the relevant List "type" for the generic containers.
In other words, now you can do:
from typing import List
def my_func(l: List[int]):
pass
As of Python 3.9, builtin types are generic with respect to type annotations (see PEP 585). This allows to directly specify the type of elements:
def my_func(l: list[int]):
pass
This also extends to most other container types of the standard library, for example collections.deque or collections.abc.Mapping.
Various tools may support this syntax earlier than Python 3.9. When annotations are not inspected at runtime, the syntax is valid using quoting or __future__.annotations.
# quoted
def my_func(l: 'list[int]'):
pass
# postponed evaluation of annotation
from __future__ import annotations
def my_func(l: list[int]):
pass
As a consequence of PEP 585, most helpers in typing corresponding to standard library types are deprecated, such as typing.List, typing.Deque or typing.Mapping. They should only be used if compatibility with Python versions prior to 3.9 is required.
Type comments have been added since PEP 484
from . import Monitor
from typing import List, Set, Tuple, Dict
active_monitors = [] # type: List[Monitor]
# or
active_monitors: List[Monitor] = []
# bonus
active_monitors: Set[Monitor] = set()
monitor_pair: Tuple[Monitor, Monitor] = (Monitor(), Monitor())
monitor_dict: Dict[str, Monitor] = {'codename': Monitor()}
# nested
monitor_pair_list: List[Dict[str, Monitor]] = [{'codename': Monitor()}]
This is currently working for me on PyCharm with Python 3.6.4
With support from the BDFL, it's almost certain now that python (probably 3.5) will provide a standardized syntax for type hints via function annotations.
https://www.python.org/dev/peps/pep-0484/
As referenced in the PEP, there is an experimental type-checker (kind of like pylint, but for types) called mypy that already uses this standard, and doesn't require any new syntax.
http://mypy-lang.org/
I'm working on a library that currently supports Python 3.6+, but having a bit of trouble with how forward references are defined in the typing module in Python 3.6. I've setup pyenv on my local Windows machine so that I can switch between different Python versions at ease for local testing, as my system interpreter defaults to Python 3.9.
The use case here essentially is I'm trying to define a TypeVar with the valid forward reference types, which I can then use for type annotation purposes. I've confirmed the following code runs without issue when I'm on 3.7+ and import ForwardRef from the typing module directly, but I'm unable to get it on Python 3.6 since I noticed forward refs can't be used as arguments to TypeVar for some reason. I also tried passing the forward ref type as an argument to Union , but I ran into a similar issue.
Here are the imports and the definition forTypeVar that I'm trying to get to work on both python 3.6.0 as well as more recent versions like 3.6.8 - I did notice I get different errors between minor versions:
from typing import _ForwardRef as PyForwardRef, TypeVar
# Errors on PY 3.6:
# 3.6.2+ -> AttributeError: type object '_ForwardRef' has no attribute '_gorg'
# 3.6.2 or earlier -> AssertionError: assert isinstance(a, GenericMeta)
FREF = TypeVar('FREF', str, PyForwardRef)
Here is a sample usage I've been able to test out, which appears to type check as expected for Python 3.7+:
class MyClass: ...
def my_func(typ: FREF):
pass
# Type checks
my_func('testing')
my_func(PyForwardRef('MyClass'))
# Does not type check
my_func(23)
my_func(MyClass)
What I've Done So Far
Here's my current workaround I'm using to support Python 3.6. This isn't pretty but it seems to at least get the code to run without any errors. However this does not appear to type check as expected though - at least not in Pycharm.
import typing
# This is needed to avoid an`AttributeError` when using PyForwardRef
# as an argument to `TypeVar`, as we do below.
if hasattr(typing, '_gorg'): # Python 3.6.2 or lower
_gorg = typing._gorg
typing._gorg = lambda a: None if a is PyForwardRef else _gorg(a)
else: # Python 3.6.3+
PyForwardRef._gorg = None
Wondering if I'm on the right track, or if there's a simpler solution I can use to support ForwardRef types as arguments to TypeVar or Union in Python 3.6.
To state the obvious, the issue here appears to be due to several changes in the typing module between Python 3.6 and Python 3.7.
In both Python 3.6 and Python 3.7:
All constraints on a TypeVar are checked using the typing._type_check function (links are to the 3.6 branch of the source code on GitHub) before the TypeVar is allowed to be instantiated.
TypeVar.__init__ looks something like this in the 3.6 branch:
class TypeVar(_TypingBase, _root=True):
# <-- several lines skipped -->
def __init__(self, name, *constraints, bound=None,
covariant=False, contravariant=False):
# <-- several lines skipped -->
if constraints and bound is not None:
raise TypeError("Constraints cannot be combined with bound=...")
if constraints and len(constraints) == 1:
raise TypeError("A single constraint is not allowed")
msg = "TypeVar(name, constraint, ...): constraints must be types."
self.__constraints__ = tuple(_type_check(t, msg) for t in constraints)
# etc.
In Python 3.6:
There was a class called _ForwardRef. This class was given a name with a leading underscore to warn users that it was an implementation detail of the module, and that therefore the API of the class could change unexpectedly between Python versions.
It appears that typing._type_check did not account for the possibility that _ForwardRef might be passed to it, hence the strange AttributeError: type object '_ForwardRef' has no attribute '_gorg' error message. I assume that this possibility was not accounted for because it was assumed that users would know not to use classes marked as implementation details.
In Python 3.7:
_ForwardRef has been replaced with a ForwardRef class: this class is no longer an implementation detail; it is now part of the module's public API.
typing._type_check now explicitly accounts for the possibility that ForwardRef might be passed to it:
def _type_check(arg, msg, is_argument=True):
"""Check that the argument is a type, and return it (internal helper).
As a special case, accept None and return type(None) instead. Also wrap strings
into ForwardRef instances. Consider several corner cases, for example plain
special forms like Union are not valid, while Union[int, str] is OK, etc.
The msg argument is a human-readable error message, e.g::
"Union[arg, ...]: arg should be a type."
We append the repr() of the actual value (truncated to 100 chars).
"""
# <-- several lines skipped -->
if isinstance(arg, (type, TypeVar, ForwardRef)):
return arg
# etc.
Solutions
I'm tempted to argue that it's not really worth the effort to support Python 3.6 at this point, given that Python 3.6 is kind of old now, and will be officially unsupported from December 2021. However, if you do want to continue to support Python 3.6, a slightly cleaner solution might be to monkey-patch typing._type_check rather than monkey-patching _ForwardRef. (By "cleaner" I mean "comes closer to tackling the root of the problem, rather than a symptom of the problem" — it's obviously less concise than your existing solution.)
import sys
from typing import TypeVar
if sys.version_info < (3, 7):
import typing
from typing import _ForwardRef as PyForwardRef
from functools import wraps
_old_type_check = typing._type_check
#wraps(_old_type_check)
def _new_type_check(arg, message):
if arg is PyForwardRef:
return arg
return _old_type_check(arg, message)
typing._type_check = _new_type_check
# ensure the global namespace is the same for users
# regardless of the version of Python they're using
del _old_type_check, _new_type_check, typing, wraps
else:
from typing import ForwardRef as PyForwardRef
However, while this kind of thing works fine as a runtime solution, I have honestly no idea whether there is a way to make type-checkers happy with this kind of monkey-patching. Pycharm, MyPy and the like certainly won't be expecting you to do something like this, and probably have their support for TypeVars hardcoded for each version of Python.
Using Python 3's function annotations, it is possible to specify the type of items contained within a homogeneous list (or other collection) for the purpose of type hinting in PyCharm and other IDEs?
A pseudo-python code example for a list of int:
def my_func(l:list<int>):
pass
I know it's possible using Docstring...
def my_func(l):
"""
:type l: list[int]
"""
pass
... but I prefer the annotation style if it's possible.
Answering my own question; the TLDR answer is No Yes.
Update 2
In September 2015, Python 3.5 was released with support for Type Hints and includes a new typing module. This allows for the specification of types contained within collections. As of November 2015, JetBrains PyCharm 5.0 fully supports Python 3.5 to include Type Hints as illustrated below.
Update 1
As of May 2015, PEP0484 (Type Hints) has been formally accepted. The draft implementation is also available at github under ambv/typehinting.
Original Answer
As of Aug 2014, I have confirmed that it is not possible to use Python 3 type annotations to specify types within collections (ex: a list of strings).
The use of formatted docstrings such as reStructuredText or Sphinx are viable alternatives and supported by various IDEs.
It also appears that Guido is mulling over the idea of extending type annotations in the spirit of mypy: http://mail.python.org/pipermail/python-ideas/2014-August/028618.html
Update: please see other answers, this one is outdated.
Original Answer (2015):
Now that Python 3.5 is officially out, there is the Type Hints supporting module - typing and the relevant List "type" for the generic containers.
In other words, now you can do:
from typing import List
def my_func(l: List[int]):
pass
As of Python 3.9, builtin types are generic with respect to type annotations (see PEP 585). This allows to directly specify the type of elements:
def my_func(l: list[int]):
pass
This also extends to most other container types of the standard library, for example collections.deque or collections.abc.Mapping.
Various tools may support this syntax earlier than Python 3.9. When annotations are not inspected at runtime, the syntax is valid using quoting or __future__.annotations.
# quoted
def my_func(l: 'list[int]'):
pass
# postponed evaluation of annotation
from __future__ import annotations
def my_func(l: list[int]):
pass
As a consequence of PEP 585, most helpers in typing corresponding to standard library types are deprecated, such as typing.List, typing.Deque or typing.Mapping. They should only be used if compatibility with Python versions prior to 3.9 is required.
Type comments have been added since PEP 484
from . import Monitor
from typing import List, Set, Tuple, Dict
active_monitors = [] # type: List[Monitor]
# or
active_monitors: List[Monitor] = []
# bonus
active_monitors: Set[Monitor] = set()
monitor_pair: Tuple[Monitor, Monitor] = (Monitor(), Monitor())
monitor_dict: Dict[str, Monitor] = {'codename': Monitor()}
# nested
monitor_pair_list: List[Dict[str, Monitor]] = [{'codename': Monitor()}]
This is currently working for me on PyCharm with Python 3.6.4
With support from the BDFL, it's almost certain now that python (probably 3.5) will provide a standardized syntax for type hints via function annotations.
https://www.python.org/dev/peps/pep-0484/
As referenced in the PEP, there is an experimental type-checker (kind of like pylint, but for types) called mypy that already uses this standard, and doesn't require any new syntax.
http://mypy-lang.org/
In newly released Python 3.8 there is a new type annotation typing.TypedDict. Its documentation mentions that
The type info for introspection can be accessed via Point2D.__annotations__ and Point2D.__total__. [....]
While __annotations__ is well-known, having been introduced in PEP 3107, I cannot find any information on __total__. Could anyone explain its meaning and if possible linking to authoritative sources?
I am guessing that the __total__ field signifies whether instances must be complete (the default) or not (all fields optional). I started my search at PEP 589, which introduced TypedDict and describes totality as such. It used a total argument, which it would make sense to rename dunder-style for the class syntax. However, I did not find when such a renaming took place.
Looking into MyPy, which is the actual type checker that cares about these annotations, there is similar documentation on TypedDict and totality, but again no reference to the dunder syntax. Digging into its implementation led to more confusion, as TypedDictType in types.py doesn't have a total field, but separate items and required_keys. Totality would imply that items.keys()==required_keys but the implementation makes different assumptions, such as can_be_false relying on items alone. total=False should in principle mean required_keys is empty.
The CPython source for _TypedDictMeta at least reveals that the total argument and __total__ dunder are one and the same, although the source describes TypedDict itself as "may be added soon".
TypedDict was accepted in Python 3.8 via PEP 589. From Python, it appears __total__ is a boolean flag set to True by default:
tot = TypedDict.__total__
print(type(tot))
print(tot)
# <class 'bool'>
# True
As mentioned in other posts, details on this method are limited in the docs, but #Yann Vernier's link to the CPython source code strongly suggests __total__ is related to the new total keyword introduced in Python 3.8:
# cypthon/typing.py
class _TypedDictMeta(type):
def __new__(cls, name, bases, ns, total=True):
"""Create new typed dict class object.
...
"""
...
if not hasattr(tp_dict, '__total__'):
tp_dict.__total__ = total
...
How does it work?
Synopsis: by default, all keys are required when instantiating a defined TypedDict. total=False overrides this restriction and allows optional keys. See the following demonstration.
Given
A test directory tree:
Code
Files in the test directory:
# rgb_bad.py
from typing import TypedDict
class Color(TypedDict):
r: int
g: int
b: int
a: float
blue = Color(r=0, g=0, b=255) # missing "a"
# rgb_good.py
from typing import TypedDict
class Color(TypedDict, total=False):
r: int
g: int
b: int
a: float
blue = Color(r=0, g=0, b=255) # missing "a"
Demo
If a key is missing, mypy will complain at the commandline:
> mypy code/rgb_bad.py
code\rgb_bad.py:11: error: Key 'a' missing for TypedDict "Color"
...
Setting total=False permits optional keys:
> mypy code/rgb_good.py
Success: no issues found in 1 source file
See Also
Tweet by R. Hettinger demonstating totality
PEP section on totality in PEP 589
Article Section on types and TypedDict in Python 3.8 by Real Python
typing-extensions package to use TypedDict in Python 3.5, 3.6
Using Python 3's function annotations, it is possible to specify the type of items contained within a homogeneous list (or other collection) for the purpose of type hinting in PyCharm and other IDEs?
A pseudo-python code example for a list of int:
def my_func(l:list<int>):
pass
I know it's possible using Docstring...
def my_func(l):
"""
:type l: list[int]
"""
pass
... but I prefer the annotation style if it's possible.
Answering my own question; the TLDR answer is No Yes.
Update 2
In September 2015, Python 3.5 was released with support for Type Hints and includes a new typing module. This allows for the specification of types contained within collections. As of November 2015, JetBrains PyCharm 5.0 fully supports Python 3.5 to include Type Hints as illustrated below.
Update 1
As of May 2015, PEP0484 (Type Hints) has been formally accepted. The draft implementation is also available at github under ambv/typehinting.
Original Answer
As of Aug 2014, I have confirmed that it is not possible to use Python 3 type annotations to specify types within collections (ex: a list of strings).
The use of formatted docstrings such as reStructuredText or Sphinx are viable alternatives and supported by various IDEs.
It also appears that Guido is mulling over the idea of extending type annotations in the spirit of mypy: http://mail.python.org/pipermail/python-ideas/2014-August/028618.html
Update: please see other answers, this one is outdated.
Original Answer (2015):
Now that Python 3.5 is officially out, there is the Type Hints supporting module - typing and the relevant List "type" for the generic containers.
In other words, now you can do:
from typing import List
def my_func(l: List[int]):
pass
As of Python 3.9, builtin types are generic with respect to type annotations (see PEP 585). This allows to directly specify the type of elements:
def my_func(l: list[int]):
pass
This also extends to most other container types of the standard library, for example collections.deque or collections.abc.Mapping.
Various tools may support this syntax earlier than Python 3.9. When annotations are not inspected at runtime, the syntax is valid using quoting or __future__.annotations.
# quoted
def my_func(l: 'list[int]'):
pass
# postponed evaluation of annotation
from __future__ import annotations
def my_func(l: list[int]):
pass
As a consequence of PEP 585, most helpers in typing corresponding to standard library types are deprecated, such as typing.List, typing.Deque or typing.Mapping. They should only be used if compatibility with Python versions prior to 3.9 is required.
Type comments have been added since PEP 484
from . import Monitor
from typing import List, Set, Tuple, Dict
active_monitors = [] # type: List[Monitor]
# or
active_monitors: List[Monitor] = []
# bonus
active_monitors: Set[Monitor] = set()
monitor_pair: Tuple[Monitor, Monitor] = (Monitor(), Monitor())
monitor_dict: Dict[str, Monitor] = {'codename': Monitor()}
# nested
monitor_pair_list: List[Dict[str, Monitor]] = [{'codename': Monitor()}]
This is currently working for me on PyCharm with Python 3.6.4
With support from the BDFL, it's almost certain now that python (probably 3.5) will provide a standardized syntax for type hints via function annotations.
https://www.python.org/dev/peps/pep-0484/
As referenced in the PEP, there is an experimental type-checker (kind of like pylint, but for types) called mypy that already uses this standard, and doesn't require any new syntax.
http://mypy-lang.org/