Develop Reference

Raise an error if type hint is violated/ignored in Python?

After looking at this question I learned that the type hints are, by default, not enforced whilst executing Python code.
One can detect some discrepancies between the type hints and actual argument types using a slightly convoluted process of running pyannotate to generate stubs whilst running Python code, and scanning for differences after applying these stubs to the code.
However, it would be more convenient/faster to directly raise an exception if an incoming argument is not of the type included in the type hint. This can be achieved by manually including:
if not isinstance(some_argument, the_type_hint_type):
raise TypeError("Argument:{argument} is not of type:{the_type_hint_type}")
However, that is quite labour intensive. Hence, I was curious, is it possible to make Python raise an error if a type-hint is violated, using an CLI argument or pip package or something like that?

Hope this helps you - https://typeguard.readthedocs.io/en/latest/userguide.html#using-the-decorator
Thanks..

The edit queue for the Answer by #Surya_1897 is full, hence I will include a more detailed description of the solution here.
Typeguard does exactly what I was looking for. The following requirements apply:
Install typeguard with:
pip install typeguard
Import typeguard into each script, and add the #typechecked property above each function.
Example:
Change:
"""Some file description."""
def add_two(x:int):
"""Adds two to an incoming int."""
return x+2
somevar:float=42.1
add_two(somevar)
To:
"""Some file description."""
from typeguard import typechecked
#typechecked
def add_two(x:int):
"""Adds two to an incoming int."""
return x+2
somevar:float=42.1
add_two(somevar)
The latter will than throw an err:
TypeError: type of argument "x" must be int; got float instead

Testing for None on a non Optional input parameter

Let's say I have a python module with the following function:
def is_plontaria(plon: str) -> bool:
if plon is None:
raise RuntimeError("None found")
return plon.find("plontaria") != -1
For that function, I have the unit test that follows:
def test_is_plontaria_null(self):
with self.assertRaises(RuntimeError) as cmgr:
is_plontaria(None)
self.assertEqual(str(cmgr.exception), "None found")
Given the type hints in the function, the input parameter should always be a defined string. But type hints are... hints. Nothing prevents the user from passing whatever it wants, and None in particular is a quite common option when previous operations fail to return the expected results and those results are not checked.
So I decided to test for None in the unit tests and to check the input is not None in the function.
The issue is: the type checker (pylance) warns me that I should not use None in that call:
Argument of type "None" cannot be assigned to parameter "plon" of type "str" in function "is_plontaria"
Type "None" cannot be assigned to type "str"
Well, I already know that, and that is the purpose of that test.
Which is the best way to get rid of that error? Telling pylance to ignore this kind of error in every test/file? Or assuming that the argument passed will be always of the proper type and remove that test and the None check in the function?

This is a good question. I think that silencing that type error in your test is not the right way to go.
Don't patronize the user
While I would not go so far as to say that this is universally the right way to do it, in this case I would definitely recommend getting rid of your None check from is_plontaria.
Think about what you accomplish with this check. Say a user calls is_plontaria(None) even though you annotated it with str. Without the check he causes an AttributeError: 'NoneType' object has no attribute 'find' with a traceback to the line return plon.find("plontaria") != -1. The user thinks to himself "oops, that function expects a str". With your check he causes a RuntimeError ideally telling him that plon is supposed to be a str.
What purpose did the check serve? I would argue none whatsoever. Either way, an error is raised because your function was misused.
What if the user passes a float accidentally? Or a bool? Or literally anything other than a str? Do you want to hold the user's hand for every parameter of every function you write?
And I don't buy the "None is a special case"-argument. Sure, it is a common type to be "lying around" in code, but that is still on the user, as you pointed out yourself.
If you are using properly type annotated code (as you should) and the user is too, such a situation should never happen. Say the user has another function foo that he wants to use like this:
def foo() -> str | None:
...
s = foo()
b = is_plontaria(s)
That last line should cause any static type checker worth its salt to raise an error, saying that is_plontaria only accepts str, but a union of str and None was provided. Even most IDEs mark that line as problematic.
The user should see that before he even runs his code. Then he is forced to rethink and either change foo or introduce his own type check before calling your function:
s = foo()
if isinstance(s, str):
b = is_plontaria(s)
else:
# do something else
Qualifier
To be fair, there are situations where error messages are very obscure and don't properly tell the caller what went wrong. In those cases it may be useful to introduce your own. But aside from those, I would always argue in the spirit of Python that the user should be considered mature enough to do his own homework. And if he doesn't, that is on him, not you. (So long as you did your homework.)
There may be other situations, where raising your own type-errors makes sense, but I would consider those to be the exception.
If you must, use Mock
As a little bonus, in case you absolutely do want to keep that check in place and need to cover that if-branch in your test, you can simply pass a Mock as an argument, provided your if-statement is adjusted to check for anything other than str:
from unittest import TestCase
from unittest.mock import Mock
def is_plontaria(plon: str) -> bool:
if not isinstance(plon, str):
raise RuntimeError("None found")
return plon.find("plontaria") != -1
class Test(TestCase):
def test_is_plontaria(self) -> None:
not_a_string = Mock()
with self.assertRaises(RuntimeError):
is_plontaria(not_a_string)
...
Most type checkers consider Mock to be a special case and don't complain about its type, assuming you are running tests. mypy for example is perfectly happy with such code.
This comes in handy in other situations as well. For example, when the function being tested expects an instance of some custom class of yours as its argument. You obviously want to isolate the function from that class, so you can just pass a mock to it that way. The type checker won't mind.
Hope this helps.

You can disable type checking for on a specific line with a comment.
def test_is_plontaria_null(self):
with self.assertRaises(RuntimeError) as cmgr:
is_plontaria(None) # type: ignore
self.assertEqual(str(cmgr.exception), "None found")

Python change Exception printable output, eg overload __builtins__

I am searching for a way to change the printable output of an Exception to a silly message in order to learn more about python internals (and mess with a friend ;), so far without success.
Consider the following code
try:
x # is not defined
except NameError as exc:
print(exc)
The code shall output name 'x' is not defined
I would like the change that output to the name 'x' you suggested is not yet defined, my lord. Improve your coding skills.
So far, I understood that you can't change __builtins__ because they're "baked in" as C code, unless:
You use forbiddenfruit.curse method which adds / changes properties of any object
You manually override the dictionnaries of an object
I've tried both solutions, but without success:
forbiddenfruit solution:
from forbiddenfruit import curse
curse(BaseException, 'repr', lambda self: print("Test message for repr"))
curse(BaseException, 'str', lambda self: print("Test message for str"))
try:
x
except NameError as exc:
print(exc.str()) # Works, shows test message
print(exc.repr()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
Dictionnary overriding solution:
import gc
underlying_dict = gc.get_referents(BaseException.__dict__)[0]
underlying_dict["__repr__"] = lambda self: print("test message for repr")
underlying_dict["__str__"] = lambda self: print("test message for str")
underlying_dict["args"] = 'I am an argument list'
try:
x
except NameError as exc:
print(exc.__str__()) # Works, shows test message
print(exc.__repr__()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
AFAIK, using print(exc) should rely on either __repr__ or __str__, but it seems like the print function uses something else, which I cannot find even when reading all properties of BaseException via print(dir(BaseException)).
Could anyone give me an insight of what print uses in this case please ?
[EDIT]
To add a bit more context:
The problem I'm trying to solve began as a joke to mess with a programmer friend, but now became a challenge for me to understand more of python's internals.
There's no real business problem I'm trying to solve, I just want to get deeper understanding of things in Python.
I'm quite puzzled that print(exc) won't make use of BaseException.__repr__ or __str__ actually.
[/EDIT]

Intro
I'd go with a more critical approach on why you'd even want to do what you want to do.
Python provides you with an ability to handle specific exceptions. That means if you had a business problem, you'd use a particular exception class and provide a custom message for that specific case. Now, remember this paragraph and let's move on, I'll refer to this later.
TL;DR
Now, let's go top-down:
Catching all kinds of errors with except Exception is generally not a good idea if want you catch let's say a variable name error. You'd use except NameError instead. There's really not much you'd add to it that's why it had a default message that perfectly described the issue. So it's assumed you'd use it as it's given. These are called concrete exceptions.
Now, with your specific case notice the alias as exc. By using the alias you can access arguments passed to the exception object, including the default message.
try:
x # is not defined
except NameError as exc:
print(exc.args)
Run that code (I put it in app.py) and you'll see:
$ python app.py
("name 'x' is not defined",)
These args are passed to the exception as a series (list, or in this case immutable list that is a tuple).
This leads to the idea of the possibility of easily passing arguments to exceptions' constructors (__init__). In your case "name 'x' is not defined" was passed as an argument.
You can use this to your advantage to solve your problem without much effort by just providing a custom message, like:
try:
x # is not defined
except NameError as exc:
your_custom_message = "the name 'x' you suggested is not yet defined, my lord. Improve your coding skills"
# Now, you can handle it based on your requirement:
# print(your_custom_message)
# print(NameError(your_custom_message))
# raise NameError(your_custom_message)
# raise NameError(your_custom_message) from exc
The output is now what you wanted to achieve.
$ python app.py
the name 'x' you suggested is not yet defined, my lord. Improve your coding skills
Remember the first paragraph when I said I'd refer to it later? I mentioned providing a custom message for a specific case. If you build your own library when you want to handle name errors to specific variables relevant to your product, you assume your users will use your code that might raise that NameError exception. They will most likely catch it with except Exception as exc or except NameError as exc. And when they do print(exc), they will see your message now.
Summary
I hope that makes sense to you, just provide a custom message and pass it as an argument to NameError or simply just print it. IMO, it's better to learn it right together with why you'd use what you use.

Errors like this are hard-coded into the interpreter (in the case of CPython, anyway, which is most likely what you are using). You will not be able to change the message printed from within Python itself.
The C source code that is executed when the CPython interpreter tries to look up a name can be found here: https://github.com/python/cpython/blob/master/Python/ceval.c#L2602. If you would want to change the error message printed when a name lookup fails, you would need to change this line in the same file:
#define NAME_ERROR_MSG \
"name '%.200s' is not defined"
Compiling the modified source code would yield a Python interpreter that prints your custom error message when encountering a name that is not defined.

I'll just explain the behaviour you described:
exc.__repr__()
This will just call your lambda function and return the expected string. Btw you should return the string, not print it in your lambda functions.
print(repr(exc))
Now, this is going a different route in CPython and you can see this in a GDB session, it's something like this:
Python/bltinmodule.c:builtin_repr will call Objects/object.c:PyObject_Repr - this function gets the PyObject *v as the only parameter that it will use to get and call a function that implements the built-in function repr(), BaseException_repr in this case. This function will format the error message based on a value from args structure field:
(gdb) p ((PyBaseExceptionObject *) self)->args
$188 = ("name 'x' is not defined",)
The args value is set in Python/ceval.c:format_exc_check_arg based on a NAME_ERROR_MSG macro set in the same file.
Update: Sun 8 Nov 20:19:26 UTC 2020
test.py:
import sys
import dis
def main():
try:
x
except NameError as exc:
tb = sys.exc_info()[2]
frame, i = tb.tb_frame, tb.tb_lasti
code = frame.f_code
arg = code.co_code[i + 1]
name = code.co_names[arg]
print(name)
if __name__ == '__main__':
main()
Test:
# python test.py
x
Note:
I would also recommend to watch this video from PyCon 2016.

Parameterized generics cannot be used with class or instance checks

I wrote the code, but I get the following message in pycharm(2019.1):
"Parameterized generics cannot be used with class or instance checks"
def data_is_valid(data):
keys_and_types = {
'comment': (str, type(None)),
'from_budget': (bool, type(None)),
'to_member': (int, type(None)),
'survey_request': (int, type(None)),
}
def type_is_valid(test_key, test_value):
return isinstance(test_value, keys_and_types[test_key])
type_is_valid('comment', 3)
I really do not understand this message well. Did I do something wrong or is it a bug in pycharm?
The error disappears if I explicitly typecast to tuple.
def type_is_valid(test_key, test_value):
return isinstance(test_value, tuple(keys_and_types[test_key]))

That looks like a bug in pycharm where it's a bit overeager in assuming that you're using the typing module in an unintended way. See this example here where that assumption would have been correct:
The classes in the typing module are only useful in a type annotation context, not to inspect or compare to actual classes, which is what isinstance tries to do. Since pycharm sees a simple object with square brackets that do not contain a literal, it jumps to the wrong conclusion you are seeing.
Your code is fine, you can use it exactly as it is.

I will not repeat after others that this is a pycharm bug. Just if you are a perfectionist and the error hurts your eyes, add the comment
# noqa
to the line where the "error" is

This was a known bug in PyCharm 2018, reported here.
There are some related bugs still in more recent PyCharm versions, e.g. PyCharm 2021.2.2, here.
In general, when you found that some PyCharm warning is incorrect, I would first isolate a simple test case where it becomes more clear what PyCharm is actually incorrect about. When it is clear that PyCharm is wrong with the warning, then you should always fill a bug report about it (or maybe search for existing bug reports first). Here this is clear because PyCharm says you cannot do sth, while in fact you can, so sth is wrong.

Since it's agreed it's a bug, you can suppress it in Pycharm by the line:
# noinspection PyTypeHints

Can I suppress mypy errors in-line?

I recently made the mistake of opening my $PYTHONSTARTUP file with mypy syntax checking enabled. As a result, I started getting this error:
startup.py|79 col 2 error| Incompatible types in assignment (expression has type "HistoryPrompt", variable has type "str")
On line 79:
sys.ps1 = HistoryPrompt()
I immediately thought, "By Jove, mypy! You're entirely correct! And yet, that is exactly what I want to do, so you're also wrong!"
I went looking to see if there was some kind of "stub" for the sys module, but couldn't find anything. I'm guessing that mypy is determining the type by looking at the value stored in the variable (default is ">>> ", which is a str).
In reality, of course, the type needs to be the non-existant typing.Stringifiable, indicating an object that will respond to str(x).
Having reached that dead end, I went looking for a way to tell mypy to suppress the error. So many of the other tools support # noqa: xxx that I figured there must be something, right?
Wrong. Or at least, I couldn't find it in my version, which is: mypy 0.670
So I devised a hack clever work-around:
import typing
# Work around mypy error: Incompatible types in assignment
suppress_mypy_error: typing.Any = HistoryPrompt()
sys.ps1 = suppress_mypy_error
My question is this: Is there a way to suppress this particular error in-line (best), or in mypy.ini, or by submitting a PR to python/mypy, or ...?

To explicitly suppress MyPy on a specific line, add a comment of the form # type: ignore.
To suppress MyPy for an entire module, add the same comment at the top of the module.
Source: https://mypy.readthedocs.io/en/latest/common_issues.html#spurious-errors-and-locally-silencing-the-checker

Overview
I prefer to suppress mypy errors based on 2 things:
specific lines (your question), and
specific error.
Example
For example, the # type: ignore [no-untyped-call]:
# ignore mypy error because azure has broken type hints.
# See https://github.com/Azure/azure-sdk-for-python/issues/20083 (the issue is closed but the problem remains)
exception = azure.core.exceptions.ResourceNotFoundError("Test") # type: ignore [no-untyped-call]
You can find out the "error code" (e.g. no-untyped-call) in the mypy output by configuring mypy with:
in pyproject.toml
[tool.mypy]
show_error_codes = true
or in mypy.ini
[mypy]
show_error_codes = True
Benefits
Documentation in code: You can see exactly what error is being suppressed.
You won't ignore other errors by mypy. Otherwise, that line could be a source of bugs in the future and mypy would not warn you.