Develop Reference

Running `python -m unittest` changes the way exceptions with overriden `__name__` are printed in the traceback

I have a piece of code that dynamically creates Exceptions. Every exception class created in this way gets its __name__ overwritten:
def exception_injector(name, parent, module_dict):
class product_exception(parent):
pass
product_exception.__name__ = name
product_exception.__module__ = module_dict["__name__"]
module_dict[name] = product_exception
When I use it regularly it prints everything out just fine:
>>> namespace = {"__name__": "some.module"}
>>> exception_injector("TestError", Exception, namespace)
>>> raise namespace["TestError"]("What's going on?")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
some.module.TestError: What's going on?
But when I use the unittest module the original __name__ is printed:
>>> import unittest
>>> class DemoTestCase(unittest.TestCase):
... def test_raise(self):
... namespace = {"__name__": "some.module"}
... exception_injector("TestError", Exception, namespace)
... namespace["TestError"]("What's going on?")
...
>>> unittest.main(defaultTest="DemoTestCase", argv=["demo"], exit=False)
E
======================================================================
ERROR: test_raise (__main__.DemoTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<stdin>", line 3, in test_raise
some.module.exception_injector.<locals>.product_exception: What's going on?
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
<unittest.main.TestProgram object at 0x1017c3eb8>
From where on the exception object could the unittest module be getting this original information?

Unittests use the traceback module to format exceptions. You can reproduce your output by doing the same:
>>> import traceback
>>> try:
... raise namespace["TestError"]("What's going on?")
... except Exception as e:
... tb = traceback.TracebackException.from_exception(e)
... print(*tb.format())
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
some.module.exception_injector.<locals>.product_exception: What's going on?
What is being printed is the object __module__ attribute value*, with object.__qualname__ attribute appended (with a dot separator), rather than __module__ plus __name__:
>>> namespace["TestError"].__qualname__
'exception_injector.<locals>.product_exception'
The qualified name includes the full scope where the class was created (function names here, but this could also include class names).
If your aim is to add exceptions to the global namespaces of a module, you can just set it to the same value as name:
>>> namespace["TestError"].__qualname__ = "TestError"
>>> try:
... raise namespace["TestError"]("What's going on?")
... except Exception as e:
... tb = traceback.TracebackException.from_exception(e)
... print(*tb.format())
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
some.module.TestError: What's going on?
or in the context of your code:
def exception_injector(name, parent, module_dict):
class product_exception(parent, details=args):
pass
product_exception.__name__ = name
product_exception.__qualname__ = name
product_exception.__module__ = module_dict["__name__"]
module_dict[name] = product_exception
* The traceback module omits the exception module if the module name is either __main__ or builtins.

Raise custom ImportError when user imports a deprecated variable [duplicate]

This question already has answers here:
How to mark a global as deprecated in Python?
(4 answers)
Closed 4 years ago.
I have written a package where a sub-module contains a module-level variable deprecated_var that I want to remove, because it was a horrible mistake.
mypkg
- mymodule
- __init__.py
But instead of just leaving my end users with a generic ImportError, I want to print a message that says their import is deprecated, and what they should do. So instead of:
>>> from mypkg.mymodule import deprecated_var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'deprecated_var'
I want users to see something like this:
>>> from mypkg.mymodule import deprecated_var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: mypkg.mymodule.deprecated_var was removed. Replace
with "from foo.bar import Baz; deprecated_var = Baz()"
How can I achieve that?

I don't think this is possible before python 3.7.
However, in python 3.7 or later, this can be achieved by using the module level __getattr__ added in PEP562.
You'd use like so:
#_deprecated_vars is a dict of keys -> alternatives (or None)
_deprecated_vars: Dict[str, Optional[str]] = {
'deprecated_var': 'from foo.bar import Baz; deprecated_var = Baz()',
'other_deprecated_var': None
}
def __getattr__(name):
if name in _deprecated_vars:
alt_text = '{name} was removed from module {__name__}'
replace_text = _deprecated_vars[name]
if replace_text is not None:
alt_text += f'. Replace with {replace_text!r}.'
raise AttributeError(alt_text)
raise AttributeError(f"module {__name__} has no attribute {name}")
However, I'm not sure this works for your use case of from a.b import deprecated_var. This is more for import a.b; a.b.deprecated_var. See the other answer for the former.

For your specific example, you could use the following:
mymodule/__init__.py:
#deprecated_var = 5
replacement_var = 6
mymodule/deprecated_var.py:
raise ImportError("deprecated_var is deprecated. Use mypkg.mymodule.replacement_var instead")
While this raises the custom ImportError when importing the variable directly:
>>> from mymodule import deprecated_var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File ".../mymodule/deprecated_var.py", line 1, in <module>
raise ImportError("deprecated_var is deprecated. Use mypkg.mymodule.replacement_var instead")
ImportError: deprecated_var is deprecated. Use mypkg.mymodule.replacement_var instead
it does nothing when accessing it as a module attribute. Or rather, it throws an AttributeError instead of a deprecation warning:
>>> import mymodule
>>> mymodule.deprecated_var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'mymodule' has no attribute 'deprecated_var'

python reload module for beginner. importlib.reload doesn't seem to work

I have a file called skdb and class called skmysqldb. I am trying to force reload.
I tried reloading "skdb", "skdb.skmysqldb" "skmysqldb" and none of them seem to work.
>>> from skdb import skmysqldb
>>> importlib.reload(skdb.skmysqldb)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'skdb' is not defined
>>> importlib.reload(skmysqldb)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py", line 139, in reload
raise TypeError("reload() argument must be a module")
TypeError: reload() argument must be a module
>>> importlib.reload(skdb)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'skdb' is not defined

When you import some object using the from <module> import <obj> syntax as in
from skdb import skmysqldb
the module itself is not added to the current namespace, hence why you get a NameError when you try to do reload(skdb).
Instead try:
import skdb
importlib.reload(skdb)
Be cautious when using reload. If the module your reload imports other modules, those modules are not reloaded recursively, so depending on the exact code you can wind up in a rather broken state where it's better to just restart the whole interpreter.

I don't think this is supported, but try doing del sys.modules['mymodule'] for everything that vaguely matches. To find relevant ones, try something like [x for x in sys.modules if 'mymodule' in x].

ImportError: cannot import name (not a circular dependency)

I have the problem on importing the class in the same package, and it seems not a circular dependency problem. So I'm really confused now.
my-project/
lexer.py
exceptions.py
I declared an exception in exceptions.py and wants to use it in lexer.py:
exceptions.py:
class LexError(Exception):
def __init__(self, message, line):
self.message = message
self.line = line
and in lexer.py:
import re
import sys
from exceptions import LexError
...
It shouldn't be circular dependency since lexer.py is the only file has import in it.
Thanks!!

exceptions conflicts with builtin module exception.
>>> import exceptions
>>> exceptions.LexError
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'LexError'
>>> from exceptions import LexError
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name LexError
Use different module name.

python how to include a file of lists in a script

I have a file that gets generated by :
excerpt:
group0 = ['ParentPom']
group1 = ['Commons','http', 'availability','ingestPom','abcCommons','solrIndex','123Service']
...
group10=['totalCommons','Generator']
How can I include this in my python script, tried import but no luck
>>> import dependencies_custom
>>> print (group2[0])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'group2' is not defined

In the import form you're using, you should be able to access the groups by
dependencies_custom.group2[0]
type notation. If you want just use just group2[0] notation, try using:
from dependencies_custom import *