How can you find where python imported a particular module from?
Each module object has a __file__ attribute:
import module
print module.__file__
Some modules are part of the Python executable; these will not have the attribute set.
Demo:
>>> import urllib2
>>> urllib2.__file__
'/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/urllib2.pyc'
>>> import sys
>>> sys.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'
You could also run Python in verbose mode, with the -v command line switch or the PYTHONVERBOSE environment variable; Python then prints out every imported file as it takes place.
Related
I have a script named requests.py that needs to use the third-party requests package. The script either can't import the package, or can't access its functionality.
Why isn't this working, and how do I fix it?
Trying a plain import and then using the functionality results in an AttributeError:
import requests
res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "/Users/me/dev/rough/requests.py", line 1, in <module>
import requests
File "/Users/me/dev/rough/requests.py", line 3, in <module>
requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'
In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import).
Using from-import of a specific name results in an ImportError:
from requests import get
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import get
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests import get
ImportError: cannot import name 'get'
In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py).
Using from-import for a module inside the package results in a different ImportError:
from requests.auth import AuthBase
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests.auth import AuthBase
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package
Using a star-import and then using the functionality raises a NameError:
from requests import *
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import *
File "/Users/me/dev/rough/requests.py", line 3, in <module>
res = get('http://www.google.ca')
NameError: name 'get' is not defined
This happens because your local module named requests.py shadows the installed requests module you are trying to use. The current directory is prepended to sys.path, so the local name takes precedence over the installed name.
An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:
Notice the name you used in your script:
File "/Users/me/dev/rough/requests.py", line 1, in <module>
The module you are trying to import: requests
Rename your module to something else to avoid the name collision.
Python may generate a requests.pyc file next to your requests.py file (in the __pycache__ directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc file in __pycache__ should not affect your code if the py file has been removed.
In the example, renaming the file to my_requests.py, removing requests.pyc, and running again successfully prints <Response [200]>.
The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.
For example: suppose that you have a script mydecimal.py that includes import decimal, intending to use the standard library decimal library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal. However, it so happens that decimal imports numbers (another standard library module) for internal use, so a script called numbers.py in your project would cause the problem.
In one especially pernicious case, having a file named token.py in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)
The traceback tells us all we need to know: calling help triggers a deferred import of the standard library pydoc, which indirectly attempts to import the standard library token, but finds our token.py which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize would do a star-import from token, and then its top-level code would try to use a name defined there, resulting in NameError - and a stack trace not mentioning the file name token.py.
If you still encounter problems like this after tracking own and renaming or removing the appropriate .py files in your project, also check for .pyc files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).
I have a script named requests.py that needs to use the third-party requests package. The script either can't import the package, or can't access its functionality.
Why isn't this working, and how do I fix it?
Trying a plain import and then using the functionality results in an AttributeError:
import requests
res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "/Users/me/dev/rough/requests.py", line 1, in <module>
import requests
File "/Users/me/dev/rough/requests.py", line 3, in <module>
requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'
In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import).
Using from-import of a specific name results in an ImportError:
from requests import get
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import get
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests import get
ImportError: cannot import name 'get'
In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py).
Using from-import for a module inside the package results in a different ImportError:
from requests.auth import AuthBase
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests.auth import AuthBase
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package
Using a star-import and then using the functionality raises a NameError:
from requests import *
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import *
File "/Users/me/dev/rough/requests.py", line 3, in <module>
res = get('http://www.google.ca')
NameError: name 'get' is not defined
This happens because your local module named requests.py shadows the installed requests module you are trying to use. The current directory is prepended to sys.path, so the local name takes precedence over the installed name.
An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:
Notice the name you used in your script:
File "/Users/me/dev/rough/requests.py", line 1, in <module>
The module you are trying to import: requests
Rename your module to something else to avoid the name collision.
Python may generate a requests.pyc file next to your requests.py file (in the __pycache__ directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc file in __pycache__ should not affect your code if the py file has been removed.
In the example, renaming the file to my_requests.py, removing requests.pyc, and running again successfully prints <Response [200]>.
The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.
For example: suppose that you have a script mydecimal.py that includes import decimal, intending to use the standard library decimal library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal. However, it so happens that decimal imports numbers (another standard library module) for internal use, so a script called numbers.py in your project would cause the problem.
In one especially pernicious case, having a file named token.py in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)
The traceback tells us all we need to know: calling help triggers a deferred import of the standard library pydoc, which indirectly attempts to import the standard library token, but finds our token.py which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize would do a star-import from token, and then its top-level code would try to use a name defined there, resulting in NameError - and a stack trace not mentioning the file name token.py.
If you still encounter problems like this after tracking own and renaming or removing the appropriate .py files in your project, also check for .pyc files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).
When starting the interactive Python interpreter with default
settings, is there any module implicitly imported/loaded into the
interpreter, without explicitly running import <modulename>?
I thought that modules like sys or builtins would be, but when I
type their module names,
>>> sys
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'sys' is not defined
>>> builtins
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'builtins' is not defined
So is it correct that by default, there is no module imported/loaded implicitly?
When executing a python script, is there any module implicitly
imported/loaded into the script, without explicitly specifying
import <modulename> in the script?
Thanks.
One module that is usually imported automatically is site.py. And it imports a lot of other modules. But even if you prevent it from importing using option -S Python still imports many modules. Try the following script:
#! /usr/bin/python2.7 -ESs
import sys
print(sys.modules)
and see how many modules are there. Change shebang to
#! /usr/bin/python3 -EISs
and say "Wow!" :-)
Only __builtins__:
#! /usr/bin/python2.7 -ESs
print(dir())
=> ['__builtins__', '__doc__', '__file__', '__name__', '__package__']
This question already has answers here:
How do I find the location of Python module sources?
(20 answers)
Closed 5 years ago.
I apparently have multiple versions of a module installed and I am trying to figure out where the files are because only one of them is from package management and I want to delete the other.
Is there a simple way to ask Python where it found a module after importing it?
if the module isn't built-in, you can do this:
import your_module
print(your_module.__file__)
test:
>>> import os
>>> print(os.__file__)
L:\Python34\lib\os.py
if module is built-in, you get an error:
>>> import sys
>>> print(sys.__file__)
Traceback (most recent call last):
File "<string>", line 301, in runcode
File "<interactive input>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'
>>>
(check if module has the __file__ attribute using hasattr is also an option to avoid errors; if hasattr(module_name, '__file__'):)
also: by directly printing the module:
>>> print(os)
<module 'os' from 'L:\\Python34\\lib\\os.py'>
You can use __file__ after importing the module:
>>> import os
>>> os.__file__
'/usr/lib/python2.7/os.pyc'
I am trying to import the importlib module but I get this message:
>>> importlib
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'importlib' is not defined
I have manually located the importlib file within the Python folder, so I know I have it. I am running Python version 3.4.1. I also tried to pip install importlib but it wasn't found either.
What's going on?
You need to use an import statement to load it in:
>>> import importlib
>>> importlib
<module 'importlib' from 'C:\\Python33\\lib\\importlib\\__init__.py'>