My question is specific to scikit-learn python module, but I had similar issues with matplotlib as well.
When I want to use sklearn, if I just do 'import sklearn' and then call whatever submodule I need, like ' sklearn.preprocessing.scale()', I get an error
"AttributeError: 'module' object has no attribute 'preprocessing'"
On the other hand, when I do 'from sklearn import preprocessing'
and then use 'preprocessing.scale()' it works normally.
When I use other modules like Numpy, it is sufficient to just 'import numpy' and it works well.
Therefore, I would like to ask if anyone can tell me why is this happening and if I am doing something wrong?
Thanks.
A python package is defined in the __init__.py file inside the directory.
This file determines whether or not submodules are include.
When you do import sklearn python finds the file sklearn/__init__.py and executes it to create the sklearn module. This object is the bound to the name sklearn. Submodules are not implicitly imported by the interpreter.
However when doing from sklearn import preprocessing python will first load the sklearn module as before. Then it will check if preprocessing is an attribute of that module (e.g. a function), and if not it will look for the file sklearn/preprocessing.py and improt that module too.
It happens that numpy does something like the following in its __init__.py file:
import .random
Thus when importing numpy as import numpy the execution of that module triggers the importing of numpy.random which is then added as an attribute.
This is useful because sometimes you want to use only part of a package and loading all of it could take a significant amount of time. For example importing numpy does take something like half a second. This is time wasted if you only need a very small subset of its functionality.
You may be interested in reading the documentation for packages.
Numpy conveniently imports its submodules in its __init__.py file and adds them to __all__. There's not much you can do about it when using a library - it either does it or not. sklearn apparently doesn't.
Related
I have a project which worked perfectly fine, but I wanted to separate some of the code into a separate project (with its own packages). I have successfully done that, but now in that newly separated package I need to explicitly import local modules using from . import <module> instead of import <module> - the latter of which had worked before separating the modules into their own package.
The new imports (where I include the dot representing the local dir) works but it seems like it should also just work without having to specify the module locations since the files are in the same directory as the module that is trying to import them. Am I missing something here?
Artificial example:
|-root
|--ProjectA
|---app.py
|-root
|--ProjectB
|---ProjectB
|----__init__.py
|----libA.py
|----libB.py
In app.py I am able to import a module from ProjectB (I have adjusted PYTHONPATH to include /root/ProjectB). So the following works just fine as I would expect:
from ProjectB import libA.py
But the entire thing fails because in libA.py the following does not work:
import libB
Instead I have to use:
from . import libB
I have tried to wrap my head around how python imports work across packages and I thought I had sorted it in my head, but apparently I am missing something. Unless this is the way it is supposed to work? (packages need to explicitly indicate that a module is local). But I thought python would always implicitly add the path of the current module that is doing the importing.
Any advice (or even just links to tutorials that explain this - I have watched a lot but they all tend to focus on different aspects of importing than this) is greatly appreciated.
I was trying to some code in python and noted this peculiar case.
import importlib
print(importlib.abc)
The above code runs fine in python 3.7 but not in python 3.10.2. In python 3.10.2, i get the following error:
AttributeError: module 'importlib' has no attribute 'abc'. Did you mean: '_abc'?
I tried to look at the documentation as to whether the behaviour of the above code got changed somehow. But couldn't find it. The documentations i tried looking are Python 3.7 importlib documentation and Python 3.10.2 importlib documentation.
So i wish to know how to find things imported by an import statement on a python standard library through documentation.
Note: I do know that i can look at dir(importlib) to see everything importlib has after it is imported. But i want to find it through documentation so i can know them without using that specific python version.
If you want to import a submodule of a package, you should do so explicitly:
import importlib.abc
Python will not automatically load all submodules of a package when you import the package. Whether submodules will be available without an explicit submodule import depends on whether some code somewhere else in the program has already loaded the module.
As for the question in the title, documentation will usually not have the information you're asking for. Good documentation will thoroughly describe a module's entire public API, but you're asking for information beyond that.
You could use python dir() function to know what importlib contains
import importlib
dir(importlib)
Or a slightly different method is hasattr(importlib,'abc') which returns a boolean.
Yet again another way is via the inspect module
import importlib
import inspect
inspect.getmembers(importlib) #Show all methods,attribute and ...
inspect.getmembers(importlib, inspect.ismodule) #Show only modules
iPython Notebook
Python 2
Complaining about this line:
from sklearn.model_selection import train_test_split
Why isn't model selection working?
In order to remedy this issue, you need to first find out if you are importing the actual sklearn package, and not just some script with the name sklearn.py saved somewhere in your working directory. The way Python imports modules is somewhat similar to the way it finds variables in its namespace (Local, Enclosed, Global, Built-in). In this case, Python will start importing module by first looking in the current directory and then the site-packages. If it looks in the current working directory and finds a python script with the same name as the module you are trying to import, then it will import that script instead of the actual module.
You can usually find out whether or not the actual module is imported by checking its __file__ or __path__ attribute:
import sklearn
print(sklearn.__file__)
print(sklearn.__path__)
Reviewing the output of those print statements will tell you whether the imported package is the module you are after, or just some script lying somewhere in your working directory. If, in case, the output does not point to the site-packages of your Python version then you have imported some script somewhere that's not the module itself. Your quick fix would be to exit the console first, rename the .py script and its compiled version (the .pyc file) and go back to your console and try again.
However, if the output points to your python version's site-packages, then there is something wrong with how the package was installed in the first place. In which case, you will probably need to update or reinstall it.
Particular to this, it turns out that the issue is with the version of sklearn you are using; because the model_selection module in sklearn is available in versions 0.18+. If you're using a version number (sklearn.__version__) lower than 0.18, then you will have to use the old cross_validation module instead of the model_selection module:
from sklearn.cross_validation import train_test_split
You can also just upgrade to the latest version of the package with your preferred package management system.
I hope this is helpful.
On my system I have two versions of Python (to call them I type python and python2 in the command line). When I use the first version of Python, I cannot import sklearn module but I can do it in the second version of Python.
I would like to use the first version of python (because other modules are available there) and, at the same time, I would like to be able to import sklearn from this version of Python.
My solution was to use:
import sys
sys.path.append('location_of_the_sklearn_module')
To find the location of the sklearn module I started a python session (using the second version of python, in which sklearn works). The I type:
import sklearn
sklearn.__file__
As a result I got:
/home/name/my_name/numpy/local/lib/python2.7/site-packages/sklearn/__init__.pyc
In the session of the first version of Python I tried:
import sys
sys.path.append('/home/name/my_name/numpy/local/lib/python2.7/site-packages/sklearn')
import sklearn
Unfortunately it did not work. As a result I got: ImportError: No module named sklearn
Does anybody know what I am doing wrong and if it is possible to reach the goal in the way I try?
When importing packages, you need to add the parent directory of the package to PYTHONPATH, not the package directory itself, so just change...
sys.path.append('/home/name/my_name/numpy/local/lib/python2.7/site-packages/sklearn')
...to...
sys.path.append('/home/name/my_name/numpy/local/lib/python2.7/site-packages')
...although it may not necessarily import correctly in Python 3.x.
I am new to python and I am running into some troubles importing modules. I have very limited prior knowledge to programming (bit of Fortran and C) and I usually work with Matlab.
I searched the web for import methods and the preferred method according to most tutorials is: import X
If I use the console (iPython respectively) I can execute the following:
import scipy
scipy.stats.cauchy.pdf
However when I use the same line in a function it does not work, I have to use
import scipy.stats as s
s.stats.cauchy.pdf
or I get: AttributeError: 'module' object has no attribute 'stats'
By reading this 'import module' or 'from module import' I got the impression that both should work.
If someone could quickly clear that up for me I would be very thankful!
Best Wishes,
Chris
It should not happen on a newly opened ipython instance. You would not normally be able to access like this:
import scipy
scipy.stats.cauchy.pdf
Without getting AttributeError, in either ipython or regular python interpreter. You would need to do
import scipy.stats
somewhere, because it is a submodule.
Either you have an earlier imported scipy.stats hanging around in your ipython namespace, or you have some automatic startup script importing it when ipython starts.
Note: if you are using ipython's run magic function to execute a script, any global variables and imported modules will remain in scope.