I am using Python3.3 and I have installed a package called python-bidi, which can be found here.
I have installed the package by running: python setup.py install, which was instructed in the package documentation.
Now I am running a code that uses that package but I get this error:
File "C:\Python33\lib\site-packages\python_bidi-0.3.4-py3.3.egg\bidi\algorithm.py", line 24, in <module>
from mirror import MIRRORED
ImportError: No module named 'mirror'
However, looking at this folder: C:\Python33\lib\site-packages\python_bidi-0.3.4-py3.3.egg\bidi I can see that there is a file called mirror.py.
What is the reason for the error that I get?
Thanks for any help on this.
In Python 2.7, you can do import bidi.algorithm, but it will fail in Python 3.3. Actually, it comes from changes in the way modules are imported. Notice the python-bidi library was last updated in 2010.
See PEP 328, which also dates back 2010. This PEP addresses the following issue: "Imports can be ambiguous in the face of packages; within a package, it's not clear whether import foo refers to a module within the package or some module outside the package. (More precisely, a local module or package can shadow another hanging directly off sys.path.)"
With the new notation, that is used in Python 3.x, you would write, in "bidi/algorithm.py", from .mirror import MIRRORED instead of from mirror import MIRRORED. Notice the dot!
The import would work, but the module bidi.algorithm would still fail to load, because of other incompatibilities. Namely, there is at some point X2_X5_MAPPINGS.keys() + ['BN', 'PDF', 'B'], which fails because dict.keys() does not return a list any longer, but a "dict_keys" object.
Therefore, you won't be able to use this package in Python 3.x, unless you are willing to do some work to resolve incompatibilities.
As a side note, the new notation from .mirror import MIRRORED is also understood by Python 2.7, so it should be possible to make python-bidi to be compatible with both Python 2.x and 3.x lines.
The only thing left is to change line 41 and 42 by adding list around X2_X5_MAPPINGS.keys() and X2_X5_MAPPINGS.keys() to become:
list(X2_X5_MAPPINGS.keys())
and
list(X2_X5_MAPPINGS.keys())
and also change line 600 from:
if isinstance(unicode_or_str, unicode)
to
if isinstance(unicode_or_str, str):
Related
I found the following line in Python code:
from six.moves import urllib
Simultaneously, I can find urllib.py anywhere. I found that there is a file six.py in package root and it has class Module_six_moves_urllib(types.ModuleType): inside.
Is this it? How is this defined?
UPDATE
Sorry I am new to Python and the question is about Python syntax. I learned, that what is after import is Python file name without a py extension. So, where is this file in this case?
six is a package that helps in writing code that is compatible with both Python 2 and Python 3.
One of the problems developers face when writing code for Python2 and 3 is that the names of several modules from the standard library have changed, even though the functionality remains the same.
The six.moves module provides those modules under a common name for both Python2 and 3 (mostly by providing the Python2 module under the name of the Python 3 module).
So your line
from six.moves import urllib
imports urllib when run with Python3 and imports a mixture of urllib, urllib2 and urlparse with Python2, mimicking the structure of Python3's urllib. See also here.
EDIT to address the update of the question:
TLDR; There is not necessarily a direct relation between the imported module urllib and a file on the filesystem in this case. The relevant file is exactly what six.__file__ points to.
Third party modules are defined in a file/directory that is
listed in sys.path. Most of the time you can find the name of the file a module is imported from by inspecting the __file__ attribute of the module in question, e.g. six.__file__. However with six.moves things are not as simple, precisely because the exposed modules might not actually map one to one to actual Python modules but hacked versions of those.
I was believing that when importing a package, it would search from sys.path and use the first hit for import. However, it seems not true:
import mpl_toolkits
print(mpl_toolkits.__path__)
And it outputs:
['/Library/Python/2.7/site-packages/matplotlib-1.5.0-py2.7-macosx-10.11-intel.egg/mpl_toolkits', '/usr/local/lib/python2.7/site-packages/mpl_toolkits']
Can someone please explain to me how exactly python looks for packages if it is installed multiple times in the machine (in different location searchable by sys.path)? Or a pointer to relevant reference would be good.
when you import a module, python uses PYTHON PATH (system variable that contains a list of folders) and loops to search for importable module.
Python will test if it is a package (folder containing init.py) or a module (*.py). it will stop on first module found if no module is found python raises an import error
The default version of python (ie, one that opens on typing "python" in command line) is 2.6 on my server. I also have 2.7 installed. How do I import a module, in this case numpy, in python2.7? When I try to import right now, it gives me an error -
ImportError: No module named numpy
Is there any workaround, apart from downloading the package and doing a build install?
I agree with the comment above. You will have to compile it separately. I once tried a hack of importing and modifying the sys.path however I ran into issues with the .so files.
I was just wondering why __import__() calls a __init__ module twice when loading a package.
test.py
testpkg/
__init__.py
test.py:
pkg = __import__("testpkg", fromlist=[''])
__init__.py:
print "Called."
After calling python test.py, Called. will be printed out twice. Why does python execute the __init__ "module" twice?
This is a Python bug. Passing the null string as an element of fromlist is illegal, and should raise an exception.
There's no need to include "" in fromlist; that's implicit--the module itself is always loaded. What's actually happening is the module.submodule string is using the null string, resulting in the module name testpkg., with a trailing period. That gets imported literally, and since it has a different name than testpkg, it's imported as a separate module.
Try this:
pkg = __import__("testpkg", fromlist=[''])
import sys
print sys["testpkg"]
print sys["testpkg."]
... and you'll see the duplicate module.
Someone should probably file a ticket on this if there isn't already one; too tired to do it myself right now.
Using the fromlist=[''] hack to import a specific module is explicitly frowned upon by python-dev. While it has been filed as an issue, the chances of it being fixed are low specifically because this is viewed as a mis-use of fromlist to begin with instead of necessarily a bug and a better solution is available.
What you should be doing is using importlib.import_module (available in the standard library for Python 2.7 and Python 3.1, or from PyPI with compatibility back to Python 2.3 along with being included in Django since 1.1 as django.utils.importlib). It will prevent this problem from happening, provides a better programmatic interface for importing modules, and even lets you use relative imports when you specify the package you are importing from.
If you really cannot use importlib (e.g., PyPI dependencies are not allowed even though the code you can freely copy thanks to the PSF license and it being rather short), then you should be doing __import__("some.module"); mod = sys.modules["some.module"]. That is the official, python-dev sanctioned solution to the problem (but only after you cannot use importlib).
Many third-party Python modules have an attribute which holds the version information for the module (usually something like module.VERSION or module.__version__), however some do not.
Particular examples of such modules are libxslt and libxml2.
I need to check that the correct version of these modules are being used at runtime. Is there a way to do this?
A potential solution wold be to read in the source at runtime, hash it, and then compare it to the hash of the known version, but that's nasty.
Is there a better solutions?
Use pkg_resources. Anything installed from PyPI at least should have a version number.
>>> import pkg_resources
>>> pkg_resources.get_distribution("blogofile").version
'0.7.1'
If you're on python >=3.8 you can use a module from the built-in library for that. To check a package's version (in this example lxml) run:
>>> from importlib.metadata import version
>>> version('lxml')
'4.3.1'
This functionality has been ported to older versions of python (<3.8) as well, but you need to install a separate library first:
pip install importlib_metadata
and then to check a package's version (in this example lxml) run:
>>> from importlib_metadata import version
>>> version('lxml')
'4.3.1'
Keep in mind that this works only for packages installed from PyPI. Also, you must pass a package name as an argument to the version method, rather than a module name that this package provides (although they're usually the same).
I'd stay away from hashing. The version of libxslt being used might contain some type of patch that doesn't effect your use of it.
As an alternative, I'd like to suggest that you don't check at run time (don't know if that's a hard requirement or not). For the python stuff I write that has external dependencies (3rd party libraries), I write a script that users can run to check their python install to see if the appropriate versions of modules are installed.
For the modules that don't have a defined 'version' attribute, you can inspect the interfaces it contains (classes and methods) and see if they match the interface they expect. Then in the actual code that you're working on, assume that the 3rd party modules have the interface you expect.
Some ideas:
Try checking for functions that exist or don't exist in your needed versions.
If there are no function differences, inspect function arguments and signatures.
If you can't figure it out from function signatures, set up some stub calls at import time and check their behavior.
I found it quite unreliable to use the various tools available (including the best one pkg_resources mentioned by this other answer), as most of them do not cover all cases. For example
built-in modules
modules not installed but just added to the python path (by your IDE for example)
two versions of the same module available (one in python path superseding the one installed)
Since we needed a reliable way to get the version of any package, module or submodule, I ended up writing getversion. It is quite simple to use:
from getversion import get_module_version
import foo
version, details = get_module_version(foo)
See the documentation for details.
You can use
pip freeze
to see the installed packages in requirements format.
For modules which do not provide __version__ the following is ugly but works:
#!/usr/bin/env python3.6
import sys
import os
import subprocess
import re
sp = subprocess.run(["pip3", "show", "numpy"], stdout=subprocess.PIPE)
ver = sp.stdout.decode('utf-8').strip().split('\n')[1]
res = re.search('^Version:\ (.*)$', ver)
print(res.group(1))
or
#!/usr/bin/env python3.7
import sys
import os
import subprocess
import re
sp = subprocess.run(["pip3", "show", "numpy"], capture_output=True)
ver = sp.stdout.decode('utf-8').strip().split('\n')[1]
res = re.search('^Version:\ (.*)$', ver)
print(res.group(1))