What is the recommended way getting hold of the package version of a package in the $PYTHONPATH or sys.path?
I remember that the pkg_resource module has some functionality for this but I can not find any related information. Please don't point me to solution using a version.py file and reading it somehow. Using pkg_resources is the way to go but how exactly?
>>> import pkg_resources
>>> pkg_resources.get_distribution("PIL").version
'1.1.7'
I found it quite unreliable to use the various tools available (including the best one pkg_resources mentioned by this other answer), as most of them do not cover all cases. For example built-in modules and modules not installed but just added to the python path (by your IDE for example). Since we needed a reliable way to get the version of any package, module or submodule, I ended up writing getversion. It is quite simple to use:
from getversion import get_module_version
import foo
version, details = get_module_version(foo)
See the documentation for details.
Related
I was trying to some code in python and noted this peculiar case.
import importlib
print(importlib.abc)
The above code runs fine in python 3.7 but not in python 3.10.2. In python 3.10.2, i get the following error:
AttributeError: module 'importlib' has no attribute 'abc'. Did you mean: '_abc'?
I tried to look at the documentation as to whether the behaviour of the above code got changed somehow. But couldn't find it. The documentations i tried looking are Python 3.7 importlib documentation and Python 3.10.2 importlib documentation.
So i wish to know how to find things imported by an import statement on a python standard library through documentation.
Note: I do know that i can look at dir(importlib) to see everything importlib has after it is imported. But i want to find it through documentation so i can know them without using that specific python version.
If you want to import a submodule of a package, you should do so explicitly:
import importlib.abc
Python will not automatically load all submodules of a package when you import the package. Whether submodules will be available without an explicit submodule import depends on whether some code somewhere else in the program has already loaded the module.
As for the question in the title, documentation will usually not have the information you're asking for. Good documentation will thoroughly describe a module's entire public API, but you're asking for information beyond that.
You could use python dir() function to know what importlib contains
import importlib
dir(importlib)
Or a slightly different method is hasattr(importlib,'abc') which returns a boolean.
Yet again another way is via the inspect module
import importlib
import inspect
inspect.getmembers(importlib) #Show all methods,attribute and ...
inspect.getmembers(importlib, inspect.ismodule) #Show only modules
I'm trying to use the nuitka tool to turn my python program into executable on ubuntu. It works fine if the program doesn't have any import statements but breaks when I use it on a program that imports something e.g.
test.py
import numpy
print "hello, world."
type this on commandline
nuitka --recurse-all --python-version=2.7 test.py
and gives me these errors
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/numarray/functions.py:45: Cannot find 'copyreg' in package 'numpy.numarray' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/distutils/npy_pkg_config.py:11: Cannot find 'configparser' in package 'numpy.distutils' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/distutils/system_info.py:1765: Cannot find 'Numeric' in package 'numpy.distutils' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/distutils/system_info.py:1770: Cannot find 'numarray' in package 'numpy.distutils' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/f2py/diagnose.py:48: Cannot find 'numpy_distutils' in package 'numpy.f2py' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/f2py/diagnose.py:87: Cannot find 'numpy_distutils.command.build_flib' in package 'numpy.f2py' as absolute import.
I don't know about your particular use case but I also faced similar Cannot find '' in Package errors when using nuitka.
I was using sqlalchemy and had a similar issue with configparser.
After about a day of debugging I found out that Nuitka trips up with SWIG (Dynamically Loaded shared objects).
What it means basically is, some programs/modules try to increase compatibility by using conditional imports.
For eg:
If python_version==3.5:
import thislibrary
else:
import thatlibrary
specifically the configparser library is named configparser in python3 and ConfigParser in python2.
So what basically is happening is that nuitka is trying to import python 3 stuff when you clearly are using python 2.
For me the fix was to modify the source code of sqlalchemy and change the if else construct to:
import thatlibrary
You can find more information in this Guide
written by Tom Sheffler
Official answer from Nuitaka.
What problem you facing same like another problem one user facing package 'matplotlib' as absolute import. this problem, then there nuitaka given comments below check if it useful for below comment.
I think you might be using 32 bits Python on Windows and hit the 2GB RAM boundary. Use 64 bits for better luck. Incidentally I am working on scalability improvements for the next releases which ought to make this not happen. For now Nuitka loads all say 1000 modules into RAM and compiles them all globally. Needs a lot of RAM.
For the modules:
required_modules = ['nose', 'coverage', 'webunit', 'MySQLdb', 'pgdb', 'memcache']
and programs:
required_programs = ['psql', 'mysql', 'gpsd', 'sox', 'memcached']
Something like:
# Report on the versions of programs installed
for module in required_modules:
try:
print module.__version__
except:
exit
Unfortunately, module.__version__ isn't present in all modules.
A workaround is to use a package manager. When you install a library using easy_install or pip, it keeps a record of the installed version. Then you can do:
import pkg_resources
version = pkg_resources.get_distribution("nose").version
I found it quite unreliable to use the various tools available (including the best one pkg_resources mentioned by moraes' answer), as most of them do not cover all cases. For example
built-in modules
modules not installed but just added to the python path (by your IDE for example)
two versions of the same module available (one in python path superseding the one installed)
Since we needed a reliable way to get the version of any package, module or submodule, I ended up writing getversion. It is quite simple to use:
from getversion import get_module_version
import foo
version, details = get_module_version(foo)
See the documentation for details.
I have a python script that used to run, although since moving servers at work it now throws up a strange error:
>>> import _md5
ImportError: No module named _md5
The general setup is all correct, as is my python path and seemingly everything else. I was told that I need to install the relevant RPM for this to work, but have no idea what this might be - could anyone please point me in the right direction?
These machines have a setup that prohibits me using yum, so I need to make a request to those maintaining the system about which RPM I want installed.
Based on extra bit of information from the OP, they use Python 2.5+ on the new server.
Suggested remedy is to use standard hashlib module. Which provides MD5 hash implementation among other things.
Install openssl-devel and rebuild.
Or better yet, build your own Python package and deploy everywhere.
Many third-party Python modules have an attribute which holds the version information for the module (usually something like module.VERSION or module.__version__), however some do not.
Particular examples of such modules are libxslt and libxml2.
I need to check that the correct version of these modules are being used at runtime. Is there a way to do this?
A potential solution wold be to read in the source at runtime, hash it, and then compare it to the hash of the known version, but that's nasty.
Is there a better solutions?
Use pkg_resources. Anything installed from PyPI at least should have a version number.
>>> import pkg_resources
>>> pkg_resources.get_distribution("blogofile").version
'0.7.1'
If you're on python >=3.8 you can use a module from the built-in library for that. To check a package's version (in this example lxml) run:
>>> from importlib.metadata import version
>>> version('lxml')
'4.3.1'
This functionality has been ported to older versions of python (<3.8) as well, but you need to install a separate library first:
pip install importlib_metadata
and then to check a package's version (in this example lxml) run:
>>> from importlib_metadata import version
>>> version('lxml')
'4.3.1'
Keep in mind that this works only for packages installed from PyPI. Also, you must pass a package name as an argument to the version method, rather than a module name that this package provides (although they're usually the same).
I'd stay away from hashing. The version of libxslt being used might contain some type of patch that doesn't effect your use of it.
As an alternative, I'd like to suggest that you don't check at run time (don't know if that's a hard requirement or not). For the python stuff I write that has external dependencies (3rd party libraries), I write a script that users can run to check their python install to see if the appropriate versions of modules are installed.
For the modules that don't have a defined 'version' attribute, you can inspect the interfaces it contains (classes and methods) and see if they match the interface they expect. Then in the actual code that you're working on, assume that the 3rd party modules have the interface you expect.
Some ideas:
Try checking for functions that exist or don't exist in your needed versions.
If there are no function differences, inspect function arguments and signatures.
If you can't figure it out from function signatures, set up some stub calls at import time and check their behavior.
I found it quite unreliable to use the various tools available (including the best one pkg_resources mentioned by this other answer), as most of them do not cover all cases. For example
built-in modules
modules not installed but just added to the python path (by your IDE for example)
two versions of the same module available (one in python path superseding the one installed)
Since we needed a reliable way to get the version of any package, module or submodule, I ended up writing getversion. It is quite simple to use:
from getversion import get_module_version
import foo
version, details = get_module_version(foo)
See the documentation for details.
You can use
pip freeze
to see the installed packages in requirements format.
For modules which do not provide __version__ the following is ugly but works:
#!/usr/bin/env python3.6
import sys
import os
import subprocess
import re
sp = subprocess.run(["pip3", "show", "numpy"], stdout=subprocess.PIPE)
ver = sp.stdout.decode('utf-8').strip().split('\n')[1]
res = re.search('^Version:\ (.*)$', ver)
print(res.group(1))
or
#!/usr/bin/env python3.7
import sys
import os
import subprocess
import re
sp = subprocess.run(["pip3", "show", "numpy"], capture_output=True)
ver = sp.stdout.decode('utf-8').strip().split('\n')[1]
res = re.search('^Version:\ (.*)$', ver)
print(res.group(1))