I'm trying to use sphinx to generate documentation for a package that I wrote. Everything was working great until I added some code that used another package I wrote that wraps some boost-python modules.
Now when I run make html I get this (edited for clarity):
$ make html
sphinx-build -b html -d build/doctrees source build/html
Running Sphinx v1.3.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] my_project
/path/to/my_project/my_project/docs/source/my_project.rst:19: WARNING: autodoc: failed to import module u'my_project.my_module'; the following exception was raised:
Traceback (most recent call last):
File "/Users/harding/anaconda/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg/sphinx/ext/autodoc.py", line 385, in import_object
__import__(self.modname)
File "/path/to/my_project/__init__.py", line 12, in <module>
from . import module_that_uses_boost_python_classes
File "/path/to/module_that_uses_boost_python_classes.py", line 10, in <module>
import package_that_wraps_boost_python_classes
File "/path/to/package_that_wraps_boost_python_classes/__init__.py", line 21, in <module>
from native_boost_python_module import *
ImportError: dlopen(/path/to/native_boost_python_module.so, 2): Library not loaded: cpp_library.dylib
Referenced from: /path/to/native_boost_python_module.so
Reason: image not found
The problem is that when sphinx trys to import my module, it fails to load the linked c++ library from my boost-python related package. That package is installed via pip and it loads fine in all other situations.
I manually specified the full path to all the c++ libraries with install_name_tool -change and then sphinx works fine.
My question is this: how can I get sphinx to import my boost-python package without manually editing the c++ libraries manually with install_name_tool?
I have DYLD_LIBRARY_PATH set to include the directory that contains all the c++ libraries.
I'm on MacOS Sierra
Related
I'm building a c++ python extension. So far, I created versions for Linux and Windows. Currently, I'm struggling with the MacOS version. CMake produces 2 libraries:
43898860 Aug 29 13:40 libslide_io.dylib
214876 Aug 29 13:40 slideio.cpython-35m-darwin.so
I pack them to a whl file. After installation with pip, when I try to import the package, I'm getting the following error:
(sld-35)dist % python -c "import slideio"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: dlopen(/Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/slideio.cpython-35m-darwin.so, 2): Library not loaded: libslide_io.dylib
Referenced from: /Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/slideio.cpython-35m-darwin.so
Reason: image not found
Both libraries are correctly placed to the python environment directory:
/Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/
Moreover, if I unzip the whl file and import the package from the current directory, it works.
I suspect that it is something with rpath settings in my cmake files, but cannot find the correct solution. All my attempts are failed.
I would appreciate any help.
If somebody is interested. I solved the problem by adding a post-build with the execution of install_name utility with the #loader_path parameter. The command instructs the system looking for the library at the loader (in my case - the python package) folder.
if (APPLE)
add_custom_command(TARGET ${BINDLIB_NAME}
POST_BUILD COMMAND
${CMAKE_INSTALL_NAME_TOOL} -change libslide_io.dylib #loader_path/libslide_io.dylib
$<TARGET_FILE:${BINDLIB_NAME}>)
endif()
I got a project which consists of Python and C++. I don't understand all of it but in order to compile and run everything I run an included BAT file. I have already installed the dependencies needed. Now when I run the script, I get this:
MainProcess - [INFO] os_utils: Disabling idle sleep not supported on this OS version.
world - [ERROR] launchables.world: Process Capture crashed with trace:
Traceback (most recent call last):
File "C:\work\pupil\pupil_src\launchables\world.py", line 118, in world
from plugin_manager import Plugin_Manager
File "C:\work\pupil\pupil_src\shared_modules\plugin_manager.py", line 15, in <module>
from video_capture import Base_Manager, Base_Source
File "C:\work\pupil\pupil_src\shared_modules\video_capture\__init__.py", line 36, in <module>
from .file_backend import FileCaptureError, FileSeekError
File "C:\work\pupil\pupil_src\shared_modules\video_capture\file_backend.py", line 13, in <module>
import av
File "C:\Users\XXX\AppData\Local\Programs\Python\Python36\lib\site-packages\av\__init__.py", line 9, in <module>
from av._core import time_base, pyav_version as __version__
ImportError: DLL load failed: The specified module could not be found.
It couldn't find pyav? But if I run:
C:\Users\XXX\Downloads>pip install av-0.3.1-cp36-cp36m-win_amd64.whl
Requirement already satisfied: av==0.3.1 from file:///C:/Users/XXX/Downloads/av-0.3.1-cp36-cp36m-win_amd64.whl in c:\users\anton\appdata\local\programs\python\python36\lib\site-packages
I already have it installed. What am doing wrong here?
If I open ...site-packages\av__init__.py" I can see this:
from av._core import time_base, pyav_version as __version__
is it something here?
I suspect that the module has successfully installed, but is dynamically linked against FFMPEG. You can obtain built distributions of FFMPEG from their website (https://www.ffmpeg.org/download.html). They provide both statically linked and dynamically linked builds, though it is going to be the dynamically linked one that is providing the dll's you need. Looking at my own copy of PyAV, it seems that the current release version (3.4.2) is the one that it is linked against.
When you download it, it will have a name like ffmpeg-date-build-win64-shared. In the bin directory, you will find all the relevant DLLs. You can either add this directory to the PATH, or more easily, copy the DLLs to your python location.
I have installed Python 3.4 on Mac for developmental purposes. So when I use which python on my terminal I get the below
/usr/bin/python
And when I do which python3.4, I get
/usr/local/bin/python3.4.
Now I have installed Sphinx1.3b2 and on doing which sphinx-build, i get
/usr/local/bin/sphinx-build
The problems are
I have installed many packages for Python3.4 like elasticsearch, Jinja2 etc. which are not in Python2.7 (my system's python)
Make html always uses python 2.7 and I get the following error when i try to make the html for my project;
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v1.3b2
making output directory...
loading pickled environment... not yet created
loading intersphinx inventory from http://docs.python.org /objects.inv...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 3 source files that are out of date
updating environment: 3 added, 0 changed, 0 removed
reading sources... [100%] index
/Users/sourav/Catalobo/catalobo-webapp/SphinxDocs/Code.rst:4: WARNING: autodoc: failed to import module u'catalobo'; the following exception was raised:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/Sphinx-1.3b2-py2.7.egg/sphinx/ext/autodoc.py", line 378, in import_object
__import__(self.modname)
File "/Users/sourav/Catalobo/catalobo-webapp/SphinxDocs/catalobo.py", line 4, in <module>
from elasticsearch import Elasticsearch
ImportError: No module named elasticsearch
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 1 warning.
Build finished. The HTML pages are in _build/html.
As you can see the sphinx-build is not able to load the python 3.4 packages.
Can anyone help me in resolving this?
FYI, I have already tried changing the sphinx-build file with the below;
#!/usr/bin/python3.4 and /usr/local/bin/python3.4. none works
Here is how I resolved it;
which sphinx-build
/usr/local/bin/sphinx-build
Now did a vim /usr/local/bin/sphinx-build (or use any text editor) and the below is what I changed.
It was earlier Sphinx==1.3b2 and python2.7
Changed the Sphinx==1.3b2 to 1.2.3 (the installed version of the Sphinx (this was installed usin brew.)
Changed the python to python3.4
#!/usr/local/bin/python3.4
# EASY-INSTALL-ENTRY-SCRIPT: 'Sphinx==1.3b2','console_scripts','sphinx-build'
__requires__ = 'Sphinx==1.2.3'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point('Sphinx==1.2.3', 'console_scripts', 'sphinx-build')()
)
So I used pip to download filemagic:
E:\cuckoo-current\cuckoo>c:\Python27\Scripts\pip.exe install filemagic
Downloading/unpacking filemagic
Downloading filemagic-1.6.tar.gz
Running setup.py (path:c:\users\vaclav~1.vas\appdata\local\temp\pip_build_vaclav.vasenka\filemagic\setup.py) egg_info for package filemagic
Installing collected packages: filemagic
Running setup.py install for filemagic
Successfully installed filemagic
Cleaning up...
I've created test.py with content:
import magic
print "test"
And when I run the file import throws exception:
Traceback (most recent call last):
File "E:\sand\test.py", line 2, in <module>
import magic
File "C:\Python27\lib\site-packages\magic\__init__.py", line 18, in <module>
from magic.identify import Magic, MagicError
File "C:\Python27\lib\site-packages\magic\identify.py", line 16, in <module>
from magic import api
File "C:\Python27\lib\site-packages\magic\api.py", line 22, in <module>
raise ImportError('Unable to find magic library')
ImportError: Unable to find magic library
Is the filemagic corrupted ? Is this module even working ?
From the PYPI Description (emphasis mine)
A Python API for libmagic, the library behind the Unix file command
and
libmagic is the library that commonly supports the file command on
Unix system, other than Max OSX which has its own implementation. The
library handles the loading of database files that describe the magic
numbers used to identify various file types, as well as the associated
mime types. The library also handles character set detections.
You cannot use this on Windows, as you are attempting.
You can attempt to use the mimetypes module in the standard library (mimetypes.guess_type would be what to look at).
I faced the same problem with docker containers with Alpine Linux image:
It seems like in Alpine Linux the library search doesn't work, hence it fails to find 'libmagic.so.1' which is required for the working of magic library( dependency filemagic and libmagic).
Workaround:
Use Python slim instead of Alpine Linux
mime, mimetypes are the other python libraries which can be used (the only downside to these two libraries is, that they find the mime type based on file extension and not file headers.
I'm trying to learn PyInstaller. I created two simple files, Test.py:
import os
and Test.spec:
anal = Analysis (["Test.py"])
pyz = PYZ(anal.pure)
exe = EXE(anal.scripts, pyz, name="Test.exe", exclude_binaries=1, debug=1)
coll = COLLECT(exe, anal.binaries, name="dist")
Then I ran:
Build.py Test.spec
This ran without any error mesages, and produced a directory dist with several files, including Test.exe. When I ran Test.exe, it failed with the output:
Found embedded PKG: C:\Documents and Settings\Rade\My Documents\Development\Test\Test.exe
Extracting binaries
manifestpath: C:\Documents and Settings\Rade\My Documents\Development\Test\Test.
exe.manifest
Error activating the context
python27.dll
Manipulating evironment
PYTHONPATH=C:/Documents and Settings/Rade/My Documents/Development/Test
importing modules from CArchive
extracted iu
extracted struct
extracted archive
Installing import hooks
outPYZ1.pyz
Running scripts
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named os
RC: -1 from Test
OK.
I'm sure I have made some stupid beginner's mistake, but what?
For simple files like this, you should use Makespec.py for creating spec's instead of writing manually. For large projects, you could use the Makespec.py's output as a template and edit them.
http://www.pyinstaller.org/export/latest/tags/1.4/doc/Manual.html#create-a-spec-file-for-your-project