I host a sphinx-doc website on readthedocs.org which depends on a private sphinx-doc extension/sphinxcontrib-package [extension_name]. On my local pc this extension is located in the subdirectory [sphinx-doc project root]/exts/sphinxcontrib-[extension name] and "referenced" in conf.py with
sys.path.append(os.path.abspath('exts/sphinxcontrib-<extension name>/sphinxcontrib'))
extensions = [
'sphinx.ext.graphviz',
'[extension name]'
]
The .py file is located in the "source" directory sphinxcontrib. The local sphinx-doc build output is as expected. However the build on readthedocs.org fails due to an
ImportError: No module named 'sphinxcontrib'
...
Extension error:
Could not import extension sphinxcontrib.p3 (exception: No module named 'sphinxcontrib')
As the extension is in beta phase i do not want to distribute it online. (If i would do so i should be able to install it as readthedocs.org pre-build from the web source i guess.) How can i handle this issue?
Related
I'm working on a python package which is based on pybind11 and build by cmake.
In this project a prebuild thirdparty library along with header files will be downloaded as a dependancy to pybind11 module.
cmake file is like this:
# unpackage.sh will download a tar file contains libvega.so and some so files libvega.so depends, and they will be unpacked to build dir
add_custom_target(
build-vega
COMMAND bash ${CMAKE_BINARY_DIR}/unpack.sh)
list(APPEND VEGA_LIBRARIES ${VEGA_INSTALL_DIR}/lib/libvega.so)
pybind11_add_module(${CMAKE_PROJECT_NAME})
add_dependencies(${CMAKE_PROJECT_NAME} build-vega)
list(APPEND SRC_FILES vegapy.cpp)
target_sources(${CMAKE_PROJECT_NAME} PRIVATE ${SRC_FILES})
target_link_libraries(${CMAKE_PROJECT_NAME} PRIVATE ${VEGA_LIBRARIES})
the setup file is based on cmake-example, only changes some names.
when install package by pip install ., vegapy.cpython-38-x86_64-linux-gnu.so will be installed to python dist-packages, but it depends on the so file inside project build dir.
root#ubuntu:/usr/local/lib/python3.8/dist-packages# ldd vegapy.cpython-38-x86_64-linux-gnu.so
libvega.so => /data/jgq/pv/build/temp.linux-x86_64-3.8/vegapy/vega/lib/libvega.so (0x00007f65a00cc000)
libexpreval.so => not found # depended by libvega.so
libmanage.so => not found # depended by libvega.so
in this way, when vegapy is imported in python, it will work, but if /data/jgq/pv/build/temp.linux-x86_64-3.8/vegapy/vega/lib is deleted, import will fail for libvega.so can not be found even I copy those so files into dist-packages.
I expect that:
libvega.so and its dependancies will be installed along with vegapy.cpython-38-x86_64-linux-gnu.so into a same dir.
vegapy should link to installed so files in dist-packages when python import vegapy package
I have an extension python module built with cmake. Module requires third party libutil.dll which linkes against it at runtime. Both module and libutil.dll package into wheel.
Are there some ways to help python find libutil.dll for module through setup.py config?
Module and .dll located in the same folder in package.
I investigated some ways but all of them were rejected:
Add a path to dll in PATH variable or some else environment variables;
Launch python from directory where dll stored
setup.py has next content:
from setuptools import setup
setup(
name ='my_ext_module',
version = '0.0.1',
classifiers = ["Programming Language :: Python :: 3"],
packages = ['my_ext_module'],
package_dir = {'my_ext_module' : 'src/module_dir'},
package_data = {'' : ['*']},
python_requires = ">=3.5"
)
The project has next structure:
setup.py
src
|---module_dir
|---__init__.py
|---ext_module.pyd
|---libutil.dll
I'm trying to use the feed-parser module for this project im working on. When I upload the files to App Engine and I run the script it comes back with the error that the there is no module named feed-parser.
So I'm wondering if and how can I install this module on App Engine, so that I can fix this error and use RSS.
Error:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~vlis-mannipulus-bot/1.391465315184045822/main.py", line 7, in <module>
import feedparser
ImportError: No module named feed parser
Development 1:
So I've tried installing the module in the lib directory i created(in this fail example i forgot the /lib at the --prefix=..). And i get PYTHONERROR as is shown in the shell. Ive done some research on python paths and the solutions i tried didn't work for me.
kevins-MacBook-Pro-2:~ KevinH$ cd /Users/KevinH/Downloads/feedparser -5.2.1
kevins-MacBook-Pro-2:feedparser-5.2.1 KevinH$ sudo python setup.py install --prefix=/Users/KevinH/Documents/Thalia\ VMbot/Thalia-VMbot/
Password:
running install
Checking .pth file support in /Users/KevinH/Documents/Thalia VMbot/Thalia-VMbot//lib/python2.7/site-packages/
/usr/bin/python -E -c pass
TEST FAILED: /Users/KevinH/Documents/Thalia VMbot/Thalia- VMbot//lib/python2.7/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/Users/KevinH/Documents/Thalia VMbot/Thalia-VMbot//lib/python2.7/site- packages/
and your PYTHONPATH environment variable currently contains:
''
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://pythonhosted.org/setuptools/easy_install.html#custom- installation-locations
Please make the appropriate changes for your system and try again.
Then i tried with the "pip" command but then i get this:
can't open file 'pip': [Errno 2] No such file or directory
According to what I have read "pip" should be a default program installed with python 2.7 and up. So to be sure i did install python3.5 and ran it with that and still get the same error. I typed this with both pythons:
kevins-MacBook-Pro-2:feedparser KevinH$ python3 pip -m install feedparse
--
Not sure if this would work, but via terminal i went to the default directory where feed parser has been installed on my system and copied it to the lib directory i made. Then I've created the config file with the following:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
Deployed it and im still getting the same error as above no module named feeedparser.
Apologies if im doing something stupidly wrong, im still in the learning process.
The App Engine documentation that explains how to add third party modules is here
In summary, you will need to add a folder, usually named 'lib', to the top level off your app, and then install feedparser into that folder using the commands described in the documentation. You will also need to create an appengine_config.py file as descibed in the documentation.
Note that not all third party packages can be uploaded to App Engine - those with C extensions are forbidden. Feedparser looks OK to me though.
EDIT: further comments based on edit "development1" to the question.
Your appengine_config.py looks good.
You "lib" folder should be your application folder, that is the same folder as your app.yaml and appengine_config.py files.
You need to install the feedparser package into the lib folder. The Google docs recommend that you do this by running the command
pip install -t lib feedparser
This command will install the feedparser package into your lib folder.
You need to install and run a version of pip that works with Python2.x - the Python3 version will create a version of feedparser that only runs under Python3.
If you can't install a pip for Python2 this question might provide the right solution, otherwise I'd suggest you ask a separate question about how to install feedparser into a custom install directory on a Mac.I don't have a Mac so I can't help with this.
Once you have feedparser installed in your lib folder, verify that your app works locally; in particular verify that it's using your lib/feedparser installation: try logging feedparser.__file__ after importing feedparser to check the location of the file being imported.
Once everything is working locally you should be able to upload to GAE.
So in summary, your appengine_config.py looks good, but you need to make sure that the right version of feedparser is installed into the lib folder in your app folder before uploading to App Engine.
I'm trying to set up Boost.Python 1.54.0 on Windows 7, python 2.7.3. I installed boost_1_54_0 and built it with Visual Studio 2010. I can build and run the quickstart and tutorial examples in the distribution (I had to add some MSVC lib dirs as library-path requirements in Jamroot, but that's OK). But when I copy the tutorial example to a different dir, not inside the boost dist dir, to make it more like a real project, bjam gives build errors. I changed the use-project boost location in Jamroot to /local/boost_1_54_0/ which is where boost wanted to install itself, and added a simple one-line boost-build.jam (based on something I found online):
boost-build C:/local/boost_1_54_0/tools/build/v2 ;
but ultimately bjam gives me this error:
% bjam
notice: no Python configured in user-config.jam
notice: will use default configuration
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:262: in find-jamfile from module project
error: Unable to load Jamfile.
error: Could not find a Jamfile in directory '/local/boost_1_54_0'.
error: Attempted to find it with pattern '[Bb]uild.jam [Jj]amfile.v2 [Jj]amfile [Jj]amfile.jam'.
error: Please consult the documentation at 'http://www.boost.org'.
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:280: in load-jamfile from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:64: in load from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:89: in load-used-projects from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:75: in load from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:145: in project.find from module project
C:/local/boost_1_54_0/tools/build/v2\build-system.jam:535: in load from module build-system
C:\local\boost_1_54_0\tools\build\v2/kernel\modules.jam:289: in import from module modules
C:\local\boost_1_54_0\tools\build\v2/kernel/bootstrap.jam:139: in boost-build from module
C:\tmp\tutorial\boost-build.jam:1: in module scope from module
What else do I need to add, or what am I doing wrong?
I ended up giving up on bjam for this, and just used SCons. A simple SConstruct was enough:
# SConstruct for building boost python tutorial example
import os
boost_python_lib = 'boost_python-vc100-gd-1_54'
boost_top = 'c:/boost'
python_top = 'c:/python27'
env=Environment(TARGET_ARCH='x86',
CCFLAGS=['/MDd', '/DEBUG', '/EHsc'],
CPPPATH=[os.path.join(boost_top,'include/boost-1_54'),
os.path.join(python_top, 'include')],
LIBPATH=[os.path.join(boost_top, 'lib/i386'),
os.path.join(python_top, 'libs')])
dll=env.SharedLibrary('hello_ext', 'hello.cpp',
LIBS=boost_python_lib)
env.InstallAs('hello_ext.pyd', dll[0])
# Copy the boost python lib into this dir so hello_ext will find it at runtime
env.Install('.', os.path.join(boost_top, 'lib/i386', '%s.dll'%boost_python_lib))
Of course you could make a real SCons Tool out of this, but that was enough to get me going. Hope it's useful to others.
I'm writing a setup.py which uses setuptools/distutils to install a python package I wrote.
It need to install two DLL files (actually a DLL file and a PYD file) into location which is available for python to load. Thought this is the DLLs directory under the installation directory on my python distribution (e.g. c:\Python27\DLLs).
I've used data_files option to install those files and all work when using pip:
data_files=[(sys.prefix + "/DLLs", ["Win32/file1.pyd", "Win32/file2.dll"])]
But using easy_install I get the following error:
error: Setup script exited with error: SandboxViolation: open('G:\\Python27\\DLLs\\file1.pyd', 'wb') {}
The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted.
So, what's the right way to install those files?
I was able to solve this issue by doing the following changes:
1. All data_files path changed to be relative
data_files=["myhome", ["Win32/file1.pyd", "Win32/file2.dll"])]
2. I try to find the location of "myhome" in the package init file so I'll be able to use them. This require some nasty code, because they are either under current Python's root directory or under an egg directory dedicated to the package. So I just look where a directory exits.
POSSIBLE_HOME_PATH = [
os.path.join(os.path.dirname(__file__), '../myhome'),
os.path.join(sys.prefix, 'myhome'),
]
for p in POSSIBLE_HOME_PATH:
myhome = p
if os.path.isdir(myhome) == False:
print "Could not find home at", myhome
else:
break
3. I then need to add this directory to the path, so my modules will be loaded from there.
sys.path.append(myhome)