My console_scripts/entry_points are not working in "develop" mode install and produce a traceback saying ImportError cannot load the module containing the entry points. Need help understanding how entry_points figure out which module to load and what paths to setup and stuff. Help fixing my error.
I have a setup.py script that has some entry points as follows
entry_points = {'console_scripts':[
'pyjampiler=pyjs.pyjampiler:Builder',
'pyjscompile=pyjs.translator:main',
'pyjsbuild=pyjs.browser:build_script',
]}
my code is organized as
workspace/
setup.py
pyjs/
src/
pyjs/
browser.py
My setup.py makes use of packages and package_dir arguments to setup() in setup.py to make sure the pyjs package gets picked up from pyjs/src/pyjs and so a regular install produces the following console scripts containing the following and it runs fine. it is able to load the module and call the entry point fine.
sys.exit(
load_entry_point('pyjs==0.8.1', 'console_scripts', 'pyjsbuild')()
)
But when I install and run it in development as "python setup.py develop", the install goes fine and I see the egg.lnk files getting created. but executing the console script causes the following errors
localhost:pyjs sarvi$ lpython/bin/pyjsbuild
Traceback (most recent call last):
File "lpython/bin/pyjsbuild", line 8, in <module>
load_entry_point('pyjs==0.8.1', 'console_scripts', 'pyjsbuild')()
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 318, in load_entry_point
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2221, in load_entry_point
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1954, in load
ImportError: No module named browser
I suspect it has something to do with sys.path and the directory structure in the development sources. How can I get this to work in "develop" mode install?
Related
I have a project which I am packaging with cx_freeze. When I try to run the resulting exe I get an ImportError: DLL load failed: The specified module could not be found.
Here is my setup.py:
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need
# fine tuning.
buildOptions = dict(build_exe='liquidation', packages=['scipy', 'numpy'], excludes=['scipy.spatial.cKDTree'],
includes=['scipy', 'numpy', 'numpy.core._methods', 'scipy.sparse.csgraph._validation',
'numpy.lib.format', 'numpy.linalg._umath_linalg', 'scipy.sparse._csparsetools',
'scipy.sparse.linalg.isolve._iterative', 'scipy.sparse.linalg.eigen.arpack._arpack',
'scipy.special._ufuncs_cxx', 'scipy.special.specfun', 'scipy.integrate._odepack',
'scipy.integrate._quadpack', 'scipy.integrate.vode', 'scipy.integrate._dop',
'scipy.integrate.lsoda', 'scipy.optimize._minpack', 'scipy.optimize._zeros',
'scipy.spatial', 'scipy.spatial.ckdtree', 'scipy.spatial.kdtree',
'scipy._distributor_init', 'numpy.core._multiarray_umath'])
base = 'Console'
executables = [
Executable('liquidation.py', base=base, targetName='liquidation.exe')
]
setup(name='liquidation',
version='2.0',
description='Program to run and test Liquidation algorithms',
options=dict(build_exe=buildOptions),
executables=executables)
When I run python setup.py build_exe it seems to complete successfully, but when I run the exe I get the following output:
Traceback (most recent call last):
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
from . import multiarray
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\multiarray.py", line 14, in <module>
from . import overrides
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\cx_Freeze\initscripts\__startup__.py", line 40, in run
module.run()
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\cx_Freeze\initscripts\Console.py", line 37, in run
exec(code, {'__name__': '__main__'})
File "liquidation.py", line 64, in <module>
from io_tools.stock_wrapper import StockWrapper
File "C:\Users\tamar\PycharmProjects\liquidation\io_tools\stock_wrapper.py", line 4, in <module>
from io_tools.io_helper import *
File "C:\Users\tamar\PycharmProjects\liquidation\io_tools\io_helper.py", line 7, in <module>
from numpy import float_
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\__init__.py", line 54, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "C:\Users\tamar\PycharmProjects\liquidation\liquidation\liquidation.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.3" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
I understand that I need to point cx_freeze to the missing DLL's (either by adding them to buildOptions.include_files or by actually copying the DLL's to the build folder) but I am at a loss as to how to know which DLL's I am missing. numpy.core._multiarray_umath is in the build folder so I don't know what it could be.
I believe I am using the latest versions of numpy (1.18.3), scipy (1.4.1) and cx-freeze (6.1). Any help would be much appreciated!
Update: I used Dependency Walker to try and see what I am missing and it listed a file by the name of LIBOPENBLAS.SVHFG5YE3RK3Z27NVFUDAPL2O3W6IMXW.GFORTRAN-WIN32.DLL which cannot be found. All of the other dependencies have been copied into the build so it makes sense that this would be the missing one. Does anyone know where I can find this file?
I am trying to build executable file of my project(Python 2.7.5) but getting this below error when trying to run the executable.
ERROR \ProjectPython\library.zip\distutils\__init__.py:14: UserWarning: The virtualenv
distutils package at %s appears to be in the same location as the system distutils?
Traceback (most recent call last):
File "main.py", line 130, in <module>
...
File "pandas\compat\__init__.pyo", line 32,in <module>
File "zipextimporter.pyo", line 82, in load_module
File "distutils\__init__.pyo", line 25, in <module>
ERROR ImportError: cannot import name dist
After searching for solutions to this issue, I got to know that this must be because of distutils in virtualenv is different from whats in Python installation Lib directory. So, I have been trying to exclude/remove distutils in virtualenv and add distutils package of my origin interpreter during executable build.
That exact solution exists for cxfreeze, but couldn't find alternate solution for py2exe. My setup.py options looks something like
distutils.core.setup(
options = {"py2exe": {
"compressed": 1,
"optimize": 1,
"bundle_files": 1,
"packages": ['psycopg2', 'lxml', 'sqlalchemy', 'openpyxl', 'pandas', 'numpy']
}}
Any help on this is much appreciated. Thanks !
Given the information you've shared, it seems that you might be going the wrong direction with excluding distutils. The error (warning) is stating that distutils package at %s appears to be in the same location as the system distutils.
Try creating a fresh virtual environment with the needed libraries and building again.
I've got a cython module that is a wrapper around a c++ library that communicates with a database. That library is called libpersistence.so. That library itself is dynamically linked with libhiredis.so.
I wrote a cython wrapper around the libpersistence.so to create a py_persistence.cpython-35m-x86_64-linux-gnu.so file to import as a module in my python scripts to use the cpp persistence layer.
However, upon importing the py_persistence module there's two issues in PyCharm:
Importing the Persistence class from the module does not get recognized by PyCharm. This also means there is no code completing
Running my tests leads to an ImportError where the libpersistence.so file shared object file can not be opened.
I use the following directory structure.
top_level/
indexing_service/
bin/
libhiredis.so.0.13
libpersistence.so
kCore/
include/
hiredis/
persistence/
query_service/
__init__
runpython.sh
py_persistence/
__init__
py_persistence.cpython-35m-x86_64-linux-gnu.so
py_persistence.pxd
py_persistence.pyx
setup.py
setup.sh
test/
test_persistence.py
The setup.sh file just specifies python3 setup.py build_ext --inplace
The setup.py file is the following:
from distutils.core import setup
from Cython.Distutils import build_ext, Extension
setup(
cmdclass = { 'build_ext' : build_ext },
ext_modules = [
Extension(
'py_persistence',
sources=['py_persistence.pyx'],
libraries=['persistence'],
language='c++',
extra_link_args=['-L../../indexing_service/bin/'],
runtime_library_dirs=['../../indexing_service/bin/'],
extra_compile_args=[
'-std=c++14',
'-I../../indexing_service/kCore/include/persistence',
'-I../../indexing_service/kCore/include/hiredis'
],
cython_directives={
'c_string_type':'unicode',
'c_string_encoding':'utf8',
},
)],
)
So when building the cython py_persistence library it links to the both the libpersistence.so and the libhiredis.so.0.13 files for dynamic linking. This works, this compiles and it builds the py_persistence.cpython-35m-x86_64-linux-gnu.so file correctly.
When I run my tests using the runpython.py helper script I wrote everything works perfectly fine.
The runpython.py explicitely adds the required directories to the runtime library path:
export PYTHONPATH=$PYTHONPATH:.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../indexing_service/bin/:.
python3 ${#:1}
Running the tests:
./runpython.sh -m unittest discover py_persistence/test -v
This works perfectly.
However getting this setup working so that I can run it from PyCharm is giving me a lot of trouble. I don't seem to be able to correctly specfify the paths. In the test_persistence.py file I get an ImportError while importing the py_persistence library.
Import code:
from py_persistence.py_persistence import Persistence
The error:
Error
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/case.py", line 58, in testPartExecutor
yield
File "/usr/lib/python3.5/unittest/case.py", line 600, in run
testMethod()
File "/usr/lib/python3.5/unittest/loader.py", line 34, in testFailure
raise self._exception
ImportError: Failed to import test module: test_persistence
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name
__import__(name)
File "/home/tom/work/dev/alexandria.works/query_service/py_persistence/test/test_persistence.py", line 3, in <module>
from py_persistence.py_persistence import Persistence
ImportError: libpersistence.so: cannot open shared object file: No such file or directory
I've already tried adding the indexing_service/bin/ folder to the path in PyCharm for the virtual environment:
Interpreter Paths in PyCharm
The run configuration used to run the tests is the following:
Run Configuration
The problems I'm experiencing are the ImportError and the fact that code completion isn't working.
I'm at a loss about what else to try here... Printing sys.path in the test_persistence.py file tells me that the indexing_service/bin/ folder is on the path.
['/home/tom/Applications/pycharm-2017.1.1/helpers/pycharm',
'/home/tom/work/dev/alexandria.works/query_service/py_persistence/test',
'/home/tom/work/dev/alexandria.works/indexing_service/bin',
'/home/tom/work/dev/alexandria.works/query_service',
'/home/tom/Applications/pycharm-2017.1.1/helpers/pycharm',
'/home/tom/work/dev/alexandria.works/query_service/dev-env/lib/python35.zip',
'/home/tom/work/dev/alexandria.works/query_service/dev-env/lib/python3.5',
'/home/tom/work/dev/alexandria.works/query_service/dev-env/lib/python3.5/plat-x86_64-linux-gnu',
'/home/tom/work/dev/alexandria.works/query_service/dev-env/lib/python3.5/lib-dynload', '/usr/lib/python3.5',
'/usr/lib/python3.5/plat-x86_64-linux-gnu',
'/home/tom/work/dev/alexandria.works/query_service/dev-env/lib/python3.5/site-packages']
Anybody know what I missed?
I am having trouble making a standalone application for Mac from a python Qt-based program, both when using cx_freeze and Py2App. (note that i did it successfully on Windows with cx_freeze).
Actually, the 'PyQt4' example provided with cx_freeze does not run properly on Mac. Is anyone able to "build" and run this example on a Mac?! I couldn't find any information on this seemingly basic problem on forums.
Indeed, let us build first:
~$ cd cx_Freeze-4.3.3/cx_Freeze/samples/PyQt4/
PyQt4$ ls
PyQt4app.py setup.py
PyQt4$ python setup.py build
running build
running build_exe
creating directory build/exe.macosx-10.5-x86_64-2.7
copying //anaconda/lib/python2.7/site-packages/cx_Freeze-4.3.3-py2.7-
macosx-10.5-x86_64.egg/cx_Freeze/bases/Console -> build/exe.macosx-10.5-x86_64-2.7/PyQt4app
TD bug fix: adding //anaconda/lib to relative path libpython2.7.dylib!
copying //anaconda/lib/libpython2.7.dylib -> build/exe.macosx-10.5-x86_64-2.7/libpython2.7.dylib
TD bug fix: adding //anaconda/lib to relative path libpython2.7.dylib!
writing zip file build/exe.macosx-10.5-x86_64-2.7/library.zip
…
copying //anaconda/lib/python2.7/lib-dynload/zlib.so -> build/exe.macosx-10.5-x86_64-2.7/zlib.so
PyQt4$
note the lines "TD bug fix…": in order to have the above run without 'file not found' error, i had to add a correction to function _GetDependentFiles in cx_freeze file freezer.py:
for i in range(len(dependentFiles)):
filei = dependentFiles[i]
if not os.path.isabs(filei):
print 'TD bug fix: adding ' + sys.prefix + '/lib to relative path ' + filei + '!'
dependentFiles[i] = os.path.join(sys.prefix,'lib',filei)
Now, when trying to run the created standalone program, I get:
PyQt4$ ls
PyQt4app.py build setup.py
PyQt4$ cd build/exe.macosx-10.5-x86_64-2.7/
exe.macosx-10.5-x86_64-2.7$ ls
PyQt4.QtCore.so _codecs_iso2022.so _struct.so libQtCore.dylib libz.1.dylib
PyQt4.QtGui.so _codecs_jp.so binascii.so libQtGui.4.dylib sip.so
PyQt4app _codecs_kr.so bz2.so libQtGui.dylib unicodedata.so
_codecs_cn.so _codecs_tw.so itertools.so libpython2.7.dylib zlib.so
_codecs_hk.so _multibytecodec.so libQtCore.4.dylib library.zip
exe.macosx-10.5-x86_64-2.7$ ./PyQt4app
Traceback (most recent call last):
File "//anaconda/lib/python2.7/site-packages/cx_Freeze-4.3.3-py2.7-macosx-10.5-x86_64.egg/cx_Freeze/initscripts/Console.py", line 27, in <module>
exec(code, m.__dict__)
File "PyQt4app.py", line 5, in <module>
File "ExtensionLoader_PyQt4_QtGui.py", line 22, in <module>
File "ExtensionLoader_PyQt4_QtGui.py", line 14, in __bootstrap__
ImportError: dlopen(/Users/thomasdeneux/cx_Freeze-4.3.3/cx_Freeze/samples/PyQt4/build/exe.macosx-10.5-x86_64-2.7/PyQt4.QtGui.so, 2): Library not loaded: #loader_path/../../../libQtGui.4.dylib
Referenced from: /Users/thomasdeneux/cx_Freeze-4.3.3/cx_Freeze/samples/PyQt4/build/exe.macosx-10.5-x86_64-2.7/PyQt4.QtGui.so
Reason: image not found
exe.macosx-10.5-x86_64-2.7$
It happens that some Qt libraries are not found because they are referenced with a complete relative path, while in the distribution all libraries were copied inside the same folder.
I have then moved to Py2App, hoping another package would not run into such problem. I don't show all the details now, only the most important lines when trying to build (this time i did not need to modify the code to get the building run) and run another simple Qt-based helloworld program:
test$ py2applet --make-setup helloworld.py
Wrote setup.py
test$ python setup.py py2app
running py2app
creating /Users/thomasdeneux/Documents/spyderworkspace/securebox/python/build/bdist.macosx-10.5-x86_64
...
Done!
test$ ./dist/helloworld.app/Contents/MacOS/helloworld
Traceback (most recent call last):
File "/Users/thomasdeneux/Documents/spyderworkspace/securebox/python/dist/helloworld .app/Contents/Resources/__boot__.py", line 351, in <module>
_run()
File "/Users/thomasdeneux/Documents/spyderworkspace/securebox/python/dist/helloworld .app/Contents/Resources/__boot__.py", line 336, in _run
exec(compile(source, path, 'exec'), globals(), globals())
File "/Users/thomasdeneux/Documents/spyderworkspace/securebox/python/dist/helloworld .app/Contents/Resources/helloworld.py", line 4, in <module>
from PyQt4.QtGui import *
File "PyQt4/QtGui.pyc", line 14, in <module>
File "PyQt4/QtGui.pyc", line 10, in __load
ImportError: dlopen(/Users/thomasdeneux/Documents/spyderworkspace/securebox/python/dist/helloworld.app/Contents/Resources/lib/python2.7/lib-dynload/PyQt4/QtGui.so, 2): Library not loaded: #loader_path/../../../libQtGui.4.dylib
Referenced from: /Users/thomasdeneux/Documents/spyderworkspace/securebox/python/dist/helloworld.app/Contents/Resources/lib/python2.7/lib-dynload/PyQt4/QtGui.so
Reason: image not found
2015-05-16 20:40:41.546 helloworld[983:707] helloworld Error
test$
Exactly the same kind of problem appears!
Thanks to anyone being able to help, whether by directly solving the issue or by advising another strategy for creating standalone programs for Mac from python/Qt projects.
I'm trying to add pretty printing for STL in my GDB on Ubuntu 14.04. Some details on the tools:
OS: Ubuntu 14.04
gdb version: 7.7
python version: 2.7.6
python3 version: 3.4.0
But after I setup exactly as what the instruction said. I still get the following errors:
Traceback (most recent call last):
File "<string>", line 3, in <module>
File "/home/jerry/myLib/gdb_stl_support/python/libstdcxx/v6/__init__.py", line 19, in <module>
from printers import register_libstdcxx_printers
ImportError: No module named 'printers'
/home/jerry/.gdbinit:6: Error in sourced command file:
Error while executing Python code.
Reading symbols from main...done.
Then I haved double checked my pretty printing installation directory. Under the directory /home/jerry/myLib/gdb_stl_support/python/libstdcxx/v6/, I can clearly see I have the printers.py file. And I also view the content of printers.py, I'm sure it also has the register_libstdcxx_printers class. Why the python interpreter is still complaining the printers module is missing? This seems really strange to me.
I just tried something myself, and luckily, now it's working. At least it can print out the map and vector content as expected. Here is what I did:
Since it's complaining that it can't find the printer.py module, then I think should probably I tell python interpreter where this file is located. So I first added this extra line to my ~/.gdbinit:
sys.path.append("/home/jerry/myLib/gdb_stl_support/python/libstdcxx/v6")
(After the line sys.path.insert(0, '/home/jerry/myLib/gdb_stl_support/python') )
Then running gdb again, I got the following error:
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "/home/jerry/myLib/gdb_stl_support/python/libstdcxx/v6/printers.py", line 1247, in register_libstdcxx_printers
gdb.printing.register_pretty_printer(obj, libstdcxx_printer)
File "/usr/share/gdb/python/gdb/printing.py", line 146, in register_pretty_printer
printer.name)
RuntimeError: pretty-printer already registered: libstdc++-v6
/home/jerry/.gdbinit:7: Error in sourced command file:
Error while executing Python code.
Given the error information, I edited the ~/.gdbinit file and commented the line register_libstdcxx_printers (None).
And then after running gdb, it works.
But I'm still wondering if directory in sys.path is searched recursively? I mean in my mind, the python interpreter should work like this: once you have added one directory to sys.path, then the subdirectory under that directory will also get searched for a module file.