Develop Reference

Cx_freeze: How do I resolve conflicts caused by multiple .dylib files having the same name

I have created a python application that I now want to make into a standalone application. I am using cx_freeze from a virtualenv, which has worked before on my machine with scripts that have less dependencies.
I am currently running into an issue where cx_freeze errors when it tries to copy dylib files that are named the same to the build folder.
I am running the command:
python3 setup.py build
My setup.py file looks like:
from cx_Freeze import setup, Executable
build_options = {'packages': ["PySimpleGUI", "matplotlib", "pandas", "descartes", "geopandas", "cities_coordinates"],
#'excludes': ["PIL", "pyproj", "setuptools"],
"zip_exclude_packages": [],}
import sys
base = 'Win32GUI' if sys.platform=='win32' else None
executables = [
Executable('main.py', base=base, targetName = 'data_visualizer')
]
setup(name='data_visualizer',
version = '0.1',
description = 'Data visualization tool',
options = {'build_exe': build_options},
executables = executables)
This is the error I am getting:
...
copying /Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/fiona/.dylibs/libproj.12.dylib -> build/exe.macosx-10.9-x86_64-3.7/libproj.12.dylib
copying /Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/fiona/.dylibs/libjson-c.2.dylib -> build/exe.macosx-10.9-x86_64-3.7/libjson-c.2.dylib
Traceback (most recent call last):
File "setup.py", line 18, in <module>
executables = executables)
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/dist.py", line 392, in setup
distutils.core.setup(**attrs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/dist.py", line 260, in run
freezer.Freeze()
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/freezer.py", line 788, in Freeze
self._WriteModules(fileName, self.finder)
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/freezer.py", line 712, in _WriteModules
relativeSource=True,
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/freezer.py", line 231, in _CopyFile
resolvedPath=dependent_file
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/freezer.py", line 231, in _CopyFile
resolvedPath=dependent_file
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/freezer.py", line 173, in _CopyFile
sourcePath=normalizedSource, targetPath=normalizedTarget
File "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/cx_Freeze/darwintools.py", line 446, in getDarwinFile
raise DarwinException(exceptionString)
cx_Freeze.darwintools.DarwinException: Attempting to copy two files to "build/exe.macosx-10.9-x86_64-3.7/libwebp.7.dylib"
source 1: "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/PIL/.dylibs/libwebp.7.dylib" (real: "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/PIL/.dylibs/libwebp.7.dylib")
source 2: "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/fiona/.dylibs/libwebp.7.dylib" (real: "/Users/AidenTheJaunty/projects/test/lib/python3.7/site-packages/fiona/.dylibs/libwebp.7.dylib")
(This may be caused by including modules in the zip file that rely on binary libraries with the same name.)
If I exclude libraries that introduce a dylib that conflicts with another (see commented out excludes build option in the setup.py) then the command will run successfully, but my built application will fail because of missing dependencies.
Does anyone know of a way to get around this issue?
Some other info if it matters ->
Operating System: Mac OS Big Sur
Python Version: 3.7.8
cx_freeze version: 6.2

This same name error appears to have been reported in this cx_Freeze issue, introduced by this pull request that raises an error when copying a second file onto the path of another.
A workaround was merged to print a warning and simply use the first copy of the file rather than raising an error. As it doesn't appear to have been released yet (as of 2021-03-09, v6.5.3), I installed cx_Freeze from Git (after cloning and entering the repo, I installed it by pip install -e .), ran cxfreeze, and saw warnings such as this:
*** WARNING ***
Attempting to copy two files to '../distcx/libopenblas.0.dylib'
source 1: '/venv/lib/python3.6/site-packages/numpy/.dylibs/libopenblas.0.dylib' (real: '/venv/lib/python3.6/site-packages/numpy/.dylibs/libopenblas.0.dylib')
source 2: '/venv/lib/python3.6/site-packages/scipy/.dylibs/libopenblas.0.dylib' (real: '/venv/lib/python3.6/site-packages/scipy/.dylibs/libopenblas.0.dylib')
(This may be caused by including modules in the zip file that rely on binary libraries with the same name.)
Using only source 1.
where the conflicting file was simply skipped, allowing the build to proceed.

Value error '/anaconda3/lib/libpython3.7.dylib' does not exist?

so I've created a virtual environment in my iterm2 window. this is the ques I asked earlier which has not been solved yet and my app is not running is alias modeso i tried to do the whole process again but it was showing error with plistlib so that was fixed by this answer I tried this whole process again which means I did this all this before but my app was not working so i deleted my py2app and some venv's got deleted to I downloaded miniconda3 again and then it was working until it comes to running the app even in alias mode . but I continued and deleted the dist and build and when I executed
python setup.py py2app but then i got this error
Traceback (most recent call last):
File "setup.py", line 18, in <module>
setup_requires=['py2app'],
File "/Users/divyabansal/divya/lib/python3.7/site-packages/setuptools/__init__.py", line
145, in setup
return distutils.core.setup(**attrs)
File "/Users/divyabansal/miniconda3/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/Users/divyabansal/miniconda3/lib/python3.7/distutils/dist.py", line 966, in
run_commands
self.run_command(cmd)
File "/Users/divyabansal/miniconda3/lib/python3.7/distutils/dist.py", line 985, in
run_command
cmd_obj.run()
File "/Users/divyabansal/divya/lib/python3.7/site-packages/py2app/build_app.py", line 756,
in run
self._run()
File "/Users/divyabansal/divya/lib/python3.7/site-packages/py2app/build_app.py", line 966,
in _run
self.run_normal()
File "/Users/divyabansal/divya/lib/python3.7/site-packages/py2app/build_app.py", line 1074,
in run_normal
self.create_binaries(py_files, pkgdirs, extensions, loader_files)
File "/Users/divyabansal/divya/lib/python3.7/site-packages/py2app/build_app.py", line 1384,
in create_binaries
mm.mm.run_file(runtime)
File "/Users/divyabansal/divya/lib/python3.7/site-packages/macholib/MachOGraph.py", line 89,
in run_file
raise ValueError("%r does not exist" % (pathname,))
ValueError: '/Users/divyabansal/miniconda3/lib/libpython3.7.dylib'
does not exist
please help me with this problem
Thank You!

This problem can often be fixed by creating a symlink, see here for more.
ln -s /path/to/lib/libpython3.7m.dylib /path/to/lib/libpython3.7.dylib
For the above command replace it with your own path.

Py2Exe - Addressing NumPy and SciPy dependencies

Objective (End-Goal):
I want to create a stand-alone executable Python script (only one file) that includes NumPy and SciPy dependencies for my application.
Background:
From my understanding, to create an executable script in Python - there are three options that are available:
PyInstaller
Py2exe
CxFreeze
I went ahead and tried Py2exe for my development. It appears that CxFreeze does not support the single-file option (from the documentation here). I also considered the option of using PyInstaller, but ran into issues regarding missing DLLs (similar to what is found here). The issue continued to persist even after installing the Microsoft Visual C++ 2010 Redistributable Package in my laptop.
I followed the tutorial to use Py2exe here and was able to get a dummy script executable (Hello World!) working. However, I tried to re-modify the setup.py script specific to my application to include numpy and scipy dependencies (see below):
from distutils.core import setup
import py2exe,sys,numpy,scipy
sys.argv.append('py2exe')
setup(
console=['Application.py'],
options={
'py2exe': {
'includes':['numpy','scipy','scipy.integrate','scipy.special.*','scipy.linalg.*'],
'bundle_files':1,
'compressed':True
}
},
zipfile=None)
This is the resulting error I received when I tried running the script:
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\pydev_run_in_console.py", line 52, in run_file
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/prest/PycharmProjects/Application/setup.py", line 15, in <module>
zipfile=None
File "C:\Python34\lib\distutils\core.py", line 149, in setup
dist.run_commands()
File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\prest\PycharmProjects\Application\venv\lib\site-packages\py2exe\distutils_buildexe.py", line 188, in run
self._run()
File "C:\Users\prest\PycharmProjects\Application\venv\lib\site-packages\py2exe\distutils_buildexe.py", line 267, in _run
builder.analyze()
File "C:\Users\prest\PycharmProjects\Application\venv\lib\site-packages\py2exe\runtime.py", line 164, in analyze
mf.import_hook(modname)
File "C:\Users\prest\PycharmProjects\Application\venv\lib\site-packages\py2exe\mf3.py", line 120, in import_hook
module = self._gcd_import(name)
File "C:\Users\prest\PycharmProjects\Application\venv\lib\site-packages\py2exe\mf3.py", line 274, in _gcd_import
return self._find_and_load(name)
File "C:\Users\prest\PycharmProjects\Application\venv\lib\site-packages\py2exe\mf3.py", line 337, in _find_and_load
raise ImportError(name)
ImportError: scipy.linalg.*
These are the versions that I am using relevant for my application:
Python 3.4
NumPy 1.14.5
SciPy 1.1.0
Question:
Can anyone provide any insight as to why I am receiving this error and any next steps to address this? I appreciate any input!
Thanks,
Preston

Closing - went ahead and use PyInstaller for the single-file executable. I re-modified my script to address specific dependencies (only used NumPy).

What changed in Heroku-16 stack that made GDAL uninstallable in Python?

I am trying out Heroku-16 stack and it seem's GDAL upgrade may need changes to the Python buildpack.
On cedar-14 i was using cyberdelia/heroku-geo-buildpack and heroku's own python buildpack.
Same combination used on Heroku-16 — with the only change in gdal version change in requirements to gdal==2.1.3 — results in a build error: No such file or directory: 'gdal-config'
Do I need to change something in the way I build heroku slug or does build pack's need to be upgraded.
Any tips greatly appreciated.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-w_oq7nbx/gdal/setup.py", line 339, in <module>
**extra )
File "/app/.heroku/python/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/app/.heroku/python/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/app/.heroku/python/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/app/.heroku/python/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 280, in run
self.find_sources()
File "/app/.heroku/python/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 295, in find_sources
mm.run()
File "/app/.heroku/python/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 526, in run
self.add_defaults()
File "/app/.heroku/python/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 562, in add_defaults
sdist.add_defaults(self)
File "/app/.heroku/python/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
self._add_defaults_ext()
File "/app/.heroku/python/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/app/.heroku/python/lib/python3.6/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/app/.heroku/python/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/tmp/pip-build-w_oq7nbx/gdal/setup.py", line 214, in finalize_options
self.gdaldir = self.get_gdal_config('prefix')
File "/tmp/pip-build-w_oq7nbx/gdal/setup.py", line 188, in get_gdal_config
return fetch_config(option)
File "/tmp/pip-build-w_oq7nbx/gdal/setup.py", line 135, in fetch_config
raise gdal_config_error(e)
__main__.gdal_config_error: [Errno 2] No such file or directory: 'gdal-config'

The Python buildpack way: 1.11.x works when using the cedar-14 stack only. If you set the environment variable: BUILD_WITH_GEO_LIBRARIES=1 the Python buildpack knows to download GDAL, GEOS, and PROJ. GeoDjango seems to be able to find the libraries just fine in this case. This method downloads the libraries and binaries from an S3 bucket, which contains GDAL version 1.11.5.
The custom buildpack way: One of the more popular geo-enabled buildpacks is https://github.com/cyberdelia/heroku-geo-buildpack. On the cedar-14 stack it works out of the box (remove the BUILD_WITH_GEO_LIBRARIES, purge your Heroku builder cache, redeploy).
On the heroku-16 stack, it doesn't correctly set up GDAL (explained below) nor environment variables for Django as of a6f8e24. Instead, you can use the fix (hopefully it will be merged eventually) in the buildpack at https://github.com/dschep/heroku-geo-buildpack. You'll need to set GDAL_LIBRARY_PATH and GEOS_LIBRARY_PATH in your settings.py (you can use os.getenv('GDAL_LIBRARY_PATH')).
Extended reading:
When running the Python buildpack (or a6f8e24 commit of the cyberdelia/heroku-geo-buildpack buildpack) on the heroku-16 stack, libjasper.so is missing, so the GDAL library will not successfully load. Moreover, the $PATH variable in the Python buildpack is missing the vendor binaries, so gdal-config won't work out of the box.
Appendix A - geo library versions installed:
heroku-16: Using geos version: 3.6.1, Using gdal version: 2.1.3, Using proj version: 4.9.3
cedar-14: Using geos version: 3.4.2, Using gdal version: 1.11.1, Using proj version: 4.8.0_1

How to install pybrain

I am attempting to install pybrains but I am getting this error:
C:\Python34\python.exe "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.0.4\helpers\pycharm\pycharm_setup_runner.py" C:\Users\PycharmProjects\Youtube\pybrain-pybrain-87c7ac3\setup.py
Testing started at 14:08 ...
running pycharm_test
running egg_info
writing PyBrain.egg-info\PKG-INFO
writing dependency_links to PyBrain.egg-info\dependency_links.txt
writing top-level names to PyBrain.egg-info\top_level.txt
reading manifest file 'PyBrain.egg-info\SOURCES.txt'
writing manifest file 'PyBrain.egg-info\SOURCES.txt'
running build_ext
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.0.4\helpers\pycharm\pycharm_setup_runner.py", line 26, in <module>
exec (fh.read(), globals(), locals())
File "<string>", line 21, in <module>
File "C:\PYTHON34\LIB\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\PYTHON34\LIB\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\PYTHON34\LIB\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Python34\lib\site-packages\setuptools\command\test.py", line 138, in run
self.with_project_on_sys_path(self.run_tests)
File "C:\Python34\lib\site-packages\setuptools\command\test.py", line 118, in with_project_on_sys_path
func()
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.0.4\helpers\pycharm\pycharm_commands\pycharm_test.py", line 18, in run_tests
testLoader=loader_class()
File "C:\PYTHON34\LIB\unittest\main.py", line 92, in __init__
self.parseArgs(argv)
File "C:\PYTHON34\LIB\unittest\main.py", line 139, in parseArgs
self.createTests()
File "C:\PYTHON34\LIB\unittest\main.py", line 146, in createTests
self.module)
File "C:\PYTHON34\LIB\unittest\loader.py", line 146, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "C:\PYTHON34\LIB\unittest\loader.py", line 146, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "C:\PYTHON34\LIB\unittest\loader.py", line 105, in loadTestsFromName
module = __import__('.'.join(parts_copy))
File "c:\users\dhowarth\pycharmprojects\youtube\pybrain-pybrain-87c7ac3\pybrain\__init__.py", line 1, in <module>
from structure.__init__ import *
ImportError: No module named 'structure'
Process finished with exit code 1
Where do I find the 'Structure' module?
Anyone having the same issue?
I have followed the instructions on: https://github.com/pybrain/pybrain/wiki/installation.
what am I missing?

The Pybrain can be setup by pip install pybrain directly.
The FNN in pybrain have a bit problem of convergence and performance. As substitution, there are lots of advices on Python for AI, such as FANN (An example) and scikit-learn (for lots of ML method but not contains FNN directly).

Try not to use pip. I always prefer getting the entire folder from PyPI or github (if available) and then doing the simple python setup.py install.
Also, whenever you do some work with the packages which are under active development (or not very commonly use), please read their documentation.
If you see the documentation : Here. You will see that they use Python 2.x.
And you are clearly using Python 3.x.
Get Python 2.x and download the folder from github. Unzip it and install.
Hope it helps.
Cheers.

Try following queries:
pip install numpy
pip install scipy
pip install pybrain
OR:
download ez_setup.py and run it:
python ez_setup.py
OR:
use easy_install:
easy_install pybrain