Can python distutils compile CUDA code? - python

I have CUDA code which I want to build a dynamic library to Python using distutils. But it seems distutils doesn't recognize ".cu" file even if the "nvcc" compiler is installed. Not sure how to get it done.

Distutils is not able to compile CUDA by default, because it doesn't support using multiple compilers simultaneously. By default, it sets to compiler just based on your platform, not on the type of source code you have.
I have an example project on github that contains some monkey patches into distutils to hack in support for this. The example project is a C++ class that manages a some GPU memory and a CUDA kernel, wrapped in swig, and all compiled with just python setup.py install. The focus is on array operations, so we're also using numpy. All the kernel does for this example project is increment each element in an array by one.
The code is here: https://github.com/rmcgibbo/npcuda-example. Here's the setup.py script. The key to whole code is customize_compiler_for_nvcc().
import os
from os.path import join as pjoin
from setuptools import setup
from distutils.extension import Extension
from distutils.command.build_ext import build_ext
import subprocess
import numpy
def find_in_path(name, path):
"Find a file in a search path"
#adapted fom http://code.activestate.com/recipes/52224-find-a-file-given-a-search-path/
for dir in path.split(os.pathsep):
binpath = pjoin(dir, name)
if os.path.exists(binpath):
return os.path.abspath(binpath)
return None
def locate_cuda():
"""Locate the CUDA environment on the system
Returns a dict with keys 'home', 'nvcc', 'include', and 'lib64'
and values giving the absolute path to each directory.
Starts by looking for the CUDAHOME env variable. If not found, everything
is based on finding 'nvcc' in the PATH.
"""
# first check if the CUDAHOME env variable is in use
if 'CUDAHOME' in os.environ:
home = os.environ['CUDAHOME']
nvcc = pjoin(home, 'bin', 'nvcc')
else:
# otherwise, search the PATH for NVCC
nvcc = find_in_path('nvcc', os.environ['PATH'])
if nvcc is None:
raise EnvironmentError('The nvcc binary could not be '
'located in your $PATH. Either add it to your path, or set $CUDAHOME')
home = os.path.dirname(os.path.dirname(nvcc))
cudaconfig = {'home':home, 'nvcc':nvcc,
'include': pjoin(home, 'include'),
'lib64': pjoin(home, 'lib64')}
for k, v in cudaconfig.iteritems():
if not os.path.exists(v):
raise EnvironmentError('The CUDA %s path could not be located in %s' % (k, v))
return cudaconfig
CUDA = locate_cuda()
# Obtain the numpy include directory. This logic works across numpy versions.
try:
numpy_include = numpy.get_include()
except AttributeError:
numpy_include = numpy.get_numpy_include()
ext = Extension('_gpuadder',
sources=['src/swig_wrap.cpp', 'src/manager.cu'],
library_dirs=[CUDA['lib64']],
libraries=['cudart'],
runtime_library_dirs=[CUDA['lib64']],
# this syntax is specific to this build system
# we're only going to use certain compiler args with nvcc and not with gcc
# the implementation of this trick is in customize_compiler() below
extra_compile_args={'gcc': [],
'nvcc': ['-arch=sm_20', '--ptxas-options=-v', '-c', '--compiler-options', "'-fPIC'"]},
include_dirs = [numpy_include, CUDA['include'], 'src'])
# check for swig
if find_in_path('swig', os.environ['PATH']):
subprocess.check_call('swig -python -c++ -o src/swig_wrap.cpp src/swig.i', shell=True)
else:
raise EnvironmentError('the swig executable was not found in your PATH')
def customize_compiler_for_nvcc(self):
"""inject deep into distutils to customize how the dispatch
to gcc/nvcc works.
If you subclass UnixCCompiler, it's not trivial to get your subclass
injected in, and still have the right customizations (i.e.
distutils.sysconfig.customize_compiler) run on it. So instead of going
the OO route, I have this. Note, it's kindof like a wierd functional
subclassing going on."""
# tell the compiler it can processes .cu
self.src_extensions.append('.cu')
# save references to the default compiler_so and _comple methods
default_compiler_so = self.compiler_so
super = self._compile
# now redefine the _compile method. This gets executed for each
# object but distutils doesn't have the ability to change compilers
# based on source extension: we add it.
def _compile(obj, src, ext, cc_args, extra_postargs, pp_opts):
if os.path.splitext(src)[1] == '.cu':
# use the cuda for .cu files
self.set_executable('compiler_so', CUDA['nvcc'])
# use only a subset of the extra_postargs, which are 1-1 translated
# from the extra_compile_args in the Extension class
postargs = extra_postargs['nvcc']
else:
postargs = extra_postargs['gcc']
super(obj, src, ext, cc_args, postargs, pp_opts)
# reset the default compiler_so, which we might have changed for cuda
self.compiler_so = default_compiler_so
# inject our redefined _compile method into the class
self._compile = _compile
# run the customize_compiler
class custom_build_ext(build_ext):
def build_extensions(self):
customize_compiler_for_nvcc(self.compiler)
build_ext.build_extensions(self)
setup(name='gpuadder',
# random metadata. there's more you can supploy
author='Robert McGibbon',
version='0.1',
# this is necessary so that the swigged python file gets picked up
py_modules=['gpuadder'],
package_dir={'': 'src'},
ext_modules = [ext],
# inject our custom trigger
cmdclass={'build_ext': custom_build_ext},
# since the package has c code, the egg cannot be zipped
zip_safe=False)

As an alternative to distutils/setuptools, you could use scikit-build (along with CMakeLists.txt, pyproject.toml, and setup.cfg/setup.py):
import sys
from pathlib import Path
from skbuild import setup
from setuptools import find_packages
# https://github.com/scikit-build/scikit-build/issues/521#issuecomment-753035688
for i in (Path(__file__).resolve().parent / "_skbuild").rglob("CMakeCache.txt"):
i.write_text(re.sub("^//.*$\n^[^#].*pip-build-env.*$", "", i.read_text(), flags=re.M))
setup(cmake_args=[f"-DPython3_ROOT_DIR={sys.prefix}"],
packages=find_packages(exclude=["tests"]))

Related

Keyerror 'pywintypes'

I solved my last problem, but now, it pretty much:
Traceback (most recent call last):
File "C:\Users\Qihua Huang\AppData\Local\Programs\Python\Python310\Lib\site-packages\win32\lib\pywintypes.py", line 115, in <module>
__import_pywin32_system_module__("pywintypes", globals())
File "C:\Users\Qihua Huang\AppData\Local\Programs\Python\Python310\Lib\site-packages\win32\lib\pywintypes.py", line 104, in __import_pywin32_system_module__
old_mod = sys.modules[modname]
KeyError: 'pywintypes'
I accessed the pwintypes.py, and inserted print statements to fish out error:
# Magic utility that "redirects" to pywintypesxx.dll
import importlib.util, importlib.machinery, sys, os
def __import_pywin32_system_module__(modname, globs):
# This has been through a number of iterations. The problem: how to
# locate pywintypesXX.dll when it may be in a number of places, and how
# to avoid ever loading it twice. This problem is compounded by the
# fact that the "right" way to do this requires win32api, but this
# itself requires pywintypesXX.
# And the killer problem is that someone may have done 'import win32api'
# before this code is called. In that case Windows will have already
# loaded pywintypesXX as part of loading win32api - but by the time
# we get here, we may locate a different one. This appears to work, but
# then starts raising bizarre TypeErrors complaining that something
# is not a pywintypes type when it clearly is!
# So in what we hope is the last major iteration of this, we now
# rely on a _win32sysloader module, implemented in C but not relying
# on pywintypesXX.dll. It then can check if the DLL we are looking for
# lib is already loaded.
# See if this is a debug build.
suffix = "_d" if "_d.pyd" in importlib.machinery.EXTENSION_SUFFIXES else ""
filename = "%s%d%d%s.dll" % (modname,sys.version_info[0],sys.version_info[1],suffix)
if hasattr(sys, "frozen"):
# If we are running from a frozen program (py2exe, McMillan, freeze)
# then we try and load the DLL from our sys.path
# XXX - This path may also benefit from _win32sysloader? However,
# MarkH has never seen the DLL load problem with py2exe programs...
for look in sys.path:
# If the sys.path entry is a (presumably) .zip file, use the
# directory
if os.path.isfile(look):
look = os.path.dirname(look)
found = os.path.join(look, filename)
if os.path.isfile(found):
break
else:
raise ImportError(
"Module '%s' isn't in frozen sys.path %s" % (modname, sys.path)
)
else:
# First see if it already in our process - if so, we must use that.
from win32 import _win32sysloader
found = _win32sysloader.GetModuleFilename(filename)
if found is None:
# We ask Windows to load it next. This is in an attempt to
# get the exact same module loaded should pywintypes be imported
# first (which is how we are here) or if, eg, win32api was imported
# first thereby implicitly loading the DLL.
# Sadly though, it doesn't quite work - if pywintypesxx.dll
# is in system32 *and* the executable's directory, on XP SP2, an
# import of win32api will cause Windows to load pywintypes
# from system32, where LoadLibrary for that name will
# load the one in the exe's dir.
# That shouldn't really matter though, so long as we only ever
# get one loaded.
found = _win32sysloader.LoadModule(filename)
if found is None:
# Windows can't find it - which although isn't relevent here,
# means that we *must* be the first win32 import, as an attempt
# to import win32api etc would fail when Windows attempts to
# locate the DLL.
# This is most likely to happen for "non-admin" installs, where
# we can't put the files anywhere else on the global path.
# If there is a version in our Python directory, use that
if os.path.isfile(os.path.join(sys.prefix, filename)):
found = os.path.join(sys.prefix, filename)
if found is None:
# Not in the Python directory? Maybe we were installed via
# easy_install...
if os.path.isfile(os.path.join(os.path.dirname(__file__), filename)):
found = os.path.join(os.path.dirname(__file__), filename)
print(found)
found='C:\\Users\Qihua Huang\\AppData\\Local\\Programs\\Python\\Python310\\Lib\\site-packages\\pywin32_system32\\pywintypes310.dll'
print(found)
# There are 2 site-packages directories - one "global" and one "user".
# We could be in either, or both (but with different versions!). Factors include
# virtualenvs, post-install script being run or not, `setup.py install` flags, etc.
# In a worst-case, it means, say 'python -c "import win32api"'
# will not work but 'python -c "import pywintypes, win32api"' will,
# but it's better than nothing.
# We prefer the "user" site-packages if it exists...
if found is None:
import site
maybe = os.path.join(site.USER_SITE, "pywin32_system32", filename)
print(maybe)
if os.path.isfile(maybe):
found = maybe
print(found)
# Or the "global" site-packages.
if found is None:
import sysconfig
maybe = os.path.join(
sysconfig.get_paths()["platlib"], "pywin32_system32", filename
)
print(maybe)
if os.path.isfile(maybe):
found = maybe
print(found)
if found is None:
# give up in disgust.
raise ImportError("No system module '%s' (%s)" % (modname, filename))
# After importing the module, sys.modules is updated to the DLL we just
# loaded - which isn't what we want. So we update sys.modules to refer to
# this module, and update our globals from it.
old_mod = sys.modules[modname]
# Load the DLL.
loader = importlib.machinery.ExtensionFileLoader(modname, found)
spec = importlib.machinery.ModuleSpec(name=modname, loader=loader, origin=found)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
# Check the sys.modules[] behaviour we describe above is true...
assert sys.modules[modname] is mod
# as above - re-reset to the *old* module object then update globs.
sys.modules[modname] = old_mod
globs.update(mod.__dict__)
__import_pywin32_system_module__("pywintypes", globals())
The output that followed before error was:
C:\Users\*user*\AppData\Local\Programs\Python\Python310\Lib\site-packages\win32\lib\pywintypes310.dll
C:\Users\*user*\AppData\Local\Programs\Python\Python310\Lib\site-packages\pywin32_system32\pywintypes310.dll

scon portable and cleaner way for debug and release build together

SConstruct : This file is implemented to use build library for debug and release build.
variant_dir is set to build/debug for debug build
& set to build/release for release build
import os
env = Environment()
releaseEnv = env.Clone(CCFLAGS = ['-O3'])
debugEnv = env.Clone(CCFLAGS = ['-O0', '-g'])
debugDirPath = os.path.join('build', 'debug') # build/debug
releaseDirPath = os.path.join('build', 'release') # build/release
if os.name == 'nt':
releaseEnv.Replace(CCFLAGS = ['EHsc'])
# windows specific flags
debugEnv.Replace(CCFLAGS = ['EHsc', 'Zi', 'MTd'])
SConscript(dirs = 'src', name = 'SConscript', exports = {'env' : releaseEnv}, variant_dir = releaseDirPath, duplicate = 0)
SConscript(dirs = 'src', name = 'SConscript', exports = {'env': debugEnv}, variant_dir = debugDirPath, duplicate = 0)
SConscript: (present inside source directory which contains a1.cpp and b1.cpp)
import os
Import('env')
src_list = Glob(os.path.join(Dir('#').abspath, 'src', '*.cpp'))
env.SharedLibrary(target='sum', source= src_list)
env.StaticLibrary(target='sum', source= src_list)
Directory structure is like:
root_dir -> SConstruct
-> src
-> SConscript
-> sum.cpp
-> mul.cpp
1) Running scons from root_dir generates following warning and although it's a warning message build is stop, library doesn't gets created.
scons: * Two environments with different actions were specified for the same target: /home/xyz/temp/src/mul.os
File "/home/xyz/temp/src/SConscript", line 7, in
This issue has been resolved after using src_list = Glob('*.cpp');
2) What is the proper (portable) way to create environment object for debug and release build ?
The way I have implemented is it correct ?
Kindly suggest necessary changes to avoid the warning and running build successfully.
Your problem is not related to build variants, but the fact that you have two targets with the same name (SharedLibrary and StaticLibrary both build sum).
To fix that, either just give one of them another name or add an extension to at least one of them. If you add an extension, you might want to check for OS if you want to keep your cross-platform compatibility.

Automate compilation of protobuf specs into python classes in setup.py

I have a python project that uses google protobufs as a message format for communicating over the network. Generating python files from the .proto files is straight-forward using the protoc program. How can I configure my setup.py file for the project so that it automatically calls the protoc command?
In a similar situation, I ended up with this code (setup.py, but written in a way to allow extraction into some external Python module for reuse). Note that I took the generate_proto function and several ideas from the setup.py file of the protobuf source distribution.
from __future__ import print_function
import os
import shutil
import subprocess
import sys
from distutils.command.build_py import build_py as _build_py
from distutils.command.clean import clean as _clean
from distutils.debug import DEBUG
from distutils.dist import Distribution
from distutils.spawn import find_executable
from nose.commands import nosetests as _nosetests
from setuptools import setup
PROTO_FILES = [
'goobuntu/proto/hoststatus.proto',
]
CLEANUP_SUFFIXES = [
# filepath suffixes of files to remove on "clean" subcommand
'_pb2.py',
'.pyc',
'.so',
'.o',
'dependency_links.txt',
'entry_points.txt',
'PKG-INFO',
'top_level.txt',
'SOURCES.txt',
'.coverage',
'protobuf/compiler/__init__.py',
]
CLEANUP_DIRECTORIES = [ # subdirectories to remove on "clean" subcommand
# 'build' # Note: the build subdirectory is removed if --all is set.
'html-coverage',
]
if 'PROTOC' in os.environ and os.path.exists(os.environ['PROTOC']):
protoc = os.environ['PROTOC']
else:
protoc = find_executable('protoc')
def generate_proto(source):
"""Invoke Protocol Compiler to generate python from given source .proto."""
if not os.path.exists(source):
sys.stderr.write('Can\'t find required file: %s\n' % source)
sys.exit(1)
output = source.replace('.proto', '_pb2.py')
if (not os.path.exists(output) or
(os.path.getmtime(source) > os.path.getmtime(output))):
if DEBUG:
print('Generating %s' % output)
if protoc is None:
sys.stderr.write(
'protoc not found. Is protobuf-compiler installed? \n'
'Alternatively, you can point the PROTOC environment variable at a '
'local version.')
sys.exit(1)
protoc_command = [protoc, '-I.', '--python_out=.', source]
if subprocess.call(protoc_command) != 0:
sys.exit(1)
class MyDistribution(Distribution):
# Helper class to add the ability to set a few extra arguments
# in setup():
# protofiles : Protocol buffer definitions that need compiling
# cleansuffixes : Filename suffixes (might be full names) to remove when
# "clean" is called
# cleandirectories : Directories to remove during cleanup
# Also, the class sets the clean, build_py, test and nosetests cmdclass
# options to defaults that compile protobufs, implement test as nosetests
# and enables the nosetests command as well as using our cleanup class.
def __init__(self, attrs=None):
self.protofiles = [] # default to no protobuf files
self.cleansuffixes = ['_pb2.py', '.pyc'] # default to clean generated files
self.cleandirectories = ['html-coverage'] # clean out coverage directory
cmdclass = attrs.get('cmdclass')
if not cmdclass:
cmdclass = {}
# These should actually modify attrs['cmdclass'], as we assigned the
# mutable dict to cmdclass without copying it.
if 'nosetests' not in cmdclass:
cmdclass['nosetests'] = MyNosetests
if 'test' not in cmdclass:
cmdclass['test'] = MyNosetests
if 'build_py' not in cmdclass:
cmdclass['build_py'] = MyBuildPy
if 'clean' not in cmdclass:
cmdclass['clean'] = MyClean
attrs['cmdclass'] = cmdclass
# call parent __init__ in old style class
Distribution.__init__(self, attrs)
class MyClean(_clean):
def run(self):
try:
cleandirectories = self.distribution.cleandirectories
except AttributeError:
sys.stderr.write(
'Error: cleandirectories not defined. MyDistribution not used?')
sys.exit(1)
try:
cleansuffixes = self.distribution.cleansuffixes
except AttributeError:
sys.stderr.write(
'Error: cleansuffixes not defined. MyDistribution not used?')
sys.exit(1)
# Remove build and html-coverage directories if they exist
for directory in cleandirectories:
if os.path.exists(directory):
if DEBUG:
print('Removing directory: "{}"'.format(directory))
shutil.rmtree(directory)
# Delete generated files in code tree.
for dirpath, _, filenames in os.walk('.'):
for filename in filenames:
filepath = os.path.join(dirpath, filename)
for i in cleansuffixes:
if filepath.endswith(i):
if DEBUG:
print('Removing file: "{}"'.format(filepath))
os.remove(filepath)
# _clean is an old-style class, so super() doesn't work
_clean.run(self)
class MyBuildPy(_build_py):
def run(self):
try:
protofiles = self.distribution.protofiles
except AttributeError:
sys.stderr.write(
'Error: protofiles not defined. MyDistribution not used?')
sys.exit(1)
for proto in protofiles:
generate_proto(proto)
# _build_py is an old-style class, so super() doesn't work
_build_py.run(self)
class MyNosetests(_nosetests):
def run(self):
try:
protofiles = self.distribution.protofiles
except AttributeError:
sys.stderr.write(
'Error: protofiles not defined. MyDistribution not used?')
for proto in protofiles:
generate_proto(proto)
# _nosetests is an old-style class, so super() doesn't work
_nosetests.run(self)
setup(
# MyDistribution automatically enables several extensions, including
# the compilation of protobuf files.
distclass=MyDistribution,
...
tests_require=['nose'],
protofiles=PROTO_FILES,
cleansuffixes=CLEANUP_SUFFIXES,
cleandirectories=CLEANUP_DIRECTORIES,
)
Here's the solution that I have used for setup.py. The only thing you need to keep in mind is the version of the protoc compiler is compatible with the installed protobuf version.
'''
# here you can specify the proto folder and
# output folder or input them as parameters
# to script
protoc_command = [
"python", "-m", "grpc_tools.protoc",
f"--proto_path={proto_folder}",
f"--python_out={output_folder}",
f"--grpc_python_out={output_folder}",
]
'''

Reliable way to get the "build" directory from within setup.py

Inside the setup.py script I need to create some temporary files for the installation. The natural place to put them would be the "build/" directory.
Is there a way to retrieve its path that works if installing via pypi, from source, easy_install, pip, ...?
Thanks a lot!
By default distutils create build/ in current working dir, but it can be changed by argument --build-base. Seems like distutils parses it when executing setup and parsed argument does not accessible from outside, but you can cut it yourself:
import sys
build_base_long = [arg[12:].strip("= ") for arg in sys.argv if arg.startswith("--build-base")]
build_base_short = [arg[2:].strip(" ") for arg in sys.argv if arg.startswith("-b")]
build_base_arg = build_base_long or build_base_short
if build_base_arg:
build_base = build_base_arg[0]
else:
build_base = "."
This naive version of parser still shorter than optparse's version with proper error handling for unknown flags. Also you can use argparse's parser, which have try_parse method.
distutils/setuptools provide an abstract Command class that users can use to add custom commands to their package's setup process. This is the same class that built-in setup commands like build and install are subclasses of.
Every class that is a subclass of the abstract Command class must implement the initialize_options, finalize_options, and run methods. The "options" these method names refer to are class attributes that are derived from command-line arguments provided by the user (they can also have default values). The initialize_options method is where a class's options are defined, the finalize_options method is where a class's option values are assigned, and the run method is where a class's option values are used to perform the function of the command.
Since command-line arguments may affect more than one command, some command classes may share options with other command classes. For example, all the distutils/setuptools build commands (build, build_py, build_clib, build_ext, and build_scripts) and the install command need to know where the build directory is. Instead of having every one of these command classes define and parse the same command-line arguments into the same options, the build command, which is the first of all these commands to be executed, defines and parses the command-line arguments and options, and all the other classes get the option values from the build command in their finalize_options method.
For example, the build class defines the build_base and build_lib options in its initialize_options method and then computes their values from the command-line arguments in its finalize_options method. The install classes also defines the build_base and build_lib options in its initialize_options method but it gets the values for these options from the build command in its finalize_options method.
You can use the same pattern to add a custom sub-command to the build command as follows (it would be similar for install)
import setuptools
from distutils.command.build import build
class BuildSomething(setuptools.Command):
def initialize_options(self):
# define the command's options
self.build_base = None
self.build_lib = None
def finalize_options(self):
# get the option values from the build command
self.set_undefined_options('build',
('build_base', 'build_base'),
('build_lib', 'build_lib'))
def run(self):
# do something with the option values
print(self.build_base) # defaults to 'build'
print(self.build_lib)
build_something_command = 'build_something'
class Build(build):
def has_something(self):
# update this to check if your build should run
return True
sub_commands = [(build_something_command, has_something)] + build.sub_commands
COMMAND_CLASS = {
build_something_command: BuildSomething, # custom command
'build': Build # override distutils/setuptools build command
}
setuptools.setup(cmdclass=COMMAND_CLASS)
Alternatively, you could just subclass one of the distutils/setuptools classes if you just want to extend its functionality and it already has the options you need
import setuptools
from setuptools.command.build_py import build_py
class BuildPy(build_py):
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
# do something with the option values
print(self.build_lib) # inherited from build_py
build_py.run(self) # make sure the regular build_py still runs
COMMAND_CLASS = {
'build_py': BuildPy # override distutils/setuptools build_py command
}
setuptools.setup(cmdclass=COMMAND_CLASS)
Unfortunately, none of this is very well documented anywhere. I learned most of it from reading the distutils and setuptools source code. Any of the build*.py and install*.py files in either repository's command directory is informative. The abstract Command class is defined in distutils.
perhaps something like this? works in my case with python 3.8
...
from distutils.command.build import get_platform
import sys
import os
...
def configuration(parent_package='', top_path=None):
config = Configuration('', parent_package, top_path)
# add xxx library
config.add_library('xxx',['xxx/src/fil1.F90',
'xxx/src/file2.F90',
'xxx/src/file3.F90'],
language='f90')
# check for the temporary build directory option
_tempopt = None
_chkopt = ('-t','--build-temp')
for _opt in _chkopt:
if _opt in sys.argv:
_i = sys.argv.index(_opt)
if _i < len(sys.argv)-1:
_tempopt = sys.argv[_i+1]
break
# check for the base directory option
_buildopt = 'build'
_chkopt = ('-b','--build-base')
for _opt in _chkopt:
if _opt in sys.argv:
_i = sys.argv.index(_opt)
if _i < len(sys.argv)-1:
_buildopt = sys.argv[_i+1]
break
if _tempopt is None:
# works with python3 (check distutils/command/build.py)
platform_specifier = ".%s-%d.%d" % (get_platform(), *sys.version_info[:2])
_tempopt = '%s%stemp%s'%(_buildopt,os.sep,platform_specifier)
# add yyy module (wraps fortran code in xxx library)
config.add_extension('fastpost',sources=['yyy/src/fastpost.f90'],
f2py_options=['--quiet'],
libraries=['xxx'])
# to access the mod files produced from fortran modules comppilaton, add
# the temp build directory to the include directories of the configuration
config.add_include_dirs(_tempopt)
return config
setup(name="pysimpp",
version="0.0.1",
description="xxx",
author="xxx",
author_email="xxx#yyy",
configuration=configuration,)

How do I use cx_freeze?

I've created my setup.py file as instructed but I don't actually.. understand what to do next. Typing "python setup.py build" into the command line just gets a syntax error.
So, what do I do?
setup.py:
from cx_Freeze import setup, Executable
setup(
name = "On Dijkstra's Algorithm",
version = "3.1",
description = "A Dijkstra's Algorithm help tool.",
exectuables = [Executable(script = "Main.py", base = "Win32GUI")])
Add import sys as the new topline
You misspelled "executables" on the last line.
Remove script = on last line.
The code should now look like:
import sys
from cx_Freeze import setup, Executable
setup(
name = "On Dijkstra's Algorithm",
version = "3.1",
description = "A Dijkstra's Algorithm help tool.",
executables = [Executable("Main.py", base = "Win32GUI")])
Use the command prompt (cmd) to run python setup.py build. (Run this command from the folder containing setup.py.) Notice the build parameter we added at the end of the script call.
I'm really not sure what you're doing to get that error, it looks like you're trying to run cx_Freeze on its own, without arguments. So here is a short step-by-step guide on how to do it in windows (Your screenshot looks rather like the windows command line, so I'm assuming that's your platform)
Write your setup.py file. Your script above looks correct so it should work, assuming that your script exists.
Open the command line (Start -> Run -> "cmd")
Go to the location of your setup.py file and run python setup.py build
Notes:
There may be a problem with the name of your script. "Main.py" contains upper case letters, which might cause confusion since windows' file names are not case sensitive, but python is. My approach is to always use lower case for scripts to avoid any conflicts.
Make sure that python is on your PATH (read http://docs.python.org/using/windows.html)1
Make sure are are looking at the new cx_Freeze documentation. Google often seems to bring up the old docs.
I ran into a similar issue. I solved it by setting the Executable options in a variable and then simply calling the variable. Below is a sample setup.py that I use:
from cx_Freeze import setup, Executable
import sys
productName = "ProductName"
if 'bdist_msi' in sys.argv:
sys.argv += ['--initial-target-dir', 'C:\InstallDir\\' + productName]
sys.argv += ['--install-script', 'install.py']
exe = Executable(
script="main.py",
base="Win32GUI",
targetName="Product.exe"
)
setup(
name="Product.exe",
version="1.0",
author="Me",
description="Copyright 2012",
executables=[exe],
scripts=[
'install.py'
]
)
You can change the setup.py code to this:
from cx_freeze import setup, Executable
setup( name = "foo",
version = "1.1",
description = "Description of the app here.",
executables = [Executable("foo.py")]
)
I am sure it will work. I have tried it on both windows 7 as well as ubuntu 12.04
find the cxfreeze script and run it. It will be in the same path as your other python helper scripts, such as pip.
cxfreeze Main.py --target-dir dist
read more at:
http://cx-freeze.readthedocs.org/en/latest/script.html#script
I usually put the calling setup.py command into .bat file to easy recall.
Here is simple code in COMPILE.BAT file:
python setup.py build
#ECHO:
#ECHO . : ` . * F I N I S H E D * . ` : .
#ECHO:
#Pause
And the setup.py is organized to easy customizable parameters that let you set icon, add importe module library:
APP_NAME = "Meme Studio"; ## < Your App's name
Python_File = "app.py"; ## < Main Python file to run
Icon_Path = "./res/iconApp48.ico"; ## < Icon
UseFile = ["LANGUAGE.TXT","THEME.TXT"];
UseAllFolder = True; ## Auto scan folder which is same level with Python_File and append to UseFile.
Import = ["infi","time","webbrowser", "cv2","numpy","PIL","tkinter","math","random","datetime","threading","pathlib","os","sys"]; ## < Your Imported modules (cv2,numpy,PIL,...)
Import+=["pkg_resources","xml","email","urllib","ctypes", "json","logging"]
################################### CX_FREEZE IGNITER ###################################
from os import walk
def dirFolder(folderPath="./"): return next(walk(folderPath), (None, None, []))[1]; # [ Folder ]
def dirFile(folderPath="./"): return next(walk(folderPath), (None, None, []))[2]; # [ File ]
if UseAllFolder: UseFile += dirFolder();
import sys, pkgutil;
from cx_Freeze import setup, Executable;
BasicPackages=["collections","encodings","importlib"] + Import;
def AllPackage(): return [i.name for i in list(pkgutil.iter_modules()) if i.ispkg]; # Return name of all package
#Z=AllPackage();Z.sort();print(Z);
#while True:pass;
def notFound(A,v): # Check if v outside A
try: A.index(v); return False;
except: return True;
build_msi_options = {
'add_to_path': False,
"upgrade_code": "{22a35bac-14af-4159-7e77-3afcc7e2ad2c}",
"target_name": APP_NAME,
"install_icon": Icon_Path,
'initial_target_dir': r'[ProgramFilesFolder]\%s\%s' % ("Picox", APP_NAME)
}
build_exe_options = {
"includes": BasicPackages,
"excludes": [i for i in AllPackage() if notFound(BasicPackages,i)],
"include_files":UseFile,
"zip_include_packages": ["encodings"] ##
}
setup( name = APP_NAME,
options = {"build_exe": build_exe_options},#"bdist_msi": build_msi_options},#,
executables = [Executable(
Python_File,
base='Win32GUI',#Win64GUI
icon=Icon_Path,
targetName=APP_NAME,
copyright="Copyright (C) 2900AD Muc",
)]
);
The modules library list in the code above is minimum for workable opencv + pillow + win32 application.
Example of my project file organize:
========== U P D A T E ==========
Although cx_Freeze is a good way to create setup file. It's really consume disk space if you build multiple different software project that use large library module like opencv, torch, ai... Sometimes, users download installer, and receive false positive virus alert about your .exe file after install.
Thus, you should consider use SFX archive (.exe) your app package and SFX python package separate to share between app project instead.
You can create .bat that launch .py file and then convert .bat file to .exe with microsoft IExpress.exe.
Next, you can change .exe icon to your own icon with Resource Hacker: http://www.angusj.com/resourcehacker/
And then, create SFX archive of your package with PeaZip: https://peazip.github.io/
Finally change the icon.
The Python Package can be pack to .exe and the PATH register can made with .bat that also convertable to .exe.
If you learn more about command in .bat file and make experiments with Resource Hacker & self extract ARC in PeaZip & IExpress, you can group both your app project, python package into one .exe file only. It'll auto install what it need on user machine. Although this way more complex and harder, but then you can custom many install experiences included create desktop shorcut, and install UI, and add to app & feature list, and uninstall ability, and portable app, and serial key require, and custom license agreement, and fast window run box, and many more,..... but the important features you get is non virus false positive block, reduce 200MB to many GB when user install many your python graphic applications.

Categories

Resources