Build VTK with Python Bindings and CUDA, TBB, & MPI on Windows - python

Problem
I cannot load VTK into python on Windows with support for CUDA, MPI, and TBB. Volume rendering is unreasonably slow for production with the default PyPI distribution (hours or more) and I need to leverage my hardware to speed up computations.
I have successfully built VTK 9.3.0 for Python 3.8.10-x64 with CMake and Visual Studio and have bin, lib, include, and share folders and have added these to PATH, but Python does not see vtk:
Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'vtk'
>>>
I attempted to place the generated Lib/site-packages, bin, lib, and include folders in appropriate places in C:\Program Files\Python38, but I still get a DLL load error:
> py -3.8
Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Traceback (most recent call last):
File "C:\Program Files\Python38\lib\site-packages\vtkmodules\__init__.py", line 13, in <module>
from . import vtkCommonCore
ImportError: DLL load failed while importing vtkCommonCore: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files\Python38\lib\site-packages\vtk.py", line 30, in <module>
all_m = importlib.import_module('vtkmodules.all')
File "C:\Program Files\Python38\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Program Files\Python38\lib\site-packages\vtkmodules\__init__.py", line 15, in <module>
import _vtkmodules_static
ModuleNotFoundError: No module named '_vtkmodules_static'
>>>
Setup
OS: Windows 10 Enterprise, x64-bit, Build 1909
CPU: 2x Intel(R) Xeon(R) Gold 6248R
Disk: 2TB NVMe M.2 SSD
RAM: 192 GB DDR4
Compute GPUs: 2x NVIDIA Quadro RTX8000 in TCC mode with NVLink
Display GPU: 1x NVIDIA Quadro RTX4000
Visual Studio: Visual Studio 16 2019 Community
Windows SDK: 10.0.19041.0, targeting 10.0.18363
Python: 3.8.10 x64-bit
CMake: 3.21.1 (I use CMake-GUI)
vcpkg: 2021-08-12
VTK Source: vtk-9.0.3-cp38-cp38-win_amd64 (download from VTK website; not GitHub master)
VTKPythonPackage: GitHub Link (...not sure if I need this...)
CUDA Toolkit: 11.4
TBB: 2021.3.0 (via Intel(R) oneAPI Base Toolkit)
MPI: 2021.3.0 (via Intel(R) oneAPI HPC Toolkit)
NOTE: If desired, Embree, OSPRay, and OpenVKL are available in the Intel(R) oneAPI Rendering Toolkit
Preparation
NOTE: There are A LOT of programs to be downloaded, installed, and configured, and they are important for proper building. In an attempt to shorten this post to an MRE (Minimal Reproducable Example), please go straight to the Steps to Reproduce section.
NOTE: When setting Environment Variables, you may get an error that a PATH length greater than 2047 is not permitted. To overcome this hurdle, it is simplest to edit the path directly in the Windows Registry. Open regedit as an Admin, navigate to
Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment
and set the Path variable there.
TIP: To edit PATH, I copy and paste from the Registry into Word, then selected and replace ;'s with paragraph endings ^p by selecting Ctrl+H and then using the Replace tool as follows:
When finished, I simply reverse the changes by replacing ^p's with ;s. This is simpler for me than struggling with regular expressions.
1. Install Visual Studio
Download the Visual Studio Installer
Run the installer, choose Visual Studio Community 2019, and select Desktop development with C++. Ensure the following are selected in Individual Components
Windows 10 SDK
C++ ATL for latest...
C++ MFC for latest...
C++ CMake tools for Windows
NOTE: (For me on Windows 10 with Visual Studio 2019, these are v142)
Ensure the path to the corresponding bin, lib, and include folders are in the PATH of System Environment Variables. These can be either:
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\lib
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include
and/or
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\lib\x64\
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include
NOTE: (I recommend the latter at a minimum. )
Ensure the following additional System Environment Variables have been set by the installer:
VS140COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\
VS160COMCOMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\Tools\
VS2019INSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
VSSDK140INSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VSSDK\
NOTE: Even though the installer downloads these to Program Files (x86), they are in fact the x64-bit programs (I know...Windows is weird...)
2. Install CUDA
Download and run cuda_11.4.1_471.41_win10.exe (link)
TIP: The local installer will take longer to download, but your installs will run faster. I recommend saving the installer for the future anyway in case your CUDA Toolkit becomes corrupted or you need to install on another machine.
Ensure the following are added to PATH in System Environment Variables
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\libnvvp
ASIDE: You MUST use version 11.4 for VTK 9. However, as an aside, when using 10.2 in CMake, if CMake cannot find CUDA, this is probably because the CUDA Toolkit 10.2 installer for some reason does not copy the Visual Studio MSBuild Extensions over to your Visual Studio folder. Copy the files at
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\extras\visual_studio_integration\MSBuildExtensions\
to
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\VC\VCTargets\BuildCustomizations
NOTE: If you want to run your GPU in TCC mode (WARNING: This disables graphics output), and/or if you have two GPUs connected via NVLink and want to activate NVLink, run the following command in command prompt:
> nvidia-smi -i 0 -dm 1
> nvidia-smi -i 1 -dm 1
Assuming you have two GPUs with ID's 0 and 1 you want to run in TCC mode. (To check, simply run nvidia-smi and see which GPUs you want to activate in TCC mode with the -dm switch). The PugetSystems Instructions have change with CUDA 11. NVIDIA's newer drivers, which ship with the NVIDIA Control Panel, no longer have the Configure SLI, Surround, PhysX under 3D Settings, particularly if using a Quadro card rather than a GeForce card. Instead, NVLink will be activated once TCC mode is set, which disables graphics output. The program they have provided only works for CUDA 10, so you will get an error with it when using CUDA 11 even though it is enabled.
NOTE: If you get an error "Intel Graphics Driver for Windows not found", ignore it. You only need the drivers for the CPU.
NOTE: Don't add more than the /lib/ and /libnvvp/ folders to path. CMake might not be able to find CUDA if you do. If you have to add other folders (e.g. CUPTI), I recommend the lib/ and libnvvp/ folder appear FIRST in PATH.
3. Install CMake
Download and install CMake (...that was easy)
4. Install vcpkg
Download and configure vcpkg:
cd C:\
git clone https://github.com/microsoft/vcpkg
cd vcpkg
.\bootstrap-vcpkg.bat
vcpkg integrate install
Set the default triplet in the System Environment Variables to x64-windows:
VCPKG_DEFAULT_TRIPLET=x64-windows
Install ports using vcpkg install. Press Configure in CMake and every time it cannot find a dependency you want installed, search for the dependency with vcpkg search and install it with vcpkg install (to remove a package, use vcpkg remove)._
NOTE: BE AWARE that depending on what you install, these dependencies take A LOT of space and take A LONG TIME to download.
5. Install Intel oneAPI Toolkits
NOTE: These include TBB, MPI, OSPRay, and the Intel Fortran compiler ifort.exe, which you need.
Intel(R) oneAPI Base Toolkit
Intel(R) oneAPI HPC Toolkit
Intel(R) oneAPI Rendering Toolkit (for Embree, OSPRay, and OpenVKL, optionally)
and add the corresponding lib, bin, and include folders to PATH.
6. Install Intel SPMD Compiler
Intel(R) Implicit SPMD Program Compiler
7. Additional Dependencies with Installers
NOTE: You may not need or want need these. I am adding them here for completeness. If CMake complains about not finding these, I recommend installing them.
Nullsoft Scriptable Install System
WiX Toolset
MySQL Community (GPL) Installer for Windows
MySQL ODBC Connector
Strawberry Perl
8. Dependencies that are Unsuccessful on Windows
NOTE: I have tried installing binaries for these and adding them to PATH, but they just don't seem to work on Windows..
NOTE: For each of these, you can also try downloading corresponding python 3.8 zip source file (on GitHub, under Releases, and on conda forge, under Files) to a folder on C:` and then add the subfolders bin, lib, include (and share if available) to the PATH System Environment Variable.
OpenTurns 1.17
GDAL 3.3.1
Steps to Reproduce
I set my PATH first. VTK will make bin, include, lib, and share folders, so I make sure these are at the front of PATH. Here is what I have (HINT: Refer to setting PATH in regedit to avoid the Windows error of PATH length greater than 2047 characters in the Preparation section above)
Open an x64 Native Tools Command Prompt for VS 2019 terminal as an Administrator and run
> cmake-gui
This ensures vcvarsall.bat is run and the corresponding Environment Variables it sets are passed on to CMake-GUI.
Set the Source folder to C:\VTK\src and Build folder to C:\VTK
UPDATE 27-AUG-2021:
Step 3 was the problem. The build folder was previously C:\VTK\build. It needed to be C:\VTK. I corrected this.
Press Configure and select Visual Studio 16 2019 as the generator with x64 as the optional platform:
NOTE: I do not know if libraries for VTK and CUDA should be built for 32-bit. I am using 64-bit as my best guess.
After Configuration completes, you will see many variables, which is understandably overwhelming and it is unclear what the differences are between DEFAULT, YES, NO, WANT, and DON'T WANT. Unfortunately, the answer is hidden tucked away in VTK's CMake Module API Doxygen docs
VTK CMake Module API
For now, I leave everything default and edit the following CMake variables:
TIP: Check the Grouped and Advanced check boxes next to the Add Entry button to see all variables grouped by corresponding project/module:
CMAKE_CXX_MP_FLAG Checked
CMAKE_INSTALL_PREFIX C:/VTK
VTK_GROUP_ENABLE_Imaging WANT
VTK_GROUP_ENABLE_MPI WANT
VTK_GROUP_ENABLE_Qt WANT
VTK_PYTHON_VERSION 3
VTK_SMP_IMPLEMENTATION_TYPE TBB
VTK_USE_CUDA Checked
VTK_USE_MPI Checked
VTK_WRAP_PYTHON Checked
VTK_ENABLE_KITS Checked
and delete
Python2_EXECUTABLE
under Ungrouped Entries
Rerun Configure
Change the following CMake Variables:
Python3_EXECUTABLE C:/Program Files/Python38/python.exe
Python3_INCLUDE_DIR C:/Program Files/Python38/include
Python3_LIBRARY C:/Program Files/Python38/libs/python38.lib
Rerun Configure
Output from 3rd Configure:
Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.18363.
Found Python3: C:/Program Files/Python38/python.exe (found suitable version "3.8.10", minimum required is "3.2") found components: Interpreter Development.Module Development.Embed
Configuring done
My CMake variables now look like this:
ASIDE: Would be nice if the CMake GUI had an option to output variables to a text file, especially for debugging...
ASIDE: Would be nice if CMake GUI could tell you what had actually been activiated/used since everything shows up with DEFAULT value and we don't want to change that...
ASIDE: Would be nice if there was a high level overview somewhere of what all of these modules entail so users can understand what they want/need...
Click Generate
Output from Generate
Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.18363.
Configuring done
Generating done
Open Visual Studio 2019 as an Administrator
Open the Solution file VTK.sln in C:\VTK\build
Change the build type from Debug to Release for speed:
Right-click on ALL_BUILD and select Build. In the end, the output is
========== Build: 398 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
Right-click on INSTALL and select Build. In the end, the output is:
========== Build: 116 succeeded, 0 failed, 283 up-to-date, 0 skipped ==========
Issue
Please see problem at top of post.
Update: 26-AUG-2021
Per #JensMunk suggestion, I copied the VTK master branch source code to overwrite vtkmodules/__init__.py with support for os.add_dll_directory(), but to no avail:
r"""
Currently, this package is experimental and may change in the future.
"""
from __future__ import absolute_import
import sys
def _windows_dll_path():
import os
_vtk_python_path = 'Lib/site-packages/vtkmodules'
_vtk_dll_path = 'bin'
# Compute the DLL path based on the location of the file and traversing up
# the installation prefix to append the DLL path.
_vtk_dll_directory = os.path.dirname(os.path.abspath(__file__))
# Loop while we have components to remove.
print("Folder Search:")
print("==============")
while _vtk_python_path not in ("", ".", "/"):
# Strip a directory away.
_vtk_python_path = os.path.dirname(_vtk_python_path)
_vtk_dll_directory = os.path.dirname(_vtk_dll_directory)
if _vtk_python_path != "C:\\":
print(_vtk_python_path)
print("==============")
print()
_vtk_dll_directory = os.path.join(_vtk_dll_directory, _vtk_dll_path)
print(f"VTK DLL Directory: {_vtk_dll_directory}")
print()
print("Adding DLLs...")
if os.path.exists(_vtk_dll_directory):
# We never remove this path; it is required for VTK to work and there's
# no scope where we can easily remove the directory again.
print(f"DLL: {_vtk_dll_directory}")
_ = os.add_dll_directory(_vtk_dll_directory)
print()
print("Build tree...")
# Build tree support.
try:
from . import _build_paths
# Add any paths needed for the build tree.
for path in _build_paths.paths:
if os.path.exists(path):
print(f"Path: {path}")
_ = os.add_dll_directory(path)
except ImportError:
# Relocatable install tree (or non-Windows).
print("Passing: Relocatable install tree")
pass
print()
print("Imported!")
# CPython 3.8 added behaviors which modified the DLL search path on Windows to
# only search "blessed" paths. When importing SMTK, ensure that SMTK's DLLs are
# in this set of "blessed" paths.
if sys.version_info >= (3, 8) and sys.platform == "win32":
_windows_dll_path()
# ------------------------------------------------------------------------------
# this little trick is for static builds of VTK. In such builds, if
# the user imports this Python package in a non-statically linked Python
# interpreter i.e. not of the of the VTK-python executables, then we import the
# static components importer module.
try:
from . import vtkCommonCore
except ImportError:
import _vtkmodules_static
# ------------------------------------------------------------------------------
# get the version
__version__ = "9.0.3"
Attempting to import again, I get the same error, so DLLs are still missing:
Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Folder Search:
==============
Lib/site-packages
Lib
==============
VTK DLL Directory: C:\Program Files\Python38\bin
Adding DLLs...
DLL: C:\Program Files\Python38\bin
Build tree...
Passing: Relocatable install tree
Imported!
Traceback (most recent call last):
File "C:\Program Files\Python38\lib\site-packages\vtkmodules\__init__.py", line 69, in <module>
from . import vtkCommonCore
ImportError: DLL load failed while importing vtkCommonCore: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files\Python38\lib\site-packages\vtk.py", line 30, in <module>
all_m = importlib.import_module('vtkmodules.all')
File "C:\Program Files\Python38\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Program Files\Python38\lib\site-packages\vtkmodules\__init__.py", line 71, in <module>
import _vtkmodules_static
ModuleNotFoundError: No module named '_vtkmodules_static'

Update: 26-AUG-2021 (Continued...)
NOTE: This is NOT an answer.
Pretty much doing the exact same thing, but in vtk.py instead, plus including all other system dependencies, using os.add_dll_directory() does not work:
"""This is the vtk module."""
import sys
if sys.version_info < (3, 5):
# imp is deprecated in 3.4
import imp
import importlib
# import vtkmodules package.
vtkmodules_m = importlib.import_module("vtkmodules")
# import vtkmodules.all
all_m = importlib.import_module("vtkmodules.all")
# create a clone of the `vtkmodules.all` module.
vtk_m = imp.new_module(__name__)
for key in dir(all_m):
if not hasattr(vtk_m, key):
setattr(vtk_m, key, getattr(all_m, key))
# make the clone of `vtkmodules.all` act as a package at the same location
# as vtkmodules. This ensures that importing modules from within the vtkmodules package
# continues to work.
vtk_m.__path__ = vtkmodules_m.__path__
# replace old `vtk` module with this new package.
sys.modules[__name__] = vtk_m
else:
if sys.version_info >= (3, 8):
import os
if os.name == "nt":
WIN_DLLS = set([r"C:\WINDOWS\System32\downlevel"])
PY_ROOT = sys.exec_prefix
PY_DLLS = set(
[
os.path.join(PY_ROOT, "DLLs"),
]
)
try:
VTK_DIR = os.environ["VTK_DIR"]
except KeyError:
VTK_DIR = None
VTK_DLLS = set([])
if VTK_DIR is not None:
# Assumes default folder "bin" used in CMake configuration
for root, dirs, files in os.walk(os.path.join(VTK_DIR, "bin")):
for file_ in files:
if file_.lower().endswith(".dll") or file_.lower().endswith(
".pyd"
):
if root not in VTK_DLLS:
VTK_DLLS.add(root)
break
try:
VULKAN_PATH = os.environ["VK_SDK_PATH"]
except KeyError:
VULKAN_PATH = None
VULKAN_DLLS = set([])
if VULKAN_PATH is not None:
for root, dirs, files in os.walk(VULKAN_PATH):
for file_ in files:
if file_.lower().endswith(".dll"):
if root not in VULKAN_DLLS:
VULKAN_DLLS.add(root)
break
try:
CUDA_PATH = os.environ["CUDA_PATH"]
except KeyError:
CUDA_PATH = None
CUDA_DLLS = set([])
if CUDA_PATH is not None:
for root, dirs, files in os.walk(CUDA_PATH):
for file_ in files:
if file_.lower().endswith(".dll"):
if root not in CUDA_DLLS:
CUDA_DLLS.add(root)
break
try:
ONEAPI_ROOT = os.environ["ONEAPI_ROOT"]
except KeyError:
ONEAPI_ROOT = None
ONEAPI_DLLS = set([])
if ONEAPI_ROOT is not None:
for root, dirs, files in os.walk(CUDA_PATH):
for file_ in files:
if file_.lower().endswith(".dll"):
if root not in ONEAPI_DLLS:
ONEAPI_DLLS.add(root)
break
try:
ISPC_EXECUTABLE = os.environ["ISPC_EXECUTABLE"]
except KeyError:
ISPC_EXECUTABLE = None
ISPC_DLLS = set([])
if ISPC_EXECUTABLE is not None:
ISPC_DLLS.add(os.path.dirname(ISPC_EXECUTABLE))
# INCLUDE = os.path.join(PY_ROOT, "include")
# LIB = os.path.join(PY_ROOT, "Lib")
# SITEPACKAGES = os.path.join(PY_ROOT, "Lib", "site-packages")
# LIBS = os.path.join(PY_ROOT, "libs")
# SCRIPTS = os.path.join(PY_ROOT, "Scripts")
dll_directories = [
WIN_DLLS,
PY_DLLS,
VTK_DLLS,
VULKAN_DLLS,
CUDA_DLLS,
ONEAPI_DLLS,
ISPC_DLLS,
]
print(f"WIN_DLLS: {WIN_DLLS}")
print(f"PY_DLLS: {PY_DLLS}")
print(f"VTK_DLLS: {VTK_DLLS}")
print(f"VULKAN_DLLS: {VULKAN_DLLS}")
print(f"CUDA_DLLS: {CUDA_DLLS}")
print(f"ONEAPI_DLLS: {ONEAPI_DLLS}")
print(f"ISPC_DLLS: {ISPC_DLLS}")
for dll in dll_directories:
if dll is not None:
for dll_ in dll:
os.add_dll_directory(dll_)
import importlib
# import vtkmodules.all
all_m = importlib.import_module("vtkmodules.all")
# import vtkmodules
vtkmodules_m = importlib.import_module("vtkmodules")
# make vtkmodules.all act as the vtkmodules package to support importing
# other modules from vtkmodules package via `vtk`.
all_m.__path__ = vtkmodules_m.__path__
# replace old `vtk` module with the `all` package.
sys.modules[__name__] = all_m
gives output
❯ py
Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
WIN_DLLS: {'C:\\WINDOWS\\System32\\downlevel'}
PY_DLLS: {'C:\\Program Files\\Python38\\DLLs'}
VTK_DLLS: {'C:\\VTK\\bin\\Lib\\site-packages\\mpi4py', 'C:\\VTK\\bin', 'C:\\VTK\\bin\\Lib\\site-packages\\vtkmodules'}
VULKAN_DLLS: {'C:\\VulkanSDK\\1.2.182.0\\Tools', 'C:\\VulkanSDK\\1.2.182.0\\Tools32\\styles', 'C:\\VulkanSDK\\1.2.182.0\\Third-Party\\Bin32', 'C:\\VulkanSDK\\1.2.182.0\\Tools\\styles', 'C:\\VulkanSDK\\1.2.182.0\\Tools32\\iconengines', 'C:\\VulkanSDK\\1.2.182.0\\Tools\\imageformats', 'C:\\VulkanSDK\\1.2.182.0\\Tools\\platforms', 'C:\\VulkanSDK\\1.2.182.0\\Tools32\\bearer', 'C:\\VulkanSDK\\1.2.182.0\\Bin32', 'C:\\VulkanSDK\\1.2.182.0\\Tools\\bearer', 'C:\\VulkanSDK\\1.2.182.0\\Bin', 'C:\\VulkanSDK\\1.2.182.0\\Third-Party\\Bin', 'C:\\VulkanSDK\\1.2.182.0\\Tools32', 'C:\\VulkanSDK\\1.2.182.0\\Tools32\\imageformats', 'C:\\VulkanSDK\\1.2.182.0\\Tools\\iconengines', 'C:\\VulkanSDK\\1.2.182.0\\Tools32\\platforms'}
CUDA_DLLS: {'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\extras\\visual_studio_integration\\MSBuildExtensions', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\extras\\demo_suite', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\extras\\CUPTI\\lib64', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\nvvm\\bin', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\compute-sanitizer', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\libnvvp\\plugins\\org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\bin'}
ONEAPI_DLLS: {'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\extras\\visual_studio_integration\\MSBuildExtensions', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\extras\\demo_suite', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\extras\\CUPTI\\lib64', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\nvvm\\bin', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\compute-sanitizer', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\libnvvp\\plugins\\org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\bin'}
ISPC_DLLS: {'C:\\Program Files\\ISPC\\ispc-v1.16.1-windows\\bin'}
Traceback (most recent call last):
File "C:\Program Files\Python38\lib\site-packages\vtkmodules\__init__.py", line 13, in <module>
from . import vtkCommonCore
ImportError: DLL load failed while importing vtkCommonCore: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files\Python38\lib\site-packages\vtk.py", line 143, in <module>
all_m = importlib.import_module("vtkmodules.all")
File "C:\Program Files\Python38\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Program Files\Python38\lib\site-packages\vtkmodules\__init__.py", line 15, in <module>
import _vtkmodules_static
ModuleNotFoundError: No module named '_vtkmodules_static'
>>>

This is also not an answer - but a short recipe for what worked for me.
I use Python 3.8.8 from python.org - should not make any difference.
I use VTK-9.0.1 - before they attempted to fix the os.add_dll_directory issue.
I use Qt5.15.2
I am not using vcpkg
Make sure the right DLLs are found
In [1]: import sys
In [2]: sys.path.insert(0, "c:/{VTK_BUILD_DIR}/bin/Lib/site-packages/")
Make sure that they are loaded and no errors occur (some are silent)
In [5]: import os
In [6]: os.add_dll_directory("c:/{VTK_BUILD_DIR}/bin/Lib/site-packages/vtkmodules/")
In [7]: os.add_dll_directory("c:/{VTK_BUILD_DIR}/bin/Release/")
In [8]: os.add_dll_directory("c:/QT/Qt5.15/5.15.2/msvc2019_64/bin/")
This is made on top of an empty virtual environment, where only ipython is added.

I was putting my generated CMake files into C:\VTK\build\ and then compiling and installing in C:\VTK, which is why Python couldn't find the DLLs. I have corrected the corresponding step in my post so the install instructions are complete and correct.
See section Steps to Reproduce, step 3

Related

MacOS, c++ python extension cannot find a dynamic library

I'm building a c++ python extension. So far, I created versions for Linux and Windows. Currently, I'm struggling with the MacOS version. CMake produces 2 libraries:
43898860 Aug 29 13:40 libslide_io.dylib
214876 Aug 29 13:40 slideio.cpython-35m-darwin.so
I pack them to a whl file. After installation with pip, when I try to import the package, I'm getting the following error:
(sld-35)dist % python -c "import slideio"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: dlopen(/Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/slideio.cpython-35m-darwin.so, 2): Library not loaded: libslide_io.dylib
Referenced from: /Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/slideio.cpython-35m-darwin.so
Reason: image not found
Both libraries are correctly placed to the python environment directory:
/Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/
Moreover, if I unzip the whl file and import the package from the current directory, it works.
I suspect that it is something with rpath settings in my cmake files, but cannot find the correct solution. All my attempts are failed.
I would appreciate any help.
If somebody is interested. I solved the problem by adding a post-build with the execution of install_name utility with the #loader_path parameter. The command instructs the system looking for the library at the loader (in my case - the python package) folder.
if (APPLE)
add_custom_command(TARGET ${BINDLIB_NAME}
POST_BUILD COMMAND
${CMAKE_INSTALL_NAME_TOOL} -change libslide_io.dylib #loader_path/libslide_io.dylib
$<TARGET_FILE:${BINDLIB_NAME}>)
endif()

Failed to load GLFW3 shared library on Windows

I am trying to run python code which imports GLFW3 library. It gives this error:
''' File ".\run_simulation.py", line 15, in
from modules.graphics.simulator_visualizer import SimulatorVisualizer File
"F:\webapp\p1_aksu_rubius\utils..\modules\graphics\simulator_visualizer.py",
line 2, in
import glfw File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\glfw__init__.py",
line 34, in < module>
raise ImportError("Failed to load GLFW3 shared library.") ImportError: Failed to load GLFW3 shared library. '''
I use Windows Server 2012 R2 and Python version 3.7.4 (64 bit)
I found that there is glfw3.dll file in the directory C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\glfw. Then I add this path into Windows environment variable - PATH. Anyway is does not help. What is the problem?
Thanks!
If the .dll is there (it's in the right place), then it was found. Adding its path to %PATH% won't do anything. It's one (at least) of its dependencies that's missing.
According to [PyPI]: GLFW:
Windows
The GLFW shared library is included in the Python wheels for Windows, but the correct Microsoft Visual C++ Redistributable will be required:
VC 2010 for 32-bit Python, or
VC 2012 for 64-bit Python.
So, you need [MS]: Visual C++ Redistributable for Visual Studio 2012 Update 4 (x64). Select VSU_4\vcredist_x64.exe, download, install it and try again.
Let me know how it works out.

Installing Theano with GPU on Windows 8.1 64-bit with Visual Studio 2013 [duplicate]

This question already has answers here:
How do I install theano in Anaconda ver. 2.1 Windows 64 bit for Python 3.4?
(3 answers)
Closed 6 years ago.
This Theano Installation is making me mad :(
So, I've followed the instructions here on the most voted answer because it seemed like the most similar condiguration from mine and up-to-date version : Installing theano on Windows 8 with GPU enabled
1- I've installed Cuda v6.5, launched deviceQuery and it works fine.
2- I already have Visual Studio 2013 so I haven't installed Visual Studio 2010
3- > At the time of writing, Theano on GPU only allows working with 32-bit floats and is primarily built for 2.7 version of Python.
So i don't know exactly what is the current state now but I have a friend with the same configuration than mine and he managed to make it work so I guess it's possible. I've installed Python through Anaconda.
4- I've installed MinGW and Cygwin
5- I've fixed msvc9compiler.py
6- Here's the bottleneck : the PyCUDA Installation
Here's what I've done:
- I've used cygwin to extract the pycuda tar file
- I've executed python configure.py through VS2013 x64 Native Tools Command Prompt than configured siteconfig.py as followed:
BOOST_INC_DIR = []
BOOST_LIB_DIR = []
BOOST_COMPILER = 'gcc43'
USE_SHIPPED_BOOST = True
BOOST_PYTHON_LIBNAME = ['boost_python']
BOOST_THREAD_LIBNAME = ['boost_thread']
CUDA_TRACE = False
CUDA_ROOT = 'C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v6.5'
CUDA_ENABLE_GL = False
CUDA_ENABLE_CURAND = True
CUDADRV_LIB_DIR = ['${CUDA_ROOT}/lib', '${CUDA_ROOT}/lib/x64']
CUDADRV_LIBNAME = ['cuda']
CUDART_LIB_DIR = ['${CUDA_ROOT}/lib', '${CUDA_ROOT}/lib/x64']
CUDART_LIBNAME = ['cudart']
CURAND_LIB_DIR = ['${CUDA_ROOT}/lib', '${CUDA_ROOT}/lib/x64']
CURAND_LIBNAME = ['curand']
CXXFLAGS = ['/EHsc', '-DBOOST_NO_TYPEID']
LDFLAGS = ['/FORCE']
I've executed python setup.py build --compiler="msvc" through VS2013 x64 Native Tools Command Prompt
I've executed python setup.py install through VS2013 x64 Native Tools Command Prompt
When I execute the little test in python, here's what's happening:
PS C:\users\jmm\desktop> python test.py
Vendor: Continuum Analytics, Inc.
Package: mkl
Message: trial mode expires in 29 days
Traceback (most recent call last):
File "test.py", line 7, in <module>
a_doubled = (2*a_gpu).get()
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\gpuarray.py", line 471, in __rm
ul__
return self._axpbz(scalar, 0, result)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\gpuarray.py", line 333, in _axp
bz
func = elementwise.get_axpbz_kernel(self.dtype, out.dtype)
File "<string>", line 2, in get_axpbz_kernel
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\tools.py", line 423, in context
_dependent_memoize
result = func(*args)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\elementwise.py", line 417, in g
et_axpbz_kernel
"axpb")
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\elementwise.py", line 157, in g
et_elwise_kernel
arguments, operation, name, keep, options, **kwargs)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\elementwise.py", line 143, in g
et_elwise_kernel_and_types
keep, options, **kwargs)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\elementwise.py", line 71, in ge
t_elwise_module
options=options, keep=keep)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\compiler.py", line 251, in __in
it__
arch, code, cache_dir, include_dirs)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\compiler.py", line 241, in comp
ile
return compile_plain(source, options, keep, nvcc, cache_dir)
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\compiler.py", line 73, in compi
le_plain
checksum.update(preprocess_source(source, options, nvcc).encode("utf-8"))
File "C:\Users\jmm\Anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\compiler.py", line 52, in prepr
ocess_source
cmdline, stderr=stderr)
pycuda.driver.CompileError: nvcc preprocessing of c:\users\jmm\appdata\local\temp\tmp32jnzb.cu failed
[command: nvcc --preprocess -arch sm_30 -m64 -Ic:\users\jmm\anaconda\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg
\pycuda\cuda c:\users\jmm\appdata\local\temp\tmp32jnzb.cu --compiler-options -EP]
[stderr:
tmp32jnzb.cu
'C:\Program' n'est pas reconnu en tant que commande interne
ou externe, un programme exécutable ou un fichier de commandes.
]
Could you please tell me why the hell this doesn't work ?
I have written a practical guide for the whole process:
https://my6266blog.wordpress.com/2015/01/21/installing-theano-pylearn2-and-even-gpu-on-windows/
Good luck! It's not that complicated, just follow the steps one by one.
You probably need to add the path to the executables for Visual Studio in your nvcc.profile
(you can find it in your CUDA bin folder. On my system: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v6.5\bin).
In my case, since I have Visual studio 2010, I added at the end of nvcc.profile:
"compiler-bindir = C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64"
Actually you don't need to install pycuda to get theano working on your windows machine. I'm not an expert but I have Theano installed on Windows 8.1.
This is my laptop config: 64-bit, nvcc/cuda 6.5, Python 2.7.9, WinPython-64bit-2.7.9.3, Windows 8.1, VS2013 and two graphic units (Intel HD Graphics 4600 and NVIDIA GeForce GT 750M).
The process was a pretty big hassle, so here is a tutorial for anyone that is interested:
All of this has been tested using a clean install of Windows 8.1, with nothing else on it, though it should work fine if you don't have a clean install because this will install all the required versions of the software for you.
You need 64-bit windows, 32 bit will not work. You will also need a CUDA compatible graphics card, so if you don’t have one you’re stuck for now, sadly. This means that you need a relatively modern NVIDIA graphics card, AMD will not work (it can run OpenCL but not CUDA because CUDA is lame and proprietary).
I am installing this on Windows 8.1, but I suspect it should all still work on windows 7 as well.
First download WinPython, (make sure to get python 2.7, version 2.7.10.3, this link points to there) and install it TO A PATH THAT DOES NOT HAVE SPACES IN IT. OTHERWISE THINGS BREAK. I made an Other folder in C:\ (C:\Other) and then made a folder named Python27 (C:\Other\Python27) and tell the installer to install it in there.
Once it is done installing, you will need to add it to your path. Press the windows key and type environment variables, then click “Edit the System Environment Variables”, click Environment Variables in the windows that pops up, scroll down to Path, and then append
C:\Other\Python27\python-2.7.10.amd64
Or wherever else you installed WinPython to
Then add a semicolon after it, so you get
C:\Other\Python27\python-2.7.10.amd64;
This is how you add a specific path to the Path variables, in the future, I will just say to add it here and now give specific steps about how to do that. Note that if you update the system path, the current command prompt windows that are open won’t get that update, and you will have to open a new command prompt window to actually have it use the new path.
The purpose of a path is so your command prompt window knows where programs are, because if you call, say
python
in the command prompt, it will look through every folder in your path until it finds a python.exe. If it can’t find any, it will get angry as it typically would if that program didn’t exist.
If you don’t want to clutter your path variable/if your path variable is full, I put a tutorial here about how to make it so your path is just appended to when you open a command prompt window via a text file that stores all the paths, instead of having to edit the environment variable itself, if you are interested.
You then need to add
C:\Other\Python27\python-2.7.10.amd64\DLL;
C:\Other\Python27\python-2.7.10.amd64\Scripts;
to your path as well (again, or wherever else you installed python. For later on I will just say where I installed it and if you installed it somewhere else it should be pretty easy to just tweak the commands accordingly)
Next, install visual studio 2015 community and visual studio 2013, making sure to install all the tools related to c++ development as well (using custom installation, then under Programming Languages). These don’t need to be in a path without spaces, and they probably won’t let you store them anywhere else anyway and that is OK.
Add
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin;
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\amd64;
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib;
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\amd64;
To your system path.
Install NOT NEWEST DRIVER BECAUSE THEY ARE UNSTABLE, but instead 355.60 because it is known to be very reliable, and new enough. Then install the CUDA toolkit (it’s also okay to store this in a path with spaces, it probably won’t give you the option either, but even if it does, just let it store it in the default place it wants to store it to). Version 6.5 is needed because version 7 and above aren’t supported by pycuda. If you have a GTX 9__ you will need to download CUDA from here instead.
This will probably automatically append
C:\Program Files (x86)\Windows Kits\8.1\Redist\D3D\x64;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5\libnvvp;
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
To your path, if not you will need to do so now.
Those are the only three things that can be stored to paths with spaces (these will be in like Program Files or Program Files (x86)), with everything else be very careful to store them to paths that don’t have spaces.
Download the boost binaries (1_55_0 for 64 bit which is the version this link points to), and run the installer, then select to store them to a path without spaces (I stored them to C:\Other\boost)
Navigate to that directory in the command line, then run
bootstrap.bat
and then when it is done run
.\b2
This will start building, and take a long time, and use a lot of space (about 6 GB).
It will probably say that 8 targets failed, 8 targets were skipped, and 1075 were updated. This is what one should expect, and is not a problem.
Install Git-2.7.0-64-bit to some path without a space in it
Choose to use Git from the Windows Command Prompt, checkout Windows-style, commit Unix-style endings, use Window’s default console window, and do not enable file system caching.
Add
C:\Other\Git\bin;
To your system path
Next, run the installer for VCForPython.
Download pycuda source (pycuda-2015.1.3)
Navigate inside that directory, then run
python configure.py
this will create a file named siteconf.py.
Open this file, and it should look something like
BOOST_INC_DIR = []
BOOST_LIB_DIR = []
BOOST_COMPILER = 'gcc43'
USE_SHIPPED_BOOST = True
BOOST_PYTHON_LIBNAME = ['boost_python-py27']
BOOST_THREAD_LIBNAME = ['boost_thread']
CUDA_TRACE = False
CUDA_ROOT = 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v6.5'
CUDA_INC_DIR = ['${CUDA_ROOT}/include']
CUDA_ENABLE_GL = False
CUDA_ENABLE_CURAND = True
CUDADRV_LIB_DIR = ['${CUDA_ROOT}/lib/Win32', '${CUDA_ROOT}/lib/x64']
CUDADRV_LIBNAME = ['cuda']
CUDART_LIB_DIR = ['${CUDA_ROOT}/lib/Win32', '${CUDA_ROOT}/lib/x64']
CUDART_LIBNAME = ['cudart']
CURAND_LIB_DIR = ['${CUDA_ROOT}/lib/Win32', '${CUDA_ROOT}/lib/x64']
CURAND_LIBNAME = ['curand']
CXXFLAGS = []
LDFLAGS = []
modify it so it looks like:
BOOST_INC_DIR = ['C:/Other/boost']
BOOST_LIB_DIR = ['C:/Other/boost/lib64-msvc-12.0']
BOOST_COMPILER = 'msvc'
USE_SHIPPED_BOOST = True
BOOST_PYTHON_LIBNAME = ['boost_python-vc120-mt-1_55']
BOOST_THREAD_LIBNAME = ['boost_thread-vc110-mt-1_55']
CUDA_TRACE = False
CUDA_ROOT = 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v6.5'
CUDA_ENABLE_GL = False
CUDA_ENABLE_CURAND = True
CUDADRV_LIB_DIR = ['${CUDA_ROOT}/lib/Win32', '${CUDA_ROOT}/lib/x64']
CUDADRV_LIBNAME = ['cuda']
CUDART_LIB_DIR = ['${CUDA_ROOT}/lib/Win32', '${CUDA_ROOT}/lib/x64']
CUDART_LIBNAME = ['cudart']
CURAND_LIB_DIR = ['${CUDA_ROOT}/lib/Win32', '${CUDA_ROOT}/lib/x64']
CURAND_LIBNAME = ['curand']
CXXFLAGS = ['/DBOOST_PYTHON_STATIC_LIB', '/EHsc']
LDFLAGS = ['/LIBPATH:C:\\Other\\boost\\/lib64-msvc-12.0', '/FORCE']
then run
python setup.py build
followed by
python setup.py install
This should install pycuda for you =)
To install Theano (with GPU enabled), download release 0.7, unzip it, navigate inside it using a command prompt, and then type
python setup.py install
then go and edit system environment variables, and create one named
THEANO_FLAGS
and set it’s value to
device=gpu,floatX=float32
Then open a new command prompt, and if you have completed all of the steps above, this should work well =) You can run the code here to make sure that you are actually running off the GPU.
I was able to get Theano installed on my ASUS K501LX Windows 8.1 laptop, with an NVIDIA GeForce 950M GPU, without any hassle whatsoever. I largely followed Maor's post above from March 29th. I was actually shocked at how easy it was! All I needed was the Community Edition of Visual Studio 2013 and the CUDA 7 Toolkit. I then installed Anaconda 3.4 (I used the latest version that's out there now). The one modification I made to Maor's post was installing mingw, via conda install mingw libpython immediately after installing Anaconda. Also, since I am using Python 3, I had to change the flags parameter in .theanorc.txt to point to C:\Anaconda3\libs.
Upon importing theano, it returned that it was using my GeForce GTX 950M device, and running the theano\misc\check_blas.py check returned no errors and carried out its tests on my GPU.
Happy times!

python opencv dll load failed Missing environment variables?

I want to use Blender 2.72b with OpenCV, so I had to build it for Python3 (I have 3.4.2 installed since Blender is using that too). I'm working on Win7 64bit, so I used a 64bit version of Python and NumPy (Blender too of course). For OpenCV I used 3.0.0-beta.
For building I used CMake 2.8 and Visual Studio 2010 Professional (since I have a student-license > Visual Studio 10 Win64 in CMake).
I followed these instructions (lower ones), modified for python3.
These are my folders:
Target build-folder for CMake (OpenCV.sln executed and build in here too of course):
C:\Users\Gunnar\Desktop\build\
OpenCV-source-folder:
C:\Users\Gunnar\Downloads\opencv\sources\
Python installed here:
C:\Python34\
The build went fine, I adjusted the following before generating with CMake (python2 left blank):
PYTHON3_EXECUTABLE -> C:/Python34/python.exe
PYTHON3_INCLUDE_DIR -> C:/Python34/include
PYTHON3_LIBRARY -> C:/Python34/libs/python34.lib
PYTHON3_NUMPY_INCLUDE_DIRS -> C:/Python34/Lib/site-packages/numpy/core/include
PYTHON3_PACKAGES_PATH -> C:/Python34/Lib/site-packages
BUILD_opencv_python3 -> true
The build-folder has a cv2.pyd at
C:\Users\Gunnar\Desktop\build\lib\Release\
The cv2.pyd got copied to
C:\Python34\Lib\site-packages\
automatically and I copied it to
M:\Programme\blender-2.72b-windows64\2.72\python\lib\site-packages\
If I now open the python IDLE, I get the following error (same in Blender of course):
>>> import cv2
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import cv2
ImportError: DLL load failed: Das angegebene Modul wurde nicht gefunden.
(in english: DLL load failed: Module not found.)
I'm assuming that I'm missing environment variables (from what I've found googling for it).
Looking into them I have no corresponding entry for PATH and no PYTHONPATH.
My problem now is that I don't know what I need to set there. I never really had to change something in there and I don't know what exactly is needed.
If more information are needed let me know.
EDIT:
I got i working now. I'm not quite sure, which one it did in the end, but I first used
cmd > setx -m OPENCV_DIR C:\Users\Gunnar\Desktop\build
and than edited the PATH-variable with these:
C:\Users\Gunnar\Desktop\build;
C:\Users\Gunnar\Desktop\build\lib\Release;
C:\Users\Gunnar\Desktop\build\x64\Release;
C:\Users\Gunnar\Desktop\build\bin\Release
And right now it works even though I removed all of them...

Help installing cx_Oracle

I'm trying to install the cx_Oracle for Python 2.6, but it is failing. I don't know enough about C or MS Vis. Studio's compiler to even approach fixing it myself.
This is what is output on the command line:
C:\pydev\cx_Oracle-5.0.1>C:\python26\python setup.py install
running install
running build
running build_ext
building 'cx_Oracle' extension
C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Oracle\instantclient\sdk\include -IC:\p
ython26\include -IC:\python26\PC /Tccx_Oracle.c /Fobuild\temp.win32-2.6-11g\Release\cx_Oracle.obj -DBUILD_VERSION=5.0.1
cx_Oracle.c
c:\pydev\cx_oracle-5.0.1\StringVar.c(392) : warning C4018: '>' : signed/unsigned mismatch
c:\pydev\cx_oracle-5.0.1\StringVar.c(417) : warning C4018: '>' : signed/unsigned mismatch
c:\pydev\cx_oracle-5.0.1\ObjectVar.c(117) : warning C4018: '<' : signed/unsigned mismatch
c:\pydev\cx_oracle-5.0.1\ObjectVar.c(134) : warning C4018: '<' : signed/unsigned mismatch
c:\pydev\cx_oracle-5.0.1\Variable.c(331) : error C2036: 'void *' : unknown size
c:\pydev\cx_oracle-5.0.1\Variable.c(878) : warning C4244: '=' : conversion from 'sb2' to 'sb1', possible loss of data
c:\pydev\cx_oracle-5.0.1\Variable.c(914) : warning C4018: '<' : signed/unsigned mismatch
error: command '"C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe"' failed with exit status 2
Does anyone have any idea how to fix this?
(I also may have an issue with that cl.exe file, I think I remember seeing this as a problem somewhere else, but I have four Windows machines which could have had issue there.).
----------------------------- EDIT -----------------------------
The reason I do not use the pre-compiled installer is because when I try to run the module I get the following:
Python 2.6.3 (r263rc1:75186, Oct 2 2009, 20:40:30) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
Traceback (most recent call last):
File "", line 1, in
ImportError: DLL load failed: The specified module could not be found.
I assume that means that it has not been properly installed. If someone can tell me what I did wrong here, that would also be a perfectly acceptable answer.
----------------------------- EDIT2 -----------------------------
I have retrieved all files from Oracle's Basic Client version 11.1.0.7. I have not tried installing a more substantial client. I am downloading one now.
%ORACLE_HOME%=C:\Oracle\instantclient\ #The directory where the above files have been extracted.
More information
The current directory for the cx_Oracle's setup.py is C:\pydev\cx_Oracle-5.0.1
The version of Python is 2.6.3
The current directory for Python is C:\Python26 -- to access python 2.6 -> type py26 on the command line and it will map to C:\python26\python
Why don't you use a binary package like Windows Installer (Oracle 10g, Python 2.6)?
See http://cx-oracle.sourceforge.net/ for other binary packages
Addendum (as requested): you must ensure to use the cx_Oracle that :
set ORACLE_HOME if this environment variable doesn't exist (see this Oracle FAQ)
python can find oraocci11.dll for Oracle 11g, oraocci10.dll for Oracle 10g or oraclient9.dll for Oracle 9i by adding the folder (should be %ORACLE_HOME%/bin) where the DLL is in you PATH environment variable
use the correct binary package (i.e. built for the right Python/Oracle versions)
You also need some type of Oracle client installed on your machine, since cx_Oracle is just a bridge between Python and the Oracle Client. Valid Oracle Clients include a full Oracle installation (like Standard or XE) or the Instant Client
From the README:
Please note that an Oracle client (or
server) installation is required in
order to use cx_Oracle. If you do not
require the tools that come with a
full client installation, it is
recommended to install the Instant
Client which is far easier to install.
I have not installed on Windows before, but you may have to set the ORACLE_HOME environment variable so that cx_Oracle knows where to look.
If you use Anaconda Python x64, things are easier:
conda install -c https://conda.anaconda.org/anaconda cx_oracle
(It will install the right Oracle client)
If you get the following error:
[Anaconda2] C:\Users\User>conda install -c https://conda.anaconda.org/anaconda c
x_oracle
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ......
Solving package specifications: .................
Package plan for installation in environment C:\Anaconda2:
The following NEW packages will be INSTALLED:
cx_oracle: 5.1.2-py27_0
oracle-instantclient: 11.2.0.4.0-0
The following packages will be UPDATED:
conda: 3.19.1-py27_0 --> 4.0.5-py27_0
pip: 8.0.2-py27_0 --> 8.1.1-py27_0
pyyaml: 3.11-py27_2 --> 3.11-py27_3
setuptools: 19.6.2-py27_0 --> 20.3-py27_0
wheel: 0.26.0-py27_1 --> 0.29.0-py27_0
Proceed ([y]/n)? y
Unlinking packages ...
An unexpected error has occurred, please consider sending the | 0%
following traceback to the conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Include the output of the command 'conda info' in your report.
Traceback (most recent call last):
File "C:\Anaconda2\Scripts\conda-script.py", line 4, in <module>
sys.exit(main())
File "C:\Anaconda2\lib\site-packages\conda\cli\main.py", line 173, in main
args_func(args, p)
File "C:\Anaconda2\lib\site-packages\conda\cli\main.py", line 180, in args_fun
c
args.func(args, p)
File "C:\Anaconda2\lib\site-packages\conda\cli\main_install.py", line 45, in e
xecute
install.install(args, parser, 'install')
File "C:\Anaconda2\lib\site-packages\conda\cli\install.py", line 423, in insta
ll
plan.execute_actions(actions, index, verbose=not args.quiet)
File "C:\Anaconda2\lib\site-packages\conda\plan.py", line 538, in execute_acti
ons
inst.execute_instructions(plan, index, verbose)
File "C:\Anaconda2\lib\site-packages\conda\instructions.py", line 148, in exec
ute_instructions
cmd(state, arg)
File "C:\Anaconda2\lib\site-packages\conda\instructions.py", line 95, in UNLIN
K_CMD
install.unlink(state['prefix'], arg)
File "C:\Anaconda2\lib\site-packages\conda\install.py", line 714, in unlink
os.unlink(meta_path)
WindowsError: [Error 5] Access is denied: 'C:\\Anaconda2\\conda-meta\\conda-3.19
.1-py27_0.json'
[Anaconda2] C:\Users\User>
Try running the Anaconda prompt as admin:

Categories

Resources