Develop Reference

Import numpy without installing

Is there a way to import numpy without installing it?
I have a general application built into an .exe with PyInstaller. The application has a plugin system which allows it to be extended through Python scripts. The plugin import system works fine for basic modules (lone .py files, classes, functions, and simple packages). Internally, it globs a plugin directory and then imports accordingly using __import__ or importlib.import_module.
The application is built with minimal dependencies in order to reduce the overall size of the executable. Also, it's not possible to know what dependencies a future plugin will require nor practical to include everything. However, some plugins will inevitably require dependencies. numpy is a good test case for solving this class of problems.
Here's what I've tried.
A wheel file is really just a directory. It can be added to sys.path and the contents imported.
import sys
sys.path.append(r"C:\path\to\numpy-1.16.3+mkl-cp36-cp36m-win_amd64.whl")
import numpy as np
The wheel file is read, but the import generates an error.
*** ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using c:\projects\zip_test\venv\Scripts\python.exe),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named 'numpy.core._multiarray_umath'
The puzzling part is that the wheel file contains a .pyd for _multiarray_umath.
C:\projects\zip_test\plugins\New folder\numpy\core>dir
Volume in drive C is OS
Volume Serial Number is FE3D-6596
Directory of C:\projects\zip_test\plugins\New folder\numpy\core
07/25/2019 02:37 PM <DIR> .
07/25/2019 02:37 PM <DIR> ..
....
04/22/2019 02:55 AM 101,376 _multiarray_tests.cp36-win_amd64.pyd
04/22/2019 02:55 AM 2,494,976 _multiarray_umath.cp36-win_amd64.pyd
....
74 File(s) 583,173,551 bytes
5 Dir(s) 309,925,851,136 bytes free
This feels like a pathing issue. Yet adding the direct path for core/ to sys.path generates the same error.
sys.path.append(r"C:\projects\zip_test\plugins\numpy-1.16.3+mkl-cp36-cp36m-win_amd64.whl\numpy\core")
What's the deal? Why can't Python find numpy.core._multiarray_umath?
In response to repsones:
The wheel file was obtained from Christoph Gohlke.
My operating system is Windows 10 Pro 64-bit.
I am running Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32 (installed via Python Foundation build 3.6.8150.0) in a venv. The "system" Python is not on PATH.
I can install the numpy-1.16.3+mkl-cp36-cp36m-win_amd64.whl file via pip and import numpy as np works fine. I then uninstall numpy via pip uninstall -y numpy.
Running dumpbin /dependents _multiarray_umath.cp36-win_amd64.pyd produces:
Microsoft (R) COFF/PE Dumper Version 14.22.27905.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file _multiarray_umath.cp36-win_amd64.pyd
File Type: DLL
Image has the following dependencies:
mkl_rt.dll
python36.dll
KERNEL32.dll
VCRUNTIME140.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-time-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
Summary
77000 .data
1000 .gfids
1C000 .pdata
30000 .rdata
3000 .reloc
1000 .rsrc
1BD000 .text
Of the dependencies, I can find mkl_rt.dll, python36.dll, and VCRUNTIME140.dll in the either the .whl or the venv. There are apparently two versions of KERNEL32.dll, one for x86 and one for x64. One of these resides in C:\Windows\System32.
The only information I could find about the api-ms-win-crt-*.dll are described on MSDN within the "Update for Universal C Runtime in Windows",
The Windows 10 Universal CRT is a Windows operating system component
that enables CRT functionality on the Windows operating system. This
update allows Windows desktop applications that depend on the Windows
10 Universal CRT release to run on earlier Windows operating systems.
These are for non-Windows 10 systems. There seems to be no "official" way to obtain them for a Windows 10 system. I was able to copy the dlls from a Windows 7 system. Naively putting them in the PYTHONPATH (not surprisingly) doesn't work. If they were to work at all, they would likely need to be registered. But 1) registering Windows 7 dlls on Windows 10, I am told, can make the system unstable and 2) definitely doesn't seem like a universal, portable solution.

First.
Build application with and without numpy. Check difference between builds.
Original error was: No module named 'numpy.core._multiarray_umath' Can means two things:
cannot found file
some dependence of this this pyd file is not satisfied. You may try to check it with http://dependencywalker.com/.

Yes, depending on how you define "install".
Numpy requires the use of .pyd files. A pyd file is essentially a dll. These are compiled code files which the Python interpreter references during run-time. Unfortunately, the Windows API calls used to load dll files requires them to exist at the filesystem level. When you try to import directly from a wheel, the pyd files (and the functions/classes they contain) aren't accessible. Hence, the error.
The (Relatively) Easy solution
One solution is to make the pyd files accessible on the filesystem. To do that, it's probably simplest to unpack the whole wheel file to disk1.
In that case, consider "how you define 'install'":
If you could run numpy straight from the wheel, wouldn't that mean numpy was installed?
What difference does it really make whether you're writing a wheel file or the contents of that wheel file?
Since the plugin system is downloading a wheel file onto the filesystem, the application must have write access. Download the wheel file and extract it into the plugins directory. As long as the plugin directory is on the PYTHONPATH (i.e. sys.path.append), the import will work. Smarts and so forth can be added from there. Of course, what level of smarts you need is a whole other issue. But that's the basic idea.
The Wizard's Path
Another solution is to create your own functionality to import a dll from memory. This is precisely the goal of Joachim Bauch's MemoryModule project. It seems the Py2Exe project used the MemoryModule at one point, through a module called zipextimporter.py. Unfortunately, the code is hard to find and seems outdated (Python 2.4). There is also a similar module called pymemimporter.py which is slightly more recent.
1 Numpy expects the pyd files to be in specific locations. If you want to extract only the pyd files, they'll need to be nested in the appropriate folders (e.g. numpy/core/_multiarray_umath.cp36-win_amd64.pyd). This is sufficient to import numpy, but unless the other modules are available, rather pointless.

How do I change the name of the python DLL when compiling a library like pillow?

Question:
I want to compile a third-party library like Pillow or Numbpy but I want to change the name of the python27.dll to corepython27.dll. How do I do this during the compile process? Is it something I need to change in the setup.py? Or the distutils library?
I should explain that I have no experience in compiling at all. I just know that I will need to make this change as I learn more about the basics of compiling.
Explanation:
Corel's PaintShop Pro uses an embedded python interpreter to run scripts inside the program. And I would like to be able to use third-party libraries like pillow and numpy but they always fail to load. The version of python that is included with PaintShop Pro is 2.7.5. I've made sure to download the appropriate versions of these libraries but it always fails with a "DLL module doesn't exist" type error.
Using a PE viewer I was able to see that other libraries like TKinter were using imports corepython27.dll instead of python27.dll like pillow was.
Also pillow for 2.7 was using msvcr90.dll but the custom version of the tkinter library included with PSP was compiled with msvcr110.dll. Do you think this will be an issue? Do I need to compile pillow with the appropriate version of msvcr DLL? Or is matching versions (2.7) and making sure it uses the correct python.dll (corepython27.dll) the only important thing?

You can create a symbolic link named corepython27.dll showing to the installed python27.dll. You can do this in your console via the command
MKLINK <path_to_corepython.dll> <path_to_python27.dll>

PostgreSQL unable to create plpythonu extension

I'm trying to write a function in PostgreSQL on Windows with a Python script in the body and i'm running into an error message when trying to create the plpythonu extension. The command I'm running is:
CREATE EXTENSION plpythonu;
Which produces the following error message:
ERROR: could not access file "$libdir/plpython2": No such file or directory
SQL state: 58P01
I also tried running:
CREATE EXTENSION plpython3u;
which results in this error:
ERROR: could not load library "C:/Program Files (x86)/PostgreSQL/9.2/lib/plpython3.dll": The specified module could not be found.
SQL state: 58P01
The plpython3.dll file exists at this location, but apparently is missing some critical dependency. I've searched everywhere and found nothing helpful on this. I have both Python 2 and 3 installed on the machine...

The newest (9.4 or later) binary installations from EnterpriseDB contain only plpython3u.dll. In versions 9.4 to 9.6 I had to install python 3.3 to get plpython3u run.
You can check which version of Python is needed by plpython3u.dll using Dependency Walker.

A full answer can be found:
https://postgresrocks.enterprisedb.com/t5/PostgreSQL/unable-to-install-pl-python-extension/m-p/4090
It assumes you have used stackbuilder to install the edb language pack.
Do check the commands for correctness in your installation.
E.g. path to postgresql data, install path of edb and python version.
When you use depency walker (depends.exe), only pay attention to the pythonxx.dll. With older PG versions, this may or may not agree to the version installed by the EDB languages package. For version 10.7, version 3.4 Python is required. For windows, the later 3.4 Python versions do not appear to have a msi installer. You may have to install 3.4.4, or try to upgrade PG 10 to the latest version (10.11) first. This version requires python 3.7, so then you can use the EDB download.
But the python version may already exist and be found.
could not load library plpython3.dll (here on stackoverflow) was somewhat close, but did not detail the environment vars needed.
the solution proposed does not require you to change env vars permanently, which is a great help when using several python installations.

"R6034 An application has made an attempt to load the C runtime library incorrectly" after pygtk being installed

I'm using python 2.7.9 and encountered a problem when installing pygtk.
It displayed "Runtime error!...R6034 An application has made an attempt to load the C runtime library incorrectly" when installing numpy/scipy after pygtk being installed.
I tried to figure it out by searching it in stackoverflow and found two similar questions: Runtime error R6034 in embedded Python application and An application has made an attempt to load the C runtime library incorrectly.
So following the first one, I deleted the path corresponding to msvcr90.dll, however, it still cannot work. Then I chose to simply delete msvcr90.dll; at this time, this error wasn't presented when installing numpy/scipy, however, these two modules cannot work when simply typing "importing numpy/scipy".
I also renamed gtk-2.0 following the second one. Then numpy and scipy can be successfully installed. But it displayed "Error processing line 3 of C:\Python27\lib\site-packages\pygtk.pth" when installing matplotlib using pip.
I'm really confused about it. Can anybody provide some methods to fix it?

I've installed Python and PyGTK on 5+ machines, at least two of them brand new, clean builds of Win 7.
I've got the An application has made an attempt to load the C runtime library incorrectly error whenever I install a Python package as a windows installer (rather than using pip) on all these machines. It's annoying, but has never made a jot of difference, both Python and Gtk function correctly.
You've deleted msvcr90.dll, and that is why you get your Error processing line 3... If you look at this file, you'll see that line 3 is import runtime, and if you look further into the 'runtime' package, you'll see that this then tries to find the missing dll.
I think your best bet is to try to restore the missing file. If it's still in your recycle bin - great!
If not, the best thing to do is reinstall the Visual C++ runtime library

I made this video to show my way: https://www.youtube.com/watch?v=s6jhR1VBfeU. I use Anaconda to embedded Python in my C++ application. I simply changed "msvcr90.dll" to "msvcr90.dll_hihi" in 3 folders:
C:\Users\your user\Anaconda2\Library\bin, C:\Users\your user\Anaconda2 and C:\Program Files\Intel\iCLS Client (for x64)

How to use OpenCV in Python?

I have just installed OpenCV on my Windows 7 machine. As a result, I get a new directory:
C:\OpenCV2.2\Python2.7\Lib\site-packages
In this directory, I have two files: cv.lib and cv.pyd.
Then I try to use the opencv from Python. I do the following:
import sys
sys.path.append('C:\OpenCV2.2\Python2.7\Lib\site-packages')
import cv
As a result I get the following error message:
File "<stdin>", line 1, in <module>
ImportError: DLL load failed: The specified module could not be found.
What am I doing wrong?
ADDED
As it was recommended here, I have copied content of C:\OpenCV2.0\Python2.6\Lib\site-packages to the C:\Python26\Lib\site-packages. It did not help.
ADDED 2
My environment variables have the following values:
Path=C:\Program Files\MiKTex\miktex\bin;C:\OpenCV2.2\bin;C:\Python26;
PYTHONPATH=C:\OpenCV2.2\Python2.7\Lib\site-packages
Do I need to change something? Do I need to add something?
ADDED 3
I think my question is general: How to use a library? Probably I need to find a *.ddl file somewhere? Then I need to use the name of the directory containing this file as a value to some environment variables? Or maybe I need to use sys.addpath? I also need to know how the way to call the library is related to the name of the file that contains the library.
ADDED 4
It is interesting that when I type import cv, I get:
ImportError: DLL load failed: The specified module could not be found.
But when I type import opencv I get:
ImportError: No module named opencv
ADDED 5
It has been suggested that I usthe e inconsistent version of python. In more details, OpenCV tries to use Python2.7 and I had Python2.6. So, I have installed Python 2.7. It makes difference. Now I do not have the old error message, but I have a new one:
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: numpy.core.multiarray failed to import
ADDED 6
I have managed to resolve the problem by installing numpy. It took some time because I did not realized that there are different numpy installer corresponding to different versions of Python. Some details can be found in my answer to my own question (see bellow).

The problem was resolved. The following steps has been done:
A new version of python (version 2.7) has been installed.
After that I still was unable to run OpenCV because I had some problems with the numpy library.
I tired to install numpy but the installer did not see my new version of the Python.
I deleted the old version of Python as well as links to the old version in the Path system vatriable.
After that numpy installer was not able to finish the installation.
I have realized that I need to run another numpy installer that is associated with the Python 2.7. It can be found here.
Finally everything worked. I was able to "import cv".

I suspect you have the same problem I've run into. If you have a 64-bit version of Python, it cannot load 32-bit DLLs. OpenCV currently only ships 32-bit binaries. If you want 64-bit .pyd and .dll files, you have to compile them yourself. There are some instructions on the OpenCV Wiki, but it's not for the faint of heart. Expect to have a substantial time investment.
The easiest solution is to:
Uninstall 64-bit Python
Install a 32-bit distribution.
The PythonXY distribution includes pyopencv -- a good set of OpenCV hooks. The only limitation is that it's 32-bit, so don't make plans to process gigapixel astronomy data with it! ;)
If you must have the 64-bit version, follow these instructions to get it OpenCV to compile with Visual Studio 2010. There's a discussion on stackoverflow that describes building 64-bit apps with VC Express.
EDIT: OpenCV now ships with 64-bit Python binaries. The .dll files need to go somewhere in your path (I put them in the scripts folder), and the .pyd files go in your site-packages directory.

I had trouble interfacing OpenCV with Python, and I was looking all over the place for help. Here's what worked for me. I basically followed this post: http://opencvpython.blogspot.com/2012/05/install-opencv-in-windows-for-python.html. After downloading and extracting OpenCV 2.4.6, you basically get a folder called "opencv" with a bunch of stuff in it. Navigate to build->python->2.7. Inside, there is only one file called "cv2.pyd". I copied this file and pasted it in "python-2.7.5\Lib\site-packages". I'm actually using the Spyder IDE, and it works fine. In the python interpreter, typing in "import cv" worked for me.

Maybe you should edit your environment variable
right click on the "My Computer" or something like this, click on properties.
In the properties window click on the Advanced tab.
Then, the environment variables button.
Change the path.