I installed in Anaconda the tk package which is 8.6.10. I run Python 3.8.5
When I run within Spyder IDE the simple script
import tkinter as tk
win = tk.Tk()
I got the classic message "TclError: Can't find a usable init.tcl in the following directories:"
In the list of directories reported I do have tcl.
I tried in the cmd from anaconda to run tclsh and wish and they are both working. By running wish and looking for the version it reports correctly Tcl and Tk 8.6.10
So I took the C:\ProgramData\Anaconda3\Library\lib\tcl8.6\init.tcl and moved it in C:/ProgramData/library as this is one of the directories I get in the error report.
When I do this I got another error "C:/ProgramData/library/init.tcl: version conflict for package "Tcl": have 8.6.9, need exactly 8.6.10"
So it is clear that the tclsh in bin and the init.tcl are not from the same version (why?) but how can now enforce a correct behavior of my python script looking for the right Tcl/Tk?
I was expecting this to work out of the box
p.s. THe directory "C:\ProgramData\Anaconda3\Library\lib\tcl8.6" is also in the list of directories that is returned by the error.
The problem is that the version of the DLLs that are being used (which might be TCL86.DLL and TK86.DLL on Windows, but the exact names can vary) must match exactly the version of their support files (the primary one is init.tcl, but there's a lot more besides). That's a hard requirement; the software checks it (rightly!) and definitely won't work without them. As the binary code is looking for the files from 8.6.9 and yet has the files from 8.6.10 (a version mismatch) it's not going to work. (I don't know exactly how Tkinter was built.)
But at least we know what files it is looking for, and exactly where to get them. The official release of Tcl and Tk 8.6.9 is at https://sourceforge.net/projects/tcl/files/Tcl/8.6.9/ so get tcl869-src.zip and tk869-src.zip from there, and copy the contents of the library directory in each ZIP to where you want them to be (presumably one of the directories that are searched for). You don't need to do anything fancier than that, except perhaps to delete the old mismatched files if they're earlier on the path.
I would like to install older versions of python for use with virtual environments, ideally from source as these versions are readily available on the official python website python 3.6 here however I can't seem to find any decent instructions.
The link above has version 3.6 full link to the tar.gz here. It downloads, I move into to program files and unzip it with 7zip:
C:\Program Files\7-Zip\7z.exe e Python-3.6.9.tgz
Then unpack the tar file:
tar -xvzf Python-3.6.9
There is no python.exe file in the unpacked directory to add to my path?
Am I missing a build step perhaps?
I found some more instructions here - where there really should be good instructions!, git is installed and setup, as is vs-code. I ran :
PCBuild\build.bat
The file 'PCBuild\pcbuild.sln' appears to have been successfully produced. Now what? I run the test described here
rt.bat -q
And I get: The system cannot find the path specified
For some reason when I run build.bat, a lot of what is printed to the seems to be mentioning python 3.8 which is now the default install on my system, its as if this method is not even pointing to the right version of python.
Please note I am looking specifically for native python approaches to this - NOT conda approaches as it is incompatible with a number of things I am doing.
I've tried; another SO queation, another SO question, here, and here, also here.
Many install guides seem to show only how to install the latest version of python using the python installer, which doesn't seem to be able to be used for legacy versions of python.
Also pythons official site has a link at the bottom labelled ' tools for unpacking archive files' which one would presume would help you to unpack, possibly even install archive files, however the link just takes you back to the top of the page?
Any help would be much appreciated?
Is there a way to import numpy without installing it?
I have a general application built into an .exe with PyInstaller. The application has a plugin system which allows it to be extended through Python scripts. The plugin import system works fine for basic modules (lone .py files, classes, functions, and simple packages). Internally, it globs a plugin directory and then imports accordingly using __import__ or importlib.import_module.
The application is built with minimal dependencies in order to reduce the overall size of the executable. Also, it's not possible to know what dependencies a future plugin will require nor practical to include everything. However, some plugins will inevitably require dependencies. numpy is a good test case for solving this class of problems.
Here's what I've tried.
A wheel file is really just a directory. It can be added to sys.path and the contents imported.
import sys
sys.path.append(r"C:\path\to\numpy-1.16.3+mkl-cp36-cp36m-win_amd64.whl")
import numpy as np
The wheel file is read, but the import generates an error.
*** ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using c:\projects\zip_test\venv\Scripts\python.exe),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named 'numpy.core._multiarray_umath'
The puzzling part is that the wheel file contains a .pyd for _multiarray_umath.
C:\projects\zip_test\plugins\New folder\numpy\core>dir
Volume in drive C is OS
Volume Serial Number is FE3D-6596
Directory of C:\projects\zip_test\plugins\New folder\numpy\core
07/25/2019 02:37 PM <DIR> .
07/25/2019 02:37 PM <DIR> ..
....
04/22/2019 02:55 AM 101,376 _multiarray_tests.cp36-win_amd64.pyd
04/22/2019 02:55 AM 2,494,976 _multiarray_umath.cp36-win_amd64.pyd
....
74 File(s) 583,173,551 bytes
5 Dir(s) 309,925,851,136 bytes free
This feels like a pathing issue. Yet adding the direct path for core/ to sys.path generates the same error.
sys.path.append(r"C:\projects\zip_test\plugins\numpy-1.16.3+mkl-cp36-cp36m-win_amd64.whl\numpy\core")
What's the deal? Why can't Python find numpy.core._multiarray_umath?
In response to repsones:
The wheel file was obtained from Christoph Gohlke.
My operating system is Windows 10 Pro 64-bit.
I am running Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32 (installed via Python Foundation build 3.6.8150.0) in a venv. The "system" Python is not on PATH.
I can install the numpy-1.16.3+mkl-cp36-cp36m-win_amd64.whl file via pip and import numpy as np works fine. I then uninstall numpy via pip uninstall -y numpy.
Running dumpbin /dependents _multiarray_umath.cp36-win_amd64.pyd produces:
Microsoft (R) COFF/PE Dumper Version 14.22.27905.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file _multiarray_umath.cp36-win_amd64.pyd
File Type: DLL
Image has the following dependencies:
mkl_rt.dll
python36.dll
KERNEL32.dll
VCRUNTIME140.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-time-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
Summary
77000 .data
1000 .gfids
1C000 .pdata
30000 .rdata
3000 .reloc
1000 .rsrc
1BD000 .text
Of the dependencies, I can find mkl_rt.dll, python36.dll, and VCRUNTIME140.dll in the either the .whl or the venv. There are apparently two versions of KERNEL32.dll, one for x86 and one for x64. One of these resides in C:\Windows\System32.
The only information I could find about the api-ms-win-crt-*.dll are described on MSDN within the "Update for Universal C Runtime in Windows",
The Windows 10 Universal CRT is a Windows operating system component
that enables CRT functionality on the Windows operating system. This
update allows Windows desktop applications that depend on the Windows
10 Universal CRT release to run on earlier Windows operating systems.
These are for non-Windows 10 systems. There seems to be no "official" way to obtain them for a Windows 10 system. I was able to copy the dlls from a Windows 7 system. Naively putting them in the PYTHONPATH (not surprisingly) doesn't work. If they were to work at all, they would likely need to be registered. But 1) registering Windows 7 dlls on Windows 10, I am told, can make the system unstable and 2) definitely doesn't seem like a universal, portable solution.
First.
Build application with and without numpy. Check difference between builds.
Original error was: No module named 'numpy.core._multiarray_umath' Can means two things:
cannot found file
some dependence of this this pyd file is not satisfied. You may try to check it with http://dependencywalker.com/.
Yes, depending on how you define "install".
Numpy requires the use of .pyd files. A pyd file is essentially a dll. These are compiled code files which the Python interpreter references during run-time. Unfortunately, the Windows API calls used to load dll files requires them to exist at the filesystem level. When you try to import directly from a wheel, the pyd files (and the functions/classes they contain) aren't accessible. Hence, the error.
The (Relatively) Easy solution
One solution is to make the pyd files accessible on the filesystem. To do that, it's probably simplest to unpack the whole wheel file to disk1.
In that case, consider "how you define 'install'":
If you could run numpy straight from the wheel, wouldn't that mean numpy was installed?
What difference does it really make whether you're writing a wheel file or the contents of that wheel file?
Since the plugin system is downloading a wheel file onto the filesystem, the application must have write access. Download the wheel file and extract it into the plugins directory. As long as the plugin directory is on the PYTHONPATH (i.e. sys.path.append), the import will work. Smarts and so forth can be added from there. Of course, what level of smarts you need is a whole other issue. But that's the basic idea.
The Wizard's Path
Another solution is to create your own functionality to import a dll from memory. This is precisely the goal of Joachim Bauch's MemoryModule project. It seems the Py2Exe project used the MemoryModule at one point, through a module called zipextimporter.py. Unfortunately, the code is hard to find and seems outdated (Python 2.4). There is also a similar module called pymemimporter.py which is slightly more recent.
1 Numpy expects the pyd files to be in specific locations. If you want to extract only the pyd files, they'll need to be nested in the appropriate folders (e.g. numpy/core/_multiarray_umath.cp36-win_amd64.pyd). This is sufficient to import numpy, but unless the other modules are available, rather pointless.
I am attempting to put together a simple c++ test project that uses an embedded python 3.2 interpreter. The project builds fine but Py_Initialize raises a fatal error:
Fatal Python error: Py_Initialize: unable to load the file system codec
LookupError: no codec search functions registered: can't find encoding
Minimal code:
#include <Python.h>
int main (int, char**)
{
Py_Initialize ();
Py_Finalize ();
return 0;
}
The OS is 32bit Vista.
The python version used is a python 3.2 debug build, built from sources using VC++ 10.
The python_d.exe file from the same build runs without any problems.
Could someone explain the problem and how to fix it? My own google-fu fails me.
EDIT 1
After going through the python source code I've found that, as the error says, no codec search functions have been registered. Both codec_register and PyCodec_Register are as they should be. It's just that nowhere in the code are any of these functions called.
I don't really know what this means as I still have no idea when and from where these functions should have been called. The code that raises the error is entirely missing from the source of my other python build (3.1.3).
EDIT 2
Answered my own question below.
Check the PYTHONPATH and PYTHONHOME environment variables and make sure they don't point to Python 2.x.
http://bugs.python.org/issue11288
Parts of this have been mentioned before, but in a nutshell this is what worked for my environment where I have multiple Python installs and my global OS environment set-up to point to a different install than the one I attempt to work with when encountering the problem.
Make sure your (local or global) environment is fully set-up to point to the install you aim to work with, e.g. you have two (or more) installs of, let's say a python27 and python33 (sorry these are windows paths but the following should be valid for equivalent UNIX-style paths just as well, please let me know about anything I'm missing here (probably the DLLs path might differ)):
C:\python27_x86
C:\python33_x64
Now, if you intend to work with your python33 install but your global environment is pointing to python27, make sure you update your environment as such (while PATH and PYTHONHOME may be optional (e.g. if you temporarily work in a local shell)):
PATH="C:\python33_x64;%PATH%"
PYTHONPATH="C:\python33_x64\DLLs;C:\python33_x64\Lib;C:\python33_x64\Lib\site-packages"
PYTHONHOME=C:\python33_x64
Note, that you might need/want to append any other library paths to your PYTHONPATH if required by your development environment, but having your DLLs, Lib and site-packages properly set-up is of prime importance.
Hope this helps.
The core reason is quite simple: Python does not find its modules directory, so it can of course not load encodings, too
Python doc on embedding says "Py_Initialize() calculates the module search path based upon its best guess" ... "In particular, it looks for a directory named lib/pythonX.Y"
Yet, if the modules are installed in (just) lib - relative to the python binary - above guess is wrong.
Although docs says that PYTHONHOME and PYTHONPATH are regarded, we observed that this was not the case; their actual presence or content was completely irrelevant.
The only thing that had an effect was a call to Py_SetPath() with e.g. [path-to]\lib as argument before Py_Initialize().
Sure this is only an option for an embedding scenario where one has direct access and control over the code; with a ready-made solution, special steps may be necessary to solve the issue.
Ran into the same thing trying to install brew's python3 under Mac OS! The issue here is that in Mac OS, homebrew puts the "real" python a whole layer deeper than you think. You would think from the homebrew output that
$ echo $PYTHONHOME
/usr/local/Cellar/python3/3.6.2/
$ echo $PYTHONPATH
/usr/local/Cellar/python3/3.6.2/bin
would be correct, but invoking $PYTHONPATH/python3 immediately crashes with the abort 6 "can't find encodings." This is because although that $PYTHONHOME looks like a complete installation, having a bin, lib etc, it is NOT the actual Python, which is in a Mac OS "Framework". Do this:
PYTHONHOME=/usr/local/Cellar/python3/3.x.y/Frameworks/Python.framework/Versions/3.x
PYTHONPATH=$PYTHONHOME/bin
(substituting version numbers as appropriate) and it will work fine.
From python3k, the startup need the encodings module, which can be found in PYTHONHOME\Lib directory.
In fact, the API Py_Initialize () do the init and import the encodings module.
Make sure PYTHONHOME\Lib is in sys.path and check the encodings module is there.
I had this issue with python 3.5, anaconda 3, windows 7 32 bit. I solved it by moving my pythonX.lib and pythonX.dll files into my working directory and calling
Py_SetPythonHome(L"C:\\Path\\To\\My\\Python\\Installation");
before initialize so that it could find the headers that it needed, where my path was to "...\Anaconda3\". The extra step of calling Py_SetPythonHome was required for me or else I'd end up getting other strange errors where python import files.
I just ran into the exact same problem (same Python version, OS, code, etc).
You just have to copy Python's Lib/ directory in your program's working directory ( on VC it's the directory where the .vcproj is )
There appears to be something going wrong with the release build either failing to include the appropriate codecs or else misidentifying the codec to use for system APIs. Since the python_d executable is working, what does that return for os.getfsencoding()? (Use the C API to call that between your Initialize/Finalize calls)
I had the same issue and found this question. However from the answers here I was not able to solve my problem. I started debugging the cpython code and thought that I might be discovered a bug. Therefore I opened a issue on the python issue tracker.
My mistake was that I did not understand that Py_SetPath clears all inferred paths.
So one needs to set all paths when calling this function.
Link to the issue conversation
For completion I also copied the most important part of the conversation below.
My original issue text
I compiled the source of CPython 3.7.3 myself on Windows with Visual Studio 2017 together with some packages like e.g numpy. When I start the Python Interpreter I am able to import and use numpy. However when I am running the same script via the C-API I get an ModuleNotFoundError.
So the first thing I did, was to check if numpy is in my site-packages directory and indeed there is a folder named numpy-1.16.2-py3.7-win-amd64.egg. (Makes sense because the python interpreter can find numpy)
The next thing I did was to get some information about the sys.path variable created when running the script via the C-API.
#### sys.path content ####
C:\Work\build\product\python37.zip
C:\Work\build\product\DLLs
C:\Work\build\product\lib
C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\PROFESSIONAL\COMMON7\IDE\EXTENSIONS\TESTPLATFORM
C:\Users\rvq\AppData\Roaming\Python\Python37\site-packages
Examining the content of sys.path I noticed two things.
C:\Work\build\product\python37.zip has the correct path 'C:\Work\build\product\'. There was just no zip file. All my files and directory were unpacked. So I zipped the files to an archive named python37.zip and this resolved the import error.
C:\Users\rvq\AppData\Roaming\Python\Python37\site-packages is wrong it should be C:\Work\build\product\Lib\site-packages but I dont know how this wrong path is created.
The next thing I tried was to use Py_SetPath(L"C:/Work/build/product/Lib/site-packages") before calling Py_Initialize(). This led to
Fatal Python Error 'unable to load the file system encoding'
ModuleNotFoundError: No module named 'encodings'
I created a minimal c++ project with exact these two calls and started to debug Cpython.
int main()
{
Py_SetPath(L"C:/Work/build/product/Lib/site-packages");
Py_Initialize();
}
I tracked the call of Py_Initialize() down to the call of
static int
zipimport_zipimporter___init___impl(ZipImporter *self, PyObject *path)
inside of zipimport.c
The comment above this function states the following:
Create a new zipimporter instance. 'archivepath' must be a path-like
object to a zipfile, or to a specific path inside a zipfile. For
example, it can be '/tmp/myimport.zip', or
'/tmp/myimport.zip/mydirectory', if mydirectory is a valid directory
inside the archive. 'ZipImportError' is raised if 'archivepath'
doesn't point to a valid Zip archive. The 'archive' attribute of the
zipimporter object contains the name of the zipfile targeted.
So for me it seems that the C-API expects the path set with Py_SetPath to be a path to a zipfile. Is this expected behaviour or is it a bug?
If it is not a bug is there a way to changes this so that it can also detect directories?
PS: The ModuleNotFoundError did not occur for me when using Python 3.5.2+, which was the version I used in my project before. I also checked if I had set any PYTHONHOME or PYTHONPATH environment variables but I did not see one of them on my system.
Answer
This is probably a documentation failure more than anything else. We're in the middle of redesigning initialization though, so it's good timing to contribute this feedback.
The short answer is that you need to make sure Python can find the Lib/encodings directory, typically by putting the standard library in sys.path. Py_SetPath clears all inferred paths, so you need to specify all the places Python should look. (The rules for where Python looks automatically are complicated and vary by platform, which is something I'm keen to fix.)
Paths that don't exist are okay, and that's the zip file. You can choose to put the stdlib into a zip, and it will be found automatically if you name it the default path, but you can also leave it unzipped and reference the directory.
A full walk through on embedding is more than I'm prepared to type on my phone. Hopefully that's enough to get you going for now.
I had the problem and was tinkering with different solutions mentioned here. Since I was running my project from Visual Studio, apparently, I needed to set the environment path inside Visual Studio and not the system path.
Adding a simple PYTHONHOME=PATH\TO\PYTHON\DIR in the project solution\properties\environment solved the problem.
For me this happened when I updated Python 64 bit from 3.6.4 to 3.6.5. It threw some error like "unable to extract python.dll. Do you have permissions."
Pycharm also failed to load interpreter, even though I reloaded it in settings. Running python command gave same error, with and without administrator mode.
Reason
There was error in installation of Python, include folder in python installation directory C:\Users\USERNAME\AppData\Local\Programs\Python\Python36 was missing
Reinstalling Python also dint solve the issue.(Not removal and install)
Solution
Uninstall Python and Install of Python again.
Because running installer was just extracting same files excluding include folder
In my cases, for windows, if you have multiple python versions installed, if PYTHONPATH is pointing to one version the other ones didn't work. I found that if you just remove PYTHONPATH, they all work fine
For those working in Visual Studio simply add the include, Lib and libs directories to the Include Directories and Library Directories under
Projects Properties -> Configuration Properties > VC++ Directories :
For example I have Anaconda3 on my system and working with Visual Studio 2015 This is how the settings looks like (note the Include and Library directories) :
Edit:
As also pointed out by bossi setting PYTHONPATH in your user Environment Variables section seems necessary.
a sample input can be like this (in my case):
C:\Users\Master\Anaconda3\Lib;C:\Users\Master\Anaconda3\libs;C:\Users\Master\Anaconda3\Lib\site-packages;C:\Users\Master\Anaconda3\DLLs
is necessary it seems.
Also, you need to restart Visual Studio after you set up the PYTHONPATH in your user Environment Variables for the changes to take effect.
Also note that :
Make sure the PYTHONHOME environment variable is set to the Python
interpreter you want to use. The C++ projects in Visual Studio rely on
this variable to locate files such as python.h, which are used when
creating a Python extension.
So, for some reason the python dll fails to locate the encodings module. The python.exe executable apparently finds it because it has the expected relative path. Modifying the search path works.
The reason for all of this? Don't know but at least it works. I highly suspect a typo on my part somewhere, that's usually the reason for odd bugs it seems.
I was trying to execute a third party prolog program for my research. In the meanwhile, I discovered that it requires the pyswip package to execute, due to the program declaration
from pyswip import Prolog
So, after installing this package following the instructions in the INSTALL file, in the root folder of the pyswip installation, the package still could no be found, as with the output error:
ImportError: Could not find the SWI-Prolog library in this platform. If you are sure it is installed, please open an issue.
So after some investigation, I've discovered that the use of the package from python is called from the core.py file, in my case located at
/Library/Python/2.7/site-packages/pyswip
So analyzing this file I could locate the output error text and conclude that the path was not being found according with the specific line
elif platform == "dar": # Help with MacOS is welcome!!
So I noticed too that comment to help with MacOS. Inside the method that tries to found the path properly
(path, swiHome) = _findSwiplDar()
I noticed two array: one with two candidate names of package names and another with the path's candidates. So I've located the first package name "libswipl.dylib" doing a search on my OSX. Finishing, I've added the specific package path to the path array, and everything is working now.
I know this is not the best solution, but I've discovered how things are working and resolved the issue. If someone knows a better way to configure this kind of library without updating the core.py file, I'll be glad to know ;-)
I hope this helps someone!