I am trying to get KDDockWidgets 2.0 into my PySide2 Python 3.10 application. I have followed the Python bindings guide they provide (with great confusion on a Windows machine) and now am able to build KDDockWidgets with CMake, however, it is absolutely unclear what I do from here to actually be able to use the bindings in my QML code. I see in one of their examples what would be imported in QML, but how would PySide2 know where to look for it?
The guide gives me the impression that it would generate a Python module in my virtual environment's site-packages, but there is no evidence of it doing that, nor do I see any errors at all.
My build script:
setlocal
set VS_YEAR="2022"
set KDDW_REPO="KDDW"
set PY_ENV_NAME="env"
set PY_SITE_PKG="F:\Projects\Code\Python\boslin\env\Lib\site-packages\"
call "C:\Program Files\Microsoft Visual Studio\%VS_YEAR%\Community\VC\Auxiliary\Build\vcvarsall.bat" x64
cd %KDDW_REPO%
cmake . -DCMAKE_BUILD_TYPE=Release -DKDDockWidgets_PYTHON_BINDINGS=True -DKDDockWidgets_PYTHON_BINDINGS_INSTALL_PREFIX=%PY_SITE_PKG%
endlocal
I try to compile PJSUA2 for Python. I could manage to compile the source with Visual Studio 2015 Community edition.
My question is, how can I install the Python module now?
The guide suggests to use make and make install. I tried to install also minwg, but it is not working as the project was compiled with VS.
I have also tried to compile with minwg, but I never could succeed due to undefined requirements. Also official PJSUA guide recommend to use VS for Windows.
Following settings have been applied to build from source:
Swig location has been added to path
JAVA_home system variable has been added
Java location has been added to path
Empty pjlib/include/pj/config_site.h has been created
Opened source in VS 2015 Community (all suggested VS package and module has been installed)
Set following project to do not compile:
pjsua_cli_uwp_comp
pjsua_cli_wp8
pjsua_cli_wp8_comp
Set swig_java_pjsua2 to build.
Add following folders to swig_java_pjsua2 VC++ include directories
c:\Program Files\Java\jdk-11.0.2\include
c:\Program Files\Java\jdk-11.0.2\include\win32
Build solution
I could make a workaround to compile pjsua2 for Python 3.7. If you have a better solution or just suggestion to this workaround, let me know
Preconditions:
List item
Download and extract swig (swigwin 4.0.0)
Download and install JDK
Download and install Python (Python 3.7.3 x64) and set at custom installation:
tick "Add Python to environment variables"
tick "Precompile standard library"
tick "Download debugging symbols"
tick "Download debug binaries (requires VS 2015 or later)"
set custom path if you want
add swigwin location to system path
add Python location to system path
add JAVA_HOME system variable
add %JAVA_HOME%\bin to system path
Install Visual Studio 2015 Community edition
Steps:
Download and extract pjsip 2.8
Create empty config_site.h under pjlib/inlclude/pj/ folder
Open pjproject-vs14.sln in VS
There will be an unsupported warning window, press OK
At the "Install Missing Features" windows, press install
VS installer will be started, and VS has to be closed to complete installation.
I have also added "Windows 10 SDK (10.0.10586)" to the installation.
I have also enabled developer mode on my Windows 10, but this could be optional
Open project again in VS
Set project to Release and x64 on the top dropdown
Right click on the solution, and go to Configuration Properties> Configuration
Remove all uwp and wp8 related projects from selection, like pjsua_cli_uwp_comp, pjsua_cli_wp8_comp
press OK, and build solution
I have had 34 succeeded and 0 failed solution after build.
Add following lines to pjsua2.i in pjsip-apps/src/swig folder
%inline %{
pj_ssize_t new_pj_ssize_t(int s) {
return (pj_ssize_t) s;
}
%}
this extra inline swig function provides workaround for create recorder issue
Open a command line (powershell did not work for this)
go to pjsip-apps/src/swig/python
Execute following command:
swig -I../../../../pjlib/include -I../../../../pjlib-util/include -I../../../../pjmedia/include -I../../../../pjsip/include -I../../../../pjnath/include -py3 -c++ -python -threads ../pjsua2.i
Add new "Empty Project" (Visual C++) to solution with swig_python_pjsua2 name
Add libpjproject and pjsua2_lib as reference to this new project
Right click on the Header Files>Add>Existing Item...
Add pjsip-apps\src\swig\pjsua2_wrapp.h
Add a "new filter" to the project with name "Generated Code"
Right click on "Generated code" and add new existing item.
Add pjsip-apps\src\swig\pjsua2_wrapp.cxx
Right click on the project and Linker>Input
Add "Ws2_32.Lib" to "Additional Dependencies"
Go to "VC++ Directories"
Add following folder to include path(I have used full path!):
c:\python37\include
pjnath\include
pjsip\include
pjmedia\include
pjlib-util\include
pjlib\include
Add following folder to Library Directories:
C:\Python36\libs
Set at General:
Target Name: _$(ProjectName)
Target Extension: .pyd
Configuration Type: Dynamic Library (.dll)
Go to properties of the solution, and select swig_python_pjsua2 to build
Right click on swig_python_pjsua2 and build
Build should complete successfully
You will need to file to use pjsua2 library in python.
pjsua.py located under pjsip-apps/src/swig/python
x64/Release/_swig_python_pjsua2.pyd (rename this to _pjsua2.pyd)
To test, copy those to file into a folder, start python from same folder and type:
import pjsua2
Thank You very much Krisz for this awesome work!
I just got it working for Python 2.7 / 32bit and some things are slightly different, so I will add them here:
Environment:
Windows 10
SWIG 4.0.1
PJSIP 2.9
Python 2.7.14 (32bit)
Visual Studio Express 2015 (V 14.0.25431.01 Update 3)
Whaddado:
Important: Check if your Python-interpreter is really x64 or win32 like the following:
start a python session and enter:
import platform
platform.architecture()
If you have 32bit, consider this in your choice for target platform in VS.
when you add this %inline%-Snippet to pjsip-apps/src/swig/pjsua2.i, add it at the end of the file!
when starting the swig-command, omit the "-py3"-parameter, if you use Python 2.x
there is a little typo in Krisz' description, at "pjsua.py located under pjsip-apps/src/swig/python" -> of course pjsua2.py is meant. Do not use the pjsua.py.
finally copy _pjsua2.lib and pjsua2.py into \Lib\site-packages\ and the _pjsua2.pyd into \DLLs\ to make them available everywhere.
I am attempting to put together a simple c++ test project that uses an embedded python 3.2 interpreter. The project builds fine but Py_Initialize raises a fatal error:
Fatal Python error: Py_Initialize: unable to load the file system codec
LookupError: no codec search functions registered: can't find encoding
Minimal code:
#include <Python.h>
int main (int, char**)
{
Py_Initialize ();
Py_Finalize ();
return 0;
}
The OS is 32bit Vista.
The python version used is a python 3.2 debug build, built from sources using VC++ 10.
The python_d.exe file from the same build runs without any problems.
Could someone explain the problem and how to fix it? My own google-fu fails me.
EDIT 1
After going through the python source code I've found that, as the error says, no codec search functions have been registered. Both codec_register and PyCodec_Register are as they should be. It's just that nowhere in the code are any of these functions called.
I don't really know what this means as I still have no idea when and from where these functions should have been called. The code that raises the error is entirely missing from the source of my other python build (3.1.3).
EDIT 2
Answered my own question below.
Check the PYTHONPATH and PYTHONHOME environment variables and make sure they don't point to Python 2.x.
http://bugs.python.org/issue11288
Parts of this have been mentioned before, but in a nutshell this is what worked for my environment where I have multiple Python installs and my global OS environment set-up to point to a different install than the one I attempt to work with when encountering the problem.
Make sure your (local or global) environment is fully set-up to point to the install you aim to work with, e.g. you have two (or more) installs of, let's say a python27 and python33 (sorry these are windows paths but the following should be valid for equivalent UNIX-style paths just as well, please let me know about anything I'm missing here (probably the DLLs path might differ)):
C:\python27_x86
C:\python33_x64
Now, if you intend to work with your python33 install but your global environment is pointing to python27, make sure you update your environment as such (while PATH and PYTHONHOME may be optional (e.g. if you temporarily work in a local shell)):
PATH="C:\python33_x64;%PATH%"
PYTHONPATH="C:\python33_x64\DLLs;C:\python33_x64\Lib;C:\python33_x64\Lib\site-packages"
PYTHONHOME=C:\python33_x64
Note, that you might need/want to append any other library paths to your PYTHONPATH if required by your development environment, but having your DLLs, Lib and site-packages properly set-up is of prime importance.
Hope this helps.
The core reason is quite simple: Python does not find its modules directory, so it can of course not load encodings, too
Python doc on embedding says "Py_Initialize() calculates the module search path based upon its best guess" ... "In particular, it looks for a directory named lib/pythonX.Y"
Yet, if the modules are installed in (just) lib - relative to the python binary - above guess is wrong.
Although docs says that PYTHONHOME and PYTHONPATH are regarded, we observed that this was not the case; their actual presence or content was completely irrelevant.
The only thing that had an effect was a call to Py_SetPath() with e.g. [path-to]\lib as argument before Py_Initialize().
Sure this is only an option for an embedding scenario where one has direct access and control over the code; with a ready-made solution, special steps may be necessary to solve the issue.
Ran into the same thing trying to install brew's python3 under Mac OS! The issue here is that in Mac OS, homebrew puts the "real" python a whole layer deeper than you think. You would think from the homebrew output that
$ echo $PYTHONHOME
/usr/local/Cellar/python3/3.6.2/
$ echo $PYTHONPATH
/usr/local/Cellar/python3/3.6.2/bin
would be correct, but invoking $PYTHONPATH/python3 immediately crashes with the abort 6 "can't find encodings." This is because although that $PYTHONHOME looks like a complete installation, having a bin, lib etc, it is NOT the actual Python, which is in a Mac OS "Framework". Do this:
PYTHONHOME=/usr/local/Cellar/python3/3.x.y/Frameworks/Python.framework/Versions/3.x
PYTHONPATH=$PYTHONHOME/bin
(substituting version numbers as appropriate) and it will work fine.
From python3k, the startup need the encodings module, which can be found in PYTHONHOME\Lib directory.
In fact, the API Py_Initialize () do the init and import the encodings module.
Make sure PYTHONHOME\Lib is in sys.path and check the encodings module is there.
I had this issue with python 3.5, anaconda 3, windows 7 32 bit. I solved it by moving my pythonX.lib and pythonX.dll files into my working directory and calling
Py_SetPythonHome(L"C:\\Path\\To\\My\\Python\\Installation");
before initialize so that it could find the headers that it needed, where my path was to "...\Anaconda3\". The extra step of calling Py_SetPythonHome was required for me or else I'd end up getting other strange errors where python import files.
I just ran into the exact same problem (same Python version, OS, code, etc).
You just have to copy Python's Lib/ directory in your program's working directory ( on VC it's the directory where the .vcproj is )
There appears to be something going wrong with the release build either failing to include the appropriate codecs or else misidentifying the codec to use for system APIs. Since the python_d executable is working, what does that return for os.getfsencoding()? (Use the C API to call that between your Initialize/Finalize calls)
I had the same issue and found this question. However from the answers here I was not able to solve my problem. I started debugging the cpython code and thought that I might be discovered a bug. Therefore I opened a issue on the python issue tracker.
My mistake was that I did not understand that Py_SetPath clears all inferred paths.
So one needs to set all paths when calling this function.
Link to the issue conversation
For completion I also copied the most important part of the conversation below.
My original issue text
I compiled the source of CPython 3.7.3 myself on Windows with Visual Studio 2017 together with some packages like e.g numpy. When I start the Python Interpreter I am able to import and use numpy. However when I am running the same script via the C-API I get an ModuleNotFoundError.
So the first thing I did, was to check if numpy is in my site-packages directory and indeed there is a folder named numpy-1.16.2-py3.7-win-amd64.egg. (Makes sense because the python interpreter can find numpy)
The next thing I did was to get some information about the sys.path variable created when running the script via the C-API.
#### sys.path content ####
C:\Work\build\product\python37.zip
C:\Work\build\product\DLLs
C:\Work\build\product\lib
C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\PROFESSIONAL\COMMON7\IDE\EXTENSIONS\TESTPLATFORM
C:\Users\rvq\AppData\Roaming\Python\Python37\site-packages
Examining the content of sys.path I noticed two things.
C:\Work\build\product\python37.zip has the correct path 'C:\Work\build\product\'. There was just no zip file. All my files and directory were unpacked. So I zipped the files to an archive named python37.zip and this resolved the import error.
C:\Users\rvq\AppData\Roaming\Python\Python37\site-packages is wrong it should be C:\Work\build\product\Lib\site-packages but I dont know how this wrong path is created.
The next thing I tried was to use Py_SetPath(L"C:/Work/build/product/Lib/site-packages") before calling Py_Initialize(). This led to
Fatal Python Error 'unable to load the file system encoding'
ModuleNotFoundError: No module named 'encodings'
I created a minimal c++ project with exact these two calls and started to debug Cpython.
int main()
{
Py_SetPath(L"C:/Work/build/product/Lib/site-packages");
Py_Initialize();
}
I tracked the call of Py_Initialize() down to the call of
static int
zipimport_zipimporter___init___impl(ZipImporter *self, PyObject *path)
inside of zipimport.c
The comment above this function states the following:
Create a new zipimporter instance. 'archivepath' must be a path-like
object to a zipfile, or to a specific path inside a zipfile. For
example, it can be '/tmp/myimport.zip', or
'/tmp/myimport.zip/mydirectory', if mydirectory is a valid directory
inside the archive. 'ZipImportError' is raised if 'archivepath'
doesn't point to a valid Zip archive. The 'archive' attribute of the
zipimporter object contains the name of the zipfile targeted.
So for me it seems that the C-API expects the path set with Py_SetPath to be a path to a zipfile. Is this expected behaviour or is it a bug?
If it is not a bug is there a way to changes this so that it can also detect directories?
PS: The ModuleNotFoundError did not occur for me when using Python 3.5.2+, which was the version I used in my project before. I also checked if I had set any PYTHONHOME or PYTHONPATH environment variables but I did not see one of them on my system.
Answer
This is probably a documentation failure more than anything else. We're in the middle of redesigning initialization though, so it's good timing to contribute this feedback.
The short answer is that you need to make sure Python can find the Lib/encodings directory, typically by putting the standard library in sys.path. Py_SetPath clears all inferred paths, so you need to specify all the places Python should look. (The rules for where Python looks automatically are complicated and vary by platform, which is something I'm keen to fix.)
Paths that don't exist are okay, and that's the zip file. You can choose to put the stdlib into a zip, and it will be found automatically if you name it the default path, but you can also leave it unzipped and reference the directory.
A full walk through on embedding is more than I'm prepared to type on my phone. Hopefully that's enough to get you going for now.
I had the problem and was tinkering with different solutions mentioned here. Since I was running my project from Visual Studio, apparently, I needed to set the environment path inside Visual Studio and not the system path.
Adding a simple PYTHONHOME=PATH\TO\PYTHON\DIR in the project solution\properties\environment solved the problem.
For me this happened when I updated Python 64 bit from 3.6.4 to 3.6.5. It threw some error like "unable to extract python.dll. Do you have permissions."
Pycharm also failed to load interpreter, even though I reloaded it in settings. Running python command gave same error, with and without administrator mode.
Reason
There was error in installation of Python, include folder in python installation directory C:\Users\USERNAME\AppData\Local\Programs\Python\Python36 was missing
Reinstalling Python also dint solve the issue.(Not removal and install)
Solution
Uninstall Python and Install of Python again.
Because running installer was just extracting same files excluding include folder
In my cases, for windows, if you have multiple python versions installed, if PYTHONPATH is pointing to one version the other ones didn't work. I found that if you just remove PYTHONPATH, they all work fine
For those working in Visual Studio simply add the include, Lib and libs directories to the Include Directories and Library Directories under
Projects Properties -> Configuration Properties > VC++ Directories :
For example I have Anaconda3 on my system and working with Visual Studio 2015 This is how the settings looks like (note the Include and Library directories) :
Edit:
As also pointed out by bossi setting PYTHONPATH in your user Environment Variables section seems necessary.
a sample input can be like this (in my case):
C:\Users\Master\Anaconda3\Lib;C:\Users\Master\Anaconda3\libs;C:\Users\Master\Anaconda3\Lib\site-packages;C:\Users\Master\Anaconda3\DLLs
is necessary it seems.
Also, you need to restart Visual Studio after you set up the PYTHONPATH in your user Environment Variables for the changes to take effect.
Also note that :
Make sure the PYTHONHOME environment variable is set to the Python
interpreter you want to use. The C++ projects in Visual Studio rely on
this variable to locate files such as python.h, which are used when
creating a Python extension.
So, for some reason the python dll fails to locate the encodings module. The python.exe executable apparently finds it because it has the expected relative path. Modifying the search path works.
The reason for all of this? Don't know but at least it works. I highly suspect a typo on my part somewhere, that's usually the reason for odd bugs it seems.
I'm trying to build an example using cmake which needs python and mpi.I have several python versions installed, pvpython python ipython and anaconda python. I set normal python in my PATH variable (I'm working in ubuntu-linux)
I'm new to cmake stuff. Some people stated I have to change toolchainfile.cmake but I cannot locate it in my example files. Any lead on how to solve this? Thanks in advance!
Following is the error I get while running ccmake.
CMake Warning at CMakeLists.txt:14 (ADD_EXECUTABLE):
Cannot generate a safe runtime search path for target Fortran90FullExample
because files in some directories may conflict with libraries in implicit
directories:
runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by
files in:/home/xxx/anaconda/lib
runtime library [libpython2.7.so.1.0] in /usr/lib/x86_64-linux-gnu may be
hidden by files in:
/home/xxx/anaconda/libSome of these libraries may not be found correctly.
I searched for the file libz.so.1 in /usr/lib/x86_64-linux-gnu directory and it was in it. So I set the paths specifically for this directory and not the anaconda directories.
This time I used ccmake instead of cmake and I was able to easily give the paths on /usr/lib/x86_64-linux-gnu instead of the anaconda paths.
Also I changed my python path from anaconda python to the usual python location which was /usr/bin for me.
I added this to the path
PATH=/usr/bin:$PATH
This adds it to the front of the PATH variable and does not effect what is already there.
Also I had to set PYTHONHOME=$PYTHONPATH to get rid of all the relatedd errors
I am trying to install Boost Python, however http://www.boost.org/doc/libs/1_55_0/libs/python/doc/tutorial/doc/html/python/hello.html is too compressed for me. I am switchig to Windows from Linux, setting up all the technical stuff drives me crazy :)
So when I follow the instructions http://www.boost.org/doc/libs/1_55_0/more/getting_started/windows.html#prepare-to-use-a-boost-library-binary and try to execute bootstrap.bat it complains about a missing VCVARS32.BAT and it does not recognice the "cl" command - I have however both MS VS Express 2012 and 2013 installed, both boost versions (32, 64) are installed in C:/local/, documentations says that both should work. So why does it not find the compiler?
Update: So http://msdn.microsoft.com/en-us/library/9s7c9wdw%28v=vs.110%29.aspx says that you cannot use the compiler from the command line... how do I build my bjam then? This must be something totally trivial and stupid... :/
Update: Okay, one step further. vcvars32.bat actually sets all environment variables to go, so i can invoke the command line compiler now! Also there are actually prebuild binaries of bjam from the official boost site http://sourceforge.net/projects/boost/files/, I got it and copied it to by boost_xxx/tools folder and added the site to the path, so i can invoke bjam from command line as well.
Next I put a user-config.jam in my home folder, it only says:
# MSVC configuration
using msvc : 11.0 ;
# Python configuration
using python : : C:/Users/me/Anaconda ;
In the Jamroot file provided with C:\local\boost_1_55_0\libs\python\example\tutorial, I only edited
use-project boost
: C:/local/boost_1_55_0 ;
then I call bjam from the very same directory which gives my some cryptic output
C:\local\boost_1_55_0\libs\python\example\tutorial>bjam
warning: mismatched versions of Boost.Build engine and core
warning: Boost.Build engine (bjam) is 03.1.18
warning: Boost.Build core (at C:/local/boost_1_55_0/tools/build/v2) is 2011.12-s
vn
link.jam: No such file or directory
C:/local/boost_1_55_0/tools/build/v2/util\path.jam:458: in path.makedirs
rule MAKEDIR unknown in module path.
C:/local/boost_1_55_0/tools/build/v2/build\configure.jam:233: in configure.set-l
og-file
C:/local/boost_1_55_0/tools/build/v2\build-system.jam:695: in load
C:\local\boost_1_55_0\libs\python\example\..\..\..\tools\build\v2/kernel\modules
.jam:289: in import
C:\local\boost_1_55_0\libs\python\example\..\..\..\tools\build\v2\kernel\bootstr
ap.jam:139: in boost-build
C:\local\boost_1_55_0\libs\python\example\boost-build.jam:7: in module scope
C:\local\boost_1_55_0\libs\python\example\tutorial>
Now, can you give me any hints as to what went wrong and where to go from here? I really want a stable, robust working copy of boost.python - it doesn't make sense to me if I need to hack something together. I thought it was quite standard/reliable, isn't there something like "boost.python for guys who did not study computer science"? Can I ignore the warnings?
Perhaps you can use the binary from http://www.lfd.uci.edu/~gohlke/pythonlibs/#boost.python
Regarding your compiler problems: there is a bunch of questions on this on SO already. Basically, the compiler you need depends on your version of python. You will probably need to install some Windows SDK and setup some stuff manually. It has always been a pain in the ass for me and in the end you might be better of with using some python distribution like Canopy (there are a few others out there).