Develop Reference

Using PyBind11 with a virtual environment created at runtime

My goal is to embed the python interpreter using PyBind11, but to use the interpreter from a virtual env, such that installing dependencies using pip does not clutter the system paths.
There is not much information online around this topic. Embedding python with pybind11. Virtual environment doesn't work hardcodes the venv at compile time. This is insufficient, since the venv does not exist at that time, but will be created when the scripting engine is enabled at runtime.
The plan for now is to pip install --target into a cache dir and add it to the sys.path. Using the system interpreter. This is "okayish" but not using the system interpreter would be peferable.

You do not need a virtual environment to ship isolated dependencies with your software, or to use them locally, you simply need what you already have: an isolated lib folder.
pybind11 does not have its own interpreter (AFAIK), it embeds the one from the Python build you're using when compiling, so you should already have a matching version of the Python interpreter you're embedding (this is the executable you pass to PYTHON_EXECUTABLE if you're using CMake).
The only things that you should change is to not use sys.path but rather site.addsitedir, since sys.path will not handle everything (e.g., .pth files), while site.addsitedir will, so simply:
py::module_::import("site").attr("addsitedir")(/* whatever */);

How to Configure Poetry Environments in Pycharm With Windows + WSL2?

TL;DR: can't configure a Python Interpreter on PyCharm (Windows) using an existing Poetry environment in WSL. When trying to set the Poetry environment path under Add Python Interpreter > Poetry Environment > Existing Environment, the needed Python executable simply does not show. What am I doing wrong?
====================================================
Details:
I'm using PyCharm Pro 2021.3 on Windows 11, with Python running on WSL2 (Ubuntu 20.04). I am trying to add a python interpreter for an existing Poetry environment I created on WSL2, and it just does not seem to work.
PyCharm's current support of Poetry is via adopting this plugin. From what I could gather from the plugin's official documentation, in order to define an interpreter with an existing Poetry environment, I go to Python Interpreter > Add > Poetry Environment, choose Existing environment, and put in the path to that specific environment:
In order to find the path to that environment, I run "poetry env info", which gives a path in the following pattern:
\\wsl$\Ubuntu-20.04\home\$USER$\.cache\pypoetry\virtualenvs\my-pretty-project-<some-hash>-py3.8\
When running which python in the environment, I see the python executable is at:
\\wsl$\Ubuntu-20.04\home\$USER$\.cache\pypoetry\virtualenvs\my-pretty-project-<some-hash>-py3.8\bin\python
However - when I browse to that location in PyCharm, the Python file simple does not show. The bin directory appears as empty - as also described in this question. However - and similarly to what described in said question - if I try to redefine the default interpreter path for WSL to be the path to that Poetry environment, the Python executable is there and kicking:
(The solution described in the aforementioned question, sadly, does not work for my problem, as I am already using the patch to the Poetry environment).
What can I do to make this work?

Let me get this straight: You want PyCharm for Windows to execute Python binaries in WSL?
That cannot happen.
Binaries in WSL are "ELF" binaries which Windows cannot execute (outside WSL). If the virtualenv was created by poetry from within WSL, it will contain ELF Python binaries. And that is why PyCharm for Windows won't ever pick it up. Because ultimately PyCharm for Windows relies on Windows to execute the binaries, as long as you don't choose the WSL option.
Explicitly selecting the "WSL" option indicates to PyCharm that for this particular virtualenv you want PyCharm to invoke the binaries using WSL.
The solution is either to re-create the virtualenv in Windows, or just use the "WSL" option but you have to manage the poetry manually via WSL shell.

Is there a way to create a virtual environment with an earlier version of python?

I have python 3.8 installed on my pc, but i need 3.7 for a specific task. When i try to set up a virtual environment via
virtualenv -p "my/path/to/python37.exe"
it calls the installer, so i am to install py37 manually, but then it ends up with an error:
Error 0x80070666: Cannot install a product when a newer version is installed
Is there a proper way to implement such a thing?
Thanks!

Have you tried anaconda or miniconda (the lighter version of anaconda)? Having more python versions in different environments and switching between them is quite easy.

I haven't installed Anaconda. If you want to have multiple different environments with pure Python you can do it like this:
1) Install the python versions that you want with the exe installer, ie Python 2.x.x, Python 3.7.x, Python 3.8.x etc, maybe inside a common folder like C:\Python
2) Then edit the System Variables path and pinpoint to the folder for the version you want to create a virtual environment. You need two entries here, one to python folder (for the python.exe) and one to the Scripts folder (for the pip.exe)
3) Open command prompt and hit python. You ll see that it's showing the apporpriate version. Install the virtual environment with "python -m venv name_of_env_you_want"
4) If you want to create a virtual environment with a different version change the paths

Python installation on cygwin

I am using Windows 8 where I have Python 2.7 installed. In addition I have cygwin installed where I installed also Python 2.7.
I need to install a Python library whose installation on Windows is not supported but it is on Linux so I want to try to use it on Cygwin.
The library has a "configure" script who checks dependencies for many packages and settings and complaints because some python packages are not installed.
But when I try to install them, the system says they are installed. From this, I have concluded that my python windows installation is overlapping my python cygwin installation.
I think that the cygwin is importing in the Path all the windows Path and while the script checking the configuration can find the cygwin installation, when executing from the command-line it is not the case.
Of course, I don't want to remove from the PATH my windows python.
My question is then, what option do I have to work from cygwin only with the cygwin python installed?
As far as I have seen in other questions, usually the problem is the opposite (making cygwin to use windows python, what is problematic and not recommended in some cases).

Set the path in your .profile then.
So if e.g. your Python is in C:\Python27
remove the windows paths to /cygdrive/c/Python27 in .profile
E.g.:
$ echo $PATH
/usr/local/bin:/usr/bin:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:
/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:
/usr/bin:/cygdrive/c/Program Files/Microsoft SQL Server/110/Tools/Binn:/bin:/cygdrive/c/Python27:
/cygdrive/c/Python27/Scripts
=>
.profile:
export PATH=/usr/local/bin:/usr/bin:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:
/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:
/cygdrive/c/Program Files/Microsoft SQL Server/110/Tools/Binn

mingw -- cl is not recognized as an internal or external command -- my mingw in new and environment is true

recently I have downloaded latest version of mingw for win7 64bit.
also i have a program to build with scons but when i try to run it I see
"cl is not recognized as an internal or external command"
I search on internet and this site they suggest two solutions
fist checking environment varible
second install this mingw from here
http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds/
but I am sure my environment is true c:\mingw\bin
and I have a new version of mingw for win 7 64 bits
I will appreciate any suggestion.

As I mentioned in the comment to your question, its looking for the Windows compiler, which you probably dont have installed. According to the SCons man page, you can fix this as follows:
MinGW
The MinGW bin directory must be in your PATH environment variable or
the PATH variable under the ENV construction variable for SCons to
detect and use the MinGW tools. When running under the native Windows
Python interpreter, SCons will prefer the MinGW tools over the Cygwin
tools, if they are both installed, regardless of the order of the bin
directories in the PATH variable. If you have both MSVC and MinGW
installed and you want to use MinGW instead of MSVC, then you must
explictly tell SCons to use MinGW by passing
tools=['mingw']
to the Environment() function, because SCons will prefer the MSVC
tools over the MinGW tools.
So, change the instantiation of your Environment to something like this:
env = Environment(tools=['mingw'])
If you need to specify more in the Environment constructor, you can do something like this:
env = Environment(tools=['mingw'], variables = command_line_vars)