Develop Reference

Python ctypes: where is _ctypes.py file supposed to be located?

In my Python installation for python3.7, ctypes is installed. For example, __init__.py is located in: /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py. This makes sense to me.
Note: my OS is MacOS Mojave, 10.14.3.
ctypes constantly refers to _ctypes. I use PyCharm, and when I control-left click on _ctypes, it takes me to: /Users/myuser/Library/Caches/PyCharmCE2018.3/python_stubs/-1199943126/_ctypes.py.
Why is _ctypes.py in this random folder, and not in the folder installed with Python 3.7?

There's no real _ctypes.py. That file you're seeing is something PyCharm created so it could show you fake source code. It's in that weird place because it's fake. I don't know why PyCharm does that.
The real source code for _ctypes is in C, and it doesn't make it into your Python distribution. You can see it in the CPython repository under the Modules/_ctypes folder. You can see where the compiled file for _ctypes is by importing _ctypes and inspecting _ctypes.__file__, but it's compiled, so for most purposes, you'll want to look at the source code instead.

Using Module for different python version

The default version of python (ie, one that opens on typing "python" in command line) is 2.6 on my server. I also have 2.7 installed. How do I import a module, in this case numpy, in python2.7? When I try to import right now, it gives me an error -
ImportError: No module named numpy
Is there any workaround, apart from downloading the package and doing a build install?

I agree with the comment above. You will have to compile it separately. I once tried a hack of importing and modifying the sys.path however I ran into issues with the .so files.

top-level package handling with setuptools (or another python egg builder)

I am writing a little python app. I want to be able to easily deploy the app. I know that python 2.6 will allow one to execute an egg directly if there is a main module at the egg's root. I actually have that working.
The one kink is that when I try to use the argparse library, I cannot include the library in the egg without installing it into my source directory (or symlinking in the argparse.py into my source dir) since the argparse module is in the top-level package.
If I install it into a subdirectory called "argparse", I have to import it like "from argparse import argparse" instead of the normal "import argparse".
I would like to be able to specify a site-packages type directory in the egg where I could just install the third party modules/packages. Is there any way to do this with setuptools (or some other egg builder)?
Thanks!

I believe you can create a subdirectory called toplevel and in your entry point do
import sys
sys.path.insert(0, './toplevel')
Untested, though.

ImportError: no module named py2exe

I get this error when I try to use one of the py2exe samples with py2exe.
File "setup.py", line 22, in ?
import py2exe
ImportError: no module named py2exe
I've installed py2exe with the installer, and I use python 2.6. I have downloaded the correct installer from the site (The python 2.6 one.)
My path is set to C:\Python26 and I can run normal python scripts from within the command prompt.
Any idea what to do?
Thanks.
Edit: I had python 3.1 installed first but removed it afterwards. Could that be the problem?

Sounds like something has installed Python 2.4.3 behind your back, and set that to be the default.
Short term, try running your script explicitly with Python 2.6 like this:
c:\Python26\python.exe setup.py ...
Long term, you need to check your system PATH (which it sounds like you've already done) and your file associations, like this:
C:\Users\rjh>assoc .py
.py=Python.File
C:\Users\rjh>ftype Python.File
Python.File="C:\Python26\python.exe" "%1" %*
Simply removing Python 2.4.3 might be a mistake, as presumably something on your system is relying on it. Changing the PATH and file associations to point to Python 2.6 probably won't break whatever thing that is, but I couldn't guarantee it.

Seems like you need to download proper py2exe distribution.
Check out if your c:\Python26\Lib\site-packages\ contains py2exe folder.

If you have any other versions of Python installed, it may be that another Python version is the default Python. Could this be the case? I believe the default Python installation is determined from a registry setting.

I had the exact same issue and I just managed to get it solved - so I thought I would share my solution.
It turned out that my installation of CollabNet SVN contained an old version of Python that interfered with my recent Python2.7 installation.
Replacing CollabNet SVN with VisualSVN (including a couple of reboots) did the trick. I know this is not a "pretty" solution, as it caused me to uninstall CollabNet SVN - a prettier solution might have been doing modifications to the PATH env. variable... However, I am now able to use py2exe :)

For the record, my very similar problem was caused by using a Cygwin prompt. Using as standard cmd.exe shell instead worked (given all paths and correctly installed versions of python).
This was because (stupid me) the Cygwin install had pulled down it's own version of /usr/bin/python. I equally fixed it by adding the Windows installed python location to the head of the Cygwin PATH=/cygdrive/c/Python27:$PATH.

You might need to install the py2exe module globally, perhaps you might have installed to a virtualenv. I had the same problem, installing the module globally fixed the problem

python module dlls

Is there a way to make a python module load a dll in my application directory rather than the version that came with the python installation, without making changes to the python installation (which would then require I made an installer, and be careful I didn't break other apps for people by overwrting python modules and changing dll versions globaly...)?
Specifically I would like python to use my version of the sqlite3.dll, rather than the version that came with python (which is older and doesn't appear to have the fts3 module).

If you're talking about Python module DLLs, then simply modifying sys.path should be fine. However, if you're talking about DLLs linked against those DLLs; i.e. a libfoo.dll which a foo.pyd depends on, then you need to modify your PATH environment variable. I wrote about doing this for PyGTK a while ago, but in your case I think it should be as simple as:
import os
os.environ['PATH'] = 'my-app-dir' + os.pathsep + os.environ['PATH']
That will insert my-app-dir at the head of your Windows path, which I believe also controls the load-order for DLLs.
Keep in mind that you will need to do this before loading the DLL in question, i.e., before importing anything interesting.
sqlite3 may be a bit of a special case, though, since it is distributed with Python; it's obviously kind of tricky to test this quickly, so I haven't checked sqlite3.dll specifically.

The answer with modifying os.environ['PATH'] is right but it didn't work for me because I use python 3.9.
Still I was getting an error:
ImportError: DLL load failed while importing module X: The specified
module could not be found.
Turned out since version python 3.8 they added a mechanism to do this more securely.
Read documentation on os.add_dll_directory https://docs.python.org/3/library/os.html#os.add_dll_directory
Specifically see python 3.8 what's new:
DLL dependencies for extension modules and DLLs loaded with ctypes on Windows are now resolved more securely. Only the system paths, the directory containing the DLL or PYD file, and directories added with add_dll_directory() are searched for load-time dependencies. Specifically, PATH and the current working directory are no longer used, and modifications to these will no longer have any effect on normal DLL resolution. If your application relies on these mechanisms, you should check for add_dll_directory() and if it exists, use it to add your DLLs directory while loading your library.
So now this is the new way to make it work in python 3.8 and later:
import os
os.add_dll_directory('my-app-dir')
Again, the old way is still correct and you will have to use it in python 3.7 and older:
import os
os.environ['PATH'] = 'my-app-dir' + os.pathsep + os.environ['PATH']
After that my module with a dll dependency has been successfully loaded.

Ok it turns out python always loads the dll in the same directory as the pyd file, regardless of what the python and os paths are set to.
So I needed to copy the _sqlite3.pyd from python/v2.5/DLLS to my apps directory where the new sqlite3.dll is, making it load my new dll, rather than the one that comes with python (since the pyd files seem to follow the PYTHONPATH, even though the actual dlls themselves don't).

If your version of sqlite is in sys.path before the systems version it will use that. So you can either put it in the current directory or change the PYTHONPATH environment variable to do that.

I had the same issue as administrative rights to the default python library is blocked in a corporate environment and its extremely troublesome to perform installations.
What works for me:
Duplicate the sqlite3 library in a new location
Put in the latest sqlite3.dll (version you want from sqlite3 web) and the old _sqlite3.pyd into the new location or the new sqlite3 library. The old _sqlite3.pyd can be found in the default python library lib/DLLs folder.
Go to the new sqlite3 library and amend the dbapi2.py as follows:
Change "from _sqlite3 import *" to "from sqlite3._sqlite3 import *"
Make sure python loads this new sqlite3 library first. Add the path to this library if you must.

I encountered the same problem for my .pyd that depends on cuda/cudnn/tensorrt.
I came with a little function I call before importing my module.
def add_cuda_to_path():
if os.name != "nt":
return
path = os.getenv("PATH")
if not path:
return
path_split = path.split(";")
for folder in path_split:
if "cuda" in folder.lower() or "tensorrt" in folder.lower():
os.add_dll_directory(folder)
It is the easiest workaround I found, so I don't need to hardcode any path.
I guess little improvement would to actually check for .dll file, but this snippet serves well my needs.
Then you can simply:
add_cuda_to_path()
import my_module_that_depends_on_cuda