Develop Reference

Numpy in SPSS 22

I'm running SPSS 22.0.0.1 (64 bit edition) under Linux Mint 20 Ulyana (kernel 5.4.0-62-generic x86_64 GNU/Linux). Now I want to use python models not shipped with the "private" SPSS python edition.
The private Python version shipped with my version of SPSS is 2.7.1. Running the code below from SPSS syntax gives the private python version.
* SPSS syntax .
begin program .
import sys
print(sys.version)
end program .
outputs:
2.7.1 (r271:86832, Jun 14 2013, 00:22:41)
[GCC 4.3.4 [gcc-4_3-branch revision 152973]]
The system python that carries the modules I want use is 2.7.8.
~$ python2.7
outputs:
Python 2.7.18 (default, Aug 4 2020, 11:16:42)
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
From what I understand, I can add a sitecustomize.py in the private python that points to the system python. The content of the file is just:
import sys
sys.path.append(r"/usr/lib/python2.7/dist-packages")
This seems to add the desired path to my private python. Listing the paths from SPSS syntax (last row of output):
* SPSS syntax .
begin program .
import sys
for p in sys.path:
print(p)
end program .
outputs:
/opt/IBM/SPSS/Statistics/22/extensions
/home/[USER]/.IBM/SPSS/Statistics/22/extensions
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/site-packages/readline-6.1.0-py2.7-linux-x86_64.egg
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/site-packages
/opt/IBM/SPSS/Statistics/22/Python/lib/python27.zip
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/plat-linux2
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/lib-tk
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/lib-old
/opt/IBM/SPSS/Statistics/22/Python/lib/python2.7/lib-dynload
/usr/lib/python2.7/dist-packages
However, when I try to import numpy in my SPSS syntax, I get this quite verbatim message:
* SPSS syntax .
begin program .
import numpy
end program .
outputs:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 142, in <module>
from . import core
File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 71, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using ),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named _multiarray_umath
I also tried to add the system python path directly in SPSS syntax, but no love...
* SPSS syntax .
begin program .
import sys
sys.path.insert(0, "/usr/lib/python2.7/dist-packages")
import numpy
end program .
SPSS is working fine with the private python. And the system python is working fine on its own.

You need to "tell" SPSS to use your system-Python. Be aware that SPSS 22 can only use python 2.7 (not other version of 2.x, and no 3.x). So you need to have that installed on your system. Then, in SPSS, go to Edit/Options/File locations, and set the Python 2.7 location to your system-Python installation folder. You will be able to use, from SPSS syntax, the libraries available in your system-Python.
Alternatively, you could go into your SPSS-python, and install the needed libraries with pip.

How to enable an optional build dependency using nix-shell?

I would like to use gtk-enabled opencv with python 3 inside of a nix-shell.
I am able to launch a non-gtk enabled python3/opencv environment with:
nix-shell --pure -p python35Packages.opencv3
However, commands that open windows fail:
[nix-shell:~/src/nixpkgs/pkgs/development/libraries/opencv]$ python
Python 3.5.3 (default, Jan 17 2017, 07:57:56)
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.namedWindow("foo")
OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script) in cvNamedWindow, file /tmp/nix-build-opencv-3.2.0.drv-0/opencv-3.2.0-src/modules/highgui/src/window.cpp, line 565
In the package definition in /nixpkgs/pkgs/development/libraries/opencv/3.x.nix, there are function parameters enableGtk2 and enableGtk3 that switch on the optional dependency in the build.
I read through several nix tutorials and the entire syntax definition for the language.
Switching on an optional dependency is pretty fundamental, so I would expect it to be doable using the command-line tools, i.e. nix-shell. Anyone know how it is done?

This is the command that you would need to use in this case:
nix-shell -p "python35Packages.opencv3.override { enableGtk2 = true; }"
Another solution would be to override the python35Packages.opencv3 package once and for all as explained in the nixpkgs documentation. Then you would be able to use your command unchanged.

How to use pyinstaller?

Okay so I'm a complete noob in programming and I'm trying to compile a simple program I wrote that takes in a string and prints out the string in morse code it's called morse.py. I installed pyinstaller using
pip install pyinstaller
and I am trying to compile my program using pyinstaller.
Now I've searched a bit and it says that I need to write pyinstaller morse.py, but I don't really know where to write that. I tried moving to the directory of my program and doing that in CMD but it didn't work. I tried making a python program in the same directory and doing that and that also didn't work. I couldn't find anything very helpful to tell me exactly how to compile the file.
Can someone please help?

I would suggest to first read the Using Pyinstaller section in the documentation of the module itself.
You can also use some tutorials (e.g. Matt Borgerson's one).
In order to recap you should:
write your script and make sure that it works
run from the command line:
~\ pyinstaller your_file_name.py
this command will generate a your_file_name.spec file where you can include all the dll required by your application and any custom settings (Using Spec Files)
once you have decided what to include in your .exe application you can run from the command line
~\ pyinstaller [option1] [option2] your_file_name.py
You can find the full list the options in the documentation. An example could be pyinstaller.exe --onefile --windowed --icon=app.ico app.py where:
--onefile: Create a one-file bundled executable.
--windowed: Parameter to chooseif you are compiling in Mac OS X or Windows
--icon= : Choose the file to use as icon for file.
You can create your exe file very easily also with py2exe.

Try to write full path to pyinstaller (for example = C:\Users\user\AppData\Local\Programs\Python\Python35-32\Scripts\pyinstaller.exe)
full cmd string must look like:
C:\Users\user\AppData\Local\Programs\Python\Python35-32\Scripts\pyinstaller.exe --onefile myscript.py

I don’t have much experience in Python. I am able to write simple but useful scripts. I was able to convert them to .exe using pyinstaller. But the more programs I write, the more python’s versions I have (and packages and virtual environments). What was working in the beginning suddenly stopped working and got me very confused. But jumping from one page to another I figured it out. My post is to help another newbies. I’m using Windows, but some rules are the same probably for other systems.
I’m writing it in March 2020. Pyinstaller doesn’t work properly with the newest python version which is now 3.8. So firstly you need to install Python 3.7. Secondly choose the version accordingly to type of your system. It can be 32 or 64.
You can install more than one Python’s version. In command line when you type “py” your computer will look for the newest Python’s version - I mean the one with the highest version number. It’s because of the file C:\Windows\py.exe which appears when you install your first Python. If today you have installed only python 3.7, “py” will run Python 3.7. But next day when you install Python 3.8 and you type “py” it will run Python 3.8. If you want to use the older one just type py -3.7
C:\Users\Ania>py
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
C:\Users\Ania>py -3.7
Python 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 10:41:24) [MSC v.1900 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Now to work with your project you need to create a virtual environment for it. Remember to use the correct python version. First let’s make a folder for it.
D:\PYTHON>mkdir my_new_great_project
Go to that folder:
D:\PYTHON>cd my_new_great_project
Create the venv:
D:\PYTHON\my_new_great_project>py -3.7 -m venv venv
The first “venv” above is the name of a python module which we are using. The second “venv” is the name of the virtual environment which we are creating.
Activate the venv:
D:\PYTHON\my_new_great_project>venv\Scripts\activate.bat
You should see:
(venv) D:\PYTHON\my_new_great_project>
Now you can use pip to install whatever external packages your project needs.
(venv) D:\PYTHON\my_new_great_project>pip install pandas
Pip “knows” that you need packages for python 3.7.
When you create your project files don’t put them in “venv” folder.
To run your project from the venv just type its name
(venv) D:\PYTHON\my_new_great_project>great_project.py
If you are done with your project, everything works perfectly and you need .exe file, it’s time to install pyinstaller in your venv.
(venv) D:\PYTHON\my_new_great_project>pip install pyinstaller
To check what external packages you’ve already installed just type “pip list”:
(venv) D:\PYTHON\my_new_great_project>pip list
Package Version
altgraph 0.17
future 0.18.2
numpy 1.18.2
pandas 1.0.3
pefile 2019.4.18
pip 19.2.3
PyInstaller 3.6
python-dateutil 2.8.1
pytz 2019.3
pywin32-ctypes 0.2.0
setuptools 41.2.0
six 1.14.0
Now from venv you can run pyinstaller to make an .exe file:
(venv) D:\PYTHON\my_new_great_project>pyinstaller --onefile --name my_project great_project.py
Now your project is available as an single .exe file in the folder called “dist”.
I hope it helps someone.

Hello i made a code in python, i used it to turn itself into a exe.
Make sure to have it in same directory as the file you want converted.
import subprocess
import shutil
import os
cmd = input("file name: ") # asks user for filename
extra = input("extra commands? eg -w -F if using two or more put a space between: ") # asks for extra options
suros = "pyinstaller "+ extra + " " + cmd + ".py" # sets run command
current_workin_path = os.getcwd() # gets current working path
dirt = current_workin_path + "/"+ cmd # set current working path for folder creation
os.mkdir(dirt) # creates folder for working path and for a copy
shutil.copy(cmd + ".py", current_workin_path + "/" + cmd + "/" + cmd + ".py") # creats copy of python file in the newly set path
os.chdir(current_workin_path + "/" + cmd + "/") # changes current working path to newly created one
subprocess.run(suros) # runs the command set eariler
os.remove(cmd + ".py") # delete the copy of python file
Hope this helps someone.

import wx failed under GDB python interface, R6034: An application has made an attempt to load the C runtime library incorrectly

I have a self build GDB.exe under WinXP, which is link against python 2.7 dll. The GDB was built under msys+MinGW, so GDB.exe is linked against msvcrt.dll. But the python dll is linked against msvcrt90.dll. See the screen shot below of the dependency walker.
Generally, the python interpreter in GDB.exe works quite well, and I can correctly run python pretty printer and other python commands.
But I found that the wxPython can not run correctly, see the command log:
GNU gdb (GDB) 7.8.50.20140717-cvs
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) python import wx
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "E:\code\gdb\mybuild\bin\lib\site-packages\wx-3.0-msw\wx\__init__.py", li
ne 45, in <module>
from wx._core import *
File "E:\code\gdb\mybuild\bin\lib\site-packages\wx-3.0-msw\wx\_core.py", line
4, in <module>
import _core_
ImportError: DLL load failed: The specified module could not be found.
Error while executing Python code.
(gdb)
I realize that there are some dlls under folder: E:\code\gdb\mybuild\bin\Lib\site-packages\wx-3.0-msw\wx, such as "core_.pyd" and "wxmsw30u_core_vc90.dll", these dlls were build from Visual C++, and you can see, they are linked to msvcr90.dll and msvcp90.dll.
But if I run the same command under python.exe (which is E:\code\gdb\mybuild\bin\python.exe) command line shell, there is not such issue, see the log:
Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>>
Look, I don't have any issue to run import wx command.
I also noticed that the python.exe is build against msvc90.dll, see the dependency walker screen shot below:
Now, my question is: how to solve this issue? From what I know, the two different kind of DLL (msvcrt.dll and msvcrt90.dll) works peacefully, but why I can't use wxpython? Thank you.

OK, the problem is solved. The reason is that both the pyd file and the dll file under the folder E:\code\gdb\mybuild\bin\Lib\site-packages\wx-3.0-msw\wx need to be associated with a manifest. I manually add the manifest to the pyd file by a tool named Resource Hacker, but I think any tool which can edit the PE file can do this. The pyd file already have some piece of manifest, so I just add more like:
<dependency>
<dependentAssembly>
<assemblyIdentity type='win32' name='Microsoft.VC90.CRT' version='9.0.21022.8' processorArchitecture='x86' publicKeyToken='1fc8b3b9a1e18e3b' />
</dependentAssembly>
</dependency>
For the dll file, I just found there are manifest file in a wxpython development package 32-bit binaries for MSVC 9 from official wxPython site, so I just copy the manifest files to the E:\code\gdb\mybuild\bin\Lib\site-packages\wx-3.0-msw\wx.
I don't have issue now, I can correctly show a wxFrame from GDB's command line.

I have tried the process of ollydbg23. The changes taken effect only after restart.

Python: No module named contextlib?

Does anyone know where i can find this python module 'contextlib'?
root#overo:~# python
Python 2.6.6 (r266:84292, Mar 9 2011, 10:05:36)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named contextlib
I did not compile python myself personally. I'm just in this mess. It's running on an ARM based CPU so maybe some things were left out to save space.
I ran find / | grep contextlib which resulted in nothing.
Can i download this module from somewhere and just plonk it in /usr/lib/python2.6? Will that work?

I got this error in a different way.
I created a pipenv virtual environment using the 32bit version of Python 3.6.5 on Windows 10. I then realized I needed the 64bit version. Uninstalled the 32bit, installed the 64bit, and then tried to go back to my existing virtual env. The previously created env was now broken in odd ways and gave me this error.
I solved this by removing the old pipenv pipenv --rm and creating a new one with the newly installed version of python.

As others have noted, that module should be in the standard library, but if it's an embedded device, it may have been dropped to save space (if true, a foolish choice IMO, since leaving out contextlib.contextmanager robs the with statement of much of its power and convenience)
If you can name the specific device or manufacturer (or ask the vendor directly), you may be able to get a better answer.
As far as fixing it goes, grabbing http://hg.python.org/cpython/file/2.6/Lib/contextlib.py and dropping it in sys.path somewhere should do the trick (running python -m site will dump the list of directories that you can use)

It has been part of the standard library since 2.5 according to the docs. It seems a bit weird that you don't have it, it works with 2.6.6 for me (Ubuntu 10.10):
blair#blair-eeepc:~$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
>>> contextlib.__file__
'/usr/lib/python2.6/contextlib.pyc'
Somebody may have a better suggestion, but if it comes to it there is a link at the top of the documentation to the source code (which is Python, so you should be able to use it directly without any compilation or anything).
Edit: Unless, as Santiago Lezica suggested, you compiled your copy of Python manually, in which case it should be a simple matter of copying the module into the correct library path.
Edit for updated question: To the best of my knowledge, just dropping the source into a directory on the Python path should work. You could do this in the system library, but, to avoid it being deleted/replaced/otherwise borked in future updates, I'd recommend putting it in a separate directory and adding that directory to the Python path. You could put it under /usr/local, or somewhere in your home directory.

With Angsrom Linux, contextlib is included in the python-misc package. You can grab it by running:
opkg install python-misc
This won't, however, get you all expected python modules, so you may also want to install python-modules:
opkg install python-modules

I found one more occasion, which produces the same error.
I had made a virtual environment with python 3.6. After a updated my python version to 3.7 I tried to activate the old virtual environment and got this error.
The solution was to delete the old environment and recreate it with the new python version.

Check sys.path to make sure your python interpreter is looking in the right directories. It should look something like this (not necessarily identical):
>>> import sys
>>> sys.path
['', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/pymodules/python2.6']
EDIT: With the updated information in the question that this is an install of unknown origin on a constrained device, assuming that unnecessary modules were removed to save space makes sense. However, for the record, I'll mention another, perhaps more common scenario where modules cannot be found: when there are file permissions issues. For example:
$ python -c 'import contextlib; print(contextlib.__file__)'
/usr/lib/python2.6/contextlib.pyc
$ ls -l /usr/lib/python2.6/contextlib.py*
-rw-r--r-- 1 root root 4136 Dec 26 16:42 /usr/lib/python2.6/contextlib.py
-rw-r--r-- 1 root root 4127 Jan 1 21:45 /usr/lib/python2.6/contextlib.pyc
$ sudo chmod go-r /usr/lib/python2.6/contextlib.py*
$ python -c 'import contextlib; print(contextlib.__file__)'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named contextlib
Especially with custom installations, import problems due to file permission issues and path problems are some of the easiest things to check and, usually, to fix.

Python 2
sudo apt-get install python-contextlib2
Python 3
sudo apt-get install python3-contextlib2

contextlib was introduced in Python 2.5, can you remove and re-install your Python 2.6.6 again? From my copy of Python 2.6.6:
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
>>>