Develop Reference

Local installation of python 3.5 doesn't find its libraries

After building and installing Python 3.5.1 into $HOME/.local, the installed version can't seem to find its own libraries that were installed in $HOME/.local/lib64/python3.5/lib-dynload.
The python3.5 executable will show the following:
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python 3.5.1 (default, Jan 26 2016, 15:40:42)
[GCC 4.8.1 20130909 [gcc-4_8-branch revision 202388]] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "/etc/pythonstart", line 7, in <module>
import readline
ImportError: No module named 'readline'
If I set PYTHONHOME=$HOME/.local as recommended, the first two lines will disappear, but it will still fail to import readline (and math and most other modules).
If I actually start python while inside $HOME/.local/lib64/python3.5/lib-dynload, it will succeed in loading the libraries. So I'm guessing that the import path isn't set correctly.
Is the PYTHONHOME=$HOME/.local recommendation wrong? Or is there another env variable that needs to be set to let it find the libraries?

Setting PYTHONPATH=$HOME/.local/lib64/python3.5/lib-dynload fixed the problem, but I strongly suspect that's a shoddy workaround and it should find that folder automatically in some other way.

It seems like you simply need to add Python's shared libraries to system-wide dynamically linked libraries, something like that:
echo "/usr/local/python35/lib/" >> /etc/ld.so.conf.d/python35.conf
echo "/wherever/your/.so/files/used/by/python35/live/" >> /etc/ld.so.conf.d/python35.conf
Then on commandline: ldconfig -v.
Incidentally, you should not have dynamic libraries in your $HOME/.dir directory. This should be system wide as Python runtime is typically understood as system-wide program. Why did you configure Python compilation to $HOME/.local? Typical prefix should be something like /usr/local/python35 (for "local" installation).
The only "normal" situation for having Python installation to a subdirectory of your $HOME is where you do not have access to root account on the system and the sysadmin for some reason refuses to compile Python for you, but you really have to have it in this particular version.
If you're on Debian/Ubuntu, typically you do not even have to compile latest Python or other new software yourself, there are "backport" repos for this.
UPDATE
Since it seems you cannot add it system-wide indeed, the only reasonable option I see is like this:
add LD_LIBRARY_PATH to your .bash_profile like so:
LD_LIBRARY_PATH=/your/python35/libs:/other/libs:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
This ensures your .so files are seen by dynamic linker (ld), but that's visible in your login shell only.
add your Python's installation bin dir in PATH in .bash_profile in a similar way:
PATH=/your/python35/bin:$PATH
export PATH
This puts your Python 3.5 first in PATH so you have it found first (I guess that's what you could want). It also allows you to use shebang "mantra" for your code:
#!/usr/bin/env python
If you use this, you can run it on your account and elsewhere (e.g. in some virtualenv, on system Python 3.5 elsewhere, etc) without modification.

ImportError python WINFUNCTYPE

I'm trying to run some code from here:http://code.activestate.com/recipes/577654-dde-client/
On my Windows machine, using Python 2.7.8 via x86_64 Cygwin
I keep getting:
ImportError: cannot import name WINFUNCTYPE
>>> from ctypes import WINFUNCTYPE
It appears that I am missing a standard library from somewhere with this and probably other functions... But I cannot figure out how to get this library or from where? I do not have a lot of experience with python, especially not with importing native libraries... Is there like a cpan perl install module type mechanism or... How can I update this to get my code to work?

Most likely you started Cygwin's Python (by simply typing python in the Cygwin console) and not the one you want (Cygwin comes with Python in /usr/bin). You can test that by typing in the Python console:
import sys
sys.platform
It will output cygwin instead of win32. To launch the correct Python, you must specify the full path cygwinified (meaning that if your python is installed under "C:\dir1\Dir 2\python.exe", you'll have to type /cygdrive/c/dir1/Dir\ 2/python.exe).

Importing numpy package into ABAQUS 6.10 Python environment

I have not been successful at importing numpy package to ABAQUS Python environment from previous discussions on this similar problem.
I am working on Microsoft Windows 7 (64-bit). I have installed ABAQUS 6.10. And by typing the command abaqus python in MS-DOS prompt I figured that ABAQUS 6.10 uses Python 2.6.2 version. For an important reason, my python script file needs to use the array function available in the numpy package. I undertook the following steps based on previous discussions:
Python 2.6.2 installation:
Installed Python 2.6.2 version (32-bit) to C:\Python26\ (I already had Python 2.7.6 installed in folder C:\Python27\)
Changed the environment system variables under PATH variable to C:\Python26\; (previously it was C:\Python27\)
Changed the environment user variables under PYTHONPATH variable to C:\Python26\Lib\site-packages\ (previously it was C:\Python27\)
Loaded Python 2.6.2 successfully using command python in MS-DOS prompt.
Numpy installation for Python 2.6.2:
Downloaded and installed numpy (32-bit) MSI installer file numpy-1.3.0-win32-superpack-python2.6 from Sourceforge.net- NumPy 1.3.0 in C:\Python26\Lib\site-packages folder.
Loaded Python 2.6.2 successfully using command python in MS-DOS prompt.
Imported numpy successfully using the command import numpy in python environment.
Numpy installation for ABAQUS 6.10:
Copied the numpy folder from C:\Python26\Lib\site-packages\ and pasted to D:\SIMULIA\Abaqus\6.10-1\Python\Lib\site-packages, which is the python modules folder for ABAQUS. Please note I have installed ABAQUS 6.10 in D: drive.
Created a python script file, myScript.py, containing just the following line of code: from numpy import * and saved it in the desktop.
Opened MS-DOS prompt and changed folder destination to my desktop containing myScript.py.
I finally typed the following commands in MS-DOS prompt to run the script file in the ABAQUS Python environment: abaqus cae noGUI=myScript.py and I got the following errors:
Abaqus License Manager checked out the following license(s):
"cae" release 6.10 from 127.0.0.1
<2010 out of 2011 licenses remain available>.
ImportError: DLL load failed: %1 is not a valid Win32 application.
File "myScript.py", line 4, in <module> from numpy import array
File "C:\Python26\Lib\site-packages\numpy\__init__.py", line 130, in <module>
import add_newdocs
File "C:\Python26\Lib\site-packages\numpy\add_newdocs.py", line 9, in <module>
from lib import add_newdoc
File "C:\Python26\Lib\site-packages\numpy\lib\__init__.py", line 4, in <module>
from type_check import *
File "C:\Python26\Lib\site-packages\numpy\lib\type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "C:\Python26\Lib\site-packages\numpy\core\__init__.py", line 5, in <module>
import multiarray
Abaqus Error: cae exited with an error.
Note: I installed Python 2.6.2 32-bit because the numpy available for 64-bit was showing a warning that it is unstable and would subsequently crash.
My basic question is: "How can I successfully import numpy package into ABAQUS environment?"

Honestly, the 32-bit version of numpy will not work with a 64-bit version of Python. Therefore, the error is quite expected. Even if Abaqus were to use a 32-bit version, I would strongly advise not to copy and paste the directory of the library.
dim_voly answer is an easy way to get around this problem, but here are some instructions if you really want to use numpy inside Abaqus
This link explains quite clearly how to install numpy. You will need to include the Abaqus Python directory in the PATH environmental variable. Once pip is installed, it should be easy to install numpy and any other library you need.
I am currently using Abaqus 6.13 which already comes installed with numpy. Thus, upgrading your software can also be a viable solution.
Hope this helps

I had trouble importing anything into the Abaqus python environment. The workaround I used was to generate the .py script using a main .py script. So all the calculations are done and pasted in as text into a template .py script. Then later the main script calls abaqus to run the python script.
I generally found that the python version within Abaqus is behind the mainstream python version and doing anything remotely out of the ordinary doesn't work as it depends on Simulia to update things. I use too many custom packages as well.
In any case you end up wanting a master python script to set up high level things like folders and reading in data from tables. And if you're running a script to build a model, I don't doubt that you're doing a parametric study so your loops for that can be in the main script.
What also sometimes works for passing data into the abaqus .py file is setting up a struct or class and saving it using pickle. As far as I recall pickle works and is importable as long as the class does not contain any custom imports.

Importing urllib2 in SublimeREPL gives an ImportError

I installed SublimeREPL in Sublime Text 3, and it's working great. However, whenever I try to import a module that uses _socket such as urllib2 and urllib, it gives me an ImportError. I ran os.path to verify that the path was correct. It also works perfectly fine from the python command line, just not in SublimeREPL.
>>> import urllib2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "E:\Python27\lib\urllib2.py", line 94, in <module>
import httplib
File "E:\Python27\lib\httplib.py", line 71, in <module>
import socket
File "E:\Python27\lib\socket.py", line 47, in <module>
import _socket
ImportError: DLL load failed: %1 is not a valid Win32 application.

You're almost certainly mixing Python installations. That is, you've got SublimeREPL using a 64-bit Python, but you've also got a 32-bit Python on the same machine, and your PYTHONPATH is configured to point at the 32-bit Python's library instead of/ahead of its own.
Or, worse, you installed both Pythons to the same directory, and you have a single installation which is part 64-bit and part 32-bit. (If that's the case, you'll get the same error using Python from the command line.)
Mixing native and Cygwin Pythons, CPython and IronPython, or occasionally even two builds compiled with different flags, or two different X.Y versions, can also cause this, but 32-bit and 64-bit are the most common reason.
The reason urllib2 itself loads is that Python 2.7 source is Python 2.7 source code; it doesn't matter what build it comes from. But C extensions compiled into DLLs are compiled against a specific Python interpreter, and only work on that interpreter. If the Windows DLL loader hadn't refused to let Python get any further, you'd just get a different error a moment later.
While we're at it, Python improved the error handling for this case somewhere around 3.3, making it a little easier to tell what's going on, but of course if you stick with 2.7 you don't get new features.
From inside Python, the quickest way to tell if you're in a 32-bit or 64-bit interpreter is sys.getmaxsize. If it's about 2 or 4 billion, you're 32-bit; if it's about 9 or 18 too-many-digits-to-count-illion, you're 64-bit.
To find out whether a DLL is 32-bit or 64-bit is apparently a lot harder on Windows than on any other platform in the universe. See this question or this one for details.

I ran into the same problem. I suspected it had to do with using SublimeText 3 64bit and using a 32bit Python 3 for SublimeREPL.
I uninstalled Python 3 32bit, and installed 64bit Python 3 but this presented a new error about a version mismatch. I had installed Python 3.4 and my current version of SublimeText bundles Python 3.3.
So, I installed Python 3.3 64bit to match the bundled python of SublimeText 3 (Build 3083) and it finally worked. This is from within a REPL tab in Sublime Text:
Python 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 10:35:05) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>>

How to move old libraries of previous version python to new version of python

I am using Ubuntu. I install newer version of python. But all my installed libraries such as imdbPy, NumPy, Cython etc. can run in previous version of python whose version number is 2.6.6.
When I import Cython in 2.6.6, tt works, but I try same thing in 2.7.0+ version of python
import cython
occurs an error as following:
import cython
ImportError: No module named cython
What I need is to change default version of python which is used in bash. I think, I can handle it by modifying bash profile file. I also want to move all of these libs to appropriate place so that new version of python can reach them permanently.

Don't mess with your system installs - the package manager should be responsible for that. Make sure that when you install from source, you do it in a separate directory.
I install things in my home directory. I like jhbuild so this is what I use to maintain different versions of python side by side: https://thomas.apestaart.org/thomas/trac/browser/jhbuild/python
Running make from that checkout will build python 2.3-2.7 and create scripts to go into their environment:
[thomas#level ~]$ py-2.4
Entering interactive py shell /bin/bash
[py-2.4] [thomas#level ~]$ python
Python 2.4.5 (#1, Dec 16 2010, 18:51:06)
[GCC 4.4.4 20100630 (Red Hat 4.4.4-10)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Try using setuptools to reinstall the libraries. Actually, I recommend reinstalling instead of trying to use the old 2.6 libs with Python 2.7, since there may be some incompabilities.

You need to reinstall all the libraries for the new version.
I'd recommend that you first download pip and install it. After that you can install most packages with /path/to/Python27/bin/pip install <packagename>, for example
/opt/python27/bin/pip install Cython

clearly a package management tool is the way to go, because they should be able to set all the stuff I'm about to mention (plus check compatibility and dependencies) but here's some stuff that might be helpful if you run into any problems with the package manager you choose:
before you make changes, check PATH by using echo $PATH in bash. The directories are listed in the order that they are searched.
you can determine which version of python is used in bash by changing the order of directories in PATH such that the version you want appears first in PATH.
For example, if you want a python version in opt/local/bin rather than one in usr/local/bin, you can set PATH in your .bash_profile or .bashrc file with a line like this:
export PATH=opt/local/bin:$PATH
You can check the file path of the python bash is using with:
which python (entered in bash)
As for the libraries, ensure that PYTHONPATH in bash and sys.path in python are set correctly, so that it can find the appropriate libraries, and uses the updated and compatible versions instead of the old ones. You can also create .pth files containing paths to directories that you want to add to sys.path, and can use PYTHONSTARTUP in bash to run a module at startup that sets sys.path.
Unless you have a whole new set of libraries in a separate folder for the specific version of python, you may encounter problems, especially since many libraries will rely on others, and may therefore use outdated versions of each other.
If you have more than one module on sys.path that uses the same name, they may conflict.
you can check the directory path of a python module you have imported like this:
import Cython
print Cython.__file__
Hopefully all of this will be taken care of by the package manager of your choice, but you can at least check your paths carefully if you encounter trouble.