Develop Reference

Changing order of imports results in error in Python

I'm trying to understand the cause of the following error. First if I type the following into python
>>> import scipy.sparse
>>> import torch
it runs without error. However, when I type in in
>>> import torch
>>> import scipy.sparse
I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/global/software/sl-7.x86_64/modules/langs/python/3.6/lib/python3.6/site-packages/scipy/sparse/__init__.py", line 229, in <module>
from .csr import *
File "/global/software/sl-7.x86_64/modules/langs/python/3.6/lib/python3.6/site-packages/scipy/sparse/csr.py", line 15, in <module>
from ._sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \
ImportError: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /global/software/sl-7.x86_64/modules/langs/python/3.6/lib/python3.6/site-packages/scipy/sparse/_sparsetools.cpython-36m-x86_64-linux-gnu.so)
I can even go the directory "/global/software/sl-7.x86_64/modules/langs/python/3.6/lib/python3.6/site-packages/scipy/sparse/" and import the binary "_sparsetools.cpython-36m-x86_64-linux-gnu.so" followed by torch without issue. But if I try it the other way around I again get the above error.
Does anyone have any idea why changing the order of these imports should have a different effect?

The simple search strategy for shared objects assumes that a single version of each exists—or at least that directories containing newer versions are put first on the search path. The path includes $LD_LIBRARY_PATH (which should be avoided), DT_RPATH and its newer variant DT_RUNPATH (which crucially depend on the client being loaded), and system directories like /lib. This works well for systems following the FHS with global package management, but packages that are installed, with copies of their dependencies, in a single per-package directory (as is common on Windows and with some “normal user” package managers) can easily produce multiple versions of a shared object with the same soname.
The expectation is that sharing that name is harmless because one can be used in place of the other (and therefore be put first on the path). The reality is that there is no single directory that is newest for all libraries, and there’s not even a single path to configure given the DT_ tags.
The result is that whichever one is loaded first wins: the dynamic loader can’t load both of them (since they provide many of the same symbols), so the second request has only the effect of checking the library version tags, which here are found to be inadequate. In this case, one client (torch) is relying on the system’s C++ standard library, while the other (_sparsetools) has its own, newer version. It may or may not even need its newer version: since it was built against it, it is conservatively marked as needing it by default.
This is a hard problem: not even tools like virtual environments or environment modules can handle it, since the problem lies in the incompatible compilation of extant packages. Systems that rebuild everything from source (like Nix or Spack) can, but only at their usual cost of controlling all relevant builds. It may be that simply controlling the import order is, unfortunately, the best choice available.

#DavisHerring's answer gives you an explanation what is going on, here is a possible workaround to ensure that the right version is loaded - the LD_PRELOAD-trick:
1. Step:
Find the right libc++.so-version via console:
$ ldd _sparsetools.cpython-36m-x86_64-linux-gnu.so
libstdc++.so.6 => <path to right version>/libstdc++.so.6(...)
2. Step:
While starting python, preload the right version, so the loader picks the right version:
$ LD_PRELOAD=<path to right version>/libstdc++.so python
However the best solution would to rebuild pytorch with the right libc++-dependency.

Getting a 'module not found' error after installing a package

I tried installing the deuces package in python using
python -m pip install deuces
which installed perfectly. But whenever I attempt to call upon the function, I get an error message:
Traceback (most recent call last): File "", line 1, in
File
"C:\Python\Anaconda\lib\site-packages\deuces__init__.py", line 1, in
from card import Card ModuleNotFoundError: No module named 'card'
Even though I can't find anything wrong with the module calling. Would someone be able to check what's going wrong here?
PS: I did read the post Import Error Python: No module named 'card' but found no solution.

It seems the package is using python 2 only relative imports here, which was removed py PEP 404.
These should be changed to either douces.xxx or relative imports .xxx. Currently, your best hope would be to make a PR to fix this, or to fork the library and fix it yourself.

You are most likely trying to run this code in Python 3. Sadly the deuces module's page in the PyPI repository does not make it clear that the module currently only appears to support Python 2, under which the module imports perfectly.
Since it doesn't look like the module has received much attention lately, if you want to run it under Python 3 you may end up doing the port yourself. It doesn't look as though it would take too much work.

Python - How do you import downloaded directories/modules?

This is the first time I have attempted to use anything other than what's provided by python.
I have recently gotten into pythons provided Tkinter, though due to some issues I decided to use another GUI, and heard that PyQt was highly recommended, so I downloaded that and looked into various tutorials. In these tutorials, I cannot seem to execute any of the import statements in said tutorials that relate to PyQt, primarily PyQt5 (I have checked I have the correct version number by the way).
So for instance:
import PyQt5
raises the error:
Traceback (most recent call last):
File "/Users/MEBO/PycharmProjects/Music/testing.py", line 1, in <module>
import Qt
ImportError: No module named 'Qt'
[Finished in 0.1s with exit code 1]
I have a lot of research into this. I've heard people talk of using pip to install modules, and I have done this be safe (as well as downloading it from the internet), I've tried changing the project interpreter to versions Python3/ 2.7/ 2.6, appending the path name to the sys.path directory, (which I really know nothing about to be honest, I was hoping I'd get lucky), though nothing seems to work.
Are you supposed to be able to just import a module off the bat, or do you have to set some things up first?

For windows download the package and extract it to (path where python installed)\Python27\Lib and then try to import.

Specific to PyQt
This package cannot just be downloaded and imported, it must be built because it is not pure python, it uses Qt (C++) and requires dependancies. Read this tutorial on installation.
There is also a very complete python package distribution, Anaconda, that includes pyqt and much more. Almost all the packages I ever looked at are in there.
In general to pure python code
In other cases, if you place modules/code that has been download into the directory that your python script is run from, you can import off the bat, or you can append/insert any folder to the sys.path.
# importer will search here last
sys.path.append('/path/to/code/')
# importer will search here second, right after script's directory
# this can be useful to override a module temporarily...
sys.path.insert(1,'/path/to/code/')

Can only import win32com.client with IDLE. What extra work do i need to set up pywin32?

I have a Tkinter program that i use for time management. I decided to have it scan my inbox in outlook to check emails for tags, and then based on the tag, add it to my list of tasks to do for the night.
The code i wrote works just fine, but i'm getting an error when I import win32com.client. I can import it in IDLE, but it is having problems importing when i try to run the code with a .bat file, or double clicking the .py file.
I have found several people with similar problems, but from what i can tell, it's having problems with win32api module, or pywin32
Traceback (most recent call last):
File "my_program_filename.py", line 1, in <module>
import win32com.client
File "c:/Python27/lib/site-packages/win32com/__init__.py", line 5, in <module>
import win32api, sys, os
ImportError: DLL load failed: The specified module could not be found
I'm really confused. When i get the sys.path it's the same with IDLE as it is running from the .py file with the exception of an added "c:/users/username/desktop/timer" for my .py file.
I'm really lost and haven't had to mess with the system path, but I'm not able to figure out what i need to do to fix this.
What do I need to do to get pywin32 working so I can use win32com.client?
Any help is greatly appreciated. Thanks!

IIRC, the problem was attempting to link to the debug build. I think i had to dupe the release build and rename it to be debug or something.
try building release and see if it "just works". if so, you have a direction to explore

this is an issue of not having the correct paths in the sys.path. If you make sure that all of the pywin32 folders are in the sys.path(If you check in IDLE it can show that the folders are included even when they aren't?!?!?).
You also have to make sure you run your code from inside your Python directory or it will fail to import win32api. I also found that if you do anything in a function that uses pywin32 and you accidentally misspell the function when you call it, the whole import fails without telling you your function is misspelled. You can also navigate to the /Python27/Lib/site-packages/win32com/client folder and run makepy.py to make sure the right Object library is installed.
When you run makepy.py, you select the COM object you want to use and it creates packages specific to what you want to use. I don't fully understand why this is, but once i did this and ran my file from the Python folder it worked! There is a more in depth explanation on how to get this working right here.

I found this link to be the solution rather than the win32com/client as indicated above: win32com import error python 3.4

in my case typing in cmd:
python C:\Python27\Scripts\pywin32_postinstall.py -install
I hope this helps

Python program run in MATLAB can't import pygame

I'm trying to run a Python program, which uses the pygame modules, from MATLAB. I know I can use either
system('python program.py')
or just
! python program.py
However, I keep getting the error:
Traceback (most recent call last):
File "program.py", line 1, in <module>
import pygame
ImportError: No module named pygame
What is strange is that if I run the program from the command line, it works just fine. Does anyone know why if run from within MATLAB, Python can't find pygame?

The problem may be that MATLAB is not seeing your PYTHONPATH, which normally stores Python libraries and modules. For custom modules, PYTHONPATH should also include the path to your custom folders.
You can try setting the value of PYTHONPATH from within a MATLAB running session:
PATH_PYTHON = '<python_lib_folder>'
setenv('PYTHONPATH', PATH_PYTHON); % set env path (PYTHONPATH) for this session
system('python program.py');
See also the possibly relevant SO answer here: How can I call a Qtproject from matlab?

As I haven't used matlab too often and don't have the program available now I cannot say for sure, but matlab may be creating a custom environment with custom paths (this happens a lot, so the user has a very consistent experience in their software). When matlab installs it may not export paths to its own modules to your default environment. So when calling for pygame.py outside of matlab, python cannot find pygame.py under its usual lookup paths.
Solutions could be:
find the pygame.py, and map the path to it directly in your code, though this could cause you headaches later on during deployment
Try just copying the pygame.py file to your working directory, could have dependences that need to addressed.
Install pygame directly from its developer at http://www.pygame.org. Version differences could be a problem but pygame gets put under the usual lookup paths for python. (This would be my preferred solution personally.)
Or just export the location of path to pygame in matlab's library to your default enivronment. This could be a problem during deployment too.

For posterity, first try everything that Stewie noted here ("Undefined variable "py" or class" when trying to load Python from MATLAB R2014b?). If it doesnt work then it's possible that you have multiple pythons. You can try and check which python works (with all the related installed modules) on your bash/terminal. And then use
pyversion PYTHONPATH
to let matlab know the right path.
Also use py.importlib.import_module('yourmodule') to import the module after that.
That should get you started.