Develop Reference

Cannot install nlopt python module

I am trying to install nlopt onto macOS 10.15.5. I downloaded the nlopt-2.6.2.tar.gz file from the NLopt docs and ran the following from the nlopt-2.6.2 directory:
mkdir build
cd build
cmake -DNLOPT_OCTAVE=Off -DNLOPT_MATLAB=Off -DNLOPT_GUILE=Off ..
make
sudo make install
I got the following output: cmake.txt.
The header file (nlopt.h) installs correctly to /usr/local/include and the dynamic library (libnlopt.dylib) installs correctly to /usr/local/lib/, but neither the dist-info file nor the nlopt module itself installs.
I have also tried installing via pip, brew, and conda, none of which have worked. I have also tried cloning from this Github, which didn't work either.
I appreciate any help with this, because I am completely lost. I am relatively new to this kind of stuff, and I couldn't find any good answers online.

The official docs are somewhat laconic about the exact steps required for building nlopt with Python bindings. First of all, you'll need SWIG installed:
$ brew install swig
Then, you'll need numpy to be available for the target Python interpreter. It is already preinstalled for the system Python, otherwise install it via Homebrew or pip, depending on your Python installation.
Now run cmake:
$ cmake -DNLOPT_GUILE=OFF -DNLOPT_MATLAB=OFF -DNLOPT_OCTAVE=OFF -DNLOPT_TESTS=OFF
This will build bindings against the default Python 2.7 installation preinstalled on MacOS. If you need to build against custom Python installation (e.g. when you've installed Python 3 via Homebrew or PKG installer from https://www.python.org/downloads), pass it via the PYTHON_EXECUTABLE arg:
$ cmake -DNLOPT_GUILE=OFF -DNLOPT_MATLAB=OFF -DNLOPT_OCTAVE=OFF -DNLOPT_TESTS=OFF -DPYTHON_EXECUTABLE=/usr/local/bin/python3
Inspect the log now - Python, SWIG and numpy headers should be successfully located. Example output snippet (you may have different paths/versions printed):
-- Found PythonInterp: /usr/local/bin/python3.8 (found version "3.8.3")
-- Found PythonLibs: /Library/Frameworks/Python.framework/Versions/3.8/lib/libpython3.8.dylib (found suitable exact version "3.8.3")
-- Found NumPy: /Users/hoefling/Library/Python/3.8/lib/python/site-packages/numpy/core/include (found version "1.19")
-- Found SWIG: /usr/local/bin/swig (found version "4.0.2")
If any of those conditions is not satisfied (e.g. you see Could NOT find NumPy, Could NOT find PythonLibs or Could NOT find SWIG), then stop and make sure the configuration succeeds before proceeding next.
Now compile:
$ make
...
Scanning dependencies of target nlopt_python_swig_compilation
[ 96%] Swig compile nlopt.i for python
[ 96%] Built target nlopt_python_swig_compilation
Scanning dependencies of target nlopt_python
[ 98%] Building CXX object src/swig/CMakeFiles/nlopt_python.dir/CMakeFiles/nlopt_python.dir/nloptPYTHON_wrap.cxx.o
[100%] Linking CXX shared module _nlopt.so
[100%] Built target nlopt_python
Install:
$ make install
...
-- Installing: /usr/local/lib/python3.8/site-packages/nlopt.py
-- Installing: /usr/local/lib/python3.8/site-packages/_nlopt.so
Test the Python bindings are importable:
$ python -c "import nlopt; print(nlopt.__version__)"
2.6.2

python installing thread-local storage

I'm trying to install a pysnap library from github. This is part of the output I get. I have no idea how to proceed, any help would be greatly appreciated.
Installed /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pysnap-0.1.1-py2.7.egg
Processing dependencies for pysnap==0.1.1
Searching for cryptography>=1.2.2
Reading https://pypi.python.org/simple/cryptography/
Best match: cryptography 1.2.3
Downloading https://pypi.python.org/packages/source/c/cryptography/cryptography-1.2.3.tar.gz#md5=5474d2b3e8c7555a60852e48d2743f85
Processing cryptography-1.2.3.tar.gz
Writing /tmp/easy_install-7ZnmT5/cryptography-1.2.3/setup.cfg
Running cryptography-1.2.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-7ZnmT5/cryptography-1.2.3/egg-dist-tmp-rcoXcx
_configtest.c:1:1: error: thread-local storage is unsupported for the current target
__thread int some_threadlocal_variable_42;
1 error generated.
error: unable to create target: 'No available targets are compatible with this triple, see -version for the
available targets.'
1 error generated.

What platform are you trying to install pysnap?
I downloaded and installed pysnap sucessfully from this github location
I do have windows 10 platform and python 2.7 though. I dont' have a linux to replicate this issue.
If you are trying to install on Linux, depending on flavor and its setup , sometimes swap is setup as as /tmp and some applications are not able to write to /tmp. In your case it does looks like its trying to write to /tmp and erroring out.
Can you try pip install cryptography
and see if its able to install cryptography part without issues?
Once that dependency is out of way, you can install pysnap so that pysnap will not try to install cryptography

Error installing Willie irc bot for Python

Trying to install this Python IRC bot called Willie. This is what happens when I run setup.py:
C:\Python34\python.exe "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.5.1\helpers\pycharm\pycharm_setup_runner.py" C:\Users\Quibbles\Documents\willie-5.3.0\setup.py
Testing started at 1:35 PM ...
running pycharm_test
Searching for pygeoip
Best match: pygeoip 0.3.2
Processing pygeoip-0.3.2-py3.4.egg
Using c:\users\quibbles\documents\willie-5.3.0\.eggs\pygeoip-0.3.2-py3.4.egg
Searching for pyenchant
Best match: pyenchant 1.6.6
Processing pyenchant-1.6.6-py3.4-win32.egg
Using c:\users\quibbles\documents\willie-5.3.0\.eggs\pyenchant-1.6.6-py3.4-win32.egg
Searching for praw
Best match: praw 2.1.21
Processing praw-2.1.21-py3.4.egg
Using c:\users\quibbles\documents\willie-5.3.0\.eggs\praw-2.1.21-py3.4.egg
Searching for lxml
Reading https://pypi.python.org/simple/lxml/
Best match: lxml 3.4.4
Downloading https://pypi.python.org/packages/source/l/lxml/lxml-3.4.4.tar.gz#md5=a9a65972afc173ec7a39c585f4eea69c
Processing lxml-3.4.4.tar.gz
Writing C:\Users\Quibbles\AppData\Local\Temp\easy_install-p_x9naya\lxml-3.4.4\setup.cfg
Running lxml-3.4.4\setup.py -q bdist_egg --dist-dir C:\Users\Quibbles\AppData\Local\Temp\easy_install-p_x9naya\lxml-3.4.4\egg-dist-tmp-0expfysa
Building lxml version 3.4.4.
Building without Cython.
ERROR: b"'xslt-config' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"
** make sure the development packages of libxml2 and libxslt are installed **
Using build configuration of libxslt
C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
cl : Command line warning D9025 : overriding '/W3' with '/w'
lxml.etree.c
C:\Users\Quibbles\AppData\Local\Temp\easy_install-p_x9naya\lxml-3.4.4\src\lxml\includes\etree_defs.h(14) : fatal error C1083: Cannot open include file: 'libxml/xmlversion.h': No such file or directory
error: Setup script exited with error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\BIN\\cl.exe' failed with exit status 2
Process finished with exit code 1
What am I doing wrong? I downloaded the source code from https://github.com/embolalia/willie/releases/tag/v5.3.0 and tried running setup.py in PyCharm.

It should be fairly obvious by the error messages:
** make sure the development packages of libxml2 and libxslt are installed **
You need to have the dev libxml2 and libxslt installed. It appears either you have the wrong versions perhaps, or don't have them installed properly.

I'L'I's answer gives you a hint of where to start searching next, but isn't a useful answer.
After several attempts on my own, I found that to get the lxml library installed, I needed to do the following:
First, download the lxml library from http://www.lfd.uci.edu/~gohlke/pythonlibs/
This is necessary because the PiPy archives (https://pypi.python.org/pypi/lxml/3.4.4) are way out of date, only giving you the option to install for Python 3.2 for the 3+ series, which of course is useless if you're on 3.4. There's also issues if you're using 64-bit. They may be sufficient if you're using 2.7, though.
Then use the instructions here: How do I install a Python package with a .whl file? to install the .whl file.
However that only fixes half the problem. The willie package appears to be broken in other ways (on Windows) because of trying to reference a unix-like /tmp directory (absolute path) instead of the proper Windows Temp directory.
This issue describes the problem, though no fix has yet been made: https://github.com/embolalia/willie/issues/811
It indicates that it's a Windows-specific bug in the installer. To work around that, download the .tar.gz from https://pypi.python.org/pypi/willie/5.3.0
Unpack it. Go to the willie-5.3.0\willie.egg-info directory, and edit the SOURCES.txt file. The fifth line is /tmp/tmpeAhjCF/willie. Delete that line.
Repackage the directory into a .tar.gz archive. Install via pip install .\willie-5.3.0.tar.gz.
Now it's installed, and you're ready for the next round of problems of actually getting it to run on Windows. Since I haven't actually succeeded in doing that yet, that's as far as I can help for right now.

How to deal with Linux/Python dependencies?

Due to lack of support for some libraries I want to use, I moved some Python development from Windows to Linux development. I've spent most of the day messing about getting nowhere with dependencies.
The question
Whenever I pick up Linux, I usually run into some kind of dependency issue, usually with development libraries, whether they're installed via apt-get, easy_install or pip. I can waste days on what should be simple tasks, spending longer on getting libraries to work than writing code. Where can I learn about strategy for dealing with these kind of issues rather than aimlessly googling for someone who's come across the same problem before?
An example
Just one example: I wanted to generate some QR codes. So, I thought I'd use github.com/bitly/pyqrencode which is based on pyqrcode.sourceforge.net but supposedly without the Java dependencies. There are others (pyqrnative, github.com/Arachnid/pyqrencode) but that one seemed like the best bet for my needs.
So, I found the package on pypi and thought using that would make life easier:
(I've perhaps made life more difficult for myself by using virtualenv to keep things neat and tidy.)
(myenv3)mat#ubuntu:~/myenv3$ bin/pip install pyqrencode
Downloading/unpacking pyqrencode
Downloading pyqrencode-0.2.tar.gz
Running setup.py egg_info for package pyqrencode
Installing collected packages: pyqrencode
Running setup.py install for pyqrencode
building 'qrencode' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c qrencode.c -o build/temp.linux-i686-2.7/qrencode.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions build/temp.linux-i686-2.7/qrencode.o -lqrencode -o build/lib.linux-i686-2.7/qrencode.so
Successfully installed pyqrencode
Cleaning up...
(I guess I probably sudo apt-get install libqrencode-dev at some point prior to that too.)
So then I tried to run the test script:
(myenv3)mat#ubuntu:~/myenv3$ python test_qr.py
Traceback (most recent call last):
File "test_qr.py", line 1, in <module>
from qrencode import Encoder
File "qrencode.pyx", line 1, in init qrencode (qrencode.c:1520)
ImportError: No module named ImageOps
:(
Well, investigations revealed that ImageOps appears to be part of PIL...
(myenv3)mat#ubuntu:~/myenv3$ pip install pil
Downloading/unpacking pil
Downloading PIL-1.1.7.tar.gz (506Kb): 122Kb downloaded
Operation cancelled by user
Storing complete log in /home/mat/.pip/pip.log
(myenv3)mat#ubuntu:~/myenv3$ bin/pip install pil
Downloading/unpacking pil
Downloading PIL-1.1.7.tar.gz (506Kb): 506Kb downloaded
Running setup.py egg_info for package pil
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
Installing collected packages: pil
Running setup.py install for pil
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
building '_imaging' extension
gcc ...
building '_imagingmath' extension
gcc ...
--------------------------------------------------------------------
PIL 1.1.7 SETUP SUMMARY
--------------------------------------------------------------------
version 1.1.7
platform linux2 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)
[GCC 4.5.2]
--------------------------------------------------------------------
*** TKINTER support not available
*** JPEG support not available
*** ZLIB (PNG/ZIP) support not available
*** FREETYPE2 support not available
*** LITTLECMS support not available
--------------------------------------------------------------------
To add a missing option, make sure you have the required
library, and set the corresponding ROOT variable in the
setup.py script.
To check the build, run the selftest.py script.
...
Successfully installed pil
Cleaning up...
Hmm, PIL's installed but hasn't picked up the libraries I installed with sudo apt-get install libjpeg62 libjpeg62-dev libpng12-dev zlib1g zlib1g-dev earlier. I'm not sure how to tell pip to feed the library locations to setup.py. Googling suggests a variety of ideas which I've tried, but none of them seem to help much other than to send me round in circles.
Ubuntu 11.04: Installing PIL into a virtualenv with PIP suggests using the pillow package instead, so let's try that:
(myenv3)mat#ubuntu:~/myenv3$ pip install pillow
Downloading/unpacking pillow
Downloading Pillow-1.7.5.zip (637Kb): 637Kb downloaded
Running setup.py egg_info for package pillow
...
Installing collected packages: pillow
Running setup.py install for pillow
building '_imaging' extension
gcc ...
--------------------------------------------------------------------
SETUP SUMMARY (Pillow 1.7.5 / PIL 1.1.7)
--------------------------------------------------------------------
version 1.7.5
platform linux2 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)
[GCC 4.5.2]
--------------------------------------------------------------------
*** TKINTER support not available
--- JPEG support available
--- ZLIB (PNG/ZIP) support available
--- FREETYPE2 support available
*** LITTLECMS support not available
--------------------------------------------------------------------
To add a missing option, make sure you have the required
library, and set the corresponding ROOT variable in the
setup.py script.
To check the build, run the selftest.py script.
...
Successfully installed pillow
Cleaning up...
Well, we seem to have the JPEG and PNG support this time, yay!
(myenv3)mat#ubuntu:~/myenv3$ python test_qr.py
Traceback (most recent call last):
File "test_qr.py", line 1, in <module>
from qrencode import Encoder
File "qrencode.pyx", line 1, in init qrencode (qrencode.c:1520)
ImportError: No module named ImageOps
Still no ImageOps though. Now I'm stumped, is ImageOps missing from pillow, or is it a different problem that was also there with pil.

I see two separate problems here:
Keeping track of all the python modules you need for your project.
Keeping track of all the dynamic libraries you need for the python modules in your project.
For the first problem, I have found that buildout is good help, althought it takes a litle while to grasp.
In your case, I would start by creating a directory for my new project. I would then go into that directory and download bootstrap.py
wget http://python-distribute.org/bootstrap.py
I would then create a buildout.cfg file:
[buildout]
parts = qrproject
python
eggs = pyqrencode
[qrproject]
recipe = z3c.recipe.scripts
eggs = ${buildout:eggs}
entry-points= qrproject=qrprojectmodule:run
extra-paths = ${buildout:directory}
# This is a simple way of creating an interpreter that will have
# access to all the eggs / modules that this project uses.
[python]
recipe = z3c.recipe.scripts
interpreter = python
eggs = ${buildout:eggs}
extra-paths = ${buildout:directory}
In this buildout.cfg I'm referencing the module qrprojectmodule (in entry-points under [qrproject]. This will create a bin/qrproject that runs the function run in the module qrprojectmodule. So I will also create the file qrprojectmodule.py
import qrencode
def run():
print "Entry point for qrproject. Happily imports qrencode module"
Now it's time to run bootstrap.py with the python binary you want to use:
python bootstrap.py
Then run the generated bin/buildout
bin/buildout
This will create two additional binaries in the bin/ directory - bin/qrproject and bin/python. The former is your project's main binary. It will be created automatically each time you run buildout and will have all the modules and eggs you want loaded. The second is simply a convenient way to get a python prompt where all your modules and eggs are loaded, for easy debugging. The fine thing here is that bin/buildout will automatically install any python eggs that the eggs (in your case pyqrencode) have specified as dependencies.
Actually, you will probably get a compilation error in the step where you run bin/buildout. This is because you need to address problem 2: All dynamic libraries being available on your system. On Linux, it's usually best to get help from your packaging system. I'm going to assume you're using a Debian derivate such as Ubuntu here.
The pyqrencode web site specifies that you need the libqrencode library for pyqrencode to work. So I used my package manager to search for that:
$ apt-cache search libqrencode
libqrencode-dev - QR Code encoding library -- development
libqrencode3 - QR Code encoding library
qrencode - QR Code encoder into PNG image
In this case, I want the -dev package, as that installs linkable libraries and header files required to compile python C-modules. Also, the dependency system in the package manager will make sure that if I install libqrencode-dev, I will also get libqrencode3, as that is required at runtime, i.e. after compilation of the module.
So, I install the package:
sudo apt-get install libqrencode-dev
Once that has completed, rerun bin/buildout and the pyqrencode module will (hopefully) compile and install successfully. Now try to run bin/qrproject
$ bin/qrproject
Entry point for qrproject. Happily imports qrencode module
Success! :-)
So, in summary:
Use buildout to automatically download and install all the python modules/eggs you need for your project.
Use your system's package manager to install any dynamic (C) libraries required by the python modules you use.
Be aware that in many cases there are already packaged versions of your python modules available in the package system. For example, pil is available by installing the python-imaging package on Ubuntu. In this case, you don't need to install it via buildout, and you don't need to worry about libraries being available - the package manager will install all dependencies required for the module to run. Doing it via buildout can however make it easier to distribute your project and make it run on other systems.

Your story reminds me of my early experiences with Linux, and why I love APT.
There is no universal solution to your general problem; the best you can do is to take advantage of the work or others. The Debian packagers do a great job of flagging the dependencies of packages, so apt-get will pull in what you need. So, my strategy is simply to avoid building and installing stuff on my own, and use apt-get wherever possible.
Note that Ubuntu is based on Debian and thus gains the benefit of the work of the Debian packagers. I haven't used Fedora but I hear that the packages are not as well-organized as the ones from Debian.

Trouble Installing iGraph on Windows (i386 and x64)

I usually install R packages using the command below, but it will not work for the network graph ad-on iGraph. T think this is because I am using the new version of R (2.12) which seems to have changed package installation (now split between i386/x64).
My question is, what is the easiest way to get up and running with iGraph. Should I get up to speed troubleshooting compiling in windows, or should i downgrade to an earlier version of R?
> install.packages('igraph', type='source')
trying URL 'http://ftp.heanet.ie/mirrors/cran.r-project.org/src/contrib/igraph_0.5.5.tar.gz'
Content type 'application/x-gzip' length 1135577 bytes (1.1 Mb)
opened URL
downloaded 1.1 Mb
* installing *source* package 'igraph' ...
ERROR: configuration failed for package 'igraph'
* removing 'C:/PROGRA~1/R/R-212~1.0/library/igraph'
The downloaded packages are in
‘C:\Users\SSSS\AppData\Local\Temp\RtmpN4LiQG\downloaded_packages’
Warning message:
In install.packages("igraph", type = "source") :
installation of package 'igraph' had non-zero exit status

In version 2.12.0 using the following GUI commands worked for me
Packages -> Select Repositories -> CTRL Click all repositories
Packages -> Install Packages -> Click igraph
Once you have this done, be sure to add "library(igraph)" to your script.

Generally users of Windows packages are discouraged from be installing from source, but I see that the Windows version has not passed CRAN Package checks. Have you gotten Murdoch's RTools bundle downloaded and properly configured?
Here's a link that may be useful if your target is Win64:
http://r.789695.n4.nabble.com/Rtools-for-building-64-bit-windows-packages-td2021034.html