Develop Reference

Errors when installing MySQL-python package after mysql-connector-c got upgraded

Previously, my installation works normally, since it uses pip's cache directory. But when I installed a brew package, brew automatically removed mysql-connector-c, and added mysql-client 8.0.19 instead. From that moment, my application can't connect to the DB anymore:
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: dlopen([venv_location]/lib/python2.7/site-packages/_mysql.so, 2):
Library not loaded: /usr/local/opt/mysql-connector-c/lib/libmysqlclient.18.dylib
Referenced from: [venv_location]/lib/python2.7/site-packages/_mysql.so Reason: image not found.
It seems that the MySQL-python package still points to the old mysql-connector-c. I decided to reinstall the package without using the cache:
pip install MySQL-python==1.2.5 --no-cache-dir
But I got this error instead:
creating build/temp.macosx-10.15-x86_64-2.7
cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -I/usr/local/opt/openssl/include -arch x86_64 -pipe -Dversion_info=(1,2,5,'final',1) -D__version__=1.2.5 -I/usr/local/opt/mysql-client/include/mysql -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _mysql.c -o build/temp.macosx-10.15-x86_64-2.7/_mysql.o
_mysql.c:44:10: fatal error: 'my_config.h' file not found
#include "my_config.h"
^~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
I have searched for this error, but the answers are usually related to brew install mysql-connector-c, while this package is not available from brew anymore. So how can I handle this exception?
Here is my /usr/local/opt (../Cellar/mysql-client/6.1.11 is invalid since it has been upgraded):
mysql-client -> ../Cellar/mysql-client/8.0.19
mysql-connector-c -> ../Cellar/mysql-client/6.1.11
Should I reinstall the old version of mysql-client to install MySQL-python from cache and keep using the old version? Or is there another way to deal with 'my_config.h' file not found?

This procedure worked for me (I'm adding the mysql-client and the mysql-connector-c removal to it):
brew remove mysql mysql-client mysql-connector-c
brew install mysql#5.7
brew link --force mysql#5.7
pip install mysql-python
I read a post and adjusted the procedure to link things in the /usr/local/bin instead of adding a new route to the PATH variable.
Reference: MacOS: "pip install MySQL-python" returning an error: "_mysql.c:44:10: fatal error: 'my_config.h' file not found"

Can't install C-based modules like lxml and Cython on Python 3.6 on Raspberry Pi

I built Python 3.6 from source and am trying to install lxml. Trying to install it from pip gives the following error:
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -Isrc -Isrc/lxml/includes -I/usr/local/include/python3.6m -c src/lxml/etree.c -o build/temp.linux-armv6l-3.6/src/lxml/etree.o -w
src/lxml/etree.c:91:20: fatal error: Python.h: No such file or directory
compilation terminated.
Compile failed: command 'gcc' failed with exit status 1
cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInitln68cjsn.c -o tmp/xmlXPathInitln68cjsn.o
cc tmp/xmlXPathInitln68cjsn.o -L/usr/lib -lxml2 -o a.out
error: command 'gcc' failed with exit status 1
The same error is raised if I try to install Cython to compile lxml from source. I installed libxml2-dev and libxslt1-dev, but python3.6-dev, which is apparently also required, is not found in Raspbian repository. I added the following line to /etc/apt/sources.list:
deb http://ftp.de.debian.org/debian sid main
But when trying to install it afterwards, I receive the following error:
The following packages have unmet dependencies:
libdbus-1-3 : Breaks: dbus (< 1.9.16-1~) but 1.6.8-1+deb7u1 is to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
In fact, this error is raised on any package that is found in Debian repository that is not in Raspbian one. What can I do about this?

Since you installed Python from source, you should have the header files already. Should also use header files for the version that you built as opposed to the version in the package manager.
Either
Add the source code directory to includes used by GCC (gcc -I<dir>) or
Install the from source package in a system directory that is automatically included like /usr/include or /usr/local/include
The second option is probably easiest and should be possible from the standard configure/make scripts.

Fatal error when trying to install PyCrypto on OS X El Capitan

I am trying to install PyCrypto on OS X 10.11.3 (El Capitan). I am using Python 3.5.1. I downloaded the gzip file from https://pypi.python.org/pypi/pycrypto and decompressed it. Then I ran python setup.py build like the instructions said and it appeared to do something, then it produced this output:
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -fwrapv -Wall -Wstrict-prototypes -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/MD2.c -o build/temp.macosx-10.6-intel-2.7/src/MD2.o
src/MD2.c:30:10: fatal error: 'string.h' file not found
#include <string.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
I tried python3 setup.py build and got some very similar output:
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -fwrapv -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c src/MD2.c -o build/temp.macosx-10.6-intel-3.5/src/MD2.o
src/MD2.c:30:10: fatal error: 'string.h' file not found
#include <string.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
I tried Googling to figure out what to do, but I couldn't find anything useful. How can I install PyCrypto?
EDIT: I also tried several other things like pip install pycrypto and sudo pip3 install pycrypto and they didn't work. #l'L'l helped me get it to work by doing several strange, complex things that I never would have though have myself. They are summarized in the answer below.

Overview:
The manual build you're trying looks like it might be failing because it's referencing the OS X 10.6 SDK, which you likely don't have, and is outdated for the most part. Also, SDKs are now stored in a completely different location than when the 10.6 SDK was in it's prime.
New SDKs location:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/
Old SDKs location:
/Developer/SDKs/
Non-existent / outdated SDK:
Since it appears that when trying to build PyCrypto it's referencing the MacOSX10.6.sdk there are several things to consider:
Why does it reference an outdated SDK
Where is the SDK it's referencing set
What should be done to correct the issue
Unless we audit the source code carefully we might not know exactly where the incorrect flags are set, but we can do our best to work with the information we have. From the error we can see that there are several instances where the 10.6 SDK's name pops up:
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -fwrapv -Wall -Wstrict-prototypes -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library
Building from source:
/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/MD2.c -o build/temp.macosx-10.6-intel-2.7/src/MD2.o
src/MD2.c:30:10: fatal error: 'string.h' file not found
#include <string.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
Analyzing this we can see that the PyCrypto's MD2.c file is trying to be built using the flag -isysroot /Developer/SDKs/MacOSX10.6.sdk. It might be worth trying pip instead:
Installing with pip:
...
fatal error: 'string.h' file not found #include <string.h>
...
Same error; we should probably find out if the <string.h> header even exists on the system — Let's make a quick test C application to find out:
Testing the C headers:
$ echo "#include <string.h>
#include <stdio.h>
int main() { printf(\"TEST\n\"); return 0; }" > t.c
$ clang t.c -o t
$ ./t
TEST
It's apparent the header does exist because the test worked fine. This tells us that the problem is more likely related directly to the 10.6 SDK (which doesn't seem to exist on the system).
Symlinking (non-existing) 10.6 SDK to 10.11 SDK:
Since we haven't determined where the SDK is actually getting set we'll go ahead and try to create symlinks so that any reference of the old 10.6 SDK links to the latest SDK (10.11 at this time):
$ cd /Developer/SDKs
$ sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk MacOSX10.6.sdk
We can verify the symlink by issuing the following command:
$ ls -lat
total 8
drwxr-xr-x 3 root wheel 102 Feb 21 15:54 .
lrwxr-xr-x 1 root wheel 99 Feb 21 15:54 MacOSX10.6.sdk -> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk
drwxr-xr-x 3 root wheel 102 Feb 21 15:52 ..
Now that we've successfully created the symlink let's try installing PyCrypto with pip once more:
$ sudo pip install pycrypto
Collecting pycrypto
Downloading pycrypto-2.6.1.tar.gz (446kB)
100% |████████████████████████████████| 446kB 1.2GB/s
Installing collected packages: pycrypto
Running setup.py install for pycrypto ... done
Successfully installed pycrypto-2.6.1
No errors! It looks like our problem is solved! Well, almost...
We still need to figure out what is responsible for setting the wrong (10.6) SDK during builds. Let's use the xcrun tool to see what the defaults are set at:
$ xcrun --show-sdk-version
10.11
The system default SDK is set to 10.11, so it must be get set incorrectly to 10.6 by Python, PyCrypto, or some other anomaly we might not have considered.
UPDATE:
After doing some recon it was discovered that Python 3 appears to be built with the OS X 10.6 SDK. In addition it's also setting the SDK to 10.6 and setting the (outdated) path in numerous places throughout the Python_Framework. There are so many references I won't bother listing them all, although here's an example:
Python_Framework Folder/Versions/3.5/lib/python3.5/config-3.5m/Makefile:79:CONFIGURE_CFLAGS= -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk
I can only assume the developers were trying to be as backward compatible as possible, however, it's unfortunately breaking forward compatibility in the process.
Notes:
Installing Python packages with pip can make life much easier in a lot of ways (package management, updating, uninstalling, etc.). For example installing PyCrypto should just be a matter of issuing the command:
$ sudo pip install pycrypto
If you have multiple Python's you can use the version number to install for that Python accordingly:
$ sudo pip3.5 install pycrypto
↳ https://pip.pypa.io/en/stable/installing/

I had what might be a related issue on OSX El Capitan when I would run pip install pycrypto. I was seeing RuntimeError: autoconf error. All I had to do was run sudo xcodebuild -license and type agree after reviewing the licence agreement. Afterward I was able to use pip to install pycrypto.

Installing Matplotlib via pip on Mac OS X Mavericks failing due to Coacoa error?

I've been wrestling with getting the matplotlib library installed for python for a few days now. I've got freetype, XCode, X11, and the bindings sorted out, but now when I try to install matplotlib I get the following error:
building 'matplotlib.backends._macosx' extension
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2
-DNDEBUG -g -O3 -DPY_ARRAY_UNIQUE_SYMBOL=MPL_matplotlib_backends__macosx_ARRAY_API
-DPYCXX_ISO_CPP_LIB=1 -I/Library/Frameworks/Python.framework/Versions/2.7/lib
/python2.7/site-packages/numpy/core/include -I/usr/local/include -I/usr/include
-I/usr/X11/include -I/opt/local/include -I. -Iagg24/include
-I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-c src/_macosx.m
-o build/temp.macosx-10.6-intel-2.7/src/_macosx.o
In file included from /System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:161,
from /System/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12,
from src/_macosx.m:1:
/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:16:
error: expected ‘,’ or ‘}’ before ‘__attribute__’
This comes along with a number of warnings about deprecated numpy API's, but those seem to be causing no problems. Can anyone tell me what the problem is here?
For the record, I'm using gcc-4.2.

Just edit /System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:
sudo nano /System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h
Change line 16 from:
NSUserNotificationActivationTypeReplied NS_AVAILABLE(10_9, NA) = 3
to:
NSUserNotificationActivationTypeReplied /* NS_AVAILABLE(10_9, NA) */ = 3
Then retry installing matplotlib:
sudo pip install matplotlib
That solved it for me!
Source: https://www.mail-archive.com/macosx-port-dev#openjdk.java.net/msg00215.html

Check the output of which -a gcc-4.2. If its /usr/bin/gcc-4.2 you aren't using the Xcode supplied gcc. To remedy this, run your build command (or pip, etc) prefixed with CC=/usr/bin/gcc. That should fix this error.

Trying to install ephem-3.7.5.1 on Mac OS X 10.8.4

When I execute the install script, I get the following:
creating build/temp.macosx-10.6-intel-3.3/extensions/data
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -arch i386 -arch x86_64 -Ilibastro-3.7.5 -I/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m -c extensions/_libastro.c -o build/temp.macosx-10.6-intel-3.3/extensions/_libastro.o
In file included from extensions/_libastro.c:3:
/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m/Python.h:25:10: fatal error:
'stdio.h' file not found
#include <stdio.h>
^
1 error generated.
error: command '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang' failed with exit status 1
Any clues? Thanks

Rather than try to debug your reported error, it is probably easier to just to use:
sudo pip install pyephem
If you don't have pip installed, you should get it using easy_install, but you could also just install it with the other command recommended in the INSTALL instructions:
sudo easy_install ephem

That .h file is a fundamental C standard library file, so its absence indicates that your problem may be that you are missing the full install of your system's C compiler. Try installing the Xcode command line tools that I think you can find here:
https://developer.apple.com/xcode/
A similar Stack Overflow questioner found that they were also necessary to build Python itself from source:
fatal error 'stdio.h' Python 2.7.3 on Mac OS X 10.7.5
Update: the following command might also help, so here it is just in case:
http://www.bitemyapp.com/post/os-x-fsevent-fatal-error-stdio-h-file-not-found/