Develop Reference

PYTHON 3.7.4 NOT USING SQLITE 3.29.0

OS/Software installed:
root#TACIT admin]# cat /etc/*release*
CentOS Linux release 7.6.1810 (Core)
root#TACIT admin]# python3.7 --version
Python 3.7.4
[root#TACIT admin]# sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6
(T3PSA) [root#TACIT src]# django-admin --version
2.2
Software locations:
[root#TACIT admin]# which python3.7
/usr/local/bin/python3.7
[root#TACIT admin]# which sqlite3
/usr/bin/sqlite3
(T3PSA) [root#TACIT src]# which django-admin
/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/bin/django-admin
I compiled Python 3.7.4 from source and installed following these instructions without any problems:
https://tecadmin.net/install-python-3-7-on-centos/
I upgraded from Sqlite 3.7.17 to Sqlite 3.29.0 following these instructions without any problems (other than I had to install some additional ".so" libraries):
https://linuxhint.com/upgrade-to-latest-sqlite3-on-centos7/
Unfortunately Python 3.7.4 is still using the old version of Sqlite3 (3.7.17):
[root#TACIT admin]# python3.7
Python 3.7.4 (default, Aug 16 2019, 16:34:12)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.7.17'
I'm using Django and need it to run at least v3.8.3 (see the very end of the following output):
(T3PSA) [root#TACIT src]# python manage.py makemigrations
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/apps/config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/contrib/auth/models.py", line 2, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
class AbstractBaseUser(models.Model):
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/base.py", line 117, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/base.py", line 321, in add_to_class
value.contribute_to_class(cls, name)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/options.py", line 204, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/__init__.py", line 28, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/utils.py", line 201, in __getitem__
backend = load_backend(db['ENGINE'])
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
return import_module('%s.base' % backend_name)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 66, in <module>
check_sqlite_version()
File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 63, in check_sqlite_version
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
django.core.exceptions.ImproperlyConfigured: **SQLite 3.8.3 or later is required (found 3.7.17).**
I've looked at the following posts to try to figure this out but I think I don't know enough about compiling code to figure out the proper solution. Nothing I have tried from these links works.
https://superuser.com/questions/1425212/where-does-django-look-for-sqlite-instance-sqlite-3-8-3-or-later-is-required
https://unix.stackexchange.com/questions/434100/updating-the-sqlite-version-used-by-python-3-on-centos-7
django can't find new sqlite version? (SQLite 3.8.3 or later is required (found 3.7.17))
https://superuser.com/questions/1447285/python-wont-compile-with-a-new-sqlite3-on-centos-7
It could be that the instructions I'm following don't work because my software installation locations/paths are different from the users in the above posts, I'm not sure... that's why I noted "Software locations:" above. Or maybe I need to uninstall Python 3.7 somehow rather than 'reinstall over the top' of an existing installation? I installed from source, not sure if I need to do this and if so, how to go about it properly.
Any help resolving this would be greatly appreciated. Thanks in advance!
=========================================================================
This section added after getting instructions from FlipperPA
Followed instructions exactly to "Upgrade the CentOS 7 system install of SQLite to 3.29 by compiling it from source". Below is the result.
[root#TACIT src]# sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6
I had already deleted my original source compile directory (/usr/src/Python-3.7.4) several times yesterday as I tried all of the other articles from the original post. So starting from scratch, I did this:
cd /usr/src
tar -zxvf Python-3.7.4.tgz
cd Python-3.7.4
./configure --enable-optimizations
make altinstall
make clean
make install
[root#TACIT Python-3.7.4]# python3.7
Python 3.7.4 (default, Aug 17 2019, 08:20:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.7.17'
Python is still using the wrong version of SQLite. I did the 'make altinstall' command because CentOS 7.6 comes with Python 2.7.5 by default and I read that I shouldn't mess with that. So 'make altinstall' installs Python 3.7.4 in another location so as to leave the default installation of python fully intact.
=========================================================================
Manually uninstalling Python 3.7.4 (installed from source)
So I did the following:
find / -name *3.7*
The resultant list of files is below. The first group of files/folders I left alone. The second group of files/folders I removed.
/var/lib/yum/yumdb/s/27f7f0189d0898d0e87007d97102619629a4de6d-sqlite-3.7.17-8.el7-x86_64
/var/lib/yum/yumdb/s/e942412b453fd568720258d674b18e7fcd06a20f-sqlite-devel-3.7.17-8.el7-x86_64
/usr/lib/python2.7/site-packages/backports.ssl_match_hostname-3.7.0.1.dist-info
/usr/lib/python2.7/site-packages/configparser-3.7.3.dist-info
/usr/lib/python2.7/site-packages/flake8-3.7.7.dist-info
/usr/share/doc/sqlite-3.7.17
/usr/share/doc/git-1.8.3.1/RelNotes/1.5.3.7.txt
/usr/bin/easy_install-3.7
/usr/bin/pip3.7
/usr/bin/python3.7m
/usr/bin/python3.7
/usr/bin/python3.7m-config
/usr/bin/pydoc3.7
/usr/bin/idle3.7
/usr/bin/2to3-3.7
/usr/bin/pyvenv-3.7
/usr/lib/pkgconfig/python-3.7.pc
/usr/lib/python3.7
/usr/lib/python3.7/lib2to3/Grammar3.7.4.final.0.pickle
/usr/lib/python3.7/lib2to3/PatternGrammar3.7.4.final.0.pickle
/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu
/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a
/usr/lib/libpython3.7m.a
/usr/share/man/man1/python3.7.1
/usr/include/python3.7m
/usr/local/bin/easy_install-3.7
/usr/src/Python-3.7.4.tgz
/usr/src/Python-3.7.4
/usr/src/Python-3.7.4/Doc/whatsnew/3.7.rst
/usr/src/Python-3.7.4/build/lib.linux-x86_64-3.7
/usr/src/Python-3.7.4/build/temp.linux-x86_64-3.7
/usr/src/Python-3.7.4/build/temp.linux-x86_64-3.7/usr/src/Python-3.7.4
/usr/src/Python-3.7.4/build/scripts-3.7
/usr/src/Python-3.7.4/build/scripts-3.7/pydoc3.7
/usr/src/Python-3.7.4/build/scripts-3.7/idle3.7
/usr/src/Python-3.7.4/build/scripts-3.7/2to3-3.7
/usr/src/Python-3.7.4/build/scripts-3.7/pyvenv-3.7
/usr/src/Python-3.7.4/libpython3.7m.a
I then did the following steps:
tar -zxvf Python-3.7.4.tgz
cd Python-3.7.4
./configure --prefix=/usr --enable-optimizations
make
make altinstall
[root#TACIT Python-3.7.4]# python3.7
Python 3.7.4 (default, Aug 17 2019, 16:34:42)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.7.17'
Not sure what else I can do here.

I think that OP must fix the path to sqlite3 while installing Python:
sudo LD_RUN_PATH=/usr/lib ./configure --enable-optimizations
sudo LD_RUN_PATH=/usr/lib make altinstall

UPDATE 2020-10-03:
I've compiled a version of _sqlite.so which you can drop at the root of your Python 3.8 project:
https://github.com/FlipperPA/django-s3-sqlite/raw/master/shared-objects/python-3-8/_sqlite3.so
If you drop this at the root of a Python or Django project, it should be picked up when Python's SQLite 3 wrapped attempts to _import sqlite, overriding this module: https://github.com/python/cpython/tree/b6f5b9dd21a37fa97af38521941025259f1556e0/Modules/_sqlite
Good luck!
ORIGINAL MESSAGE:
This is an issue with the included version of SQLite on CentOS 7 being quite old. Here's how I solved it, IIRC:
Upgrade the CentOS 7 system install of SQLite to 3.29 by compiling it from source. Download the source, and then be sure to ./configure --prefix=/usr to upgrade the system version. Then make and sudo make install. You must include the prefix or it'll install to /usr/local instead by default!
Make sure the system version installed is 3.29 with sqlite3 --version
Re-compile Python 3.7.x. Be sure to do a make clean before sudo make install if you've already installed it.
Open python3.7 and check the version. import sqlite3, followed by sqlite3.sqlite_version
That should work; the only difference is I installed from Python 3.6 from IUS Community. After upgrading the system SQLite, I did a sudo yum remove python36u then a sudo yum install python36u and the Python included SQLite had successful upgraded to 3.29. Good luck!

change
def check_sqlite_version():
if Database.sqlite_version_info < (3, 8, 3):
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
to:
def check_sqlite_version():
if Database.sqlite_version_info < (3, 7, 17):
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)

Django - ImportError: No module named djangochat.settings

Here is my error trace of python manage.py runserver I am using django 1.10.7 with python 2.7
(faavenv) user#ubutnu:~/Documents/faa-backend$ python manage.py runserver
Traceback (most recent call last):
File "manage.py", line 14, in <module>
execute_from_command_line(sys.argv)
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 316, in execute
settings.INSTALLED_APPS
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 97, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named djangochat.settings
Here is my requirements.txt
amqp==2.2.1
anyjson==0.3.3
asn1crypto==0.22.0
Babel==2.4.0
backports-abc==0.5
bcrypt==3.1.3
BeautifulSoup==3.2.1
beautifulsoup4==4.6.0
billiard==3.5.0.3
bleach==2.0.0
boto==2.48.0
boto3==1.4.4
botocore==1.5.80
celery==4.1.0
certifi==2017.4.17
cffi==1.10.0
chardet==3.0.4
cryptography==1.9
Django==1.10.7
django-celery-beat==1.0.1
django-contrib-comments==1.8.0
django-debug-toolbar==1.8
django-environ==0.4.3
django-nocaptcha-recaptcha==0.0.19
django-storages-redux==1.3.3
docutils==0.13.1
enum34==1.1.6
Fabric==1.13.2
feedparser==5.2.1
filebrowser-safe==0.4.7
flower==0.9.2
future==0.16.0
futures==3.1.1
grappelli-safe==0.4.6
html5lib==0.999999999
idna==2.5
ipaddress==1.0.18
jmespath==0.9.3
kombu==4.1.0
Mezzanine==4.2.3
nltk==3.2.4
numpy==1.13.0
oauthlib==2.0.2
olefile==0.44
pandas==0.20.2
paramiko==2.2.1
Pillow==4.1.1
psycopg2==2.7.1
pyasn1==0.2.3
pycparser==2.17
pycryptodomex==3.4.6
pyjwkest==1.3.6
PyJWT==1.5.2
PyNaCl==1.1.2
python-dateutil==2.6.0
python-openid==2.2.5
pytz==2017.2
requests==2.18.1
requests-oauthlib==0.8.0
s3transfer==0.1.10
scikit-learn==0.18.2
scipy==0.19.1
singledispatch==3.4.0.3
six==1.10.0
sklearn==0.0
social-auth-app-django==1.2.0
social-auth-core==1.4.0
sqlparse==0.2.3
tornado==4.5.1
tzlocal==1.4
urllib3==1.21.1
vine==1.1.4
webencodings==0.5.1
Problem is that I haven't install Django chat in my environment. Still, it is showing above error. Not only this project but in all project, I am receiving this error.
I tried to reinstall all library again but doesn't help.
Any suggestion would be appreciated.
(faavenv) user#ubuntu:~/Documents/faa-backend$ python manage.py shell
Traceback (most recent call last):
File "manage.py", line 14, in <module>
execute_from_command_line(sys.argv)
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 316, in execute
settings.INSTALLED_APPS
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/home/user/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/home/soham/Documents/faa-backend/faavenv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 97, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named djangochat.settings
(faavenv) user#ubuntu:~/Documents/faa-backend$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import djangoChat
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named djangoChat
>>>

This error usually happens because you have a file that uses django and django is not imported.
You probably have a script somewhere in your project that is imported and has a executable function without the if __name__ == "__main__": part. So runserver imports this script, try to execute and fail, because django is not imported. So, if you have something like this, be sure that the running function is inside the above if statement, so runserver does not try to run the script when it is imported.
So, if you want to run a script that uses django (and it is not inside the models, views, forms usual stuff), you need to setup django first. You need to find the problematic file and add the following at the top:
import django
sys.path.append("path/to/django/settingsapp")
os.environ["DJANGO_SETTINGS_MODULE"] = "djangochat.settings"
django.setup()
If you find the problematic file and add the above, it will probably fix it.
Hope this helps.

Thank you for all answers, I just moved my project location and set it up again and It works. still, I don't know what was the issue but after moving project directory and environment It is working.

tarfile compressionerror bz2 module is not available

I'm trying to install twisted
pip install https://pypi.python.org/packages/18/85/eb7af503356e933061bf1220033c3a85bad0dbc5035dfd9a97f1e900dfcb/Twisted-16.2.0.tar.bz2#md5=8b35a88d5f1a4bfd762a008968fddabf
This is for a django-channels project and I'm having the following error problem
Exception:
Traceback (most recent call last):
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/tarfile.py", line 1655, in bz2open
import bz2
File "/usr/local/lib/python3.5/bz2.py", line 22, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ImportError: No module named '_bz2'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/petarp/.virtualenvs/CloneFromGitHub/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/commands/install.py", line 310, in run
wb.build(autobuilding=True)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/wheel.py", line 750, in build
self.requirement_set.prepare_files(self.finder)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/req/req_set.py", line 370, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/req/req_set.py", line 587, in _prepare_file
session=self.session, hashes=hashes)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/download.py", line 810, in unpack_url
hashes=hashes
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/download.py", line 653, in unpack_http_url
unpack_file(from_path, location, content_type, link)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/utils/__init__.py", line 605, in unpack_file
untar_file(filename, location)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/utils/__init__.py", line 538, in untar_file
tar = tarfile.open(filename, mode)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/tarfile.py", line 1580, in open
return func(name, filemode, fileobj, **kwargs)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/tarfile.py", line 1657, in bz2open
raise CompressionError("bz2 module is not available")
tarfile.CompressionError: bz2 module is not available
Clearly I'm missing bz2 module, so I've tried to installed it manually, but that didn't worked out for python 3.5, so how can I solved this?
I've did what #e4c5 suggested but I did it for python3.5.1, the output is
➜ ~ python3.5
Python 3.5.1 (default, Apr 19 2016, 22:45:11)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bz2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/bz2.py", line 22, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ImportError: No module named '_bz2'
>>>
[3] + 18945 suspended python3.5
➜ ~ dpkg -S /usr/local/lib/python3.5/bz2.py
dpkg-query: no path found matching pattern /usr/local/lib/python3.5/bz2.py
I am on Ubuntu 14.04 LTS and I have installed python 3.5 from source.

I don't seem to have any problem with import bz2 on my python 3.4 installation. So I did
import bz2
print (bz2.__file__)
And found that it's located at /usr/lib/python3.4/bz2.py then I did
dpkg -S /usr/lib/python3.4/bz2.py
This reveals:
libpython3.4-stdlib:amd64: /usr/lib/python3.4/bz2.py
Thus the following command should hopefully fix this:
apt-get install libpython3.4-stdlib
Update:
If you have compiled python 3.5 from sources, it's very likely the bz2 hasn't been compiled in. Please reinstall by first doing
./configure --with-libs='bzip'
The same applies for python 3.6 as well. Note that this will probably complain about other missing dependencies. You will have to install the missing dependencies one by one until everything is covered.

I was able to solve it by removing the _ and changing the import to
from bz2 import BZ2Compressor, BZ2Decompressor

On ubuntu, apt-get install libbz2-dev then compile python again.

cloudera 5 python API cannot create cluster

I have installed the new cloudera 5 manager by manual installation
wget http://archive.cloudera.com/cdh5/one-click-install/redhat/6/x86_64/cloudera-cdh-5-0.x86_64.rpm
rpm -ivh cloudera-cdh-5-0.x86_64.rpm
cd /etc/yum.repos.d/
wget http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/cloudera-manager.repo
cd /usr/src
wget http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update45-1.x86_64.rpm
rpm -ivh oracle-j2sdk1.7-1.7.0+update45-1.x86_64.rpm
yum -y install cloudera-manager-server cloudera-manager-server-db-2
/etc/init.d/cloudera-scm-server-db stop
/etc/init.d/cloudera-scm-server-db start
/etc/init.d/cloudera-scm-server stop
/etc/init.d/cloudera-scm-server start
I am using the python API to create the cluster...
http://cloudera.github.io/cm_api/docs/python-client/
however I am getting this error
[kshk#localhost ~]$ python
Python 2.7.5 (default, Nov 12 2013, 16:19:08)
[GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from cm_api.api_client import ApiResource
>>> import socket
>>> cm_host = "192.168.174.190"
>>> api = ApiResource(cm_host, username="admin", password="admin")
>>> cluster = api.create_cluster("clusterD", "CDH5")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/api_client.py", line 108, in create_cluster
return clusters.create_cluster(self, name, version)
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/clusters.py", line 34, in create_cluster
data=[apicluster])[0]
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/types.py", line 141, in call
return ApiList.from_json_dict(ret, method.im_self, ret_type)
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/types.py", line 419, in from_json_dict
items = [ attr.from_json(resource_root, x) for x in dic[ApiList.LIST_KEY] ]
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/types.py", line 97, in from_json
return self._atype.from_json_dict(data, resource_root)
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/types.py", line 286, in from_json_dict
obj._set_attrs(dic, allow_ro=True)
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/types.py", line 221, in _set_attrs
attr = self._check_attr(k, allow_ro)
File "/usr/lib/python2.7/site-packages/cm_api-6.0.1-py2.7.egg/cm_api/endpoints/types.py", line 234, in _check_attr
(name, self.__class__.__name__))
AttributeError: Invalid property fullVersion for class ApiCluster.
any ideas???

there was an issue with the python cloudera API, after four hours of this post; the API was updated and it works fine now...

Lost sqlite3 module from python after OSX Mavericks

After installing OS X Mavericks, I have been having issues running django's server:
(rango_tutorial)christohersmbp2:tango_with_django_project christopherspears$ python manage.py runserver
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/core/management/base.py", line 280, in execute
translation.activate('en-us')
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 130, in activate
return _trans.activate(language)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 188, in activate
_active.value = translation(language)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 177, in translation
default_translation = _fetch(settings.LANGUAGE_CODE)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 159, in _fetch
app = import_module(appname)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
__import__(name)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/contrib/admin/__init__.py", line 6, in <module>
from django.contrib.admin.sites import AdminSite, site
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 4, in <module>
from django.contrib.admin.forms import AdminAuthenticationForm
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/contrib/admin/forms.py", line 6, in <module>
from django.contrib.auth.forms import AuthenticationForm
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/contrib/auth/forms.py", line 17, in <module>
from django.contrib.auth.models import User
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/contrib/auth/models.py", line 48, in <module>
class Permission(models.Model):
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/models/base.py", line 96, in __new__
new_class.add_to_class('_meta', Options(meta, **kwargs))
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/models/base.py", line 264, in add_to_class
value.contribute_to_class(cls, name)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/models/options.py", line 124, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/__init__.py", line 34, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/utils.py", line 198, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/utils.py", line 113, in load_backend
return import_module('%s.base' % backend_name)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
__import__(name)
File "/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 35, in <module>
raise ImproperlyConfigured("Error loading either pysqlite2 or sqlite3 modules (tried in that order): %s" % exc)
django.core.exceptions.ImproperlyConfigured: Error loading either pysqlite2 or sqlite3 modules (tried in that order): dlopen(/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/lib-dynload/_sqlite3.so, 2): Symbol not found: _sqlite3_enable_load_extension
Referenced from: /Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/lib-dynload/_sqlite3.so
Expected in: /usr/lib/libsqlite3.dylib
in /Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/lib-dynload/_sqlite3.so
I think the issue is that the sqlite3 module has gone missing for some reason:
(rango_tutorial)christohersmbp2:tango_with_django_project christopherspears$ python
Python 2.7.5 (default, Dec 31 2013, 11:46:08)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: dlopen(/Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/lib-dynload/_sqlite3.so, 2): Symbol not found: _sqlite3_enable_load_extension
Referenced from: /Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/lib-dynload/_sqlite3.so
Expected in: /usr/lib/libsqlite3.dylib
in /Users/christopherspears/.virtualenvs/rango_tutorial/lib/python2.7/lib-dynload/_sqlite3.so
I'm not sure what is causing this. Isn't the sqlite3 module normally just part of Python?

Updating OS X is notorious for messing up all sorts of Developer Tools.
What is your output when you type which sqlite3? How about other developer tools like which gcc? If they are missing you may need to reinstall Xcode. I know many users had to reinstall XCode after updating to Mavericks.
Here's the link for more information: https://developer.apple.com/xcode/

Uninstalling and reinstalling python with brew seemed to help with the earlier problem. Of course now, python is broken in that virtual environment:
(rango_tutorial)christohersmbp2:tango_with_django_project christopherspears$ python manage.py runserver
dyld: Library not loaded: #executable_path/../.Python
Referenced from: /Users/christopherspears/.virtualenvs/rango_tutorial/bin/python
Reason: image not found
Trace/BPT trap: 5
(rango_tutorial)christohersmbp2:tango_with_django_project christopherspears$ python
dyld: Library not loaded: #executable_path/../.Python
Referenced from: /Users/christopherspears/.virtualenvs/rango_tutorial/bin/python
Reason: image not found
Trace/BPT trap: 5