Develop Reference

tarfile compressionerror bz2 module is not available

I'm trying to install twisted
pip install https://pypi.python.org/packages/18/85/eb7af503356e933061bf1220033c3a85bad0dbc5035dfd9a97f1e900dfcb/Twisted-16.2.0.tar.bz2#md5=8b35a88d5f1a4bfd762a008968fddabf
This is for a django-channels project and I'm having the following error problem
Exception:
Traceback (most recent call last):
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/tarfile.py", line 1655, in bz2open
import bz2
File "/usr/local/lib/python3.5/bz2.py", line 22, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ImportError: No module named '_bz2'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/petarp/.virtualenvs/CloneFromGitHub/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/commands/install.py", line 310, in run
wb.build(autobuilding=True)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/wheel.py", line 750, in build
self.requirement_set.prepare_files(self.finder)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/req/req_set.py", line 370, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/req/req_set.py", line 587, in _prepare_file
session=self.session, hashes=hashes)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/download.py", line 810, in unpack_url
hashes=hashes
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/download.py", line 653, in unpack_http_url
unpack_file(from_path, location, content_type, link)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/utils/__init__.py", line 605, in unpack_file
untar_file(filename, location)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/site-packages/pip/utils/__init__.py", line 538, in untar_file
tar = tarfile.open(filename, mode)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/tarfile.py", line 1580, in open
return func(name, filemode, fileobj, **kwargs)
File "/home/petarp/.virtualenvs/ErasmusCloneFromGitHub/lib/python3.5/tarfile.py", line 1657, in bz2open
raise CompressionError("bz2 module is not available")
tarfile.CompressionError: bz2 module is not available
Clearly I'm missing bz2 module, so I've tried to installed it manually, but that didn't worked out for python 3.5, so how can I solved this?
I've did what #e4c5 suggested but I did it for python3.5.1, the output is
➜ ~ python3.5
Python 3.5.1 (default, Apr 19 2016, 22:45:11)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bz2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/bz2.py", line 22, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ImportError: No module named '_bz2'
>>>
[3] + 18945 suspended python3.5
➜ ~ dpkg -S /usr/local/lib/python3.5/bz2.py
dpkg-query: no path found matching pattern /usr/local/lib/python3.5/bz2.py
I am on Ubuntu 14.04 LTS and I have installed python 3.5 from source.

I don't seem to have any problem with import bz2 on my python 3.4 installation. So I did
import bz2
print (bz2.__file__)
And found that it's located at /usr/lib/python3.4/bz2.py then I did
dpkg -S /usr/lib/python3.4/bz2.py
This reveals:
libpython3.4-stdlib:amd64: /usr/lib/python3.4/bz2.py
Thus the following command should hopefully fix this:
apt-get install libpython3.4-stdlib
Update:
If you have compiled python 3.5 from sources, it's very likely the bz2 hasn't been compiled in. Please reinstall by first doing
./configure --with-libs='bzip'
The same applies for python 3.6 as well. Note that this will probably complain about other missing dependencies. You will have to install the missing dependencies one by one until everything is covered.

I was able to solve it by removing the _ and changing the import to
from bz2 import BZ2Compressor, BZ2Decompressor

On ubuntu, apt-get install libbz2-dev then compile python again.

Cannot use python-nmap

I have multiple times installed python-nmap following the instructions on their site, but it just doesn't work. Every time I try to test it by doing:
>>> import nmap
>>> nm = nmap.PortScanner()
I get the following error:
Python 2.7.4 (default, Apr 19 2013, 18:28:01)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nmap
>>> nm = nmap.PortScanner()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "nmap/nmap.py", line 118, in __init__
p = subprocess.Popen(['nmap', '-V'], bufsize=10000, stdout=subprocess.PIPE)
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
All help is greatly appreciated. Thanks in advance.

This error is fixed if you install a later version of nmap for python 2.*
Follow the link here: http://nmap.org/book/inst-macosx.html

You need to install nmap by the looks of it

AttributeError: 'XPathExpr' object has no attribute 'add_post_condition'

I'm trying to install pyquery on Windows and I get the following error when I try to do selects like this d('p:first'). Everything else seems to be working. Any idea what am I missing? This issue happens only on my windows machine, on my MAC works fine.
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyquery import PyQuery as pq
>>> d = pq("<p></p>")
>>> d('p:first')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\pyquery-1.2.4-py2.7.egg\pyquery\pyquery.py
", line 237, in __call__
result = self.__class__(*args, parent=self, **kwargs)
File "C:\Python27\lib\site-packages\pyquery-1.2.4-py2.7.egg\pyquery\pyquery.py
", line 213, in __init__
xpath = self._css_to_xpath(selector)
File "C:\Python27\lib\site-packages\pyquery-1.2.4-py2.7.egg\pyquery\pyquery.py
", line 223, in _css_to_xpath
return self._translator.css_to_xpath(selector, prefix)
File "build\bdist.win32\egg\cssselect\xpath.py", line 188, in css_to_xpath
File "build\bdist.win32\egg\cssselect\xpath.py", line 188, in <genexpr>
File "build\bdist.win32\egg\cssselect\xpath.py", line 208, in selector_to_xpat
h
File "build\bdist.win32\egg\cssselect\xpath.py", line 230, in xpath
File "build\bdist.win32\egg\cssselect\xpath.py", line 272, in xpath_pseudo
File "C:\Python27\lib\site-packages\pyquery-1.2.4-py2.7.egg\pyquery\cssselectp
atch.py", line 19, in xpath_first_pseudo
xpath.add_post_condition('position() = 1')
AttributeError: 'XPathExpr' object has no attribute 'add_post_condition'
>>>

I had installed pyquery with "easy_install pyquery" and this was causing this error.
I removed it and then installed it from https://github.com/gawel/pyquery with python setup.py install and now it's working.

what's wrong, why python manage.py runserver failed?

failed to run django for lack of sqlite? in fact sqlite is already install in my system.
(py1)[mirror#home ci]$ python manage.py runserver
Validating models...
Unhandled exception in thread started by <bound method Command.inner_run of <django.contrib.staticfiles.management.commands.runserver.Command object at 0x9ea3d8c>>
Traceback (most recent call last):
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 91, in inner_run
self.validate(display_num_errors=True)
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/core/management/base.py", line 266, in validate
num_errors = get_validation_errors(s, app)
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/core/management/validation.py", line 23, in get_validation_errors
from django.db import models, connection
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/db/__init__.py", line 40, in <module>
backend = load_backend(connection.settings_dict['ENGINE'])
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/db/__init__.py", line 34, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/db/utils.py", line 92, in __getitem__
backend = load_backend(db['ENGINE'])
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/db/utils.py", line 24, in load_backend
return import_module('.base', backend_name)
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/home/mirror/build/py1/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 31, in <module>
raise ImproperlyConfigured("Error loading either pysqlite2 or sqlite3 modules (tried in that order): %s" % exc)
django.core.exceptions.ImproperlyConfigured: **Error loading either pysqlite2 or sqlite3 modules** (tried in that order): No module named _sqlite3
the focus is that : Error loading either pysqlite2 or sqlite3 modules
evidence to show that sqlite is already installed:
(py1)[mirror#home ci]$ rpm -qa | grep sqlite
python-sqlite-1.1.7-1.2.1
sqlite-devel-3.3.6-5
sqlite-3.3.6-5
(py1)[mirror#home ci]$ python -V
Python 2.7.3
(py1)[mirror#home ci]$ python manage.py runserver
so, why python manage.py runserver failed?
Edit 1:
now i know that i haven't build sqlite modue:
(py1)[mirror#home Python-2.7.3]$ python
Python 2.7.3 (default, Sep 23 2012, 21:42:51)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/mirror/insp/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/home/mirror/insp/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named _sqlite3
i find sqlite code is included by python source code, but how to build it?
[mirror#home Python-2.7.3]$ ls Modules/_s
_sqlite/ _sre.c _sre.o _ssl.c _struct.c
(py1)[mirror#home Python-2.7.3]$ ls Modules/_sqlite/
cache.c connection.h microprotocols.c module.h row.c statement.c util.h
cache.h cursor.c microprotocols.h prepare_protocol.c row.h statement.h
connection.c cursor.h module.c prepare_protocol.h sqlitecompat.h util.c
./configure --help | grep sqlite returns nothing....
(py1)[mirror#home Python-2.7.3]$ ./configure --help | grep sqlite
(py1)[mirror#home Python-2.7.3]$

You need to apt-get install libsqlite3-dev (on debian - sqlite-devel possibly elsewhere) and recompile python.

PyDev and Django: PyDev breaking Django shell?

I've set up a new project, and populated it with simple models. (Essentially I'm following the tut.)
When I run python manage.py shell on the command line, it works fine:
>python manage.py shell
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from mysite.myapp.models import School
>>> School.objects.all()
[]
Works great. Then, I try to do the same thing in Eclipse (using a Django project that is composed of the same files.)
Right click on mysite project >>
Django >> Shell with Django
environment
This is the output from the PyDev Console:
>>> import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))
C:\Python26\python.exe 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
>>>
>>> from django.core import management;import mysite.settings as settings;management.setup_environ(settings)
'path\\to\\mysite'
>>> from mysite.myapp.models import School
>>> School.objects.all()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 68, in __repr__
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 83, in __len__
self._result_cache.extend(list(self._iter))
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 238, in iterator
for row in self.query.results_iter():
File "C:\Python26\lib\site-packages\django\db\models\sql\query.py", line 287, in results_iter
for rows in self.execute_sql(MULTI):
File "C:\Python26\lib\site-packages\django\db\models\sql\query.py", line 2368, in execute_sql
cursor = self.connection.cursor()
File "C:\Python26\lib\site-packages\django\db\backends\__init__.py", line 81, in cursor
cursor = self._cursor()
File "C:\Python26\lib\site-packages\django\db\backends\sqlite3\base.py", line 170, in _cursor
self.connection = Database.connect(**kwargs)
OperationalError: unable to open database file
What am I doing wrong here?

The error is about not being open the database file. So I'd guess that the path to the database in your settings.py is a relative path, and PyDev starts the shell with a different current directory than you normally use.
If this is the case, change the DATABASE_NAME setting to an absolute path and it should work.