Develop Reference

python3 venv vs ansible 2.9.x installation fails

I'm trying to update our venv of ansible to a newer version on an older RHEL 7 box :), only it seems to fail for some unclear reasons in some python code.
Running python 3.6.8 on RHEL 7.9 (Maipo)
Any hints are most appreciated, TIA!
$ source a2.9.27/bin/activate
$ type which python
which is aliased to `alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'
python is /opt/pjp_ansible/virtual_env/a2.9.27/bin/python
$ which pip
/opt/pjp_ansible/virtual_env/a2.9.27/bin/pip
$ pip install ansible==2.9.27
Collecting ansible==2.9.27
Using cached ansible-2.9.27.tar.gz (14.8 MB)
ERROR: Exception:
Traceback (most recent call last):
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
status = run_func(*args)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/commands/install.py", line 339, in run
reqs, check_supported_wheels=not options.target_dir
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 93, in resolve
collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 482, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 349, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
return bool(self._sequence)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
return any(self)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
version=version,
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 287, in __init__
version=version,
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 528, in _prepare_linked_requirement
link, req.source_dir, self._download, self.download_dir, hashes
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/operations/prepare.py", line 223, in unpack_url
unpack_file(file.path, location, file.content_type)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 247, in unpack_file
untar_file(filename, location)
File "/opt/pjp_ansible/virtual_env/a2.9.27/lib64/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 218, in untar_file
with open(path, "wb") as destfp:
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 112: ordinal not in range(128)

found this: realpython.com/intro-to-pyenv which enabled me to create a newer python env and apply in multiple virtual env each with their version of ansible compiled from sources.

python2.7 install python daemon package using pip

When I try to install python daemon package I get this error all the time (installation tried by using pip command & downloading the archive and using pip to install):
pip install python-daemon-2.1.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/7p/x12ctdl94d59qnm_chhj2y240000gn/T/pip-2bGssA-build/setup.py", line 86, in <module>
"Topic :: Software Development :: Libraries :: Python Modules",
File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "version.py", line 640, in __init__
super(ChangelogAwareDistribution, self).__init__(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 317, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 372, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 851, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1123, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1135, in obtain
return installer(requirement)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 438, in fetch_build_egg
cmd.ensure_finalized()
File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 251, in finalize_options
'dist_version': self.distribution.get_version(),
File "version.py", line 656, in get_version
version_info = self.get_version_info()
File "version.py", line 651, in get_version_info
changelog_path = get_changelog_path(self)
File "version.py", line 552, in get_changelog_path
setup_dirname = os.path.dirname(distribution.script_name)
File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 122, in dirname
i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/7p/x12ctdl94d59qnm_chhj2y240000gn/T/pip-2bGssA-build/
I'm new to python, searching for the error code and using the steps provided by other threads doesn't solved the problem.

Just write pip install python-daemon==2.1.2
No need to download python-daemon-2.1.2.tar.gz file while installing from PyPI using pip.
If you want to install using downloaded package follow these steps
tar xvf python-daemon-2.1.2.tar.gz
cd python-daemon-2.1.2
python setup.py install

Error when creating Database Tables - CKAN

i'm trying to install CKAN 2.5.2 on Centos 6.8
When i run paster db init -c /etc/ckan/default/development.ini
i get error
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
load_entry_point('PasteScript==2.0.2', 'console_scripts', 'paster')()
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 102, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 141, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 236, in run
result = self.command()
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 205, in command
self._load_config(cmd!='upgrade')
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 142, in _load_config
conf = self._get_config()
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 139, in _get_config
return appconfig('config:' + self.filename)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 261, in appconfig
global_conf=global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
global_conf=global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
return loader.get_context(object_type, name, global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
section)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
object_type, name=use, global_conf=global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
global_conf=global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
global_conf=global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
return loader.get_context(object_type, name, global_conf)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
object_type, name=name)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 646, in find_egg_entry_point
possible.append((entry.load(), protocol, entry.name))
File "/usr/lib/ckan/default/lib/python2.6/site-packages/pkg_resources/__init__.py", line 2229, in load
return self.resolve()
File "/usr/lib/ckan/default/lib/python2.6/site-packages/pkg_resources/__init__.py", line 2235, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/lib/ckan/default/src/ckan/ckan/config/middleware.py", line 28, in <module>
from ckan.config.environment import load_environment
File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 18, in <module>
import ckan.lib.helpers as h
File "/usr/lib/ckan/default/src/ckan/ckan/lib/helpers.py", line 30, in <module>
from bleach import clean as clean_html
File "/usr/lib/ckan/default/lib/python2.6/site-packages/bleach/__init__.py", line 8, in <module>
from html5lib.sanitizer import HTMLSanitizer
ImportError: No module named sanitizer
I was following wiki instructions
I'm stuck on this step and don't know how to proceed.
Module html5lib is imported and updated to latest version. Paster script is executed in virtualenv under root account.
Also i'm running all of this in python2.6 since that is default on Centos.
Some additional information
when i run python and import html5lib and then help(html5lib) i get this
>>> import html5lib
>>> help(html5lib)
Help on package html5lib:
NAME
html5lib
FILE
/usr/lib/python2.6/site-packages/html5lib/__init__.py
DESCRIPTION
HTML parsing library based on the WHATWG "HTML5"
specification. The parser is designed to be compatible with existing
HTML found in the wild and implements well-defined error recovery that
is largely compatible with modern desktop web browsers.
Example usage:
import html5lib
f = open("my_document.html")
tree = html5lib.parse(f)
PACKAGE CONTENTS
_ihatexml
_inputstream
_tokenizer
_trie (package)
_utils
constants
filters (package)
html5parser
serializer
treeadapters (package)
treebuilders (package)
treewalkers (package)
CLASSES
__builtin__.object
html5lib.html5parser.HTMLParser
class HTMLParser(__builtin__.object)
| HTML parser. Generates a tree structure from a stream of (possibly
| malformed) HTML
|
| Methods defined here:
:
There is no sanitizer here. Do i need to use specific version of html5lib?
Can anyone help?

It looks like you have 2 copies of html5lib installed. When you did help(html5lib) it is showing a copy that is installed in your user's python directory (/usr/lib/python2.6/site-packages/) not the virtualenv that you have ckan (and bleach) installed (/usr/lib/ckan/default/lib/python2.6/site-packages/). So get rid of the former to avoid confusion.
Yes I think you have the wrong version of html5lib, since sanitizer is listed in the package contents when I do help.
This is the correct version (at the time of writing - in future check what is in requirements.txt):
$ pip freeze | grep html5lib
html5lib==0.9999999

Python -pip install Exception for any Libary\Package i'm trying to install

Im using Python 2.7 on windows 8.1, using Visual Studio as f.w
Anyway,
After installing python and some packages couple of days ago, I'm unable anymore to pip install packages (Any of them -Just name it) and keep getting the exception below, which contains tens of error
P.S :
No changes were done in my environment.
I have tried to upgrade pip as well and got same error
Thanks for the help
The same error i'm getting all time:
----- Installing 'webdriver' -----
Collecting webdriver
Exception:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\pip\basecommand.py", line 211, in main
status = self.run(options, args)
File "C:\Python27\lib\site-packages\pip\commands\install.py", line 294, in run
requirement_set.prepare_files(finder)
File "C:\Python27\lib\site-packages\pip\req\req_set.py", line 334, in
prepare_files
functools.partial(self._prepare_file, finder))
File "C:\Python27\lib\site-packages\pip\req\req_set.py", line 321, in
_walk_req_to_install
more_reqs = handler(req_to_install)
File "C:\Python27\lib\site-packages\pip\req\req_set.py", line 461, in
_prepare_file
req_to_install.populate_link(finder, self.upgrade)
File "C:\Python27\lib\site-packages\pip\req\req_install.py", line 250, in
populate_link
self.link = finder.find_requirement(self, upgrade)
File "C:\Python27\lib\site-packages\pip\index.py", line 486, in
find_requirement
all_versions = self._find_all_versions(req.name)
File "C:\Python27\lib\site-packages\pip\index.py", line 404, in
_find_all_versions
index_locations = self._get_index_urls_locations(project_name)
File "C:\Python27\lib\site-packages\pip\index.py", line 378, in
_get_index_urls_locations
page = self._get_page(main_index_url)
File "C:\Python27\lib\site-packages\pip\index.py", line 818, in _get_page
return HTMLPage.get_page(link, session=self.session)
File "C:\Python27\lib\site-packages\pip\index.py", line 949, in get_page
trusted=link.trusted,
File "C:\Python27\lib\site-packages\pip\index.py", line 866, in __init__
namespaceHTMLElements=False,
File "C:\Python27\lib\site-packages\pip\_vendor\html5lib\html5parser.py",
line 27, in parse
tb = treebuilders.getTreeBuilder(treebuilder)
File "C:\Python27\lib\site-
packages\pip\_vendor\html5lib\treebuilders\__init__.py", line 73, in
getTreeBuilder
return etree.getETreeModule(implementation, **kwargs).TreeBuilder
File "C:\Python27\lib\site-packages\pip\_vendor\html5lib\utils.py", line 98,
in moduleFactory
objs = factory(baseModule, *args, **kwargs)
File "C:\Python27\lib\site-
packages\pip\_vendor\html5lib\treebuilders\etree.py", line 17, in
getETreeBuilder
ElementTreeCommentType = ElementTree.Comment("asd").tag
AttributeError: 'module' object has no attribute 'Comment'
You are using pip version 7.1.2, however version 8.0.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip'
command.

syntax error on production server where running bin/python setup.py test-q

When I attempt to do the following on a production server:
$ sudo -u vretnet ../bin/python setup.py test -q
It will show the following error, how should I fix this?
Or could it be that my development.ini went wrong?
Traceback (most recent call last):
File "setup.py", line 34, in <module>
paster_plugins=['pyramid'],
File "/opt/python3.2.3/lib/python3.2/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/python3.2.3/lib/python3.2/distutils/dist.py", line 917, in run_commands
self.run_command(cmd)
File "/opt/python3.2.3/lib/python3.2/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/home/vretnet/env/lib/python3.2/site-packages/distribute-0.6.34-py3.2.egg/setuptools/command/test.py", line 138, in run self.with_project_on_sys_path(self.run_tests)
File "/home/vretnet/env/lib/python3.2/site-packages/distribute-0.6.34-py3.2.egg/setuptools/command/test.py", line 118, in with_project_on_sys_path
func()
File "/home/vretnet/env/lib/python3.2/site-packages/distribute-0.6.34-py3.2.egg/setuptools/command/test.py", line 164, in run_tests
testLoader = cks
File "/opt/python3.2.3/lib/python3.2/unittest/main.py", line 123, in __init__
self.parseArgs(argv)
File "/opt/python3.2.3/lib/python3.2/unittest/main.py", line 191, in parseArgs
self.createTests()
File "/opt/python3.2.3/lib/python3.2/unittest/main.py", line 198, in createTests
self.module)
File "/opt/python3.2.3/lib/python3.2/unittest/loader.py", line 132, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/python3.2.3/lib/python3.2/unittest/loader.py", line 132, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/python3.2.3/lib/python3.2/unittest/loader.py", line 91, in loadTestsFromName
module = __import__('.'.join(parts_copy))
File "/home/vretnet/env/ECommerce/ecommerce/__init__.py", line 1, in <module>
from pyramid.config import Configurator
File "/home/vretnet/env/lib/python3.2/site-packages/pyramid-1.4-py3.2.egg/pyramid/config/__init__.py", line 10, in <module>
from webob.exc import WSGIHTTPException as WebobWSGIHTTPException
File "/home/vretnet/env/lib/python3.2/site-packages/WebOb-1.2.3-py3.2.egg/webob/exc.py", line 1115, in <module>
from paste import httpexceptions
File "/home/vretnet/env/lib/python3.2/site-packages/Paste-1.7.5.1-py3.2.egg/paste/httpexceptions.py", line 634
except HTTPException, exc:
^
SyntaxError: invalid syntax

It is Python 3, you should write as:
except HTTPException as exc:

This took me a couple of days to figure out ><. But make sure you install the correct mod_wsgi version. You can install that uses python2 or one that uses python3.
If you install the python3 wsgi mod and use virtualenv2 to build your python environment you will get packages that aren't written for python3.

you just use this patch bellow.
$ 2to3 example.py
https://docs.python.org/2.7/library/2to3.html#to3-python-2-3
translate code from python2 to python3 automatically and recursively.