pandas install issues in gitlab and docker - python

Collecting numpy (from -r requirements.txt (line 21))
Downloading numpy-1.11.1.zip (4.7MB)
Collecting pandas (from -r requirements.txt (line 22))
Downloading pandas-0.18.1.tar.gz (7.3MB)
Complete output from command python setup.py egg_info:
Download error on https://pypi.python.org/simple/numpy/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) -- Some packages may not be found!
Couldn't find index page for 'numpy' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) -- Some packages may not be found!
No local packages or download links found for numpy>=1.7.0
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-8puw9oba/pandas/setup.py", line 631, in <module>
**setuptools_kwargs)
File "/usr/local/lib/python3.5/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/local/lib/python3.5/site-packages/setuptools/dist.py", line 269, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/local/lib/python3.5/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init__.py", line 826, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1092, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1104, in obtain
return installer(requirement)
File "/usr/local/lib/python3.5/site-packages/setuptools/dist.py", line 380, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 634, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('numpy>=1.7.0')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-8puw9oba/pandas/
ERROR: Build failed: exit code 1
Trying continuous Integration with gitlab and am running into an issue after pandas has been added as a requirement. when running the pytest the error above happens. the yaml for the gitlab-ci looks like this:
pytest:
image: python:3-alpine
script:
- pip install -r requirements.txt
- python -m pytest tests --ignore=tests/test_routes.py
eslint:
image: node:4.4.7
cache:
paths:
- src/static/node_modules/
script:
- cd src/static
- npm --loglevel=silent install
- npm --loglevel=silent install gulp -g
- gulp lint
pytest is the one that is failing before it even gets to running the tests
the contents of our requirements.txt are as follows:
astroid==1.4.5
blinker==1.4
click==6.3
colorama==0.3.7
Flask==0.10.1
Flask-DebugToolbar==0.10.0
Flask-Login==0.3.2
Flask-Mail==0.9.1
Flask-Principal==0.4.0
Flask-WTF==0.12
Jinja2==2.8
lazy-object-proxy==1.2.1
MarkupSafe==0.23
passlib==1.6.5
pylint==1.5.5
requests==2.9.1
six==1.10.0
Werkzeug==0.11.4
wrapt==1.10.6
WTForms==2.1
pandas
pyaml
rtyaml
webtest
hypothesis
beautifulsoup4
pytest
I attempted manually adding numpy before pandas but got the same result. since it complained about numpy >=1.7.0 I also attempted explicitly telling it that version but that did not resolve the issue either. Is there anything I am missing in this configuration that would be causing this problem?

pip is unable to verify the certificate. You need to manually say which certificate it should use to verify it.
This should work:
pip --cert /etc/ssl/certs/DigiCert_High_Assurance_EV_Root_CA.pem install -r requirements.txt

Related

setuptools pip wheel failed with error code 2

I am going through the Google Cloud Platform's Python Bookshelf App Tutorial and I am at the step of starting up my virtualenv where I am stuck.
Running virtualenv -p python3 env produces this error:
⇒ virtualenv -p python3 env
Running virtualenv with interpreter /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/puoygae/Lab/gae-bookshelf-app-python-flexible-experiment/getting-started-python/2-structured-data/env/bin/python3
Also creating executable in /Users/puoygae/Lab/gae-bookshelf-app-python-flexible-experiment/getting-started-python/2-structured-data/env/bin/python
Please make sure you remove any previous custom paths from your /Users/puoygae/.pydistutils.cfg file.
Installing setuptools, pip, wheel...
Complete output from command /Users/puoygae/Lab/ga...data/env/bin/python3 - setuptools pip wheel:
Collecting setuptools
Using cached https://files.pythonhosted.org/packages/75/b3/0a106dfaf7f48aef638da80b32608617cc8de4b24a22c8cd3759c32e5d30/setuptools-41.1.0-py2.py3-none-any.whl
Collecting pip
Using cached https://files.pythonhosted.org/packages/8d/07/f7d7ced2f97ca3098c16565efbe6b15fafcba53e8d9bdb431e09140514b0/pip-19.2.2-py2.py3-none-any.whl
Collecting wheel
Using cached https://files.pythonhosted.org/packages/bb/10/44230dd6bf3563b8f227dbf344c908d412ad2ff48066476672f3a72e174e/wheel-0.33.4-py2.py3-none-any.whl
Installing collected packages: setuptools, pip, wheel
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/req/req_set.py", line 731, in install
**kwargs
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/req/req_install.py", line 841, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/req/req_install.py", line 1040, in move_wheel_files
isolated=self.isolated,
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/wheel.py", line 343, in move_wheel_files
clobber(source, lib_dir, True)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/wheel.py", line 321, in clobber
shutil.copyfile(srcfile, destfile)
File "/Users/puoygae/Lab/gae-bookshelf-app-python-flexible-experiment/getting-started-python/2-structured-data/env/lib/python3.6/shutil.py", line 121, in copyfile
with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/lib/python3.6/site-packages/easy_install.py'
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/virtualenv.py", line 2327, in <module>
main()
File "/Library/Python/2.7/site-packages/virtualenv.py", line 711, in main
symlink=options.symlink)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 944, in create_environment
download=download,
File "/Library/Python/2.7/site-packages/virtualenv.py", line 900, in install_wheel
call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 795, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /Users/puoygae/Lab/ga...data/env/bin/python3 - setuptools pip wheel failed with error code 2
I had followed this guide to set up my python development environment. I searched the internet for a solution but I haven't had luck getting pass this step.
What is causing the issue and how can I fix it?
EDIT: Running it with sudo produces the following error:
⇒ sudo virtualenv -p python3 env
Password:
Running virtualenv with interpreter /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/myUserName/Lab/gae-bookshelf-app-python-flexible-experiment/getting-started-python/2-structured-data/env/bin/python3
Not overwriting existing python script /Users/myUserName/Lab/gae-bookshelf-app-python-flexible-experiment/getting-started-python/2-structured-data/env/bin/python (you must use /Users/myUserName/Lab/gae-bookshelf-app-python-flexible-experiment/getting-started-python/2-structured-data/env/bin/python3)
Please make sure you remove any previous custom paths from your /Users/myUserName/.pydistutils.cfg file.
Installing setuptools, pip, wheel...
Complete output from command /Users/myUserName/Lab/ga...data/env/bin/python3 - setuptools pip wheel:
The directory '/Users/myUserName/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/myUserName/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting setuptools
Downloading https://files.pythonhosted.org/packages/75/b3/0a106dfaf7f48aef638da80b32608617cc8de4b24a22c8cd3759c32e5d30/setuptools-41.1.0-py2.py3-none-any.whl (576kB)
Collecting pip
Downloading https://files.pythonhosted.org/packages/8d/07/f7d7ced2f97ca3098c16565efbe6b15fafcba53e8d9bdb431e09140514b0/pip-19.2.2-py2.py3-none-any.whl (1.4MB)
Collecting wheel
Downloading https://files.pythonhosted.org/packages/bb/10/44230dd6bf3563b8f227dbf344c908d412ad2ff48066476672f3a72e174e/wheel-0.33.4-py2.py3-none-any.whl
Installing collected packages: setuptools, pip, wheel
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/req/req_set.py", line 731, in install
**kwargs
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/req/req_install.py", line 841, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/req/req_install.py", line 1040, in move_wheel_files
isolated=self.isolated,
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/wheel.py", line 476, in move_wheel_files
generated.extend(maker.make(spec))
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/_vendor/distlib/scripts.py", line 353, in make
self._make_script(entry, filenames, options=options)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/_vendor/distlib/scripts.py", line 257, in _make_script
self._write_script(scriptnames, shebang, script, filenames, ext)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/_vendor/distlib/scripts.py", line 231, in _write_script
self._fileop.write_binary_file(outname, script_bytes)
File "/Library/Python/2.7/site-packages/virtualenv_support/pip-8.0.2-py2.py3-none-any.whl/pip/_vendor/distlib/util.py", line 388, in write_binary_file
with open(path, 'wb') as f:
PermissionError: [Errno 1] Operation not permitted: '/bin/easy_install'
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/virtualenv.py", line 2327, in <module>
main()
File "/Library/Python/2.7/site-packages/virtualenv.py", line 711, in main
symlink=options.symlink)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 944, in create_environment
download=download,
File "/Library/Python/2.7/site-packages/virtualenv.py", line 900, in install_wheel
call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
File "/Library/Python/2.7/site-packages/virtualenv.py", line 795, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /Users/myUserName/Lab/ga...data/env/bin/python3 - setuptools pip wheel failed with error code 2
#puoygae fixed his issue by creating a new User account on his MacOS and installed python separately. The environment in his Admin account was convoluted.
Other fixes to this known issue can be :
Removing all virtualenv* things in
/usr/local/lib/python2.7/dist-packages/ and reinstalling it as per
this GitHub issue.
Uninstall any setuptools and wheels from pip. Also uninstall them
from the apt-get remove wheel setuptools virutalenv(if they exist,
In some cases wheels and setuptools both can exist and may conflict
with each other from pip and apt-get installations) > Download fresh
setuptools-35.0.2 from here and virutalenv-15.1.0 from here.
Install both the tools freshly.
Follow the instructions on updating openSSL here and then try
setting up a virtual environment using pip install.
Could try to open the system terminal and type pip install
virtualenv. Restart IDE.
Try installing pysocks - sudo pip install pysocks Use - unset
all_proxy Retry
Also you can try upgrading to platformio-ide 2.1.5
"setenv all_proxy https://proxy_here:port_here/". Solves the proxy
problem for not only this setup but for all installations.
Also have a look at this stackoverflow thread fixes for the
same issue.
Reference Links :
https://github.com/platformio/platformio-atom-ide/issues/1245
https://github.com/pypa/virtualenv/issues/876
I had this problem as well.
My pip was looking at a different repo, the one I created using AWS's CodeArtifact, which needs login and credentials.
In my case, all I needed to do was erase this file ~/.config/pip/pip.config

Ignore ssl certs for easy install python

I have my lib which has setup as
setup.py
from setuptools import setup
setup(
setup_requires=['pbr>=0.11.0'],
pbr=True
)
setup.cfg
[metadata]
name = MyLib
[options]
zip_safe = False
include_package_data = True
install_requires =
cython
pyrex
pymssql
elementtree
bleach
BeautifulSoup4
pytz
MySQL_python
retry
elementtree
Pyrex
dependency_links =
https://pypi.mydomain.com/packages/
[files]
packages =
MyLib
MyLib.package1
MyLib.package1.new1
[tool:wheel]
universal = 1
[flake8]
exclude =
venv,
.tox,
.git,
__pycache__,
*.pyc,
*.egg-info,
.cache,
.eggs,
max-line-length = 80
[tool:pytest]
testpaths=MyLib/tests
ignore=
.tox
.cache
docs
config.py
venv
[coverage:run]
omit=
MyLib/config.py
I am using this MyLib in my application requirements.txt
myapp/requirements.txt
...
...
MyLib==1.2.3
...
...
When I installed, with /usr/local/pyenv/versions/myapp/bin/pip install -i https://pypi.mydomain.com --trusted-host pypi.mydomain.com -r /opt/school/apps/myapp/requirements.txt, it was redirecting to pypi.python.org. Then I get that easy_install is using another config as ~/.pydistutils.cfg. I added as
~/.pydistutils.cfg
[easy_install]
index_url=https://pypi.mydomain.com/
find_links = https://pypi.mydomain.com/simple/
allow_hosts = pypi.mydomain.com
Now its not redirecting to pypi.python.org but it gives error for SSL.
/usr/local/pyenv/versions/myapp/bin/pip install -i https://pypi.mydomain.com --trusted-host pypi.mydomain.com -r /opt/school/apps/myapp/requirements.txt
Requirement already satisfied: appdirs==1.4.3 in /usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages (from -r /opt/school/apps/myapp/requirements.txt (line 1))
Requirement already satisfied: beautifulsoup4 in /usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages (from -r /opt/school/apps/myapp/requirements.txt (line 2))
Requirement already satisfied: bleach==2.0.0 in /usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages (from -r /opt/school/apps/myapp/requirements.txt (line 3))
Requirement already satisfied: click==6.7 in /usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages (from -r /opt/school/apps/myapp/requirements.txt (line 4))
Collecting MyLib==1.2.3 (from -r /opt/school/apps/myapp/requirements.txt (line 5))
Downloading https://pypi.mydomain.com/packages/MyLib-1.2.3.tar.gz (221kB)
100% |████████████████████████████████| 225kB 31.8MB/s
Complete output from command python setup.py egg_info:
Download error on https://pypi.mydomain.com/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661) -- Some packages may not be found!
Download error on https://pypi.mydomain.com/pbr/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661) -- Some packages may not be found!
Couldn't find index page for 'pbr' (maybe misspelled?)
Download error on https://pypi.mydomain.com/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661) -- Some packages may not be found!
No local packages or working download links found for pbr>=0.11.0
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-Lsc4UJ/MyLib/setup.py", line 5, in <module>
pbr=True
File "/usr/local/pyenv/versions/2.7.13/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/setuptools/dist.py", line 315, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/pkg_resources/__init__.py", line 850, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1122, in best_match
return self.obtain(req, installer)
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1134, in obtain
return installer(requirement)
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/setuptools/dist.py", line 429, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/pyenv/versions/2.7.13/envs/myapp/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 659, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pbr>=0.11.0')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Lsc4UJ/MyLib/
Is there any way to force dependency_links not to use ~/.pydistutils.cfg ?
If answer of 1 is NO, then how to ignore SSL for my pypi server?
Tried the solution given below.
# /usr/local/pyenv/versions/myapp/bin/python get-pip.py
Collecting pip
Using cached https://files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 9.0.1
Uninstalling pip-9.0.1:
Successfully uninstalled pip-9.0.1
Successfully installed pip-18.0
/usr/local/pyenv/versions/myapp/bin/pip install -i https://pypi.mydomain.com --trusted-host pypi.mydomain.com -r /opt/school/apps/myapp/requirements.txt
Looking in indexes: https://pypi.mydomain.com
Collecting MyLib==1.2.3 (from -r /opt/school/apps/myapp/requirements.txt (line 1))
Downloading https://pypi.mydomain.com/packages/MyLib-1.2.3.tar.gz (221kB)
100% |████████████████████████████████| 225kB 21.2MB/s
Complete output from command python setup.py egg_info:
Download error on https://pypi.mydomain.com/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661) -- Some packages may not be found!
Download error on https://pypi.mydomain.com/pbr/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661) -- Some packages may not be found!
Couldn't find index page for 'pbr' (maybe misspelled?)
Download error on https://pypi.mydomain.com/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661) -- Some packages may not be found!
No local packages or working download links found for pbr>=0.11.0
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-JD4AuE/MyLib/setup.py", line 5, in <module>
pbr=True
File "/usr/local/pyenv/versions/2.7.13/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/setuptools/dist.py", line 315, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/pkg_resources/__init__.py", line 850, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1122, in best_match
return self.obtain(req, installer)
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1134, in obtain
return installer(requirement)
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/setuptools/dist.py", line 429, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/pyenv/versions/cvcollect/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 659, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pbr>=0.11.0')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-JD4AuE/MyLib/
The issue, as mentioned here is,
setuptools is trying to talk to PyPI, not pip,
and that 3.6.0 on macOS does not have access to a CA Bundle by default
and setuptools doesn't bundle one like pip does. You'll need to
install something like certifi or raise an issue with setuptools.
You can force install new version of pip,
curl 'https://bootstrap.pypa.io/get-pip.py' > get-pip.py
sudo python get-pip.py
You can also try installing certifi as mentioned in the above issue,
pip install certifi
Certifi is a carefully curated collection of Root Certificates for
validating the trustworthiness of SSL certificates while verifying the
identity of TLS hosts. It has been extracted from the Requests
project.
TLS can be disabled using --trusted-host, but it is dangerous and not recommend at all.
pip install --trusted-host pypi.python.org pip --upgrade

Can't install package pysftp, No package 'libffi' found, failed with error code 1

I am on a clean raspian stretch installation on a pi zero w and trying to install pysftp
pip install pysftp
gives the following error:
Collecting pysftp
Using cached https://files.pythonhosted.org/packages/36/60/45f30390a38b1f92e0a8cf4de178cd7c2bc3f874c85430e40ccf99df8fe7/pysftp-0.2.9.tar.gz
Collecting paramiko>=1.17 (from pysftp)
Using cached https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl
Collecting pynacl>=1.0.1 (from paramiko>=1.17->pysftp)
Using cached https://files.pythonhosted.org/packages/08/19/cf56e60efd122fa6d2228118a9b345455b13ffe16a14be81d025b03b261f/PyNaCl-1.2.1.tar.gz
Complete output from command python setup.py egg_info:
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^
compilation terminated.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-63NrlL/pynacl/setup.py", line 251, in <module>
"Programming Language :: Python :: 3.6",
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/__init__.py", line 128, in setup
_install_setup_requires(attrs)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/__init__.py", line 123, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/home/pi/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 770, in resolve
replace_conflicting=replace_conflicting
File "/home/pi/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1053, in best_match
return self.obtain(req, installer)
File "/home/pi/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in obtain
return installer(requirement)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 699, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 884, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1152, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/home/pi/.local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1140, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-63NrlL/pynacl/
I tried to upgrade the setuptools and install ez_setup
pip install --upgrade setuptools
pip install ez_setup
This doesn't changed anything, also tried to install the missing package with
pip install libffi
pip install cffi
But it all failed. I tried everything I found on the internet, but nothing works. Wasted hours on this... What the heck am I missing?
EDIT:
Also tried:
pip install libffi-dev
But this results in:
Could not find a version that satisfies the requirement libffi-dev (from versions: )
No matching distribution found for libffi-dev
Tried:
sudo apt-get libffi-dev
but it results in:
Package libffi-dev is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'libffi-dev' has no installation candidate
It happened to me as well. I finally found an answer that worked. I simply added the cffi module before adding pysftp.
pip install cffi
and then
pip install pysftp.

pip install census for census.gov API

I am very new to python, and I am trying to speak with the census.gov API to find the population of several zipcodes. I was able to get my API key, and I am using this python wrapper: https://github.com/datamade/census. I was able to install pip using the sudo easy_install pip command. Here's an example of the output when I run it again:
Best match: pip 9.0.1
Processing pip-9.0.1-py2.7.egg
pip 9.0.1 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin
Using /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip
I am trying to use the commands:
pip install census
pip install us
pip install us seems to have worked, but I am getting a Traceback when trying pip install census:
Using cached census-0.8.3-py2.py3-none-any.whl
Collecting requests>=1.1.0 (from census)
Using cached requests-2.18.1-py2.py3-none-any.whl
Collecting urllib3<1.22,>=1.21.1 (from requests>=1.1.0->census)
Using cached urllib3-1.21.1-py2.py3-none-any.whl
Collecting idna<2.6,>=2.5 (from requests>=1.1.0->census)
Using cached idna-2.5-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests>=1.1.0->census)
Using cached certifi-2017.4.17-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests>=1.1.0->census)
Using cached chardet-3.0.4-py2.py3-none-any.whl
Installing collected packages: urllib3, idna, certifi, chardet, requests, census
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install
**kwargs
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 316, in clobber
ensure_dir(destdir)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 83, in ensure_dir
os.makedirs(path)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/urllib3'
I am not sure what to do. Can anyone help? I am using python 2.7.10 by the way.
Your user doesn't have write permissions on /System/Library. Either run pip install from the root user or use pip install -u to install the library to your own directory instead of the system one.

python setup.py egg_info gcc error installing cassandra python

For some reason, the specific version that I need to install manually cannot be accessed directly:
rpm -i ftp://195.220.108.108/linux/centos/7.1.1503/updates/x86_64/Packages/python-2.7.5-18.el7_1.1.x86_64.rpm
So tried a brand new installation of python, then tried updating and installing setup tools
yum -y update
yum -y install python-pip
pip install -U pip
pip install -U setuptools
pip install cassandra-driver
yum install patch
then tried
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
python ez_setup.py
pip install cassandra-driver
Full stacktrace
[root#d8174406546b cassandradump]# pip install cassandra-driver
Collecting cassandra-driver
/usr/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject
Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validat
ion failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. Thi
s prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/e
n/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached cassandra-driver-3.0.0.tar.gz
Complete output from command python setup.py egg_info:
warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
warning: no files found matching '*.h' under directory 'Cython/Debugger/Tests'
warning: no files found matching '*.pxd' under directory 'Cython/Utility'
unable to execute gcc: No such file or directory
Unable to find pgen, not compiling formal grammar.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-cP9FYQ/cassandra-driver/setup.py", line 375, in <module>
run_setup(None)
File "/tmp/pip-build-cP9FYQ/cassandra-driver/setup.py", line 373, in run_setup
**kw)
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 269, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 827, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1072, in best_match
return self.obtain(req, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1084, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 380, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 640, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 670, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 853, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1081, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1069, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-cP9FYQ/cassandra-driver
Typed enter too early, missed gcc library and got carried away by similar posts.
/tmp/easy_install-N19BCp/Cython-0.23.4/Cython/Plex/Scanners.c:4:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
Unable to find pgen, not compiling formal grammar.
Traceback (most recent call last):
Then python-devel solved the problem.
install yum install python-devel
I am working inside docker so prefixed the 'RUN'. On real systems, you will need to use sudo in place of RUN. Sometimes, the pip is tool old. In my case it was 7.x so I also upgraded it. cython depends on python-devel
RUN pip install --upgrade pip
RUN yum -y install python-devel
RUN pip install cython

Categories

Resources