Im trying to setup elasticsearch-curator (version 5.6.0) to delete indices in elasticsearch (version 7.3.1).
Theirs versions should be compatible (https://www.elastic.co/guide/en/elasticsearch/client/curator/current/version-compatibility.html).
Elasticseach is protected by SSL using self-signed certificate, so I need to turn off certificate verification.
This is my curator.yml conf:
client:
hosts:
- 127.0.0.1
port: 9201
url_prefix:
use_ssl: True
certificate: /opt/elastic-stack/curator/security/ca.crt
client_cert:
client_key:
ssl_no_validate: True
http_auth: curator:************
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /var/log/elastic-stack/curator/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']
When I run
curator --config /opt/elastic-stack/curator/curator.yml /opt/elastic-stack/curator/actions.yml
Even though ssl_no_validate is set to True, I am getting:
/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
return yaml.load(read_file(path))
/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py:175: UserWarning: Connecting to 127.0.0.1 using SSL with verify_certs=False is insecure.
% host
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connection.py", line 394, in connect
ssl_context=context,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 383, in ssl_wrap_socket
return context.wrap_socket(sock)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 217, in perform_request
method, url, body, retries=Retry(False), headers=request_headers, **kw
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 376, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connection.py", line 394, in connect
ssl_context=context,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 383, in ssl_wrap_socket
return context.wrap_socket(sock)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 899, in get_client
check_version(client)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 685, in check_version
version_number = get_version(client)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 658, in get_version
version = client.info()['version']['number']
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 245, in info
return self.transport.perform_request("GET", "/", params=params)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/transport.py", line 353, in perform_request
timeout=timeout,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 226, in perform_request
raise SSLError("N/A", str(e), e)
elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/bin//curator", line 11, in <module>
sys.exit(cli())
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/cli.py", line 213, in cli
run(config, action_file, dry_run)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/cli.py", line 160, in run
client = get_client(**client_args)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 906, in get_client
'Error: {0}'.format(e)
elasticsearch.exceptions.ElasticsearchException: Unable to create client connection to Elasticsearch. Error: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777))
I got successful response, when I run
curl -k -u curator:******** https://127.0.0.1:9201
Also kibana and logstash communicates with elasticsearch properly.
Does anyone knows, what might be the problem or how to get more info?
Edit 1:
unfortunately I dont have permissions to use yum, when I installed elasticsearch-curator-5.8.1-1.x86_64.rpm to my home dir using
cd {{ python_installation_dest }} && rpm2cpio ../elasticsearch-curator-5.8.1-1.x86_64.rpm| cpio -idmB
and then run curator, I got:
Fatal Python error: initfsencoding: Unable to get the locale encoding Traceback (most recent call last): File "/opt/python/3.7.4/lib/python3.7/encodings/__init__.py", line 31, in <module> zipimport.ZipImportError: can't decompress data; zlib not available
I dont use pythin 3.7.4, how can I change path to python?
The error appears to be tied to Python issues within RedHat and associated variants. As the official RPM version of Curator 5.8.x now bundles both its own version of Python 3.7.4 and its own up-to-date OpenSSL shared library (1.1.1c in Curator 5.8.1), you will get better results using the official RPM build.
I also faced a similar issue when I was using Curator 5.5.4 for my Elastic Search 5.6. The issue seems to be tied to some python dependency associated with Curator with versions below 5.8.
To fix this, I first installed Curator 5.8 which internally upgraded all the other Python dependencies of Curator. I then uninstalled Curator 5.8 which just removed Curator 5.8 and not its upgraded dependencies.
In the end, I installed Curator 5.5.4. It worked like a charm after that.
Just FYI, it has been working well on production.
pip install --no-cache-dir elasticsearch_curator==5.8 --user
pip uninstall elasticsearch_curator==5.8
pip install --no-cache-dir elasticsearch_curator==5.5.4 --user
Related
C:\Users\matth>pip3 install pygame
Collecting pygame
Using cached pygame-2.1.2.tar.gz (10.1 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [80 lines of output]
WARNING, No "Setup" File Exists, Running "buildconfig/config.py"
Using WINDOWS configuration...
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 975, in send
self.connect()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1075, in _create
self.do_handshake()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\setup.py", line 359, in <module>
buildconfig.config.main(AUTO_CONFIG)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\config.py", line 225, in main
deps = CFG.main(**kwds)
^^^^^^^^^^^^^^^^
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\config_win.py", line 497, in main
and download_win_prebuilt.ask(**download_kwargs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 290, in ask
update(x86=x86, x64=x64)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 273, in update
download_prebuilts(download_dir, x86=x86, x64=x64)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 124, in download_prebuilts
download_sha1_unzip(url, checksum, temp_dir, 1)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 54, in download_sha1_unzip
response = urllib.urlopen(request).read()
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)>
Making dir :prebuilt_downloads:
Downloading... https://www.libsdl.org/release/SDL2-devel-2.0.18-VC.zip ed561079ec622b0bab5a9e02976f5d540b0622da
---
For help with compilation see:
https://www.pygame.org/wiki/CompileWindows
To contribute to pygame development see:
https://www.pygame.org/contribute.html
---
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
New answer (Feb 2023)
pip install pygame-ce
^ pygame-ce (pygame community edition) is a modern fork of pygame by many pygame contributors that supports 3.11 out of the box, just recently launched. See https://github.com/pygame-community/pygame-ce/tree/main
Short answer
pip install pygame --pre is necessary right now due to your Python version
Long answer
Your traceback shows you are using Python 3.11, which was just released earlier today. Pygame is a "C extension" and needs to re-release itself with new "wheels" (binaries) for every new minor release of Python (3.9, 3.10, 3.11, etc.)
The latest "full" pygame release, 2.1.2, was released a while back and doesn't come with wheels for 3.11, so your system tries to build pygame from source. It fails to do so, because you don't have a pygame development environment set up.
However, the latest pre release of pygame, 2.1.3.dev8 (released recently) has wheels for 3.11, so your system will not need to compile pygame from source.
Once pygame 2.1.3 is released, this extra flag (which tells it to look for pre releases), won't be necessary.
If you are using poetry. Do the following:
poetry add pygame=2.1.3.dev8
If you are using pip. Do the following:
pip install pygame --pre
Install python 3.10 and wait for pygame to be ready for 3.11 and then install pygame for that version. You can also have 2 version of python installed.
To work with each one use:
py -3.10 script.py
py -3.11 script.py
C:\Users\matth>pip3 install pygame
Collecting pygame
Using cached pygame-2.1.2.tar.gz (10.1 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [80 lines of output]
WARNING, No "Setup" File Exists, Running "buildconfig/config.py"
Using WINDOWS configuration...
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 975, in send
self.connect()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1075, in _create
self.do_handshake()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\setup.py", line 359, in <module>
buildconfig.config.main(AUTO_CONFIG)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\config.py", line 225, in main
deps = CFG.main(**kwds)
^^^^^^^^^^^^^^^^
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\config_win.py", line 497, in main
and download_win_prebuilt.ask(**download_kwargs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 290, in ask
update(x86=x86, x64=x64)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 273, in update
download_prebuilts(download_dir, x86=x86, x64=x64)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 124, in download_prebuilts
download_sha1_unzip(url, checksum, temp_dir, 1)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 54, in download_sha1_unzip
response = urllib.urlopen(request).read()
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)>
Making dir :prebuilt_downloads:
Downloading... https://www.libsdl.org/release/SDL2-devel-2.0.18-VC.zip ed561079ec622b0bab5a9e02976f5d540b0622da
---
For help with compilation see:
https://www.pygame.org/wiki/CompileWindows
To contribute to pygame development see:
https://www.pygame.org/contribute.html
---
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
New answer (Feb 2023)
pip install pygame-ce
^ pygame-ce (pygame community edition) is a modern fork of pygame by many pygame contributors that supports 3.11 out of the box, just recently launched. See https://github.com/pygame-community/pygame-ce/tree/main
Short answer
pip install pygame --pre is necessary right now due to your Python version
Long answer
Your traceback shows you are using Python 3.11, which was just released earlier today. Pygame is a "C extension" and needs to re-release itself with new "wheels" (binaries) for every new minor release of Python (3.9, 3.10, 3.11, etc.)
The latest "full" pygame release, 2.1.2, was released a while back and doesn't come with wheels for 3.11, so your system tries to build pygame from source. It fails to do so, because you don't have a pygame development environment set up.
However, the latest pre release of pygame, 2.1.3.dev8 (released recently) has wheels for 3.11, so your system will not need to compile pygame from source.
Once pygame 2.1.3 is released, this extra flag (which tells it to look for pre releases), won't be necessary.
If you are using poetry. Do the following:
poetry add pygame=2.1.3.dev8
If you are using pip. Do the following:
pip install pygame --pre
Install python 3.10 and wait for pygame to be ready for 3.11 and then install pygame for that version. You can also have 2 version of python installed.
To work with each one use:
py -3.10 script.py
py -3.11 script.py
C:\Users\matth>pip3 install pygame
Collecting pygame
Using cached pygame-2.1.2.tar.gz (10.1 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [80 lines of output]
WARNING, No "Setup" File Exists, Running "buildconfig/config.py"
Using WINDOWS configuration...
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 975, in send
self.connect()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\http\client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1075, in _create
self.do_handshake()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\setup.py", line 359, in <module>
buildconfig.config.main(AUTO_CONFIG)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\config.py", line 225, in main
deps = CFG.main(**kwds)
^^^^^^^^^^^^^^^^
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\config_win.py", line 497, in main
and download_win_prebuilt.ask(**download_kwargs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 290, in ask
update(x86=x86, x64=x64)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 273, in update
download_prebuilts(download_dir, x86=x86, x64=x64)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 124, in download_prebuilts
download_sha1_unzip(url, checksum, temp_dir, 1)
File "C:\Users\matth\AppData\Local\Temp\pip-install-g18yr5c2\pygame_2c1b94c764cc404881a71cf0ac51d63e\buildconfig\download_win_prebuilt.py", line 54, in download_sha1_unzip
response = urllib.urlopen(request).read()
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.193.0_x64__qbz5n2kfra8p0\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)>
Making dir :prebuilt_downloads:
Downloading... https://www.libsdl.org/release/SDL2-devel-2.0.18-VC.zip ed561079ec622b0bab5a9e02976f5d540b0622da
---
For help with compilation see:
https://www.pygame.org/wiki/CompileWindows
To contribute to pygame development see:
https://www.pygame.org/contribute.html
---
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
New answer (Feb 2023)
pip install pygame-ce
^ pygame-ce (pygame community edition) is a modern fork of pygame by many pygame contributors that supports 3.11 out of the box, just recently launched. See https://github.com/pygame-community/pygame-ce/tree/main
Short answer
pip install pygame --pre is necessary right now due to your Python version
Long answer
Your traceback shows you are using Python 3.11, which was just released earlier today. Pygame is a "C extension" and needs to re-release itself with new "wheels" (binaries) for every new minor release of Python (3.9, 3.10, 3.11, etc.)
The latest "full" pygame release, 2.1.2, was released a while back and doesn't come with wheels for 3.11, so your system tries to build pygame from source. It fails to do so, because you don't have a pygame development environment set up.
However, the latest pre release of pygame, 2.1.3.dev8 (released recently) has wheels for 3.11, so your system will not need to compile pygame from source.
Once pygame 2.1.3 is released, this extra flag (which tells it to look for pre releases), won't be necessary.
If you are using poetry. Do the following:
poetry add pygame=2.1.3.dev8
If you are using pip. Do the following:
pip install pygame --pre
Install python 3.10 and wait for pygame to be ready for 3.11 and then install pygame for that version. You can also have 2 version of python installed.
To work with each one use:
py -3.10 script.py
py -3.11 script.py
Summary
We are using a custom CA in our enviroment. I have imported the CA certificates in the trust store by downloading the Base64 certs and updating the trust store using update-ca-certificates.
I am able to run cURL queries to my REST API, however the request library fails with an SSL error when running the same.
What I have tried
I have tried specifying the root ca certificate file path to the library, but got the same error. How do I troubleshoot this issue? Setting verify to false is not an option.
Code Run
Works
curl -X GET https://api.me.com/admin/ -H 'Authorization: Token 4ae5'
Doesn't work
requests.get('https://api.me.com/admin/', headers={'Authorization': 'Token 4ae5'}, verify='/etc/ssl/certs/root.pem')
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 345, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 844, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 326, in connect
ssl_context=context)
File "/usr/local/lib/python3.5/dist-packages/urllib3/util/ssl_.py", line 325, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket
_context=self)
File "/usr/lib/python3.5/ssl.py", line 752, in __init__
self.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 630, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/requests/api.py", line 67, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 480, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 588, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 447, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
... requests.get(..., verify='/etc/ssl/certs/root.pem')
Given your description of using update-ca-certificates it looks like you are using Debian or similar (i.e. Ubuntu). In this case the relevant path should be /etc/ssl/certs/ca-certificates.crt.
The reason is that Python Requests uses certificates from the python-certifi package., not those of the underlying operating system.
certifi includes all CA certificates from Mozilla, and it is also relatively straightforward to add missing certificates to it.
See this post for details on adding certificates to certifi: https://stackoverflow.com/a/66111417/516699
The good thing is also that you do not tweak the SSL of the underlying system, just that of the Python environment your are working in.
Edit: Turns out I'm an idiot and can't read something I already looked up, but the fix was to brew uninstall openssl then rm -rf /usr/local/openssl and then run brew install openssl there's an issue with openssl installed before OS X 10.10.3 and reinstalling it fixes it.
I'm trying to create a reddit bot that searches specified users on youtube, and submits their posts to a subreddit. When it tries to log in it gives an SSL error.
I'm on Python 2.7.9, and installed the gdata through pip.
Here's my code:
import gdata.youtube
import gdata.youtube.service
import praw
def main():
yt_service = gdata.youtube.service.YouTubeService()
yt_service.ssl = True
yt_service.email = ''
yt_service.password = ''
yt_service.source = 'reddit-youtube-submitter-bot'
yt_service.ProgrammaticLogin()
if __name__ == "__main__":
main()
Here's the error:
Traceback (most recent call last):
File "youtube_submitter_bot_mine.py", line 15, in <module>
main()
File "youtube_submitter_bot_mine.py", line 12, in main
yt_service.ProgrammaticLogin()
File "/usr/local/lib/python2.7/site-packages/gdata/service.py", line 771, in ProgrammaticLogin
headers={'Content-Type':'application/x-www-form-urlencoded'})
File "/usr/local/lib/python2.7/site-packages/atom/http.py", line 163, in request
connection.endheaders()
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 997, in endheaders
self._send_output(message_body)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 850, in _send_output
self.send(msg)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 812, in send
self.connect()
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1212, in connect
server_hostname=server_hostname)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 350, in wrap_socket
_context=self)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 566, in __init__
self.do_handshake()
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 788, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)