Using: Flower 0.9.5 (installed Tornado 6.0.4), Celery 4.4.6, Python 3.7
When starting Flower with
celery -A myProj flower
everything works as expected. Flower serves at http://localhost:5555.
When starting Flower with
celery -A myProj flower --keyfile=/home/me/cert/key.pem --certfile=/home/me/cert/cert.pem
it serves at https://localhost:5555 but when trying to access it, Chrome states ERR_CONNECTION_RESET and Flower logs
2020-09-16 17:19:37,421 - tornado.general - ERROR - Uncaught exception, closing connection.
Traceback (most recent call last):
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 711, in _handle_events
self._handle_read()
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 1498, in _handle_read
self._do_ssl_handshake()
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 1458, in _do_ssl_handshake
if not self._verify_cert(self.socket.getpeercert()):
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 1481, in _verify_cert
assert verify_mode in (ssl.CERT_NONE, ssl.CERT_REQUIRED, ssl.CERT_OPTIONAL)
UnboundLocalError: local variable 'verify_mode' referenced before assignment
2020-09-16 17:19:37,423 - asyncio - ERROR - Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
File "/home/me/python/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/home/me/.env/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 139, in _handle_events
handler_func(fileobj, events)
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 711, in _handle_events
self._handle_read()
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 1498, in _handle_read
self._do_ssl_handshake()
File "/home/me/.env/lib/python3.7/site-packages/tornado/iostream.py", line 1458, in _do_ssl_handshake
if not self._verify_cert(self.socket.getpeercert()):
File "/home/me/.env/lib/python/site-packages/tornado/iostream.py", line 1481, in _verify_cert
assert verify_mode in (ssl.CERT_NONE, ssl.CERT_REQUIRED, ssl.CERT_OPTIONAL)
UnboundLocalError: local variable 'verify_mode' referenced before assignment
Note: Everything works when running Flower with
celery -B brokerURL flower --keyfile=/home/me/cert/key.pem --certfile=/home/me/cert/cert.pem
In /home/me/.env/lib/python3.7/site-packages/tornado/iostream.py there is:
def _verify_cert(self, peercert: Any) -> bool:
"""Returns ``True`` if peercert is valid according to the configured
validation mode and hostname.
The ssl handshake already tested the certificate for a valid
CA signature; the only thing that remains is to check
the hostname.
"""
if isinstance(self._ssl_options, dict):
verify_mode = self._ssl_options.get("cert_reqs", ssl.CERT_NONE)
elif isinstance(self._ssl_options, ssl.SSLContext):
verify_mode = self._ssl_options.verify_mode
assert verify_mode in (ssl.CERT_NONE, ssl.CERT_REQUIRED, ssl.CERT_OPTIONAL) # LINE 1481
if verify_mode == ssl.CERT_NONE or self._server_hostname is None:
return True
cert = self.socket.getpeercert()
if cert is None and verify_mode == ssl.CERT_REQUIRED:
gen_log.warning("No SSL certificate given")
return False
try:
ssl.match_hostname(peercert, self._server_hostname)
except ssl.CertificateError as e:
gen_log.warning("Invalid SSL certificate: %s" % e)
return False
else:
return True
How can I pass verify_mode = ssl.CERT_REQUIRED to tornado via Flower? Setting it manually inside _verify_cert does work.
Related
I am using python docker SDK to create multiple chrome containers. Below is my script
Here first I have pull docker image then try to create 2 containers out it. But it is failing with an error message that ports are already in use but I am incrementing the ports value as per container count.
import docker, sys
class CreateContainer:
def __init__(self):
self.client = CreateContainer.create_client()
#staticmethod
def create_client():
client = docker.from_env()
return client
def pull_image(self, image_name):
image = client.images.pull(image_name)
print(image.name)
def create_containers(self, image, container_name, expose_port, countainer_count=1):
container = self.client.containers.run(
image,
name=container_name,
hostname=container_name,
ports=expose_port,
detach=True
)
for line in container.logs():
print(line)
return container
if __name__ == '__main__':
threads = int(sys.argv[1])
c_obj = CreateContainer()
for i in range(1, threads+1):
c_obj.create_containers("selenium/standalone-chrome", "Chrome_{0}".format(i), expose_port={5550+i:4444})
------Run----------
python test.py 2
------error-----
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\docker\api\client.py", line 268, in _raise_for_status
response.raise_for_status()
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localnpipe/v1.40/containers/260894dbaec6946e5f31fdbfb5307182d2f621c12a38f328f6efac58df58854d/start
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Desktop\FreeLance\Utility\src\docker_engine\create_container.py", line 53, in <module>
c_obj.create_containers("selenium/standalone-chrome", "Chrome_{0}".format(i), expose_port={5550+i:4444})
File "C:\Users\Desktop\FreeLance\Utility\src\docker_engine\create_container.py", line 20, in create_containers
container = self.client.containers.run(
File "C:\Program Files\Python39\lib\site-packages\docker\models\containers.py", line 818, in run
container.start()
File "C:\Program Files\Python39\lib\site-packages\docker\models\containers.py", line 404, in start
return self.client.api.start(self.id, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\docker\utils\decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\docker\api\container.py", line 1111, in start
self._raise_for_status(res)
File "C:\Program Files\Python39\lib\site-packages\docker\api\client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "C:\Program Files\Python39\lib\site-packages\docker\errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localnpipe/v1.40/containers/260894dbaec6946e5f31fdbfb5307182d2f621c12a38f328f6efac58df58854d/start: Internal Server Error ("driver failed programming external connectivity on endpoint Chrome_2 (c0c77528743a1e3153201565b2cb520243b66adbb903bb69e91a00c4399aca62): Bind for 0.0.0.0:4444 failed: port is already allocated")
While proving port the syntax is container_port:host_port
c_obj.create_containers("selenium/standalone-chrome", "Chrome_{0}".format(i), expose_port={4444:5550+i})
documentation : https://docker-py.readthedocs.io/en/stable/containers.html
I have a server running locally. When I run it on AWS EC2 and send a request from outside on port 8000, I get the following error:
$ uvicorn sql_app.main:app --host="0.0.0.0" --port=8000
INFO: Started server process [9806]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 170, in handle_events
event = self.conn.next_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 425, in next_event
event = self._extract_next_receive_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
event = self._reader(self._receive_buffer)
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_readers.py", line 73, in maybe_read_from_IDLE_client
request_line_re, lines[0], "illegal request line: {!r}", lines[0]
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 88, in validate
raise LocalProtocolError(msg)
h11._util.RemoteProtocolError: illegal request line: bytearray(b'\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03\x91\xa5\xe2Y\xf0\xa1\xdd\x1d+\x08\x1c\r\x15X\x1d#\x1e/\xb1N\x00\xb5\xe5\xec\xf3F\x1fm\x03\xa1{> \xa80\xb4\x14\x1aUs\xaa\xcd\xc3<s\xcd\xd1\x17\xdf3\x0e\xdbh\xd1c\x88}\x8c\x1f\xa5\x15\x9aa\x14I\x00 ')
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 170, in handle_events
event = self.conn.next_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 425, in next_event
event = self._extract_next_receive_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
event = self._reader(self._receive_buffer)
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
raise LocalProtocolError("illegal request line")
h11._util.RemoteProtocolError: illegal request line
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 170, in handle_events
event = self.conn.next_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 425, in next_event
event = self._extract_next_receive_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
event = self._reader(self._receive_buffer)
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
raise LocalProtocolError("illegal request line")
h11._util.RemoteProtocolError: illegal request line
WARNING: Invalid HTTP request received.
It would be very nice if you could tell me how to do it on port 80.
I was getting the same arcane WARNING: Invalid HTTP request received. error with an unhelpful stack trace. I tried all of the environment variable tweaks recommended and none worked (see FastAPI issue #680, uvicorn issue #441).
My issue was that when I was calling my FastAPI microservice I was using https when my microservice did not have HTTPS support. I changed the url from https to http and it started working as expected.
Note that if your service requires HTTPS support you can add HTTPS support as Ilgizar Murzakov suggests.
Had the same issue. Disabling http2 on application load balancer helped me.
I had the same issue and solved it by adding HTTPS support. I configured NGINX with LetsEncrypt certificate. Make sure that the ports that you use are open at your security group settings.
this is because uvicorn updated to display detailed error. uvicorn #886
then, use uvicorn 0.13.1 which is before the update. this version won't display the error.
P.S: this update is to fix this error. and uvicorn haven't fixed the error for now. uvicorn #1296
I am trying to use Azure Service Bus as the broker for my celery app.
I have patched the solution by referring to various sources.
The goal is to use Azure Service Bus as the broker and PostgresSQL as the backend.
I created an Azure Service Bus and copied the credentials for the RootManageSharedAccessKey to the celery app.
Following is the task.py
from time import sleep
from celery import Celery
from kombu.utils.url import safequote
SAS_policy = safequote("RootManageSharedAccessKey") #SAS Policy
SAS_key = safequote("1234222zUY28tRUtp+A2YoHmDYcABCD") #Primary key from the previous SS
namespace = safequote("bluenode-dev")
app = Celery('tasks', backend='db+postgresql://afsan.gujarati:admin#localhost/local_dev',
broker=f'azureservicebus://{SAS_policy}:{SAS_key}=#{namespace}')
#app.task
def divide(x, y):
sleep(30)
return x/y
When I try to run the Celery app using the following command:
celery -A tasks worker --loglevel=INFO
I get the following error
[2020-10-09 14:00:32,035: CRITICAL/MainProcess] Unrecoverable error: AzureHttpError('Unauthorized\n<Error><Code>401</Code><Detail>claim is empty or token is invalid. TrackingId:295f7c76-770e-40cc-8489-e0eb56248b09_G5S1, SystemTracker:bluenode-dev.servicebus.windows.net:$Resources/Queues, Timestamp:2020-10-09T20:00:31</Detail></Error>')
Traceback (most recent call last):
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 918, in create_channel
return self._avail_channels.pop()
IndexError: pop from empty list
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/azure/servicebus/control_client/servicebusservice.py", line 1225, in _perform_request
resp = self._filter(request)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/azure/servicebus/control_client/_http/httpclient.py", line 211, in perform_request
raise HTTPError(status, message, respheaders, respbody)
azure.servicebus.control_client._http.HTTPError: Unauthorized
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/worker/worker.py", line 203, in start
self.blueprint.start(self)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start
step.start(parent)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/bootsteps.py", line 365, in start
return self.obj.start()
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 311, in start
blueprint.start(self)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start
step.start(parent)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/worker/consumer/connection.py", line 21, in start
c.connection = c.connect()
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 398, in connect
conn = self.connection_for_read(heartbeat=self.amqheartbeat)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 404, in connection_for_read
return self.ensure_connected(
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 430, in ensure_connected
conn = conn.ensure_connection(
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/connection.py", line 383, in ensure_connection
self._ensure_connection(*args, **kwargs)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/connection.py", line 435, in _ensure_connection
return retry_over_time(
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/utils/functional.py", line 325, in retry_over_time
return fun(*args, **kwargs)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/connection.py", line 866, in _connection_factory
self._connection = self._establish_connection()
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/connection.py", line 801, in _establish_connection
conn = self.transport.establish_connection()
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 938, in establish_connection
self._avail_channels.append(self.create_channel(self))
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 920, in create_channel
channel = self.Channel(connection)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/kombu/transport/azureservicebus.py", line 64, in __init__
for queue in self.queue_service.list_queues():
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/azure/servicebus/control_client/servicebusservice.py", line 313, in list_queues
response = self._perform_request(request)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/azure/servicebus/control_client/servicebusservice.py", line 1227, in _perform_request
return _service_bus_error_handler(ex)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/azure/servicebus/control_client/_serialization.py", line 569, in _service_bus_error_handler
return _general_error_handler(http_error)
File "/Users/afsan.gujarati/.pyenv/versions/3.8.1/envs/celery-servicebus/lib/python3.8/site-packages/azure/servicebus/control_client/_common_error.py", line 41, in _general_error_handler
raise AzureHttpError(message, http_error.status)
azure.common.AzureHttpError: Unauthorized
<Error><Code>401</Code><Detail>claim is empty or token is invalid. TrackingId:295f7c76-770e-40cc-8489-e0eb56248b09_G5S1, SystemTracker:bluenode-dev.servicebus.windows.net:$Resources/Queues, Timestamp:2020-10-09T20:00:31</Detail></Error>
I don't see a straight solution for this anywhere. What am I missing?
P.S. I did not create the Queue in Azure Service Bus. I am assuming that celery would create the Queue by itself when the celery app is executed.
P.S.S. I also tried to use the exact same credentials in Python's Service Bus Client and it seemed to work. It feels like a Celery issue, but I am not able to figure out exactly what.
If you want to use Azure Service Bus Transport to connect Azure service bus, the URL should be azureservicebus://{SAS policy name}:{SAS key}#{Service Bus Namespace}.
For example
Get Shared access policies RootManageSharedAccessKey
Code
from celery import Celery
from kombu.utils.url import safequote
SAS_policy = "RootManageSharedAccessKey" # SAS Policy
# Primary key from the previous SS
SAS_key = safequote("X/*****qyY=")
namespace = "bowman1012"
app = Celery('tasks', backend='db+postgresql://<>#localhost/<>',
broker=f'azureservicebus://{SAS_policy}:{SAS_key}#{namespace}')
#app.task
def add(x, y):
return x + y
I try to implement Apache Airflow with the CeleryExecutor. For the database I use Postgres, for the celery message queue I use Redis. When using LocalExecutor everything works fine, but when I set the CeleryExecutor in the airflow.cfg and want to set the Postgres database as the result_backend
result_backend = postgresql+psycopg2://airflow_user:*******#localhost/airflow
I get this error when running the Airflow scheduler no matter which DAG I trigger:
[2020-03-18 14:14:13,341] {scheduler_job.py:1382} ERROR - Exception when executing execute_helper
Traceback (most recent call last):
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
return obj.__dict__[self.__name__]
KeyError: 'backend'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 1380, in _execute
self._execute_helper()
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 1441, in _execute_helper
if not self._validate_and_run_task_instances(simple_dag_bag=simple_dag_bag):
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 1503, in _validate_and_run_task_instances
self.executor.heartbeat()
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/airflow/executors/base_executor.py", line 130, in heartbeat
self.trigger_tasks(open_slots)
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 205, in trigger_tasks
cached_celery_backend = tasks[0].backend
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/celery/local.py", line 146, in __getattr__
return getattr(self._get_current_object(), name)
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/celery/app/task.py", line 1037, in backend
return self.app.backend
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/celery/app/base.py", line 1227, in backend
return self._get_backend()
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/celery/app/base.py", line 944, in _get_backend
self.loader)
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/celery/app/backends.py", line 74, in by_url
return by_name(backend, loader), url
File "<PATH_TO_VIRTUALENV>/lib/python3.6/site-packages/celery/app/backends.py", line 60, in by_name
backend, 'is a Python module, not a backend class.'))
celery.exceptions.ImproperlyConfigured: Unknown result backend: 'postgresql'. Did you spell that correctly? ('is a Python module, not a backend class.')
The exact same parameter to direct to the database works
sql_alchemy_conn = postgresql+psycopg2://airflow_user:*******#localhost/airflow
Setting Redis as the celery result_backend works, but I read it is not the recommended way.
result_backend = redis://localhost:6379/0
Does anyone see what I am doing wrong?
You need to add the db+ prefix to the database connection string:
f"db+postgresql+psycopg2://{user}:{password}#{host}/{database}"
This is also mentioned in the docs: https://docs.celeryproject.org/en/stable/userguide/configuration.html#database-url-examples
You need to add the db+ prefix to the database connection string:
result_backend = db+postgresql://airflow_user:*******#localhost/airflow
I have a RabbitMQ 3.6.1 server on Ubuntu 14.04 running properly. I tried to configure an SSL listener according to official documentation. No problems during the startup.
However when trying to establish a connection, I get the following error on Python/pika side (full transcript below):
pika.exceptions.AuthenticationError: EXTERNAL
What does EXTERNAL mean here? How to debug / get further details of the error?
Course of actions (to test I used a Vagrant box and a local connection):
RabbitMQ starts SSL Listener on port 5671 (per /var/log/rabbitmq/rabbit#rabbitmq-server.log):
started SSL Listener on [::]:5671
I execute the pika.BlockingConnection on the client side.
On the server side I can see an incoming connection:
=INFO REPORT==== 17-Apr-2016::17:07:15 ===
accepting AMQP connection <0.2788.0> (127.0.0.1:48404 -> 127.0.0.1:5671)
Client fails with:
pika.exceptions.AuthenticationError: EXTERNAL
Server timeouts:
=ERROR REPORT==== 17-Apr-2016::17:07:25 ===
closing AMQP connection <0.2788.0> (127.0.0.1:48404 -> 127.0.0.1:5671):
{handshake_timeout,frame_header}
Full transcript of the client side:
>>> import pika, ssl
>>> from pika.credentials import ExternalCredentials
>>> ssl_options = ({"ca_certs": "/etc/rabbitmq/certs/testca/cacert.pem",
... "certfile": "/etc/rabbitmq/certs/client/cert.pem",
... "keyfile": "/etc/rabbitmq/certs/client/key.pem",
... "cert_reqs": ssl.CERT_REQUIRED,
... "server_side": False})
>>> host = "localhost"
>>> connection = pika.BlockingConnection(
... pika.ConnectionParameters(
... host, 5671, credentials=ExternalCredentials(),
... ssl=True, ssl_options=ssl_options))
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 339, in __init__
self._process_io_for_connection_setup()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 374, in _process_io_for_connection_setup
self._open_error_result.is_ready)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 410, in _flush_output
self._impl.ioloop.poll()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/select_connection.py", line 602, in poll
self._process_fd_events(fd_event_map, write_only)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/select_connection.py", line 443, in _process_fd_events
handler(fileno, events, write_only=write_only)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 364, in _handle_events
self._handle_read()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 415, in _handle_read
self._on_data_available(data)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1347, in _on_data_available
self._process_frame(frame_value)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1414, in _process_frame
if self._process_callbacks(frame_value):
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1384, in _process_callbacks
frame_value) # Args
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 60, in wrapper
return function(*tuple(args), **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 92, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 236, in process
callback(*args, **keywords)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1298, in _on_connection_start
self._send_connection_start_ok(*self._get_credentials(method_frame))
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1077, in _get_credentials
raise exceptions.AuthenticationError(self.params.credentials.TYPE)
pika.exceptions.AuthenticationError: EXTERNAL
>>>
The Python / pika code in the question is correct.
The error:
pika.exceptions.AuthenticationError: EXTERNAL
is reported when client certificate authorisation is not enabled on the RabbitMQ server side. The word EXTERNAL in the error refers to the authentication mechanism as described here.
To enable:
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl