Develop Reference

Error 111 connection refused (Python, celery, redis)

I tried to get all the active/scheduled/reserved tasks in redis:
from celery.task.control import inspect
inspect_obj = inspect()
inspect_obj.active()
inspect_obj.scheduled()
inspect_obj.reserved()
But was greeted with a list of errors as follows:
My virtual environment ==> HubblerAPI.
Iam using this from the ec2 console
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 81, in active
return self._request('dump_active', safe=safe)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 71, in _request
timeout=self.timeout, reply=True,
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 316, in broadcast
limit, callback, channel=channel,
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/pidbox.py", line 283, in _broadcast
chan = channel or self.connection.default_channel
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 771, in default_channel
self.connection
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 756, in connection
self._connection = self._establish_connection()
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 711, in _establish_connection
conn = self.transport.establish_connection()
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/transport/pyamqp.py", line 116, in establish_connection
conn = self.Connection(**opts)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
**OSError: [Errno 111] Connection refused**
My celery config file is as follows:
BROKER_TRANSPORT = 'redis'
BROKER_TRANSPORT_OPTIONS = {
'queue_name_prefix': 'dev-',
'wait_time_seconds': 10,
'polling_interval': 30,
# The polling interval decides the number of seconds to sleep
between unsuccessful polls
'visibility_timeout': 3600 * 5,
# If a task is not acknowledged within the visibility_timeout, the
task will be redelivered to another worker and executed.
}
CELERY_MESSAGES_DB = 6
BROKER_URL = "redis://%s:%s/%s" % (AWS_REDIS_ENDPOINT, AWS_REDIS_PORT,
CELERY_MESSAGES_DB)
What am i doing wrong here as the error log suggests that its not using the redis broker.

Looks like your python code doesn't recognize your configs since it is attempting to use RabbitMQ's ampq protocol instead of the configured broker.
I suggest the following
https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/redis.html
Your configs look similar to Django configs for Celery yet it doesn't seem you are using Celery with Django.
https://docs.celeryq.dev/en/latest/django/first-steps-with-django.html

The issue is using "BROKER_URL" instead of "CELERY_BROKER_URL" in settings.py. Celery wasn't finding the URL and was defaulting to the rabbitmq port instead of the redis port.

Neo4j Protocol Error: Server closed connection

I am new to neo4j world. I have successfully used it on my macbook. Now I am deploying it on a remote Linux machine with the same setup. But I keep getting this Protocol error. What caused this issue? How to fix this? I have been banging my head on this error for days.
Traceback (most recent call last):
File "/root/dev/knowledgeGraphH/knowledge/media_entity_mapper.py", line 31, in <module>
main()
File "/root/dev/knowledgeGraphH/knowledge/media_entity_mapper.py", line 28, in main
map_media_to_entities()
File "/root/dev/knowledgeGraphH/knowledge/media_entity_mapper.py", line 7, in map_media_to_entities
data_manager = DataManager()
File "/root/dev/knowledgeGraphH/knowledge/data_manager/data_manager.py", line 13, in __init__
self.graphDB = Neo4jManager()
File "/root/dev/knowledgeGraphH/knowledge/neo4j_manager.py", line 10, in __init__
self.session = self.driver.session()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/session.py", line 148, in session
session = Session(self)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/session.py", line 461, in __init__
self.connection = connect(driver.host, driver.port, driver.ssl_context, **driver.config)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 465, in connect
return Connection(s, der_encoded_server_certificate=der_encoded_server_certificate, **config)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 237, in __init__
self.fetch()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 326, in fetch
self.acknowledge_failure()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 273, in acknowledge_failure
fetch()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 311, in fetch
raw.writelines(self.channel.chunk_reader())
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 169, in chunk_reader
chunk_header = self._recv(2)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 152, in _recv
raise ProtocolError("Server closed connection")
neo4j.v1.exceptions.ProtocolError: Server closed connection

Seems to be port issue. Is the bolt port open or not? You have access to the port or not?
Check the output of the following command:
lsof -i tcp:7687
change the port number if you have changed bolt port address.

It turns out it was because I used the wrong credentials for this connection.

How to debug "pika.exceptions.AuthenticationError: EXTERNAL" error when establishing TLS connection to RabbitMQ?

I have a RabbitMQ 3.6.1 server on Ubuntu 14.04 running properly. I tried to configure an SSL listener according to official documentation. No problems during the startup.
However when trying to establish a connection, I get the following error on Python/pika side (full transcript below):
pika.exceptions.AuthenticationError: EXTERNAL
What does EXTERNAL mean here? How to debug / get further details of the error?
Course of actions (to test I used a Vagrant box and a local connection):
RabbitMQ starts SSL Listener on port 5671 (per /var/log/rabbitmq/rabbit#rabbitmq-server.log):
started SSL Listener on [::]:5671
I execute the pika.BlockingConnection on the client side.
On the server side I can see an incoming connection:
=INFO REPORT==== 17-Apr-2016::17:07:15 ===
accepting AMQP connection <0.2788.0> (127.0.0.1:48404 -> 127.0.0.1:5671)
Client fails with:
pika.exceptions.AuthenticationError: EXTERNAL
Server timeouts:
=ERROR REPORT==== 17-Apr-2016::17:07:25 ===
closing AMQP connection <0.2788.0> (127.0.0.1:48404 -> 127.0.0.1:5671):
{handshake_timeout,frame_header}
Full transcript of the client side:
>>> import pika, ssl
>>> from pika.credentials import ExternalCredentials
>>> ssl_options = ({"ca_certs": "/etc/rabbitmq/certs/testca/cacert.pem",
... "certfile": "/etc/rabbitmq/certs/client/cert.pem",
... "keyfile": "/etc/rabbitmq/certs/client/key.pem",
... "cert_reqs": ssl.CERT_REQUIRED,
... "server_side": False})
>>> host = "localhost"
>>> connection = pika.BlockingConnection(
... pika.ConnectionParameters(
... host, 5671, credentials=ExternalCredentials(),
... ssl=True, ssl_options=ssl_options))
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 339, in __init__
self._process_io_for_connection_setup()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 374, in _process_io_for_connection_setup
self._open_error_result.is_ready)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 410, in _flush_output
self._impl.ioloop.poll()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/select_connection.py", line 602, in poll
self._process_fd_events(fd_event_map, write_only)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/select_connection.py", line 443, in _process_fd_events
handler(fileno, events, write_only=write_only)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 364, in _handle_events
self._handle_read()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 415, in _handle_read
self._on_data_available(data)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1347, in _on_data_available
self._process_frame(frame_value)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1414, in _process_frame
if self._process_callbacks(frame_value):
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1384, in _process_callbacks
frame_value) # Args
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 60, in wrapper
return function(*tuple(args), **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 92, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 236, in process
callback(*args, **keywords)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1298, in _on_connection_start
self._send_connection_start_ok(*self._get_credentials(method_frame))
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1077, in _get_credentials
raise exceptions.AuthenticationError(self.params.credentials.TYPE)
pika.exceptions.AuthenticationError: EXTERNAL
>>>

The Python / pika code in the question is correct.
The error:
pika.exceptions.AuthenticationError: EXTERNAL
is reported when client certificate authorisation is not enabled on the RabbitMQ server side. The word EXTERNAL in the error refers to the authentication mechanism as described here.
To enable:
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl

Error using paho-mqtt in App Engine Python App

I am trying to right a Google Cloud Platform app in python with Flask that makes an MQTT connection. I have included the paho python library by doing pip install paho-mqtt -t libs/. However, when I try to run the app, even if I don't try to connect to MQTT. I get a weird error about IP address checking:
RuntimeError: error('illegal IP address string passed to inet_pton',)
It seems something in the remote_socket lib is causing a problem. Is this a security issue? Is there someway to disable it?
Relevant code:
from flask import Flask
import paho.mqtt.client as mqtt
import logging as logger
app = Flask(__name__)
# Note: We don't need to call run() since our application is embedded within
# the App Engine WSGI application server.
#callback to print out connection status
def on_connect(mosq, obj, rc):
logger.info('on_connect')
if rc == 0:
logger.info("Connected")
mqttc.subscribe('test', 0)
else:
logger.info(rc)
def on_message(mqttc, obj, msg):
logger.info(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
mqttc = mqtt.Client("mqttpy")
mqttc.on_message = on_message
mqttc.on_connect = on_connect
As well as full stack trace:
ERROR 2014-06-03 15:14:57,285 wsgi.py:262]
Traceback (most recent call last):
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 239, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 298, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 84, in LoadObject
obj = __import__(path[0])
File "/Users/cbarnes/code/ignite/tank-demo/appengine-flask-demo/main.py", line 24, in <module>
mqttc = mqtt.Client("mqtthtpp")
File "/Users/cbarnes/code/ignite/tank-demo/appengine-flask-demo/lib/paho/mqtt/client.py", line 403, in __init__
self._sockpairR, self._sockpairW = _socketpair_compat()
File "/Users/cbarnes/code/ignite/tank-demo/appengine-flask-demo/lib/paho/mqtt/client.py", line 255, in _socketpair_compat
listensock.bind(("localhost", 0))
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/dist27/socket.py", line 222, in meth
return getattr(self._sock,name)(*args)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 668, in bind
self._SetProtoFromAddr(request.mutable_proxy_external_ip(), address)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 632, in _SetProtoFromAddr
proto.set_packed_address(self._GetPackedAddr(address))
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 627, in _GetPackedAddr
AI_NUMERICSERV|AI_PASSIVE):
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 338, in getaddrinfo
canonical=(flags & AI_CANONNAME))
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 211, in _Resolve
canon, aliases, addresses = _ResolveName(name, families)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 229, in _ResolveName
apiproxy_stub_map.MakeSyncCall('remote_socket', 'Resolve', request, reply)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 328, in MakeSyncCall
rpc.CheckSuccess()
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl
self.request, self.response)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 200, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 234, in _MakeRealSyncCall
raise pickle.loads(response_pb.exception())
RuntimeError: error('illegal IP address string passed to inet_pton',)
INFO 2014-06-03 15:14:57,291 module.py:639] default: "GET / HTTP/1.1" 500 -
Thanks!

You're trying to connect to host test and port 0 where typically you should use localhost and 1883
See the getting started example here:
https://pypi.python.org/pypi/paho-mqtt/0.9#usage-and-api

Redis example giving HTTP 400: Bad request error

I am trying to cache MySQL queries in my Cherrypy server.
I could not figure out how to solve the error when I was installing pylibmc, so I decided to use Redis-py.
Here I am trying a very simple example.
import redis
cache = redis.StrictRedis(host='localhost', port=8080, db=0)
...
...
cache.set('0', '1') # I also tested with other string keys, but failed with same error
and it's throwing the following error!
[05/May/2014:13:11:13] HTTP Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/Library/Python/2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/Library/Python/2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "server.py", line 92, in submit_data
cache.set(str(idx), '1')#res)
File "/Library/Python/2.7/site-packages/redis/client.py", line 897, in set
return self.execute_command('SET', *pieces)
File "/Library/Python/2.7/site-packages/redis/client.py", line 461, in execute_command
return self.parse_response(connection, command_name, **options)
File "/Library/Python/2.7/site-packages/redis/client.py", line 471, in parse_response
response = connection.read_response()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 339, in read_response
response = self._parser.read_response()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 118, in read_response
(str(byte), str(response)))
InvalidResponse: Protocol Error: H, TTP/1.1 400 Bad Request
I could not figure out what was wrong, and my website runs without a problem on localhost at port 8080 when I am not using Redis.

Your webserver is running on port 8080, not your Redis server. Your Redis server is most likely running on port 6379, unless you changed your config for some reason. Right now you are trying to run Redis queries against your webserver, and that is not going to work. Make sure you are connecting to the correct Redis server address and port and then try again.