I have an issue since I added gevent to my flask project: the page spawning greenlets always timeout / bad gateway.
The part of code causing this issue is this: https://github.com/Kyria/LazyBlacksmith/blob/master/lazyblacksmith/views/ajax.py#L145-L221
Here what i've tested:
Running my flask app with python app.py : works fine with or without gevent/greenlets.
Running behind uwsgi without gevent/greenlet : works fine.
Running behind uwsgi with gevent/greenlet : fails.
So, after searching for a while, reading uwsgi doc, i tried to add some stuff to my uwsgi conf :
loop = async
async = 10
greenlet = true
ugreen = true
After restarting my uwsgi, i didn't have to wait for error, the Bad Gateway was really quick, but with an error in the log this time:
Traceback (most recent call last):
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "./lazyblacksmith/views/ajax.py", line 152, in get_price_and_tax
crest = get_crest()
File "./lazyblacksmith/utils/crestutils.py", line 17, in get_crest
crest()
File "./lazyblacksmith/utils/pycrest/eve.py", line 212, in __call__
self._data = APIObject(self.get(self._endpoint), self)
File "./lazyblacksmith/utils/pycrest/eve.py", line 156, in get
res = self._session.get(resource, params=prms, headers=headers)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/sessions.py", line 480, in get
return self.request('GET', url, **kwargs)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
body=body, headers=headers)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
self._validate_conn(conn)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 782, in _validate_conn
conn.connect()
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connection.py", line 215, in connect
conn = self._new_conn()
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connection.py", line 135, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/requests/packages/urllib3/util/connection.py", line 66, in create_connection
for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/gevent/_socketcommon.py", line 268, in getaddrinfo
return get_hub().resolver.getaddrinfo(host, port, family, socktype, proto, flags)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/gevent/resolver_thread.py", line 64, in getaddrinfo
return self.pool.apply(_socket.getaddrinfo, args, kwargs)
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/gevent/pool.py", line 300, in apply
return self.spawn(func, *args, **kwds).get()
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/gevent/event.py", line 284, in get
result = self.hub.switch()
File "/home/lazyblacksmith/LazyBlacksmith/env/local/lib/python2.7/site-packages/gevent/hub.py", line 592, in switch
return greenlet.switch(self)
MemoryError
I also tried to add the following in my uwsgi app conf :
gevent = 10
gevent-monkey-patch = true
limit-as = 512
And i tried to increase the timeout in nginx uwsgi_read_timeout 300, but it didn't change anything...
My uwsgi conf is the following:
# lazyblacksmith.ini file
[uwsgi]
chdir = /home/lazyblacksmith/LazyBlacksmith
home = /home/lazyblacksmith/LazyBlacksmith/env/
callable = app
wsgi-file = app.py
master = true
processes = 10
socket = /home/lazyblacksmith/LazyBlacksmith/env/lazyblacksmith.socket
chmod-socket = 664
vacuum = true
uid = lazyblacksmith
gid = www-data
My nginx conf :
upstream lbflask {
server unix:/home/lazyblacksmith/LazyBlacksmith/env/lazyblacksmith.socket;
}
# configuration of the server
server {
# the port your site will be served on
listen 80;
# the domain name it will serve for
server_name site.com; #
charset utf-8;
location /static {
alias /home/lazyblacksmith/LazyBlacksmith/lazyblacksmith/static;
include /etc/nginx/mime.types;
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass lbflask;
include /etc/nginx/uwsgi_params; # the uwsgi_params file you installed
}
}
I don't find anything relevant anymore :/ I think i'll end up moving my gevent stuff in tasks (with celery), but i still want to understand what is my problem...
For information:
Debian 8
uwsgi 2.0.7-debian
nginx/1.6.2
python 2.7.9
gevent 1.1rc2
greenlet 0.4.9
Thank you !
Related
I am using python flask, and pusher API for live messaging on my website.
Everything works as expected in development flask server, but the trouble starts when I switch to gunicorn and nginx live servers.
When I trigger pusher event I get an error 500.
Maybe I have to configure something in my nginx.conf file?
/etc/nginx/sites-enabled/đir
server {
listen 80;
server_name 172.104.228.152;
location /static {
alias /home/marko/Đir/đir/static;
}
location / {
proxy_pass http://localhost:8000;
include /etc/nginx/proxy_params;
proxy_redirect off;
}
}.
This is the error message:
[2020-02-03 17:00:25,938] ERROR in app: Exception on /objave [POST]
Traceback (most recent call last):
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/marko/Đir/đir/rute.py", line 30, in dekorator
return f(*args, **kwargs)
File "/home/marko/Đir/đir/rute.py", line 143, in objave
pusher_client.trigger('objava-kanal', 'nova-objava', {'sport': objava.sport, 'mjesto': objava.mjesto, 'datum': datum, 'sat': sat, 'id': objava.id, 'broj': len(o> File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/pusher/pusher.py", line 118, in trigger
channels, event_name, data, socket_id)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/pusher/http.py", line 31, in __call__
return self.client.http.send_request(self.make_request(*args, **kwargs))
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/pusher/requests.py", line 44, in send_request
**self.options)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked,
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/urllib3/connection.py", line 394, in connect
ssl_context=context,
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 336, in ssl_wrap_socket
context.load_verify_locations(ca_certs, ca_cert_dir)
File "/home/marko/\udcc4\udc90ir/venv/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 450, in load_verify_locations
cafile = cafile.encode("utf-8")
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 12-13: surrogates not allowed
Edit:
'routes.py'
pusher_client = pusher.Pusher(
app_id='930370',
key='43251c740e8c7fdc4747',
secret='6cc68633eec00ebf9b9d',
cluster='eu',
ssl=True
)
#app.route("/objave", methods=["GET", "POST"])
#potrebna_prijava
def objave():
#obrazac za novi događaj
obrazac = ObjavaObrazac()
#obrazac za filter događaja
_filter = Filter()
if obrazac.validate_on_submit():
datum = datetime.strptime(obrazac.datum.data, "%Y/%m/%d %H:%M")
objava = Objava(sport=obrazac.sport.data.capitalize(), mjesto=obrazac.mjesto.data.capitalize(), datum=datum, opis=obrazac.opis.data, korisnik_id=session["korisnik_id"])
db.session.add(objava)
db.session.commit()
datum = objava.datum.strftime("%a, %d.").capitalize()
sat = objava.datum.strftime("%H:%M")
#pusher trigger
pusher_client.trigger('objava-kanal', 'nova-objava', {'sport': objava.sport, 'mjesto': objava.mjesto, 'datum': datum, 'sat': sat, 'id': objava.id, 'broj': len(objava.sudionici)})
flash('Kreirano', 'dobro')
return redirect(url_for('objave'))
if _filter.validate_on_submit():
return redirect(url_for('objave', mjesto_filter=_filter.f_mjesto.data, sport_filter=_filter.f_sport.data))
objave = selektiraj()
#slika korisnika
avatar = Korisnik.query.get(session.get("korisnik_id")).avatar
return render_template("objave.html", obrazac=obrazac, objave=objave, avatar=avatar, filter=_filter)
P.S.
translations: objava = post, objave = posts, obrazac = form
I managed to solve this by setting pusher ssl = False.
It is really strange solution, nowhere to be sad in documentation or error log.
We are creating an application which consists of a frontend (flask api) and a backend that uses celery. The API starts a celery task and retrieves the result like this:
result = data_source_tasks.add_data_point.delay(tok, uuid, source_type, datum, request_counter)
return result.get(timeout=5)
We use RabbitMQ as broker and result backend:
celery_broker_url = pyamqp://guest#localhost//
celery_result_backend = rpc://
After everything runs fine for a while (multiple thousand api calls) I get the following error:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.4/dist-packages/connexion/decorators/decorator.py", line 66, in wrapper
response = function(request)
File "/usr/local/lib/python3.4/dist-packages/connexion/decorators/validation.py", line 122, in wrapper
response = function(request)
File "/usr/local/lib/python3.4/dist-packages/connexion/decorators/validation.py", line 293, in wrapper
return function(request)
File "/usr/local/lib/python3.4/dist-packages/connexion/decorators/decorator.py", line 42, in wrapper
response = function(request)
File "/usr/local/lib/python3.4/dist-packages/connexion/decorators/parameter.py", line 219, in wrapper
return function(**kwargs)
File "/mynedata/lib/api/apicalls.py", line 747, in store_datum
return result.get(timeout=5)
File "/usr/local/lib/python3.4/dist-packages/celery/result.py", line 224, in get
on_message=on_message,
File "/usr/local/lib/python3.4/dist-packages/celery/backends/async.py", line 188, in wait_for_pending
for _ in self._wait_for_pending(result, **kwargs):
File "/usr/local/lib/python3.4/dist-packages/celery/backends/async.py", line 255, in _wait_for_pending
on_interval=on_interval):
File "/usr/local/lib/python3.4/dist-packages/celery/backends/async.py", line 56, in drain_events_until
yield self.wait_for(p, wait, timeout=1)
File "/usr/local/lib/python3.4/dist-packages/celery/backends/async.py", line 65, in wait_for
wait(timeout=timeout)
File "/usr/local/lib/python3.4/dist-packages/celery/backends/rpc.py", line 63, in drain_events
return self._connection.drain_events(timeout=timeout)
File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 301, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/kombu/transport/pyamqp.py", line 103, in drain_events
return connection.drain_events(**kwargs)
File "/usr/local/lib/python3.4/dist-packages/amqp/connection.py", line 471, in drain_events
while not self.blocking_read(timeout):
File "/usr/local/lib/python3.4/dist-packages/amqp/connection.py", line 476, in blocking_read
frame = self.transport.read_frame()
File "/usr/local/lib/python3.4/dist-packages/amqp/transport.py", line 226, in read_frame
frame_header = read(7, True)
File "/usr/local/lib/python3.4/dist-packages/amqp/transport.py", line 401, in _read
s = recv(n - len(rbuf))
ConnectionResetError: [Errno 104] Connection reset by peer
I can see in the console where I started the celery worker that the task (and all following tasks) succeeded, however, result.get results in a timeout for this and all following tasks. Did my connection to the result backend somehow break? If I restart the API, neither restarting the celery worker nor rabbitmq, everything works fine again.
I am fairly new to redis and using configuring redis with flask app.
i have edited redis.conf file and uncommented requirepass
requirepass foobared
and I am able to get into redid-cli using this password.
like this ./redis-cli -h 127.0.0.1 -p 6379 -a foobared
but when I try to initialise an object of Redis class by passing in the password,
>>> import os
>>> import redis
>>> from flask import Flask
>>> from redis import Redis
>>> app = Flask('weatherApp')
>>> app.config["REDIS_HOST"] = os.getenv("REDIS_HOST")
>>> app.config["REDIS_PORT"] = os.getenv("REDIS_PORT")
>>> app.config["REDIS_PWD"] = os.getenv("REDIS_PWD")
>>> app
<Flask 'weatherApp'>
>>> app.config
>>> redis = Redis(app.config["REDIS_HOST"], port=app.config["REDIS_PORT"],
... db=0, password=app.config["REDIS_PWD"])
>>> redis.client_list()
I keep getting the error saying:
Traceback (most recent call last):
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 2000, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/flask/views.py", line 84, in view
return self.dispatch_request(*args, **kwargs)
File "/Users/ciasto-piekarz/Development/web/python/weatherApp/src/views.py", line 192, in dispatch_request
print redis.client_list()
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/client.py", line 703, in client_list
return self.execute_command('CLIENT LIST')
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/client.py", line 667, in execute_command
connection.send_command(*args)
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/connection.py", line 610, in send_command
self.send_packed_command(self.pack_command(*args))
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/connection.py", line 585, in send_packed_command
self.connect()
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/connection.py", line 493, in connect
self.on_connect()
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/connection.py", line 561, in on_connect
if nativestr(self.read_response()) != 'OK':
File "/Users/ciasto-piekarz/Development/web/python/flask/flsk-dimnd/venv/lib/python2.7/site-packages/redis/connection.py", line 629, in read_response
raise response
ResponseError: Client sent AUTH, but no password is set
Any help will be greatly appreciated!
I followed the tutorial for Werkzeug "Shortly" here
And I get this error message after submitting a valid url.
Traceback (most recent call last)
File "/home/sadik/NLM/shortly/shortly.py", line 87, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/Werkzeug-0.9.4-py2.7.egg/werkzeug/wsgi.py", line 579, in __call__
return self.app(environ, start_response)
File "/home/sadik/NLM/shortly/shortly.py", line 83, in wsgi_app
response = self.dispatch_request(request)
File "/home/sadik/NLM/shortly/shortly.py", line 33, in dispatch_request
return getattr(self, 'on_' + endpoint)(request, **values)
File "/home/sadik/NLM/shortly/shortly.py", line 45, in on_new_url
short_id = self.insert_url(url)
File "/home/sadik/NLM/shortly/shortly.py", line 72, in insert_url
short_id = self.redis.get('reverse-url:' + url)
File "/usr/local/lib/python2.7/dist-packages/redis-2.9.1-py2.7.egg/redis/client.py", line 705, in get
return self.execute_command('GET', name)
File "/usr/local/lib/python2.7/dist-packages/redis-2.9.1-py2.7.egg/redis/client.py", line 464, in execute_command
connection.send_command(*args)
File "/usr/local/lib/python2.7/dist-packages/redis-2.9.1-py2.7.egg/redis/connection.py", line 334, in send_command
self.send_packed_command(self.pack_command(*args))
File "/usr/local/lib/python2.7/dist-packages/redis-2.9.1-py2.7.egg/redis/connection.py", line 316, in send_packed_command
self.connect()
File "/usr/local/lib/python2.7/dist-packages/redis-2.9.1-py2.7.egg/redis/connection.py", line 253, in connect
raise ConnectionError(self._error_message(e))
ConnectionError: Error 111 connecting localhost:6379. Connection refused.
The error message indicates that there is something wrong with localhost:6379
The relevant part of code is here:
def create_app(redis_host='localhost', redis_port=6379, with_static=True):
app = Shortly({
'redis_host': redis_host,
'redis_port': redis_port
})
if with_static:
app.wsgi_app = SharedDataMiddleware(app.wsgi_app, {
'/static': os.path.join(os.path.dirname(__file__), 'static')
})
return app
if __name__ == '__main__':
from werkzeug.serving import run_simple
app = create_app()
run_simple('127.0.0.1', 5000, app, use_debugger=True, use_reloader=True)
That means of course that the server is running on localhost:5000. So why is there another port number in the create_app function? That confuses me a bit.
I'm not familiar with shortly or werkzeug but it looks like you're missing the redis server, install one using your favourite package manager and try again.
After running through the basic example for flask-celery(runs fine as far as I can tell) I'm trying to integrate this to my own project. Basically, I'm using this below:
from flask import Blueprint, jsonify, request, session
from flask.views import MethodView
from celery.decorators import task
blueprint = Blueprint('myapi', __name__)
class MyAPI(MethodView):
def get(self, tag):
return get_resource.apply_async(tag)
#task(name="get_task")
def get_resource(tag):
pass
with the same setup as in the example, I'm getting this error:
Traceback (most recent call last):
File "/x/venv/lib/python2.7/site-packages/flask/app.py", line 1518, in __call__
return self.wsgi_app(environ, start_response)
File "/x/venv/lib/python2.7/site-packages/flask/app.py", line 1506, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/x/venv/lib/python2.7/site-packages/flask/app.py", line 1504, in wsgi_app
response = self.full_dispatch_request()
File "/x/venv/lib/python2.7/site-packages/flask/app.py", line 1264, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/x/venv/lib/python2.7/site-packages/flask/app.py", line 1262, in full_dispatch_request
rv = self.dispatch_request()
File "/x/venv/lib/python2.7/site-packages/flask/app.py", line 1248, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/x/venv/lib/python2.7/site-packages/flask/views.py", line 84, in view
return self.dispatch_request(*args, **kwargs)
File "/x/venv/lib/python2.7/site-packages/flask/views.py", line 151, in dispatch_request
return meth(*args, **kwargs)
File "/x/api/modules/document/document.py", line 14, in get
return get_resource.apply_async(tag)
File "/x/venv/lib/python2.7/site-packages/celery/app/task/__init__.py", line 449, in apply_async
publish = publisher or self.app.amqp.publisher_pool.acquire(block=True)
File "/x/venv/lib/python2.7/site-packages/kombu/connection.py", line 657, in acquire
R = self.prepare(R)
File "/x/venv/lib/python2.7/site-packages/kombu/pools.py", line 54, in prepare
p = p()
File "/x/venv/lib/python2.7/site-packages/kombu/pools.py", line 45, in <lambda>
return lambda: self.create_producer()
File "/x/venv/lib/python2.7/site-packages/celery/app/amqp.py", line 265, in create_producer
pub = self.app.amqp.TaskPublisher(conn, auto_declare=False)
File "/x/venv/lib/python2.7/site-packages/celery/app/amqp.py", line 328, in TaskPublisher
return TaskPublisher(*args, **self.app.merge(defaults, kwargs))
File "/x/venv/lib/python2.7/site-packages/celery/app/amqp.py", line 158, in __init__
super(TaskPublisher, self).__init__(*args, **kwargs)
File "/x/venv/lib/python2.7/site-packages/kombu/compat.py", line 61, in __init__
super(Publisher, self).__init__(connection, self.exchange, **kwargs)
File "/x/venv/lib/python2.7/site-packages/kombu/messaging.py", line 79, in __init__
self.revive(self.channel)
File "/x/venv/lib/python2.7/site-packages/kombu/messaging.py", line 168, in revive
channel = channel.default_channel
File "/x/venv/lib/python2.7/site-packages/kombu/connection.py", line 581, in default_channel
self.connection
File "/x/venv/lib/python2.7/site-packages/kombu/connection.py", line 574, in connection
self._connection = self._establish_connection()
File "/x/venv/lib/python2.7/site-packages/kombu/connection.py", line 533, in _establish_connection
conn = self.transport.establish_connection()
File "/x/venv/lib/python2.7/site-packages/kombu/transport/amqplib.py", line 279, in establish_connection
connect_timeout=conninfo.connect_timeout)
File "/x/venv/lib/python2.7/site-packages/kombu/transport/amqplib.py", line 89, in __init__
super(Connection, self).__init__(*args, **kwargs)
File "/x/venv/lib/python2.7/site-packages/amqplib/client_0_8/connection.py", line 129, in __init__
self.transport = create_transport(host, connect_timeout, ssl)
File "/x/venv/lib/python2.7/site-packages/amqplib/client_0_8/transport.py", line 281, in create_transport
return TCPTransport(host, connect_timeout)
File "/x/venv/lib/python2.7/site-packages/amqplib/client_0_8/transport.py", line 85, in __init__
raise socket.error, msg
error: [Errno 111] Connection refused
-->
I'm using redis, and if I install rabbitmq I get another error, but I do not understand this right now --the broker should be redis but its isn't finding it or what? Can anyone give me more of a clue what is going on here? Do I need to import something else, etc. The point is, there is very little beyond the bare bones example and this makes no sense to me.
The most I've been able to determine as that in the Api module there is no access to the 'celery' and when it goes to try and put data there when at the app level, the celery there falls into some defaults, which aren't installed because I'm pointing to redis. Just a guess. I haven't been able to import information into the module, only determined that calling anything 'celery'(for example, output celery.conf) from the app causes an error -- although I could import celery.task.
This is the broker config the application is using, direct from the example:
BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = "redis"
CELERY_REDIS_HOST = "localhost"
CELERY_REDIS_PORT = 6379
CELERY_REDIS_DB = 0
EDIT:
If you'd like to see a demo: https://github.com/thrisp/flask-celery-example
AS it turns out having BROKER_TRANSPORT = 'redis' in your settings is important for whatever is being passed that I'm passing in (for the setup I put forth here and in the git example), I'm not entirely sure why it isn't in the example bits, but is in the ones I've added but it is -- without this it wants to dump everything onto a default ampq queue.
EDIT2:
Also, this a rather a big deal, using the upcoming version of Celery simplifies 10,000 issues when using it with Flask, making all of this unecesssary.
You must configure redis to bind the localhost. In /etc/redis/redis.conf, uncomment the line with
bind 127.0.0.1