So I am getting error while executing code in rq worker
File "/usr/local/bin/rq", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/rq/cli/cli.py", line 75, in wrapper
return ctx.invoke(func, cli_config, *args[1:], **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/rq/cli/cli.py", line 236, in worker
worker.work(burst=burst, logging_level=logging_level)
File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 493, in work
self.execute_job(job, queue)
File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 662, in execute_job
self.fork_work_horse(job, queue)
File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 599, in fork_work_horse
self.main_work_horse(job, queue)
File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 677, in main_work_horse
success = self.perform_job(job, queue)
File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 781, in perform_job
self.prepare_job_execution(job)
File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 706, in prepare_job_execution
registry.add(job, timeout, pipeline=pipeline)
File "/usr/local/lib/python3.7/site-packages/rq/registry.py", line 47, in add
return pipeline.zadd(self.key, score, job.id)
File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 2263, in zadd
for pair in iteritems(mapping):
File "/usr/local/lib/python3.7/site-packages/redis/_compat.py", line 123, in iteritems
return iter(x.items())
AttributeError: 'int' object has no attribute 'items'
14:04:29 Moving job to 'failed' queue (work-horse terminated unexpectedly; waitpid returned 256)
I am trying to run this code in decoupled containers on openshift. Same images work locally. I guess only difference is I was running rq worker on system instead on container when trying locally and my local have both python 2 and 3 while openshift have python3 only.
Can anyone resolve why it started behaving like this. I suspect its due to python version. but i dont know how to run redis/ rq worker with python 2.
That is a known issue, resulting from redis-py's new version 3 - see https://github.com/rq/rq/issues/1014
Related
When I try to sign a PDF document from the CLI using pyhanko-cli package I get the following error:
This is the command I try to run:
pyhanko sign addsig --field Sig1 pemder --key example2.key --cert example2.crt source/SCRIPT_unsigned.pdf out/SCRIPT_signed_cli.pdf
This is the output:
Key passphrase:
2022-09-22 14:56:36,559 - pyhanko.sign.signers.pdf_cms - ERROR - Could not load cryptographic material
Traceback (most recent call last):
File "/home/chris/codetest/pdfsign/pyhanko/.venv/bin/pyhanko", line 8, in <module>
sys.exit(launch())
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/pyhanko/__main__.py", line 7, in launch
cli(prog_name='pyhanko')
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/pyhanko/cli.py", line 958, in addsig_pemder
signer = pemder_config.instantiate(provided_key_passphrase=passphrase)
File "/home/chris/codetest/pdfsign/pyhanko/.venv/lib/python3.9/site-packages/pyhanko/config.py", line 369, in instantiate
raise ConfigurationError("Error while loading key material")
pyhanko.pdf_utils.config_utils.ConfigurationError: Error while loading key material
```
Add the --no-pass argument to the call
pyhanko sign addsig --field Sig1 pemder --key example2.key --cert example2.crt source/SCRIPT_unsigned.pdf out/SCRIPT_signed_cli.pdf --no-pass
Feature was added very recently. Found the answer on the discussion page
pyHanko GitHub discussions
I am using Postgres DB with Flask migration. I have changed a table name and want to upgrade it to my local & then remote.
Init give:
flask db init
Python-dotenv could not parse statement starting at line 4
Python-dotenv could not parse statement starting at line 5
Python-dotenv could not parse statement starting at line 6
Error: Directory migrations already exists and is not empty
Migrate gives:
(env) (base) Daniyal#Daniyals-MacBook-Pro Website % flask db migrate
Python-dotenv could not parse statement starting at line 4
Python-dotenv could not parse statement starting at line 5
Python-dotenv could not parse statement starting at line 6
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/script/revision.py:152: UserWarning: Revision 6ee19758a13a referenced from 6ee19758a13a -> 2e8e5e007c0a (head), empty message is not present
util.warn(
Traceback (most recent call last):
File "/Users/Daniyal/Startup/Website/env/bin/flask", line 8, in <module>
sys.exit(main())
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/flask/cli.py", line 967, in main
cli.main(args=sys.argv[1:], prog_name="python -m flask" if as_module else None)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/flask_migrate/cli.py", line 89, in migrate
_migrate(directory, message, sql, head, splice, branch_label, version_path,
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/flask_migrate/__init__.py", line 96, in wrapped
f(*args, **kwargs)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/flask_migrate/__init__.py", line 210, in migrate
command.revision(config, message, autogenerate=True, sql=sql,
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/command.py", line 214, in revision
script_directory.run_env()
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "migrations/env.py", line 96, in <module>
run_migrations_online()
File "migrations/env.py", line 90, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/runtime/migration.py", line 511, in run_migrations
for step in self._migrations_fn(heads, self):
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/command.py", line 190, in retrieve_migrations
revision_context.run_autogenerate(rev, context)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 442, in run_autogenerate
self._run_environment(rev, migration_context, True)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 454, in _run_environment
self.script_directory.get_revisions("heads")
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/script/base.py", line 227, in get_revisions
return self.revision_map.get_revisions(id_)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/script/revision.py", line 321, in get_revisions
resolved_id, branch_label = self._resolve_revision_number(id_)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/script/revision.py", line 501, in _resolve_revision_number
self._revision_map
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/util/langhelpers.py", line 230, in __get__
obj.__dict__[self.__name__] = result = self.fget(obj)
File "/Users/Daniyal/Startup/Website/env/lib/python3.8/site-packages/alembic/script/revision.py", line 156, in _revision_map
down_revision = map_[downrev]
KeyError: '6ee19758a13a'
I am not sure why upgrading such a trivial change to DB's schema has to be such a hassle.
I would really appreciate your help on how I can make such change.
I'm having issues while trying to run $pipenv --python 3.7, in this case I am running it while inside my virtual environment. As I see, this could be having trouble with another virtual environment called venvAmazon, thar I don't need or use anymore.
I noticed that, when I created my actual virtual environment, 1. it didn't created a pipfile, 2. It was created in python 2.7. (I do have 3.7 in my pc), 3. It also throws me an error if I try to run outside the venv
Traceback (most recent call last):
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 112, in get_versions
version = PythonVersion.parse(p.name)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 359, in parse
version_dict = parse_python_version(str(version))
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/utils.py", line 86, in parse_python_version
raise InvalidPythonVersion("%s is not a python version" % version_str)
pipenv.vendor.pythonfinder.exceptions.InvalidPythonVersion: venvAmazon is not a python version
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/daniel/.local/bin/pipenv", line 11, in <module>
sys.exit(cli())
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1114, in invoke
return Command.invoke(self, ctx)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 208, in cli
clear=state.clear,
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/core.py", line 574, in ensure_project
pypi_mirror=pypi_mirror,
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/core.py", line 516, in ensure_virtualenv
ensure_python(three=three, python=python)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/core.py", line 397, in ensure_python
path_to_python = find_a_system_python(python)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/core.py", line 360, in find_a_system_python
python_entry = finder.find_python_version(line)
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 113, in find_python_version
return self.system_path.find_python_version(
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 54, in system_path
ignore_unsupported=self.ignore_unsupported,
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 451, in create
ignore_unsupported=ignore_unsupported,
File "<attrs generated init 75e45f144e3d5510d54dd5fca6730b98fa0220a9>", line 38, in __init__
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 116, in __attrs_post_init__
self._setup_pyenv()
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 196, in _setup_pyenv
version_glob_path="versions/*", ignore_unsupported=self.ignore_unsupported
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 156, in create
sort_function=sort_function, version_glob_path=version_glob_path)
File "<attrs generated init 3d06d7bc6a0a13af152279598b774c410bafffe4>", line 17, in __init__
File "/home/daniel/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 114, in get_versions
entry = next(iter(version_path.find_all_python_versions()), None)
AttributeError: 'NoneType' object has no attribute 'find_all_python_versions'
After I tried pip install --upgrade --pre pipenv It gave me another error:
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/core.py", line 541, in ensure_virtualenv
pypi_mirror=pypi_mirror,
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/core.py", line 541, in ensure_virtualenv
pypi_mirror=pypi_mirror,
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/core.py", line 541, in ensure_virtualenv
pypi_mirror=pypi_mirror,
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/core.py", line 535, in ensure_virtualenv
cleanup_virtualenv(bare=True)
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/core.py", line 181, in cleanup_virtualenv
vistir.path.rmtree(project.virtualenv_location)
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/vendor/vistir/path.py", line 433, in rmtree
shutil.rmtree(directory, ignore_errors=ignore_errors, onerror=onerror)
File "/usr/lib/python2.7/shutil.py", line 270, in rmtree
rmtree(fullname, ignore_errors, onerror)
File "/usr/lib/python2.7/shutil.py", line 275, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/vendor/vistir/path.py", line 494, in handle_remove_readonly
if is_readonly_path(path):
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/vendor/vistir/path.py", line 263, in is_readonly_path
fn = fs_decode(fs_encode(fn))
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/vendor/vistir/compat.py", line 382, in fs_encode
path = _get_path(path)
File "/home/daniel/.local/lib/python2.7/site-packages/pipenv/vendor/vistir/compat.py", line 297, in _get_path
if isinstance(path, (six.string_types, bytes)):
RuntimeError: maximum recursion depth exceeded while calling a Python object
It seems this is a currently open issue with pipenv. It might worth trying what they suggest there:
pip install --upgrade --pre pipenv
I am running a lambda function related with only database. I donot need any input from event. But handler function consists of event and context. If I remove the event and context, it shows error. Is it possible to remove these things? If yes, how?
Traceback (most recent call last):
File "/home/preethi/newvenv/bin/lambda", line 187, in <module>
cli()
File "/home/preethi/newvenv/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
return self.main(*args, **kwargs)
File "/home/preethi/newvenv/local/lib/python2.7/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/home/preethi/newvenv/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/preethi/newvenv/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/preethi/newvenv/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/home/preethi/newvenv/bin/lambda", line 85, in invoke
verbose=verbose,
File "/home/preethi/newvenv/local/lib/python2.7/site-packages/aws_lambda/aws_lambda.py", line 220, in invoke
results = fn(event, None)
TypeError: handler() takes no arguments (2 given)
Thank You.
I'm using Celery with Redis to run some background tasks, but each time a task is called, it creates a new connection to Redis. I'm on Heroku and my Redis to Go plan allows for 10 connections. I'm quickly hitting that limit and getting a "max number of clients reached" error.
How can I ensure that Celery queues the tasks on a single connection rather than opening a new one each time?
EDIT - including the full traceback
File "/app/.heroku/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/object_wrapper.py", line 166, in __call__
self._nr_instance, args, kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/hooks/framework_django.py", line 447, in wrapper
return wrapped(*args, **kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 77, in wrapped_view
return view_func(*args, **kwargs)
File "/app/feedback/views.py", line 264, in zencoder_webhook_handler
tasks.process_zencoder_notification.delay(webhook)
File "/app/.heroku/venv/lib/python2.7/site-packages/celery/app/task.py", line 343, in delay
return self.apply_async(args, kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/celery/app/task.py", line 458, in apply_async
with app.producer_or_acquire(producer) as P:
File "/usr/local/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/app/.heroku/venv/lib/python2.7/site-packages/celery/app/base.py", line 247, in producer_or_acquire
with self.amqp.producer_pool.acquire(block=True) as producer:
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/connection.py", line 705, in acquire
R = self.prepare(R)
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/pools.py", line 54, in prepare
p = p()
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/pools.py", line 45, in <lambda>
return lambda: self.create_producer()
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/pools.py", line 42, in create_producer
return self.Producer(self._acquire_connection())
File "/app/.heroku/venv/lib/python2.7/site-packages/celery/app/amqp.py", line 160, in __init__
super(TaskProducer, self).__init__(channel, exchange, *args, **kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/messaging.py", line 83, in __init__
self.revive(self.channel)
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/messaging.py", line 174, in revive
channel = self.channel = maybe_channel(channel)
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/connection.py", line 879, in maybe_channel
return channel.default_channel
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/connection.py", line 617, in default_channel
self.connection
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/connection.py", line 610, in connection
self._connection = self._establish_connection()
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/connection.py", line 569, in _establish_connection
conn = self.transport.establish_connection()
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 722, in establish_connection
self._avail_channels.append(self.create_channel(self))
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 705, in create_channel
channel = self.Channel(connection)
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 271, in __init__
self.client.info()
File "/app/.heroku/venv/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/object_wrapper.py", line 166, in __call__
self._nr_instance, args, kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/function_trace.py", line 81, in literal_wrapper
return wrapped(*args, **kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/redis/client.py", line 344, in info
return self.execute_command('INFO')
File "/app/.heroku/venv/lib/python2.7/site-packages/kombu/transport/redis.py", line 536, in execute_command
conn.send_command(*args)
File "/app/.heroku/venv/lib/python2.7/site-packages/redis/connection.py", line 273, in send_command
self.send_packed_command(self.pack_command(*args))
File "/app/.heroku/venv/lib/python2.7/site-packages/redis/connection.py", line 256, in send_packed_command
self.connect()
File "/app/.heroku/venv/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/object_wrapper.py", line 166, in __call__
self._nr_instance, args, kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/newrelic-1.4.0.137/newrelic/api/function_trace.py", line 81, in literal_wrapper
return wrapped(*args, **kwargs)
File "/app/.heroku/venv/lib/python2.7/site-packages/redis/connection.py", line 207, in connect
self.on_connect()
File "/app/.heroku/venv/lib/python2.7/site-packages/redis/connection.py", line 233, in on_connect
if self.read_response() != 'OK':
File "/app/.heroku/venv/lib/python2.7/site-packages/redis/connection.py", line 283, in read_response
raise response
ResponseError: max number of clients reached
I ran into the same problem on Heroku with CloudAMQP. I do not know why, but I had no luck when assigning low integers to the BROKER_POOL_LIMIT setting.
Ultimately, I found that by setting BROKER_POOL_LIMIT=None or BROKER_POOL_LIMIT=0 my issue was mitigated. According to the Celery docs, this disables the connection pool. So far, this has not been a noticeable issue for me, however I'm not sure if it might be for you.
Link to relevant info: http://celery.readthedocs.org/en/latest/configuration.html#broker-pool-limit
I wish I was using Redis, because there is a specific option to limit the number of connections: CELERY_REDIS_MAX_CONNECTIONS.
http://docs.celeryproject.org/en/3.0/configuration.html#celery-redis-max-connections (for 3.0)
http://docs.celeryproject.org/en/latest/configuration.html#celery-redis-max-connections (for 3.1)
http://docs.celeryproject.org/en/master/configuration.html#celery-redis-max-connections (for dev)
The MongoDB has a similar backend setting.
Given these backend settings, I have no idea what BROKER_POOL_LIMIT actually does. Hopefully CELERY_REDIS_MAX_CONNECTIONS solves your problem.
I'm one of those folks using CloudAMQP, and the AMQP backend does not have its own connection limit parameter.
Try those settings :
CELERY_IGNORE_RESULT = True
CELERY_STORE_ERRORS_EVEN_IF_IGNORED = True
I had a similar issue involving number of connections and Celery. It wasn't on Heroku, and it was Mongo and not Redis though.
I initiated the connection outside of the task function definition at the task module level. At least for Mongo this allowed the tasks to share the connection.
Hope that helps.
https://github.com/instituteofdesign/wander/blob/master/wander/tasks.py
mongoengine.connect('stored_messages')
#celery.task(default_retry_delay = 61)
def pull(settings, google_settings, user, folder, messageid):
'''
Pulls a message from zimbra and stores it in Mongo
'''
try:
imap = imap_connect(settings, user)
imap.select(folder, True)
.......