RabbitMQ crashes when I tried to send message - python

I am very new to RabbitMQ. I use Python to send a message between the Sender.py and Receiver.py (from the Rabbit tutorial). It works well if I run the Sender.py and Receiver.py independtly. However, I want to make the third file, namely, RunToGether.py with very very simple code:
import Sender, Receiver
Sender.send()
Receiver.receive()
and it crashes:
File "/home/my_direct/Receive.py", line 50, in PV_simulator
channel.start_consuming()
File "/home/my_direct/venv/lib/python3.8/site-packages/pika/adapters/blocking_connection.py", line 1865, in start_consuming
self._process_data_events(time_limit=None)
File "/home/rmy_direct/venv/lib/python3.8/site-packages/pika/adapters/blocking_connection.py", line 2026, in _process_data_events
self.connection.process_data_events(time_limit=time_limit)
File "/home/my_direct/lib/python3.8/site-packages/pika/adapters/blocking_connection.py", line 824, in process_data_events
self._flush_output(common_terminator)
File "/home/my_direct/venv/lib/python3.8/site-packages/pika/adapters/blocking_connection.py", line 514, in _flush_output
self._impl.ioloop.poll()
File "/home/my_direct/venv/lib/python3.8/site-packages/pika/adapters/select_connection.py", line 579, in poll
self._poller.poll()
File "/home/my_direct/venv/lib/python3.8/site-packages/pika/adapters/select_connection.py", line 1184, in poll
events = self._poll.poll(self._get_max_wait())
KeyboardInterrupt
Please help!

Related

Python/Pyrogram on Mac Server Suddenly Turning up Error 61 Connection Refused

I have a Pyrogram app that has been running for ages with no problems from a server running MacOS. A few days ago, the code suddenly hung up and refuses to work. As soon as I attempt to start the Pyrogram Client(), I immediately get "Err 61 Connection Refused"/"network issues" errors. I have confirmed that my Telegram API account is totally fine. The next assumption was that it was an IP ban issue, but I can't even get the Client to open a socket to an http proxy. Pyrogram documentation has nothing about an Error 61, and now I'm thinking that the problem isn't with Pyrogram but rather with a core Python library (corrupted?).
I have seen references on Stackoverflow to folks running into analogous problems with other Python api/web applications but not randomly after a year of normal operation. Some of them were able to resolve the issue by switching references to "localhost" to "127.0.0.1", but I have no idea where that would be located within the Pyrogram library, nor why that would suddenly be a problem out of the blue with no code or server changes.
Help! ;) Thanks!!
Update:
I forgot to post code:
import os
import logging
from typing import List, Tuple, Optional
import re
import math
from datetime import datetime
import os.path
import asyncio
import pyrogram
import yaml
import time
import base64
import sys
import mysql.connector
import subprocess
client = pyrogram.Client(
"media_downloader",
api_id,
api_hash,
)
... and there it ends.
INFO:pyrogram.connection.connection:Connecting...
WARNING:pyrogram.connection.connection:Unable to connect due to network issues: [Errno 61] Connection refused
INFO:pyrogram.connection.connection:Connecting...
WARNING:pyrogram.connection.connection:Unable to connect due to network issues: [Errno 61] Connection refused
INFO:pyrogram.connection.connection:Connecting...
^CTraceback (most recent call last):
File "/Users/admin/home/telegram/get_message_media_v2.py", line 99, in <module>
with client:
File "/usr/local/lib/python3.9/site-packages/pyrogram/client.py", line 251, in __enter__
return self.start()
File "/usr/local/lib/python3.9/site-packages/pyrogram/sync.py", line 66, in async_to_sync_wrap
return loop.run_until_complete(coroutine)
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
self.run_forever()
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
self._run_once()
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 1854, in _run_once
event_list = self._selector.select(timeout)
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/selectors.py", line 562, in select
kev_list = self._selector.control(None, max_ev, timeout)
KeyboardInterrupt
After attempting with a proxy:
INFO:pyrogram.connection.transport.tcp.tcp:Using proxy [PROXY REDACTED]
INFO:pyrogram.connection.connection:Connecting...
WARNING:pyrogram.connection.connection:Unable to connect due to network issues: Socket error: timed out
INFO:pyrogram.connection.transport.tcp.tcp:Using proxy [PROXY REDACTED]
INFO:pyrogram.connection.connection:Connecting...
^X^CTraceback (most recent call last):
File "/Users/admin/home/telegram/get_message_media_v2.py", line 106, in <module>
with client:
File "/usr/local/lib/python3.9/site-packages/pyrogram/client.py", line 251, in __enter__
return self.start()
File "/usr/local/lib/python3.9/site-packages/pyrogram/sync.py", line 66, in async_to_sync_wrap
return loop.run_until_complete(coroutine)
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
self.run_forever()
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
self._run_once()
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
handle._run()
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.9/site-packages/pyrogram/methods/utilities/start.py", line 52, in start
is_authorized = await self.connect()
File "/usr/local/lib/python3.9/site-packages/pyrogram/methods/auth/connect.py", line 46, in connect
await self.session.start()
File "/usr/local/lib/python3.9/site-packages/pyrogram/session/session.py", line 105, in start
await self.connection.connect()
File "/usr/local/lib/python3.9/site-packages/pyrogram/connection/connection.py", line 57, in connect
await self.protocol.connect(self.address)
File "/usr/local/lib/python3.9/site-packages/pyrogram/connection/transport/tcp/tcp_abridged_o.py", line 40, in connect
await super().connect(address)
File "/usr/local/lib/python3.9/site-packages/pyrogram/connection/transport/tcp/tcp.py", line 82, in connect
self.socket.connect(address)
File "/usr/local/lib/python3.9/site-packages/socks.py", line 47, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/socks.py", line 809, in connect
negotiate(self, dest_addr, dest_port)
File "/usr/local/lib/python3.9/site-packages/socks.py", line 443, in _negotiate_SOCKS5
self.proxy_peername, self.proxy_sockname = self._SOCKS5_request(
File "/usr/local/lib/python3.9/site-packages/socks.py", line 470, in _SOCKS5_request
chosen_auth = self._readall(reader, 2)
File "/usr/local/lib/python3.9/site-packages/socks.py", line 276, in _readall
d = file.read(count - len(data))
File "/usr/local/Cellar/python#3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
KeyboardInterrupt
If anyone else runs into this problem, I was able to eventually resolve this by using pip3 to reinstall/upgrade Pyrogram. I still don't know the exact underlying cause, and at this point, I can only assume that either my copy of Pyrogram got corrupted somehow, or else there were changes to Telegram that broke a slightly older version of Pyrogram (I had 1.7.1 installed previously and pip upgraded it to 2.0.35).
pip3 install pyrogram --upgrade

celery task raise an error while redis backend connection socket timeout

I am using celery to run task one by one with redis broker, but when i run 2 task then after completing the first redis given an timeout socket error for second task so that second task would be failed.
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/celery/result.py", line 194, in get
on_message=on_message,
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/celery/backends/async.py", line 189, in wait_for_pending
for _ in self._wait_for_pending(result, **kwargs):
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/celery/backends/async.py", line 256, in _wait_for_pending
on_interval=on_interval):
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/celery/backends/async.py", line 57, in drain_events_until
yield self.wait_for(p, wait, timeout=1)
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/celery/backends/async.py", line 66, in wait_for
wait(timeout=timeout)
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/celery/backends/redis.py", line 69, in drain_events
m = self._pubsub.get_message(timeout=timeout)
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/client.py", line 2513, in get_message
response = self.parse_response(block=False, timeout=timeout)
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/client.py", line 2430, in parse_response
return self._execute(connection, connection.read_response)
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/client.py", line 2408, in _execute
return command(*args)
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/connection.py", line 624, in read_response
response = self._parser.read_response()
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/connection.py", line 284, in read_response
response = self._buffer.readline()
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/connection.py", line 216, in readline
self._read_from_socket()
File "/home/ubuntu/.virtualenvs/aide_venv/local/lib/python2.7/site-packages/redis/connection.py", line 187, in _read_from_socket
raise TimeoutError("Timeout reading from socket")
TimeoutError: Timeout reading from socket
I am running celery by using this command:
celery -A flask_application.celery worker --loglevel=info --max-tasks-per-child=1 --concurrency=1
I am calling celery task by using: .delay() function
celery_response = run_algo.run_pipeline.delay(request.get_json())
Getting output by using: .get() function
output_file_path = celery_response.get()
There's a warning in the docs of AsyncResult.get saying that calling it it within an async task can cause a deadlock, which may be what's happening here, though it's hard to tell without more context of where these things are being called.

Password input may be echoed when run Python Fabric command in crontab

I encountered issue when trying to run my program through crontab on Mac OS. My program works fine when run it independently. Normally, I never set env.password for remote system password. For instead, I set env.key_filename. It works fine if i don't have to run sudo command. So ideally, it shouldn't prompt any password typing.
By referring to https://github.com/fabric/fabric/issues/1230, i also tried to set environment variables to pass the password. Yet, i still get the same error. What did i miss? Anyone can help pls?
Thx
Error msg:
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/getpass.py:83: GetPassWarning: Can not control echo on the terminal.
passwd = fallback_getpass(prompt, stream)
Warning: Password input may be echoed.
[xxx.xxx.xxx.xxx] Login password for 'xxx': [xxx.xxx.xxx.xxx] Executing task 'System.Monitor.free_mem'
[+] Start checking system total/free memory in MB:
[xxx.xxx.xxx.xxx] run: free -m
Traceback (most recent call last):
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/main.py", line 757, in main
*args, **kwargs
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/tasks.py", line 386, in execute
multiprocessing
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/tasks.py", line 276, in _execute
return task.run(*args, **kwargs)
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/tasks.py", line 173, in run
return self.wrapped(*args, **kwargs)
File "/Users/thomas.pan/Python-ninja/playwith/DevOps/System/Monitor.py", line 69, in free_mem
run("free -m")
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/network.py", line 687, in host_prompting_wrapper
return func(*args, **kwargs)
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/operations.py", line 1090, in run
shell_escape=shell_escape, capture_buffer_size=capture_buffer_size,
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/operations.py", line 930, in _run_command
channel=default_channel(), command=wrapped_command, pty=pty,
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/state.py", line 424, in default_channel
chan = _open_session()
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/state.py", line 416, in _open_session
return connections[env.host_string].get_transport().open_session()
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/network.py", line 159, in __getitem__
self.connect(key)
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/network.py", line 151, in connect
user, host, port, cache=self, seek_gateway=seek_gateway)
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/network.py", line 569, in connect
password = prompt_for_password(text)
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/network.py", line 652, in prompt_for_password
new_password = _password_prompt(password_prompt, stream)
File "/usr/local/var/pyenv/versions/2.7.10/lib/python2.7/site-packages/fabric/network.py", line 624, in _password_prompt
return getpass.getpass(prompt.encode('ascii', 'ignore'), stream)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/getpass.py", line 83, in unix_getpass
passwd = fallback_getpass(prompt, stream)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/getpass.py", line 118, in fallback_getpass
return _raw_input(prompt, stream)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/getpass.py", line 135, in _raw_input
raise EOFError
EOFError
Issue closed by switch to launchd with LaunchControl tool. It's not Fabric issue.
In case it helps anyone - if you're running a command that is logging into an instance, this GetPassWarning/OEFError issue can happen with a cron job, since the shell session does not know how to log into that instance.
To fix this, you may need to give ssh context to crond. ssh-cron can do this, for example, since it looks like there is difficulty getting that all set up in crontab.

How to debug "pika.exceptions.AuthenticationError: EXTERNAL" error when establishing TLS connection to RabbitMQ?

I have a RabbitMQ 3.6.1 server on Ubuntu 14.04 running properly. I tried to configure an SSL listener according to official documentation. No problems during the startup.
However when trying to establish a connection, I get the following error on Python/pika side (full transcript below):
pika.exceptions.AuthenticationError: EXTERNAL
What does EXTERNAL mean here? How to debug / get further details of the error?
Course of actions (to test I used a Vagrant box and a local connection):
RabbitMQ starts SSL Listener on port 5671 (per /var/log/rabbitmq/rabbit#rabbitmq-server.log):
started SSL Listener on [::]:5671
I execute the pika.BlockingConnection on the client side.
On the server side I can see an incoming connection:
=INFO REPORT==== 17-Apr-2016::17:07:15 ===
accepting AMQP connection <0.2788.0> (127.0.0.1:48404 -> 127.0.0.1:5671)
Client fails with:
pika.exceptions.AuthenticationError: EXTERNAL
Server timeouts:
=ERROR REPORT==== 17-Apr-2016::17:07:25 ===
closing AMQP connection <0.2788.0> (127.0.0.1:48404 -> 127.0.0.1:5671):
{handshake_timeout,frame_header}
Full transcript of the client side:
>>> import pika, ssl
>>> from pika.credentials import ExternalCredentials
>>> ssl_options = ({"ca_certs": "/etc/rabbitmq/certs/testca/cacert.pem",
... "certfile": "/etc/rabbitmq/certs/client/cert.pem",
... "keyfile": "/etc/rabbitmq/certs/client/key.pem",
... "cert_reqs": ssl.CERT_REQUIRED,
... "server_side": False})
>>> host = "localhost"
>>> connection = pika.BlockingConnection(
... pika.ConnectionParameters(
... host, 5671, credentials=ExternalCredentials(),
... ssl=True, ssl_options=ssl_options))
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 339, in __init__
self._process_io_for_connection_setup()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 374, in _process_io_for_connection_setup
self._open_error_result.is_ready)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 410, in _flush_output
self._impl.ioloop.poll()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/select_connection.py", line 602, in poll
self._process_fd_events(fd_event_map, write_only)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/select_connection.py", line 443, in _process_fd_events
handler(fileno, events, write_only=write_only)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 364, in _handle_events
self._handle_read()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/base_connection.py", line 415, in _handle_read
self._on_data_available(data)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1347, in _on_data_available
self._process_frame(frame_value)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1414, in _process_frame
if self._process_callbacks(frame_value):
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1384, in _process_callbacks
frame_value) # Args
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 60, in wrapper
return function(*tuple(args), **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 92, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pika/callback.py", line 236, in process
callback(*args, **keywords)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1298, in _on_connection_start
self._send_connection_start_ok(*self._get_credentials(method_frame))
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py", line 1077, in _get_credentials
raise exceptions.AuthenticationError(self.params.credentials.TYPE)
pika.exceptions.AuthenticationError: EXTERNAL
>>>
The Python / pika code in the question is correct.
The error:
pika.exceptions.AuthenticationError: EXTERNAL
is reported when client certificate authorisation is not enabled on the RabbitMQ server side. The word EXTERNAL in the error refers to the authentication mechanism as described here.
To enable:
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl

python xmlrpc timeout error

I am using xmlrpc to contact a local server. On the client side, Sometimes the following socket timeout error and happens and its not a consistent error.
Why is it happening? What could be the reason for socket timeout?
<class 'socket.timeout'>: timed out
args = ('timed out',)
errno = None
filename = None
message = 'timed out'
strerror = None
Traceback on the server side is as follows
Exception happened during processing of request from ('127.0.0.1', 34855)
Traceback (most recent call last):
File "/usr/lib/python2.4/SocketServer.py", line 222, in handle_request
self.process_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 241, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.4/SocketServer.py", line 521, in __init__
self.handle()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 314, in handle
self.handle_one_request()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 308, in handle_one_request
method()
File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 441, in do_POST
self.send_response(200)
File "/usr/lib/python2.4/BaseHTTPServer.py", line 367, in send_response
self.send_header('Server', self.version_string())
File "/usr/lib/python2.4/BaseHTTPServer.py", line 373, in send_header
self.wfile.write("%s: %s\r\n" % (keyword, value))
File "/usr/lib/python2.4/socket.py", line 256, in write
self.flush()
File "/usr/lib/python2.4/socket.py", line 243, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
I killed the server and restarted it. Its working fine now.
What could be the reason?
My machine's RAM went full yesterday night by a process and came back to normal today morning.
Will this error be because of some swapping of processes?
Looks like the client socket it timing out waiting for the server to respond. Is it possible that your server might take a lot time to respond some times? Also, if the server is causing the machine to go into swap, that would slow it down making a timeout possible.
If I remember right, socket timeout is not set in xmlrpc in python. Are you doing socket.setdefaulttimeout somewhere in your code?
If it is expected that your server will take time once in a while, then you could set a higher timeout value using above.
HTH

Categories

Resources