airflow task is unable to connect to remote oracle database but i am able to connect to the same remote oracle database using the same code from python shell.
i checked running environment is same for both the shell and airflow.
i am attaching the log error which i am getting.
[2018-07-18 13:12:11,037] {models.py:1428} INFO - Executing <Task(AadharNumberValidation): validating_data> on 2018-07-18 07:42:05.573491
[2018-07-18 13:12:11,037] {base_task_runner.py:115} INFO - Running: ['bash', '-c', 'airflow run data_validation validating_data 2018-07-18T07:42:05.573491 --job_id 206 --raw -sd /Users/b0204890/Desktop/python/airflow_home/dags/data_validaton.py']
[2018-07-18 13:12:11,531] {base_task_runner.py:98} INFO - Subtask: [2018-07-18 13:12:11,531] {__init__.py:45} INFO - Using executor SequentialExecutor
[2018-07-18 13:12:11,588] {base_task_runner.py:98} INFO - Subtask: [2018-07-18 13:12:11,588] {models.py:189} INFO - Filling up the DagBag from /Users/b0204890/Desktop/python/airflow_home/dags/data_validaton.py
[2018-07-18 13:12:11,661] {cli.py:374} INFO - Running on host LTB0204890-Mac.local
[2018-07-18 13:12:11,669] {base_task_runner.py:98} INFO - Subtask: [2018-07-18 13:12:11,669] {validation_operators.py:37} INFO - operator_param yadav: Script one validation
[2018-07-18 13:12:11,678] {models.py:1595} ERROR - DPI-1047: 64-bit Oracle Client library cannot be loaded: "dlopen(libclntsh.dylib, 1): image not found". See https://oracle.github.io/odpi/doc/installation.html#macos for help
Traceback (most recent call last):
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/models.py", line 1493, in _run_raw_task
result = task_copy.execute(context=context)
File "/Users/b0204890/Desktop/python//airflow_home/plugins/validation_operators.py", line 38, in execute
cursor = create_connection(user="USERNAME",port="PORT",host="HOST",pwd="password",sid="SID")
File "/Users/b0204890/Desktop/python/airflow_home/utility/validation.py", line 30, in create_connection
connection = cx_Oracle.connect(user, pwd, service)
cx_Oracle.DatabaseError: DPI-1047: 64-bit Oracle Client library cannot be loaded: "dlopen(libclntsh.dylib, 1): image not found". See https://oracle.github.io/odpi/doc/installation.html#macos for help
[2018-07-18 13:12:11,681] {models.py:1616} INFO - Marking task as UP_FOR_RETRY
[2018-07-18 13:12:11,682] {base_task_runner.py:98} INFO - Subtask: [2018-07-18 13:12:11,682] {configuration.py:206} WARNING - section/key [smtp/smtp_user] not found in config
[2018-07-18 13:12:11,684] {models.py:1628} ERROR - Failed to send email to: ['tushar.smartcorp#gmail.com']
[2018-07-18 13:12:11,684] {models.py:1629} ERROR - [Errno 61] Connection refused
Traceback (most recent call last):
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/models.py", line 1493, in _run_raw_task
result = task_copy.execute(context=context)
File "/Users/b0204890/Desktop/python//airflow_home/plugins/validation_operators.py", line 38, in execute
cursor = create_connection(user="USERNAME",port="PORT",host="HOST",pwd="PASSWORD",sid="SID")
File "/Users/b0204890/Desktop/python/airflow_home/utility/validation.py", line 30, in create_connection
connection = cx_Oracle.connect(user, pwd, service)
cx_Oracle.DatabaseError: DPI-1047: 64-bit Oracle Client library cannot be loaded: "dlopen(libclntsh.dylib, 1): image not found". See https://oracle.github.io/odpi/doc/installation.html#macos for help
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/models.py", line 1618, in handle_failure
self.email_alert(error, is_retry=True)
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/models.py", line 1779, in email_alert
send_email(task.email, title, body)
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/utils/email.py", line 44, in send_email
return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/utils/email.py", line 87, in send_email_smtp
send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
File "/Users/b0204890/venv/python3/lib/python3.6/site-packages/airflow/utils/email.py", line 107, in send_MIME_email
s = smtplib.SMTP_SSL(SMTP_HOST, SMTP_PORT) if SMTP_SSL else smtplib.SMTP(SMTP_HOST, SMTP_PORT)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/smtplib.py", line 251, in __init__
(code, msg) = self.connect(host, port)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/smtplib.py", line 336, in connect
self.sock = self._get_socket(host, port, self.timeout)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/smtplib.py", line 307, in _get_socket
self.source_address)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 724, in create_connection
raise err
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 713, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
Without following all of the instructions the connector will not work. Simply doing an installation for the Python package cx-Oracle is not enough. In this case it could also be that the venv of Airflow cannot access all neccessary files or that there is some rights issue.
As stated in the message, you need to follow all steps seen in: https://oracle.github.io/odpi/doc/installation.html#macos
Related
I'm using Airflow 2.5.1 and setting up Airflow scheduler (Using LocalExecutor) + webserver on Deb9 Instance.
MySQL DB is on another instance and I checked using PING and airflow db check that the connection to the MySQL server is successful. I had even run airflow db init from this instance and it was able to create all the tables successfully.
When I start the scheduler using airflow scheduler command, getting the following error:
airflow-scheduler.err log below:
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')
(Background on this error at: https://sqlalche.me/e/14/e3q8)
[2023-02-09 01:12:34 +0530] [15451] [ERROR] Connection in use: ('0.0.0.0', 8793)
[2023-02-09 01:12:34 +0530] [15451] [ERROR] Retrying in 1 second.
[2023-02-09 01:12:35 +0530] [15451] [ERROR] Connection in use: ('0.0.0.0', 8793)
[2023-02-09 01:12:35 +0530] [15451] [ERROR] Retrying in 1 second.
[2023-02-09 01:12:36 +0530] [15451] [ERROR] Connection in use: ('0.0.0.0', 8793)
[2023-02-09 01:12:36 +0530] [15451] [ERROR] Retrying in 1 second.
[2023-02-09 01:12:37 +0530] [15451] [ERROR] Can't connect to ('0.0.0.0', 8793)
airflow-scheduler.log below:
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')
(Background on this error at: https://sqlalche.me/e/14/e3q8)
2023-02-09 01:12:33,040 INFO - Shutting down LocalExecutor; waiting for running tasks to finish. Signal again if you don't want to wait.
2023-02-09 01:12:34,066 INFO - Sending Signals.SIGTERM to group 15492. PIDs of all processes in the group: [15492]
2023-02-09 01:12:34,066 INFO - Sending the signal Signals.SIGTERM to group 15492
2023-02-09 01:12:34,158 INFO - Process psutil.Process(pid=15492, status='terminated', exitcode=0, started='01:12:32') (15492) terminated with exit code 0
2023-02-09 01:12:34,159 INFO - Exited execute loop
Any idea why is this happening?
[Update]
Connection in use: ('0.0.0.0', 8793) was because of some processes running from previous run. Have killed those processes. But I'm still getting MySQL server has gone away error.
airflow-scheduler.err log:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/airflow_venv/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/__main__.py", line 39, in main
args.func(args)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 52, in command
return func(*args, **kwargs)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/utils/cli.py", line 108, in wrapper
return f(*args, **kwargs)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py", line 68, in scheduler
_run_scheduler_job(args=args)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py", line 43, in _run_scheduler_job
job.run()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/jobs/base_job.py", line 258, in run
self._execute()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 759, in _execute
self._run_scheduler_loop()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 840, in _run_scheduler_loop
self.adopt_or_reset_orphaned_tasks()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 75, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 36, in create_session
session.commit()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1451, in commit
self._transaction.commit(_to_root=self.future)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 836, in commit
trans.commit()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2459, in commit
self._do_commit()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2649, in _do_commit
self._connection_commit_impl()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2620, in _connection_commit_impl
self.connection._commit_impl()
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1091, in _commit_impl
self._handle_dbapi_exception(e, None, None, None, None)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2125, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1089, in _commit_impl
self.engine.dialect.do_commit(self.connection)
File "/home/airflow/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 686, in do_commit
dbapi_connection.commit()
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')
(Background on this error at: https://sqlalche.me/e/14/e3q8)
When running the LocalExecutor, the scheduler also starts a process to serve log files, by default on port 8793. The error tells you you've already got something running on port 8793, therefore it can't start and returns an error. You've probably already got a scheduler running.
The port is configurable by AIRFLOW__LOGGING__WORKER_LOG_SERVER_PORT. For more information, see https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#worker-log-server-port.
I tried to deploy my Telethon bot on PythonAnywhere but got following exception:
Attempt 1 at connecting failed: ConnectionRefusedError: [Errno 111] Connect call failed ('149.154.167.51', 443)
Attempt 2 at connecting failed: ConnectionRefusedError: [Errno 111] Connect call failed ('149.154.167.51', 443)
Attempt 3 at connecting failed: ConnectionRefusedError: [Errno 111] Connect call failed ('149.154.167.51', 443)
Attempt 4 at connecting failed: ConnectionRefusedError: [Errno 111] Connect call failed ('149.154.167.51', 443)
Attempt 5 at connecting failed: ConnectionRefusedError: [Errno 111] Connect call failed ('149.154.167.51', 443)
Attempt 6 at connecting failed: ConnectionRefusedError: [Errno 111] Connect call failed ('149.154.167.51', 443)
Traceback (most recent call last):
File "/home/hellbowe/popa3/main.py", line 51, in <module>
clientg.start(phone_tg, password_tg)
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/client/auth.py", line 133, in start
else self.loop.run_until_complete(coro)
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/client/auth.py", line 140, in _start
await self.connect()
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/client/telegrambaseclient.py", line 525, in connect
if not await self._sender.connect(self._connection(
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/network/mtprotosender.py", line 127, in connect
await self._connect()
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/network/mtprotosender.py", line 253, in _connect
raise ConnectionError('Connection to Telegram failed {} time(s)'.format(self._retries))
ConnectionError: Connection to Telegram failed 5 time(s)
It works fine on my device.
I tried to add proxy to TelegramClient() thing but got this:
Traceback (most recent call last):
File "/home/hellbowe/popa3/main.py", line 51, in <module>
clientg = TelegramClient('popa3', api_id_tg, api_hash_tg, proxy = ('http', '***', 80))
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/client/telegrambaseclient.py", line 292, in __init__
session.set_dc(
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/sessions/sqlite.py", line 168, in set_dc
self._update_session_table()
File "/home/hellbowe/.local/lib/python3.9/site-packages/telethon/sessions/sqlite.py", line 194, in _update_session_table
c.execute('delete from sessions')
sqlite3.OperationalError: database is locked
You have some other process that has locked your sqlite database (or the database is corrupted) Try deleting the database file or moving it somewhere else so that it can be recreated.
As far as I know, you need to log in into Telegram to use telethon. In your machine you can do it with command line interface, but in hosting you can't. I solved this problem when rent a virtual server, where I can log in Telegram. Then I just run python script with some options in order to always working
I'm building a Airflow server from the official docker image:
curl -LfO "https://airflow.apache.org/docs/apache-airflow/2.3.0/docker-compose.yaml"
It creates an standard user (airflow) with standard password (airflow). I have been trying to connect to my Airflow DAG with a PostgresHook, but it keeps returning this:
[2022-05-18, 01:29:32 UTC] {base.py:68} INFO - Using connection ID 'postgres_db' for task execution.
[2022-05-18, 01:29:32 UTC] {taskinstance.py:1889} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/python.py", line 171, in execute
return_value = self.execute_callable()
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/python.py", line 189, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/dags/etl_hover.py", line 35, in insert_data_to_table
pg_conn = hook.get_conn()
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/postgres/hooks/postgres.py", line 113, in get_conn
self.conn = psycopg2.connect(**conn_args)
File "/home/airflow/.local/lib/python3.7/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: connection to server at "host.docker.internal" (192.168.65.2), port 5432 failed: FATAL: password authentication failed for user "***"
And, this is the connection:
As you can see, It is pretty much a standard connection (I have created succesfully an entry point with the database api_db, by the way. But It keeps telling me something about a wrong password. Any ideas?
I tried to get all the active/scheduled/reserved tasks in redis:
from celery.task.control import inspect
inspect_obj = inspect()
inspect_obj.active()
inspect_obj.scheduled()
inspect_obj.reserved()
But was greeted with a list of errors as follows:
My virtual environment ==> HubblerAPI.
Iam using this from the ec2 console
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 81, in active
return self._request('dump_active', safe=safe)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 71, in _request
timeout=self.timeout, reply=True,
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 316, in broadcast
limit, callback, channel=channel,
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/pidbox.py", line 283, in _broadcast
chan = channel or self.connection.default_channel
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 771, in default_channel
self.connection
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 756, in connection
self._connection = self._establish_connection()
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 711, in _establish_connection
conn = self.transport.establish_connection()
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/transport/pyamqp.py", line 116, in establish_connection
conn = self.Connection(**opts)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
**OSError: [Errno 111] Connection refused**
My celery config file is as follows:
BROKER_TRANSPORT = 'redis'
BROKER_TRANSPORT_OPTIONS = {
'queue_name_prefix': 'dev-',
'wait_time_seconds': 10,
'polling_interval': 30,
# The polling interval decides the number of seconds to sleep
between unsuccessful polls
'visibility_timeout': 3600 * 5,
# If a task is not acknowledged within the visibility_timeout, the
task will be redelivered to another worker and executed.
}
CELERY_MESSAGES_DB = 6
BROKER_URL = "redis://%s:%s/%s" % (AWS_REDIS_ENDPOINT, AWS_REDIS_PORT,
CELERY_MESSAGES_DB)
What am i doing wrong here as the error log suggests that its not using the redis broker.
Looks like your python code doesn't recognize your configs since it is attempting to use RabbitMQ's ampq protocol instead of the configured broker.
I suggest the following
https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/redis.html
Your configs look similar to Django configs for Celery yet it doesn't seem you are using Celery with Django.
https://docs.celeryq.dev/en/latest/django/first-steps-with-django.html
The issue is using "BROKER_URL" instead of "CELERY_BROKER_URL" in settings.py. Celery wasn't finding the URL and was defaulting to the rabbitmq port instead of the redis port.
I am completely new to MQs and pika.
When I am trying to execute this code(server code) I am getting errors:
(I am getting same error for receiver code also.)
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host="localhost:5672"))
channel = connection.channel()
channel.queue_declare(queue='hello')
channel.basic_publish(exchange='',
routing_key='hello',
body='Hello World!')
print(" [x] Sent 'Hello World!'")
connection.close()
Error :
Traceback (most recent call last):
File "D:\Workspace\Luna_WS\MQ\RabbitMQ\Sample\Src\Sample.py", line 10, in <module>
host="localhost:5672"))
File "C:\Python34\lib\site-packages\pika\adapters\blocking_connection.py", line 339, in __init__
self._process_io_for_connection_setup()
File "C:\Python34\lib\site-packages\pika\adapters\blocking_connection.py", line 374, in _process_io_for_connection_setup
self._open_error_result.is_ready)
File "C:\Python34\lib\site-packages\pika\adapters\blocking_connection.py", line 395, in _flush_output
raise exceptions.ConnectionClosed()
pika.exceptions.ConnectionClosed
I had installed: (using windows 7, 32 bit)
pikka package - pika-0.10.0-py2.py3-none-any.whl
Erlang - esl-erlang_18.2-1-windows_i386
RabbitMq server - rabbitmq-server-3.6.0
Try splitting the host into separate host and port connection parameters:
connection = pika.BlockingConnection(
pika.ConnectionParameters(host="localhost", port=5672))