I am currently trying to use CosmosDB as state storage but encountering some issues when the bot attempts to save the state in CosmosDB.
Below is the code that I am using CosmosDB as the middleware
LOOP = asyncio.get_event_loop()
APP = Flask(__name__)
APP.config.update(VaultConfig.get_config())
APP.wsgi_app = BotTelemetryMiddleware(APP.wsgi_app)
PORT = APP.config["Settings"]["Port"]
INSTRUMENTATION_KEY = APP.config['Settings']["appinsight_instrumentation_key"]
TELEMETRY_CLIENT = ApplicationInsightsTelemetryClient(INSTRUMENTATION_KEY)
SETTINGS = BotFrameworkAdapterSettings(
APP.config["Settings"]["AppId"], APP.config["Settings"]["AppPassword"]
)
cosmos_config = APP.config['COSMOS']
# Create CosmosStorage and ConversationState
cosmos = CosmosDbStorage(CosmosDbConfig(endpoint=cosmos_config['endpoint'], masterkey=cosmos_config['masterKey'],
database=cosmos_config['database'], container=cosmos_config['container'], partition_key='/id'))
# Create MemoryStorage, UserState and ConversationState
USER_STATE = UserState(cosmos)
CONVERSATION_STATE = ConversationState(cosmos)
ADAPTER = BotFrameworkAdapter(SETTINGS)
ADAPTER.on_turn_error = on_error
The error that I am receiving:
Traceback (most recent call last):
File "/bot/venv/lib/python3.7/site-packages/botbuilder/core/bot_adapter.py", line 95, in run_pipeline
context, callback
File "/bot/venv/lib/python3.7/site-packages/botbuilder/core/middleware_set.py", line 69, in receive_activity_with_status
return await self.receive_activity_internal(context, callback)
File "/bot/venv/lib/python3.7/site-packages/botbuilder/core/middleware_set.py", line 79, in receive_activity_internal
return await callback(context)
File "/bot/main.py", line 146, in aux_func
await BOT.on_turn(turn_context)
File "/bot/bots/dialog_bot.py", line 46, in on_turn
await self.conversation_state.save_changes(turn_context, False)
File "/bot/venv/lib/python3.7/site-packages/botbuilder/core/bot_state.py", line 88, in save_changes
await self._storage.write(changes)
File "/bot/venv/lib/python3.7/site-packages/botbuilder/azure/cosmosdb_storage.py", line 196, in write
raise error
File "/bot/venv/lib/python3.7/site-packages/botbuilder/azure/cosmosdb_storage.py", line 168, in write
e_tag = change.e_tag
AttributeError: 'dict' object has no attribute 'e_tag'
Attempting to modify this file /bot/venv/lib/python3.7/site-packages/botbuilder/azure/cosmosdb_storage.py for a temporary patch seems to lead me down a rabbit hole that makes it seems like CosmosDB is not fully implemented/tested to be used yet. Am I doing this right?
There were no current samples to work with.
CosmosDB has been fixed with the following commit: https://github.com/microsoft/botbuilder-python/commit/843edfc1f4c563d58ba639b23f2bb019bdbc381b
Related
I create a bot that notifies the user at certain times, but from time to time gives this error
dispatcher.py [LINE:390] ERROR | 2022-10-03 04:10:16,846 : Cause exception while getting updates.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/aiogram/dispatcher/dispatcher.py", line 381, in start_polling
updates = await self.bot.get_updates(
File "/usr/local/lib/python3.8/dist-packages/aiogram/bot/bot.py", line 110, in get_updates
result = await self.request(api.Methods.GET_UPDATES, payload)
File "/usr/local/lib/python3.8/dist-packages/aiogram/bot/base.py", line 231, in request
return await api.make_request(await self.get_session(), self.server, self.__token, method, data, files,
File "/usr/local/lib/python3.8/dist-packages/aiogram/bot/api.py", line 140, in make_request
return check_result(method, response.content_type, response.status, await response.text())
File "/usr/local/lib/python3.8/dist-packages/aiogram/bot/api.py", line 128, in check_result
raise exceptions.TelegramAPIError(description)
aiogram.utils.exceptions.TelegramAPIError: Bad Gateway
I think this problem is on the telegram side itself and is solved via webhook, but I don't want to use them.
import discord
from discord.ext import commands
import random
import praw
cl = commands.Bot(command_prefix = '!')
reddit = praw.Reddit(client_id = "",
client_secret = "",
username = "",
password = "",
user_agent = "")
#cl.event
async def on_ready():
print("Bot is ready, get ready to do wutever u want with it")
#cl.command()
async def meme(ctx, amount=50, subr="memes", filter="top"):
all_submission = []
subreddit = reddit.subreddit("subr")
subs = subreddit.filter(limit = amount)
for submission in subs:
all_submission.append(submission)
random_sub = random.choice(all_submission)
name = random_sub.title
url = random_sub.url
em = discord.embed(title = name)
em.set_image = url
await ctx.send(embed=em)
print("embed sent")
cl.run("")
when I was running this nothing showed up but when I debugged it and !meme in discord it was showing me this traceback error thing
It appears that you are using PRAW in an asynchronous environment.
It is strongly recommended to use Async PRAW: https://asyncpraw.readthedocs.io.
Ignoring exception in command meme:
Traceback (most recent call last):
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/discord/ext/commands/core.py", line 85, in wrapped
ret = await coro(*args, **kwargs)
File "main.py", line 20, in meme
subs = subreddit.filter(limit = amount)
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/praw/models/reddit/base.py", line 34, in __getattr__
self._fetch()
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/praw/models/reddit/subreddit.py", line 584, in _fetch
data = self._fetch_data()
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/praw/models/reddit/subreddit.py", line 581, in _fetch_data
return self._reddit.request("GET", path, params)
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/praw/reddit.py", line 885, in request
return self._core.request(
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/sessions.py", line 330, in request
return self._request_with_retries(
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/sessions.py", line 228, in _request_with_retries
response, saved_exception = self._make_request(
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/sessions.py", line 185, in _make_request
response = self._rate_limiter.call(
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/rate_limit.py", line 33, in call
kwargs["headers"] = set_header_callback()
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/sessions.py", line 283, in _set_header_callback
self._authorizer.refresh()
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/auth.py", line 425, in refresh
self._request_token(
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/prawcore/auth.py", line 158, in _request_token
raise OAuthException(
prawcore.exceptions.OAuthException: invalid_grant error processing request
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/discord/ext/commands/bot.py", line 939, in invoke
await ctx.command.invoke(ctx)
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/discord/ext/commands/core.py", line 863, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "/home/runner/memes-bot/venv/lib/python3.8/site-packages/discord/ext/commands/core.py", line 94, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: OAuthException: invalid_grant error processing request
This is the error thing I was talking about and it is very weird like telling me to download async praw which I have never had a problem with and pls help it is needed
This doesn't appear to be a discord.py issue, it appears to be related to the praw 0auth flow.
According to the traceback, there's an authentication issue in your praw credentials. Double, triple, and quadruple check your authentication credentials and flow with the workflow and methodology in the docs
I just want to learn how to store data in Firestore using Python and Google Cloud Platform, so I'm calling an API to query some example data.
For it, I'm using requests library and Firebase library from google.cloud package.
Here is the code that I'm running at Cloud Shell:
import requests
from google.cloud import firestore
url = "https://api.coindesk.com/v1/bpi/currentprice.json"
r = requests.get(url)
resp: str = r.text
if not (resp=="null" or resp=="[]"):
db = firestore.Client()
doc_ref=db.collection("CoinData").add(r.json())
When the code try to connect to Firebase to add the json of the API response I got this error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 66, in error_remapped_callable
return callable_(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Invalid resource field value in the request."
debug_error_string = "{"created":"#1634689546.004704998","description":"Error received from peer ipv4:74.125.134.95:443","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"Invalid resource field value in the request.","grpc_status":3}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/antonyare_93/cloudshell_open/pruebas/data_coin_query.py", line 11, in <module>
doc_ref=db.collection("CoinData").add(r.json())
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/collection.py", line 107, in add
write_result = document_ref.create(document_data, **kwargs)
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/document.py", line 99, in create
write_results = batch.commit(**kwargs)
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/batch.py", line 60, in commit
request=request, metadata=self._client._rpc_metadata, **kwargs,
File "/home/antonyare_93/.local/lib/python3.7/site-packages/google/cloud/firestore_v1/services/firestore/client.py", line 815, in commit
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/gapic_v1/method.py", line 142, in __call__
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 288, in retry_wrapped_func
on_error=on_error,
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 190, in retry_target
return target()
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 68, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Invalid resource field value in the request.
Anyone knows how I can fix it?
I've just got it.
It was a little mistake, I was missing the name of the project at the firestore client method call, here's the code working:
import requests
from google.cloud import firestore
url = "https://api.coindesk.com/v1/bpi/currentprice.json"
r = requests.get(url)
resp: str = r.text
if not (resp=="null" or resp=="[]"):
db = firestore.Client(project="mytwitterapitest")
doc_ref=db.collection("CoinData").add(r.json())
I'm trying to use Celery with SQS as broker. In order to use the SQS from my container I need to assume a role and for that I'm using STS. My code looks like this:
role_info = {
'RoleArn': 'arn:aws:iam::xxxxxxx:role/my-role-execution',
'RoleSessionName': 'roleExecution'
}
sts_client = boto3.client('sts', region_name='eu-central-1')
credentials = sts_client.assume_role(**role_info)
aws_access_key_id = credentials["Credentials"]['AccessKeyId']
aws_secret_access_key = credentials["Credentials"]['SecretAccessKey']
aws_session_token = credentials["Credentials"]["SessionToken"]
os.environ["AWS_ACCESS_KEY_ID"] = aws_access_key_id
os.environ["AWS_SECRET_ACCESS_KEY"] = aws_secret_access_key
os.environ["AWS_DEFAULT_REGION"] = 'eu-central-1'
os.environ["AWS_SESSION_TOKEN"] = aws_session_token
broker = "sqs://"
backend = 'redis://redis-service:6379/0'
celery = Celery('tasks', broker=broker, backend=backend)
celery.conf["task_default_queue"] = 'my-queue'
celery.conf["broker_transport_options"] = {
'region': 'eu-central-1',
'predefined_queues': {
'my-queue': {
'url': 'https://sqs.eu-central-1.amazonaws.com/xxxxxxx/my-queue'
}
}
}
In the same file I have the following task:
#celery.task(name='my-queue.my_task')
def my_task(content) -> int:
print("hello")
return 0
When I execute the following code I get an error:
[2020-09-24 10:38:03,602: CRITICAL/MainProcess] Unrecoverable error: ClientError('An error occurred (AccessDenied) when calling the ListQueues operation: Access to the resource https://eu-central-1.queue.amazonaws.com/ is denied.',)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line 921, in create_channel
return self._avail_channels.pop()
IndexError: pop from empty list
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 208, in start
self.blueprint.start(self)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 369, in start
return self.obj.start()
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/connection.py", line 23, in start
c.connection = c.connect()
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 405, in connect
conn = self.connection_for_read(heartbeat=self.amqheartbeat)
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 412, in connection_for_read
self.app.connection_for_read(heartbeat=heartbeat))
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 439, in ensure_connected
callback=maybe_shutdown,
File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 422, in ensure_connection
callback, timeout=timeout)
File "/usr/local/lib/python3.6/site-packages/kombu/utils/functional.py", line 341, in retry_over_time
return fun(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 275, in connect
return self.connection
File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 823, in connection
self._connection = self._establish_connection()
File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 778, in _establish_connection
conn = self.transport.establish_connection()
File "/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line 941, in establish_connection
self._avail_channels.append(self.create_channel(self))
File "/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line 923, in create_channel
channel = self.Channel(connection)
File "/usr/local/lib/python3.6/site-packages/kombu/transport/SQS.py", line 100, in __init__
self._update_queue_cache(self.queue_name_prefix)
File "/usr/local/lib/python3.6/site-packages/kombu/transport/SQS.py", line 105, in _update_queue_cache
resp = self.sqs.list_queues(QueueNamePrefix=queue_name_prefix)
File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 337, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 656, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListQueues operation: Access to the resource https://eu-central-1.queue.amazonaws.com/ is denied.
If I use boto3 directly without Celery, I'm able to connect to the queue and retrieve data without this error. I don't know why Celery/Kombu try to list queues when I specify the predefined_queues configuration, tha is used to avoid these behavior (from docs):
If you want Celery to use a set of predefined queues in AWS, and to never attempt to list SQS queues, nor attempt to create or delete them, pass a map of queue names to URLs using the predefined_queue_urls setting
Source here
Anyone know what happens? How I should modify my code in order to make it work?. Seems that Celery is not using the credentials at all.
The versions I'm using:
celery==4.4.7
boto3==1.14.54
kombu==4.5.0
Thanks!
PS: I created and issue in Github to track if this can be a library error or not...
I solved the problem updating dependencies to the latest versions:
celery==5.0.0
boto3==1.14.54
kombu==5.0.2
pycurl==7.43.0.6
I was able to get celery==4.4.7 and kombu==4.6.11 working by setting the following configuration option:
celery.conf["task_create_missing_queues"] = False
Im working on a slack bot using the new slack 2.0 python library. I am new to python decorators and I suspect that is part of my problem.
Here is my code...
#!/opt/rh/rh-python36/root/usr/bin/python
import os
import slack
# instantiate Slack client
slack_token = os.environ['SLACK_BOT_TOKEN']
rtmclient = slack.RTMClient(token=slack_token)
webclient = slack.WebClient(token=slack_token)
# get the id of my user
bot_id = webclient.auth_test()['user_id']
print('Bot ID: {0}'.format(bot_id))
def get_user_info(user_id):
user_info = webclient.users_info(user=user_id)['ok']
return user_info
#slack.RTMClient.run_on(event='message')
def parse_message(**payload):
data = payload['data']
user_id = data['user']
print(get_user_info(user_id))
rtmclient.start()
It outputs the Bot ID(using the webclient) when started but then crashes with RuntimeError: This event loop is already running when I make another call to webclient.
[root#slackbot-01 bin]# scl enable rh-python36 /root/slackbot/bin/slackbot.py
Bot ID: UBT547D31
Traceback (most recent call last):
File "/root/slackbot/bin/slackbot.py", line 24, in <module>
rtmclient.start()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 197, in start
return self._event_loop.run_until_complete(future)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/asyncio/base_events.py", line 467, in run_until_complete
return future.result()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 339, in _connect_and_read
await self._read_messages()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 390, in _read_messages
await self._dispatch_event(event, data=payload)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 440, in _dispatch_event
self._execute_in_thread(callback, data)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 465, in _execute_in_thread
future.result()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/root/slackbot/bin/slackbot.py", line 22, in parse_message
print(get_user_info(user_id))
File "/root/slackbot/bin/slackbot.py", line 15, in get_user_info
user_info = webclient.users_info(user=user_id)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/web/client.py", line 1368, in users_info
return self.api_call("users.info", http_verb="GET", params=kwargs)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/web/base_client.py", line 154, in api_call
return self._event_loop.run_until_complete(future)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/asyncio/base_events.py", line 454, in run_until_complete
self.run_forever()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/asyncio/base_events.py", line 408, in run_forever
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
The really confusing part to me is that if I comment out the line that makes the first call to webclient.auth_test(), I have no issues at all. My call to webclient.users_info() works every time rtmclient sends me data.
#!/opt/rh/rh-python36/root/usr/bin/python
import os
import slack
# instantiate Slack client
slack_token = os.environ['SLACK_BOT_TOKEN']
rtmclient = slack.RTMClient(token=slack_token)
webclient = slack.WebClient(token=slack_token)
# get the id of my user
#bot_id = webclient.auth_test()['user_id']
#print('Bot ID: {0}'.format(bot_id))
def get_user_info(user_id):
user_info = webclient.users_info(user=user_id)['ok']
return user_info
#slack.RTMClient.run_on(event='message')
def parse_message(**payload):
data = payload['data']
user_id = data['user']
print(get_user_info(user_id))
rtmclient.start()
[root#slackbot-01 bin]# scl enable rh-python36 /root/slackbot/bin/slackbot.py
True
True
^C[root#slackbot-01 bin]#
I need to get the bot id so that I can make sure it doesnt answer it's own messages. I don't why my code doesnt work after I get the bot id outside of the parse message function with a decorator.
What am I doing wrong here?
The python event loop is a tricky thing to program libraries around and there are some issues with the way the event queue is managed in the 2.0 version of SlackClient. It looks like some improvements were made with 2.1 but it appears to be a work in progress, and I still encounter this. I'd expect there will be future updates to make it more robust.
In the meantime, the following code at the top of your file (use pip to install) usually resolves it for me:
import nest_asyncio
nest_asyncio.apply()
Keep in mind this will alter the way the rest of your application is handling the event queue, if that's a factor.
If you're using RTM, the RTMClient creates a WebClient for you. The handle for it should be getting passed to you in the payload when you handle an event. You can check your ID by looking for the 'open' event which is always dispatched after RTM successfully connects and doing the lookup inside your 'open' event handler.