I've tried setting up a WebSocket server, and an API to access it via browser
I'm a week new into Python, and I hope this somewhat makes sense.
I have a Server class
**Server.py**
# WS server that sends messages at random intervals
import asyncio
import datetime
import random
import websockets
async def time(websocket, path):
while True:
now = datetime.datetime.utcnow().isoformat() + "Z"
await websocket.send(now)
await asyncio.sleep(random.random() * 3)
async def echo(websocket, path):
async for message in websocket:
await websocket.send(message)
async def router(websocket, path):
if path == "/get":
await time(websocket)
elif path == "/post":
await echo(websocket)
asyncio.get_event_loop().run_until_complete(
websockets.serve(router, "127.0.0.1", 8081, ping_interval=None))
asyncio.get_event_loop().run_forever()
and an API class
**API.py**
from quart import Quart
import json
import websockets
app = Quart(__name__)
#app.route('/get',methods=['GET'])
async def foo1():
uri = "ws://localhost:8081"
async with websockets.connect(uri) as websocket:
try:
receivedMessage = await websocket.recv()
print(f"< {receivedMessage}")
except:
print('Reconnecting')
websocket = await websockets.connect(uri)
return ' '
#app.route('/post', methods=['POST'])
async def foo2():
uri = "ws://localhost:8081"
async with websockets.connect(uri) as websocket:
await websocket.send("Hello world!")
try:
await websocket.recv()
except:
print('Reconnecting')
websocket = await websockets.connect(uri)
if __name__ == '__main__':
app.run()
My aim was to be able to access different functions from the Server, based on the API's Path which was being accessed (seen it here: https://github.com/aaugustin/websockets/issues/547)
The problem I'm getting is that "Reconnecting" appears constantly, and nothing actually happens when I make use of /get or /post (i.e. it seems to be closing after .recv() occurs - I know this because I had seen ConnectionClosedOk - no reason 1000 earlier.
I really don't understand what I'm doing right now lol, it is very confusing. I did get things to work when Server only had 1 function (constantly spitting out timestamps), and the API only had a GET function (to display the current time of the request).. things now don't really work after I implemented the router function.
Any help would be greatly, greatly appreciated. My Python knowledge is incredibly limited, so as verbose an answer as possible would really help me out.
I'm assuming that the POST method is going to result in "Hello world!" appearing on the console for the Server, but I'm probably wrong about that :( I just seen it in some tutorial documentation.
EDIT: Also, I'm not able to access localhost:(port)/post; it says Specified method is invalid for this resource; why is that?
Related
I'm using telethon and quart on the same loop.
I need to listen for new messages and in the same time i would like to read all channels when a page is loaded.
But doing this will end in a ConnectionError: Cannot send requests while disconnected and i don't know how to handle this. Adding async with client: before the loop will cause sqlite3.OperationalError: database is locked. There's a chance to archieve what i want?
Here the relevant code:
...
executor = ProcessPoolExecutor(1)
...
# Telethon client
client = TelegramClient('bot', int(api_id), api_hash)
# Quart app
app = Quart(__name__, template_folder=None, static_folder=None, static_url_path=None)
main = Blueprint('main', __name__, template_folder='templates', static_folder='static', static_url_path='static')
...
#app.before_serving
async def startup():
client.loop.run_in_executor(executor, telethon_start)
...
#client.on(events.NewMessage(incoming=True))
async def new_chat(event):
#Do something with messages
...
#main.route('/new', methods=['GET', 'POST'])
async def new():
channels = []
async for dialog in client.iter_dialogs(): # Here the problem
channels.append((dialog.id, dialog.name))
return await render_template(
'new_channel.html',
channels=channels
)
def telethon_start():
with client:
client.run_until_disconnected()
async def start(conf):
app.register_blueprint(main, url_prefix=base_url)
await hypercorn.asyncio.serve(app, conf)
if __name__ == '__main__':
config = hypercorn.Config()
config.bind = "127.0.0.1:" + str(expose_port)
client.loop.run_until_complete(start(config))
Please help me!
The executor usage seems suspicious. You shouldn't really be needing that.
Assuming #app.before_serving is called every time a request occurs, you will also be connecting the same client many times per request. This is wrong because each client instance should ideally only be connected once (you cannot reuse the same session multiple times).
My recommendation is that you simply connect the client before serving your application like so:
# #app.before_serving, startup and telethon_start removed entirely
async def start(conf):
async with client:
# ^~~~~~~~~~~ starting the client now occurs here
app.register_blueprint(main, url_prefix=base_url)
await hypercorn.asyncio.serve(app, conf)
I'm trying to scrape a website.
I found this site is using websocket, and in my opinion I also found the message that should be send to get my needed data.
Code is below.
If I run this code I'm getting a response, but nothing is in the message.
The original link = https://sport.circus.be/en/sport/sports-bets/844/227875758
Code I'm trying to run:
import asyncio
import websockets
var = []
messageToSend = '''{"Id":"dc5bc662-f718-370a-b1c4-55709cdcb4e3","TTL":10,"MessageType":1000,"Message":"{\"Direction\":1,\"Id\":\"42320c69-6447-561c-636f-8b14d870fd37\",\"Requests\":[{\"Id\":\"b2077eb6-fe48-6dd9-369c-ed3d56d88157\",\"Type\":201,\"Identifier\":\"ContinueLeaguesDataSourceFromCache\",\"AuthRequired\":false,\"Content\":\"{\\\"Entity\\\":{\\\"CacheId\\\":\\\"e857cc38-6f9a-48c0-b3d6-6b435101c94d\\\",\\\"EventIdList\\\":[3427106301,3427106501,3427106701,3427106901,9614598,9391158,9391160,9391159,9391162,9391155]},\\\"InitialRequest\\\":{\\\"Language\\\":\\\"en\\\",\\\"BettingActivity\\\":0,\\\"PageNumber\\\":0,\\\"OnlyShowcaseMarket\\\":true,\\\"IncludeSportList\\\":true,\\\"EventSkip\\\":0,\\\"EventTake\\\":200,\\\"EventType\\\":0,\\\"SportId\\\":844,\\\"RequestString\\\":\\\"LeagueIds=227875758&OnlyMarketGroup=Main\\\"}}\"}],\"Groups\":[]}"}'''
async def hello():
async with websockets.connect("wss://wss02.circus.be:443", timeout=10, ping_interval=None) as websocket:
await websocket.send(messageToSend)
recv_msg = await websocket.recv()
print(recv_msg)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(hello())
It would be great if someone could help me out.
Thanks a lot already.
Quart is a Python web framework which re-implements the Flask API on top of the asyncio coroutine system of Python. In my particular case, I have a Quart websocket endpoint which is supposed to have not just one source of incoming events, but two possible sources of events which are supposed to continue the asynchronous loop.
An example with one event source:
from quart import Quart, websocket
app = Quart(__name__)
#app.websocket("/echo")
def echo():
while True:
incoming_message = await websocket.receive()
await websocket.send(incoming_message)
Taken from https://pgjones.gitlab.io/quart/
This example has one source: the incoming message stream. But what is the correct pattern if I had two possible sources, one being await websocket.receive() and another one being something along the lines of await system.get_next_external_notification() .
If either of them arrives, I'd like to send a websocket message.
I think I'll have to use asyncio.wait(..., return_when=FIRST_COMPLETED), but how do I make sure that I miss no data (i.e. for the race condition that websocket.receive() and system.get_next_external_notification() both finish almost exactly at the same time) ? What's the correct pattern in this case?
An idea you could use is a Queue to join the events together from different sources, then have an async function listening in the background to that queue for requests. Something like this might get you started:
import asyncio
from quart import Quart, websocket
app = Quart(__name__)
#app.before_serving
async def startup():
print(f'starting')
app.q = asyncio.Queue(1)
asyncio.ensure_future(listener(app.q))
async def listener(q):
while True:
returnq, msg = await q.get()
print(msg)
await returnq.put(f'hi: {msg}')
#app.route("/echo/<message>")
async def echo(message):
while True:
returnq = asyncio.Queue(1)
await app.q.put((returnq, message))
response = await returnq.get()
return response
#app.route("/echo2/<message>")
async def echo2(message):
while True:
returnq = asyncio.Queue(1)
await app.q.put((returnq, message))
response = await returnq.get()
return response
I am getting a stream from a source which I made so that it can be accessed at a particular websocket URL (Not sure if this ideal and would appreciate any other architectures as well). I need to now do object detection on this video stream, and thought of the architecture that I will connect to the websocket URL through a client websocket library like websocket in a server which is made through flask or fastapi, and then again stream the object detected video to multiple clients through another websocket URL.
The problem is I am unable to receive any images even after connecting to the websocket URL and am not sure how to handle asyncio in a server scenario as in where to put the line run_until_complete.
Any suggestions or help would be greatly appreciated
Server code
import uvicorn
from fastapi import FastAPI, WebSocket
# import websockets
# import asyncio
# init app
app = FastAPI()
async def clientwebsocketconnection():
uri = "wss://<websocket URL here>"
async with websockets.connect(uri) as websocket:
print("reached here")
data = await websocket.recv()
print(f"< {data}")
# Routes
#app.get('/')
async def index():
return {"text": "Its working"}
#app.websocket('/websocketconnection') # FIXME: change this to a websocket endpoint
async def websocketconnection():
return {"text": "Its working"}
if __name__ == '__main__':
uvicorn.run(app, host="127.0.0.1", port=8000)
# asyncio.get_event_loop().run_until_complete(clientwebsocketconnection())
I assume you want to send a text via websocket. TO do this you need the following code:
#app.websocket('/websocketconnection')
async def websocketconnection(websocket: WebSocket) -> None:
await websocket.accept()
await websocket.send_text("It's working!")
await websocket.close()
You may find more examples in the official FastAPI docs.
I am building a Discord bot in Python and would like to receive HTTP requests from Twitch.tv's API (See Webhooks Guide & Webhooks Reference) (To subscribe to events like; X streamer has gone live) and based on the content of the HTTP (POST or GET) request received from Twitch, do something on the Discord bot, e.g: Output a message on a text channel.
I am using the discord.py Python Discord API/Library.
I've looked into the matter and found that Flask seemed like a good minimalist choice for a webserver to receive these requests on.
I should preface this by saying I'm very new to Python and I've never used Flask before.
Now. The problem is I can't seem to figure out a way to run the Flask server inside of my discord bot.
I've tried adding this simple code into my discord.py script:
from flask import Flask, request
app = Flask(__name__)
#app.route('/posts', methods=['POST'])
def result():
print(request.form['sched'])
# Send a message to a discord text channel etc...
return 'Received !'
When I run my discord.py script which looks something like this:
(Stripped away some commands and features for the sake of keeping this shorter)
import discord
import asyncio
from flask import Flask, request
app = Flask(__name__)
#app.route('/posts', methods=['POST'])
def result():
print(request.form['sched'])
# Send a message to a discord text channel etc...
return 'Received !'
client = discord.Client()
#client.event
async def on_ready():
print('Logged in as')
print(client.user.name)
print(client.user.id)
print('------')
#client.event
async def on_message(message):
if message.author == client.user:
return
content = message.content
fullUser = message.author.name+'#'+message.author.discriminator
print(str(message.timestamp)+" #"+message.channel.name+" "+fullUser+": "+str(content.encode('ascii', 'ignore').decode('ascii')))
if content.startswith('!'):
content = content[1:]
if content.startswith('test'):
counter = 0
tmp = await client.send_message(message.channel, 'Calculating messages...')
async for log in client.logs_from(message.channel, limit=100):
if log.author == message.author:
counter += 1
await client.edit_message(tmp, 'You have {} messages.'.format(counter))
client.run('MyTokenHere')
It seems like if I point flask to discord.py (the above) and run it, it'll start the code, get to the "client.run('MyTokenHere')" part for discord, and just stop at that and run the discord bot. It's not until I exit out of the bot by doing Ctrl+C that the actual Flask server starts, but now the discord bot is disconnected and no longer does any processing.
The same problem persists if I were to for example add "app.run()" somewhere in my code (before calling "client.run()" which starts the Discord bot part) to launch the Flask server; It'll just run the flask, get stuck on that until I Ctrl+C out of the Flask server, then it'll proceed to start the Discord bot.
Ultimately, I need to use the Discord API and I need to be connected to the Discord API gateway and all that good jazz to actually send messages to a channel with the bot, so I don't really know what to do here.
So. I think I've tried my best to explain what I'm ultimately trying to achieve here, and hopefully someone can help me find a way to either make this work with Flask, or if there's a better and easier way, provide a different solution.
This is cog example in discord.py
I made this thing for dbl (Discord Bot Lists) you can implement, the thing you need.
Note: run your heroku app with webprocess
example :
web: python main.py
Then go on https://uptimerobot.com/
and setup to ping your web app after every 5 minutes
from aiohttp import web
from discord.ext import commands, tasks
import discord
import os
import aiohttp
app = web.Application()
routes = web.RouteTableDef()
def setup(bot):
bot.add_cog(Webserver(bot))
class Webserver(commands.Cog):
def __init__(self, bot):
self.bot = bot
self.web_server.start()
#routes.get('/')
async def welcome(request):
return web.Response(text="Hello, world")
#routes.post('/dbl')
async def dblwebhook(request):
if request.headers.get('authorization') == '3mErTJMYFt':
data = await request.json()
user = self.bot.get_user(data['user']) or await self.bot.fetch_user(data['user'])
if user is None:
return
_type = f'Tested!' if data['type'] == 'test' else f'Voted!'
upvoted_bot = f'<#{data["bot"]}>'
embed = discord.Embed(title=_type, colour=discord.Color.blurple())
embed.description = f'**Upvoter :** {user.mention} Just {_type}' + f'\n**Upvoted Bot :** {upvoted_bot}'
embed.set_thumbnail(url=user.avatar_url)
channel = self.bot.get_channel(5645646545142312312)
await channel.send(embed=embed)
return 200
self.webserver_port = os.environ.get('PORT', 5000)
app.add_routes(routes)
#tasks.loop()
async def web_server(self):
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, host='0.0.0.0', port=self.webserver_port)
await site.start()
#web_server.before_loop
async def web_server_before_loop(self):
await self.bot.wait_until_ready()
As the kind commenters informed me; threading seems like the way to go.
Thanks guys!
Another cool way if you aren't going to use flask extensions, use quart instead of flask, then it will super easy for you.
Note: Run your heroku app with webprocess
example :
web: python main.py
Then go on https://uptimerobot.com/ and setup to ping your web app after every 5 minutes
# Code Example
from discord.ext import commands
from quart import Quart
import os
app = Quart(__name__)
bot = commands.Bot('!')
#bot.command()
async def something(ctx):
...
"""
Note: On Heroku you cant bind your webserver with 5000 port as they aren't static.
To fix above problem you will have to get dynamic port from the environment variable and you are good to go.
"""
PORT = os.environ.get('PORT')
bot.loop.create_task(app.run_task('0.0.0.0', PORT))
bot.run('Token')
Or you can use the Terminal Multiplexer, tmux to run them independently!.
If you are running on a Linux platform, tmux python3 flaskapp.py would run the flask app, while you can independently run the discord bot.
You can use asyncio with flask to perform this
import discord
import asyncio
class PrintDiscordChannelsClient(discord.Client):
def __init__(self, *args, **kwargs):
self.guild_id = kwargs.pop('guild_id')
super().__init__(*args, **kwargs)
async def on_ready(self):
try:
await self.wait_until_ready()
guild = self.get_guild(int(self.guild_id))
print(f'{guild.name} is ready!')
channels = guild.text_channels
print(f'{(channels)} channels found')
await self.close()
except Exception as e:
print(e)
await self.close()
async def print_discord_channels(guild_id):
client = PrintDiscordChannelsClient(guild_id=guild_id)
await client.start('DISCORD_BOT_TOKEN')
#application.route("/discord-api", methods=["GET"])
def discord_api():
guild_id = request.args.get('guild_id')
asyncio.run(print_discord_channels(guild_id=guild_id))
return make_response("Success", 200)