I have the following code for a Sanic hello world based off combining different endpoints here:
https://sanic.readthedocs.io/en/latest/sanic/response.html
https://sanic.readthedocs.io/en/latest/sanic/websocket.html
Code is:
from sanic import Sanic
from sanic import response
from sanic.websocket import WebSocketProtocol
app = Sanic()
#app.route("/")
async def test(request):
return response.json({"hello": "world"})
#app.route('/html')
async def handle_request(request):
return response.html('<p>Hello world!</p>')
#app.websocket('/feed')
async def feed(request, ws):
while True:
data = 'hello!'
print('Sending: ' + data)
await ws.send(data)
data = await ws.recv()
print('Received: ' + data)
#app.route('/html2')
async def handle_request(request):
return response.html("""<html><head><script>
var exampleSocket = new WebSocket("wss://0.0.0.0:8000/feed", "protocolOne");
exampleSocket.onmessage = function (event) {
console.log(event.data)};</script></head><body><h1>Hello socket!</h1><p>hello</p></body></html>""")
app.run(host="0.0.0.0", port=8000)
# app.run(host="0.0.0.0", port=8000, protocol=WebSocketProtocol) # ws
The routes "/" and "/html" work fine, but
http://0.0.0.0:8000/feed
produces:
Error: Invalid websocket request
and "/html2" renders the page fine, but doesn't log to console, showing in the debugger:
Firefox can’t establish a connection to the server at wss://0.0.0.0:8000/feed.
What should I change or add to make a viable websocket endpoint that plays nicely with the http ones, too?
Using 0.0.0.0 as your endpoint within your client html doesn't make any sense and you're not using SSL so you want to use ws:// rather than wss://. In other words,
from sanic import Sanic
from sanic import response
from sanic.websocket import WebSocketProtocol
app = Sanic()
#app.websocket('/feed')
async def feed(request, ws):
while True:
data = 'hello!'
print('Sending: ' + data)
await ws.send(data)
data = await ws.recv()
print('Received: ' + data)
#app.route('/html2')
async def handle_request(request):
return response.html("""<html><head><script>
var exampleSocket = new WebSocket("ws://" + location.host + '/feed');
exampleSocket.onmessage = function (event) {
console.log(event.data)};</script></head><body><h1>Hello socket!</h1><p>hello</p></body></html>""")
app.run(host="0.0.0.0", port=8000)
Related
I have a rest api server which makes call to some other Apis,I am accessing the data I get from the server on a react js frontend,But for certain usecases I need to fetch real time data from backed,is there any way do this together,below is my code
from flask import Flask,request
from flask_cors import CORS
from tuya_connector import TuyaOpenAPI, TUYA_LOGGER
app = Flask(__name__)
CORS(app)
#app.get("/api/device/<string:deviceid>")
def getdata(deviceid):
ACCESS_ID = ""
ACCESS_KEY = ""
API_ENDPOINT = ""
# Enable debug log
# Init OpenAPI and connect
openapi = TuyaOpenAPI(API_ENDPOINT, ACCESS_ID, ACCESS_KEY)
openapi.connect()
# Set up device_id
DEVICE_ID = deviceid
# Call APIs from Tuya
# Get the device information
response = openapi.get("/v1.0/devices/{}".format(DEVICE_ID))
return response
I want to have traditional request response service along with real time data fetching
Websockets endpoints are exactly what you're looking for. If that is not too late, I'd recommend switching to FastAPI which supports WebSockets "natively" (out-of-the-box) - https://fastapi.tiangolo.com/advanced/websockets
If you need to keep using Flask, there are a few packages that allow you to add WebSockets endpoints: https://flask-sock.readthedocs.io/en/latest/
With FastAPI, this is that simple:
#app.get("/")
async def get():
return {"msg": "This is a regular HTTP endpoint"}
#app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
while True:
data = await websocket.receive_text()
await websocket.send_text(f"Message text was: {data}")
I'm in a situation where I need to retry async request even when the request returns 200 response. For some specific cases, I need to check if there's a key in the output. If so, we need to retry. The following sample code (which can be executed in a Jupyter notebook) works for retries whenever the request fails (non-200). How can I tweak this to cater to this particular need?
P.S. Ideally, the response should've been non-200 but this is the option I'm left with.
# load required libraries
import json
import asyncio
import aiohttp
from async_retrying import retry
base_url = "http://localhost:1050/hello?rid="
# async ginger call
#retry(attempts=3)
async def async_ginger_call():
connector = aiohttp.TCPConnector(limit=3)
async with aiohttp.ClientSession(connector=connector) as session:
async with session.post(url, raise_for_status=True, timeout=300) as response:
result = await response.text()
# condition here; if key in result then retry
return json.loads(result)
reqs = 2
tasks = []
connector = aiohttp.TCPConnector(limit=reqs)
async with aiohttp.ClientSession(connector=connector) as session:
for i in range(reqs):
url = base_url + str(i)
# encode sentence
tasks.append(async_ginger_call())
results = await asyncio.gather(*tasks, return_exceptions=True)
Sample flask server code
# sample api
import time
import json
import datetime
from flask import Flask, request
from flask import Response
app = Flask(__name__)
#app.route('/hello', methods=['GET', 'POST'])
def welcome():
rid = request.args.get('rid', default=3, type=int)
valid_response = json.dumps({
"Result": {
"Warnings": [
{
"Code": 1100,
"Message": "A technical error occurred during execution."
}
]
}
}
)
# testing for failure
if rid == 2:
# pass
# return valid_response
return Response("{'Result': ''}", status=500, mimetype='application/json')
return valid_response
if __name__ == '__main__':
app.run(host='0.0.0.0', port=1050)
Now I’m working on developing WebSocket to get data (ex.btcusdt) from the website FTX
since the ftx.com you can trade the crypto without having to pay fees and I have a minimum budget
so now I want to try out to get some data and making my own bot
but now I’m having a problem with how I connect to the website
since I watch Binance video I was trying the same way but still didn’t get any message from the " wss://ftx.com/ws/ "
I do not quite understand the document they provide
my question is how can I connect to the data stream for example if I want to get the JSON file of BTCUSDT or BULLUSDT
this is the document they provide
https://docs.ftx.com/#websocket-api
Thank you
My code
import websocket
SOCKET = "wss://ftx.com/ws/"
def on_open(ws):
print('opened connection')
def on_close(ws):
print('closed connection')
def on_message(ws, message):
print("got message")
ws = websocket.WebSocketApp(SOCKET, on_open=on_open, on_close=on_close, on_message=on_message)
ws.run_forever()
This works find with Binance
Using the example code here modified to accept api keys as arguments, here is an example of grabbing ticker data:
if __name__ == '__main__':
# rest = client.FtxClient(api_key=key, api_secret=secret)
ws = ws_client.FtxWebsocketClient(api_key=key, api_secret=secret)
ws.connect()
for i in range(1, 10):
print(ws.get_ticker(market='BTC-PERP'))
time.sleep(1)
As Chev_603 say you can copy the two files into your directory.
Then, import the files at the begging of your app and use:
from client import FtxWebsocketClient
from websocket_manager import WebsocketManager
if __name__ == '__main__':
for i in range(1, 1000):
print(ws.get_ticker(market='BTC-PERP'))
time.sleep(0.2)
# your own api_key and api_secret must be written into the client.py lines 20 and 21
JSON message:
import websocket
import json
this = json.dumps({'op': 'subscribe', 'channel': 'trades', 'market': 'BTC-
PERP'})
def on_open(wsapp):
wsapp.send(this)
def on_message(wsapp, message):
print(message)
wsapp = websocket.WebSocketApp("wss://ftx.com/ws/", on_message=on_message,
on_open=on_open)
wsapp.run_forever()
I can use the following way to send message when I open one website(http://localhost:api/notify).
I want to send message at the specific time(eg. 5 pm everyday)automatically, how can i do that?
from aiohttp import web
from aiohttp.web import Request, Response, json_response
async def notify() :
await _send_proactive_message()
return Response(status=201, text="Proactive messages have been sent")
APP = web.Application()
APP.router.add_get("/api/notify", notify)
if __name__ == "__main__":
try:
web.run_app(APP, host="localhost")
except Exception as error:
raise error
I have a RESTFUL Flask API I am serving with gunicorn and I'm trying to continue running parse_request() after sending a response to whoever made a POST request so they're not left waiting for it to finish
I'm not too sure if this will even achieve what I want but this is the code I have so far.
from threading import Thread
import subprocess
from flask import Flask
import asyncio
application = Flask(__name__)
async def parse_request(data):
try:
command = './webscraper.py -us "{user}" -p "{password}" -url "{url}"'.format(**data)
output = subprocess.check_output(['bash','-c', command])
except Exception as e:
print(e)
#application.route('/scraper/run', methods=['POST'])
def init_scrape():
try:
thread = Thread(target=parse_request, kwargs={'data': request.json})
thread.start()
return jsonify({'Scraping this site: ': request.json["url"]}), 201
except Exception as e:
print(e)
if __name__ == '__main__':
try:
application.run(host="0.0.0.0", port="8080")
except Exception as e:
print(e)
I am sending a POST request similar to this.
localhost:8080/scraper/run
data = {
"user": "username",
"password": "password",
"url": "www.mysite.com"
}
The error I get when sending a POST request is this.
/usr/lib/python3.6/threading.py:864: RuntimeWarning: coroutine 'parse_request' was never awaited
self._target(*self._args, **self._kwargs)
So first things first, why are you calling webscraper.py with subprocess? This is completely pointless. Because webscraper.py is a python script you should be importing the needed functions/classes from webscraper.py and using them directly. Calling it this way is totally defeating what you are wanting to do.
Next, your actual question you have got mixed up between async and threading. I suggest you learn more about it but essentially you want something like the following using Quart which is an async version of Flask, it would suit your situation well.
from quart import Quart, response, jsonify
import asyncio
from webscraper import <Class>, <webscraper_func> # Import what you need or
import webscraper # whatever suits your needs
app = Quart(__name__)
async def parse_request(user, password, url):
webscraper_func(user, password, url)
return 'Success'
#app.route('/scraper/run', methods=['POST'])
async def init_scrape():
user = request.args.get('user')
password = request.args.get('password')
url = request.args.get('url')
asyncio.get_running_loop().run_in_executor(
None,
parse_request(user, password, url)
)
return 'Success'
if __name__ == '__main__':
app.run(host='0.0.0.0', port='8080')