I am using the third-party websocket library of Python
(https://github.com/miguelgrinberg/flask-sock)
it is ran with flask framework
I am sending over a data with 2568 length of characters, the server sometimes can receive full data, sometimes it receive only total length of 1420
The flask web application is running in server, and my data is sending from my localhost pc, i checked with wireshark, my data was successfully send over to server with 2568 length of characters, but my server sometimes receiving only 1420 length of data/string
i just contacted the developer of this third-party library, here is it https://github.com/miguelgrinberg/flask-sock/issues/6
my question is to understanding, will websocket lost data in the middle or i am sending too much sizes of data?
Update #1
Server
from flask_sock import Sock
app = Flask(__name__)
sock = Sock(app)
#sock.route('/websocket/instance_state')
def instance_state(ws):
while ws.connected:
print("Receiving :: ")
req = ws.receive()
print(f"{len(req)}")
ws.send(req)
Client
import websockets
async with websockets.connect(uri) as websocket:
while True:
the_string = "a" * 2600
print(f"Sending :: {the_string}")
await websocket.send(the_string)
resp = await websocket.recv()
I had overcome the situation by implementing Flask-SocketIO instead of Flask-Sock, so the issue was not solved but due to project timeline, Flask-SocketIO works fine
Related
I wrote a code in python 3.9.0 which opens a text file, read line by line and host it at web socket for clients to consume. It works just fine. The problem is, when I consume the file from the web socket server it is really slow. I can see that server sends it faster in thousands but client is receiving it slow. I am not sure what am I doing wrong with my client code. Here is my client code
async def consumeStream():
while True:
async with websockets.connect('ws://localhost:6800') as ws:
ingestCounter = 0
while True:
try:
consumedStream = await ws.recv()
except websockets.ConnectionClosed:
print(color.red+'Websocket connection closed, retrying !'+color.reset)
break
if ingestCounter % 100 == 0:
print('\r'+f'Consuming: {ingestCounter}', end='')
ingestCounter += 1
await asyncio.sleep(0)
Is streaming good idea with web sockets? Should I use Flask or FastAPI with yield ? What is the best practice for web sockets streaming and consuming ?
I am opening a GRPC bidirectional stream with a server (python3.8 specifically). After I get some data from the server, I have to do a time consuming task (it takes about 3 minutes). While doing this, I am keeping the channel open to send the results to the server as soon as the task is done
Exactly 1 minute after the server's last message, my connection closes. And I get this error
<AioRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "Received RST_STREAM with error code 1"
debug_error_string = "{"created":"#1625250019.779494905","description":"Error received from peer ipv4:3.101.44.139:443","file":"src/core/lib/surface/call.cc","file_line":1066,"grpc_message":"Received RST_STREAM with error code 1","grpc_status":13}"
>
To me it seems like some timeout. I am a beginner to grpc. I referred to this article and gave my channel the client options as specified in the article with every number close to 5mins. But it did not change anything. Can someone please help me understand what is going on here and how I should solve this?
TL;DR
My connection to the server is an async secure channel. Please note that my two streams are independant which is why I have to do async
channel=grpc.aio.secure_channel(address, credentials=creds, options=options)
I have two coroutines, one is waiting for the long running task to be excecuted, other receiving and sensing messages on the bi directional stream
def parse_response(resp):
# collect data from server response
# make `task_available=True` when all data is received
async def stream_handler():
stub = TaskServiceStub(channel)
req_gen = get_requests() # An async generator for request messages
stream: grpc.aio.StreamStreamCall = stub.Task(req_gen)
async for resp in stream.__aiter__():
parse_response(resp)
async def task_handler():
if task_available:
do_task()
async def main():
await asyncio.gather(stream_handler(), task_handler())
Adding a read timeout to the server nginx configuration solved the issue
nginx.ingress.kubernetes.io/server-snippet: |
client_header_timeout 3h;
client_body_timeout 3h;
grpc_read_timeout 3h;
grpc_send_timeout 3h;
I'm having a lot of difficulty with a very simple task. I'm attempting to set up a socket.io client in node js, which should then communicate with a local socket.io server setup in python (using the python bindings here. The issue I'm having is the server is detecting the client, but the client never seems to receive the 'connect' event. I suspect this is an issue with the way I've deployed the server asynchronously, but I'm really not sure. The code for each file is below:
server.py
import socketio
from aiohttp import web
HOST = '127.0.0.1'
PORT = 10001
# create a Socket.IO server
sio = socketio.AsyncServer(async_mode='aiohttp', logger=True, engineio_logger=True)
app = web.Application()
sio.attach(app)
#sio.on('connect')
def connect(sid, environ):
print('connect ', sid)
if __name__ == '__main__':
web.run_app(app, host=HOST, port=PORT)
client.js
const io = require('socket.io-client');
const HOST = '127.0.0.1';
const PORT = '10001';
const socket = io(`http://${HOST}:${PORT}`);
console.log('Socket instantiated!');
socket.on('connect', () => {
console.log(`socket connected: ${socket.connected}`);
});
The output I would expect is to see the server print out that the client has connected, and then for the client to print out that it has connected too. However, the client never seems to receive the 'connect' event, so never prints anything to the console.
Finally, an example of the server's output is:
Server initialized for aiohttp.
======== Running on http://127.0.0.1:10001 ========
(Press CTRL+C to quit)
1c17586e4c7e49b48abefea2fba460e6: Sending packet OPEN data {'sid': '1c17586e4c7e49b48abefea2fba460e6', 'upgrades': ['websocket'], 'pingTimeout': 60000, 'pingInterval': 25000}
connect 1c17586e4c7e49b48abefea2fba460e6
1c17586e4c7e49b48abefea2fba460e6: Sending packet MESSAGE data 0
While the client's output is annoyingly just
$ node mySocket.js
Socket instantiated!
and then it just hangs doing nothing.
I'm clearly misunderstanding something here, so thank you in advance!
Small update
I quickly tested using the python socketio client, and succesfully got an actual connection, so this should narrow it down to something I've done in the JS client.
Well, I ended up downgrading from socket.io-client 3.00 (did not see there was this major release 3 days ago) back to 2.3.1, and everything started working again! However, based on the lack of issues listed on Github, I'm guessing this is not a bug that is affecting everyone.
I am currently working on a django project where I use aiohttp to communicate between the backend and frontend. I wanted to get the IP address of a client when a request is made from the frontend. Looked in different docs but none seems to point to exactly how to get the IP address using aiohttp. Someone Help!
from aiohttp import web
async def handler(request):
ws = web.WebSocketResponse()
await ws.prepare(request)
try:
async for msg in ws:
# handle incoming messages
# use ws.send_str() to send data back
...
finally:
task.cancel()
Based on the aiohttp docs you can get the originating IP address of a client initiated HTTP request from requests remote parameter (request.remote).
I would like to connect with several address using asyncio library, like
'127.0.0.1:8000/hello'
But where can I put '/hello' part ? and how can I call this part? (endpoint? static url? )
asyncio.open_connection('127.0.0.1', 8000)
But where can I put /hello part ?
asyncio.open_connection opens a raw TCP connection to the given remote endpoint. The address of the form 127.0.0.1:8000/hello strongly indicates an HTTP connection. HTTP is a protocol built on top of TCP, where you connect to the given host and port (127.0.0.1 and port 8000 in your case) using TCP and send a request in a specified format, and to which the server answers with a response.
HTTP communication is a complex topic and is not something you want to implement from scratch. Fortunately there are mature asyncio-aware libraries that handle it for you, such as aiohttp. Using that library connecting to a server might look like this:
import aiohttp, asyncio
async def connect_to(url, session):
async with session.get(url) as response:
return await response.data()
async def main():
async with aiohttp.ClientSession() as session:
data = connect_to('http:/127.0.0.1:8000/hello')
# do something with data, or connect to other urls
asyncio.run(main())
and how can I call this part? (endpoint? static url? )
That part is typically called the path of the URL, see e.g. section 3.3 of the RFC for more details.