Develop Reference

Python websocket server command stream (lua client)?

After seeing a video about a youtuber who uses a websocket server to interacte with an entity in a video game (minecraft) video : https://www.youtube.com/watch?v=pwKRbsDbxqc&t=25s&ab_channel=Ottomated
I wanted to recreate the same idea, but with some little tweaks:
I wanted the websocket server to be host in python
I wanted to be able to interact without any html or javascript
So I built the websocket server:
#!/usr/bin/env python
import asyncio
import websockets
async def server(websocket, path):
data = await websocket.recv()
while True:
msg = input('type the command: ')
await websocket.send(msg)
start_server = websockets.serve(server, "localhost", 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
(really simple)
and the client side (in lua):
print('connecting')
local ws, err = http.websocket("ws://localhost:8765")
if not ws then
return printError(err)
ws.close()
end
ws.send("message")
repeat
command = ws.receive()
load(command)()
until command == "stop"
ws.close()
The only thing is that after the first message gets executed by the lua programm, it stops processing the other messages.
I don't know what I did wrong!
Am I doing it the wrong way? or maybe there is a better way?

well i am still learning sockets and improving my server-side skill but it is easy you just need to make it a loop 'while true:' or i prefer a controllable variable 'kim = True
while kim :' then you can make false or true to close or open the loop under certian conditions and i think you need somthing like threads to not block the other clients input
well i didnt study web-server but the problem is as i think its that you only receive one message because its not in a loop so it will excute the code and then close

How to connect to multiple channels using websocket multiprocessing?

I wrote a script allowing to collect data through different websockets channels but I can't manage to listen to multiple channel at once with this script. I would like to find a solution such as "multiprocessing" or "thread".
I managed to listen to several channel by running several times the script in different terminals.
For example if I want to listen to 10 channels, I launch 10 times the script with the desired channel as input argument, but I am sure there must be a smarter and cleaner way to do so using multiprocessing or other process.
Indeed, if I launch to many terminals my computer starts being really slow and the finality of this program would be to run it with a Raspberry Pi which is less powerful than my laptop.
The script is the following:
import websocket
import sys
def getData(uri, channel, pathCSV):
ws = websocket.create_connection(uri)
try:
print("Sending data")
ws.send(channel)
print("Receiving...")
result = ws.recv()
print ("Received '%s'" % result)
while True:
result = ast.literal_eval(ws.recv())
print("Received ast '%s'" % result)
# Here is a function which write the collected data to
# a CSV.
exportDataToCSV(result, pathCSV)
except websocket.WebSocketConnectionClosedException as e:
print('This caught the exception')
ws.close()
# In case of error I simply relaunch the script
getMarketData(uri, channel, pathCSV)
except KeyboardInterrupt:
ws.close()
return(result)
pathCSV = "/path/to/CSV"
uri = "websocket adress"
channelList = ["channel1", "channel2", "channel3", "channel4",
"channel5"]
#channel : for now I have to select one
while True:
getData(uri, channel, pathCSV)
So the question is "How can I manage to listen to all channels in only one instance of the script designed to collect and write received data to a CSV?".
Thank you in advance if you have any ideas to share.
Edit:
I found some information with "asyncio" library which lead me to the following code:
import asyncio
import websockets
import ast
uri = "websocket uri"
channelList = ["channel1", "channel2", "channel3", "channel4",
"channel5"]
async def getData(channelList):
uri = "websocket uri"
for channel in channelList:
async with websockets.connect(uri) as ws:
await ws.send(channel)
print("Receiving...")
result = await ws.recv()
# Confirmation ofsubscription
print ("Received '%s'" % result)
result = ast.literal_eval(await ws.recv())
# Getting Data
print("Received ast '%s'" % result)
asyncio.get_event_loop().run_until_complete(getData(channelList))
This way I can subscribe and listen to all channels by launching only one script.
But this doesn't really help because in each loop I have to reconnect to the channel and if a channel took to long to answer I am missing lots of information from the other channels while I am not connected to them.
Could anyone help me to optimize the process?

Understanding async await in python socket io / aiohttp server

I am trying to setup an socket.io server using python-socketio.
Here is a minimal working example:
import asyncio
from aiohttp import web
import socketio
import random
sio = socketio.AsyncServer(async_mode='aiohttp')
app = web.Application()
sio.attach(app)
#sio.on('connect')
def connect(sid, environ):
print("connected: ", sid)
#sio.on('sendText')
async def message(sid, data):
print("message ", data)
# await asyncio.sleep(1 * random.random())
# print('waited', data)
#sio.on('disconnect')
def disconnect(sid):
print('disconnect ', sid)
if __name__ == '__main__':
web.run_app(app, host='0.0.0.0', port=8080)
This runs fine, and I can execute (here in node.js) for instance
const io = require('socket.io-client');
const socket = io('ws://localhost:8080');
socket.emit('sendText', 'hey 1')
socket.emit('sendText', 'hey 2')
socket.emit('sendText', 'hey 3')
If I run the server and run the node script above I get server-side
connected: c1e687f0e2724b339fcdbefdb5aaa8f8
message hey 1
message hey 2
message hey 3
However, if I uncomment the lines with await sleep in the code, I only receive the first message:
connected: 816fb6700f5143f7875b20a252c65f33
message hey 1
waited hey 1
I don't understand why the next messages are not appearing.
Can only one instance of async def message run at the same time? Or why?
I am sure that I am not understanding something very fundamental about how this works. I would be very grateful if someone could point out what I am not understanding.

I'm the author of the python-socketio package. There are two problems here, I think. I can answer your question:
Can only one instance of async def message run at the same time? Or why?
My Socket.IO server serializes the events that are received from a given client. So for example, if client A sends an event that runs for one minute, any additional events sent by A during that minute will be queued, waiting for the first event to complete first. If client B sends an event during that minute, it will be handled immediately. The reason why events from a client are artificially serialized is to prevent race conditions or other side effects from occurring as a result of two or more handlers for the same client running in parallel. This serialization of events can be turned off, with the async_handlers option:
sio = socketio.AsyncServer(async_mode='aiohttp', async_handlers=True)
Using aiohttp 2.3.7 and async_handlers=True your three events are received at more or less the same time, and then all handlers wait in parallel during their sleep periods.
Unfortunately this does not explain the 2nd and 3rd events never reaching the server. I have verified that these events are properly queued and executed in sequence with aiohttp 2.2.5, but this breaks with 2.3.0 all the way to 2.3.7. My current theory is that a change that was introduced in 2.3.0 is causing these messages that arrive while the task is sleeping to get dropped, but haven't found why that happens yet.

Python IRC ChatBot hangs on socket.recv after seemingly random time even though socket.settimeout is 8

Hey so I decided to create an IRC ChatBot whose sole purpose it is to read incoming messages from Twitch Chat and if a giveaway is recognized by a keyword it's supposed to enter the giveaway by sending !enter in Chat.
I build the Bot upon this source: https://github.com/BadNidalee/ChatBot. I only changed things in the Run.py so thats the only Code I'm going to post. The unaltered ChatBot does work but it has no reconnect ability and regularly stops receiving data because the socket closes or other reasons.
All I wanted to change was make it so that the ChatBot is stable and can just stay in the IRC Chat constantly without disconnecting. I tried to achieve this by setting a timeout of 8 seconds for my socket and catching timeout exceptions that would occur and reconnect after they occur.
And all in all it does seem to work, my Bot does what it's supposed to even when alot of messages are coming in, it recognizes when a Giveaway starts and answers acordingly. IRC Server PING Messages are also handled and answered correctly. If there is no message in Chat for over 8 seconds the Exception gets thrown correctly and the Bot also reconnects correctly to IRC.
BUT heres my Problem: After seemingly random times the socket will literally just Stop working. What I find strange is it will sometimes work for 20 minutes and sometimes for an hour. It doesn't occur when special events, like lots of messages or something else happens in Chat, it really seems random. It will not timeout there's just nothing happening anymore. If I cancel the program with CTRL-C at this point the console sais the last call was "readbuffer = s.recv(1024)" But why is it not throwing a timeout exception at that point? If s.recv was called the socket should timeout if nothing is received after 8 seconds but the program just stops and there is no more output until you manually abort it.
Maybe I went about it the wrong way completely. I just want a stable 24/7-able ChatBot that scans for one simple keyword and answers with one simple !enter.
This is also my first Time programming in Python so If I broke any conventions or made any grave mistakes let me know.
The getUser Method returns the username of the line of chat that is scanned currently.
The getMessage Method returns the message of the line of chat that is scanned.
The openSocket Method opens the Socket and sends JOIN NICK PASS etc to the IRC
#!/usr/bin/python
import string
import socket
import datetime
import time
from Read import getUser, getMessage
from Socket import openSocket, sendMessage
from Initialize import joinRoom
connected = False
readbuffer = ""
def connect():
print "Establishing Connection..."
irc = openSocket()
joinRoom(irc)
global connected
connected = True
irc.settimeout(8.0)
print "Connection Established!"
return irc
while True:
s = connect()
s.settimeout(8.0)
while connected:
try:
readbuffer = s.recv(1024)
temp = string.split(readbuffer, "\n")
readbuffer = temp.pop()
for line in temp:
if "PING" in line:
s.send(line.replace("PING", "PONG"))
timern = str(datetime.datetime.now().time())
timern = timern[0:8]
print timern + " PING received"
break
user = getUser(line)
message = getMessage(line)
timern = str(datetime.datetime.now().time())
timern = timern[0:8]
print timern +" " + user + ": " + message
if "*** NEW" in message:
sendMessage(s, "!enter")
break
except socket.timeout:
connected = False
print "Socket Timed Out, Connection closed!"
break
except socket.error:
connected = False
print "Socket Error, Connection closed!"
break

I think you've missunderstood how timeout work on the socket.
s.settimeout(8.0)
Will only set s.connect(...) to timeout if it can't reach the destination host.
Further more, usually what you want to use instead if s.setblocking(0) however this alone won't help you either (probably).
Instead what you want to use is:
import select
ready = select.select([s], [], [], timeout_in_seconds)
if ready[0]:
data = s.recv(1024)
What select does is check the buffer to see if any incoming data is available, if there is you call recv() which in itself is a blocking operation. If there's nothing in the buffer select will return empty and you should avoid calling recv().
If you're running everything on *Nix you're also better off using epoll.
from select import epoll, EPOLLIN
poll = epoll()
poll.register(s.fileno(), EPOLLIN)
events = poll.poll(1) # 1 sec timeout
for fileno, event in events:
if event is EPOLLIN and fileno == s.fileno():
data = s.recv(1024)
This is a crude example of how epoll could be used.
But it's quite fun to play around with and you should read more about it

How to keep a python 3 script (Bot) running

(Not native English, sorry for probably broken English. I'm also a newbie at programming).
Hello, I'm trying to connect to a TeamSpeak server using the QueryServer to make a bot. After days of struggling with it... it works, with only 1 problem, and I'm stuck with that one.
If you need to check, this is the TeamSpeak API that I'm using: http://py-ts3.readthedocs.org/en/latest/api/query.html
And this is the summary of what actually happens in my script:
It connects.
It checks for channel ID (and it's own client ID)
It joins the channel
Script ends so it disconnects.
My question is: How can I make it doesn't disconnects? How can I make the script stay in a "waiting" state so it can read if someone types "hi bot" in the channel? All the code needed to read texts and answer to them seems easy to program, however I'm facing a problem where I can't keep the bot "running" since it closes the file as soon as it ends running the script.
More info:
I am using Python 3.4.1.
I tried learning Threading http://www.tutorialspoint.com/python/python_multithreading.htm but either M'm dumb or it doesn't work the way I though it would.
In the API there's a function named on_event that I would like to keep running all the time. The bot code should only be run once and then stay "waiting" until an event happens. How should i do that? No clue.
Code:
import ts3
import telnetlib
import time
class BotPrincipal:
def Conectar(ts3conn):
MiID = [i["client_id"] for i in ts3conn.whoami()]
ChannelToJoin = "[Pruebas] Bots"
ts3conn.on_event = BotPrincipal.EventHappened()
try:
BuscandoIDCanal = ts3conn.channelfind(pattern=ChannelToJoin)
IDCanal = [i["cid"] for i in BuscandoIDCanal]
if not IDCanal:
print("No channel found with that name")
return None
else:
MiID = str(MiID).replace("'", "")
MiID = str(MiID).replace("]", "")
MiID = str(MiID).replace("[", "")
IDCanal = str(IDCanal).replace("'", "")
IDCanal = str(IDCanal).replace("]", "")
IDCanal = str(IDCanal).replace("[", "")
print("ID de canal " + ChannelToJoin + ": " + IDCanal)
print("ID de cliente " + Nickname + ": " + MiID)
try:
print("Moving you into: " + ChannelToJoin)
ts3conn.clientmove(cid=IDCanal, clid=MiID) #entra al canal
try:
print("Asking for notifications from: " + ChannelToJoin)
ts3conn.servernotifyregister(event="channel", id_=IDCanal)
ts3conn.servernotifyregister(event="textchannel", id_=IDCanal)
except ts3.query.TS3QueryError:
print("You have no permission to use the telnet command: servernotifyregister")
print("------- Bot Listo -------")
except ts3.query.TS3QueryError:
print("You have no permission to use the telnet command: clientmove")
except ts3.query.TS3QueryError:
print("Error finding ID for " + ChannelToJoin + ". telnet: channelfind")
def EventHappened():
print("Doesn't work")
# Data needed #
USER = "thisisafakename"
PASS = "something"
HOST = "111.111.111.111"
PORT = 10011
SID = 1
if __name__ == "__main__":
with ts3.query.TS3Connection(HOST, PORT) as ts3conn:
ts3conn.login(client_login_name=USER, client_login_password=PASS)
ts3conn.use(sid=SID)
print("Connected to "+HOST)
BotPrincipal.Conectar(ts3conn)

From a quick glimpse at the API, it looks like you need to explicitly tell the ts3conn object to wait for events. There seem to be a few ways to do it, but ts3conn.recv(True) seems like the most obvious:
Blocks untill all unfetched responses have been received or forever, if recv_forever is true.
Presumably as each command comes in, it will call your on_event handler, then when you return from that it will go back to waiting forever for the next command.
I don't know if you need threads here or not, but the docs for recv_in_thread make it sound like you might:
Calls recv() in a thread. This is useful, if you used servernotifyregister and you expect to receive events.
You presumably want to get both servernotify events and also commands, and I guess the way this library is written you need threads for that? If so, just call ts3conn.recv_in_thread() instead of ts3conn.recv(True). (If you look at the source, all that does is start a background thread and call self.recv(True) on that thread.)