I'm building a telegram bot and for the start I used the structure from an example of the api wrapper. In the py script there is an infinite loop which is polling the telegram api to get new messages for the bot. And processes each new message one by one.
while True:
for update in bot.getUpdates(offset=LAST_UPDATE_ID, timeout=10):
chat_id = update.message.chat.id
update_id = update.update_id
if update.message.text:
#do things with the message \ start other functions and so on
What I foresee already now, is that with some messages\requests - i'll have a longer processing time and other messages, if the even came at the same time - will wait. For the user it will look like a delay in answering. Which boils down to a simple dependency: more user chatting = more delay.
I was thinking this: Can I have this main script bot.py run and check for new messages and each time a message arrived - this script will kickstart another script answer.py to do the processing of the message and reply.
And to start as many as needed those answer.py scripts in parallel.
I can also use bot.py to log all incoming things into DB with reference data about the user who is sending a message and then have another process processing all newly logged data and marking it as answered - but also then it should process each new entry parallel to each other.
I'm not a guru in python and is asking for some ideas and guidance on how to approach this? Thank you!
What you need are threads, or some frameworks that can handle many requests asynchronously, e.g. Twisted, Tornado, or asyncio in Python 3.4.
Here is an implementation using threads:
import threading
def handle(message):
##### do your response here
offset = None
while True:
for update in bot.getUpdates(offset=offset, timeout=10):
if update.message.text:
t = threading.Thread(target=handle, args=(update.message,))
t.start()
offset = update.update_id + 1
##### log the message if you want
This way, the call to handle() would not block, and the loop can go on handling the next message.
For more complicated situations, for example if you have to maintain states across messages from the same chat_id, I recommend taking a look at telepot, and this answer:
Handle multiple questions for Telegram bot in python
In short, telepot spawns threads for you, freeing you from worrying about the low-level details and letting you focus on the problem at hand.
Related
I created a basic process for printing server sent events as the python client receives them. I will later change the state of the module depending on the coming data. Most of the examples are for testing the python side as the event emitter. I would like to use the python as the client.
#subscriber.py
class Subscriber():
def __init__(self):
def start_listening(self):
messages = sseclient.SSEClient(SOME_URL_THAT_PUBLISHES_EVENT_STREAM)
for event in messages:
print(event)
I would like to test this but I could not find a way to mock streaming data that the sseclient uses. I tried to use responses library as below but it did not work out. I would like to at least see the prints of the given events when I test it. Is there a simple way to achieve this?
def test_subscriber():
events = ["event1", "event2", "event3", "event4"]
def sse_success(request):
headers = {'X-EventSource-Event': "new_event"}
for event in events:
time.sleep(3)
yield 200, headers, {"data": event}
responses.add_callback(
responses.GET,
SOME_URL_THAT_PUBLISHES_EVENT_STREAM,
callback=sse_success
)
sub = Subscriber()
sub.start_listening()
The subscriber class is blocking the python execution when it waits for messages. The easiest way to get around this is to run the subscriber from one process, and then in another process run a script which tests the subscriber
A better way to do this, in general, is to use an asynchronous library so that the subscriber is not blocking the python process whilst waiting for data. I have used pyzmq before - here
I have a telegram bot written in python that lets users create EC2 instances in AWS. The code is the following:
# We create the new EC2 instance for the new user
instance_id, ec2 = generateInstanceForUser(user_id)
i = ec2.Instance(id=instance_id) # instance id
i.start()
i.wait_until_running()
i.load()
time.sleep(45)
# Create account in DB
createAccountDB(user_id, username, user.mail, instance_id)
# Now that the instance and the account have been created, now settings have to be updated too
updateSettings(user_id, dictChange)
The problem is that function generateInstanceForUser(user_id) is blocking the workflow, as well as the following 5 lines (obvious, with time.sleep() function). The last function updateSettings() connects via SSH to the just created machine and do some operations. Without considering delays, this workflow works well.
HOWEVER, since I am using a Telegram bot, during this portion of the code the bot freezes during 2 minutes. As a result, if there are other users sending commands, the bot does not respond, and that is not desirable, obviously.
NOTE: functions used are held in the boto3 library.
QUESTION
Do you know some alternative to avoid workflow blocking during the execution of the given code in order to avoid bad UX with Telegram bot? Thank you.
I found myself the answer. I just encapsuled the blocking portion of the code inside another function, and used threading in order to create a parallel thread. This way, the main thread would not block and the bot would still keep working normally:
threads = []
t = threading.Thread(target=workerFunc, args=(apiKey, apiSecret, user_id, startStepValue, username, user, bot, update, leverageValue))
threads.append(t)
t.start()
I would like to create a Twilio app in python that runs in the background on my mac osx desktop and performs an action whenever a new text message arrives. I'm only interested in processing the most recently received text message.
My current solution is a polling mechanism where I have an infinite loop and in the loop I call TwilioRestClient(...).messages.list() every single time and process the first message in the returned array. I sleep for 1 second in each iteration in order to avoid taxing the processor too much.
The current implementation usually works but sometimes lags at the TwilioRestClient(...).messages.list() call for multiple seconds. I'm assuming this is because I'm using the trial version of Twilio. However, any advice on how to fix this would be greatly appreciated. The other main issue is that the implementation is extremely wasteful. The app should really be sleeping for the vast majority of the time it is running and there should be brief spurts of activity when new messages arrive.
Here's the current implementation:
def run(self):
last_message = self.get_most_recent_message_body().lower()
self.is_running = True
while self.is_running:
message = self.get_most_recent_message_body().lower()
if (message != last_message):
last_message = message
self.commander.execute(message)
sleep(1);
def get_most_recent_message_body(self):
messages = TwilioRestClient(self.TWILIO_ACCOUNT_SID, self.TWILIO_AUTH_TOKEN).messages.list()
most_recent_message = messages[0]
return most_recent_message.body
I want the app to be as responsive as possible. The optimal solution would be some kind of Observer pattern or some kind of signal use where Twilio alerts my app to "wake up" whenever a new message arrives. Is there a way to implement this?
Thanks!
Twilio developer evangelist here.
You can absolutely do that without polling. Let me explain.
When you buy a number with Twilio you can set two URLs that point to an application that you build, one for voice and one for messaging. Then, when Twilio receives a call or a text message it will make an HTTP request (a webhook) to the url you set with the details of the call or the message. Your application can respond with some XML (called TwiML) to tell Twilio what to do with the message or call.
I'm guessing you're writing a python application as you tagged this question with Python. Here's a quick guide to getting started receiving SMS messages with Python.
I try to solve problem with sending mails(or any long task) in web.py project. What I want is to start sending any mail and return the http response. But this task (sending) is taking a long time. Is there any solution?
Example:
import web
''some settings urls, etc.''
class Index:
def GET(self):
''task''
sending_mail()
return 'response'
I found many examples about async tasks but I think that if this task put to background and return 'response' it will fail.
You could get away with sending email in a separate thread (you can spawn one when you need to send an email):
import threading
threading.Thread(target=sending_email).start()
However, the all-around best (and standard) solution would be to use an asynchronous task processor, such as Celery. In your web thread, simply create a new task, and Celery will asynchronously execute it.
There is no reason why "returning response" would fail when using a message queue, unless your response depends on the email being sent prior to sending the response (but in that case, you have an architectural problem).
Moving the sending_email() task to a background queue would be the best solution. This would allow you to return the response immediately and get the results of the sending_email task later on.
Let me also suggest taking a look at RQ
It is a lightweight alternative to Celery that I find easier to get up and running. I have used it in the past for sending emails in the background and it didn't disappoint.
I'm writing a code for a simple chat client in python. I have the GUI, a php server to store strings and other data. I want to make my code capable of updating the chat (conversation Text field) each 1 second.
I post a bit of pseudo-code:
Initialize Gui
Setup Users
UserX write messageX
messageX sent to server
At this point I need something that checks each second if userX(that could be user1 or user2) has new messages to display.
If I put something like:
while True:
time.sleep(1)
checkAndDisplayNewMessages()
the GUI doesn't appear! Because at the end of the code I got a mainloop()
To resume, I want my code to give the possibility to the user to send and receive messages asynchronously! With a part of code for sending messages if the user type in any message and the other part to constantly check for new messages while the program runs.
You did not mention which GUI toolkit you are using; from mainloop() I guess it's Tk.
The answer to this question explains how to set up a recurring event. Multithreading is not required.
You need to detach the way you fetch for new messages from the main thread of your applications. That can be easily done with threads in Python, it'd look something like this:
import threading
def fetch_messages(ui):
while not ui.ready():
#this loop syncs this process with the UI.
#we don't want to start showing messages
#until the UI is not ready
time.sleep(1)
while True:
time.sleep(1)
checkAndDisplayNewMessages()
def mainlogic():
thread_messages = threading.Thread(target=fetch_messages,args=(some_ui,))
thread_messages.start()
some_ui.show() # here you can go ahead with your UI stuff
# while messages are fetched. This method should
# set the UI to ready.
This implementation will run in parallel the process to seek for more messages and also will launch the UI. It is important that the UI is sync with the process to seek for messages otherwise you'd end up with funny exceptions. This is achieved by the first loop in the fetch_messages function.