Develop Reference

How to get the next telegram messages from specific users

I'm implementing a telegram bot that will serve users. Initially, it used to get any new message sequentially, even in the middle of an ongoing session with another user. Because of that, anytime 2 or more users tried to use the bot, it used to get all jumbled up. To solve this I implemented a queue system that put users on hold until the ongoing conversation was finished. But this queue system is turning out to be a big hassle. I think my problems would be solved with just a method to get the new messages from a specific chat_id or user. This is the code that I'm using to get any new messages:
def get_next_message_result(self, update_id: int, chat_id: str):
"""
get the next message the of a given chat.
In case of the next message being from another user, put it on the queue, and wait again for
expected one.
"""
update_id += 1
link_requisicao = f'{self.url_base}getUpdates?timeout={message_timeout}&offset={update_id}'
result = json.loads(requests.get(link_requisicao).content)["result"]
if len(result) == 0:
return result, update_id # timeout
if "text" not in result[0]["message"]:
self.responder(speeches.no_text_speech, message_chat_id)
return [], update_id # message without text
message_chat_id = result[0]["message"]["chat"]["id"]
while message_chat_id != chat_id:
self.responder(speeches.wait_speech, message_chat_id)
if message_chat_id not in self.current_user_queue:
self.current_user_queue.append(message_chat_id)
print("Queuing user with the following chat_id:", message_chat_id)
update_id += 1
link_requisicao = f'{self.url_base}getUpdates?timeout={message_timeout}&offset={update_id}'
result = json.loads(requests.get(link_requisicao).content)["result"]
if len(result) == 0:
return result, update_id # timeout
if "text" not in result[0]["message"]:
self.responder(speeches.no_text_speech, message_chat_id)
return [], update_id # message without text
message_chat_id = result[0]["message"]["chat"]["id"]
return result, update_id
On another note: I use the queue so that the moment the current conversation ends, it calls the next user in line. Should I just drop the queue feature and tell the concurrent users to wait a few minutes? While ignoring any messages not from the current chat_id?

Waiting for API response in python3

(background)
I have an ERP application which is managed from a Weblogic Console. Recently we noticed that the same activities that we perform from the console can be performed using the vendor provided REST API calls. So we wanted to utilize this approach programatically and try to build some automations.
This is the page from where we can control one of the instance ConsoleImage
The same button acts as Stop and Start to manage the start and stop instance.
Both the start and stop have different API calls which makes sense.
The complete API doc is at : https://docs.oracle.com/cd/E61420_01/doc.92/e80710/smcrestapis.htm#BABFHBJI
(Now)
I wrote a program in python using the request method to call these APIs and it works fine.
The API response can take anywhere between 20 to 30 seconds when I use the stopInstance API
And normally takes 60 to 90 seconds when I use the startInstance API, but if there is an issue when starting the instance it takes more than 300 seconds and goes into indefinate wait.
My problem is, while starting an instance I want to wait maximum only for 100 seconds for the response. If it takes more than 100 seconds the program should display a message like "Instance was not able to start in 100 seconds"
This is my program. I am taking input from a text file and all the values present there have been verified.
import requests
import json
import importlib.machinery
import importlib.util
import numpy
import time
import sys
loader = importlib.machinery.SourceFileLoader('SM','sm_details.txt')
spec = importlib.util.spec_from_loader(loader.name, loader)
mod = importlib.util.module_from_spec(spec)
loader.exec_module(mod)
username = str(mod.username)
password = str(mod.password)
hostname = str(mod.servermanagerHostname)
portnum = str(mod.servermanagerPort)
instanceDetails = numpy.array(mod.instanceName)
authenticationAPI = "http://"+hostname+":"+portnum+"/manage/mgmtrestservice/authenticate"
startInstanceAPI = "http://"+hostname+":"+portnum+"/manage/mgmtrestservice/startinstance"
headers = {
'Content-Type':'application/json',
'Cache-Control':'no-cache',
}
data = {}
data['username']= username
data['password']= password
instanceNameDict = {'instanceName':''}
#Authentication request and storing token
response = requests.post(authenticationAPI, data=json.dumps(data), headers=headers)
token = response.headers['TOKEN']
head2 = {}
head2['TOKEN']=token
def start(instance):
print(f'\nTrying to start instance : '+instance['instanceName'])
startInstanceResponse = requests.post(startInstanceAPI,data=json.dumps(instance), headers=head2) #this is where the program is stuck and it does not move to the time.sleep step
time.sleep(100)
if startInstanceResponse.status_code == 200:
print('Instance '+instance['instanceName']+' started.')
else:
print('Could not start instance in 100 seconds')
sys.exit(1)

I would suggest you to use the timeout parameter in requests:
requests.post(startInstanceAPI,data=json.dumps(instance), headers=head2, timeout=100.0)
You can tell Requests to stop waiting for a response after a given
number of seconds with the timeout parameter. Nearly all production
code should use this parameter in nearly all requests. Failure to do
so can cause your program to hang indefinitely.
Source
Here's the requests timeout documentation, you will also find more details in there and Exception handling.

receiving and sending mavlink messages using pymavlink library

I have created a proxy between QGC(Ground Control Station) and vehicle in Python. Here is the code:
gcs_conn = mavutil.mavlink_connection('tcpin:localhost:15795')
gcs_conn.wait_heartbeat()
print("Heartbeat from system (system %u component %u)" %(gcs_conn.target_system, gcs_conn.target_system))
vehicle = mavutil.mavlink_connection('tcp:localhost:5760')
vehicle.wait_heartbeat() # recieving heartbeat from the vehicle
print("Heartbeat from system (system %u component %u)" %(vehicle.target_system, vehicle.target_system))
while True:
gcs_msg = gcs_conn.recv_match()
if gcs_msg == None:
pass
else:
vehicle.mav.send(gcs_msg)
print(gcs_msg)
vcl_msg = vehicle.recv_match()
if vcl_msg == None:
pass
else:
gcs_conn.mav.send(vcl_msg)
print(vcl_msg)
I need to receive the messages from the QGC and then forward them to the vehicle and also receive the messages from the vehicle and forward them to the QGC.
When I run the code I get this error.
is there any one who can help me?

If you print your message before sending you'll notice it always fails when you try to send a BAD_DATA message type.
So this should fix it (same for vcl_msg):
if gcs_msg and gcs_msg.get_type() != 'BAD_DATA':
vehicle.mav.send(gcs_msg)
PD: I noticed that you don't specify tcp as input or output, it defaults to input. Than means both connections are inputs. I recommend setting up the GCS connection as output:
gcs_conn = mavutil.mavlink_connection('tcp:localhost:15795', input=False)
https://mavlink.io/en/mavgen_python/#connection_string

For forwarding MAVLink successfully a few things need to happen. I'm assuming you need a usable connection to a GCS, like QGroundControl or MissionPlanner. I use QGC, and my design has basic testing with it.
Note that this is written with Python3. This snippet is not tested, but I have a (much more complex) version tested and working.
from pymavlink import mavutil
import time
# PyMAVLink has an issue that received messages which contain strings
# cannot be resent, because they become Python strings (not bytestrings)
# This converts those messages so your code doesn't crash when
# you try to send the message again.
def fixMAVLinkMessageForForward(msg):
msg_type = msg.get_type()
if msg_type in ('PARAM_VALUE', 'PARAM_REQUEST_READ', 'PARAM_SET'):
if type(msg.param_id) == str:
msg.param_id = msg.param_id.encode()
elif msg_type == 'STATUSTEXT':
if type(msg.text) == str:
msg.text = msg.text.encode()
return msg
# Modified from the snippet in your question
# UDP will work just as well or better
gcs_conn = mavutil.mavlink_connection('tcp:localhost:15795', input=False)
gcs_conn.wait_heartbeat()
print(f'Heartbeat from system (system {gcs_conn.target_system} component {gcs_conn.target_system})')
vehicle = mavutil.mavlink_connection('tcp:localhost:5760')
vehicle.wait_heartbeat()
print(f'Heartbeat from system (system {vehicle.target_system} component {vehicle.target_system})')
while True:
# Don't block for a GCS message - we have messages
# from the vehicle to get too
gcs_msg = gcs_conn.recv_match(blocking=False)
if gcs_msg is None:
pass
elif gcs_msg.get_type() != 'BAD_DATA':
# We now have a message we want to forward. Now we need to
# make it safe to send
gcs_msg = fixMAVLinkMessageForForward(gcs_msg)
# Finally, in order to forward this, we actually need to
# hack PyMAVLink so the message has the right source
# information attached.
vehicle.mav.srcSystem = gcs_msg.get_srcSystem()
vehicle.mav.srcComponent = gcs_msg.get_srcComponent()
# Only now is it safe to send the message
vehicle.mav.send(gcs_msg)
print(gcs_msg)
vcl_msg = vehicle.recv_match(blocking=False)
if vcl_msg is None:
pass
elif vcl_msg.get_type() != 'BAD_DATA':
# We now have a message we want to forward. Now we need to
# make it safe to send
vcl_msg = fixMAVLinkMessageForForward(vcl_msg)
# Finally, in order to forward this, we actually need to
# hack PyMAVLink so the message has the right source
# information attached.
gcs_conn.mav.srcSystem = vcl_msg.get_srcSystem()
gcs_conn.mav.srcComponent = vcl_msg.get_srcComponent()
gcs_conn.mav.send(vcl_msg)
print(vcl_msg)
# Don't abuse the CPU by running the loop at maximum speed
time.sleep(0.001)
Notes
Make sure your loop isn't being blocked
The loop must quickly check if a message is available from one connection or the other, instead of waiting for a message to be available from a single connection. Otherwise a message on the other connection will not go through until the blocking connection has a message.
Check message validity
Check that you actually got a valid message, as opposed to a BAD_DATA message. Attempting to send BAD_DATA will crash
Make sure the recipient gets the correct information about the sender
By default PyMAVLink, when sending a message, will encode YOUR system and component IDs (usually left at zero), instead of the IDs from the message. A GCS receiving this may be confused (ie, QGC) and not properly connect to the vehicle (despite showing the messages in MAVLink inspector).
This is fixed by hacking PyMAVLink such that your system and component IDs match the forwarded message. This can be revered after the message is sent if necessary. See the example to see how I did it.
Loop update rate
It's important that the update rate is fast enough to handle high traffic conditions (especially, say, for downloading params), but it shouldn't peg out the CPU either. I find that a 1000hz update rate works well enough.

Fix "Nickname is already in use" in my Python IRC client

Error:
NOTICE Auth :*** Looking up your hostname...
433 * testbot:Nickname is already in use.
NOTICE Auth :*** Could not resolve your hostname: Request timed out; using your IP address () instead.
451 837AAAABB JOIN :You have not registered
The script works fine, the only issue I'm having is when another user has the same name and so the bot won't join, how can I fix this?
#IRC Info, Where the bot connects too
server="Server"
botnick="testbot"
channel="#test"
What I have tried:
Google, YouTube, Looking at other github IRC bot's and stackoverflow.
One idea I had was to use an random string generator, so if the name "testbot" was taken the script would generator something random and try again. I'm unsure how I would add this.

It is difficult to help you without seeing some code that you tried. Or any code at all.
The overall idea would be to detect when an incoming message is a 433 (aka ERR_NICKNAMEINUSE), and then send a new NICK command with a new nickname; and try again until you find a free nickname.
Pseudocode:
MAINNICK = 'testbot'
nick_suffix = 1
send_msg('NICK {}'.format(MAINNICK))
while True:
msg = recv_msg()
if msg.split(' ')[1] == '433':
send_msg('NICK {}{}'.format(MAINNICK, nick_suffix))
nick_suffix += 1

This answer assumes that the nick is registered by you.
When receiving ERR_NICKNAMEINUSE (433), send REGAIN to nickserv. Personally I also take care to not send REGAIN more than 3 times in 30 seconds, otherwise I disconnect and reconnect because something else is wrong.
REGAIN YourRegisteredNick YourPassword
Once in a while, NOTICE with a second argument containing can not regain your nickname is then received. This indicates a REGAIN failure. The only way I am aware of to handle this error is to disconnect, reconnect, and rejoin channels.

irc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def ircwrite(message):
global irc
irc.send(str(message).encode('utf-8'))
botnick = "ME_Number-1" # Nickname of the bot
InUse_alt = "Me2" # if Nickname: "('botnick')" is already in use, A.K.A. 433, uses this alternative option instead
while 1:
text = irc.recv(2048).decode('utf-8')
print(text)
if "433" in text:
print("Bot's nick IN USE or has been regged by another; switching to: "+ (InUse_alt))
if text.find("433") != -1:
ircwrite("NICK "+ InUse_alt +" \r\n")
---
Something like that would work in most situations, but it can be set off by: someone saying: "433"; trying to work on that.

Send one nick but not both nick1 and nick2. To send new nick2
send_msg(f'NICK {}'.format(nick2_suffix))
In my case:
socket.socket.send(f'NICK {NICK2}')
Another way. In entry box.
/NICK NICK2

Using Tweepy to listen to stream and search for tweets. How to stop previous search and only listen for new stream?

I'm using Flask and Tweepy to search for live tweets. On the front-end I have a user text input, and button called "Search". Ideally, when a user gives a search-term into the input and clicks the "Search" button, the Tweepy should listen for the new search-term and stop the previous search-term stream. When the "Search" button is clicked it executes this function:
#app.route('/search', methods=['POST'])
# gets search-keyword and starts stream
def streamTweets():
search_term = request.form['tweet']
search_term_hashtag = '#' + search_term
# instantiate listener
listener = StdOutListener()
# stream object uses listener we instantiated above to listen for data
stream = tweepy.Stream(auth, listener)
if stream is not None:
print "Stream disconnected..."
stream.disconnect()
stream.filter(track=[search_term or search_term_hashtag], async=True)
redirect('/stream') # execute '/stream' sse
return render_template('index.html')
The /stream route that is executed in the second to last line in above code is as follows:
#app.route('/stream')
def stream():
# we will use Pub/Sub process to send real-time tweets to client
def event_stream():
# instantiate pubsub
pubsub = red.pubsub()
# subscribe to tweet_stream channel
pubsub.subscribe('tweet_stream')
# initiate server-sent events on messages pushed to channel
for message in pubsub.listen():
yield 'data: %s\n\n' % message['data']
return Response(stream_with_context(event_stream()), mimetype="text/event-stream")
My code works fine, in the sense that it starts a new stream and searches for a given term whenever the "Search" button is clicked, but it does not stop the previous search. For example, if my first search term was "NYC" and then I wanted to search for a different term, say "Los Angeles", it will give me results for both "NYC" and "Los Angeles", which is not what I want. I want just "Los Angeles" to be searched. How do I fix this? In other words, how do I stop the previous stream? I looked through other previous threads, and I know I have to use stream.disconnect(), but I'm not sure how to implement this in my code. Any help or input would be greatly appreciated. Thanks so much!!

Below is some code that will cancel old streams when a new stream is created. It works by adding new streams to a global list, and then calling stream.disconnect() on all streams in the list whenever a new stream is created.
diff --git a/app.py b/app.py
index 1e3ed10..f416ddc 100755
--- a/app.py
+++ b/app.py
## -23,6 +23,8 ## auth.set_access_token(access_token, access_token_secret)
app = Flask(__name__)
red = redis.StrictRedis()
+# Add a place to keep track of current streams
+streams = []
#app.route('/')
def index():
## -32,12 +34,18 ## def index():
#app.route('/search', methods=['POST'])
# gets search-keyword and starts stream
def streamTweets():
+ # cancel old streams
+ for stream in streams:
+ stream.disconnect()
+
search_term = request.form['tweet']
search_term_hashtag = '#' + search_term
# instantiate listener
listener = StdOutListener()
# stream object uses listener we instantiated above to listen for data
stream = tweepy.Stream(auth, listener)
+ # add this stream to the global list
+ streams.append(stream)
stream.filter(track=[search_term or search_term_hashtag],
async=True) # make sure stream is non-blocking
redirect('/stream') # execute '/stream' sse
What this does not solve is the problem of session management. With your current setup a search by one user will affect the searches of all users. This can be avoided by giving your users some identifier and storing their streams along with their identifier. The easiest way to do this is likely to use Flask's session support. You could also do this with a requestId as Pierre suggested. In either case you will also need code to notice when a user has closed the page and close their stream.

Disclaimer: I know nothing about Tweepy, but this appears to be a design issue.
Are you trying to add state to a RESTful API? You may have a design problem.
As JRichardSnape answered, your API shouldn't be the one taking care of canceling a request; it should be done in the front-end. What I mean here is in the javascript / AJAX / etc calling this function, add another call, to the new function
#app.route('/cancelSearch', methods=['POST'])
With the "POST" that has the search terms. So long as you don't have state, you can't really do this safely in an async call: Imagine someone else makes the same search at the same time then canceling one will cancel both (remember, you don't have state so you don't know who you're canceling). Perhaps you do need state with your design.
If you must keep using this and don't mind breaking the "stateless" rule, then add a "state" to your request. In this case it's not so bad because you could launch a thread and name it with the userId, then kill the thread every new search
def streamTweets():
search_term = request.form['tweet']
userId = request.form['userId'] # If your limit is one request per user at a time. If multiple windows can be opened and you want to follow this limit, store userId in a cookie.
#Look for any request currently running with this ID, and cancel them
Alternatively, you could return a requestId, which you would then keep in the front-end can call cancelSearch?requestId=$requestId. In cancelSearch, you would have to find the pending request (sounds like that's in tweepy since you're not using your own threads) and disconnect it.
Out of curiosity I just watched what happens when you search on Google, and it uses a GET request. Have a look (debug tools -> Network; then enter some text and see the autofill). Google uses a token sent with every request (every time you type something)). It doesn't mean it's used for this, but that's basically what I described. If you don't want a session, then use a unique identifier.

Well I solved it by using timer method But still I'm looking for pythonic way.
from streamer import StreamListener
def stream():
hashtag = input
#assign each user an ID ( for pubsub )
StreamListener.userid = random_user_id
def handler(signum, frame):
print("Forever is over")
raise Exception("end of time")
def main_stream():
stream = tweepy.Stream(auth, StreamListener())
stream.filter(track=track,async=True)
redirect(url_for('map_stream'))
def close_stream():
# this is for closing client list in redis but don't know it's working
obj = redis.client_list(tweet_stream)
redis_client_list = obj[0]['addr']
redis.client_kill(redis_client_list)
stream = tweepy.Stream(auth, StreamListener())
stream.disconnect()
import signal
signal.signal(signal.SIGALRM, handler)
signal.alarm(300)
try:
main_stream()
except Exception:
close_stream()
print("function terminate")