Develop Reference

Return multiple times from one api call in Flask Restful

I want to call a generate() function and send a user a message, but then continue executing a function.
#application.route("/api/v1.0/gen", methods=['POST'])
def generate():
return "Your id for getting the generated data is 'hgF8_dh4kdsRjdr'"
main() #generate a data
return "Successfully generated something. Use your id to get the data"
I understand that this is not a correct way of returning, but I hope you get the idea of what I am trying to accomplish. Maybe Flask has some build-in method to return multiple times from one api call?

Basically, what are you describing is called Server-Sent Events (aka SSE)
The difference of this format, that they returned an 'eventstream' Response type instead of usual JSON/plaintext
And if you want to use it with python/flask, you need generators.
Small code example (with GET request):
#application.route("/api/v1.0/gen", methods=['GET'])
def stream():
def eventStream():
text = "Your id for getting the generated data is 'hgF8_dh4kdsRjdr'"
yield str(Message(data = text, type="message"))
main()
text = "Successfully generated something. Use your id to get the data"
yield str(Message(data = text, type="message"))
resp.headers['Content-Type'] = 'text/event-stream'
resp.headers['Cache-Control'] = 'no-cache'
resp.headers['Connection'] = 'keep-alive'
return resp
Message class you can find here: https://gist.github.com/Alveona/b79c6583561a1d8c260de7ba944757a7
And of course, you need specific client that can properly read such responses.
postwoman.io supports SSE at Real-Time tab

How to get payload value from quick replies on Facebook Messenger

I'm trying to get the payload from the quick replies that I'm using to create a chatbot on Messenger, but I keep getting this error: KeyError: 'quick_reply'. I am using python to create bot. I've tried everything imaginable such as:
#app.route('/', methods=['POST'])
def webhook():
data = request.get_json()
log(data)
if data["object"] == "page":
for entry in data["entry"]:
for messaging_event in entry["messaging"]:
if messaging_event.get("message"):
recieved(messaging_event)
message_text = messaging_event["message"]["quick_reply"]["payload"]
if messaging_event.get("delivery"):
pass
if messaging_event.get("optin"):
pass
if messaging_event.get("postback"):
pass
return "ok", 200
But each one keeps giving me the same problem. I've looked online and at docs but there seems to be no answer. Any insights?

When a Quick Reply is tapped, a text message will be sent to your webhook Message Received Callback. The text of the message will correspond to the title of the Quick Reply. The message object will also contain a field named quick_reply containing the payload data on the Quick Reply.
so quick reply payload will be like message.quick_reply.payload

tweepy wait_on_rate_limit not working

So, first off, I realize there's a number of questions regarding handling the twitter rate limits. I have no idea why, but none of the ones's I've found so far work for me.
I'm using tweepy. I'm trying to get a list of all the followers of the followers of a user. As expected, I can't pull everything down all at once due to twitter's rate limits. I have tweepy v 3.5 installed and thus am referring to http://docs.tweepy.org/en/v3.5.0/api.html. To get the list of followers of the originating user I use:
auth = tweepy.OAuthHandler(consumer_key,consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
followerIDs = []
for page in tweepy.Cursor(api.followers_ids, screen_name=originatingUser, wait_on_rate_limit = True, wait_on_rate_limit_notify = True).pages():
followerIDs.extend(page)
followers = api.lookup_users(follower)
This works a for a bit but quickly turns into:
tweepy.error.TweepError: [{u'message': u'Rate limit exceeded', u'code': 88}]
My theory, would then to retrieve the followers of each user for each followerID using something like this:
for followerID in followerIDs:
for page in tweepy.Cursor(api.followers_ids, id=followerID, wait_on_rate_limit = True, wait_on_rate_limit_notify = True).pages():
followerIDs.extend(page)
The other problem I have is when I'm trying to look up the user names. For this, It use the grouper function from itertools to break the followers up into groups of 100 (api.lookup_users can only accept 100 id's at a time) and use
followerIDs = grouper(followerIDs,100)
for followerGroup in followerIDs:
followerGroup=filter(None, followerGroup)
followers = api.lookup_users(followerGroup,wait_on_rate_limit = True)
for follower in followers:
print (originatingUser + ", " + str(follower.screen_name))
That gets a different error, namely:
TypeError: lookup_users() got an unexpected keyword argument 'wait_on_rate_limit'
which I'm finding confusing, becuase the tweepy api suggests that that should be an accepted argument.
Any ideas as to what I'm doing wrong?
Cheers
Ben.

I know this might be a little late, but here goes.
You pass the wait_on_rate_limit argument in the Cursor constructor, while the tweepy documentation states that it should be passed on the API() constructor.

The wait_on_rate_limit argument is to be passed in the API() constructor.
In your code it would look like:
api = tweepy.API(auth,wait_on_rate_limit=True)
There's also another argument wait_on_rate_limit_notify, which informs you when tweepy is waiting for your rate limit to refresh. Adding both would finally make the line:
api = tweepy.API(auth,wait_on_rate_limit=True,wait_on_rate_limit_notify=True)

There is a rate limit for twitter API as mentioned here: https://dev.twitter.com/rest/public/rate-limiting
The quick solution to pass this could be catching the rate limit error and sleeping your application for a while then continue where you left.
pages = tweepy.Cursor(api.followers_ids, id=followerID).pages()
while True:
try:
page = pages.next()
followerIDs.extend(page)
except TweepError:
time.sleep(60 * 15)
continue
except StopIteration:
break
should do the trick. Not sure if this will work as you expect but the basic idea is this.

Reading page's messages with Python Facebook SDK

Basically i need to get all messages of a page using facebook SDK in python.
Following some tutorial i arrived to this point:
import facebook
def main():
cfg = {
"page_id" : "MY PAGE ID",
"access_token" : "LONG LIVE ACCESS TOKEN"
}
api = get_api(cfg)
msg = "Hre"
status = api.put_wall_post(msg) #used to post to wall message Hre
x = api.get_object('/'+str(MY PAGE ID)+"/conversations/") #Give actual conversations
def get_api(cfg):
graph = facebook.GraphAPI(cfg['access_token'])
resp = graph.get_object('me/accounts')
page_access_token = None
for page in resp['data']:
if page['id'] == cfg['page_id']:
page_access_token = page['access_token']
graph = facebook.GraphAPI(page_access_token)
return graph
if __name__ == "__main__":
main()
The first problem is that api.get_object('/'+str(MY PAGE ID)+"/conversations/")returns a dictionary containing many informations, but what i would like to see is the messages they sent to me, while for now it print the user id that sent to me a message.
The output look like the following:
{u'paging': {u'next': u'https://graph.facebook.com/v2.4/571499452991432/conversations?access_token=Token&limit=25&until=1441825848&__paging_token=enc_AdCqaKAP3e1NU9MGSsvSdzDPIIDtB2ZCe2hCYfk7ft5ZAjRhsuVEL7eFYOOCdQ8okvuhZA5iQWaYZBBbrZCRNW8uzWmgnKGl69KKt4catxZAvQYCus7gZDZD', u'previous': u'https://graph.facebook.com/v2.4/571499452991432/conversations?access_token=token&limit=25&since=1441825848&__paging_token=enc_AdCqaKAP3e1NU9MGSsvSdzDPIIDtB2ZCe2hCYfk7ft5ZAjRhsuVEL7eFYOOCdQ8okvuhZA5iQWaYZBBbrZCRNW8uzWmgnKGl69KKt4catxZAvQYCus7gZDZD&__previous=1'}, u'data': [{u'link': u'/communityticino/manager/messages/?mercurythreadid=user%3A1055476438&threadid=mid.1441825847634%3Af2e0247f54f5c4d222&folder=inbox', u'id': u't_mid.1441825847634:f2e0247f54f5c4d222', u'updated_time': u'2015-09-09T19:10:48+0000'}]}
which is basically paging and data.
Given this is there a way to read the conversation?

In order to get the messages content you need first to request the single messages in the conversation, accessible with the 'id' field in the dictionary you copied, result of
x = api.get_object('/'+str(MY PAGE ID)+"/conversations/") #Give actual conversations
you can request the messages in the conversation by calling
msg = api.get_object('/'+<message id>)
Here it gets tricky, because following the graph api documentation you should receive back a dictionary with ALL the possible fields, including the 'message' (content) field. The function however returns only the fields 'created_time' and 'id'.
Thanks to this other question Request fields in Python Facebook SDK I found that you can request for those fields by adding a dict with such fields specified in the arguments of the graph.get_object() function. As far as I know this is undocumented in the facebook sdk reference for python.
The correct code is
args = {'fields' : 'message'}
msg = api.get_object('/'+<message id>, **args)
Similar question: Read facebook messages using python sdk

Using Tweepy to listen to stream and search for tweets. How to stop previous search and only listen for new stream?

I'm using Flask and Tweepy to search for live tweets. On the front-end I have a user text input, and button called "Search". Ideally, when a user gives a search-term into the input and clicks the "Search" button, the Tweepy should listen for the new search-term and stop the previous search-term stream. When the "Search" button is clicked it executes this function:
#app.route('/search', methods=['POST'])
# gets search-keyword and starts stream
def streamTweets():
search_term = request.form['tweet']
search_term_hashtag = '#' + search_term
# instantiate listener
listener = StdOutListener()
# stream object uses listener we instantiated above to listen for data
stream = tweepy.Stream(auth, listener)
if stream is not None:
print "Stream disconnected..."
stream.disconnect()
stream.filter(track=[search_term or search_term_hashtag], async=True)
redirect('/stream') # execute '/stream' sse
return render_template('index.html')
The /stream route that is executed in the second to last line in above code is as follows:
#app.route('/stream')
def stream():
# we will use Pub/Sub process to send real-time tweets to client
def event_stream():
# instantiate pubsub
pubsub = red.pubsub()
# subscribe to tweet_stream channel
pubsub.subscribe('tweet_stream')
# initiate server-sent events on messages pushed to channel
for message in pubsub.listen():
yield 'data: %s\n\n' % message['data']
return Response(stream_with_context(event_stream()), mimetype="text/event-stream")
My code works fine, in the sense that it starts a new stream and searches for a given term whenever the "Search" button is clicked, but it does not stop the previous search. For example, if my first search term was "NYC" and then I wanted to search for a different term, say "Los Angeles", it will give me results for both "NYC" and "Los Angeles", which is not what I want. I want just "Los Angeles" to be searched. How do I fix this? In other words, how do I stop the previous stream? I looked through other previous threads, and I know I have to use stream.disconnect(), but I'm not sure how to implement this in my code. Any help or input would be greatly appreciated. Thanks so much!!

Below is some code that will cancel old streams when a new stream is created. It works by adding new streams to a global list, and then calling stream.disconnect() on all streams in the list whenever a new stream is created.
diff --git a/app.py b/app.py
index 1e3ed10..f416ddc 100755
--- a/app.py
+++ b/app.py
## -23,6 +23,8 ## auth.set_access_token(access_token, access_token_secret)
app = Flask(__name__)
red = redis.StrictRedis()
+# Add a place to keep track of current streams
+streams = []
#app.route('/')
def index():
## -32,12 +34,18 ## def index():
#app.route('/search', methods=['POST'])
# gets search-keyword and starts stream
def streamTweets():
+ # cancel old streams
+ for stream in streams:
+ stream.disconnect()
+
search_term = request.form['tweet']
search_term_hashtag = '#' + search_term
# instantiate listener
listener = StdOutListener()
# stream object uses listener we instantiated above to listen for data
stream = tweepy.Stream(auth, listener)
+ # add this stream to the global list
+ streams.append(stream)
stream.filter(track=[search_term or search_term_hashtag],
async=True) # make sure stream is non-blocking
redirect('/stream') # execute '/stream' sse
What this does not solve is the problem of session management. With your current setup a search by one user will affect the searches of all users. This can be avoided by giving your users some identifier and storing their streams along with their identifier. The easiest way to do this is likely to use Flask's session support. You could also do this with a requestId as Pierre suggested. In either case you will also need code to notice when a user has closed the page and close their stream.

Disclaimer: I know nothing about Tweepy, but this appears to be a design issue.
Are you trying to add state to a RESTful API? You may have a design problem.
As JRichardSnape answered, your API shouldn't be the one taking care of canceling a request; it should be done in the front-end. What I mean here is in the javascript / AJAX / etc calling this function, add another call, to the new function
#app.route('/cancelSearch', methods=['POST'])
With the "POST" that has the search terms. So long as you don't have state, you can't really do this safely in an async call: Imagine someone else makes the same search at the same time then canceling one will cancel both (remember, you don't have state so you don't know who you're canceling). Perhaps you do need state with your design.
If you must keep using this and don't mind breaking the "stateless" rule, then add a "state" to your request. In this case it's not so bad because you could launch a thread and name it with the userId, then kill the thread every new search
def streamTweets():
search_term = request.form['tweet']
userId = request.form['userId'] # If your limit is one request per user at a time. If multiple windows can be opened and you want to follow this limit, store userId in a cookie.
#Look for any request currently running with this ID, and cancel them
Alternatively, you could return a requestId, which you would then keep in the front-end can call cancelSearch?requestId=$requestId. In cancelSearch, you would have to find the pending request (sounds like that's in tweepy since you're not using your own threads) and disconnect it.
Out of curiosity I just watched what happens when you search on Google, and it uses a GET request. Have a look (debug tools -> Network; then enter some text and see the autofill). Google uses a token sent with every request (every time you type something)). It doesn't mean it's used for this, but that's basically what I described. If you don't want a session, then use a unique identifier.

Well I solved it by using timer method But still I'm looking for pythonic way.
from streamer import StreamListener
def stream():
hashtag = input
#assign each user an ID ( for pubsub )
StreamListener.userid = random_user_id
def handler(signum, frame):
print("Forever is over")
raise Exception("end of time")
def main_stream():
stream = tweepy.Stream(auth, StreamListener())
stream.filter(track=track,async=True)
redirect(url_for('map_stream'))
def close_stream():
# this is for closing client list in redis but don't know it's working
obj = redis.client_list(tweet_stream)
redis_client_list = obj[0]['addr']
redis.client_kill(redis_client_list)
stream = tweepy.Stream(auth, StreamListener())
stream.disconnect()
import signal
signal.signal(signal.SIGALRM, handler)
signal.alarm(300)
try:
main_stream()
except Exception:
close_stream()
print("function terminate")