I'm trying to reach out an event to recognize whenever specific user tweets , so im using stream.filter in tweepy , just like
class IDPrinter(tweepy.StreamingClient):
def on_tweet(self, tweet):
print(tweet)
printer = IDPrinter("bearer")
printer.sample(expansions=['author_id'],user_fields=['1234']) #consider 1234 user_id to follow
but Stream encountered HTTP error: 400 ,what am i missing? im having api v2 and allowing OAuth v2 as well
To be able to filter streaming data you need to give rules initially. So if you want to stream tweets of user with id 1234 you need give a rule for that, then when you start streaming, tweets that satisfy your rule will be retrieved.
For more information about adding rules on tweepy : StreamRule
For more about Twitter Api v2 Filtered Stream : How to build a rule
Related
I'm quite new to python and coding in general, and I'm having difficulty understanding how to interact with streamed data from the Twitter API using Tweepy.
Here's my example code which prints out any new tweet that the specified user makes.
import tweepy
auth = tweepy.OAuthHandler("****", "****")
auth.set_access_token("****", "****")
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)
class MyStreamListener(tweepy.StreamListener):
def on_status(self, status):
print (status.text)
myStreamListener = MyStreamListener()
myStream = tweepy.Stream(auth = api.auth, listener = myStreamListener)
myStream.filter(follow=['user_id_goes_here']))
If I want to do something such as check if a certain word exists inside each tweet as they are made, I do not know how, given it is a constant stream of data.
How does one analyze each tweet as it is delivered and parse it?
The tweepy documentation on streaming is very limited, but it does say
This page aims to help you get started using Twitter streams with Tweepy by offering a first walk through. Some features of Tweepy streaming are not covered here. See streaming.py in the Tweepy source code.
so searching for that file in the tweepy github repository we find
https://github.com/tweepy/tweepy/blob/master/tweepy/streaming.py
There you can find the method on_status and see that status should be an instance of the class Status
Looking at the twitter API documentation reveals that
Tweets are also known as “status updates.”
Unfortunately, looking at the source code for Status or the tweepy documentation does not yield much information.
Looking at the twitter API documentation for tweet
https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/tweet
We should expect a field called text that should be the tweet actual text
Another thing we can try is just using a breakpoint and then looking at the variable status using the debugger in order to see what fields it has (this is done a lot of times in python due to it's dynamic nature)
I believe I have found what I am looking for, the analysis of the status needs to happen within the on_status method, for example:
class MyStreamListener(tweepy.StreamListener):
def on_status(self, status):
if keyword in status.text:
print ("keyword found")
I am using this function to collect the usernames of the accounts followed by a user. The problem is that is very unefficient because it collects the whole user object. Is there a way to collect only the username or to do it more efficiently?
def collect_followed_by_user(api,target_user):
all_followed = []
ids = []
try:
for page in tweepy.Cursor(api.friends_ids, screen_name=target_user).pages():
ids.extend(page)
for identifier in ids:
name = api.get_user(identifier).screen_name
all_followed.append(name)
except:
exit(1)
return all_followed
The function uses the Twitter API with tweepy.
As I see, tweepy is based on Twitter API (without surprise!). As it is in code of tweepy get_user() used this reference to get user of Twitter API.
It is not possible to get only names or increase efficiency with tweepy. But you could try users/lookup directly with Twitter API or find the same method in another lib.
Yesterday I wrote a twitter bot with Python that takes the most recent tweet from Donald Trump, translates it over in Google Translate 45 times, and tweets back at him the final translation in English. Everything works, except the fact that I now need to add some sort of "listener" at the beginning of the code to automatically detect when he tweets so that the rest of the code can do its magic. I've been looking over the internet for some time now and I can't seem to find any sort of event handler that would allow the script to detect when he tweets. So that's why I've come to you guys. Is there any way to use Tweepy or other Python libraries to actively detect when someone tweets? I'll include my code so you guys can see what I want to happen exactly when he does Tweet. It's annotated so hopefully it's not too complicated to understand. Thanks!
import tweepy
from googletrans import Translator
#Keys for accessing the Twitter API
consumer_key = 'PLACEHOLDER'
consumer_secret = 'PLACEHOLDER'
access_token = 'PLACEHOLDER'
access_token_secret = 'PLACEHOLDER'
#Setting up authentification
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
#Scrapes my timeline for DT's latest tweet. I want this to be done AUTOMATICALLY when he tweets.
tweet = api.user_timeline(screen_name = 'realDonaldTrump', count = 1, include_rts = False, tweet_mode = 'extended')
#Translates the text from the .json file that is pulle from the previous line using the Google translate library.
for status in tweet:
translator = Translator()
translation = translator.translate(translation.text, 'mn')
translation = translator.translate(status._json["full_text"], 'ja')
#There are more translations in the actual code, but to reduce the length and complexity, I've taken those out. They don't matter to the specific question.
#Include his handle in a message so that the tweet is tweeted back at him once the translation is complete.
message = ('#realDonaldTrump', translation.text)
#Tweets the message back at him under that specific tweet using it's ID.
send = api.update_status(message, status._json["id"])
I just want the code to be able to scrape my timelines for one of DT's tweets in real time. Thanks!
To Automate your script, you might need to push it into a production server and then create a cron job that will run the script at given regular intervals. To run it locally, I use 1. NGROK - Exposes your localhost addresses and ports to the outside world ie it gives you a way to reach your localhost from the internet, and 2. Invictify - This allows you to run your scripts at a schedule. Also, the script as it is will need a web service to be triggered. Use Flask or FastAPI to create endpoints that call the script when triggered.
I'd like send query to twitter search API using the tweet id but it seems you cannot search a tweet by having its id (maybe because you don't need to search it if you already have the id). For example imagine we have a tweet https://twitter.com/great_watches/status/643389532105256961 and we want to send 643389532105256961 to the search API to see if the tweet is available on the search api or not.
I need it because I just want to compare twitter search api with twitter streaming api.
I have a python script which is listening to the stream for some keywords and whenever a tweet is comming I like to search it on twitter search api to see if it is available there also or not. meaningless huh?
You can't compare the the Search API to the Streaming API the way you're doing it due to the fact they're both retrieving different types of information.
From the Search API docs:
The Twitter Search API is part of Twitter’s v1.1 REST API. It allows
queries against the indices of recent or popular Tweets and behaves
similarily to, but not exactly like the Search feature available in
Twitter mobile or web clients, such as Twitter.com search.
Before getting involved, it’s important to know that the Search API is
focused on relevance and not completeness. This means that some Tweets
and users may be missing from search results. If you want to match for
completeness you should consider using a Streaming API instead.
Here's to explain the scenario based on the information you've given.
You're streaming the word python and you find a match.
You instantly take that match and look for it on search API
The issue with that is by the time you're going from Streaming API (which is in real time) and you're looking to find the same one on the search API you'll get in conflict where more relevant tweets and popular one that will supersede it.
You'll need to redefine the query sent to the search API to include that exact same one (i.e. include more than python as you have done with the Streaming API).
You can get it using the tweepy api. Get the consumer key, secret and access key, secret from https://apps.twitter.com/ .Then run the following:
consumer_key = 'XXXX'
consumer_secret = 'XXXX'
access_key = 'XXXX-XXXX'
access_secret = 'XXXX'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)
tweet = api.statuses_lookup(['643389532105256961'])
a1[0].text # Prints the message
More info here http://docs.tweepy.org/en/v3.5.0/api.html#API.statuses_lookup
I'm using twitter python tools for extracting twitter information. And I'm lost when it comes to a the very simple task of getting tweets from a user.
search_results = twitter_api.search.tweets(q="", user="daguilaraguilar", count=10)
tweets = search_results['statuses']
for tweet in tweets:
print tweet['text']
But I'm getting this error message at last of HTTP 400 response:
details: {"errors":[{"code":25,"message":"Query parameters are missing."}]}
You did not define your query. According to the twitter api, it is required. Looking at the docs for the python api wrapper, I see these steps for getting the tweets of a specific user. You can probably define a since variable also according to the twitter api:
# Get your "home" timeline
t.statuses.home_timeline()
# Get a particular friend's timeline
t.statuses.friends_timeline(id="billybob")