I am doing a project on aspect based sentiment analysis from tweets on twitter. I am using tweepy to fetch the tweets. I noticed that I get only a few tweets. What is the maximum number of tweets I can get using tweepy, when a product name is searched? Is there any way I can increase this number? This is the code I am referring to : https://pastebin.com/vqbkFhtf
The number of tweets will depend on how many people are talking about the product.
Also i would suggest you to use twitter streaming API so that you get the tweets as they appear.
Related
Im looking for ways to retrieve tweets from Twitter which contains certain hashtags.
I tried to use the official API and tweepy package in Python but even with academic access I was only able to retrieve tweets which are 7 days old. I want to retrieve tweets from 2019 till 2020 but Im not able to do so with tweepy.
I tried the following packages GetOldTweet3, twint but none of them seem to work due to some changes Twitter made last year.
Can someone suggest a way to get old tweets with certain hashtags. Thanks in advance for any help or suggestion provided.
If you have academic access, you are able to use the full archive search API available in the Twitter API v2. Tweepy has support for this via the tweepy.Client class. There's a full tutorial on DEV, but the code will be something like this:
import tweepy
client = tweepy.Client(bearer_token='REPLACE_ME')
# Replace with your own search query
query = 'from:andypiper -is:retweet'
tweets = client.search_all_tweets(query=query, tweet_fields=['context_annotations', 'created_at'], max_results=100)
for tweet in tweets.data:
print(tweet.text)
if len(tweet.context_annotations) > 0:
print(tweet.context_annotations)
You can use search query parameters to specify the date range.
I have tried using tweepy to extract tweets for a specific keyword.But the count of extracted tweets using tweepy is less compared those tweets for the specific keyword as seen on twitter search.
Also I want to know how to effectively extract ALL the tweets for a specific keyword of interest using any twitter data extracting library (tweepy/twython).
I also face a problem of irrelevant tweets with same keyword coming up.Is there a way to fine tune search and perform accurate extraction so that I get all the tweets extracted for the specific keyword.
Im adding the code snippet as many asked for it.But I don't have a problem with the code as its running.
tweets = api.search('Mexican Food', count=500,tweet_mode = 'extended')
data = pd.DataFrame(data=[tweet.full_text for tweet in tweets], columns
['Tweets'])
data.head(10)
print(tweets[0].created_at)
My question is that how to get ALL the tweets with a particular keyword.For example when I run the above code ,for each time I am getting different count of tweets.Also I cross checked with doing manual search on twitter and it seems that there are much more tweets than extracted through tweepy for the particular keyword.
Also I want to know if there is any way to fine tune the keyword search through python so that all the relevant tweets for my keyword of interest is fetched.
The thing is when you use tweepy It has some limitation. It won't be able to fetch older tweets.
So I will suggest you to use
https://github.com/Jefferson-Henrique/GetOldTweets-python
in place of tweepy to fetch the older tweets.
Since you refuse to help me with your question, I'll do the bare minimum with my answer:
You are probably not doing pagination correctly
ps: Check out the stack overflow guidelines. There is an important point about Helping others reproduce the problem
How to predict twitter tweet reach on a specific area ? For e.g if a user tweets in china then how can i predict about its reach there is a website named keyhole.co they tells the total reach of tweets containing the hashtag that user searched for. Below is the screenshot that is the result when i searched for hashTag (#Retweet)
Basically, they are tells reach of a hashtag.
But the question is that how to predict the about tweet reach in a specific area (country or region).
Backgroud Research:
I know that i can check count of how many times the Tweet has been viewed through twitter Engagement API. I have aslo checked that there is a api called TwitterCounter that contains the Scripts for counting tweets. Uses 'search/tweets' or 'statuses/filter' Twitter resource to get old or new tweets, respectively.
There are also ways to get counts of retweets like discussed in this answer. Moreover, there are other ways to get views/reach of old tweets and track specific word or hashtag in tweets etc.
But my question is that how can i predict about a tweet reach ?
I have been searching for this for past several days but didn't able to find a way or solution to do it.
Can anyone please help me by telling the way to get what i want or tell me any solution by which i can do it.
Thanks in advance!
I'm having a hard time formulating my question, but basically, imagine you're streaming Twitter with Tweepy and filtering the tweets on 2 keywords like that:
twitterStream = Stream(auth, listener())
twitterStream.filter(track=["keyword1", "keyword2"])
Basically, I would like to append the keywords on their respective tweets, so, for example, I would get something like this:
some tweet about keyword 1 [keyword1]
another tweet about keyword 1 [keyword1]
some tweet about keyword 2 [keyword2]
etc...
Is this possible?
Thanks!
Tweepy use twitter streaming API, from docs of streaming API, I believe its impossible to get result as you expected. Possible solutions are:
If you have very limited keywords to track, then for each of these keywords, make a streaming track request.
If you have lots of keywords to track, and you track all of these keywords in one streaming request, then you can perform keyword search on the returned tweets to determine what keywords this tweets containing. Depending one your applications, the search operation may process on many fields of tweet, e.g., text, hashtag, urls and etc.
Hope this would be help for. Thanks.
I want collect data from twitter using python Tweepy library.
I surveyed the rate limits for Twitter API,which is 180 requests per 15-minute.
What I want to know how many data I can get for one specific keyword?put it in another way , when I use the Tweepy.Cursor,when it'll stops?
I not saying the maths calculation(100 count * 180 request * 4 times/hour etc.) but the real experience.I found a view as follows:
"With a specific keyword, you can typically only poll the last 5,000 tweets per keyword. You are further limited by the number of requests you can make in a certain time period. "
http://www.brightplanet.com/2013/06/twitter-firehose-vs-twitter-api-whats-the-difference-and-why-should-you-care/
Is this correct(if this's correct,I only need to run the program for 5 minutes or so)? or I am needed to keep getting as many tweets as they are there(which may make the program keep running very long time)?
You will definitely not be getting as many tweets as exist. The way Twitter limits how far back you can go (and therefore how many tweets are available) is with a minimum since_id parameter passed to the GET search/tweets call to the Twitter API. In Tweepy, the API.search function interfaces with the Twitter API. Twitter's GET search/tweets documentation has a lot of good info:
There are limits to the number of Tweets which can be accessed through the API. If the limit of Tweets has occured since the since_id, the since_id will be forced to the oldest ID available.
In practical terms, Tweepy's API.search should not take long to get all the available tweets. Note that not all tweets are available per the Twitter API, but I've never had a search take up more than 10 minutes.