I'm new to Python and Twython, but I'm working on a project where I want to use Twython to analyze all of Miley Cyrus' tweets. Currently there are 7,193, but Twython will only let me take 200 at a time...how can I scrape all of them? Is there there a way to scrape all of them using Twython or do I have to manually scrape the Twitter website? Ideally I would preserve access to all of the tweets' metadata so I could use it in my analysis (rather than just the text of all of the tweets). Suggestions for code?
If you read the documentation, you will see that:
This method can only return up to 3,200 of a user's most recent Tweets.
So there is no way the API can be used to programatically return all historic Tweets. And, yes, you are limited to requesting 200 at a time.
Related
What I want to do is to learn which is better way to get all tweets from a user profile URL and then get all users who replied to those tweets and remove duplicates from list before adding them in to csv or excel file. I found this Git project but its not what I want
https://github.com/nirholas/Get-Tweet-Replies-With-Python-Tweepy
Could anyone guide me to a better library or idea?
Tweepy is a very good library for interacting with Twitter in Python.
I'd suggest taking a look at Tweepy's documentation and starter guide. It's quite simple to work with once you understand what functions are available.
Hi I'm new to scraping data from twitter and I've been working on a project to collect tweets and their replies. I scraped the tweets using twitter API , but I couldn't scrape their replies. Any suggestions ?
Scraping is not using API. API is a comfortable method to get data while scraping is doing it manually. If you want to get replies, but the API doesn't provide them, you can use selenium. It is well-known library used for scraping data.
Twitter API does not have an end-point to get replies to a tweet for free users, but there are some ways to achieve what you want, although there may be some limits on how many tweets you can get.
Check-out tweepy library, I think you could use the search method as is explained in this question(How to get the replies for a given tweet with tweepy and python?
). To better understand how to use the tweet object, you should definitely check the docs out.
You can achieve this by:
You want to get replies of #barackobama and Tweet Id is 1234 just example.
Search tweets having #barackobama is tweet text. then make filter on InrplytweetId=1234
by this method you can get replies of this tweet.
I have a twitter url ("http://twitter.com/1/statuses/860392275108548608") and I need to fetch the tweet using this URL.
I checked some python packages like twitter, tweepy, etc. Here there are functions to pull tweets for particular account and hashtags, but I could find any function to pull from URL.
Can someone please share the function to pull the tweet using url.
Thanks
You could use an HTML parsing library. It's not the best way to do it, but it will work. What you'll have to do is find a similarity in class or id that every tweet shares. You'll then get the text of that tweet by finding the textual value of the element.
I been successfully able to use Twitter REST API to do find tweets and retweet them automatically. Now I want to tweet user a Thank You Message, I am using Python and Tweepy for my BOT. I can do live collection of tweets, using Twitter Streaming API, but I can't understand how to use the API to send a thank you message to those who favorite my tweets. I saw both the documentations but couldn't understand and use much from there. I also tried to search for any example but couldn't find any good one for Streaming API.
Can anyone show me how to use it with tweepy to achieve the same. It would be nice if you can show how to use it when someone retweets my tweet or follows me, I want to send them a thank you tweet automatically.
PS: I understand that Twitter doesn't likes much automation. But its all for learning.
I've been trying to download the entire timelines of a number of users on Twitter. While I'm able to download the most recent 3,200 tweets from a user, I was wondering if anyone had any success downloading timelines in opposite chronological order--i.e. from first tweet to the 3,200th?
The best I can do is view these tweets in my browser via Twitter's Advanced Search feature. But there's not even a way to copy-paste these tweets into a csv where I can analyze them.
Thanks for any help you can provide!
I don't think you can get tweets older than the most recent 3,200.
This is a limitation of Twitter's underlying GET statuses/user_timeline API endpoint in the REST API:
This method can only return up to 3,200 of a user’s most recent Tweets. Native retweets of other statuses by the user is included in this total, regardless of whether include_rts is set to false when requesting this resource.
I submitted a bot TwEater on GitHub today, aiming at collecting earlier data more than 3200 for text mining tasks. Hopefully it might help you.