in the process of extracting tweets from a twitter account, should the access keys be regenerated again and again ?
like i have tried this code:
import tweepy as tw
import pandas as pd
consumer_key = "xyz"
consumer_secret = "xyz"
access_token = "xyz"
access_token_secret = "xyz"
auth = tw.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tw.API(auth)
tweets = []
likes = []
time = []
cursor = tw.Cursor(api.user_timeline, id='CirusFoundation', tweet_mode = "extended").items(1)
for i in tw.Cursor(api.user_timeline, id='CirusFoundation', tweet_mode = "extended").items(200):
tweets.append(i.full_text)
likes.append(i.favourite_count)
time.append(i.created_at)
df = pd.DataFrame({'tweets':tweets, 'likes':likes, 'time':time})
now i get a forbidden: 403 error. as output
thanks in adv.
There are several reasons for a 403:
You are requesting a feature that is not covered by your twitter dev account. Tweepy may be predecated on having the elevated access.
You are trying to access from a non-ssl/https
You have exceeded your request quota.
Your access token was not generated properly (E.g if you are trying to write, you need to generate with r/w permissions)
Related
I have a developer account as an academic and my profile page on twitter has Elevated on top of it, but when I use Tweepy to access the tweets, it only scrapes tweets from 7 days ago. How can I extend my access up to 2006?
This is my code:
import tweepy
from tweepy import OAuthHandler
import pandas as pd
access_token = '#'
access_token_secret = '#'
API_key = '#'
API_key_secret = '#'
auth = tweepy.OAuthHandler(API_key, API_key_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth, wait_on_rate_limit=True)
tweets = []
count = 1
for tweet in tweepy.Cursor(api.search_tweets, q= "#SEARCHQUERY", count=5000).items(50000):
print(count)
count += 1
try:
data = [tweet.created_at, tweet.id, tweet.text,
tweet.user._json['screen_name'], tweet.user._json['name'], tweet.user._json['created_at'], tweet.entities['urls']]
data = tuple(data)
tweets.append(data)
except tweepy.TweepError as e:
print(e.reason)
continue
except StopIteration:
break
df = pd.DataFrame(tweets, columns = ['created_at','tweet_id', 'tweet_text', 'screen_name', 'name', 'account_creation_date', 'urls'])
df.to_csv(path_or_buf = 'local address/file.csv', index=False)
The Search All endpoint is available in Twitter API v2, which is represented by the tweepy.Client object (you are using tweepy.api).
The most important thing is that you require Academic research access from Twitter. Elevated access grants addition request volume, and access to the v1.1 APIs on top of v2 (Essential) access, but you will need an account and Project with Academic access to call the endpoint. There's a process to apply for that in the Twitter Developer Portal.
I was just wondering if anyone knew how to list out the usernames that a twitter user is following, and their followers in two separate .csv cells.
This is what I have tried so far.
import tweepy
import csv
consumer_key = ""
consumer_secret = ""
access_token = ""
access_token_secret = ""
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
csvFile = open('ID.csv', 'w')
csvWriter = csv.writer(csvFile)
users = ['AindriasMoynih1', 'Fiona_Kildare', 'daracalleary', 'CowenBarry', 'BillyKelleherTD', 'BrendanSmithTD']
for user_name in users:
user = api.get_user(screen_name = user_name, count=200)
csvWriter.writerow([user.screen_name, user.id, user.followers_count, user.followers_id, user.friends_id user.description.encode('utf-8')])
print (user.id)
csvFile.close()
Tweepy is a wrapper around the Twitter API.
According to the Twitter API documentation, you'll need to call the GET friends/ids to get a list of their friends (people they follow), and GET followers/ids to get their followers.
Using the wrapper, you'll invoke those API calls indirectly by calling the corresponding method in Tweepy.
Since there will be a lot of results, you should use the Tweepy Cursor to handle scrolling through the pages of results for you.
Try the code below. I'll leave it to you to handle the CSV aspect, and to apply it to multiple users.
import tweepy
access_token = "1234"
access_token_secret = "1234"
consumer_key = "1234"
consumer_secret = "1234"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
for user in tweepy.Cursor(api.get_friends, screen_name="TechCrunch").items():
print('friend: ' + user.screen_name)
for user in tweepy.Cursor(api.get_followers, screen_name="TechCrunch").items():
print('follower: ' + user.screen_name)
I want to get tweets from specific geo-location, so i used tweepy (api.search) method. I am successfully getting tweets from the specific geo location. but I am not getting screen name or userid. I am expecting that show_user when True will return me the screen name, who is posting that tweet. but i am not getting any screen name. so please tell me the solution how can i get the screen name who is posting that tweet by given geo location. Using tweepy.
public_tweets = tweepy.Cursor(api.search, rpp=100,
geocode="44.269493,63.341332,5km",since="2018-05-01",show_user =
"True",tweet_mode="extended").items()
rpp is deprecated, use "count" instead (see https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html).
This works with "100km" ("5km" returned no result).
consumer_key = '***'
consumer_secret = '***'
access_token = '***'
access_token_secret = '***'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)
tweets = api.search(count=100,geocode="44.269493,63.341332,5km",since="2018-05-01")
for tweet in tweets:
print(tweet.user.id_str, tweet.user.screen_name, tweet.text)
****I am trying to obtain information from the twitter timeline of a specific user and I am trying to print the output in Json format, however I am getting an AttributeError: 'str' object has no attribute '_json'. I am new to python so I'm having troubles trying to resolve this so any help would be greatly appreciated. ****
Below shows the code that I have at the moment:
from __future__ import absolute_import, print_function
import tweepy
import twitter
def oauth_login():
# credentials for OAuth
CONSUMER_KEY = 'woIIbsmhE0LJhGjn7GyeSkeDiU'
CONSUMER_SECRET = 'H2xSc6E3sGqiHhbNJjZCig5KFYj0UaLy22M6WjhM5gwth7HsWmi'
OAUTH_TOKEN = '306848945-Kmh3xZDbfhMc7wMHgnBmuRLtmMzs6RN7d62o3x6i8'
OAUTH_TOKEN_SECRET = 'qpaqkvXQtfrqPkJKnBf09b48TkuTufLwTV02vyTW1kFGunu'
# Creating the authentication
auth = twitter.oauth.OAuth( OAUTH_TOKEN,
OAUTH_TOKEN_SECRET,
CONSUMER_KEY,
CONSUMER_SECRET )
# Twitter instance
twitter_api = twitter.Twitter(auth=auth)
return twitter_api
# LogIn
twitter_api = oauth_login()
# Get statuses
statuses = twitter_api.statuses.user_timeline(screen_name='#ladygaga')
# Print text
for status in statuses:
print (status['text']._json)
You seem to be mixing up tweepy with twitter, and are possibly getting a bit confused with methods as a result. The auth process for tweepy, from your code, should go as follows:
import tweepy
def oauth_login():
# credentials for OAuth
consumer_key = 'YOUR_KEY'
consumer_secret = 'YOUR_KEY'
access_token = 'YOUR_KEY'
access_token_secret = 'YOUR_KEY'
# Creating the authentication
auth = tweepy.OAuthHandler(consumer_key,
consumer_secret)
# Twitter instance
auth.set_access_token(access_token, access_token_secret)
return tweepy.API(auth)
# LogIn
twitter_api = oauth_login()
# Get statuses
statuses = twitter_api.user_timeline(screen_name='#ladygaga')
# Print text
for status in statuses:
print (status._json['text'])
If, as previously mentioned, you want to create a list of tweets, you could do the following rather than everything after # Print text
# Create a list
statuses_list = [status._json['text'] for status in statuses]
And, as mentioned in the comments, you shouldn't every give out your keys publicly. Twitter lets you reset them, which I'd recommend you do as soon as possible - editing your post isn't enough as people can still read your edit history.
I am trying to retrieve Twitter data using Tweepy, using that below code, but I'm returning 401 error, and I regenerate the access and secret tokens, and the same error appeared.
#imports
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
#setting up the keys
consumer_key = 'xxxxxxx'
consumer_secret = 'xxxxxxxx'
access_token = 'xxxxxxxxxx'
access_secret = 'xxxxxxxxxxxxx'
class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
#This is a basic listener that just prints received tweets to standard output
def on_data(self, data):
print (data)
return True
def on_error(self, status):
print (status)
#printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
stream = Stream(auth, TweetListener())
t = u"#سوريا"
stream.filter(track=[t])
Just reset your system's clock.
If an API request to authenticate comes from a server that claims it is a time that is outside of 15 minutes of Twitter time, it will fail with a 401 error.
ThankYou
You might just have made a mistake in copying the Access Token from the apps.twitter.com page.
You need to copy the entire thing that's given as Access Token, not just the string after the -.
For example, copy and paste the entire string like 74376347-jkghdui456hjkbjhgbm45gj, not just jkghdui456hjkbjhgbm45gj.
[Note the above string is just something I typed randomly for demonstration purpose. Your actual Access token will also look like this though, i.e,
"a string of number-an alphanumeric string"]
you just have to show your keys into the double quote
and you don't have to define your keys in last twitter authentication.
#Import the necessary methods from tweepy library
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
#Variables that contains the user credentials to access Twitter API
access_token = 'X3YIzD'
access_token_secret = 'PiwPirr'
consumer_key = 'ekaOmyGn'
consumer_secret = 'RkFXRIOf83r'
#This is a basic listener that just prints received tweets to stdout.
class StdOutListener(StreamListener):
def on_data(self, data):
print data
return True
def on_error(self, status):
print status
if __name__ == '__main__':
#This handles Twitter authetification and the connection to Twitter
Streaming API
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
#This line filter Twitter Streams to capture data by the keywords: 'python',
'javascript', 'ruby'
stream.filter(track=['python', 'javascript', 'ruby'])
I had the same issue - nothing here fixed it. The trick for me was that Streaming tweets with Tweepy apparently requires 1A authentication, not 2A (see - https://github.com/tweepy/tweepy/issues/1346). This means you need to use an access token as well as the consumer tokens in the authentication object.
import tweepy
# user credentials
access_token = '...'
access_token_secret = '...'
consumer_key = '...'
consumer_secret = '...'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
# this is the main difference
auth.set_access_token(access_token, access_token_secret)
stream = tweepy.Stream(auth, tweepy.StreamListener)
In my case the error occurred because I was using AppAuthHandler rather than OAuthHandler. Switching to OAuthHandler resolved the issue.
In my case, I had this problem but it did not have to do with time.
My app had a "read only" permission.
I had to change it to a "read and write" permission for the error to cease.
You can do this by going to "user authentication" in the app settings page.
After changing your read only permission, you have to regenerate your access token, then put it into your code. Thanks for the help!