I am trying to retrieve Twitter data using Tweepy, using that below code, but I'm returning 401 error, and I regenerate the access and secret tokens, and the same error appeared.
#imports
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
#setting up the keys
consumer_key = 'xxxxxxx'
consumer_secret = 'xxxxxxxx'
access_token = 'xxxxxxxxxx'
access_secret = 'xxxxxxxxxxxxx'
class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
#This is a basic listener that just prints received tweets to standard output
def on_data(self, data):
print (data)
return True
def on_error(self, status):
print (status)
#printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
stream = Stream(auth, TweetListener())
t = u"#سوريا"
stream.filter(track=[t])
Just reset your system's clock.
If an API request to authenticate comes from a server that claims it is a time that is outside of 15 minutes of Twitter time, it will fail with a 401 error.
ThankYou
You might just have made a mistake in copying the Access Token from the apps.twitter.com page.
You need to copy the entire thing that's given as Access Token, not just the string after the -.
For example, copy and paste the entire string like 74376347-jkghdui456hjkbjhgbm45gj, not just jkghdui456hjkbjhgbm45gj.
[Note the above string is just something I typed randomly for demonstration purpose. Your actual Access token will also look like this though, i.e,
"a string of number-an alphanumeric string"]
you just have to show your keys into the double quote
and you don't have to define your keys in last twitter authentication.
#Import the necessary methods from tweepy library
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
#Variables that contains the user credentials to access Twitter API
access_token = 'X3YIzD'
access_token_secret = 'PiwPirr'
consumer_key = 'ekaOmyGn'
consumer_secret = 'RkFXRIOf83r'
#This is a basic listener that just prints received tweets to stdout.
class StdOutListener(StreamListener):
def on_data(self, data):
print data
return True
def on_error(self, status):
print status
if __name__ == '__main__':
#This handles Twitter authetification and the connection to Twitter
Streaming API
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
#This line filter Twitter Streams to capture data by the keywords: 'python',
'javascript', 'ruby'
stream.filter(track=['python', 'javascript', 'ruby'])
I had the same issue - nothing here fixed it. The trick for me was that Streaming tweets with Tweepy apparently requires 1A authentication, not 2A (see - https://github.com/tweepy/tweepy/issues/1346). This means you need to use an access token as well as the consumer tokens in the authentication object.
import tweepy
# user credentials
access_token = '...'
access_token_secret = '...'
consumer_key = '...'
consumer_secret = '...'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
# this is the main difference
auth.set_access_token(access_token, access_token_secret)
stream = tweepy.Stream(auth, tweepy.StreamListener)
In my case the error occurred because I was using AppAuthHandler rather than OAuthHandler. Switching to OAuthHandler resolved the issue.
In my case, I had this problem but it did not have to do with time.
My app had a "read only" permission.
I had to change it to a "read and write" permission for the error to cease.
You can do this by going to "user authentication" in the app settings page.
After changing your read only permission, you have to regenerate your access token, then put it into your code. Thanks for the help!
Related
what do I put in my code to make it where I can force the program to stop printing data when the tweets data back to a certain point. For example, how can I get all tweets about Verratti from within a month of running this?
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
import json
access_token = the code
access_token_secret = the code
consumer_key = the code
consumer_secret = the code
#print
class StdOutListener(StreamListener):
def on_data(self, data):
print (json.loads(data)['text'])
return True
def on_error(self, status):
print (status)
#find
if __name__ == '__main__':
#This handles Twitter authetification and the connection to Twitter Streaming API
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
#This line filter Twitter Streams to capture data by the keywords: 'python', 'javascript', 'ruby'
stream.filter(track=['Verratti'])
Nice question. It turns out that the Twitter API only lets you look back one week from the current date. There is a way around it though, someone made a github library that can search for any timeframe using twitter's advanced search function and you don't even have to bother with the whole authentication process.
Check it out: https://github.com/Jefferson-Henrique/GetOldTweets-python
****I am trying to obtain information from the twitter timeline of a specific user and I am trying to print the output in Json format, however I am getting an AttributeError: 'str' object has no attribute '_json'. I am new to python so I'm having troubles trying to resolve this so any help would be greatly appreciated. ****
Below shows the code that I have at the moment:
from __future__ import absolute_import, print_function
import tweepy
import twitter
def oauth_login():
# credentials for OAuth
CONSUMER_KEY = 'woIIbsmhE0LJhGjn7GyeSkeDiU'
CONSUMER_SECRET = 'H2xSc6E3sGqiHhbNJjZCig5KFYj0UaLy22M6WjhM5gwth7HsWmi'
OAUTH_TOKEN = '306848945-Kmh3xZDbfhMc7wMHgnBmuRLtmMzs6RN7d62o3x6i8'
OAUTH_TOKEN_SECRET = 'qpaqkvXQtfrqPkJKnBf09b48TkuTufLwTV02vyTW1kFGunu'
# Creating the authentication
auth = twitter.oauth.OAuth( OAUTH_TOKEN,
OAUTH_TOKEN_SECRET,
CONSUMER_KEY,
CONSUMER_SECRET )
# Twitter instance
twitter_api = twitter.Twitter(auth=auth)
return twitter_api
# LogIn
twitter_api = oauth_login()
# Get statuses
statuses = twitter_api.statuses.user_timeline(screen_name='#ladygaga')
# Print text
for status in statuses:
print (status['text']._json)
You seem to be mixing up tweepy with twitter, and are possibly getting a bit confused with methods as a result. The auth process for tweepy, from your code, should go as follows:
import tweepy
def oauth_login():
# credentials for OAuth
consumer_key = 'YOUR_KEY'
consumer_secret = 'YOUR_KEY'
access_token = 'YOUR_KEY'
access_token_secret = 'YOUR_KEY'
# Creating the authentication
auth = tweepy.OAuthHandler(consumer_key,
consumer_secret)
# Twitter instance
auth.set_access_token(access_token, access_token_secret)
return tweepy.API(auth)
# LogIn
twitter_api = oauth_login()
# Get statuses
statuses = twitter_api.user_timeline(screen_name='#ladygaga')
# Print text
for status in statuses:
print (status._json['text'])
If, as previously mentioned, you want to create a list of tweets, you could do the following rather than everything after # Print text
# Create a list
statuses_list = [status._json['text'] for status in statuses]
And, as mentioned in the comments, you shouldn't every give out your keys publicly. Twitter lets you reset them, which I'd recommend you do as soon as possible - editing your post isn't enough as people can still read your edit history.
I've been playing with the Twitter Streaming API using the Tweepy library. I started by following my own account and streaming my own tweets as I posted them, which worked fine.
I then attempted to stream a fairly large region's tweets ([30,-85,31,-84]), to which I initially seemed to receive no data. I then started receiving 'Location Deletion Notices', or 'scrub_geo' messages, and have only ever received those since. I changed my code back to the previously working follow code, but I continue to receive 'scrub_geo' messages and not statuses from my profile.
Here's the script I'm using:
# Import the necessary methods from tweepy library
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
# Other libs
import json
# Variables that contains the user credentials to access Twitter API
access_token = "<my_access_token>"
access_token_secret = "<my_secret_token>"
consumer_key = "<my_consumer_key>"
consumer_secret = "<my_consumer_secret>"
# This is a basic listener that just prints received tweets to stdout.
class StdOutListener(StreamListener):
def on_data(self, data):
#try:
# json_data = json.loads(data)
# print json_data['created_at'] + " " + data['text']
#except:
print "Data " + str(data)
return True
def on_error(self, status):
print "Error " + str(status)
if status == 420:
print("420 error.")
return False
if __name__ == '__main__':
# This handles Twitter authetification and the connection to Twitter Streaming API
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
# Start streaming with right parameters
#tallahassee=[30,-85,31,-84]
#stream.filter(locations=tallahassee) <---- previously used
stream.filter(follow="<my_user_id>")
Your coordinates are reversed. Since we're dealing with GeoJSON always do (long,lat,alt) or (x,y,z)
So you'll need to provide tallahassee=[-85,30,-84,31]. Always provide longitude first same as you would do (x,y) in math.
There are some places, like google maps, that do latitude first. You just have to be careful as to which proper format you're dealing with.
Im doing Twitter sentiment research at the moment. For this reason, I'm using the Twitter API to download all tweets on certain keywords. But my current code is taking a lot of time to create a large datafile, so I was wondering if there's a faster method.
This is what Im using right now:
__author__ = 'gerbuiker'
import time
#Import the necessary methods from tweepy library
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
#Variables that contains the user credentials to access Twitter API
access_token = "XXXXXXXXXXXXX"
access_token_secret = "XXXXXXXX"
consumer_key = "XXXXX"
consumer_secret = "XXXXXXXXXXXXXX"
#This is a basic listener that just prints received tweets to stdout.
class StdOutListener(StreamListener):
def on_data(self, data):
try:
#print data
tweet = data.split(',"text":"')[1].split('","source')[0]
print tweet
saveThis = str(time.time())+ '::'+ tweet #saves time+actual tweet
saveFile = open('twitiamsterdam.txt','a')
saveFile.write(saveThis)
saveFile.write('\n')
saveFile.close()
return True
except BaseException, e:
print 'failed ondata,',str(e)
time.sleep(5)
def on_error(self, status):
print status
if __name__ == '__main__':
#This handles Twitter authetification and the connection to Twitter Streaming API
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
#This line filter Twitter Streams to capture data by the keywords: 'Amsterdam'
stream.filter(track=['KEYWORD which i want to check'])
This gets me about 1500 tweets in one hour, for a pretty popular keyword (Amsterdam). Does anyone now a faster method in Python?
To be clear: I want to download all tweets on a certain subject for last month/year for example. So the newest tweets don't have to keep coming in, the most recent ones for a period would be sufficient. Thanks!
I need something similar to this for an academic research.
We're you able to fix it?
Would it be possible to specify a custom range of time from which to pull the data?
Sorry for asking here, but couldn't send you private messages.
I am able to extract the mentioned details about a twitter user using Tweepy API.
I want to do it for a list of users. Can anyone help me to this?
import tweepy
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
CONSUMER_KEY = 'ABC'
CONSUMER_SECRET = 'ABC'
ACCESS_KEY = 'ABC'
ACCESS_SECRET = 'ABC'
class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
#This is a basic listener that just prints received tweets to standard output
def on_data(self, data):
print data
return True
def on_error(self, status):
print status
auth = OAuthHandler(CONSUMER_KEY,CONSUMER_SECRET)
api = tweepy.API(auth)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
twitterStream = Stream(auth,TweetListener())
user = api.get_user('User Name')
print user.screen_name
print user.description
print user.followers_count
print user.statuses_count
print user.url
This code is ready to use anyone can use it with his/her own credentials for a single user profile.
Finally exercising and reading a lot I get the answer to my question.you can try this
import tweepy
from tweepy import Stream
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
CONSUMER_KEY = 'ABC'
CONSUMER_SECRET = 'ABC'
ACCESS_KEY = 'ABC'
ACCESS_SECRET = 'ABC'
auth = OAuthHandler(CONSUMER_KEY,CONSUMER_SECRET)
api = tweepy.API(auth)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
#This is a basic listener that just prints received tweets to standard output
def on_data(self, data):
print data
return True
def on_error(self, status):
print status
#search
api = tweepy.API(auth)
twitterStream = Stream(auth,TweetListener())
test = api.lookup_users(user_ids=['17006157','59145948','157009365'])
for user in test:
print user.screen_name
print user.name
print user.description
print user.followers_count
print user.statuses_count
print user.url
This code is ready to use just put your valid keys in place of ABC & get the users profile.you need to get the IDs first.
Your code simply interacts with your twitter account; to find information on a specific user or group of users you should look them up using the api.lookup_users(user_ids=[]) query.
You'd do it like this:
#boring auth you already have
import tweepy
from tweepy import OAuthHandler
CONSUMER_KEY = 'ABC'
CONSUMER_SECRET = 'ABC'
ACCESS_KEY = 'ABC'
ACCESS_SECRET = 'ABC'
auth = OAuthHandler(CONSUMER_KEY,CONSUMER_SECRET)
api = tweepy.API(auth)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
#search
api = tweepy.API(auth)
test = api.lookup_users(user_ids=['1123728482,5539932'])
This gives you a list of two tweepy.models.User objects:
[<tweepy.models.User object at 0x103995090>, <tweepy.models.User object at 0x1039950d0>]
You can replace the list in user_ids with a list of up to 100 ids, twitter won't let you search any more than that at once, though. Once you have your list of User objects, you can access different properties (for a list, check out the tweepy doc for the User class, line 113).