Develop Reference

Using tweepy to get a keyword from a specific user

I'm trying to create a listener to a very specific twitter account (mine), so I can do some automation, if I tweet something with a "special" code at the end (could be a character like "…") it will trigger an action, like adding the previous characters to a database.
So, I used Tweepy and I'm able to create the listener, filter keywords and so, but it will filter keywords from all the Tweetverse. This is my code:
import tweepy
cfg = {
"consumer_key" : "...",
"consumer_secret" : "...",
"access_token" : "...",
"access_token_secret" : "..."
}
auth = tweepy.OAuthHandler(cfg['consumer_key'], cfg['consumer_secret'])
auth.set_access_token(cfg['access_token'], cfg['access_token_secret'])
api = tweepy.API(auth)
class MyStreamListener(tweepy.StreamListener):
def on_status(self, status):
print(status.text)
return True
def on_error(self, status):
print('error ',status)
return False
myStreamListener = MyStreamListener()
myStream = tweepy.Stream(auth=auth, listener=myStreamListener)
myStream.filter(track=['…'])
It will filter all the messages containing a "…" no matter who wrote it, so I added to the last line the parameter follow='' like:
myStream.filter(follow='myTwitterName', track=['…'])
It always gives me a 406 error, if I use myStream.userstream('myTwitterName') it will give me, not just the Tweets I write, but also my whole timeline.
So, what am I doing wrong?
EDIT
I just find my first error. I was using user's screen name, not Twitter ID. Now I got rid of the 406 error, but still doesn't work. I placed the Twitter ID in the follow parameter, but does absolutely nothing. I tried both, with my account and with an account that is too "live", like CNN (ID = 759251), I see new tweets coming in my browser, but nothing on the listener.
If you're interested on knowing your own Twitter ID, I used this service: http://gettwitterid.com/

OK, solved. It was working from the very beggining, I made two mistakes:
To solve the 406 error all it has to be done, is to use Twitter id instead of Twitter name.
The listener was apparently doing nothing, because I was sending "big" tweets, that is, tweets longer than 140 chars. In this case, you shouldn't use status.text, but status.extended_tweet['full_text']
You must check for the existance of the extended_tweet, if it is not in the status received, then you should use the text

tweepy wait_on_rate_limit not working

So, first off, I realize there's a number of questions regarding handling the twitter rate limits. I have no idea why, but none of the ones's I've found so far work for me.
I'm using tweepy. I'm trying to get a list of all the followers of the followers of a user. As expected, I can't pull everything down all at once due to twitter's rate limits. I have tweepy v 3.5 installed and thus am referring to http://docs.tweepy.org/en/v3.5.0/api.html. To get the list of followers of the originating user I use:
auth = tweepy.OAuthHandler(consumer_key,consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
followerIDs = []
for page in tweepy.Cursor(api.followers_ids, screen_name=originatingUser, wait_on_rate_limit = True, wait_on_rate_limit_notify = True).pages():
followerIDs.extend(page)
followers = api.lookup_users(follower)
This works a for a bit but quickly turns into:
tweepy.error.TweepError: [{u'message': u'Rate limit exceeded', u'code': 88}]
My theory, would then to retrieve the followers of each user for each followerID using something like this:
for followerID in followerIDs:
for page in tweepy.Cursor(api.followers_ids, id=followerID, wait_on_rate_limit = True, wait_on_rate_limit_notify = True).pages():
followerIDs.extend(page)
The other problem I have is when I'm trying to look up the user names. For this, It use the grouper function from itertools to break the followers up into groups of 100 (api.lookup_users can only accept 100 id's at a time) and use
followerIDs = grouper(followerIDs,100)
for followerGroup in followerIDs:
followerGroup=filter(None, followerGroup)
followers = api.lookup_users(followerGroup,wait_on_rate_limit = True)
for follower in followers:
print (originatingUser + ", " + str(follower.screen_name))
That gets a different error, namely:
TypeError: lookup_users() got an unexpected keyword argument 'wait_on_rate_limit'
which I'm finding confusing, becuase the tweepy api suggests that that should be an accepted argument.
Any ideas as to what I'm doing wrong?
Cheers
Ben.

I know this might be a little late, but here goes.
You pass the wait_on_rate_limit argument in the Cursor constructor, while the tweepy documentation states that it should be passed on the API() constructor.

The wait_on_rate_limit argument is to be passed in the API() constructor.
In your code it would look like:
api = tweepy.API(auth,wait_on_rate_limit=True)
There's also another argument wait_on_rate_limit_notify, which informs you when tweepy is waiting for your rate limit to refresh. Adding both would finally make the line:
api = tweepy.API(auth,wait_on_rate_limit=True,wait_on_rate_limit_notify=True)

There is a rate limit for twitter API as mentioned here: https://dev.twitter.com/rest/public/rate-limiting
The quick solution to pass this could be catching the rate limit error and sleeping your application for a while then continue where you left.
pages = tweepy.Cursor(api.followers_ids, id=followerID).pages()
while True:
try:
page = pages.next()
followerIDs.extend(page)
except TweepError:
time.sleep(60 * 15)
continue
except StopIteration:
break
should do the trick. Not sure if this will work as you expect but the basic idea is this.

Session data not being stored during testing in Django

I am currently writing tests for our project, and I ran into an issue. We have this section of a view, which will redirect the user back to the page where they came from including an error message (that's being stored in the session):
if request.GET.get('error_code'):
"""
Something went wrong or the call was cancelled
"""
errorCode = request.GET.get('error_code')
if errorCode == 4201:
request.session['errormessage'] = _('Action cancelled by the user')
return HttpResponseRedirect('/socialMedia/manageAccessToken')
Once the HttpResponseRedirect kicks in, the first thing that the new view does is scan the session, to see if any error messages are stored in the session. If there are, we place them in a dictionary and then delete it from the session:
def manageAccessToken(request):
"""
View that handles all things related to the access tokens for Facebook,
Twitter and Linkedin.
"""
contextDict = {}
try:
contextDict['errormessage'] = request.session['errormessage']
contextDict['successmessage'] = request.session['successmessage']
del request.session['errormessage']
del request.session['successmessage']
except KeyError:
pass
We should now have the error message in a dictionary, but after printing the dictionary the error message is not there. I also printed the session just before the HttpResponseRedirect, but the session is an empty dictionary there as well.
This is the test:
class oauthCallbacks(TestCase):
"""
Class to test the different oauth callbacks
"""
def setUp(self):
self.user = User.objects.create(
email='test#django.com'
)
self.c = Client()
def test_oauthCallbackFacebookErrorCode(self):
"""
Tests the Facebook oauth callback view
This call contains an error code, so we will be redirected to the
manage accesstoken page. We check if we get the error message
"""
self.c.force_login(self.user)
response = self.c.get('/socialMedia/oauthCallbackFacebook/',
data={'error_code': 4201},
follow=True,
)
self.assertEqual('Action cancelled by the user', response.context['errormessage'])
It looks like the session can not be accessed or written to directly from the views during testing. I can, however, access a value in the session by manually setting it in the test by using the following bit of code:
session = self.c.session
session['errormessage'] = 'This is an error message'
session.save()
This is however not what I want, because I need the session to be set by the view as there are many different error messages in the entire view. Does anyone know how to solve this? Thanks in advance!

After taking a closer look I found the issue, it is in the view itself:
errorCode = request.GET.get('error_code')
if errorCode == 4201:
request.session['errormessage'] = _('Action cancelled by the user')
The errorCode variable is a string, and I was comparing it to an integer. I fixed it by changing the second line to:
if int(errorCode) == 4201:

geopy openmapquest for Python throws GeocoderInsufficientPrivileges error

I am running the following:
import geopy
geolocator = geopy.geocoders.OpenMapQuest(api_key='my_key_here')
location1 = geolocator.geocode('Madrid')
where my_key_here is my consumer key for mapquest, and I get the following error:
GeocoderInsufficientPrivileges: HTTP Error 403: Forbidden
Not sure what I am doing wrong.
Thanks!

I've also tried the same with the same result. After checking the Library, I found out, that the error is referring to the line, where the request ist build and it seems, that the API Key is not transmitted. If you add no key in the init statement, the api_key='' so I tried to change the line 66 in my own Library of the file: https://github.com/geopy/geopy/blob/master/geopy/geocoders/openmapquest.py to my key.
Still no success! The key itself works, I've tested it with calling the URL that is also called in the Library:
http://open.mapquestapi.com/nominatim/v1/search.php?key="MY_KEY"&format=json&json_callback=renderBasicSearchNarrative&q=westminster+abbey
no idea why this isn't working…
Cheers.kg

I made slight progress with fixing this one. I was able to get the query written correctly, but its the json parsing that kind of have me stumped. Maybe someone knows. I know the url is being sent correctly (I checked it in the browser and it returned a json object). Maybe someone knows how to parse the returned json object to get it to finally work.
Anyways, I had to go in the openmapquest.py source code, and starting from line 66, I made the following modifications:
self.api_key = api_key
self.api = "http://www.mapquestapi.com/geocoding/v1/address?"
def geocode(self, query, exactly_one=True, timeout=None): # pylint: disable=W0221
"""
Geocode a location query.
:param string query: The address or query you wish to geocode.
:param bool exactly_one: Return one result or a list of results, if
available.
:param int timeout: Time, in seconds, to wait for the geocoding service
to respond before raising a :class:`geopy.exc.GeocoderTimedOut`
exception. Set this only if you wish to override, on this call
only, the value set during the geocoder's initialization.
.. versionadded:: 0.97
"""
params = {
'key': self.api_key,
'location': self.format_string % query
}
if exactly_one:
params['maxResults'] = 1
url = "&".join((self.api, urlencode(params)))
print url # Print the URL just to make sure it's produced correctly
Now the task remains to get the _parse_json function working.

What is AppIdentityError: Wrong recipient?

I have a simple Python server to verify Google oauth2 id_tokens. When this code executes it throws the error AppIdentityError: Wrong recipient:
def verify():
id_token = request.form['id_token']
status = {}
if id_token is not None:
try:
jwt = verify_id_token(id_token, config['google']['clientId'])
status['valid'] = True
status['jwt'] = jwt
except AppIdentityError:
status['valid'] = False
status['message'] = "Invalid id token"
pprint.pprint(status)
response = make_response(json.dumps(status))
response.headers['Content-Type'] = 'application/json'
return response
What does AppIdentityError: Wrong recipient mean and how can I address it?

The JWT that you are trying to verify includes an audience (aud) value. The error is the result of the aud value being mismatched with what is expected.
I'm not certain which library you're using, but in the identity-toolkit-python-client package the VerifyGitTotken function looks at the project_id and client_id values that are provided to verify the JWT.
When I encountered this error, it turned out that my gitkit-server-config.json file mismatched what was provided by the Google developers console. The values are provided on the API Manager >> Overview >> Identity Toolkit API page.
I corrected my json file, and the error went away.