I am trying to build a script that will take a Twitter handle and calculate its engagement rate based on the last 10 tweets or so. If I understand Twitter's API correctly I would have to make a q request for each calculation. If I understand Twitter's pricing correctly, I would be paying between $0.75 and $1 per request depending on my package. That seems very expensive for me to build such a simple tool. Am I missing something, is there a cheaper way of doing it?
Related
I'm accessing Twitter's API v2 and have Academic Research access.
I am interested in pulling as much data as possible, but am getting rate limited.
I am using tweepy in Python to extract the data, and the call "search_all_tweets"
I am looping the extraction for each day and limiting tweets extracted per day. By using time.sleep() I can modify how many tweets I can extract per 15 minutes.
Twitter has published this to answer my question, however I am still in doubt which category and thereby limit I am under:
https://developer.twitter.com/en/docs/twitter-api/rate-limits#v2-limits
Can anybody help to clarify how many tweets I can extract per 15 minutes before getting rate limited?
Thanks in advance
Go to the Twitter developper portal, select your app in your project and, at the bottom of the Settings tab, check if you have activated the Read and Write permissions in the OAuth 1.0a authentication. If not, activate them and regenerate your tokens.
We are trying to get the owned games of a lot of users but our problem is that after a while the API call limit (100.000 a day) kicks in and we stop getting results.
We use 'IPlayerService/GetOwnedGames/v0001/?key=APIKEY&steamid=STEAMID' in our call and it works for the first entries.
There are several other queries like the GetPlayerSummaries query which take multiple Steam IDs, but according to the documentation, this one only takes one.
Is there any other way to combine/ merge our queries? We are using Python and the urllib.request library to create the request.
Depending on the payload of the requests you have the following possibilities:
if each request brings only the newest updates, you could serialize the steam ID's when you get the response that you've hit the daily limit
if you have the ability to control via the request payload what data you receive, you could go for a multithreaded / multiprocessing approach that consume the request queries and the steam ID's from a couple of shared resources
As #andreihondrari indirectly stated in his comment under his answer, one can request to get an API key which can get more then the 100.000 calls/ day. This is stated under part "License to Steam Web API & Steam Data" of the documentation:
You are limited to one hundred thousand (100,000) calls to the Steam Web API per day. Valve may approve higher daily call limits if you adhere to these API Terms of Use.
This may be complicated and there is of cause the possibility that you wont get approved, but this is pretty much the only stable way you can go.
Furthermore you could theoretically use multiple Steam Web API keys, BUT:
Each API key still has the limitation of 100.000 calls/day so you'll need to implement a fail safe and a transition between used keys and possibly need to create lots of accounts.
As each user has his own specific friendlist and blocked list the API key can "see" a portion of the Steam Community exclusively (friends data is not public otherwise). So it could be that you are using one API key which can't "see" a certain user when you could've used another to "see" it properly.
You'll need a unique email adress for each created account.
Note: Having multiple accounts actually complies with Valves ToS according to this post on Arqade.
Without having a twitter-account I can search tweets, e.g. with the URL https://twitter.com/search?q=stackoverflow or https://twitter.com/StackOverflow or https://twitter.com/hashtag/stackoverflow.
To access twitter using Python, there are several API wrappers, maybe the most famous ones being tweepy and twitter.
It does not seem to be possible to access tweets without authentication.
How can I access tweets from Python without logging in to twitter?
ps. I know that I can download the page and parse the html, but this looks like a last resort rather than an elegant solution.
From the docs
Authentication on all endpoints
We require applications to
authenticate all of their requests with OAuth 1.0a or Application-only
authentication. This visibility allows us to prevent abusive behavior,
and it also helps us to further understand how categories of
applications are using the API. We apply this understanding to better
meet the needs of developers as we continue to evolve the platform.
They need to apply rate limiting (and may be other security precautions) to minimize abuse, so, they do not allow public access to APIs.
Searching tweets using Twitter's website is manual. So there are less chances of abuse. Moreover, even if you try to parse it from web, most probably they will ban your IP address as soon as you start sending more requests than they think are not abusive.
Why not to create an account? Keep in mind different circumstances.
accounts are occassionally very hard to create. For instance, it has been reported that Twitter doesn't support phone numbers from certain locations. What would you do if you cannot authenticate with your personal mobile?
account features are heavily limited. For instance, under the standard dev account you will not retrieve past data for your educational project (e.g. a sentiment analysis of celebrities tweets).
accounts will not be free Twitter has just decided to discontinue supporting free accounts
I need to fetch twitter historical data for a given set of keywords. Twitter Search API returns tweets that are not more than 9 days old, so that will not do. I'm currently using Tweepy Library (http://code.google.com/p/tweepy/) to call Streaming API and it is working fine except the fact that it is too slow. For example, when I run a search for "$GOOG" sometimes it takes more than an hour between two results. There are definitely tweets containing that keyword but it isn't returning result fast enough.
What can be the problem? Is Streaming API slow or there is some problem in my method of accessing it? Is there any better way to get that data free of cost?
How far back do you need? To fetch historical data, you might want to keep the stream on indefinitely (the stream API allows for this) and store the stream locally, then retrieve historical data from your db.
I also use Tweepy for live Stream/Filtering and it works well. The latency is typically < 1s and Tweepy is able to handle large volume streams.
streaming API too fast you get message as soon as you post it, we use twitter4j. But streamer streams only current messages, so if you not listening on streamer the moment you send tweet then message is lost.
Is there a way for me to determine the total number of Twitter messages on a given trend topic (e.g. frequency of Twitter messages with subject matter on Haiti/#Haiti) at a given instance in time using the Twitter API? I'm writing a script in Python that will monitor Twitter traffic over a long spell of time and I was wondering how I could go about doing this.
Yes. Use the Twitter Streaming API to get a representative sample.
You might take a look at this site (see below) Drew has several sources of information and was to look at the data.
Network of People who Twitter about R
http://www.drewconway.com/zia/?p=1471