I am trying to use twython package to scrap twitter data.
Due to the huge number of twitter handles that I am dealing with, I cannot use twitter.lookup_user(user_id=user_batch) command - so I instead tried to use
user_dict_batch = tw.request(user_url, method='POST').
Now, according to the twython documentation, the function request is defined as follows:
request(endpoint, method='GET', params=None, version='1.1')
Return dict of response received from Twitter’s API
Parameters:
endpoint (string) – (required) Full url or Twitter API endpoint (e.g. search/tweets)
method (string) – (optional) Method of accessing data, either GET or POST. (default GET)
params (dict or None) – (optional) Dict of parameters accepted by Twitter API endpoint
version (string) – (optional) Twitter API version to access (default 1.1)
Return type:
dict
my question is, what exactly is the "Twitter API endpoint"? For instance, if I am trying to scrap for information about the user whose twitter handle is 3Degrees_Inc, what would be the Twitter API endpoint for this specific user be? I tried to use 'https://twitter.com/3Degrees_Inc' for my value for the parameter endpoint, but it's keep throwing me the Twitter API 403 (forbidden) error....
thanks,
First, twython is a wrap-up for RESTfult twitter API, whose docs you can find here. So if you are interested in API users/lookup, its endpoint is users/lookup. Pay attention that this api accepts POST request, since request for that many users may exceed GET limits.
But brief walkthrough twython documentation shows that lookup_user is implemented in EndpointMixin and according to docs implements this same endpoint interface, so you shure can use twitter.lookup_user(user_id=[user1, user2, user3])
def lookup_user(self, **params):
"""Returns fully-hydrated user objects for up to 100 users per request,
as specified by comma-separated values passed to the user_id and/or screen_name parameters.
Docs: https://dev.twitter.com/docs/api/1.1/get/users/lookup
"""
return self.post('users/lookup', params=params)
Related
There was a question about an undocumented method in the steam api, which serves to confirm authorization in the client (https://steamapi.xpaw.me/#IAuthenticationService/UpdateAuthSessionWithMobileConfirmation). With the help of Charles proxy, I saw a discrepancy with the documentation, instead of post parameters in the request, access_token is passed in the get parameter. When you try to retry the request, a 500 error is returned. The question is, how to implement the execution of this API method and what is needed for this? Is it possible to use another undocumented method to confirm authorization - UpdateAuthSessionWithSteamGuardCode and where can I get the parameters for its implementation? if anyone has an example of the implementation of these methods in python?
I tried to make a test request in python, but in response I received a 401 error, although according to the documentation I passed the necessary parameter
import requests
url = "https://api.steampowered.com/IAuthenticationService/UpdateAuthSessionWithMobileConfirmation/v1/"
data = {
'key': 'my steam web api key from [https://steamcommunity.com/dev/apikey]'
}
req = requests.post(url,data)
print(req.status_code) # 401
maybe there are convenient libraries for python where these methods are implemented?
I'm trying to retrieve the wallet balances from the mexc.com cryptocurrency exchange. It works well on others such as Gate.io or Huobi thanks to their awesome documentation. I'm stuck on mexc.com however and confused on how to authenticate myself there.
Based on their documentation, I'd need to send a GET request to /open/api/v2/account/info. I'd also need to sign the request and this is where I'm stuck.
When a request requires a signature, I need to pass the api key, unix time stamp and a calculated signature as url parameters. I have no clue though how to calculate that signature. Also I'd need to sign a string and pass it into the header.
This is their description:
Image
I'm using google api client to call YouTube Data and YouTube Analytics APIs and get video stats. I'm authenticated as the owner of the channel I am querying.
I am interested in only getting the public videos uploaded in the channel. I first query the YouTube Data API to get the list of videos in the uploads playlist and once I have the list of video ids I call the youtube Analytics API for each video and get the stats I need.
The problem I'm having is that when I list all videos in the uploads playlist I get hundreds of thousands (!!!) of unlisted videos, which I don't need.
I cannot afford to download the entire list and then check status to keep only the public videos as the number is too big and I am reaching my daily quota. It would also be a very inefficient way to do it.
is there a way to list only videos with status public for a specific playlist?
This is the current method I use:
data = service.playlistItems().list(
part="snippet,status",
playlistId=playlistID,
maxResults="50",
).execute()
I couldn't find any in the youtube API documentation on how to achieve this.
Have you tried to use PlaylistItems endpoint by not being authenticated, but, instead, using only your API key parameter?
I'm assuming here -- though cannot check it myself -- that when not authenticated, the endpoint response will contain only videos that are public.
As per the doc, there are two ways an user can access any given API endpoint: using an API key or, otherwise, by way of an OAuth token.
Any user may request from Google developer's console an API key that he will pass to his endpoint of interest as the key parameter. In case of using the Python API Client Library, your code will have to be similar to this:
from googleapiclient.discovery import build
service = build(serviceName = 'youtube', version = 'v3', developerKey = DEVELOPER_KEY)
where DEVELOPER_KEY is the API key string obtained from Google. Please note that such an API key is an user's private information.
That's it: instead of OAuth, use an API key.
This doc text delineates clearly the API's concepts of authentication and authorization.
Basically, I was assuming that when using an API key instead of being authenticated via OAuth -- that is when build receives the parameter developerKey instead of credentials -- then the endpoint's response will include only public videos, even in the case when the respective API key (passed on as parameter developerKey) originates from your initial authentication account.
I want to code that post facebook. So I decided to use python-sdk (https://github.com/pythonforfacebook/facebook-sdk).
Then I hit a problem.
graph = facebook.GraphAPI(oauth_access_token)
How can I get this "oauth_access_token"?
You need to use an authorization flow. Access tokens are the keys used after getting proper authorization.
An access token is an opaque string that identifies a user, app, or
page and can be used by the app to make graph API calls. Access tokens
are obtained via a number of methods, each of which are covered later
in this document. The token includes information about when the token
will expire and which app generated the token. Because of privacy
checks, the majority of API calls on Facebook need to include an
access token.
There are various ways to obtain an access token all explained in https://developers.facebook.com/docs/facebook-login/access-tokens/
For testing, one must create an app at https://developers.facebook.com/apps and can be issued an access token at https://developers.facebook.com/tools/access_token
Here is a way to get the user access token :
instance = UserSocialAuth.objects.get(user=request.user, provider='facebook')
token = instance.tokens
graph = facebook.GraphAPI(token['access_token'])
Maybe you've already figured this out, just in case somebody else is looking for it
Is there a way to use Simple Access API (Developer Key) instead of oAuth2 key with Google Cloud Endpoint?
Extra fields in your protorpc request object that aren't part of the definition are still stored with the request.
If you wanted to use a key field as a query parameter, you could access it via
request.get_unrecognized_field_info('key')
even if key is not a field in your message definition.
This is done in users_id_token.py (the Auth part of the endpoints library) to allow sending bearer_token or access_token as query parameters instead of as header values.
Unfortunately, the nice quota checking and other associated pieces that a "Simple API Access" key gives are not readily available. However, you could issue your own keys and manually check a key against your list and potentially check against quotas that you have defined.
For those looking to use #bossylobster's answer in Java, use the the SO Answer here:
Getting raw HTTP Data (Headers, Cookies, etc) in Google Cloud Endpoints
P.S.
I tried to make this a comment in #bossylobster's answer, but I don't have the reputation to do that. Feel free to clean up this answer so that other's can follow the path