Does anyone have experience with the Dota 2 API library in Python called 'dota2api'? I wish to pull a list of 200 recent games filtered by various criteria. I'm using the get_match_history() request (see link). Here's my code:
import dota2api
key = '<key>'
api = dota2api.Initialise(key)
match_list = api.get_match_history(matches_requested=200)
I haven't specified any filters yet, since I can't even get the matches_requested argument to work. When I run this code, I get exactly 100 matches. I fact, no matter how I specify the matches_requested argument, I allways get 100 matches.
Does anyone know if I'm specifying the argument wrong or some other reason why it's working as intended?
Thanks in advance.
For such rarely used libraries it is hard to get an answer here.
I have found this issue on the library's Github:
You can't get more than 500 matches through get_match_history, it's
limited by valve api. One approach you can do is alternate hero_id,
like, requesting with account_id, hero_id and start_at_match_id (none
if first request), values assigned, this way you can get at least 500
matches of each hero from that account_id.
Probably that has since changed and now the parameter is ignored by the API completely. Try creating a new issue on the Github.
Related
Just built my first Shopify store and wanted to use python and API to bulk-update product tags on all our products.
However, I've run into a snag on my PUT call. I keep getting a 400 Response with the message '{"errors":{"product":"Required parameter missing or invalid"}}'.
I've seen other people with similar issues on here but none of their solutions seem to be working for me.
Has anyone else run into this problem that can help me figure it out?
Here are some screenshots of my code, a printout of the payload, and the error in the response.
Code, Payload and Response:
I can successfully get the product info using the API and was originally sending the entire JSON payload that was returned just updated it with my product tags and sent it back through the API.
To narrow potential pain points, I'm keeping it simple now and just including "id" and "tags" in the payload but still get the same error.
We figured it out! It turns out that when we initially created the store we used a domain store name that was misspelled so we created a new domain store name with the correct spelling. However, the original domain store name was not deleted (not even sure it can be), and even though the new domain store name forwards to the original one and allows for GET requests, for PUT requests we had to use the original misspelled domain store name and the PUTs work fine.
I figured this out by using fiddler to capture a manual product update via the Shopify website to see what the payload looked like and that's why I noticed the store URL was different than the one we were using.
Hope this helps someone with a similar issue!
I would like to get the information about all reviews from my server. That's my code that I used to achieve my goal.
from rbtools.api.client import RBClient
client = RBClient('http://my-server.net/')
root = client.get_root()
reviews = root.get_review_requests()
The variable reviews contains just 25 review requests (I expected much, much more). What's even stranger I tried something a bit different
count = root.get_review_requests(counts_only=True)
Now count.count is equal to 17164. How can I extract the rest of my reviews? I tried to check the official documentation but I haven't found anything connected to my problem.
According to the documentation (https://www.reviewboard.org/docs/manual/dev/webapi/2.0/resources/review-request-list/#webapi2.0-review-request-list-resource), counts_only is only a Boolean flag that indicates following:
If specified, a single count field is returned with the number of results, instead of the results themselves.
But, what you could do, is to provide it with status, so:
count = root.get_review_requests(counts_only=True, status='all')
should return you all the requests.
Keep in mind that I didn't test this part of the code locally. I referred to their repo test example -> https://github.com/reviewboard/rbtools/blob/master/rbtools/utils/tests/test_review_request.py#L643 and the documentation link posted above.
You have to use pagination (unfortunately I can't provide exact code without ability to reproduce your question):
The maximum number of results to return in this list. By default, this is 25. There is a hard limit of 200; if you need more than 200 results, you will need to make more than one request, using the “next” pagination link.
Looks like pagination helper class also available.
If you want to get 200 results you may set max_results:
requests = root.get_review_requests(max_results=200)
Anyway HERE is a good example how to iterate over results.
Also I don't recommend to get all 17164 results by one request even if it possible. Because total data of response will be huge (let's say if size one a result is 10KB total size will be more than 171MB)
I have to scrape tweets from Twitter for a specific user (#salvinimi), from January 2018. The issue is that there are a lot of tweets in this range of time, and so I am not able to scrape all the ones I need!
I tried multiple solutions:
1)
pip install twitterscraper
from twitterscraper import query_tweets_from_user as qtfu
tweets = qtfu(user='matteosalvinimi')
With this method, I get only a few teets (500~600 more or less), instead of all the tweets... Do you know why?
2)
!pip install twitter_scraper
from twitter_scraper import get_tweets
tweets = []
for i in get_tweets('matteosalvinimi', pages=100):
tweets.append(i)
With this method I get an error -> "ParserError: Document is empty"...
If I set "pages=40", I get the tweets without errors, but not all the ones. Do you know why?
Three things for the first issue you encounter:
first of all, every API has its limits and one like Twitter would be expected to monitor its use and eventually stop a user from retrieving data if the user is asking for more than the limits. Trying to overcome the limitations of the API might not be the best idea and might result in being banned from accessing the site or other things (I'm taking guesses here as I don't know what's the policy of Twitter on the matter). That said, the documentation on the library you're using states :
With Twitter's Search API you can only sent 180 Requests every 15 minutes. With a maximum number of 100 tweets per Request this means you can mine for 4 x 180 x 100 = 72.000 tweets per hour.
By using TwitterScraper you are not limited by this number but by your internet speed/bandwith and the number of instances of TwitterScraper you are willing to start.
then, the function you're using, query_tweets_from_user() has a limit argument which you can set to an integer. One thing you can try is changing that argument and seeing whether you get what you want or not.
finally, if the above does not work, you could be subsetting your time range in two, three ore more subsets if needed, collect the data separately and merge them together afterwards.
The second issue you mention might be due to many different things so I'll just take a broad guess here. For me, either setting pages=100 is too high and by one way or another the program or the API is unable to retrieve the data, or you're trying to look at a hundred pages when there is less than a hundred in pages to look for reality, which results in the program trying to parse an empty document.
Usually, when using wptools, an API wrapper for Python I get a 'mostviewed' dictionary under 'data' when I do something like this:
site = wptools.site('en.wikipedia.org') or
blah = site.get_info('en.wikipedia.org') or
fdgfdg= site.top('en.wikipedia.org')
The dictionary (under data) returned contains the most viewed pages for, in this case the English version of wikipedia.org
but lately, the dictionary is empty (I changed nothing in the original version and created a new project to test from scratch but nothing). What can I do to regain this functionality cleanly and quickly in Python 3.7?
This seemed to resolve within 24 hours so I assumed A. I had hit some rate limit as caching wasn't happening under development version. or
B. wptools or wikipedia API was undergoing some change.
...but I can't see any rate limits for this usage and issue ocurring again.
Here is Python code I am using resulting in an empty list returned [ ]
wptools_site_object = wptools.site('en.wikipedia.org')
wptools_site_object.top('en.wikipedia.org')
mostviewedtopics = wptools_site_object.data['mostviewed']
I have a simple piece of Python code to run a Radar Search with Places API and return the results coordinates in a list. I run into three problems, first, the results pulled this way do not match with doing a search on Google Maps itself using the same coordinates and parameters. Specifically, I get MANY more results on Radar Search. In a radius of 1km, I get more than 200 results for a restaurant chain name.
Second, the results go beyond 1km, my specified radius. The furthest is 1.3km away using Haversine.
Third, the results are wrong. The keyword field has no effect on the results. For example, searching for "McDonalds" or "Car" with the same parameters yield the exact same results. One of the results points to an Adidas store when I use the Place ID to find the Google description.
This is code independent, these problems are there if I just C&P this into the url bar:
https://maps.googleapis.com/maps/api/place/radarsearch/json?location=39.876186,116.439424&radius=1000&keyword=McDonalds&key=KEY
I have seen another similar post on Places API malfunctioning recently. Any help is appreciated. Thanks
I have a support ticket open with Google about this, as we're Enterprise customers, and they have confirmed there is an issue and they're working on it. From my conversations with them over the last few days:
There have been a few other reports of this issue and we've reported
the problem to the Places API team. I'll get back to you as soon as we
have more information from them.
We've received some other reports of this and the API engineers are
looking at the issue with the highest priority. There's no obvious
cause yet, but we'll let you know when they're done investigating and
have determined a fix.
I'm sorry to hear about the complaints that you're receiving, but
unfortunately the engineers haven't been able to give me an ETA yet. I
expect to hear back from them soon but can't give an estimate yet.
I'll post updates here as I get them.
UPDATE 9/8: Google's support is saying this issue will be fixed by end of the week.
UPDATE 9/12: Google fixed it. It was being tracked here: https://code.google.com/p/gmaps-api-issues/issues/detail?id=7082