I am trying to make some (JSON) API calls to our Wi-Fi controller and obtain some info. When I store the JSON response into a dict somehow it only see's a few keys, namely:
dict_keys(['totalCount', 'hasMore', 'firstIndex', 'list'])
and items:
dict_items([('totalCount', 32), ('hasMore', False), ('firstIndex', 0),
('list', [{'id': 'ehgfhafgf', 'name': 'fasdfsd
xxxx'}, {'id': 'efasfsfas',
'name': 'zxcva'}])])
I removed a lot of items so It would make some sense otherwise it would be too much text.
So as you can see the dict recognizes the wrong variables as keys. Because as keys I need id and name. Is there a way to manually assign dict keys or a trick to simulate this?
My piece of code:
#Method that retrieves all zones
def getZones():
r = requests.get("url..", verify=False, cookies=cookie)
print(type(r.json()))
jsonResponse = r.json()
print("items: \n")
print(jsonResponse.items())
print("\nkeys: \n")
print(jsonResponse.keys())
print(jsonResponse.get('id'))
return r
doing a lot of prints for debugging reasons.
Your question would have been clearer if you had shown the actual JSON response.
However, it is clear from what you have posted that id and name are indeed not top-level keys, but keys inside nested dictionaries inside a list assigned to the list key. So you should get them from there:
for item in jsonResponse['list']:
print(item['id'], item['name'])
Related
I am trying to get the subscriber count from a youtube channel using the youtube api. However the repsonse sends a nested dict with more information than just the subscriber count. Here is the code I tried using to solve the problem
from googleapiclient.discovery import build
api_key = 'my_key'
youtube = build('youtube', 'v3', developerKey=api_key)
request = youtube.channels().list(
part='statistics',
forUsername='pewdiepie'
)
response = dict(request.execute())
print(response)
print(['items']['statistics'])
However I come up with this error
list indices must be integers or slices, not str
Here is the response I get from the youtube api
{'kind': 'youtube#channelListResponse', 'etag': 'b1tpsekLNgax8TyB0hih1FujSL4', 'pageInfo': {'totalResults': 1, 'resultsPerPage': 5}, 'items': [{'kind': 'youtube#channel', 'etag': 'gdAP8rVq0n-1Bj2eyjvac-CrtdQ', 'id': 'UC-lHJZR3Gqxm24_Vd_AJ5Yw', 'statistics': {'viewCount': '28147953019', 'subscriberCount': '111000000', 'hiddenSubscriberCount': False, 'videoCount': '4459'}}]}
Change the last line in your code to:
for item in response['items']:
print(item['statistics'])
Based on the response you are posting here, if you want to know statistics from an item, you have to specify from which item you are trying to get statistics. items is a list, which means that you have to refer to it's elements by numerical index. It has a length as well, you can get by using len(response['items']). Now, let's say you want to get the subscriberCount from the first item in the list, in that case you can retrieve it with the following code:
if (len(response['items']) > 0):
response['items'][0]['statistics']['subscriberCount']
In order to access a value in a dictionary you actually have to include the dictionary in the expression.
['items']['statistics'] alone creates a list containing one string, 'items', and tries to access that list using the string 'statistics' as index. But this is bad, because list indices must be integers.
In order to access the key 'items' in the dictionary response, you can use response['items'].
Assuming this returns another, nested, dictionary, you can then access the key 'statistics' in that dictionary by using response['items']['statistics'].
/api/stats
?fields=["clkCnt","impCnt"]
&ids=nkw0001,nkw0002,nkw0003,nkw0004
&timeRange={"since":"2019-05-25","until":"2019-06-17"}
I'm currently working on a API called naver_searchad_api
link to github of the api If you want to check it out. but i don't think you need to
the final url should be a baseurl + /api/stats
and on fields and ids and timeRange, the url should be like that
the requests I wrote is like below
r = requests.get(BASE_URL + uri, params={'ids': ['nkw0001','nkw0002','nkw0003','nkw0004'], 'timeRange': {"since": "2019-05-25", "until": "2019-06-17"}}, headers=get_header(method,uri,API_KEY,SECRET_KEY,CUSTOMER_ID))
final_result = r.json()
print(final_result)
as I did below instead
print(r.url)
it returns as below
https://api.naver.com/stats?ids=nkw0001&ids=nkw0002&ids=nkw0002&ids=nkw0002&fields=clkCnt&fields=impCnt&timeRange=since&timeRange=until
the 'ids' is repeated and doesn't have dates that I put.
how would I make my code to fit with the right url?
Query strings are key-value pairs. All keys and all values are strings. Anything that is not trivially convertible to string depends on convention. In other words, there is no standard for these things, so it depends on the expectations of the API.
For example, the API could define that lists of values are to be given as comma-separated strings, or it could say that anything complex should be JSON-encoded.
In fact, that's exactly what the API documentation says:
fields string
Fields to be retrieved (JSON format string).
For example, ["impCnt","clkCnt","salesAmt","crto"]
The same goes for timeRange. The other values can be left alone. Therefore we JSON-encode those two values only.
We can do that inline with a dict comprehension.
import json
import requests
params = {
'fields': ["clkCnt", "impCnt"],
'ids': 'nkw0001,nkw0002,nkw0003,nkw0004',
'timeRange': {"since":"2019-05-25","until":"2019-06-17"},
}
resp = requests.get('https://api.naver.com/api/stats', {
key: json.dumps(value) if key in ['fields', 'timeRange'] else value for key, value in params.items()
})
On top of complying with the API's expectations, all keys and values that go into the query string need to be URL-encoded. Luckily the requests module takes care of that part, so all we need to do is pass a dict to requests.get.
I am trying to post multiple files using requests post.
The format that is stated here is:
>>> url = 'http://httpbin.org/post'
>>> multiple_files = [('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),
('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]
>>> r = requests.post(url, files=multiple_files)
>>> r.text
So I am trying to do this with a list comprehension. But only the last image is uploading. I have a feeling that the comprehension is overriding the images since all images have the same name 'visuals'. But I required all of them to have the name 'visuals'.
images=[list of image URLS]
files=[('visuals',(str(index)+'.jpg',requests.get(image).content,'image/jpeg')) for index,image in enumerate(images)]
requests.post(script.php,files=files)
For example, if there are 20 images, only 20.jpg is sent to my script.php.
Response to answer (not working):
images=response.xpath(root+'/photos//url/text()').extract()
visuals=[(str(index)+'.jpg',requests.get(image).content,'image/jpeg') for index,image in enumerate(images)]
requests.post(triggers,data={'json':json.dumps(array)},files={'visuals':visuals})
The files argument to requests.post is supposed to be a dict. See the example here. Since you pass it with a list object, it will be casted internally to a dict object. During the type casting, a latter element will overwrite the former element with the same key value! Since all element in has the same key "visuals", only the last one will remain in the final dict object.
This question was already answered in requests' issue page #737.
The list of tuples you provided to data has dict() called on it. Dictionaries (obviously) don't allow duplicate keys, but your list of tuples has duplicate keys, so the last item in the iterable takes the value for that key ... so I'd assume that this is intended behaviour.
I'm trying to pull very specific elements from a dictionary of RSS data that was fetched using the feedparser library, then place that data into a new dictionary so it can be called on later using Flask. The reason I'm doing this is because the original dictionary contains tons of metadata I don't need.
I have broken down the process into simple steps but keep getting hung up on creating the new dictionary! As it is below, it does create a dictionary object, but it's not comprehensive-- it only contains a single article's title, URL and description-- the rest is absent.
I've tried switching to other RSS feeds and had the same result, so it would appear the problem is either the way I'm trying to do it, or there's something wrong with the structure of the list generated by feedparser.
Here's my code:
from html.parser import HTMLParser
import feedparser
def get_feed():
url = "http://thefreethoughtproject.com/feed/"
front_page = feedparser.parse(url)
return front_page
feed = get_feed()
# make a dictionary to update with the vital information
posts = {}
for i in range(0, len(feed['entries'])):
posts.update({
'title': feed['entries'][i].title,
'description': feed['entries'][i].summary,
'url': feed['entries'][i].link,
})
print(posts)
Ultimately, I'd like to have a dictionary like the following, except that it keeps going with more articles:
[{'Title': 'Trump Does Another Ridiculous Thing',
'Description': 'Witnesses looked on in awe as the Donald did this thing',
'Link': 'SomeNewsWebsite.com/Story12345'},
{...},
{...}]
Something tells me it's a simple mistake-- perhaps the syntax is off, or I'm forgetting a small yet important detail.
The code example you provided does an update to the same dict over and over again. So, you only get one dict at the end of the loop. What your example data shows, is that you actually want a list of dictionaries:
# make a list to update with the vital information
posts = []
for entry in feed['entries']:
posts.append({
'title': entry.title,
'description': entry.summary,
'url': entry.link,
})
print(posts)
Seems that the problem is that you are using a dict instead of a list. Then you are updating the same keys of the dict, so each iteration you are overriding the last content added.
I think that the following code will solve your problem:
from html.parser import HTMLParser
import feedparser
def get_feed():
url = "http://thefreethoughtproject.com/feed/"
front_page = feedparser.parse(url)
return front_page
feed = get_feed()
# make a dictionary to update with the vital information
posts = [] # It should be a list
for i in range(0, len(feed['entries'])):
posts.append({
'title': feed['entries'][i].title,
'description': feed['entries'][i].summary,
'url': feed['entries'][i].link,
})
print(posts)
So as you can see the code above are defining the posts variable as a list. Then in the loop we are adding dicts to this list, so it will give you the data structure that you want.
I hope to help you with this solution.
Summary: dictionary/json object indicates it does not have a given key (using either a hasattr call or a value in object.keys boolean test even though that key shows up in an object.keys() call. So how can I access the value for that key?
Longer version: I am quite puzzled trying to parse some json coming back from an API. When I try to determine whether the json object, which is showing up as a dictionary, has a given key, the code returns false for the key even when it shows the key is there for the object.
Here is how I am retrieving the json:
r = requests.get(url, headers = {'User-Agent':UA})
try:
print(r.json())
jsonobject = r.json()
print("class of jsonobject is %s"%jsonobject.__class__.__name__)
print("here are dictionary keys %s"%jsonobject.keys())
if hasattr(jsonobject, 'laps') and jsonobject['laps'] is not None:
...
else:
print("no laps object")
if hasattr(jsonobject, 'points') and jsonobject['points'] is not None:
...
The reason I am doing this is that often I am getting encoding errors from the field nested within the 'laps' array or the 'points' array so that I cannot insert the json data into a MongoDB database. I would like to delete these fields from the json object since they don't contain useful information anyway.
The problem is that the json object is always returning false for hasattr(jsonobject, 'laps') and hasattr(jsonobject,'points'. It returned false even in the case of a record where I then printed out the keys and they showed:
here are dictionary keys dict_keys(['is_peptalk_allowed', 'show_workout', 'hydration', 'records', 'include_in_stats', 'expand', 'pb_count', 'start_time', 'calories', 'altitude_max', 'hashtags', 'laps', 'pictures', 'duration', 'playlist'\
, 'sport', 'points', 'show_map', 'local_start_time', 'speed_avg', 'tagged_users', 'distance', 'altitude_min', 'is_live', 'author', 'feed_id', 'speed_max', 'id'])
So I thought perhaps the dict was behaving strangely with hasattr, and rewrote the code as:
if 'laps' in jsonobject.keys() and jsonobject['laps'] is not None:
but that also returns false even thoug hit again prints the same array of keys that does include 'laps'.
hasattr() is entirely the wrong tool to use. It tests for attributes, but dictionary keys are not attributes.
To test for keys, use the in test directly against the dictionary:
if 'lap' in jsonobject:
Calling jsonobject.keys() is redundant and creates a new dictionary view object.
It'll be true for your dictionary, but that's not the only thing you are testing for. Your test is:
if 'lap' in jsonobject and jsonobject['lap'] is not None:
That'll fail if 'lap' is a key but the value in the dictionary is None.
The above test can be more simply and compactly stated as:
if jsonobject.get('lap') is not None:
If None is a valid value, don't test for it; stick to just 'lap' in jsonobject.