I have a JSON file that I fetch from an API that returns KeyError:0 while I attempt to remove items in a python dict. I assume its a combination of my lack of skill and format of the json.
My goal is to remove all instances of 192.168.1.1 from ip_address_1
My Code:
from api import Request
import requests, json, ordereddict
# prepare request
request = Request().service('').where({"query":"192.168.1.0"}).withType("json")
# call request
response = request.execute()
# parse response into python object
obj = json.loads(response)
# remove items
for i in xrange(len(obj)):
if obj[i]["ip_address_1"] == "192.168.1.1":
obj.pop(i)
# display
print json.dumps(obj,indent=1)
Example JSON:
{
"response": {
"alerts": [
{
"action": "New",
"ip_address_1": "192.168.1.1",
"domain": "example.com",
"ip_address_2": "192.68.1.2"
},
{
"action": "New",
"ip_address_1": "192.168.1.3",
"domain": "example2.com",
"ip_address_2": "192.168.1.1"
}
],
"total": "2",
"query": "192.168.1.0",
}
}
This is incorrect:
# remove items
for i in xrange(len(obj)):
if obj[i]["ip_address_1"] == "192.168.1.1":
obj.pop(i)
You are iterating over an object as if it were a list.
What you want to do:
for sub_obj in obj["response"]["alerts"]:
if sub_obj["ip_address_1"] == "192.168.1.1":
sub_obj.pop("ip_address_1")
I've interpreted your requirements to be:
Remove from the "alerts" list any dictionary with ip_address_1 set
to 192.168.1.1.
Create a list of all other ip_address_1 values.
json.loads(response) produces this dictionary:
{u'response': {u'alerts': [{u'action': u'New',
u'domain': u'example.com',
u'ip_address_1': u'192.168.1.1',
u'ip_address_2': u'192.68.1.2'},
{u'action': u'New',
u'domain': u'example2.com',
u'ip_address_1': u'192.168.1.3',
u'ip_address_2': u'192.168.1.1'}],
u'query': u'192.168.1.0',
u'total': u'2'}}
The "alerts" list is accessed by (assuming the dict is bound to obj):
>>> obj['response']['alerts']
[{u'action': u'New',
u'domain': u'example.com',
u'ip_address_1': u'192.168.1.1',
u'ip_address_2': u'192.68.1.2'},
{u'action': u'New',
u'domain': u'example2.com',
u'ip_address_1': u'192.168.1.3',
u'ip_address_2': u'192.168.1.1'}]
The first part can be done like this:
alerts = obj['response']['alerts']
obj['response']['alerts'] = [d for d in alerts if d.get('ip_address_1') != '192.168.1.1']
Here a list comprehension is used to filter out those dictionaries with ip_address_1 192.168.1.1 and the resulting list is then rebound the the obj dictionary. After this obj is:
>>> pprint(obj)
{u'response': {u'alerts': [{u'action': u'New',
u'domain': u'example2.com',
u'ip_address_1': u'192.168.1.3',
u'ip_address_2': u'192.168.1.1'}],
u'query': u'192.168.1.0',
u'total': u'2'}}
Next, creating a list of the other ip addresses is easy with another list comprehension run on the alerts list after removing the undesired dicts as shown above:
ip_addresses = [d['ip_address_1'] for d in obj['response']['alerts'] if d.get('ip_address_1') is not None]
Notice that we use get() to handle the possibility that some dictionaries might not have a ip_address_1 key.
>>> ip_addresses
[u'192.168.1.3']
Related
I have a list and I would like to turn it into json. but I get the following error:
trackerror:list indices must be integers or slices, not str
and
The 'operation' tag of the JSON file is not correct or does not exist
What could it be?.
Observation:
list indices must be integers or slices, not str
It is generated in this line: url = variables["url"]
I have:
event=
[[{'operacion': 'generar','url':'xxxxxxxx', 'items': [{'unidad': 'un', codigo'001'}]
}]]
I need (json):
event=
{
"operacion":"generar",
"url":"xxxxxxxx,"
"items":[
{
"unidad_de_medida":"un",
"codigo":"001"
}
]
}
Code:
def lambda_handler(event, context):
payload=json.dumps(event,indent=4)
variables= json.loads(payload)
url =variables["url"] --------->>>>I have an error here too.<<<<<<<-----------
headers={'content-type': "application/json",'authorization': "xxxxxxxx",'cache-`enter code here`control': "no-cache",'postman-token': "xxxxxxxx"}
response = requests.request("POST", url, data=payload, headers=headers)
The webservice response indicates that it does not exist.
This should work:
event=[[{'operacion': 'generar','url':'xxxxxxxx', 'items': [{'unidad': 'un', 'codigo':'001'}]}]]
json.dumps(event[0][0])
# '{"operacion": "generar", "url": "xxxxxxxx", "items": [{"unidad": "un", "codigo": "001"}]}'
PS: Your dictionary key codigo is formatted incorrectly. If this is not simply a typo when making this post, you will need to fix this before any processing can be done. However, since Python would throw an error, I'm guessing it's just a formatting error in your post?
Your event list is one level too deep to make any sense. Based on your desired output description, you could do this:
event = [
[
{'operacion': 'generar','url':'xxxxxxxx', 'items': [
{'unidad': 'un', 'codigo': '001'}
]
}
]
]
extractedEvents = []
for e in event:
extractedEvents.append(e[0])
print(extractedEvents)
It's a bit easier to see what you're dealing with if you take the time to format the original source similar to what I've done.
By traversing into the redundant list index with e[0] we're flattening the object by one level, like you indicated you wanted
I'm working with the Microsoft Azure face API and I want to get only the glasses response.
heres my code:
########### Python 3.6 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64, requests, json
###############################################
#### Update or verify the following values. ###
###############################################
# Replace the subscription_key string value with your valid subscription key.
subscription_key = '(MY SUBSCRIPTION KEY)'
# Replace or verify the region.
#
# You must use the same region in your REST API call as you used to obtain your subscription keys.
# For example, if you obtained your subscription keys from the westus region, replace
# "westcentralus" in the URI below with "westus".
#
# NOTE: Free trial subscription keys are generated in the westcentralus region, so if you are using
# a free trial subscription key, you should not need to change this region.
uri_base = 'https://westcentralus.api.cognitive.microsoft.com'
# Request headers.
headers = {
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': subscription_key,
}
# Request parameters.
params = {
'returnFaceAttributes': 'glasses',
}
# Body. The URL of a JPEG image to analyze.
body = {'url': 'https://upload.wikimedia.org/wikipedia/commons/c/c3/RH_Louise_Lillian_Gish.jpg'}
try:
# Execute the REST API call and get the response.
response = requests.request('POST', uri_base + '/face/v1.0/detect', json=body, data=None, headers= headers, params=params)
print ('Response:')
parsed = json.loads(response.text)
info = (json.dumps(parsed, sort_keys=True, indent=2))
print(info)
except Exception as e:
print('Error:')
print(e)
and it returns a list like this:
[
{
"faceAttributes": {
"glasses": "NoGlasses"
},
"faceId": "0f0a985e-8998-4c01-93b6-8ef4bb565cf6",
"faceRectangle": {
"height": 162,
"left": 177,
"top": 131,
"width": 162
}
}
]
I want just the glasses attribute so it would just return either "Glasses" or "NoGlasses"
Thanks for any help in advance!
I think you're printing the whole response, when really you want to drill down and get elements inside it. Try this:
print(info[0]["faceAttributes"]["glasses"])
I'm not sure how the API works so I don't know what your specified params are actually doing, but this should work on this end.
EDIT: Thank you to #Nuageux for noting that this is indeed an array, and you will have to specify that the first object is the one you want.
I guess that you can get few elements in that list, so you could do this:
info = [
{
"faceAttributes": {
"glasses": "NoGlasses"
},
"faceId": "0f0a985e-8998-4c01-93b6-8ef4bb565cf6",
"faceRectangle": {
"height": 162,
"left": 177,
"top": 131,
"width": 162
}
}
]
for item in info:
print (item["faceAttributes"]["glasses"])
>>> 'NoGlasses'
Did you try:
glasses = parsed[0]['faceAttributes']['glasses']
This looks more like a dictionary than a list. Dictionaries are defined using the { key: value } syntax, and can be referenced by the value for their key. In your code, you have faceAttributes as a key that for value contains another dictionary with a key glasses leading to the last value that you want.
Your info object is a list with one element: a dictionary. So in order to get at the values in that dictionary, you'll need to tell the list where the dictionary is (at the head of the list, so info[0]).
So your reference syntax will be:
#If you want to store it in a variable, like glass_var
glass_var = info[0]["faceAttributes"]["glasses"]
#Or if you want to print it directly
print(info[0]["faceAttributes"]["glasses"])
What's going on here? info[0] is the dictionary containing several keys, including faceAttributes,faceId and faceRectangle. faceRectangle and faceAttributes are both dictionaries in themselves with more keys, which you can reference to get their values.
Your printed tree there is showing all the keys and values of your dictionary, so you can reference any part of your dictionary using the right keys:
print(info["faceId"]) #prints "0f0a985e-8998-4c01-93b6-8ef4bb565cf6"
print(info["faceRectangle"]["left"]) #prints 177
print(info["faceRectangle"]["width"]) #prints 162
If you have multiple entries in your info list, then you'll have multiple dictionaries, and you can get all the outputs as so:
for entry in info: #Note: "entry" is just a variable name,
# this can be any name you want. Every
# iteration of entry is one of the
# dictionaries in info.
print(entry["faceAttributes"]["glasses"])
Edit: I didn't see that info was a list of a dictionary, adapted for that fact.
I'm querying the onename api in an effort to get the bitcoin addresses of all the users.
At the moment I'm getting all the user information as a json-esque list, and then piping the output to a file, it looks like this:
[{'0': {'owner_address': '1Q2Tv6f9vXbdoxRmGwNrHbjrrK4Hv6jCsz', 'zone_file': '{"avatar": {"url": "https://s3.amazonaws.com/kd4/111"}, "bitcoin": {"address": "1NmLvYVEZqPGeQNcgFS3DdghpoqaH4r5Xh"}, "cover": {"url": "https://s3.amazonaws.com/dx3/111"}, "facebook": {"proof": {"url": "https://facebook.com/jasondrake1978/posts/10152769170542776"}, "username": "jasondrake1978"}, "graph": {"url": "https://s3.amazonaws.com/grph/111"}, "location": {"formatted": "Mechanicsville, Va"}, "name": {"formatted": "Jason Drake"}, "twitter": {"username": "000001"}, "v": "0.2", "website": "http://1642.com"}', 'verifications': [{'proof_url': 'https://facebook.com/jasondrake1978/posts/10152769170542776', 'service': 'facebook', 'valid': False, 'identifier': 'jasondrake1978'}], 'profile': {'website': 'http://1642.com', 'cover': {'url': 'https://s3.amazonaws.com/dx3/111'}, 'facebook': {'proof': {'url': 'https://facebook.com/jasondrake1978/posts/10152769170542776'}, 'username': 'jasondrake1978'}, 'twitter': {'username': '000001'}, 'bitcoin': {'address': '1NmLvYVEZqPGeQNcgFS3DdghpoqaH4r5Xh'}, 'name': {'formatted': 'Jason Drake'}, 'graph': {'url': 'https://s3.amazonaws.com/grph/111'}, 'location': {'formatted': 'Mechanicsville, Va'}, 'avatar': {'url': 'https://s3.amazonaws.com/kd4/111'}, 'v': '0.2'}}}]
what I'm really interested in is the field {"address": "1NmLvYVEZqPGeQNcgFS3DdghpoqaH4r5Xh"}, the rest of the stuff I don't need, I just want the addresses of every user.
Is there a way that I can just write only the addresses to a file using python?
I'm trying to write it as something like:
1NmLvYVEZqPGeQNcgFS3DdghpoqaH4r5Xh,
1GA9RVZHuEE8zm4ooMTiqLicfnvymhzRVm,
1BJdMS9E5TUXxJcAvBriwvDoXmVeJfKiFV,
1NmLvYVEZqPGeQNcgFS3DdghpoqaH4r5Xh,
...
and so on.
I've tried a number of different ways using dump, dumps, etc. but I haven't yet been able to pin it down.
My code looks like this:
import os
import json
import requests
#import py2neo
import csv
# set up authentication parameters
#py2neo.authenticate("46.101.180.63:7474", "neo4j", "uni-bonn")
# Connect to graph and add constraints.
neo4jUrl = os.environ.get('NEO4J_URL',"http://46.101.180.63:7474/db/data/")
#graph = py2neo.Graph(neo4jUrl)
# Add uniqueness constraints.
#graph.run("CREATE CONSTRAINT ON (q:Person) ASSERT q.id IS UNIQUE;")
# Build URL.
apiUrl = "https://api.onename.com/v1/users"
# apiUrl = "https://raw.githubusercontent.com/s-matthew-english/26.04/master/test.json"
# Send GET request.
Allusersjson = requests.get(apiUrl, headers = {"accept":"application/json"}).json()
#print(json)])
UsersDetails=[]
for username in Allusersjson['usernames']:
usernamex= username[:-3]
apiUrl2="https://api.onename.com/v1/users/"+usernamex+"?app-id=demo-app-id&app-secret=demo-app-secret"
userinfo=requests.get(apiUrl2, headers = {"accept":"application/json"}).json()
# try:
# if('bitcoin' not in userinfo[usernamex]['profile']):
# continue
# else:
# UsersDetails.append(userinfo)
# except:
# continue
try:
address = userinfo[usernamex]["profile"]["bitcoin"]["address"]
UsersDetails.append(address)
except KeyError:
pass # no address
out = "\n".join(UsersDetails)
print(out)
open("out.csv", "w").write(out)
# f = csv.writer(open("test.csv", "wb+"))
# Build query.
query = """
RETURN {json}
"""
# Send Cypher query.
# py2neo.CypherQuery(graph, query).run(json=json)
# graph.run(query).run(json=json)
#graph.run(query,json=json)
anyway, in such a situation, what's the best way to write out those addresses as csv :/
UPDATE
I ran it, and at first it worked, but then I got the following error:
Instead of adding all the information to the UsersDetails list
UsersDetails.append(userinfo)
you can add just the relevant part (address)
try:
address = userinfo[usernamex]["profile"]["bitcoin"]["address"]
UsersDetails.append(address)
except KeyError:
pass # no address
except TypeError:
pass # illformed data
To print the values to the screen:
out = "\n".join(UsersDetails)
print(out)
(replace "\n" with "," for comma separated output, instead of one per line)
To save to a file:
open("out.csv", "w").write(out)
You need to reformat the list, either through map() or a list comprehension, to get it down to just the information you want. For example, if the top-level key used in the response from the api.onename.com API is always 0, you can do something like this
UsersAddresses = [user['0']['profile']['bitcoin']['address'] for user in UsersDetails]
I'm trying to parse JSON, below is my code.
import requests
import json
yatoken = '123123sdfsdf'
listOfId = ['11111111111', '2222222222', '33333333333']
for site in listOfId:
r = requests.get('http://api.ya-bot.net/projects/' + site + '?token=' + yatoken)
parsed = json.loads(r.text)
for url in parsed['project']:
#print url
print str(url['name'])
And JSON:
{
"project":{
"id":"123123sdfsdfs",
"urls":[],
"name":"Имя",
"group":"Группа",
"comments":"",
"sengines":[],
"wordstat_template":1,
"wordstat_regions":[]
}
}
It gives this error
print str(url['name'])
TypeError: string indices must be integers
How I can fix this problem?
Thx.
The 'project' key refers to a dictionary. Looping over that dictionary gives you keys, each a string. You are not looping over the list of URLs. One of those keys will be 'name'
Your code is confusing otherwise. You appear to want to get each URL. To do that, you'd have to loop over the 'urls' key in that nested dictionary:
for url in parsed['project']['urls']:
# each url value
In your sample response that list is empty however.
If you wanted to get the 'name' key from the nested dictionary, just print it without looping:
print parsed['project']['name']
Demo:
>>> import json
>>> parsed = json.loads('''\
... {
... "project":{
... "id":"123123sdfsdfs",
... "urls":[],
... "name":"Имя",
... "group":"Группа",
... "comments":"",
... "sengines":[],
... "wordstat_template":1,
... "wordstat_regions":[]
... }
... }
... ''')
>>> parsed['project']
{u'group': u'\u0413\u0440\u0443\u043f\u043f\u0430', u'name': u'\u0418\u043c\u044f', u'wordstat_regions': [], u'comments': u'', u'urls': [], u'sengines': [], u'id': u'123123sdfsdfs', u'wordstat_template': 1}
>>> parsed['project']['name']
u'\u0418\u043c\u044f'
>>> print parsed['project']['name']
Имя
>>> print parsed['project']['urls']
[]
for url in parsed['project'] returns a dict so you are actually iterating over the keys of the dict so "id"["name"] etc.. is going to error, you can use d = parsed['project'] to get the dict then access the dict by key to get whatever value you want.
d = parsed['project']
print(d["name"])
print(d["urls"])
...
Or iterate over the items to get key and value:
for k, v in parsed['project'].items():
print(k,v)
If you print what is returned you can see exactly what is happening:
In [17]: js = {
"project":{
"id":"123123sdfsdfs",
"urls":[],
"name":"Имя",
"group":"Группа",
"comments":"",
"sengines":[],
"wordstat_template":1,
"wordstat_regions":[]
}
}
In [18]: js["project"] # dict
Out[18]:
{'comments': '',
'group': 'Группа',
'id': '123123sdfsdfs',
'name': 'Имя',
'sengines': [],
'urls': [],
'wordstat_regions': [],
'wordstat_template': 1}
In [19]: for k in js["project"]: # iterating over the keys of the dict
....: print(k)
....:
sengines # k["name"] == error
id
urls
comments
name
wordstat_regions
wordstat_template
group
I'm new to python as was wondering how I could get the estimatedWait and routeName from this string.
{
"lastUpdated": "07:52",
"filterOut": [],
"arrivals": [
{
"routeId": "B16",
"routeName": "B16",
"destination": "Kidbrooke",
"estimatedWait": "due",
"scheduledTime": "06: 53",
"isRealTime": true,
"isCancelled": false
},
{
"routeId":"B13",
"routeName":"B13",
"destination":"New Eltham",
"estimatedWait":"29 min",
"scheduledTime":"07:38",
"isRealTime":true,
"isCancelled":false
}
],
"serviceDisruptions":{
"infoMessages":[],
"importantMessages":[],
"criticalMessages":[]
}
}
And then save this to another string which would be displayed on the lxterminal of the raspberry pi 2. I would like only the 'routeName' of B16 to be saved to the string. How do I do that?
You just have to deserialise the object and then use the index to access the data you want.
To find only the B16 entries you can filter the arrivals list.
import json
obj = json.loads(json_string)
# filter only the b16 objects
b16_objs = filter(lambda a: a['routeName'] == 'B16', obj['arrivals'])
if b16_objs:
# get the first item
b16 = b16_objs[0]
my_estimatedWait = b16['estimatedWait']
print(my_estimatedWait)
You can use string.find() to get the indices of those value identifiers
and extract them.
Example:
def get_vaules(string):
waitIndice = string.find('"estimatedWait":"')
routeIndice = string.find('"routeName":"')
estimatedWait = string[waitIndice:string.find('"', waitIndice)]
routeName = string[routeIndice:string.find('"', routeIndice)]
return estimatedWait, routeName
Or you could just deserialize the json object (highly recommended)
import json
def get_values(string):
jsonData = json.loads(string)
estimatedWait = jsonData['arrivals'][0]['estimatedWait']
routeName = jsonData['arrivals'][0]['routeName']
return estimatedWait, routeName
Parsing values from a JSON file using Python?