Develop Reference

Why am I getting this error "TypeError: string indices must be integers" when trying to fetch data from an api?

json file =
{
"success": true,
"terms": "https://curr
"privacy": "https://cu
"timestamp": 162764598
"source": "USD",
"quotes": {
"USDIMP": 0.722761,
"USDINR": 74.398905,
"USDIQD": 1458.90221
}
}
The json file is above. i deleted lot of values from the json as it took too many spaces. My python code is in below.
import urllib.request, urllib.parse, urllib.error
import json
response = "http://api.currencylayer.com/live?access_key="
api_key = "42141e*********************"
parms = dict()
parms['key'] = api_key
url = response + urllib.parse.urlencode(parms)
mh = urllib.request.urlopen(url)
source = mh.read().decode()
data = json.loads(source)
pydata = json.dumps(data, indent=2)
print("which curreny do you want to convert USD to?")
xm = input('>')
print(f"Hoe many USD do you want to convert{xm}to")
value = input('>')
fetch = pydata["quotes"][0]["USD{xm}"]
answer = fetch*value
print(fetch)
--------------------------------
Here is the
output
"fetch = pydata["quotes"][0]["USD{xm}"]
TypeError: string indices must be integers"

First of all the JSON data you posted here is not valid. There are missing quotes and commas. For example here "terms": "https://curr. It has to be "terms": "https://curr",. The same at "privacy" and the "timestamp" is missing a comma. After i fixed the JSON data I found a solution. You have to use data not pydata. This mean you have to change fetch = pydata["quotes"][0]["USD{xm}"] to fetch = data["quotes"][0]["USD{xm}"]. But this would result in the next error, which would be a KeyError, because in the JSON data you provided us there is no array after the "qoutes" key. So you have to get rid of this [0] or the json data has to like this:
"quotes":[{
"USDIMP": 0.722761,
"USDINR": 74.398905,
"USDIQD": 1458.90221
}]
At the end you only have to change data["quotes"]["USD{xm}"] to data["quotes"]["USD"+xm] because python tries to find a key called USD{xm} and not for example USDIMP, when you type "IMP" in the input.I hope this fixed your problem.

Read from JSON file with multiple objects in Python

I have a problem regarding JSON library in Python. I can't figure out a way to read data from json file that looks like this:
{"name": "LOTR", "author": "Tolkin"}{"name": "Aska", "author": "Ivo"}
because when I try to load data using this code:
with open("json_books.txt","r") as file:
json_data = json.load(file)
I get the following error:
json.decoder.JSONDecodeError: Extra data: line 1 column 37 (char 36)
I've looked it up and none of the solutions I found helped me. If anyone can help me with this one it would be much appreciated.

You can read the file content as a string, extract the "char" number, which is an index, from the error message of the JSONDecodeError exception, and reparse the slice of the string up to that index as valid JSON, and parse the rest of the string in the same way, until it no longer raises an error:
import json
import re
s = '{"name": "LOTR", "author": "Tolkin"}{"name": "Aska", "author": "Ivo"}'
json_data = []
while True:
try:
json_data.append(json.loads(s))
break
except json.JSONDecodeError as e:
match = re.match(r'Extra data: .*\(char (\d+)\)', str(e))
if match:
index = int(match.group(1))
json_data.append(json.loads(s[:index]))
s = s[index:]
else:
raise
print(json_data)
This outputs:
[{'name': 'LOTR', 'author': 'Tolkin'}, {'name': 'Aska', 'author': 'Ivo'}]

What you listed is not valid JSON. JSON must have a single list or object at top level -- this has two objects.
Perhaps the proper JSON would instead be:
[{"name": "LOTR", "author": "Tolkin"}, {"name": "Aska", "author": "Ivo"}]

Python JSON decoder error with unicode characters in request content

Using requests library to execute http GET that return JSON response i'm getting this error when response string contains unicode char:
json.decoder.JSONDecodeError: Invalid control character at: line 1 column 20 (char 19)
Execute same http request with Postman the json output is:
{ "value": "VILLE D\u0019ANAUNIA" }
My python code is:
data = requests.get(uri, headers=HEADERS).text
json_data = json.loads(data)
Can I remove or replace all Unicode chars before executing conversion with json.loads(...)?

It is likely to be caused by a RIGHT SINGLE QUOTATION MARK U+2019 (’). For reasons I cannot guess, the high order byte has been dropped leaving you with a control character which should be escaped in a correct JSON string.
So the correct way would be to control what exactly the API returns. If id does return a '\u0019' control character, you should contact the API owner because the problem should be there.
As a workaround, you can try to limit the problem for your processing by filtering out non ascii or control characters:
data = requests.get(uri, headers=HEADERS).text
data = ''.join((i for i in data if 0x20 <= ord(i) < 127)) # filter out unwanted chars
json_data = json.loads(data)
You should get {'value': 'VILLE DANAUNIA'}
Alternatively, you can replace all unwanted characters with spaces:
data = requests.get(uri, headers=HEADERS).text
data = ''.join((i if 0x20 <= ord(i) < 127 else ' ' for i in data))
json_data = json.loads(data)
You would get {'value': 'VILLE D ANAUNIA'}

The code below works on python 2.7:
import json
d = json.loads('{ "value": "VILLE D\u0019ANAUNIA" }')
print(d)
The code below works on python 3.7:
import json
d = json.loads('{ "value": "VILLE D\u0019ANAUNIA" }', strict=False)
print(d)
Output:
{u'value': u'VILLE D\x19ANAUNIA'}
Another point is that requests get return the data as json:
r = requests.get('https://api.github.com/events')
r.json()

how can i take a specific element from this list in python?

I'm working with the Microsoft Azure face API and I want to get only the glasses response.
heres my code:
########### Python 3.6 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64, requests, json
###############################################
#### Update or verify the following values. ###
###############################################
# Replace the subscription_key string value with your valid subscription key.
subscription_key = '(MY SUBSCRIPTION KEY)'
# Replace or verify the region.
#
# You must use the same region in your REST API call as you used to obtain your subscription keys.
# For example, if you obtained your subscription keys from the westus region, replace
# "westcentralus" in the URI below with "westus".
#
# NOTE: Free trial subscription keys are generated in the westcentralus region, so if you are using
# a free trial subscription key, you should not need to change this region.
uri_base = 'https://westcentralus.api.cognitive.microsoft.com'
# Request headers.
headers = {
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': subscription_key,
}
# Request parameters.
params = {
'returnFaceAttributes': 'glasses',
}
# Body. The URL of a JPEG image to analyze.
body = {'url': 'https://upload.wikimedia.org/wikipedia/commons/c/c3/RH_Louise_Lillian_Gish.jpg'}
try:
# Execute the REST API call and get the response.
response = requests.request('POST', uri_base + '/face/v1.0/detect', json=body, data=None, headers= headers, params=params)
print ('Response:')
parsed = json.loads(response.text)
info = (json.dumps(parsed, sort_keys=True, indent=2))
print(info)
except Exception as e:
print('Error:')
print(e)
and it returns a list like this:
[
{
"faceAttributes": {
"glasses": "NoGlasses"
},
"faceId": "0f0a985e-8998-4c01-93b6-8ef4bb565cf6",
"faceRectangle": {
"height": 162,
"left": 177,
"top": 131,
"width": 162
}
}
]
I want just the glasses attribute so it would just return either "Glasses" or "NoGlasses"
Thanks for any help in advance!

I think you're printing the whole response, when really you want to drill down and get elements inside it. Try this:
print(info[0]["faceAttributes"]["glasses"])
I'm not sure how the API works so I don't know what your specified params are actually doing, but this should work on this end.
EDIT: Thank you to #Nuageux for noting that this is indeed an array, and you will have to specify that the first object is the one you want.

I guess that you can get few elements in that list, so you could do this:
info = [
{
"faceAttributes": {
"glasses": "NoGlasses"
},
"faceId": "0f0a985e-8998-4c01-93b6-8ef4bb565cf6",
"faceRectangle": {
"height": 162,
"left": 177,
"top": 131,
"width": 162
}
}
]
for item in info:
print (item["faceAttributes"]["glasses"])
>>> 'NoGlasses'

Did you try:
glasses = parsed[0]['faceAttributes']['glasses']

This looks more like a dictionary than a list. Dictionaries are defined using the { key: value } syntax, and can be referenced by the value for their key. In your code, you have faceAttributes as a key that for value contains another dictionary with a key glasses leading to the last value that you want.
Your info object is a list with one element: a dictionary. So in order to get at the values in that dictionary, you'll need to tell the list where the dictionary is (at the head of the list, so info[0]).
So your reference syntax will be:
#If you want to store it in a variable, like glass_var
glass_var = info[0]["faceAttributes"]["glasses"]
#Or if you want to print it directly
print(info[0]["faceAttributes"]["glasses"])
What's going on here? info[0] is the dictionary containing several keys, including faceAttributes,faceId and faceRectangle. faceRectangle and faceAttributes are both dictionaries in themselves with more keys, which you can reference to get their values.
Your printed tree there is showing all the keys and values of your dictionary, so you can reference any part of your dictionary using the right keys:
print(info["faceId"]) #prints "0f0a985e-8998-4c01-93b6-8ef4bb565cf6"
print(info["faceRectangle"]["left"]) #prints 177
print(info["faceRectangle"]["width"]) #prints 162
If you have multiple entries in your info list, then you'll have multiple dictionaries, and you can get all the outputs as so:
for entry in info: #Note: "entry" is just a variable name,
# this can be any name you want. Every
# iteration of entry is one of the
# dictionaries in info.
print(entry["faceAttributes"]["glasses"])
Edit: I didn't see that info was a list of a dictionary, adapted for that fact.

Read text file and dump to json object

I am doing a task in python (learning phase) wherein i have a text file with list of ip's eg:
10.8.9.0
10.7.8.7
10.4.5.6 and so on. Each on one line , one below another.
I have to read its contents and create its json as [{"ip":"10.8.9.0"},{"ip":"10.7.8.7"}..]
Code:
with open("filename.txt") as file:
content = [x.strip('\n') for x in file.readlines()]
print content
print "content",type(content)
content_json=json.dumps(content)
print content_json
print type(content_json)
The output of content is ['ip adrress1','ip address2'] which is a list.
When i dump the list in content_json the type shown is "Str" .
However i need it as json
My concern is - my further task is to validate ip and add a item in existing json stating {"status":"valid/invalid"}.
I dnt know how to do that as the type of my json is showing str.
Kindly let me knw how to proceed and add status for every ip in existing json.
Also i wish to know why is the type of the json i dumped my list with is being showed as str.
The desired output should be
[
{
"ip":"10.8.9.0",
"status":"valid"
},
{
"ip":"10.7.8.A",
"status":"invalid"
}, ..so on
]

First thing: The result is a list because you're building a list with
[x.strip('\n') for x in file.readlines()]. In case you're not sure that means: Take every line x in file, remove the \n character from it and then build a list of those results. You want something like [{"ip":x.strip('\n')} for x in file.readlines()].
Now, the function json.dumps takes a Python object and attempts to create a JSON representation of it. That representation is serialized as a string so if you ask for the type of content_json that's what you'll get.

You have to make the distinction between a python list/dictionary and a JSON string.
This
>>> with open('input.txt') as inp:
... result = [dict(ip=ip.strip()) for ip in inp]
...
>>> result
[{'ip': '10.8.9.0'}, {'ip': '10.7.8.7'}, {'ip': '10.4.5.6'}]
will give you a list of dictionaries that is easy to mutate. When you are done with it, you can dump it as a JSON string:
>>> result[1]['status'] = 'valid'
>>> result
[{'ip': '10.8.9.0'}, {'status': 'valid', 'ip': '10.7.8.7'}, {'ip': '10.4.5.6'}]
>>> json.dumps(result)
'[{"ip": "10.8.9.0"}, {"status": "valid", "ip": "10.7.8.7"}, {"ip": "10.4.5.6"}]'

You should supply key:value properly for the dump. Putting just the value alone would store it as String
Refer this :
https://docs.python.org/2/library/json.html

Maybe something like this?
import json
import socket
result = list()
with open("filename.txt") as file:
for line in file:
ip = line.strip()
try:
socket.inet_aton(ip)
result.append({"ip": line.strip(), "status": "valid"})
except socket.error:
result.append({"ip": line.strip(), "status": "invalid"})
print(json.dumps(result))

Finally, I got a fix:
import os
import sys
import json
from IPy import IP
filepath="E:/Work/"
filename="data.txt"
result = list()
with open(os.path.join(filepath+filename)) as file:
for line in file:
ip = line.strip()
if ip.startswith("0"):
result.append({"ip": line.strip(), "status": "invalid"})
else:
try:
ip_add=IP(ip)
result.append({"ip": line.strip(), "status": "Valid"})
except ValueError:
result.append({"ip": line.strip(), "status": "invalid"})
print(json.dumps(result))