I have a problem regarding JSON library in Python. I can't figure out a way to read data from json file that looks like this:
{"name": "LOTR", "author": "Tolkin"}{"name": "Aska", "author": "Ivo"}
because when I try to load data using this code:
with open("json_books.txt","r") as file:
json_data = json.load(file)
I get the following error:
json.decoder.JSONDecodeError: Extra data: line 1 column 37 (char 36)
I've looked it up and none of the solutions I found helped me. If anyone can help me with this one it would be much appreciated.
You can read the file content as a string, extract the "char" number, which is an index, from the error message of the JSONDecodeError exception, and reparse the slice of the string up to that index as valid JSON, and parse the rest of the string in the same way, until it no longer raises an error:
import json
import re
s = '{"name": "LOTR", "author": "Tolkin"}{"name": "Aska", "author": "Ivo"}'
json_data = []
while True:
try:
json_data.append(json.loads(s))
break
except json.JSONDecodeError as e:
match = re.match(r'Extra data: .*\(char (\d+)\)', str(e))
if match:
index = int(match.group(1))
json_data.append(json.loads(s[:index]))
s = s[index:]
else:
raise
print(json_data)
This outputs:
[{'name': 'LOTR', 'author': 'Tolkin'}, {'name': 'Aska', 'author': 'Ivo'}]
What you listed is not valid JSON. JSON must have a single list or object at top level -- this has two objects.
Perhaps the proper JSON would instead be:
[{"name": "LOTR", "author": "Tolkin"}, {"name": "Aska", "author": "Ivo"}]
Related
json file =
{
"success": true,
"terms": "https://curr
"privacy": "https://cu
"timestamp": 162764598
"source": "USD",
"quotes": {
"USDIMP": 0.722761,
"USDINR": 74.398905,
"USDIQD": 1458.90221
}
}
The json file is above. i deleted lot of values from the json as it took too many spaces. My python code is in below.
import urllib.request, urllib.parse, urllib.error
import json
response = "http://api.currencylayer.com/live?access_key="
api_key = "42141e*********************"
parms = dict()
parms['key'] = api_key
url = response + urllib.parse.urlencode(parms)
mh = urllib.request.urlopen(url)
source = mh.read().decode()
data = json.loads(source)
pydata = json.dumps(data, indent=2)
print("which curreny do you want to convert USD to?")
xm = input('>')
print(f"Hoe many USD do you want to convert{xm}to")
value = input('>')
fetch = pydata["quotes"][0]["USD{xm}"]
answer = fetch*value
print(fetch)
--------------------------------
Here is the
output
"fetch = pydata["quotes"][0]["USD{xm}"]
TypeError: string indices must be integers"
First of all the JSON data you posted here is not valid. There are missing quotes and commas. For example here "terms": "https://curr. It has to be "terms": "https://curr",. The same at "privacy" and the "timestamp" is missing a comma. After i fixed the JSON data I found a solution. You have to use data not pydata. This mean you have to change fetch = pydata["quotes"][0]["USD{xm}"] to fetch = data["quotes"][0]["USD{xm}"]. But this would result in the next error, which would be a KeyError, because in the JSON data you provided us there is no array after the "qoutes" key. So you have to get rid of this [0] or the json data has to like this:
"quotes":[{
"USDIMP": 0.722761,
"USDINR": 74.398905,
"USDIQD": 1458.90221
}]
At the end you only have to change data["quotes"]["USD{xm}"] to data["quotes"]["USD"+xm] because python tries to find a key called USD{xm} and not for example USDIMP, when you type "IMP" in the input.I hope this fixed your problem.
After opening and before loading a json file on phython, the code end up getting a string filled with unicode blocks between every character. Its seems to be a encoding problem, any easy way to solve this problem?
import json
import io
# read file
with open('BOVA111618484700 (1).json', 'r',encoding="ASCII") as myfile:
data=myfile.read()
print(data)
# parse file
obj = json.loads(data)
print(data) shows:
[�
�{�
�"�d�a�t�a�h�o�r�a�"�:� �"�2�0�2�1�.�0�4�.�1�5� �1�1�:�0�5�:�0�0�"�,�
�"�m�i�l�i�s�e�c�o�n�d�s�"�:� �"�1�6�1�8�4�8�4�7�0�0�2�3�4�"�,�
�"�b�i�d�"�:� �"�1�1�6�.�3�2�"�,�
�"�a�s�k�"�:� �"�1�1�6�.�3�6�"�,�
�"�l�a�s�t�"�:� �"�1�1�6�.�3�2�"�,�
�"�v�o�l�u�m�e�"�:� �"�1�"�,�
�"�f�l�a�g�s�"�:� �"�2�"�
�}�,� #json string continues...
when it should show:
[
{
"datahora": "2021.04.15 11:05:00",
"miliseconds": "1618484700234",
"bid": "116.32",
"ask": "116.36",
"last": "116.32",
"volume": "1",
"flags": "2"
}, #json string continues...
After the print, the json.load function returns this error:
JSONDecodeError: Expecting value: line 1 column 2 (char 1)
Thanks #Grismar and #tevemadar the encode of the file was actually "UTF-16 LE" assigning this to the open function solve everything!
import json
import io
# read file
with open('BOVA111618484700 (1).json', 'r',encoding="UTF-16 LE") as myfile:
data=myfile.read()
print(data)
# parse file
obj = json.loads(data)
I am trying to substitute a value using safe substitute. Before this, I am converting an array using JSON dumps and then substituting. Once the substitution is done I am doing JSON loads and passing as a parameter to other utility. While doing this I am getting an error for JSON loads. Below is the code...
account_id={'ABC123', user_id='testing'}
var1 = {'account':account_id, 'user':user_id}
response = json.dumps(var1)
payload = Template.(test_template).safe_substitute(var1=var1)
output = json.loads(payload)
get an error when it comes to loads:
Expecting "," delimiter: line 1 column 448 (char 447)
It's seems to be a syntax error. Try, like below
account_id='ABC123'
user_id='testing'
var1 = {'account':account_id, 'user':user_id}
response = json.dumps(var1)
print(response)
# out: '{"account": "ABC123", "user": "testing"}'
output = json.loads(response)
print(output)
# out: {'user': 'testing', 'account': 'ABC123'}
I am doing a task in python (learning phase) wherein i have a text file with list of ip's eg:
10.8.9.0
10.7.8.7
10.4.5.6 and so on. Each on one line , one below another.
I have to read its contents and create its json as [{"ip":"10.8.9.0"},{"ip":"10.7.8.7"}..]
Code:
with open("filename.txt") as file:
content = [x.strip('\n') for x in file.readlines()]
print content
print "content",type(content)
content_json=json.dumps(content)
print content_json
print type(content_json)
The output of content is ['ip adrress1','ip address2'] which is a list.
When i dump the list in content_json the type shown is "Str" .
However i need it as json
My concern is - my further task is to validate ip and add a item in existing json stating {"status":"valid/invalid"}.
I dnt know how to do that as the type of my json is showing str.
Kindly let me knw how to proceed and add status for every ip in existing json.
Also i wish to know why is the type of the json i dumped my list with is being showed as str.
The desired output should be
[
{
"ip":"10.8.9.0",
"status":"valid"
},
{
"ip":"10.7.8.A",
"status":"invalid"
}, ..so on
]
First thing: The result is a list because you're building a list with
[x.strip('\n') for x in file.readlines()]. In case you're not sure that means: Take every line x in file, remove the \n character from it and then build a list of those results. You want something like [{"ip":x.strip('\n')} for x in file.readlines()].
Now, the function json.dumps takes a Python object and attempts to create a JSON representation of it. That representation is serialized as a string so if you ask for the type of content_json that's what you'll get.
You have to make the distinction between a python list/dictionary and a JSON string.
This
>>> with open('input.txt') as inp:
... result = [dict(ip=ip.strip()) for ip in inp]
...
>>> result
[{'ip': '10.8.9.0'}, {'ip': '10.7.8.7'}, {'ip': '10.4.5.6'}]
will give you a list of dictionaries that is easy to mutate. When you are done with it, you can dump it as a JSON string:
>>> result[1]['status'] = 'valid'
>>> result
[{'ip': '10.8.9.0'}, {'status': 'valid', 'ip': '10.7.8.7'}, {'ip': '10.4.5.6'}]
>>> json.dumps(result)
'[{"ip": "10.8.9.0"}, {"status": "valid", "ip": "10.7.8.7"}, {"ip": "10.4.5.6"}]'
You should supply key:value properly for the dump. Putting just the value alone would store it as String
Refer this :
https://docs.python.org/2/library/json.html
Maybe something like this?
import json
import socket
result = list()
with open("filename.txt") as file:
for line in file:
ip = line.strip()
try:
socket.inet_aton(ip)
result.append({"ip": line.strip(), "status": "valid"})
except socket.error:
result.append({"ip": line.strip(), "status": "invalid"})
print(json.dumps(result))
Finally, I got a fix:
import os
import sys
import json
from IPy import IP
filepath="E:/Work/"
filename="data.txt"
result = list()
with open(os.path.join(filepath+filename)) as file:
for line in file:
ip = line.strip()
if ip.startswith("0"):
result.append({"ip": line.strip(), "status": "invalid"})
else:
try:
ip_add=IP(ip)
result.append({"ip": line.strip(), "status": "Valid"})
except ValueError:
result.append({"ip": line.strip(), "status": "invalid"})
print(json.dumps(result))
This is just a part of my json file which looks like:
"network_lo": "127.0.0.0",
"ec2_block_device_mapping_root": "/dev/sda1",
"selinux": "false",
"uptime_seconds": 127412,
"ec2_reservation_id": "r-cd786568",
"sshdsakey": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"ec2_block_device_mapping_ami": "/dev/sda1",
"memorysize": "3.66 GB",
"swapsize": "0.00 kB",
"netmask": "255.255.255.192",
"uniqueid": "24wq0see",
"kernelmajversion": "3.2",
I have a Python scipt which download this file.. i want to parse this file and remove a number of objects like "swapsize","sshdsakey"
sqs = boto.sqs.connect_to_region("ap-west-1")
q = sqs.get_queue("deathvally")
m = q.read(visibility_timeout=15)
if m == None:
print "No message!"
else:
with open('download.json', 'w') as json_data:
print m.get_body()
json_data.write(m.get_body())
json_data.close()
# I want a logic here which can simply delete the specific json objects
# Something like this is what i tried but didn't work...
# clean_data = json.load(json_data)
# for element in clean_data: ##
# del element['sshdsakey']
# json_data.write(clean_data)
I basically need to parse the fetched json file and then remove the specific objects and then just write this new modified stuff in a file.
json.loads will decode JSON string into Python dictionary (Although format you provided is not a valid JSON format, there have to be curly braces on each side), then you can delete the needed keys with del , encode dictionary back to JSON string with json.dumps and write the resultit
clean_data = json.loads(json_data.read())
del clean_data[your_key]
with open(your_file_to_write, 'w') as f:
f.write(json.dumps(clean_data))
You can parse your json using loads from native json module.
Then delete an element from the dict using del
import json
keys_to_remove = ['sshdsakey', 'selinux']
json_str = '''{
"network_lo": "127.0.0.0",
"ec2_block_device_mapping_root": "/dev/sda1",
"selinux": "false",
"uptime_seconds": 127412,
"ec2_reservation_id": "r-cd786568",
"sshdsakey": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}'''
data = json.loads(json_str)
for key in keys_to_remove:
if key in data:
del data[key]
print data
You need to first convert the JSON object string into a Python dict, delete the keys from it, and then write to to the output file.
import json
sqs = boto.sqs.connect_to_region("ap-west-1")
q = sqs.get_queue("deathvally")
m = q.read(visibility_timeout=15)
if m is None:
print "No message!"
else:
KEYS_TO_REMOVE = "swapsize", "sshdsakey", "etc"
with open('download.json', 'w') as json_data:
json_obj = json.loads(m.get_body())
for key in KEYS_TO_REMOVE:
try:
del json_obj[key]
except KeyError:
pass
json_data.write(json.dumps(json_obj, indent=4))