As I know json.loads() take string as input, so I don't know why below code doesn't work.
import json
fileref = open("olympics.txt", "r").readline() # the output is str
print(fileref)
print(type(fileref))
d = json.loads(fileref)
Error:
ExternalError: SyntaxError: Unexpected token N in JSON at position 0 on line 8
Thank you!
Thats because fileref is not a syntactically valid JSON string. Unless you know the first line of the file is supposed to be a JSON object then I'm assuming the entire file is a JSON object (which is common). You can use the json.load function to read in a file https://docs.python.org/2/library/json.html.
Or you could store the file into a string and then use loads like
content = ""
with open("file.txt","r") as f:
for line in f:
content += line
json.loads(content)
EDIT: I do not know you're file structure, I am just assuming that its a JSON file (i.e its all JSON). If that is the case then what I mentioned would be accurate. Otherwise the error you are seeing is because the string being parsed is not valid JSON.
Related
I have a simple code to be able to read my json data in python and convert to dictionary so I can use it for sqlite. However, I keep running into this error.
Code I ran:
import json
with open("users.json", "r") as f:
json_str = f.read()
json_value = json.loads(json_str)
print(type(json_value))
I have a set of data like this:
{"_id":{"$od":"5f12"},"ready":true,"createdon":{"$date":1609687444800},"lastLogin":{"$date":1858},"position":"consumer","signUp":"Email","state":"WI"}
{"_id":{"$od":"5f12"},"ready":true,"createdon":{"$date":1609687444800},"lastLogin":{"$date":1858},"position":"consumer","signUp":"Email","state":"WI"}
The error I get:
Traceback (most recent call last):
File "json_to_sqlite.py", line 5, in <module>
json_value = json.loads(json_str)
File "/Users/malaba/opt/anaconda3/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/Users/malaba/opt/anaconda3/lib/python3.8/json/decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 186)
#rv.kvetch is right, the problem is that your users.json file has two distinct json strings, i.e. you have two dictionaries stored in your file when you can have at most one. An easy way to fix this is to wrap the dictionaries in a list (hence, you have one json object that holds all your other objects). The newly formatted users.json file would look something like this:
[{"_id":{"$od":"5f12"},"ready":true,"createdon":{"$date":1609687444800},"lastLogin":{"$date":1858},"position":"consumer","signUp":"Email","state":"WI"},
{"_id":{"$od":"5f12"},"ready":true,"createdon":{"$date":1609687444800},"lastLogin":{"$date":1858},"position":"consumer","signUp":"Email","state":"WI"}]
Please make note of the opening and closing braces and the extra comma separating the dictionaries in your new file. Reading the json file would be the same procedure as before, but if you wanted to pull a specific dictionary out of your returned json, you would need to index the list accordingly. More details on the json module can be found here.
EDIT:
If you have a large data file, this sort of operation is not feasible to do manually so you will have to get clever to look for additional structure to the file. For example, if you know that each dictionary in the file is separated by a newline and there are no other newline characters in your file (similar to the example you provided), then you can automatically do this conversion like so:
import json
with open("users.json", "r") as f:
new = [json.loads(x) for x in f.read().splitlines()]
with open("users.json", "w") as f:
json.dump(new, f)
The google colab code is shown below:
annotations = []
with open('/content/orig.json') as f:
for line in f:
annotations.append(json.loads(line))
The error which I get is
JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
Let me know if I need anymore details.
Assume that your file is .json extension but every line is a JSON string, then line 2 in your JSON file 'orig.json' is malformed, it's not in correct JSON syntax, eg. JSON needs doublequotes around keys (properties) and values, not singlequotes.
But normally, when your files come with .json extension, you don't read line by line that way, parse the whole file instead:
annotations = []
with open('/content/orig.json') as f:
text = f.read()
annotations.append(json.loads(text))
I have big size of json file to parse with python, however it's incomplete (i.e., missing parentheses in the end). The json file consist of one big json object which contains json objects inside. All json object in outer json object is complete, just finishing parenthese are missing.
for example, its structure is like this.
{bigger_json_head:value, another_key:[{small_complete_json1},{small_complete_json2}, ...,{small_complete_json_n},
So final "]}" are missing. however, each small json forms a single row so when I tried to print each line of the json file I have, I get each json object as a single string.
so I've tried to use:
with open("file.json","r",encoding="UTF-8") as f:
for line in f.readlines()
line_arr.append(line)
I expected to have a list with line of json object as its element
and then I tried below after the process:
for json_line in line_arr:
try:
json_str = json.loads(json_line)
print(json_str)
except json.decoder.JSONDecodeError:
continue
I expected from this code block, except first and last string, this code would print json string to console. However, it printed nothing and just got decode error.
Is there anyone who solved similar problem? please help. Thank you
If the faulty json file only miss the final "]}", then you can actually fix it before parse it.
Here is an example code to illustrate:
with open("file.json","r",encoding="UTF-8") as f:
faulty_json_str = f.read()
fixed_json_str = faulty_json_str + ']}'
json_obj = json.loads(fixed_json_str)
Below is the error code I received on my Terminal call of "python lastYearArray.py". (I'm on a MacBook Pro running OSX 10.9.5, editing my Python program in jEdit). According to this link: TypeError: expected string or buffer it appears that the function I am using doesn't take lists either. I would just be working with the JSON file instead of converting it, but this particular JSON data has no headers or element tags, such as "fruits: apple, banana, orange"; it only has a long string of numbers separated by commas and braces.
Traceback (most recent call last):
File "lastYearAnalysis.py", line 8, in <module>
PyListData = json.loads(lastYearArray)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
However,I don't know why it is telling me about a decoder, because as you can see from my rather minimal code here:
#Write json array to a python list to find the iterative location of max value
#should return a value such as "[n]" on req for max
import requests
import json
lastYearArray = requests.get('https://api.github.com/repos/mbostock/d3/stats/contributors')
PyListData = json.loads(lastYearArray)
print data[PyListData]
#TODO: ask for max value
I'm not using an explicit decoder function that I can see; I'm using json.loads. I should add that I am certainly a Python novice with regards to lists and JSON array decoding. Any info, however basic, that you can provide would be immensely helpful to me.
Thank you.
requests.get returns a Response object. That object is not a string object, so you cannot pass it to json.loads.
You have to get the response object’s using response.text, and pass that to json.loads:
lastYearArrayResponse = requests.get('…')
data = json.loads(lastYearArrayResponse.text)
Alternatively, you can also use the response.json() method to get the parsed response since requests already comes with built-in support for JSON responses:
lastYearArrayResponse = requests.get('…')
data = lastYearArrayResponse.json()
You forget to add .text at the end of line lastYearArray = to get the HTML source of the webpage.
Also, I think the last line is not correct...
I have a JSON file (stored in database.txt) I want to modify using python dictionary in method addEvent():
def addEvent(eventName, start, end, place):
newdict={} #make a new dictionnary
newdict["NAME"]=eventName
newdict["START"]=start
newdict["END"]=end
newdict["place"]=place
try:
with open("database.txt",'r') as file:
content=file.read()
dict=json.loads(content) #make dictionnary with original JSON file
liste.append(newdict)
dico["event"]=liste #combine 2dictionnaries
with open("database.txt", 'w') as file:
file.write(str(dico)) #save my new JSON file
except:
...
My problem:
I can run this method only one time, second time I receive an error message:
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
addEvent() method modify my database.txt file: it doesn't incule double quotes anymore but accents so i can't use dict=json.loads(content) second time
My question Did I correctly save my JSON file ? How can I keep my JSON format in my database.txt file (keep double quotes) ?
You produced Python syntax by converting the object with str(); Python strings can use either single or double quotes. To produce a JSON string, use json.dumps():
file.write(json.dumps(dico))
The json module has variants of json.loads() and json.dumps() that work directly with files; there is no need to read or write yourself, just use the function names without the trailing s on file objects directly:
with open("database.txt", 'r') as file:
d = json.load(file)
and
with open("database.txt", 'w') as file:
json.dump(dico, file)
The problem is here:
file.write(str(dico)) #save my new JSON file
Use json.dumps instead:
file.write(json.dumps(dico)) #save my new JSON file