I have a question concerning an issue I ran into while using the json lib in Python.
I'm tying to read a json file using the json.load(file) command using the following code:
import json
filename= '../Data/exampleFile.json'
histFile= open(filename, 'w+')
print(json.load(histFile))
The JSON file I am trying to read is valid according to some website I found: a screenshot of that validation, because I'm new and still lack the reputation...
The error message I'm getting is the following:
File ".\testLoad.py", line 5, in <module>
print(json.load(histFile))
File "C:\Users\...\Python\Python37\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Users\...\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\...\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\...\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Alright, so I believe it is not the file that is the issue, but the json.load(file) works for me in other cases.
Sadly I was not able to figure this error-message out on my own, so it would be amazing if someone with some more experience dealing with Python-JSON interaction could maybe help me out.
You opened the file for writing:
histFile= open(filename, 'w+')
# ^^^^
The w mode first truncates the file, so the file is empty (it doesn't matter here that the file can also be read from, the + sees to that but the file is truncated nonetheless). See the open() function documentation:
'w': open for writing, truncating the file first)
There is no JSON data in it to parse. This is why the exception tells you that parsing failed at the very start of the file:
Expecting value: line 1 column 1 (char 0)
There is no data in line one, column one.
If you wanted to open a file for both reading and writing without truncating it first, use 'r+' as the file mode.
Related
I'm trying to store the data published on a topic to a JSON file, but I keep getting a JSONDecodeError.
DB = '/home/path/data.json'
f = open(DB, 'w+')
json_array = json.load(fp=f)
json_array.append(data)
json.dump(json_array, f)
f.close()
The open() command successfully creates the file. But loading fails. I have tried running it with the file contents: [] and {}, both gave the same Exception:
[ERROR] [1637699609.562673]: bad callback: <function callback at 0x7fe36196b1f0>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/path/scripts/extract_info_node.py", line 43, in callback
json_array = json.load(fp=f)
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
It's not really descriptive and I have nothing to go on. Is there something about the way subscribers are run that doesn't allow writing? Can you even do this from a callback function? I have just started working in ROS so it could be something simple or obvious to someone more experienced.
.load() takes a .read() supporting file. You're only opening the file for reading. Instead try this:
f = open(DB, 'r+')
json_array = json.load(f)
As another note, if you're storing and re-reading topic data I'd suggest looking potentially using rosbag. This is, however, dependent on your actual application.
I have a file named dict_file.json in current folder with empty dict content {}
I want to open it such that I can do both read and modify the content. That is I will read the json file as dict, modify that dict and write it back to json file.
I tried r and r+ below:
import json
f = open('dict_file.json', 'r') # same output for r+
json.loads(list_file.read())
This prints
{}
When I tried w+:
f = open('dict_file.json', 'r')
this first clears the file. Then,
json.loads(list_file.read())
gives error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
When I tried a+:
f = open('dict_file.json', 'a+')
json.loads(list_file.read())
gives error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Though, it does not clear the file.
So, I guess I should be using r+ for my usecase scenario. Also I tried reading writing both :
f = open('dict_file.json', 'r+')
a = json.loads(list_file.read())
a = {'key':'value'} # this involves complex logic instead of plain assignment
json.dump(a,f)
f.flush()
Now when I open the file, its contents are weird:
{}{'key':'value'}
whereas I want it to be:
{'key':'value'}
So I have few questions:
Q1. Why w+ and a+ gives error?
Q2. Why file contained {}{'key':'value'} instead of {'key':'value'}?
Q3. How to correctly do reading and writing (with or without with block)?
PS: I am reading file, then running some loop which computes new dict and write it to file. Then loop sleeps for some time and repeats the same. Thats why I felt flush() will be correct here. That is I open file once outside loop and only flush inside the loop. No need to open file in each iteration.
Why w+ and a+ gives error?
Because the file reader is pointed at the end of the file, where there's no json object to read
its contents are weird
Seems like you're not resetting the file to an actual valid JSON object throughout your tests, and so you've managed to clear the file, read nothing, maybe write some other object, clear it again, then append one or two objects to it, etc etc.
Keep it simple. Start over
filename = "data.json"
with open(filename) as f:
data = json.load(f)
data["foo"] = "bar"
with open(filename, "w") as f:
json.dump(data, f)
You have to think in terms of what the operations do to a file when they open it:
r and r+ open a file for reading. This means that the contents of the file is intact, and the file pointer is the beginning of the file. The entire file is visible to the reader. After reading, the file pointer will be at the end, meaning that you're effectively appending at that point.
w and w+ open a file for writing. That means that the contents of the file is truncated. There is nothing to be read in, since the original contents is destroyed.
a and a+ open the file for appending. The contents of the file are unchanged. However, the file pointer is at the end, so a reader will see no data and raise an error.
The correct way to do this is to open the file twice, and use with blocks to do it:
with open('dict_file.json', 'r') as f:
a = json.load(f)
# you don't need the file to be open here
a = {'key': 'value'}
with open('dict_file.json', 'w') as f:
json.dump(a, f)
You might be tempted to try to open the file in read-write mode (r+), read it, then rewind it, so the pointer returns to the beginning, and then overwrite. There are two reasons not to do this:
Not every stream is rewindable
If the contents of the file decreases in size, you will end up with trash at the end. You would need to truncate to the current pointer after writing, which is an unnecessary layer of complexity
I am trying to make a code in python that imports a JSON file from a folder into my program in order for it to access the data inside. However, I am facing errors
global bees
with open('data/bees.json') as f:
bees = json.load(f)["bees"]
Where in the data/bees.json I have this:
{
"bees": []
}
The error I get
Traceback (most recent call last):
File "d:/Entertainment/Coding/Python/Pygame/BUG WORLD/main.py", line 70, in <module>
bees = json.load(f)
File "C:\Users\ernes\AppData\Local\Programs\Python\Python37-32\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Users\ernes\AppData\Local\Programs\Python\Python37-32\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\ernes\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\ernes\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Is there a way I am able to import JSON files from another folder without causing any errors?? Please help
Looks like you're on Windows, so:
with open(r'c:\path\to\file\bees.json') as json_file:
bees_js = json.load(json_file)
The relative path should be open("data/bees.json") without the /, starting a path with / means absolute path from the root.
You might want to try the following:
import json
with open('path_to_file/bees.json') as json_file:
bees_js = json.load(json_file)
I've downloaded a big stream of twitter data in JSON format and saved it to a text file. I now want to read it in, line by line, and decode it into a dictionary using json.reads().
My only problem is that it throws an error on the first line, which I assume means the function doesn't think the data is JSON? I have added the line I want to decode at the bottom of this post. When I just print the lines the code works fine, its only the json.reads() function that throws an error.
Here is the code:
def decodeJSON(tweet_data):
for line in tweet_data:
parsedJSON = json.loads(line)
print(parsedJSON) # I just want to print for now to confirm it works.
Here is the error:
File "/Users/cc756/Dropbox/PythonProjects/TwitterAnalysisAssignment/tweet_sentiment.py", line 17, in analyseSentiment
parsedJSON = json.loads(line) File "/Users/cc756/anaconda/envs/tensorflow/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s) File "/Users/cc756/anaconda/envs/tensorflow/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Users/cc756/anaconda/envs/tensorflow/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Here is the first string:
'b\\'{"delete":{"status":{"id":805444624881811457,"id_str":"805444624881811457","user_id":196129140,"user_id_str":"196129140"},"timestamp_ms":"1500994305560"}}\\''
I feel like it should work, I've been staring at this for an hour with no improvement!
Your strings are in the wrong format. I'm not sure what you need to do to get rid of the 'b\\'(which doesn't really make sense) at the beginning, but manually typing it in to the shell gives me this:
In [119]: json.loads(b'{"delete":{"status":{"id":805444624881811457,"id_str":"80
...: 5444624881811457","user_id":196129140,"user_id_str":"196129140"},"time
...: stamp_ms":"1500994305560"}}')
Out[119]:
{u'delete': {u'status': {u'id': 805444624881811457,
u'id_str': u'805444624881811457',
u'user_id': 196129140,
u'user_id_str': u'196129140'},
u'timestamp_ms': u'1500994305560'}}
Sorry, I'd make a comment, but imagine this post in a comment... :)
I'm not sure what's up with your pasting of the string into the question, but it's following an invalid format for Python, so you may want to correct that.
UPDATE: The issue was that the data was in binary format and just needed to be decoded with data.decode('utf-8')
I'm using Python 3.5.2 on Windows 10 x64. The JSON file I'm reading is this which is a JSON array containing 2 more arrays.
I'm trying to parse this JSON file using the json module. As described in the docs the JSON file must be compliant to RFC 7159. I checked my file here and it tells me it's perfectly fine with the RFC 7159 format, but when trying to read it using this simple python code:
with open(absolute_json_file_path, encoding='utf-8-sig') as json_file:
text = json_file.read()
json_data = json.load(json_file)
print(json_data)
I'm getting this exception:
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm 4.0.5\helpers\pydev\pydevd.py", line 2217, in <module>
globals = debugger.run(setup['file'], None, None)
File "C:\Program Files (x86)\JetBrains\PyCharm 4.0.5\helpers\pydev\pydevd.py", line 1643, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files (x86)\JetBrains\PyCharm 4.0.5\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Andres Torti/Git-Repos/MCF/Sur3D.App/shapes-json-checker.py", line 14, in <module>
json_data = json.load(json_file)
File "C:\Users\Andres Torti\AppData\Local\Programs\Python\Python35-32\lib\json\__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Users\Andres Torti\AppData\Local\Programs\Python\Python35-32\lib\json\__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "C:\Users\Andres Torti\AppData\Local\Programs\Python\Python35-32\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Andres Torti\AppData\Local\Programs\Python\Python35-32\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I can read this exact file perfectly fine on Javascript but I can't get Python to parse it. Is there anything wrong with my file or is any problem with the Python parser?
Try this
import json
with open('filename.txt', 'r') as f:
array = json.load(f)
print (array)
Based on reading over the documentation again, it appears you need to either change the third line to
json_data = json.loads(text)
or remove the line
text = json_file.read()
since read() causes the file's index to reach the end of the file. (I suppose, alternatively, you can reset the index of the file).
For python3, only the below worked for me - none of the above.
import json
with open('/emp.json', 'r') as f:
data=f.read()
print(data)