I am doing a simple wrapper program around a software tool, and this wrapper will, among other things, parse a JSON configuration file that will be coded by the user as a way for the user to specify certain configutations to the program.
Within the program, I have the following lines of code:
with open(os.path.join(args.config_bin_dir, 'basic_config.json'), 'r') as jsondata :
print "JSON File contents:", jsondata.read()
basic_config.update(json.load(jsondata))
print "The basic_config after updating with JSON:", basic_config
jsondata.close()
The basic_config object is a dictionary object that gets constructed somewhere on top earlier up in the program, and over here I want to add the new dictionary key-value pairs obtained from parsing the user's JSON configuration file into the basic_config dictionary object.
When the program is run, I get the following error, a section of which is as follows:
File "path/to/python/script.py", line 42, in main
basic_config.update(json.load(jsondata))
File "/usr/lib/python2.7/json/__init__.py", line 290, in load
**kw)
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
I'm quite puzzled by this error, but I noticed that it goes away so long as I remove the line print "JSON File contents:", jsondata.read() and run the code as either:
with open(os.path.join(args.config_bin_dir, 'basic_config.json'), 'r') as jsondata :
# print "JSON File contents:", jsondata.read()
basic_config.update(json.load(jsondata))
print "The basic_config after updating with JSON:", basic_config
jsondata.close()
or
with open(os.path.join(args.config_bin_dir, 'basic_config.json'), 'r') as jsondata :
# print "JSON File contents:", jsondata.read()
basic_config.update(json.load(jsondata))
# print "The basic_config after updating with JSON:", basic_config
jsondata.close()
I'll still get the same error if I comment out the line after the json.load method:
with open(os.path.join(args.config_bin_dir, 'basic_config.json'), 'r') as jsondata :
print "JSON File contents:", jsondata.read()
basic_config.update(json.load(jsondata))
# print "The basic_config after updating with JSON:", basic_config
jsondata.close()
So I'm guessing that it has to do with calling the read method on the JSON file object before parsing that same file. Could anyone please explain to me why this is so, and whether I am mistaken about the cause of the error?
Thank you very much!
When you call read() on a file handle, its current read position is advanced to the end of the file. This mirrors the behavior of the underlying file descriptors. You can reset to the start of a file via jsondata.seek(0, 0).
Related
I have a file named dict_file.json in current folder with empty dict content {}
I want to open it such that I can do both read and modify the content. That is I will read the json file as dict, modify that dict and write it back to json file.
I tried r and r+ below:
import json
f = open('dict_file.json', 'r') # same output for r+
json.loads(list_file.read())
This prints
{}
When I tried w+:
f = open('dict_file.json', 'r')
this first clears the file. Then,
json.loads(list_file.read())
gives error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
When I tried a+:
f = open('dict_file.json', 'a+')
json.loads(list_file.read())
gives error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "F:\ProgramFiles\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "F:\ProgramFiles\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Though, it does not clear the file.
So, I guess I should be using r+ for my usecase scenario. Also I tried reading writing both :
f = open('dict_file.json', 'r+')
a = json.loads(list_file.read())
a = {'key':'value'} # this involves complex logic instead of plain assignment
json.dump(a,f)
f.flush()
Now when I open the file, its contents are weird:
{}{'key':'value'}
whereas I want it to be:
{'key':'value'}
So I have few questions:
Q1. Why w+ and a+ gives error?
Q2. Why file contained {}{'key':'value'} instead of {'key':'value'}?
Q3. How to correctly do reading and writing (with or without with block)?
PS: I am reading file, then running some loop which computes new dict and write it to file. Then loop sleeps for some time and repeats the same. Thats why I felt flush() will be correct here. That is I open file once outside loop and only flush inside the loop. No need to open file in each iteration.
Why w+ and a+ gives error?
Because the file reader is pointed at the end of the file, where there's no json object to read
its contents are weird
Seems like you're not resetting the file to an actual valid JSON object throughout your tests, and so you've managed to clear the file, read nothing, maybe write some other object, clear it again, then append one or two objects to it, etc etc.
Keep it simple. Start over
filename = "data.json"
with open(filename) as f:
data = json.load(f)
data["foo"] = "bar"
with open(filename, "w") as f:
json.dump(data, f)
You have to think in terms of what the operations do to a file when they open it:
r and r+ open a file for reading. This means that the contents of the file is intact, and the file pointer is the beginning of the file. The entire file is visible to the reader. After reading, the file pointer will be at the end, meaning that you're effectively appending at that point.
w and w+ open a file for writing. That means that the contents of the file is truncated. There is nothing to be read in, since the original contents is destroyed.
a and a+ open the file for appending. The contents of the file are unchanged. However, the file pointer is at the end, so a reader will see no data and raise an error.
The correct way to do this is to open the file twice, and use with blocks to do it:
with open('dict_file.json', 'r') as f:
a = json.load(f)
# you don't need the file to be open here
a = {'key': 'value'}
with open('dict_file.json', 'w') as f:
json.dump(a, f)
You might be tempted to try to open the file in read-write mode (r+), read it, then rewind it, so the pointer returns to the beginning, and then overwrite. There are two reasons not to do this:
Not every stream is rewindable
If the contents of the file decreases in size, you will end up with trash at the end. You would need to truncate to the current pointer after writing, which is an unnecessary layer of complexity
I have a question concerning an issue I ran into while using the json lib in Python.
I'm tying to read a json file using the json.load(file) command using the following code:
import json
filename= '../Data/exampleFile.json'
histFile= open(filename, 'w+')
print(json.load(histFile))
The JSON file I am trying to read is valid according to some website I found: a screenshot of that validation, because I'm new and still lack the reputation...
The error message I'm getting is the following:
File ".\testLoad.py", line 5, in <module>
print(json.load(histFile))
File "C:\Users\...\Python\Python37\lib\json\__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Users\...\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\...\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\...\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Alright, so I believe it is not the file that is the issue, but the json.load(file) works for me in other cases.
Sadly I was not able to figure this error-message out on my own, so it would be amazing if someone with some more experience dealing with Python-JSON interaction could maybe help me out.
You opened the file for writing:
histFile= open(filename, 'w+')
# ^^^^
The w mode first truncates the file, so the file is empty (it doesn't matter here that the file can also be read from, the + sees to that but the file is truncated nonetheless). See the open() function documentation:
'w': open for writing, truncating the file first)
There is no JSON data in it to parse. This is why the exception tells you that parsing failed at the very start of the file:
Expecting value: line 1 column 1 (char 0)
There is no data in line one, column one.
If you wanted to open a file for both reading and writing without truncating it first, use 'r+' as the file mode.
I am trying to prints json in readable form. I already checked previous threads and tried out.
using
JSON.stringify(response)
gives error:
NameError: name 'JSON' is not defined
Using
response = json.loads(urllib.urlopen(url).read())
parsed = json.loads(response)
print json.dumps(parsed, indent=4, sort_keys=True)
gives error:
Traceback (most recent call last):
File "p6.py", line 15, in <module>
parsed = json.loads(response)
File "/usr/lib/python2.7/json/__init__.py", line 328, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
You get
NameError: name 'JSON' is not defined
because the first snippet is in JavaScript, not in Python.
As to the second snippet, you are calling json.loads() twice:
response = json.loads(urllib.urlopen(url).read()) # calling once
parsed = json.loads(response) # calling twice
Just call it once (and ensure that what you get from the HTTP server is actually JSON).
I'm new to python but would like to use urllib to download tweets, I'm following a tutorial instructions but get the same error every time, I print:
import urllib
import json
response = urllib.urlopen("https://twitter.com/search?q=Microsoft&src=tyah")
print json.load(response)
But everytime I get the error:
Traceback (most recent call last):
File "C:\Python27\print.py", line 4, in <module>
print json.load(response)
File "C:\Python27\Lib\json\__init__.py", line 278, in load
**kw)
File "C:\Python27\Lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\Lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\Lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
As noted in comments, the answer is: nothing is wrong with your code, per se.
The problem is that when json.load looks at response, it does not find JSON in there - it is finding HTML.
You need to pass a file-like object containing JSON into the json.load function, or it will raise the exception you see here.
To get JSON from Twitter, you need to call a URL that gives a JSON response. I can tell you now, that none of the Web interface URLs do this directly. You should use the Twitter API.
However, purely for sake of demonstration, if you deconstruct the page at the URL you are calling now, you will find that to load the tweet data, the page makes the following request:
https://twitter.com/i/search/timeline?q=Microsoft&src=tyah&composed_count=0&include_available_features=1&include_entities=1
And this URL does return JSON in response, which would work just fine with your current code.
Of course, I'm pretty sure doing so violates some sort of Twitter TOS, so if you do this there are all sorts of potential negative repercussions to consider. Plus it's just not good sportsmanship. :)
I piped the output of my Python script which accesses live twitter tweets to a file output.txt using:
$python scriptTweet.py > output.txt
Originally, the output returned by the script was a dictionary which got written to a text file.
Now i want to use the output.txt file to access tweets stored in it. But when i use this code to parse the text in output.txt into a python dictionary using json.loads():
tweetfile = open("output.txt")
pyresponse = json.loads('tweetfile.read()')
print type(pyresponse)
This error pops up:
pyresponse = json.loads('tweetfile.read()')
File "C:\Python27\lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
How should i convert the contents of file output.txt again into a dictionary?
'tweetfile.read()' is a string as you see it. You want to call this function:
with open("output.txt") as tweetfile:
pyresponse = json.loads(tweetfile.read())
or read it directly using json.load and let json read on the tweetfile itself:
with open("output.txt") as tweetfile:
pyresponse = json.load(tweetfile)