When I'm trying to parse a JSON dump, I get this attribute error
Traceback (most recent call last):
File "Security_Header_Collector.py", line 120, in <module>
process(sys.argv[-1])
File "Security_Header_Collector.py", line 67, in process
server_details = json.load(header_final)
File "/usr/lib/python2.7/json/__init__.py", line 274, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
Script:
finalJson[App[0]] = headerJson
header_final=json.dumps(finalJson,indent=4)
#print header_final
#json_data=open(header_final)
server_details = json.load(header_final)
with open("Out.txt",'wb') as f :
for appid, headers in server_details.iteritems():
htypes = [h for h in headers if h in (
'content-security-policy', 'x-frame-options',
'strict-transport-security', 'x-content-type-options',
'x-xss-protection')]
headers='{},{}'.format(appid, ','.join(htypes))
f.write(headers+'\n')
f.close()
json.dumps returns a JSON formatted string, but json.load expects to get file-like objects, not strings.
Solution: use json.loads instead of json.load in your code
Your code
header_final=json.dumps(finalJson,indent=4)
will give you string,
you have to use json.loads to convert string to json.
json.load - is used for files / objects
json.loads - is used for the strings or array elements.
You may also think about creating the whole JSON in the form of HEREDOC formate at once and latter apply escaping on it - this way it become easier to validate JSON format.
Related
In Python, what is the difference between json.load() and json.loads()?
I guess that the load() function must be used with a file object (I need thus to use a context manager) while the loads() function take the path to the file as a string. It is a bit confusing.
Does the letter "s" in json.loads() stand for string?
Thanks a lot for your answers!
Yes, s stands for string. The json.loads function does not take the file path, but the file contents as a string. Look at the documentation.
Just going to add a simple example to what everyone has explained,
json.load()
json.load can deserialize a file itself i.e. it accepts a file object, for example,
# open a json file for reading and print content using json.load
with open("/xyz/json_data.json", "r") as content:
print(json.load(content))
will output,
{u'event': {u'id': u'5206c7e2-da67-42da-9341-6ea403c632c7', u'name': u'Sufiyan Ghori'}}
If I use json.loads to open a file instead,
# you cannot use json.loads on file object
with open("json_data.json", "r") as content:
print(json.loads(content))
I would get this error:
TypeError: expected string or buffer
json.loads()
json.loads() deserialize string.
So in order to use json.loads I will have to pass the content of the file using read() function, for example,
using content.read() with json.loads() return content of the file,
with open("json_data.json", "r") as content:
print(json.loads(content.read()))
Output,
{u'event': {u'id': u'5206c7e2-da67-42da-9341-6ea403c632c7', u'name': u'Sufiyan Ghori'}}
That's because type of content.read() is string, i.e. <type 'str'>
If I use json.load() with content.read(), I will get error,
with open("json_data.json", "r") as content:
print(json.load(content.read()))
Gives,
AttributeError: 'str' object has no attribute 'read'
So, now you know json.load deserialze file and json.loads deserialize a string.
Another example,
sys.stdin return file object, so if i do print(json.load(sys.stdin)), I will get actual json data,
cat json_data.json | ./test.py
{u'event': {u'id': u'5206c7e2-da67-42da-9341-6ea403c632c7', u'name': u'Sufiyan Ghori'}}
If I want to use json.loads(), I would do print(json.loads(sys.stdin.read())) instead.
Documentation is quite clear: https://docs.python.org/2/library/json.html
json.load(fp[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]])
Deserialize fp (a .read()-supporting file-like object containing a
JSON document) to a Python object using this conversion table.
json.loads(s[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]])
Deserialize s (a str or unicode instance containing a JSON document)
to a Python object using this conversion table.
So load is for a file, loads for a string
QUICK ANSWER (very simplified!)
json.load() takes a FILE
json.load() expects a file (file object) - e.g. a file you opened before given by filepath like 'files/example.json'.
json.loads() takes a STRING
json.loads() expects a (valid) JSON string - i.e. {"foo": "bar"}
EXAMPLES
Assuming you have a file example.json with this content: { "key_1": 1, "key_2": "foo", "Key_3": null }
>>> import json
>>> file = open("example.json")
>>> type(file)
<class '_io.TextIOWrapper'>
>>> file
<_io.TextIOWrapper name='example.json' mode='r' encoding='UTF-8'>
>>> json.load(file)
{'key_1': 1, 'key_2': 'foo', 'Key_3': None}
>>> json.loads(file)
Traceback (most recent call last):
File "/usr/local/python/Versions/3.7/lib/python3.7/json/__init__.py", line 341, in loads
TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper
>>> string = '{"foo": "bar"}'
>>> type(string)
<class 'str'>
>>> string
'{"foo": "bar"}'
>>> json.loads(string)
{'foo': 'bar'}
>>> json.load(string)
Traceback (most recent call last):
File "/usr/local/python/Versions/3.7/lib/python3.7/json/__init__.py", line 293, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
In python3.7.7, the definition of json.load is as below according to cpython source code:
def load(fp, *, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
return loads(fp.read(),
cls=cls, object_hook=object_hook,
parse_float=parse_float, parse_int=parse_int,
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
json.load actually calls json.loads and use fp.read() as the first argument.
So if your code is:
with open (file) as fp:
s = fp.read()
json.loads(s)
It's the same to do this:
with open (file) as fp:
json.load(fp)
But if you need to specify the bytes reading from the file as like fp.read(10) or the string/bytes you want to deserialize is not from file, you should use json.loads()
As for json.loads(), it not only deserialize string but also bytes. If s is bytes or bytearray, it will be decoded to string first. You can also find it in the source code.
def loads(s, *, encoding=None, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
"""Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance
containing a JSON document) to a Python object.
...
"""
if isinstance(s, str):
if s.startswith('\ufeff'):
raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
s, 0)
else:
if not isinstance(s, (bytes, bytearray)):
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
f'not {s.__class__.__name__}')
s = s.decode(detect_encoding(s), 'surrogatepass')
I am trying to read a file called differences.txt and put it into a one line variable.
Here is differences.txt:
192.168.0.***
192.168.0.***
and my code:
with open ("/home/pi/Documents/difference.txt") as myfile:
difip=myfile.readlines()
print (difip)
and my error:
Traceback (most recent call last):
File "/home/pi/Desktop/clean.py", line 95, in <module>
body = "Different IP's:" + difip
TypeError: Can't convert 'list' object to str implicitly
Any help will be awesome! Thank you!
myfile.readlines()
returns a list of lines of the file
(remember, this includes \n for new lines). In your case it's returning
["192.168.0.***\n", "192.168.0.***\n"]
Optn 1) You should be using the strip() function instead
myfile = open('/home/pi/Documents/difference.txt', 'r')
text = myfile.read().strip() #pass ("\n") as argument to strip() to remove the newlines.
Optn 2) Optionally you can use your same code but modify the last line as follows:
with open ("/home/pi/Documents/difference.txt") as myfile:
difip=myfile.readlines()
print (difip[0] + difip[1])
This error message
TypeError: Can't convert 'list' object to str implicitly
is telling you that you are trying to print the list as a string. What this last change makes is print the strings that are stored in the first two lines of the text file.
This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed 25 days ago.
In Python I'm getting an error:
Exception: (<type 'exceptions.AttributeError'>,
AttributeError("'str' object has no attribute 'read'",), <traceback object at 0x1543ab8>)
Given python code:
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.load(jsonStr)['data']['children']
What does this error mean and what did I do to cause it?
The problem is that for json.load you should pass a file like object with a read function defined. So either you use json.load(response) or json.loads(response.read()).
Ok, this is an old thread but.
I had a same issue, my problem was I used json.load instead of json.loads
This way, json has no problem with loading any kind of dictionary.
Official documentation
json.load - Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.
json.loads - Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object using this conversion table.
You need to open the file first. This doesn't work:
json_file = json.load('test.json')
But this works:
f = open('test.json')
json_file = json.load(f)
If you get a python error like this:
AttributeError: 'str' object has no attribute 'some_method'
You probably poisoned your object accidentally by overwriting your object with a string.
How to reproduce this error in python with a few lines of code:
#!/usr/bin/env python
import json
def foobar(json):
msg = json.loads(json)
foobar('{"batman": "yes"}')
Run it, which prints:
AttributeError: 'str' object has no attribute 'loads'
But change the name of the variablename, and it works fine:
#!/usr/bin/env python
import json
def foobar(jsonstring):
msg = json.loads(jsonstring)
foobar('{"batman": "yes"}')
This error is caused when you tried to run a method within a string. String has a few methods, but not the one you are invoking. So stop trying to invoke a method which String does not define and start looking for where you poisoned your object.
AttributeError("'str' object has no attribute 'read'",)
This means exactly what it says: something tried to find a .read attribute on the object that you gave it, and you gave it an object of type str (i.e., you gave it a string).
The error occurred here:
json.load(jsonStr)['data']['children']
Well, you aren't looking for read anywhere, so it must happen in the json.load function that you called (as indicated by the full traceback). That is because json.load is trying to .read the thing that you gave it, but you gave it jsonStr, which currently names a string (which you created by calling .read on the response).
Solution: don't call .read yourself; the function will do this, and is expecting you to give it the response directly so that it can do so.
You could also have figured this out by reading the built-in Python documentation for the function (try help(json.load), or for the entire module (try help(json)), or by checking the documentation for those functions on http://docs.python.org .
Instead of json.load() use json.loads() and it would work:
ex:
import json
from json import dumps
strinjJson = '{"event_type": "affected_element_added"}'
data = json.loads(strinjJson)
print(data)
So, don't use json.load(data.read()) use json.loads(data.read()):
def findMailOfDev(fileName):
file=open(fileName,'r')
data=file.read();
data=json.loads(data)
return data['mail']
use json.loads() function , put the s after that ... just a mistake btw i just realized after i searched error
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.loads(jsonStr)['data']['children']
try this
Open the file as a text file first
json_data = open("data.json", "r")
Now load it to dict
dict_data = json.load(json_data)
If you need to convert string to json. Then use loads() method instead of load(). load() function uses to load data from a file so used loads() to convert string to json object.
j_obj = json.loads('["label" : "data"]')
In Python, I usually do simple JSON with this sort of template:
url = "url"
file = urllib2.urlopen(url)
json = file.read()
parsed = json.loads(json)
and then get at the variables with calls like:
parsed[obj name][value name]
But, this works with JSON that's formatted roughly like:
{'object':{'index':'value', 'index':'value'}}
The JSON I just encountered is formatted like:
{'index':'value', 'index':'value'},{'index':'value', 'index':'value'}
so there are no names for me to reference the different blocks. Of course the blocks give different info, but have the same "keys" -- much like XML is usually formatted. Using my method above, how would I parse through this JSON?
The following is not a valid JSON.
{'index':'value', 'index':'value'},{'index':'value', 'index':'value'}
Where as
[{'index':'value', 'index':'value'},{'index':'value', 'index':'value'}] is a valid JSON.
and python trackback shows that
import json
string = "{'index':'value', 'index':'value'},{'index':'value', 'index':'value'}"
parsed = json.loads(string)
print parsed
Traceback (most recent call last):
File "/Users/tron/Desktop/test3.py", line 3, in <module>
parsed_json = json.loads(json_string)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 369, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 27 - line 1 column 54 (char 26 - 53)
[Finished in 0.0s with exit code 1]
where is if you do
json_string = '[{"a":"value", "b":"value"},{"a":"value", "b":"value"}]'
everything works fine.
If that is the case, you can refer to it as an array of Jsons. where json_string[0] is the first JSON string. json_string[1] is the second and so on.
Otherwise if you think this is going to be an issue that you "just have to deal with". Here is one option:
Think of the ways JSON can be malformed and write a simple class to account for them. In the case above, here is a hacky way you can deal with it.
import json
json_string = '{"a":"value", "b":"value"},{"a":"value", "b":"value"}'
def parseJson(string):
parsed_json = None
try:
parsed_json = json.loads(string)
print parsed_json
except ValueError, e:
print string, "didnt parse"
if "Extra data" in str(e.args):
newString = "["+string+"]"
print newString
return parseJson(newString)
You could add more if/else to deal with various things you run into. I have to admit, this is very hacky and I don't think you can ever account for every possible mutation.
Good luck
The result must be list of dict:
[{'index1':'value1', 'index2':'value2'},{'index1':'value1', 'index2':'value2'}]
thus you can reference it using numbers: item[1]['index1']
I want to read a BSON format Mongo dump in Python and process the data. I am using the Python bson package (which I'd prefer to use rather than have a pymongo dependency), but it doesn't explain how to read from a file.
This is what I'm trying:
bson_file = open('statistics.bson', 'rb')
b = bson.loads(bson_file)
print b[0]
But I get:
Traceback (most recent call last):
File "test.py", line 11, in <module>
b = bson.loads(bson_file)
File "/Library/Python/2.7/site-packages/bson/__init__.py", line 75, in loads
return decode_document(data, 0)[1]
File "/Library/Python/2.7/site-packages/bson/codec.py", line 235, in decode_document
length = struct.unpack("<i", data[base:base + 4])[0]
TypeError: 'file' object has no attribute '__getitem__'
What am I doing wrong?
I found this worked for me with a mongodb 2.4 BSON file and PyMongo's 'bson' module:
import bson
with open('survey.bson','rb') as f:
data = bson.decode_all(f.read())
That returned a list of dictionaries matching the JSON documents stored in that mongo collection.
The f.read() data looks like this in a BSON:
>>> rawdata[:100]
'\x04\x01\x00\x00\x12_id\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02_type\x00\x07\x00\x00\x00simple\x00\tchanged\x00\xd0\xbb\xb2\x9eI\x01\x00\x00\tcreated\x00\xd0L\xdcfI\x01\x00\x00\x02description\x00\x14\x00\x00\x00testing the bu'
The documentation states :
> help(bson.loads)
Given a BSON string, outputs a dict.
You need to pass a string. For example:
> b = bson.loads(bson_file.read())
loads expects a string (that's what the 's' stands for), not a file. Try reading from the file, and passing the result to loads.