I have this function to make dictionaries into json objects and writes them to a json file:
def somethin(a1,a2,a3):
someDict = {"a": a1,
"b": a2,
"c": a3
}
json_obj = json.dumps(someDict)
file = open('somefile.json', 'a',encoding="utf-8")
file.write(json_obj)
the first written element to the file doesnt present any problem, but after adding the second json object to the file I get: SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data with the json file I made:
{
"a": 1,
"b":2,
"c": 3
}
{
"a1": 1,
"b1":2,
"c1": 3
}
How can I modify the code for the json output file to be correct?
There are a few issues here:
file = open('somefile.json', 'a',encoding="utf-8")
You're implicitly assuming that the file will be closed here. That is true on CPython, but not on all Python implementations. A better approach would be:
with open('somefile.json', 'a',encoding="utf-8") as file:
file.write(json_obj)
Because that uses a context manager to explicitly close the file.
Second, you can avoid creating an extra string by writing directly to the file:
with open('somefile.json', 'a',encoding="utf-8") as file:
json.dump(someDict, file)
Third, having multiple JSON objects in a file is not valid JSON. There are a few approaches you could take here. One is to wrap the JSON objects in a list:
[
{
"a": 1,
"b":2,
"c": 3
},
{
"a1": 1,
"b1":2,
"c1": 3
}
]
So, start the file with an open bracket, and write a comma after every JSON object, except the last one, then end the file with a close bracket.
Second approach would be to newline-separate your JSON objects, like this:
{"a": 1,"b":2,"c": 3}
{"a1": 1, "b1":2,"c1": 3}
Each line is a JSON object. You'd read this like so:
with open("filename", "rt") as file:
for line in file:
obj = json.loads(line)
# do something with obj
# ...
The advantage of this approach would be that you can now load each individual JSON object in memory, without having to load the entire file in at once. The disadvantage is that you're no longer writing valid JSON, so you can't use tools like jq on the output. (If you want the best of both worlds, you can use a package like ijson, but that's more complex.)
Related
I want to read a dictionary from a text file. This dictionary seems like {'key': [1, ord('#')]}. I read about eval() and literal_eval(), but none of those two will work due to ord().
I also tried json.loads and json.dumps, but no positive results.
Which other way could I use to do it?
So Assuming you read the text file in with open as a string and not with json.loads you could do some simple regex searching for what is between the parenthesis of ord e.g ord('#') -> #
This is a minimal solution that reads everything from the file as a single string then finds all instances of ord and places the integer representation in an output list called ord_. For testing this example myfile.txt was a text file with the following in it
{"key": [1, "ord('#')"],
"key2": [1, "ord('K')"]}
import json
import re
with open(r"myfile.txt") as f:
json_ = "".join([line.rstrip("\n") for line in f])
rgx = re.compile(r"ord\(([^\)]+)\)")
rgd = rgx.findall(json_)
ord_ = [ord(str_.replace(r"'", "")) for str_ in rgd]
json.dump() and json.load() will not work because ord() is not JSON Serializable (meaning that the function cannot be a JSON object.
Yes, eval is really bad practice, I would never recommend it to anyone for any use.
The best way I can think of to solve this is to use conditions and an extra list.
# data.json = {'key': [1, ['ord', '#']]} # first one is function name, second is arg
with open("data.json") as f:
data = json.load(f)
# data['key'][1][0] is "ord"
if data['key'][1][0] == "ord":
res = ord(data['key'][1][1])
I have a JSON file that is getting continuously appended with new data. Each time it gets updated I need it to be "well-formed". The problem is that my JSON looks like this (each item is dumped serially):
{"one": 1},
{"two": 2}
I need the data to be properly formed so enclosing in square brackets could work, or an outer curly bracket. But I'm not quite sure how to do that.
[
{"one": 1},
{"two": 2}
]
Here is the code performing the JSON writing:
def printJSONFile(data):
json_dump = json.dumps(data, default=serialize)
try:
jf = open(fullpath, "a+")
jf.write(json_dump + ",\n")
jf.close()
except IOError:
print "ERROR: Unable to open/write to {}".format(fullpath)
return
Currently I am exporting a database from firebase into a JSON but it is coming out in as an array.
[{"ConnectionTime": 19.23262298107147, "objectId": "01331oxpVT", "FirmwareRevision": "201504270003 Img-B", "DeviceID": "EDF02C74-6518-489E-8751-25C58F8C830D", "PeripheralType": 4, "updatedAt": "2015-10-09T04:01:39.569Z", "Model": "Bean", "HardwareRevision": "E", "Serial": "Serial Number", "createdAt": "2015-10-09T04:01:39.569Z", "Manufacturer": "Punch Through Design"}, {"ConnectionTime": 0.3193170428276062, "objectId": "018Mv1g6I8", "DeviceID": "42635033-DF3A-4109-A633-C3AB829BE114", "PeripheralType": 2, "updatedAt": "2015-12-08T04:20:41.950Z", "createdAt": "2015-12-08T04:20:41.950Z"}]
And then I get this error - Start of array encountered without start of object.'}]
How can I change this to not be an Array and just a list of data. I also need a line break between each set of data but Im assuming once I get the data out of the array the code I currently have will do that. My code is below. Thanks for the help!
firebase = firebase.FirebaseApplication('https://dataworks-356fa.firebaseio.com/')
result = firebase.get('/connection_info_parse', None)
# id_keys = map(str, result.keys()) #filter out ID names
with open("firetobqrestore1.json", "w") as outfile:
# for id in id_keys:
json.dump(result, outfile, indent=None)
outfile.write("\n")
It sounds like something in your workflow wants newline delimited JSON, although you haven't made it explicitly clear what is giving you this error.
With that caveat, I think this is what you are looking for:
import json
with open("firetobqrestore1.json", "w") as outfile:
for line in result:
json.dump(line, outfile, indent=None)
outfile.write("\n")
This will write individual json objects to each line.
This also assumes that result is an actual python object rather than a JSON string. If it's a string you will need to parse it first with something like:
result = json.loads(result)
If the elements of the list are not parsed ( they are strings), then loop through the list and convert each element to a json using json.loads(). Then, you can use json.dumps()
In case the elements of the list are already parsed,then just loop through the list and use json.dumps().
I have a bunch of JSON objects that I need to update in order to use the CLI for AWS.
Here is an example of the JSON format. I will need to update lbtest, lbtest-cookie-pol and 80 with different values.
{
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod":80
}
In some cases, there will be multiple values here for each Load Balancer Name.
The output will need to look like this:
{
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod":80
}
{
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol2",
"CookieExpirationPeriod":8080.
}
Suppose I had a CSV file with all these entries, what kind of python script can I write to loop through these and print out my JSON output? The part where I am having issues is the printing of the nested JSON object. print doesn't seem to like multiple lines or the curly braces that I have. Newbie here so I would appreciate any kind of solution.
you can use json.dumps method and it's options mentioned in documentations:
for example with using indent option you get this on python 2.7 :
>>> dictionary = {
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod":80 } #a dictionary object made from csv
>>> print dictionary
{'PolicyName': 'lbtest-cookie-pol', 'CookieExpirationPeriod': 80, 'LoadBalancerName': 'lbtest'}
>>> import json
>>> jobj = json.dumps(dictionary,indent=4, separators=(',', ': '))
>>> print jobj
{
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod": 80,
"LoadBalancerName": "lbtest"
}
>>> f = open(r'jtest.txt','w') #save our json object to file
>>> json.dump(dictionary,fp,indent =4 , seperators = (',',': '))
>>> f.close()
>>> f = open(r'jtest.txt!','r') #read our object from file
>>> test = json.load(f)
>>> test
{u'PolicyName': u'lbtest-cookie-pol', u'CookieExpirationPeriod': 80, u'LoadBalancerName': u'lbtest'}
>>> dict(test)
{u'PolicyName': u'lbtest-cookie-pol', u'CookieExpirationPeriod': 80, u'LoadBalancerName': u'lbtest'}
here is how our file looks like:
jtest.txt file
I have JSON data as an array of dictionaries which comes as the request payload.
[
{ "Field1": 1, "Feld2": "5" },
{ "Field1": 3, "Feld2": "6" }
]
I tried ijson.items(f, '') which yields the entire JSON object as one single item. Is there a way I can iterate the items inside the array one by one using ijson?
Here is the sample code I tried which is yielding the JSON as one single object.
f = open("metadatam1.json")
objs = ijson.items(f, '')
for o in objs:
print str(o) + "\n"
[{'Feld2': u'5', 'Field1': 1}, {'Feld2': u'6', 'Field1': 3}]
I'm not very familiar with ijson, but reading some of its code it looks like calling items with a prefix of "item" should work to get the items of the array, rather than the top-level object:
for item in ijson.items(f, "item"):
# do stuff with the item dict