Hi I have just started experimenting with python and tornado along with mongodb(I am a newbie). I have written a simple get function to get all the values from my mongodb and return it in JSON format. The problem is when I try to write the output as a JSON string I get a trailing comma(,) after the last record from the collection.
class TypeList(APIHandler):
#gen.coroutine
def get(self):
cursor = db.vtype.find()
self.write("{"'"success"'": 1, "'"data"'":[")
while (yield cursor.fetch_next):
document = cursor.next_object()
self.write(format(JSONEncoder().encode(document)))
self.write(",")
self.write("]}")
class JSONEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o,ObjectId):
return str(o)
return json.JSONEncoder.default(self, o)
And my output is like
{"success": 1, "data":[{"_id": "55a5e988545779f35d3ecdf4", "name": "fgkd", "city": "fghj"},{"_id": 12345.0, "name": "adfs", "city": "asd"},]}
Can anyone tell me how can I get rid of that trailing comma(,) after my last record, because of that comma I am getting an error malformed JSON string
I have tried using json dumps
#gen.coroutine
def get(self):
cursor = db.vtype.find({"brand": "Tata"})
while (yield cursor.fetch_next):
document = cursor.next_object()
self.write(json.dumps(document,default=json_util.default))
got the output as
{"Reg": "11ts", "_id": {"$oid": "55a5e988545779f35d3ecdf4"}, "Name": "Alex"}{"Reg": "12ts", "_id": {"$oid": "55a5eac6545779f35d3ecdf5"}, "Name": "asdf"}
When using dumps[{ "data": document }]
I am getting the output as
[{"data": {"Name": "asdf", "Reg": "asdfs", "_id": {"$oid": "55a5e988545779f35d3ecdf4"}}}]
[{"data": {"Name": "qwer", "Reg": "asdff", "_id": {"$oid": "55a5eac6545779f35d3ecdf5"}}}]
but I want the output like this
{"data": [{"Name": "asdf", "Reg": "asdfs", "_id": {"$oid": "55a5e988545779f35d3ecdf4"}},{"Name": "qwer", "Reg": "asdff", "_id": {"$oid": "55a5eac6545779f35d3ecdf5"}}]}
If I am doing something wrong please tell me I dont know how to do it.
There is no reason you should be building up JSON documents via text concatenation.
Python has a perfectly good json module in the standard library which you should be using. Build up your document as a Python list of dicts, then use json.dumps() to convert the whole thing to valid JSON.
So your problem is with MongoDB ObjectId? Then maybe you should have been using bson.json_util. It's probably already installed as part of your MongoDB driver dependecies ( which all use pymongo ), but if not then install it.
import bson
import bson.json_util
from bson.json_util import dumps
from bson import ObjectId
dumps({ "a": ObjectId() })
'{"a": {"$oid": "55a782261d41c80b0432b811"}}'
Or:
dumps([{ "a": ObjectId(), "b": 1 },{ "a": ObjectId(), "b": 2 }])
'[{"a": {"$oid": "55a79543268e150463d51799"}, "b": 1}, {"a": {"$oid": "55a79543268e150463d5179a"}, "b": 2}]'
And it works just like "dumps" except all the BSON type handling is built it.
Again, no need to re-invent the wheel here and "roll your own", because people already use this.
Your implementation of the JSONEncoder works well. Just use it the way it was intended to be used:
>>> JSONEncoder().encode({'data': [ObjectId(), ObjectId()]})
'{"data": ["<objId>", "<objId>"]}'
The encoder will take care of serializing dicts, objects, lists, tuples, strings (including unicode), ints, longs, floats, booleans and None. Your implementation makes it aware of ObjectIds as well. Perfect!
Just lose the string concatenation and use encode.
Related
I am helping my teacher by creating a game where the k-2 kids can learn their passwords, I added this json file to create a save for the teacher so he doesn't have to re-add all the computers names and passwords... But when I run my code, the json file gets wiped, and I lose all of my code... Luckily I had backups but I cant have it erase for my teacher.
Python Code:
import json
with open("accounts.json", "w") as f:
accountData = json.dumps(f)
type(accountData)
JSON:
{ // not real names and passwords for security
"accounts": [
{
"id": "1",
"uname": "scarlett",
"pword": "k,",
"points": "0"
},
{
"id": "2",
"uname": "santiago",
"pword": "k,",
"points": "0"
},
{
"id": "3",
"uname": "harper",
"pword": "k,",
"points": "0"
}
]
}
It's a wrong use of the json.dump method.
Here is part of the help to json.dump in python at version 3.7.6.
Help on function dump in module json:
dump(obj, fp, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw)
Serialize ``obj`` as a JSON formatted stream to ``fp`` (a
``.write()``-supporting file-like object).
The first parameter should be the dict object which contains the names and passwords, and you missed this parameter.
And the parameter f should be in the second place.
I'm trying to write an efficient stringification routine for logging dicts, but want to redact certain values based on key names. I see that JSONDecoder provides the object_pairs_hook which provides key and value, but I don't see a corresponding hook for JSONEncoder - just 'default' which only provides value. In my case, the values are just other strings so can't base the processing on that alone. Is there something I missed?
For example, if I have a dict with:
{
"username": "Todd",
"role": "Editor",
"privateKey": "1234ad1234e434134"
}
I would want to log:
'{"username":"Todd","role":"Editor","privateKey":"**redacted**"}'
Any good tools in python to do this? Or should I just recursively iterate the (possibly nested) dict directly?
You can "reload" it using the object hook then dump it again.
def redact(o):
if 'privateKey' in o:
o['privateKey'] = '***redacted***'
return o
>>> d
{'username': 'Todd', 'role': 'Editor', 'privateKey': '1234ad1234e434134', 'foo': ['bar', {'privateKey': 'baz'}]}
>>> json.dumps(json.loads(json.dumps(d), object_hook=redact))
'{"username": "Todd", "role": "Editor", "privateKey": "***redacted***", "foo": ["bar", {"privateKey": "***redacted***"}]}'
JSON library has two functions viz. dumps() and loads(). To convert a json to a string use dumps() and vice-versa use loads().
import json
your_dict = {
"username": "Todd",
"role": "Editor",
"privateKey": "1234ad1234e434134"
}
string_of_your_json = json.dumps(your_dict)
I'm revising my previous question. I have a collection named FileCollection with the following document:
{
"_id": {
"$oid": "5e791a53185fbb070378660a"
},
"selectedfiles": [{
"inputfile": "https://localhost/_HAC-154_1584994899979.jpg",
"Selectedby: "Joe"
}]}
I need to read the value of selectedfiles.inputfile as a string variable. I'm trying to do this in Python using this code:
from pymongo import MongoClient
mydb = MongoClient(mongodbConnection)
myCollection=mydb.FileCollection
myValue=myCollection.selectedfile[0].inputfile.value
print(myValue)
client.close
the output is a JSON without having the actual value of inputfile. Please help.
Thanks
Isn't it just because you're missing an s?
You had:
myValue=myCollection.selectedfile[0].inputfile.value
instead of:
myValue=myCollection.selectedfiles[0].inputfile.value
I'm trying to encode a somewhat large JSON in Python (v2.7) and I'm having trouble putting in my variables!
As the JSON is multi-line and to keep my code neat I've decided to use the triple double quotation mark to make it look as follows:
my_json = """{
"settings": {
"serial": "1",
"status": "2",
"ersion": "3"
},
"config": {
"active": "4",
"version": "5"
}
}"""
To encode this, and output it works well for me, but I'm not sure how I can change the numbers I have there and replace them by variable strings. I've tried:
"settings": {
"serial": 'json_serial',
but to no avail. Any help would be appreciated!
Why don't you make it a dictionary and set variables then use the json library to make it into json
import json
json_serial = "123"
my_json = {
'settings': {
"serial": json_serial,
"status": '2',
"ersion": '3',
},
'config': {
'active': '4',
'version': '5'
}
}
print(json.dumps(my_json))
If you absolutely insist on generating JSON with string concatenation -- and, to be clear, you absolutely shouldn't -- the only way to be entirely certain that your output is valid JSON is to generate the substrings being substituted with a JSON generator. That is:
'''"settings" : {
"serial" : {serial},
"version" : {version}
}'''.format(serial=json.dumps("5"), version=json.dumps(1))
But don't. Really, really don't. The answer by #davidejones is the Right Thing for this scenario.
can someone please explain to me whats happening here and whats the difference between these 2 functions. I am essentially trying to return a django object to my angular controller and so the response has to be in json.
obj = Cuisine.objects.all()
print obj
[<Cuisine: Chinese>, <Cuisine: Indian>, <Cuisine: Japanese>]
With django serializer
from django.core import serializers
obj_json = serializers.serialize('json', obj)
print obj_json
[{"pk": 1, "model": "swings.cuisine", "fields": {"name": "Chinese"}}, {"pk": 2, "model": "swings.cuisine", "fields": {"name": "Indian"}}, {"pk": 3, "model": "swings.cuisine", "fields": {"name": "Japanese"}}]
With python json, why is it unable to convert my object to json here?
import json
obj_json = json.dumps(obj)
TypeError: TypeErro...izable',)
Now django-angular has a JSONResponseMixin which automatically converts the returned object / list to json but it keeps giving me a type error. I have a feeling its connected to the same reason why json.dumps does work. I've been reading around plenty trying all sorts of work arounds to no luck. I have already tried converting my object to a list first via list(obj) but that does nothing.
Any ideas?