I have a python class that should return a json, it looks something like this:
class ScanThis():
def__init__(self, test):
data={}
if test>5:
data["amount"] = test
self.json_data = json.dumps(data)
else:
data["fail"] = test
self.json_data = json.dumps(data)
def __str__(self):
return self.json_data
and I'm trying to call it like so:
output= json.loads(ScanThis(8))
print(output["command"])
But I get this error:
TypeError: the JSON object must be str, bytes or bytearray, not 'ScanThis'
I believe my earlier clas returns an object of type ScanThis() rather than a JSon like I wanted. I just wanted to now how I'd fix this
Thank you
PS: I apologise if this code is rough or invalid, it's not the actual code, just something similar I made up
Update: Again, this isn't the real code, it's just a small basic fragment of the actual code. There's a good reason I'm using a class, and a json is used cause data transfer over the internet is involved
Use str(..)
You can't call json.loads on a ScanThis object directly. So that won't work. Like the error says, json.loads expects a str, bytes or bytearray object.
You can however use str(..) to invoke the __str__(self) method, and thus obtain the JSON data:
output = json.loads(str(ScanThis(8)))
# ^ get the __str__ result
Use another method
That being said, it is usually a better idea to define a method, for instance to_json to obtain the JSON data, since now you have made str(..) return a JSON object. So perhaps a more elegant way to do this is the following:
class ScanThis():
def__init__(self, test):
data={}
if test>5:
data["amount"] = test
self.json_data = json.dumps(data)
else:
data["fail"] = test
self.json_data = json.dumps(data)
def to_json(self):
return self.json_data
and call it with:
output = json.loads(ScanThis(8).to_json())
Now you can still use __str__ for another purpose. Furthermore by using to_json you make it explicit that the result will be a JSON string. Using str for JSON conversion is of course not forbidden, but str(..) as a name, does not provide much guarantees about the format of the result whereas to_json (or another similar name) strongly hints that you will obtain JSON data.
I don't think you are wanting to use a class there at all.
Instead, try using a function that returns a string. For example:
def scan_this(test):
data={}
if test>5:
data["amount"] = test
json_data = json.dumps(data)
else:
data["fail"] = test
json_data = json.dumps(data)
return json_data
output = json.loads(scan_this(8))
However!! Now you are just doing extra work for nothing? Why do you need to serialize a python dictionary as a json formatted string, and then load it back into a python dictionary straight away? While you are working with data in python, it's best to keep it as native data types, and only using the json module to either load from a string/file you already have, or serialize to a string/file for storage or transfer (eg sending over the internet).
Related
I'm creating a test suite for my module that uses the requests library quite a bit. However, I'm trying to mock several different return values for a specific request, and I'm having trouble doing so. Here is my code snippet that doesn't work:
class MyTests(unittest.TestCase):
#patch('mypackage.mymodule.requests.post')
def test_change_nested_dict_function(self, mock_post):
mock_post.return_value.status_code = 200
mock_post.return_value.json = nested_dictionary
modified_dict = mymodule.change_nested_dict()
self.assertEqual(modified_dict['key1']['key2'][0]['key3'], 'replaced_value')
The function I am attempting to mock:
import requests
def change_nested_dict():
uri = 'http://this_is_the_endpoint/I/am/hitting'
payload = {'param1': 'foo', 'param2': 'bar'}
r = requests.post(uri, params=payload)
# This function checks to make sure the response is giving the
# correct status code, hence why I need to mock the status code above
raise_error_if_bad_status_code(r)
dict_to_be_changed = r.json()
def _internal_fxn_to_change_nested_value(dict):
''' This goes through the dict and finds the correct key to change the value.
This is the actual function I am trying to test above'''
return changed_dict
modified_dict = _internal_fxn_to_change_nested_value(dict_to_be_changed)
return modified_dict
I know a simple way of doing this would be to not have a nested function, but I am only showing you part of the entire function's code. Trust me, the nested function is necessary and I really do not want to change that part of it.
My issue is, I don't understand how to mock requests.post and then set the return value for both the status code and the internal json decoder. I also can't seem to find a way around this issue since I can't seem to patch the internal function either, which also would solve this problem. Does anyone have any suggestions/ideas? Thanks a bunch.
I bumped here and although I agree that possibly using special purpose libraries is a better solution, I ended up doing the following
from mock import patch, Mock
#patch('requests.post')
def test_something_awesome(mocked_post):
mocked_post.return_value = Mock(status_code=201, json=lambda : {"data": {"id": "test"}})
This worked for me for both getting the status_code and the json() at the receiver end while doing the unit-test.
Wrote it here thinking that someone may find it helpful.
When you mock a class each child method is set up as a new MagicMock that in turn needs to be configured. So in this case you need to set the return_value for mock_post to bring the child attribute into being, and one to actually return something, i.e:
mock_post.return_value.status_code.return_value = 200
mock_post.return_value.json.return_value = nested_dictionary
You can see this by looking at the type of everything:
print(type(mock_post))
print(type(mock_post.json))
In both cases the type is <class 'unittest.mock.MagicMock'>
Probably it is better for you to look at some specialized libraries for requests testing:
responses
requests-mock
requests-testing
They provide clean way to mock responses in unittests.
An alternate approach is to just create an actual Response object and then do a configure_mock() on the original mock.
from requests import Response
class MyTests(unittest.TestCase):
#patch('mypackage.mymodule.requests.post')
def test_change_nested_dict_function(self, mock_post):
resp = Response()
resp.status_code = 200
resp.json = nested_dictionary
mock_post.configure_mock(return_value=resp)
...
I am trying to understand the following topic and have some outstanding questions. Can anyone help me?:
class MyObj(object):
def __init__(self, s):
self.s = s
def __repr__(self):
return '<MyObj(%s)>' % self.s
====================================
import json
import json_myobj
obj = json_myobj.MyObj('instance value goes here')
print 'First attempt'
try:
print json.dumps(obj)
except TypeError, err:
print 'ERROR:', err
def convert_to_builtin_type(obj):
print 'default(', repr(obj), ')'
# Convert objects to a dictionary of their representation
d = { '__class__':obj.__class__.__name__,
'__module__':obj.__module__,
}
d.update(obj.__dict__)
return d
print
print 'With default'
print json.dumps(obj, default=convert_to_builtin_type)
Question: what is the purpose of the following code?
d = { '__class__':obj.__class__.__name__,
'__module__':obj.__module__,
}
d.update(obj.__dict__)
I think there are two things you need to know to understand this code snippet.
JSON serialization and deserialization.
JSON is a data-exchange format. Particularly it is text-based, which means if you want to save your data into a text file, you have to determine how to represent your data as the text (The serialization process). Of course, when you load data from a text file, you also need to determine how to parse the text into the memory structure (The deserialization process). Luckily, by default, the json module of python would handle most of the built-in data types, e.g., scalar type, list, dict and etc. But for your case, you have created your own data type, thus you have to specify how to serialize your own data type. This is what function convert_to_builtin_type does.
Python data model
Now we come across the problem how to serialize the self-defined object Myobj. There is no uniform answer for this question, but the base line is that you can recover your object (deserialize) by the serialized text. In your case:
d = { '__class__':obj.__class__.__name__,
'__module__':obj.__module__,
}
d.update(obj.__dict__)
The obj.__dict__ is a built-in dictionary that stores attributes of obj. You may read the python documentation Data Model to understand it. The intention here is try to give enough information to recover obj. For example:
__class__=<c> provides the name of the class
__module__=<m> provides the module to find the class.
s=<v> provides the attribute and value of Myobj.s
With these three, you can recover the object you previously stored. For these hidden (built-in) attributes starting with __, you need to check the python document.
Hope this would be helpful.
I wanted to use Whoosh in my application and followed the tutorial here, which was written in 2011.
When I try to unpickle data in this block:
def results_to_instances(request, results):
instances = []
for r in results:
cls = pickle.loads('{0}'.format(r.get('cls')))
id = r.get('id')
instance = request.db.query(cls).get(id)
instances.append(instance)
return instances
I get an error from the pickle.loads() command:
TypeError: 'str' does not support the buffer interface
When I check what '{0}'.format(r.get('cls')) returns, it is type str, but the value is "b'foo'".
How do I get the bytes object out of the string? Encoding it just returns b"b'foo'".
The values are pickled in this block:
def first_index(self, writer):
oid = u'{0}'.format(self.id)
cls = u'{0}'.format(pickle.dumps(self.__class__))
attributes = []
for attr in self.__whoosh_value__.split(','):
if getattr(self, attr) is not None:
attributes.append(str(getattr(self, attr)))
value = u' '.join(attributes)
writer.add_document(id=oid, cls=cls, value=value)
So if there is a way to fix it at the root, that would be better.
Just use r.get('cls'). Wrapping it in '{0}'.format() makes a bytes into str in the first place, which is not what you want at all. Same goes for when you wrap pickle.dumps (immediately converting the useful bytes it returns to the useless formatted version). Basically all of your uses of '{0}'.format() make no sense, because they make str when you're trying to work with the raw data.
I am not sure I understand the purpose of the flask.jsonify method. I try to make a JSON string from this:
data = {"id": str(album.id), "title": album.title}
but what I get with json.dumps differs from what I get with flask.jsonify.
json.dumps(data): [{"id": "4ea856fd6506ae0db42702dd", "title": "Business"}]
flask.jsonify(data): {"id":…, "title":…}
Obviously I need to get a result that looks more like what json.dumps returns. What am I doing wrong?
The jsonify() function in flask returns a flask.Response() object that already has the appropriate content-type header 'application/json' for use with json responses. Whereas, the json.dumps() method will just return an encoded string, which would require manually adding the MIME type header.
See more about the jsonify() function here for full reference.
Edit:
Also, I've noticed that jsonify() handles kwargs or dictionaries, while json.dumps() additionally supports lists and others.
You can do:
flask.jsonify(**data)
or
flask.jsonify(id=str(album.id), title=album.title)
This is flask.jsonify()
def jsonify(*args, **kwargs):
if __debug__:
_assert_have_json()
return current_app.response_class(json.dumps(dict(*args, **kwargs),
indent=None if request.is_xhr else 2), mimetype='application/json')
The json module used is either simplejson or json in that order. current_app is a reference to the Flask() object i.e. your application. response_class() is a reference to the Response() class.
The choice of one or another depends on what you intend to do.
From what I do understand:
jsonify would be useful when you are building an API someone would query and expect json in return. E.g: The REST github API could use this method to answer your request.
dumps, is more about formating data/python object into json and work on it inside your application. For instance, I need to pass an object to my representation layer where some javascript will display graph. You'll feed javascript with the Json generated by dumps.
consider
data={'fld':'hello'}
now
jsonify(data)
will yield {'fld':'hello'} and
json.dumps(data)
gives
"<html><body><p>{'fld':'hello'}</p></body></html>"
When my page POSTs a form to my Django view, the view returns a response with some data but soon I ran into the issue that my views returned data in different formats and different sets of information. I've thought of using JSON as a standard format to return my data.
There are two types of statuses, success and failure. When the POST was successful, it just returns success but when it has failed, it returns a sub group called errors which contains a of fields and that field's error.
Here's a sample failure response:
{"failure": {
"errors":[
{"fieldname": "<fieldname>",
"fielderror": "<fielderror>"
},
{"fieldname": "<fieldname>",
"fielderror": "<fielderror>"
}]
}}
Here's a sample success response:
{"success": {
"data":[
{"fieldname": "<fieldname>",
"fielddata": "<fielddata>"
},
{"fieldname": "<fieldname>",
"fielddata": "<fielddata>"
}]
}}
(The success response has data fields because quite often you like to return some data i.e. key of newly created DB record.)
This is what I've come up with but so many people using Django out there, I'm wondering whether there is a standard way/more robust of doing this or some module to help with this.
Thanks.
been there. I wrote the solution my own (since it's simple. I dont know if theres a module out there for this). This is just a json response wrapper
from django.utils import simplejson
from django.http import HttpResponse
class JsonResponse(HttpResponse):
def __init__(self, data):
content = simplejson.dumps(data)
super(JsonResponse, self).__init__(content=content,
mimetype='application/json')
class Success(JsonResponse):
def __init__(self, something):
x = something # set your data here
content = {'success': {'data': x}}
super(Success, self).__init__(content)
class Failure(JsonResponse):
def __init__(self, something):
x = something # set your data
content = {'failures': {'errors': x}}
super(Failure, self).__init__(content)
something like that. In my case, I make Success and Failure accept a string and a dictionary for arguments (like Success(data)) to make it even easier
If your structure is quite complex (or youre too lazy too write them manually), write your own wrapper for that complicated structure (so you can create the response with even less typing), and make the Success/Failure accepts that as argument.
Do you know how to use Python dicts? If so then there are libraries which convert your Python dict into valid JSON. Django's built-in is for example is simplejson. Here an example:
from django.utils import simplejson
def my_view(request):
....
my_json = simplejson.dumps(my_dict) #dumps a JSON string from a dict
my_dict_again = simplejson.loads(my_json) #loads a Python dict from a JSON string
return HttpResponse(json, mimetype="application/json")
Well, I don't know how much this will help you out, but Python does have a module json.
>>> import json
>>> json.dumps({'a':'b', 'c':'d'})
'{"a": "b", "c": "d"}'
I think the front end is going to make a stronger case for how you should format your data then some "standard", Different client side frameworks expect different things, and adapt to differences from that expectations better or worse than that.
One way I try to make my own life easier is to have the 'shape' of the response always roughly the same, for example, the response will always have a 'status' property, which always has an 'errors' property, and if there were no errors, then the value of result.status.errors is null. If there were errors, it'd be a list of strings explaining what went wrong. even if there were errors, there'd be a 'value' property, which would be either null if the request couldn't be serviced, or the requested resource if it could.