I'm using Colander to convert JSON strings to Python Objects and vice versa in a Pyramid/Cornice based project.
Is there any way that you can serialize/deserialize to different names/keys ?
Here is the Colander Schema:
class CommentSchema(MappingSchema):
resource_id = SchemaNode(Int(), name="resourceID", location="body")
text = SchemaNode(String(), name="text", location="body")
And here is the input JSON
{"text":"Hello!", "resourceID":12}
It is being converted to :
{u'text': u'Hello!', u'resourceID': 12}
Here is my question, can I have the same input JSON converted to the following?
{u'full_text': u'Hello!', u'resource_id': 12}
Thanks for your help.
I eventually had to do it manually.
Whatever is received from the JSON is used to construct a Data Object. The Object would have a custom function to map the data to the desired output format, and you pass the output to the serializer:
data_schema = DataSchema().deserialize(self.request.json)
data_obj = DataObject(data_schema**) // or DataObject(full_text = data_schema['text'], resource_id = data_schema['resourceID'])
#
# ...
#
rbody = DataSchema().serialize(data_obj.map_dump())
return Response(body=rbody, status_code=201)
And the DataObject would look like this:
class DataObject(Object):
def __init__(self, text, resourceID): // or __init__(self, full_text, resource_id)
self.text = text
self.resourceID = resourceID
def map_dump(self):
output['full_text'] = self.text
output['resource_id'] = self.resource
return output
Related
I have the following model:
class Car(models.Model):
data = models.TextField(default="[]")
Also, I have the following serializer:
class CarSerializer(serializers.ModelSerializer):
data = serializers.ListField(child=serializers.CharField())
The REST API gets data and saves it as text field. In my to_dict method of Car, I want to convert self.data into JSON and return the dict:
def to_dict(self):
result = dict()
result['data']= json.loads(self.data)
return result
But it fails with the error:
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
As I understand, the reason is that self.data is:
"['a', 'b', 'c']"
And not:
'["a", "b", "c"]'
I'm familiar with JsonField, but since I'm using SQLite without JSON1 externation, I can't use it. How can I convert self.data to JSON?
You can use python json.dumps() method to convert string into json format and then use json.loads() to convert json into python object.
import json
def to_dict(self):
result = dict()
data = json.dumps(self.data)
result['data'] = json.loads(data)
return result
The simplest way to solve this problem is json.loads(self.data.replace('\'','\"')).
Replace ' to ".
Or you can try eval(self.data)
you can watch a sample here about the usage of eval
I am getting Data via a REST-Interface and I want to store those data in a class-object.
my class could looks like this:
class Foo:
firstname = ''
lastname = ''
street = ''
number = ''
and the json may look like this:
[
{
"fname": "Carl",
"lname": "any name",
"address": ['carls street', 12]
}
]
What's the easiest way to map between the json and my class?
My problem is: I want to have a class with a different structure than the json.
I want the names of the attributes to be more meaningful.
Of course I know that I could simply write a to_json method and a from_json method which does what I want.
The thing is: I have a lot of those classes and I am looking for more declarative way to write the code.
e.g. in Java I probably would use mapstruct.
Thanks for your help!
Use a dict for the json input. Use **kwargs in an __init__ method in your class and map the variables accordingly.
I had a similar problem, and I solved it by using #classmethod
import json
class Robot():
def __init__(self, x, y):
self.type = "new-robot"
self.x = x
self.y = y
#classmethod
def create_robot(cls, sdict):
if sdict["type"] == "new-robot":
position = sdict["position"]
return cls(position['x'], position['y'])
else:
raise Exception ("Unable to create a new robot!!!")
if __name__=='__main__':
input_string = '{"type": "new-robot", "position": {"x": 3, "y": 3}}'
cmd = json.loads(input_string)
bot = Robot.create_robot(cmd)
print(bot.type)
Perhaps you could you two classes, one directly aligned with the Json (your source class) and the other having the actual structure you need. Then you could map them using the ObjectMapper class[https://pypi.org/project/object-mapper/]. This is very close to the MapStruct Library for Java.
ObjectMapper is a class for automatic object mapping. It helps you to create objects between project layers (data layer, service layer, view) in a simple, transparent way.
I am trying to use Marshmallow schema to serialize the python object. Below is the schema I have defined for my data.
from marshmallow import Schema, fields
class User:
def __init__(self, name = None, age = None, is_active = None, details = None):
self.name = name
self.age = age
self.is_active = is_active
self.details = details
class UserSchema(Schema):
name = fields.Str()
age = fields.Int()
is_active = fields.Bool()
details = fields.Dict()
The input will be in dictionary format and all the values will be in string.
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
When I try to run the below snippet, values of age and is_active got converted into respective datatype but details remains unchanged.
user_schema = UserSchema()
user_dump_data = user_schema.dump(user_data)
print(user_dump_data)
Output:
{'name': 'xyz', 'is_active': True, 'details': "{'key1':'val1', 'key2':'val2'}", 'age': 20}
I need to serialize the input data into respective datatype I defined in my schema. Is there anything I am doing wrongly? Can anyone guide me how to acheive this using Marshmallow?
I am using
python 3.6
marshmallow 3.5.1
Edit
The above input data is fetched from HBase. By default HBase stores all its values as bytes and return as bytes. Below is the format I get from HBase
{b'name': b'xyz', b'age': b'20', b'is_active': b'true', b'details': b"{'key1':'val1', 'key2':'val2'}"}
Then I decode this dictionary and pass it to my UserSchema to serialize it to be used in web API.
You're confusing serializing (dumping) and deserializing (loading).
Dumping is going from object form to json-serializable basic python types (using Schema.dump) or json string (using Schema.dumps). Loading is the reverse operation.
Typically, your API loads (and validates) data from the outside world and dumps (without validation) your objects to the outside world.
If your input data is this data and you want to load it into objects, you need to use load, not dump.
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
user_loaded_data = user_schema.load(user_data)
user = User(**user_loaded_data)
Except if you do so, you'll be caught by another issue. DictField expects data as a dict, not a str. You need to enter
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details": {'key1':'val1', 'key2':'val2'}}
As Jérôme mentioned you're confusing serializing(dumping) with deserializing(loading). As per your requirement, you should use Schema.load as suggested.
Since, all the input values are expected to be of string type. You can use, pre_load to register a method for pre-processing the data as follows:
from marshmallow import Schema, fields, pre_load
class UserSchema(Schema):
name = fields.Str()
age = fields.Int()
is_active = fields.Bool()
details = fields.Dict()
#pre_load
def pre_process_details(self, data, **kwarg):
data['details'] = eval(data['details'])
return data
user_data = {"name":"xyz", "age":"20", "is_active": 'true',"details":"{'key1':'val1', 'key2':'val2'}"}
user_schema = UserSchema()
user_loaded_data = user_schema.load(user_data)
print(user_loaded_data)
Here, pre_process_details will convert string type to dictionary for correct deserialization.
I'm getting data from an API and saving into a list called data[].
After this, I'm sending these data to another class to format it.
I want to create a dict, so I can save these data. I'm trying to do something like this:
import json
import datetime
class MovieFormatter():
def format(self, data):
data = {
"movie_info_name": data['title']
"movie_info_duration": data['duration']
"movie_info_description": data['synopsis']
"movie_info_genres": data['genres']
"movie_info_actor": data['cast']
"movie_info_director": data['director']
data['trailers'] = data.get('trailers') or []
"dictionary": [{'url': x['url'], 'type': x['type']} for x in data['images'] + data['trailers']]
}
return data
Is this the right way to do?
It seems that the data object passed to your function is already a dictionary, from the way it has been indexed e.g. data['title'].
Try this :
_in_data = ["a","b","c"]
_out_data = ["x","y","z"]
_dict={}
print(_dict)
for i in range (len(_in_data)):
_dict[_in_data[i]]=_out_data[i]
print(_dict)
Lets say I have
import peewee
class Foo(Model):
name = CharField()
I would like to do the following:
f = {id:1, name:"bar"}
foo = Foo.create_from_dict(f)
Is this native in Peewee? I was unable to spot anything in the source code.
I've wrote this function which works but would rather use the native function if it exists:
#clazz is a string for the name of the Model, i.e. 'Foo'
def model_from_dict(clazz, dictionary):
#convert the string into the actual model class
clazz = reduce(getattr, clazz.split("."), sys.modules[__name__])
model = clazz()
for key in dictionary.keys():
#set the attributes of the model
model.__dict__['_data'][key] = dictionary[key]
return model
I have a web page that displays all the foos and allows the user to edit them. I would like to be able to pass a JSON string to the controller, where I would convert it to a dict and then make Foos out of it, so I can update as necessary.
If you have a dict, you can simply:
class User(Model):
name = CharField()
email = CharField()
d = {'name': 'Charlie', 'email': 'foo#bar.com'}
User.create(**d)
You could use PickledKeyStore which allows you to save any value as a python dict and it works like Python's pickle library.