I am working on something which should manage multiple water dispensers. I need to get some data from a json file and then load it into objects after that, append the objects to a list. For some reason list.append changes other object's parameters(more specific, location). Here is my code:
WaterDispenser.py
class WaterDispenser():
def __init__(self, id: int = -1, status: bool = False, location: list=[-1, -1]) -> None:
self.id = id
self.status = status
self.location = location
def Dump(self) -> dict:
"""Dumps the propoerties in a json dictionary
Returns
-------
dict
A dictionary with a collection of propoerties and their names
"""
return {"id": self.id, "status": self.status, "location":[self.location[0], self.location[1]]}
def Load(self, object: dict) -> None:
"""Loads the json dictoinary in memory
Parameters
----------
object : dict, required
The json file with the properties of the dispenser
Returns
-------
None
"""
self.id = object["id"]
self.status = object["status"]
self.location[0] = object["location"][0]
self.location[1] = object["location"][1]
return None
main.py
import json
from WaterDispenser import WaterDispenser
dispensers = []
def LoadDispensers(path: str = "dispensers.json") -> int:
"""Loads the json file in memory.
Parameters
---------
path : str, optional
The path of the file to be loaded. Defaults to "dispensers.json".
Returns
-------
int
Count of dispensers data loaded
"""
global dispensers
dispensers = []
data = json.load(open(path, "r"))
for d in data:
x = WaterDispenser()
x.Load(d)
dispensers.append(x)
return len(dispensers)
if __name__ == '__main__':
print(LoadDispensers())
print([o.Dump() for o in dispensers])
dispensers.json
[
{"id": 0, "status": true, "location": [0, 0]},
{"id": 1, "status": true, "location": [0, 1]},
{"id": 2, "status": false, "location": [1, 1]}
]
Output:
3
[{'id': 0, 'status': True, 'location': [1, 1]}, {'id': 1, 'status': True, 'location': [1, 1]}, {'id': 2, 'status': False, 'location': [1, 1]}]
The functional answer:
Change the init of WaterDispenser to
from typing import Optional
class WaterDispenser():
def __init__(self, id: int = -1, status: bool = False, location: Optional[list] = None) -> None:
self.id = id
self.status = status
self.location = location or [-1, -1]
This should result in the expected response of
3
[{'id': 0, 'status': True, 'location': [0, 0]}, {'id': 1, 'status': True, 'location': [0, 1]}, {'id': 2, 'status': False, 'location': [1, 1]}]
The why:
Generally you want to avoid using mutable values as kwarg values because they're pre-computed (so your default location argument was technically the same object in memory across your WaterDispenser instances). append wasn't the culprit here and you can read more about this all via this SO discussion or read a succinct explanation via this answer to a similar question.
Design note:
It's worth noting that the way you are using Load in the above example could just be folded into WaterDispenser.__init__, so something like
from typing import Dict, Any
class WaterDispenser():
def __init__(self, data: Dict[Any]) -> None:
self.id = data.get("id", -1)
self.status = data.get("status", False)
self.location = data.get("location", [-1, -1])
or if you want to avoid typing
class WaterDispenser():
def __init__(self, data: dict) -> None:
self.id = data.get("id", -1)
self.status = data.get("status", False)
self.location = data.get("location", [-1, -1])
That example still includes your default values but if you removed the secondary arguments from those get calls you could protect against missing data at runtime without having to check to see if you had, say, an impossible location data point like [-1, -1].
Related
I have a song class, which holds the attributes to a song, and it is a custom class. I also have a list of songs in a list called track list. When I try to json.dump the list, I get an error that says :
TypeError: Object of type 'Song' is not JSON serializable
How would I go about converting this list of songs to json?
Here is the additional relevant code that returns the error:
class Song:
def __init__(self, sname, sartist, coverart, albname, albartist, spotid):
self.sname = sname
self.sartist = sartist
self.coverart = coverart
self.albname = albname
self.albartist = albartist
self.spotid = spotid
tracklist = createDict(tracks) ##creates the list of songs, works fine
jsontracks = json.dumps(tracklist)
pp.pprint(jsontracks)
Thanks
I've solved this by adding an encode() method to the class:
def encode(self):
return self.__dict__
and adding some arguments to json.dumps:
jsontracks = json.dumps(tracklist, default=lambda o: o.encode(), indent=4)
This will "crawl" down your class tree (if you have any child classes) and encode every object as a json list/object automatically. This should work with just about any class and is fast to type. You may also want to control which class parameters get encoded with something like:
def encode(self):
return {'name': self.name,
'code': self.code,
'amount': self.amount,
'minimum': self.minimum,
'maximum': self.maximum}
or a little bit faster to edit (if you're lazy like me):
def encode(self):
encoded_items = ['name', 'code', 'batch_size', 'cost',
'unit', 'ingredients', 'nutrients']
return {k: v for k, v in self.__dict__.items() if k in encoded_items}
full code:
import json
class Song:
def __init__(self, sname, sartist, coverart, albname, albartist, spotid):
self.sname = sname
self.sartist = sartist
self.coverart = coverart
self.albname = albname
self.albartist = albartist
self.spotid = spotid
def encode(self):
return self.__dict__
tracklist = [
Song('Imagine', 'John Lennon', None, None, None, None),
Song('Hey Jude', 'The Beatles', None, None, None, None),
Song('(I Can\'t Get No) Satisfaction', 'The Rolling Stones', None, None, None, None),
]
jsontracks = json.dumps(tracklist, default=lambda o: o.encode(), indent=4)
print(jsontracks)
output:
[
{
"sname": "Imagine",
"sartist": "John Lennon",
"coverart": null,
"albname": null,
"albartist": null,
"spotid": null
},
{
"sname": "Hey Jude",
"sartist": "The Beatles",
"coverart": null,
"albname": null,
"albartist": null,
"spotid": null
},
{
"sname": "(I Can't Get No) Satisfaction",
"sartist": "The Rolling Stones",
"coverart": null,
"albname": null,
"albartist": null,
"spotid": null
}
]
Description of the problem
The problem is a classical Bill of Materials (BoM) problem;
Suppose we have the class BomEntry(object) defined as:
class BomEntry:
def __init__(self, part, quantity=0, unit="", children=[]):
self.part = part
self.quantity = quantity
self.unit = unit
self.children = children
part is a django model, and quantity and unit are two of its members.
The Django model has a method make_bom(self) which returns an instance of BomEntry(a class which doesn't use django) . Asm is the django model keeping track of BoM data in the database
def make_bom(self, depth=1):
if not self.is_asm:
return BomEntry(self, 1, "", [])
else:
children = list(Asm.objects.filter(parent=self))
children_bom = [BomEntry(obj.child, obj.quantity, obj.unit, []) for obj in children]
bom = BomEntry(self, 1, "", children=children_bom)
return bom
I'm currently including a parameter to decide the depth of the BoM, but I can't wrap my head around how I would use it.
I want to be able to traverse the nested objects, ending up with an output similar to this:
{
'part': <PartAsm: 100-1822-R1-A>,
'quantity': 1,
'unit': '',
'children':
[
{
'part': <PartAsm: 100-1823-R1-A>,
'quantity': 1,
'unit': '',
'children':
[]
},
{
'part':
<PartAsm: 100-1824-R1-A>,
'quantity': 1,
'unit': '',
'children':
[
{
'part': <PartAsm: 100-1825-R1-A>,
'quantity': Decimal('1.00'),
'unit': 'g',
'children':
[]
},
{
'part': <PartAsm: 100-1826-R1-A>,
'quantity': Decimal('1.00'),
'unit': 'g',
'children':
[]
}
]
}
]
}
The output above was acquired using the console, I would appreciate any advice on looping this or making it recursive. I hope I provided sufficient and clear information
When depth is more than 1, make_bom() should recurse, decrementing depth in the call.
def make_bom(self, depth=1):
if not self.is_asm:
return BomEntry(self, 1, "", [])
else:
if depth > 1:
children = list(Asm.objects.filter(parent=self))
children_bom = [make_bom(BomEntry(obj.child, obj.quantity, obj.unit, []), depth-1) for obj in children]
else:
children_bom = []
bom = BomEntry(self, 1, "", children=children_bom)
return bom
I finally got it working using #Barmar 's valuable input. The make_bom function now looks like:
def make_bom(self, quantity=1, unit="def_unit", depth=1):
if not self.is_asm:
return BomEntry(self, quantity, unit, [])
else:
if depth <= 1:
children = list(Asm.objects.filter(parent=self))
children_bom = [BomEntry(obj.child, quantity*obj.quantity, obj.unit, []) for obj in children]
bom = BomEntry(self, quantity, unit, children=children_bom)
else:
bom = BomEntry(self, quantity, "def_unit",
children=[a.child.make_bom(quantity*a.quantity, a.unit, depth - 1) for a in
list(Asm.objects.filter(parent=self))])
return bom
I added in the quantity and unit params for completeness.
Thanks a million #Barmar
I have a list of objects with ids (object.id) and a list of ids. What is the most efficient way to use the list of ids to order the object list? I have the following solution... just wondering whether there is a faster one?
Input:
result = list of user objects with user ids (user.id)
list_of_ids = list of user ids e.g. [3, 2, 5, 8, 9]
Output:
list of user objects ordered according to list_of_ids
Code:
ordered_result = []
for user_id in list_of_ids:
for user in result:
if user_id == user.id:
ordered_result.append(user)
break
You could first put the users in a dict:
usrdict = {}
for user in result:
usrdict[user.id] = user
And then you'd have a few options, based on looking up the users by their id, for example:
ordered_result = [usrdict[x] for x in list_of_ids]
Edit: I'd probably, unless you have a very large list or have perform the operation many times, not worry about efficiency that much, but rather focus on having it clear and readable.
You can use sorted function with custom sorting function. In this case returning index in the ids list.
def order_by(objects, ids):
def fn(obj):
return ids.index(obj["id"])
return sorted(objects, key=fn)
print(order_by(objects_list, id_list))
Example:
objects_list = [
{ "id": 3, "name": "penny"},
{ "id": 5, "name": "adam"},
{ "id": 9, "name": "meh"},
{ "id": 1, "name": "john"},
{ "id": 3, "name": "archibald"},
]
id_list = [9,1,3,5,6,4]
print(order_by(objects_list, id_list))
Results in:
[{'id': 9, 'name': 'meh'}, {'id': 1, 'name': 'john'}, {'id': 3, 'name': 'penny'}, {'id': 3, 'name': 'archibald'}, {'id': 5, 'name': 'adam'}]
Here is an O(n log(n)) solution. It requires the ids in both lists to be exactly the same. It sorts the object list by id and does an indirect sort of the id list yielding an index list containing the positions of ids in order. This is then used to move the sorted objects to the right positions.
import operator
class know_who_you_are:
def __init__(self, id_):
self.id = id_
def argsort(L):
"returns sorted, order"
return zip(*sorted(zip(L, range(len(L)))))
ids = [3, 2, 4, 5, 1]
objs = [know_who_you_are(id_) for id_ in [1, 5, 3, 2, 4]]
sid, oid = argsort(ids)
sobj = sorted(objs, key=operator.attrgetter('id'))
result = len(objs) * [None]
for dest, src in zip(oid, sobj):
result[dest] = src
# check
print(all(id_==obj.id for id_, obj in zip(ids, result)))
Prints:
True
Here's a variation of the scheme using the built-in sorted() function with a custom comparison function. (It may seem like a lot of code, but a significant portion of it is only there for setting up a somewhat realistic test case.)
from functools import cmp_to_key
import random
random.seed(13) # Gets consistent "random" ordering during testing.
class User:
def __init__(self, name, id):
self.name = name
self.id = id
def __repr__(self):
return '{}({!r}, id={!r})'.format(self.__class__.__name__, self.name, self.id)
#cmp_to_key # Converts cmp function to a key function.
def cmp(x, y):
""" Return -1 if the position of User x in list_of_ids < index of User y
otherwise return 1.
"""
p1, p2 = -1, -1
try:
p1 = list_of_ids.index(x.id)
p2 = list_of_ids.index(y.id)
except ValueError:
pass
return -1 if p1 < p2 else 1
list_of_ids = [3, 2, 5, 8, 9]
# Create a random list of users with these ids.
shuffled_ids = random.sample(list_of_ids, k=len(list_of_ids))
users = [User(name, id) for name, id in zip(['Andy', 'Simon', 'Nick', 'John',
'Roger'], shuffled_ids)]
print('Desired id order:', list_of_ids)
print()
print(' Before:', users)
ordered_result = sorted(users, key=cmp)
print('Ordered:', ordered_result)
Output:
Desired id order: [3, 2, 5, 8, 9]
Before: [User('Andy', id=5), User('Simon', id=9), User('Nick', id=8), User('John', id=3), User('Roger', id=2)]
Ordered: [User('John', id=3), User('Roger', id=2), User('Andy', id=5), User('Nick', id=8), User('Simon', id=9)]
I have a custom data file formatted like this:
{
data = {
friends = {
max = 0 0,
min = 0 0,
},
family = {
cars = {
van = "honda",
car = "ford",
bike = "trek",
},
presets = {
location = "italy",
size = 10,
travelers = False,
},
version = 1,
},
},
}
I want to collect the blocks of data, meaning string between each set of {} while maintaining a hierarhcy. This data is not a typical json format so that is not a possible solution.
My idea was to create a class object like so
class Block:
def __init__(self, header, children):
self.header = header
self.children = children
Where i would then loop through the data line by line 'somehow' collecting the necessary data so my resulting output would like something like this...
Block("data = {}", [
Block("friends = {max = 0 0,\n min = 0 0,}", []),
Block("family = {version = 1}", [...])
])
In short I'm looking for help on ways I can serialize this into useful data I can then easily manipulate. So my approach is to break into objects by using the {} as dividers.
If anyone has suggestions on ways to better approach this I'm all up for ideas. Thank you again.
So far I've just implemented the basic snippets of code
class Block:
def __init__(self, content, children):
self.content = content
self.children = children
def GetBlock(strArr=[]):
print len(strArr)
# blocks = []
blockStart = "{"
blockEnd = "}"
with open(filepath, 'r') as file:
data = file.readlines()
blocks = GetBlock(strArr=data)
You can create a to_block function that takes the lines from your file as an iterator and recursively creates a nested dictionary from those. (Of course you could also use a custom Block class, but I don't really see the benefit in doing so.)
def to_block(lines):
block = {}
for line in lines:
if line.strip().endswith(("}", "},")):
break
key, value = map(str.strip, line.split(" = "))
if value.endswith("{"):
value = to_block(lines)
block[key] = value
return block
When calling it, you have to strip the first line, though. Also, evaluating the "leafs" to e.g. numbers or strings is left as an excercise to the reader.
>>> to_block(iter(data.splitlines()[1:]))
{'data': {'family': {'version': '1,',
'cars': {'bike': '"trek",', 'car': '"ford",', 'van': '"honda",'},
'presets': {'travelers': 'False,', 'size': '10,', 'location': '"italy",'}},
'friends': {'max': '0 0,', 'min': '0 0,'}}}
Or when reading from a file:
with open("data.txt") as f:
next(f) # skip first line
res = to_block(f)
Alternatively, you can do some preprocessing to transform that string into a JSON(-ish) string and then use json.loads. However, I would not go all the way here but instead just wrap the values into "" (and replace the original " with ' before that), otherwise there is too much risk to accidentally turning a string with spaces into a list or similar. You can sort those out once you've created the JSON data.
>>> data = data.replace('"', "'")
>>> data = re.sub(r'= (.+),$', r'= "\1",', data, flags=re.M)
>>> data = re.sub(r'^\s*(\w+) = ', r'"\1": ', data, flags=re.M)
>>> data = re.sub(r',$\s*}', r'}', data, flags=re.M)
>>> json.loads(data)
{'data': {'family': {'version': '1',
'presets': {'size': '10', 'travelers': 'False', 'location': "'italy'"},
'cars': {'bike': "'trek'", 'van': "'honda'", 'car': "'ford'"}},
'friends': {'max': '0 0', 'min': '0 0'}}}
You can also do with ast or json with the help of regex substitutions.
import re
a = """{
data = {
friends = {
max = 0 0,
min = 0 0,
},
family = {
cars = {
van = "honda",
car = "ford",
bike = "trek",
},
presets = {
location = "italy",
size = 10,
travelers = False,
},
version = 1,
},
},
}"""
#with ast
a = re.sub("(\w+)\s*=\s*", '"\\1":', a)
a = re.sub(":\s*((?:\d+)(?: \d+)+)", lambda x:':[' + x.group(1).replace(" ", ",") + "]", a)
import ast
print ast.literal_eval(a)
#{'data': {'friends': {'max': [0, 0], 'min': [0, 0]}, 'family': {'cars': {'car': 'ford', 'bike': 'trek', 'van': 'honda'}, 'presets': {'travelers': False, 'location': 'italy', 'size': 10}, 'version': 1}}}
#with json
import json
a = re.sub(",(\s*\})", "\\1", a)
a = a.replace(":True", ":true").replace(":False", ":false").replace(":None", ":null")
print json.loads(a)
#{u'data': {u'friends': {u'max': [0, 0], u'min': [0, 0]}, u'family': {u'cars': {u'car': u'ford', u'bike': u'trek', u'van': u'honda'}, u'presets': {u'travelers': False, u'location': u'italy', u'size': 10}, u'version': 1}}}
In this example code I'm trying to make the whole pile of data callable:
class Data(object):
def __init__(self, data):
self._raw_data = data
for key, val in data.items():
if isinstance(val, dict):
if 'data' in val.keys():
setattr(self, key, Dataset(val))
else:
setattr(self, key, Data(val))
else:
setattr(self, key, val)
def __getattr__(self, name):
return None
#property
def raw(self):
return self._raw_data
class Dataset(Data):
def __init__(self, data):
self._raw_data = data
super().__init__(data)
for entry in self.data:
# need to debug this
self.data[self.data.index(entry)] = Data(entry)
What ends up happening is this. I'll try:
>>> example = {'dataset': {'id': 1, 'data': [{'param1': 3, 'param2': 4}, {'param1': 5, 'param2': 6}]}}
>>> data1 = Data(example)
>>> print(data1.dataset.id)
1
Now that works as expected. The problem the last line, which causes change in __raw_data attribute in Dataset object and I have no idea how (and also, looks wrong). Clearly I don't understand something important about inherritance. What I am trying to do is put all data into callable attributes, but also keep the raw data for all kinds of purposes. This is what I get however:
>>> print(data1.raw)
{'dataset': {'id': 1, 'data': [<wtf.Data object at 0x102a8da58, <wtf.Data object at 0x102a8da90>]}}
It should however return raw data as it was passed:
>>> print(data1.raw)
{'dataset': {'id': 1, 'data': [{'param1': 3, 'param2': 4}, {'param1': 5, 'param2': 6}]}}
It works for Data objects:
>>> for item in data1.dataset.data:
... print(item.raw)
{'param2': 4, 'param1': 3}
{'param2': 6, 'param1': 5}
Thanks for help.