How can i implement structure array like matlab in python ?
matlab code :
cluster.c=[]
cluster.indiv=[]
Although you can do this in Python (as I explain below), it might not be the best or most pythonic approach. For other users that have to look at your code (including yourself in 3 months) this syntax is extremely confusing. Think for example about how this deals with name conflicts, undefined values and iterating over properties.
Instead consider storing the data in a data structure that is better suited for this such as a dictionary. Then you can just store everything in
cluster = {'c':[],'indiv':[]}
Imitating Matlab in a bad way:
You can assign properties to any mutable objects in python.
If you need an object just for data storage, then you can define a custom class without any functionality in the following way:
class CustomStruct():
pass
Then you can have
struct=CustomStruct()
struct.c=[]
and change or request properties of the class in this way.
Better approach:
If you really want to store these things as properties of an object, then it might be best to define the variables in the init of that class.
class BetterStruct():
def __init__(self):
self.c=[]
self.indiv=[]
In this way, users looking at your code can immediately understand the expected values, and you can guarantee that they are initalised in a proper fashion.
Allowing data control
If you want to verify the data when it is stored, or if it has to be calculated once the user requests it (instead of storing it constantly), then consider using Python property decorators
Related
I am trying to insert a collection of objects in a flask api. We use marshmallow for deserializing. My endpoint looks like this:
#blp.arguments(SomeSchemas, location='json', as_kwargs=True)
#blp.response(200, SomeSchemas)
def post(self, some_schemas: SomeSchemas) -> dict:
The schema is a simple schema like this:
class SomeSchemas(ma.Schema):
schemas = ma.fields.List(ma.fields.Nested(SomeSchema))
class SomeSchema(ma.Schema):
a = ma.fields.String()
b = ma.fields.Integer()
When i post to the endpoint, I do get a list of the correct data, but it comes in the form of dicts, instead of being correctly translated into the object.
I have also tried explicitly using a list of objects (List[SomeSchema], SomeSchema(many=True), etc.) but I can not seem to figure it out.
I assume this is a very common use case (providing a list of arguments) and that I am missing an obvious solution, but I can't seem to find any reference as to how to do this correctly. To be clear, I am looking for the correct way to call the endpoint with a list (or some other collection type, it does not matter) and have said list be correctly deserialized and with the correct object type.
Disclaimer: flask-smorest maintainer speaking.
I don't think the issue is related to the fact that the input is a list.
IIUC, your problem is that you're gettings dicts, rather than objects, injected in the view function. This is the default marshmallow behaviour. It can be overridden in marshmallow by using a post_load hook to actually instantiate the object.
I generally don't do that. In practice I find it better to instantiate objects in the view function. For instance, in a PUT resource, I prefer to instantiate the existing item from DB then update it with new data. In this case it is better to have the new data as dict than object.
There may not be a single truth, here, it could be a matter of opinion, but while the idea of having the object instantiated in the decorator and passed to the view is appealing, it might be a bit of an abusive shortcut.
I realize this answer is of the "no idea but you shouldn't do it anyway" type. Just saying maybe you shouldn't struggle to achieve this.
This said, I'd be surprised if it worked with non-list / non-nested inputs and I'd wonder why it doesn't work specifically in this case.
This question already has answers here:
How to make a class JSON serializable
(41 answers)
Closed 6 months ago.
I'm noting that the methods I am looking at to serialize a variable into JSON in python don't really seem to handle it all that well, and for my purpose I just want to quickly dump an objects contents into a string format so I can pick out what I actually want to write custom code to handle. I want to be able to dump the main fields at the very least of any class I pass the python serializer and really if its worth the name this should work.
So take the following code:
import json
c = SomeClass()
#causes an error if any field in someclass has another class instance.
json.dumps(c)
leads to..
TypeError: Object of type {Type} is not JSON serializable
Are there any modules other people have used that would solve my problem ? I really don't see how there would not be. Or maybe one might explain how to circumvent this error ?
The goal is to simply get some output to look at. If I wrote a recursion loop in c# using reflection, excepting circular references, it wouldn't be difficult, so I cannot imagine python users have never tackled this exact issue and I'm not satisfied with the answers that I have seen in older posts which seem to suggest a lot of custom tinkering for something seems to be designed in spirit to just dump any old object's contents out.
I don't even need complex traversal is the funny part, though it would be nice. I just need a dump of the property values which are primitive types in many cases. I know this is possible because the debugger does it.
Additionally I looked at one of the methods given indicating to use default lambda to specify how the json serializer should descend into the object:
json.dumps(o, default=lambda k: k.__dict__)
and the object does not contain the standard dict member.
in the end I just ended up writing a class to do this.
edit:
Here use this now you can one way serialize a class structure with this nifty little bit of code that I added to address my problem with f**** discord.py !
end edit
There is no fire and forget option that would disentangle a mass of information.
The way of creating this solution would be to manage seperate lists of subclasses to make sure not to recurse until a stackoverflow is reached.
The slots_ can be used with getattr(o,name) when hasattr(o,'dict') is False.
But the answer is you'd have to create a solution that basically does the job that the json serializer should be doing and cut out circular reference by determining the unique complex types and writing them in seperate tabular entries in the json file and replacing them in the referencing classes with ids.
That way you could cross reference these objects while glancing at them.
However the short answer is no. Python does not offer an out of the box way of doing this and all the provided answers encountered thus far only solve a single use-case or scenario, and do not create a incorporated solution to the problem which the above mentioned algorithm WOULD by NORMALIZING the class data into unique elements.
I have a large-ish YAML file (~40 lines) that I'm loading using PyYAML. This is of course parsed into a large-ish dictionary plus a couple of arrays.
My question is: how to manage the data. I can of course leave it in the output dictionary and work through the data. But I was wondering if it's better instead to mangle the data in a class or use a nametuple to hold the data.
Any first-hand experience about that?
Whether you post-process the data structure into a class or not primarily has to do with how you are using that data. The same applies to the decision whether to use a tag or not and load (some off) the data from the YAML file into a specific instance of a class that way.
The primary advantage of using a class in both cases (post-processing, tagging) is that you can do additional tests during initialisation for consistency, that are not done on the key-value pairs of a dict or on the items of list.
A class also allows you to provide methods to check values before they are set, e.g. to make sure they are of the right type.
Whether that overhead is necessary depends on the project, who is using and/or updating the data etc and how long this project and its data is going to live (i.e. are you still going to understand the data and its implicit structure a year from now). These are all issues for which a well designed (and documented) class can help, at the cost of some extra work up-front.
I am looking for an appropriate data structure in Python for processing variably structured forms. By variably structured forms I mean that the number of form fields and the types of the form's contents are not known in advance. They are defined by the user who populates the forms with his input.
What are the pros and cons of putting data in A) object attributes (e.g. of an otherwise empty "form"-class) or B) simply lists/dicts? Consider that I have to preserve the sequence of form fields, the form field names and the types.
(Strangely, it has been difficult to find conclusive information on this topic. As I am still new to Python, it's possible that I have searched for the wrong terms. If my question is not clear enough, please ask in the comments and I will try to clarify.)
In Python, as in all object-oriented languages, the purpose of classes is to associate data and closely-related methods that act on that data. If there's no real encapsulation going on (i.e. the methods help define the ways you can interact with the data), the best choice is a conglomeration of builtin types like lists and dictionaries as you mention and perhaps some utility functions that act on those sorts of data structures.
Python classes are literally just two dicts (one for functions, one for data), a name and the rules how Python looks for keys. When you access existing keys, there is absolutely no difference to a dict (unless you overwrote the access rules of cause).
That means that there is no drawback (besides more code) to using classes at all and you should never be afraid to write a class.
In your particular case I think you should go with classes, for one simple reason: You might want to extend them later. Maybe you want to add constraints on the name (length, allowed letters, uniqueness, ...) or the value (not empty, length, type, ...) of a field one day. Maybe you want to validate all fields in a form. If you use a class you can do this without changing any code outside the class! And as I said before, even if you don't, there are no drawbacks!
I guess my rule of thumb for classes is: Don't use a class if you're absolutely sure that there is nothing to add to it. If not just write those few extra lines.
It's not very Pythonic to randomly add members to an object. It would be more Pythonic if you used member methods to do it, but still not the way things are usually done.
Every library I've seen for this kind of thing uses dictionaries or lists. So that is the idiomatically Python way to handle the problem. Sometimes they use an object that overrides __getitem__ so it can behave like a dictionary or list, but it's still dictionary syntax that's used to access the fields.
I think all the pros and cons have to do with people understanding your code, and since I've never seen code that handles this by having an object with members that can appear I don't think many people will find code that does do that to be very understandable.
A list of dictionaries (e.g. [{"type": "text", "name": "field_name", "value": "test value"}, ...]) would be a usable structure, if I understand your requirement correctly.
Whether object are better in this case depends on what you're doing later. If you use the objects just as data storage, you don't gain anything. Maybe a list of field objects, which implement some appropriate methods to deal with your data, would also be a good choice.
maybe if you set up an object to use for each field and store those in a list, but that is practically ending up like a glorified dictionary
then you could access it like
fields[2].name
fields[2].value
ect
So I have a function which creates a dynamic model. I accomplish this in a way very similar to AuditTrail (see django wiki).
Sample of code is here:
https://gist.github.com/0212845ae00891efe555
Is there any way I can make a dynamically-generated class pickle-able? Ideally something thats not a crazy monkeypatch/hack?
I am aware of the problem where pickle can't store a generated or dynamic class. I solved this by rigging in my dynamic type into the modules dict like so:
new_class = type(name, (models.Model,), attrs)
mod = sys.modules[new_class.__module__]
mod.__dict__[new_class.__name__] = new_class
It's FAR from a clean or elegant solution, so if someone can think of a more django-friendly way to make this happen, I am all ears. However, the above code does work.
The reason there aren't answers for this is because the answer is likely hackish. I don't think you can unpickle an object in Python without knowing the structure of the class on the receiving end without some sort of hackish solution. A big reason pickle doesn't support it is probably because it's a fantastic way to introduce malicious code into your application.
http://www.mofeel.net/871-comp-lang-python/2898.aspx explains a bit why dynamically created classes can't be unpickled.
In every case, I've either just serialized a dictionary of the attributes of the object using the dict method, or just figured out some awful work around. I hope you come up with something better.
Good Luck!