I'm trying to create a model in SQLAlchemy, but I'm having a hard time finding out what is the best way. I currently have a class called returns, which I want to give an additional variable. A state of which the return is in. So for example, a return can be expected, received or processed. However, in the Flask application I want to show the user a nice string. For example; processed should become "Waiting for reimboursment".
The problem however is, I don't want to send these strings to the database, since I might change them in the future or add statusses. Therefore I want some kind of translation to be made between the value saved in the DB and the 'string' value. I have tried solving this by using Enums, but it is not possible to create the 'string' values. I would like something like this to return either the 'key' or the 'value', where only the key is saved in the database.
return.status.key
return.status.value
I have tried looking for a solution but was not able to find anything that seems to be fit.
What is the best practice for these kinds of requirements?
Related
Problem.
I have query let say like that:
qr = Session.query(models.User).filter(models.User.email == email)
What I want to do is to create qr2 with all filters dropped, in my case it would equivalent to:
qr2 = Session.query(models.User)
Good solution might be either, new query based on old one but without filters or modification of old query.
Why someone might need that.
Imagine you have a complex query that was build 3 modules and 4k lines away. What you get in your tiny function is query object (an instance). You have no clue which models are being selected or in fact what is the real payload of the query. All you know that you want to modify that query to not have any filters at all.
Suggestions/Solutions?
I am implementing a web application based on Google App Engine, Relying on ndb. I am facing a strange problem when trying to update one of the many entities in my db. When I try to update one of its properties (i.e. a string property) I get
"/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1715, in _validate (value,)) BadValueError: Expected string, got (u'a test',)
The same code works when I create a new entity of the same kind.
I know there will be soon someone asking me to add relevant code. It would be practically unreasonable to copy the javascript and python code I developed. I am interested to know if this behavior is known to occur in certain conditions I can check.
UPDATE
consider that the value with which I am trying to update the property is a post parameter (i.e. self.request.get('parameter'))
It looks to me like you are trying to assign a tuple to a model's ndb.StringProperty().
I'm trying to keep to SOLID object oriented programming principles, stay DRY, etc, but my newness to Python/SQLAlchemy/Pyramid is making it very hard.
I'm trying to take what I now know to be a SQLAlchemy model used to create a simple Pyramid Framework object and use what I know to be "reflection" in C#, it may be called something different in Python (Introspection? Not sure as this is only my second week with python but I have lots of experience in other languages (C/C++/C#,Java, etc) so the trouble seems to be mapping my knowledge to the vocabulary of python, sorry), to find out the field names of the database table, and most importantly, the current field values, when I do not know the column names or ANY of the shape of the object in advance.
Thats right; I don't know that the 'derp' instance has a field named id or name, just that it has columns and a value in each of them. And thats all I care about.
The goal is to be able to take any SQLAlchemy defined data model, and convert it to a dictionary of column_name -> column_value fields of simple data types of the kind found in JSON as I want to ultimately serialize any object I create in SQLAlchemy to a json object, but I will settle for a dictionary as from there its trivial as long as the dictionary holds the correct types of data. Doing this for every object by hand is a violation of too many good clean code rules and will create too much work over time; I could spend another week on this and still save time and effort by doing it the right way.
So if I have a class defined in SQLAlchemy as:
class SimpleFooModel(Base):
id = Column(Integer, primary_key=True, autoincrement=True, nullable=False)
name = Column(VARCHAR(length=12), nullable=False, index=True)
.. and I have an instance of this equal to (in python):
derp = SimpleFooModel(id=7, name="Foobar")
I want to be able to having ONLY the 'derp' instance variable described above, and NO OTHER KNOWLEDGE of how the model is shaped, and be able to flatten it out to a python key->value dictionary for that simple object, where every value in that dictionary can be serialized to JSON using import json from python syslib.
The problem is , I have been up for 2 days looking at this and I cant find an answer that gives me the results I want in my unit tests ANYWHERE; Google keeps taking me to really old posts here on SO about really old versions of the library that either use interfaces that no longer apply, or have accepted answers that do not actually work at all; and since none of them are recent that does surprise me (but why Stack Overflow keeps them when they are wrong and allows google to mislead people does surprise me)
I know I could wire every object manually for every object to json, etc, but thats not only NOT ELEGANT, its inefficient because it just creates more work for me as I create more objects and could lead to big bugs down the road. I want to know how to do this the correct way, with introspection/reflection, but nobody seems to know, and the people who claim to have all given examples here on stack overflow that actually do not work at all (at least with the current versions of things)
This seems like a really common use case for me; and getting the column field list and then iterating through it with getattr - like many of the answers say to do - doesn't work as expected either; it just creates what look like namespaces that never return the actual value of the column, and don't actually exist in any code as none of the fields created by sqlalchmy are singleton/static.
So:
from sqlalchemy.inspection import inspect
obj = inspect(derp, raiseerr=True)
for key in obj.attrs.keys():
fields[key] = getattr(derp, key)
print fields[key]
Just gives me:
[Class Name].[Column Name]
.. or in this case just gives me:
SimpleFooModel.id
SimpleFooModel.name
NOT the values of 7 and "Foobar" for id and name respectively, that I actually expected in my tests.
In fact it seems like I cant even find WHERE the values are being stored in the object model; or I could brute force the issue and get them from there as an ugly, evil hack I would be ashamed to look at. All I get through the "official public api" is a lot of objects that seem to have no clue where the real data is being stored, but will happily tell me the name of the path used by the column name and type, restrictions, etc... just not the actual data that I want.
Yet since my requirement is that I do not know the field names in advance, using a call to derp.id or derp.name to collect the value is not an option since that would violate SOLID and force me to duplicate work for every single class. So its not an option.
Maybe its the fact I have not slept in 2 days but its really hard for me to not see this as a serious design flaw in these libs; I just want to serialize a SQLAlchemy defined Model object representing a single row in a table into a python dictionary without having to know the names of the fields in advance, and while many other languages make this easy or even trivial, this seems to be far too hard than it should be.
Can somebody please explain either a working solution or why I am wrong to want to apply SOLID to my code?
EDIT: Updated spelling.
Extend your model with following class:
class BaseModel(object):
#classmethod
def _get_keys(cls):
return sa.orm.class_mapper(cls).c.keys()
def get_dict(self):
d = {}
for k in self._get_keys():
d[k] = getattr(self, k)
return d
This will do exactly what you want, return a dict in form of {'column_name':'value'} pairs.
I'm using subqueryload/subqueryload_all pretty heavily, and I've run into the edge case where I tend to need to very explicitly define the query that is used during the subqueryload. For example I have a situation where I have posts and comments. My query looks something like this:
posts_q = db.query(Post).options(subqueryload(Post.comments))
As you can see, I'm loading each Post's comments. The problem is that I don't want all of the posts' comments, I need to also take into account a deleted field, and they need to be ordered by create time descending. The only way I have observed this being done, is by adding options to the relationship() declaration between posts and comments. I would prefer not to do this, b/c it means that that relationship cannot be reused everywhere after that, as I have other places in the app where those constraints may not apply.
What I would love to do, is explicitly define the query that subqueryload/subqueryload_all uses to load the posts' comments. I read about DisjointedEagerLoading here, and it looks like I could simply define a special function that takes in the base query, and a query to load the specified relationship. Is this a good route to take for this situation? Anyone ever run into this edge case before?
The answer is that you can define multiple relationships between Posts and Comments:
class Post(...):
active_comments = relationship(Comment,
primary_join=and_(Comment.post_id==Post.post_id, Comment.deleted=False),
order_by=Comment.created.desc())
Then you should be able to subqueryload by that relationship:
posts_q = db.query(Post).options(subqueryload(Post.active_comments))
You can still use the existing .comments relationship elsewhere.
I also had this problem and it took my some time to realize that this is an issue by design. When you say Post.comments then you refer to the relationship that says "these are all the comments of that post". However, now you want to filter them. If you'd now specify that condition somewhere on subqueryload then you are essentially loading only a subset of values into Post.comments. Thus, there will be values missing. Essentially you have a faulty representation of your data in the model.
The question here is how to approach this then, because you obviously need this value somewhere. The way I go is building the subquery myself and then specify special conditions there. That means you get two objects back: The list of posts and the list of comments. That is not a pretty solution, but at least it is not displaying data in a wrong way. If you were to access Post.comments for some reason, you can safely assume it contains all posts.
But there is room for improvement: You might want to have this attached to your class so you don't carry around two variables. The easy way might be to define a second relationship, e.g. published_comments which specifies extra parameters. You could then also control that no-one writes to it, e.g. with attribute events. In these events you could, instead of forbidding manipulation, handle how manipulation is allowed. The only problem might be when updates happen, e.g. when you add a comment to Post.comments then published_comments won't be updated automatically because they are not aware of each other. Again, I'd take events for this if this is a required feature (but with the above ugly solution you would not have that either).
As a last, hybrid, solution you could take the first approach and then just assign those values to your object, e.g. Post.deleted_comments = deleted_comments.
The thing to keep in mind here is that it is generally not a clever idea to manipulate the query the ORM makes as this could lead to problems later on. I have taken this approach and manipulated the queries (with contains_eager this is easily possible) but it has created problems on some points (while generally being functional) so I dropped that approach.
I’m working on a Python script that accesses number of websites. I don’t think writing a whole class for each website is a feasible option, so I am looking for a way to reduce the code and re-use code as much as I can and write it as efficiently as I can. Although the website share same or similar engine (e.g. Wordpress), they often differ with slight details like cookie names, changed URL paths (e.g. example.com/signin.php and example2.com/login.php), data names and values for POST requests, and so on.
In most cases I could use variables, but sometimes I need to find extra data from retrieved url in the middle of method to continue, so that means adding few lines of code that needs to be processed before proceeding. I found that possible solution for me could theoretically be to use superclasses. But I couldn’t find any information how to actually ‘step into’ a method to make changes. To better illustrate my point, take a look at this example
class Wordpress(object):
def login(self):
# Do some stuff here
def newentry(self):
# Code to check if I'm logged in code
# and do bunch of other stuff
#
# Retrieve a page that *** sometimes contain important information
# in which case must be processed to continue! ***
data = {'name' : title,
'message' : message,
'icon' : iconid}
# Post new entry here
As I already commented in the example above in some cases I need to make adjustement just inside a method itself, add snippet of code, or change value of the variable or value in the dictionary, etc.. How can I achieve that, if it's even possible? Maybe my perception of superclasses is way off and they aren't really for what I think they are.
Maybe there's other way I haven't thought of. Feel free to post your own ideas how you would solve this problem. I am looking forward to your replies. Thank you.
Why can't you just call some process function from the main method, process can do whatever you want it to do, and you can even override it in derived classes e.g.
class WordPress(object):
def newentry(self):
data = get_data()
data = process_data(data)
# do moe generi cthings
def process_data(self):
return data