Develop Reference

Update one2many form view based on onchange

I assign for each project many tasks into task view form by using many2one field related project model. Then i create one2many field into project model to retrieve tasks directly into project View form, and into the code i'm looping from all the one2many fields for all tasks and append value into the one one2many field for displaying all tasks,the project_id_for_project is many2one field inside project model used to give the abaility of when i select one project it give all the attached tasks i hope you get my idea so i created the two model like below:
class project(models.Model):
_name = 'project.project'
_description = 'project.project'
project_id_for_project = fields.Many2one('project.project' )
Project_task_aff = fields.One2many('tasks.tasks','task_aff')
#api.onchange('project_id_for_project')
def getone2manyproject(self):
for rec in self:
lines = []
for line in rec.project_id_for_project.Project_task_aff :
val = {
# 'task_aff' : line.id ,
'task_actor' : line.task_actor,
'name' : line.name,
'DATE_tsk' : line.DATE_tsk,
'Description' : line.Description,
}
lines.append((0,0, val))
rec.Project_task_aff = lines
and the task model :
class tasks(models.Model):
_name = 'tasks.tasks'
_description = 'tasks.tasks'
_inherit = ['mail.thread', 'mail.activity.mixin']
task_actor = fields.Many2one('res.users', string='Chauffeur')
task_aff = fields.Many2one('project.project')
name = fields.Char()
DATE_tsk = fields.Date()
Description = fields.Char()
project_id = fields.Many2one('project.project')
the code give update the one2many field Project_task_aff but it dont give all the taks from task module: it mean that when i go into task model view, and create 5 entries related to a project, but when the onchange methode based of project_id_for_project field didn't give the 5 stored task but append just whose are stored into the onetomany field into the project view ?

You need to apply the relational field rules to update the value.
Rules:
(0, 0, { values }) link to a new record that needs to be created with the given values dictionary
(1, ID, { values }) update the linked record with id = ID (write values on it)
(2, ID) remove and delete the linked record with id = ID (calls unlink on ID, that will delete the object completely, and the link to it as well)
(3, ID) cut the link to the linked record with id = ID (delete the relationship between the two objects but does not delete the target object itself)
(4, ID) link to existing record with id = ID (adds a relationship)
(5) unlink all (like using (3,ID) for all linked records)
(6, 0, [IDs]) replace the list of linked IDs (like using (5) then (4,ID) for each ID in the list of IDs)
rec.Project_task_aff = [(6, 0, lines)]
I think this URL help with this problem and debugging.
https://raise360.odoocommunity.net/slides/slide/how-to-update-one2many-field-from-onchange-of-field-124

Try this way.
#api.onchange('project_id_for_project')
def getone2manyproject(self):
for rec in self:
tasks = self.env['tasks.tasks'].search([('task_aff', '=', rec.id)])
rec.Project_task_aff = [(6, 0, tasks.ids)]
Edit:
What is the purpose of project_id_for_project field in project's model for child parent relationship?

May be this will help, I just retrieve the first line browse loop self, and replace the tuple (0, 0, val) with (6, 0, val) in last line
#api.onchange('project_id_for_project')
def getone2manyproject(self):
lines = []
for line in self.project_id_for_project.Project_task_aff :
val = {
# 'task_aff' : line.id ,
'task_actor' : line.task_actor.id,
'name' : line.name,
'DATE_tsk' : line.DATE_tsk,
'Description' : line.Description,
}
lines.append((6, 0, val))
self.Project_task_aff = lines

sqlalchemy orm | fastAPI querying + joining three tables to get parents and all children

I'm trying to make have a route in my fastAPI that gives back a list of all parents.portfolios and all the children or stocks that are associated with each of them PLUS the extra data that is in the association table (for that relationship).
The response is suppose to look somewhat like this
[ { "parrent1_attr1": bla,
"parrent1_attr2": bla,
"children": [ {
"child1_attr1": bla,
"child1_attr2": bla},
{"child2_attr1": bla,
"child2_attr2": bla}]
},
etc...]
Right now the route that produces this looks like this:
#router.get("/")
def get_all_portfolios(db: Session = Depends(get_db), current_user: int = Depends(oauth2.get_current_user)):
results = db.query(models.Portfolio).options(joinedload(models.Portfolio.stocks)).all()
return results
But this gives me the wrong result.
This results in this.
[ { "parrent1_attr1": bla,
"parrent1_attr2": bla,
"children": [ {
"association_table_attr1": bla
"association_table_attr2": bla},]
So I get data from the association table back instead of from the children.
The models I have are here.
class Portfolio(Base):
__tablename__ = "portfolios"
id = Column(Integer, primary_key=True, nullable=False)
...
stocks = relationship("PortfolioStock", back_populates="portfolio")
class Stock(Base):
__tablename__ = "stocks"
id = Column(Integer, primary_key=True, nullable=False)
...
portfolios = relationship("PortfolioStock", back_populates="stock")
class PortfolioStock(Base):
__tablename__ = "portfolio_stocks"
id = Column(Integer, primary_key=True)
stock_id = Column(Integer, ForeignKey("stocks.id", ondelete="CASCADE"))
portfolio_id = Column(Integer, ForeignKey("portfolios.id", ondelete="CASCADE"))
count = Column(Integer, nullable=True)
buy_in = Column(Float, nullable=True)
stock = relationship("Stock", back_populates="portfolios")
portfolio = relationship("Portfolio", back_populates="stocks")
Let me know if you need more information. I appreciate your help.

I find it to be easier to give the association some name of its own because it is confusing but in this case Portfolio.stocks is actually a list of the association objects and NOT actual stocks. You have to get those off the association object. In my example below I go and get stock with assoc.stock.id. That should not trigger another query because we used joinedload to pre-load it. If the stock had a name we'd reference it with assoc.stock.name.
with Session(engine) as session:
q = session.query(Portfolio).options(joinedload(Portfolio.stocks).joinedload(PortfolioStock.stock))
for portfolio in q.all():
print (f"Listing associated stocks for portfolio {portfolio.id}")
for assoc in portfolio.stocks:
print (f" Buy in {assoc.buy_in}, count {assoc.count} and stock id {assoc.stock.id}")
The query looks something like this:
SELECT portfolios.id AS portfolios_id, stocks_1.id AS stocks_1_id, portfolio_stocks_1.id AS portfolio_stocks_1_id, portfolio_stocks_1.stock_id AS portfolio_stocks_1_stock_id, portfolio_stocks_1.portfolio_id AS portfolio_stocks_1_portfolio_id, portfolio_stocks_1.count AS portfolio_stocks_1_count, portfolio_stocks_1.buy_in AS portfolio_stocks_1_buy_in
FROM portfolios LEFT OUTER JOIN portfolio_stocks AS portfolio_stocks_1 ON portfolios.id = portfolio_stocks_1.portfolio_id LEFT OUTER JOIN stocks AS stocks_1 ON stocks_1.id = portfolio_stocks_1.stock_id

For anybody that is looking for an answer, here is how I fixed it.
I used the query from Ian that he mentioned above, (thank you a ton for that).
And then I just manually declared the structure I wanted to have.
The whole code looks like this
results = (
db.query(models.Portfolio)
.options(joinedload(models.Portfolio.stocks).joinedload(models.PortfolioStock.stock))
.all()
)
result_list = []
for portfolio in results:
result_dict = portfolio.__dict__
stock_list = []
for sto in result_dict["stocks"]:
sto_dict = sto.__dict__
temp_sto = {}
temp_sto = sto_dict["stock"]
setattr(temp_sto, "buy_in", sto_dict["buy_in"])
setattr(temp_sto, "count", sto_dict["count"])
stock_list.append(temp_sto)
result_dict["stocks"] = stock_list
result_list.append(result_dict)
return result_list
What I'm doing here is firstly declare an empty list where our final results will be stored and which will be returned.
Then we iterate over the query (because the query gives us a list back).
So we have each "SQL alchemy Model" now as portfolio in there.
Then we can convert this into a dictionary and assign it a new variable with result_dict = portfolio.__dict__ The __dict__ method converts the model into a Python dictionary that you can work with easily.
Since that result_dict contains a list of PortfolioStock models which is the association table model in. These are stored in the stocks key we have to iterate over them to get those values as well. And here we just repeat the process.
We convert the model into a dictionary with __dict__ then make a new empty dictionary temp_sto={} and set it equal to the stock key which is the key that is linking the child to our association table. So now we have the child or stock we want to access. We can simply set our new empty dicitonary equal to that so we inherit all the information contained within.
And then we just have to add all other information from the association table that we might want which can be accessed via the dictionary we defined at the beginning of the for loop sto_dict.
Once we have this we append it to an empty list we have defined outside of this for loop but inside the portfolio loop.
Set the result_dict["stocks"] key (so basically the key where you want all children to be contained in) equal to that list we just appended all the dictionaries to and then append that dictionary to our result_list.
And last thing to do is just to return that list, and we're done.
I have provided an agnostic approach hopefully down below
query = db.query(Parent).options(joinedload(Parent.relationship).joinedload(AssociationTable.child).all()
result_list = []
for parent in query:
parent_dict = parent.__dict__
child_list = []
for association in parent_dict["relationship_name"]:
association_dict = association.__dict__
temp_child_dict = {}
temp_child_dict = association_dict["child"]
setattr(temp_child_dict, "name_you_want", association_dict["key_of_value_you_want"])
# repeat as many times as you need
child_list.append(temp_child_dict)
parent_dict["children"] = child_list
result_list.append(parent_dict)
return result_list
I hope this helps you if you are in a similar situation.

Filter elements by optional lists of related elements

I have a feeling that I've made things more complex than they need to be - this can't possibly be such a rare case. It seems to me that it should be possible - or perhaps that I'm doing something fundamentally wrong.
The issue at hand is this: I've declared a database element, Element, which consists of about 10 many-to-many relations with other elements, one of which is Tag.
I want to enable the user of my application to filter Element by all of these relations, some of them or none of them. Say the user wants to see only Elements which are related to a certain Tag.
To make things even more difficult, the function that will carry out this objective is called from a graphql API, meaning it will recieve ID's instead of ORM objects.
I'm trying to build a resolver in my Python Flask project, using SQLAlchemy, which will provide an interface like so:
# graphql request
query getElements {
getElements(tags:[2, 3] people:[8, 13]) {
id
}
}
# graphql response
{
"data": {
"getElements": [
{
"id": "2"
},
{
"id": "3"
},
{
"id": "8"
}
]
}
}
I imagine the resolver would look something like this simplified pseudo-code, but I can't for the life of me figure out how to pull it off:
def get_elements(tags=None, people=None):
args = {'tags' : tags, 'people' : people}
if any(args):
data_elements = DataElement.query.filter_by(this in args) # this is the tricky bit - for each of DataElements related elements, I want to check if its ID is given in the corresponding argument
else:
data_elements = DataElement.query.all()
return data_elements
Here's a peek at the simplified database model, as requested. DataElement holds a lot of relations like this, and it works perfectly:
class DataElement(db.Model):
__tablename__ = 'DataElement'
id = db.Column(db.Integer, primary_key=True)
tags = db.relationship('Tag', secondary=DataElementTag, back_populates='data_elements')
class Tag(db.Model):
__tablename__ = 'Tag'
id = db.Column(db.Integer, primary_key=True)
data_elements = db.relationship('DataElement', secondary=DataElementTag, back_populates='tags')
DataElementTag = db.Table('DataElementTag',
db.Column('id', db.Integer, primary_key=True),
db.Column('data_element_id', db.Integer, db.ForeignKey('DataElement.id')),
db.Column('tag_id', db.Integer, db.ForeignKey('Tag.id'))
)
Please, ORM wizards and python freaks, I call upon thee!

I've solved it in a rather clunky manner. I suppose there must be a more elegant way to pull this off, and am still holding out for better answers.
I ended up looping over all the given arguments and using eval() (not on user input, don't worry) to get the corresponding database model. From there, I was able to grab the DataElement object with the many-to-many relationship. My final solutions looks like this:
args = {
'status' : status,
'person' : people,
'tag' : tags,
'event' : events,
'location' : locations,
'group' : groups,
'year' : year
} # dictionary for args for easier data handling
if any(args.values()):
final = [] # will contain elements matching criteria
for key, value in args.items():
if value:
model = eval(key.capitalize()) # get ORM model from dictionary key name (eval used on hardcoded string, hence safe)
for id in value:
filter_element = model.query.filter_by(id=id).one_or_none() # get the element in question from db
if filter_element:
elements = filter_element.data_elements # get data_elements linked to element in question
for element in elements:
if not element in final: # to avoid duplicates
final.append(element)
return final

Updating a MongoDB document if field doesn't exist

Whenever I updated my insert_one with a new field to use, I had to always delete the old posts in the collection. I know there are manual methods of updating such fields using update_many but I know it's inefficient.
For example:
posts.insert_one({
"id": random.randint(1,10000)
"value1": "value1",
"value2": "value2"
})
I use the following code to check if the document exists or not. How would this work for a field?
if posts.find({'id': 12312}).count() > 0:
I know I can easily overwrite the previous data but I know people won't enjoy having their data wiped every other month.
Is there a way to add the field to a document in Python?

How would this work for a field?
You can use $exists to check whether a field exists in a doc.
In your case, you can combine this with find
find({ 'id':1, "fieldToCheck":{$exists:"true"}})
It will return the doc if it exists with id = 1, fieldToCheck is present in doc with id = 1
You can skip id=1, in that case, it will return all docs where fieldToCheck exists
Is there a way to add the field to a document in Python?
You could use update with new field, it will update if it is present else it will insert.
update({"_id":1}, {field:"x"})
If field is present, it will set to x else it will add with field:x
Beware of update options like multi, upsert

Yes you can you use update command in mongoDB shell to do that. check here
This is the command to use...
db.collection.update({},{$set : {"newfield":1}},false,true)
The above will work in the mongoDB shell. It will add newfield in all the documents, if it is not present.
If you want to use Python, use pymongo.
For python, following command should work
db.collection.update({},{"$set" : {"newfield":1}},False, True)

Thanks to john's answer I have made an entire solution that automatically updates documents without the need to run a task meaning you don't update inactive documents.
import datetime
import pymongo
database = pymongo.MongoClient("mongodb://localhost:27017") # Mongodb connection
db = database.maindb # Database
posts = db.items # Collection within a database
# A schema equivalent function that returns the object
def user_details(name, dob):
return {
"username": name, # a username/id
"dob": dob, # some data
"level": 0, # some other data
"latest_update": datetime.datetime.fromtimestamp(1615640176)
# Must be kept to ensure you aren't doing it that often
}
# The first schema changed for example after adding a new feature
def user_details2(name, dob, cake):
return {
"username": name, # a username/id
"dob": dob, # Some data
"level": 0, # Some other data
"cake": cake, # Some new data that isn't in the document
"latest_update": datetime.datetime.utcnow() # Must be kept to ensure you aren't doing it that often
}
def check_if_update(find, main_document,
collection): # parameters: What you find a document with, the schema dictionary, then the mongodb collection
if collection.count_documents(find) > 0: # How many documents match, only proceed if it exists
fields = {} # Init a dictionary
for x in collection.find(find): # You only want one for this to work
fields = x
if "latest_update" in fields: # Just in case it doesn't exist yet
last_time = fields["latest_update"] # Get the time that it was last updated
time_diff = datetime.datetime.utcnow() - last_time # Get the time difference between the utc time now and the time it was last updated
if time_diff.total_seconds() < 3600: # If the total seconds of the difference is smaller than an hour
print("return")
return
db_schema = main_document # Better naming
db_schema["_id"] = 0 # Adds the _id schema_key into the dictionary
if db_schema.keys() != fields:
print("in")
for schema_key, schema_value in db_schema.items():
if schema_key not in fields.keys(): # Main key for example if cake is added and doesn't exist in db fetched fields
collection.update_one(find, {"$set": {schema_key: schema_value}})
else: # Everything exists and you want to check for if a dictionary within that dictionary is changed
try:
sub_dict = dict(schema_value) # Make the value of it a dictionary
# It exists in the schema dictionary but not in the db fetched document
for key2, value2 in sub_dict.items():
if key2 not in fields[schema_key].keys():
new_value = schema_value
new_value[
key2] = value2 # Adding the key and value from the schema dictionary that was added
collection.update_one(find,
{"$set": {schema_key: new_value}})
# It exists in the db fetched document but not in the schema dictionary
for key2, value2 in fields[schema_key].items():
if key2 not in sub_dict.keys():
new_dict = {} # Get all values, filter then so that only the schema existent ones are passed back
for item in sub_dict:
if item != key2:
new_dict[item] = sub_dict.get(item)
collection.update_one(find, {"$set": {schema_key: new_dict}})
except: # Wasn't a dict
pass
# You removed a value from the schema dictionary and want to update it in the db
for key2, value2 in fields.items():
if key2 not in db_schema:
collection.update_one(find, {"$unset": {key2: 1}})
else:
collection.insert_one(main_document) # Insert it because it doesn't exist yet
print("start")
print(posts.find_one({"username": "john"}))
check_if_update({"username": "john"}, user_details("john", "13/03/2021"), posts)
print("inserted")
print(posts.find_one({"username": "john"}))
check_if_update({"username": "john"}, user_details2("john", "13/03/2021", "Lemon drizzle"), posts)
print("Results:")
print(posts.find_one({"username": "john"}))
It is available as a gist

How do I look get an associated value in a json variable using python?

How do I look up the 'id' associated with the a person's 'name' when the 2 are in a dictionary?
user = 'PersonA'
id = ? #How do I retrieve the 'id' from the user_stream json variable?
json, stored in a variable named "user_stream"
[
{
'name': 'PersonA',
'id': '135963'
},
{
'name': 'PersonB',
'id': '152265'
},
]

You'll have to decode the JSON structure and loop through all the dictionaries until you find a match:
for person in json.loads(user_stream):
if person['name'] == user:
id = person['id']
break
else:
# The else branch is only ever reached if no match was found
raise ValueError('No such person')
If you need to make multiple lookups, you probably want to transform this structure to a dict to ease lookups:
name_to_id = {p['name']: p['id'] for p in json.loads(user_stream)}
then look up the id directly:
id = name_to_id.get(name) # if name is not found, id will be None
The above example assumes that names are unique, if they are not, use:
from collections import defaultdict
name_to_id = defaultdict(list)
for person in json.loads(user_stream):
name_to_id[person['name']).append(person['id'])
# lookup
ids = name_to_id.get(name, []) # list of ids, defaults to empty
This is as always a trade-off, you trade memory for speed.

Martijn Pieters's solution is correct, but if you intend to make many such look-ups it's better to load the json and iterate over it just once, and not for every look-up.
name_id = {}
for person in json.loads(user_stream):
name = person['name']
id = person['id']
name_id[name] = id
user = 'PersonA'
print name_id[user]

persons = json.loads(...)
results = filter(lambda p:p['name'] == 'avi',persons)
if results:
id = results[0]["id"]
results can be more than 1 of course..