Problem: I get a ValidationError when trying to perform a .save() when appending a value to an EmbeddedDocumentListField because I am missing required fields that already exist on the document.
Note that at this point the User document has already been created as part of the signup process, so it already has an email and password in the DB.
My classes:
class User(gj.Document):
email = db.EmailField(required=True, unique=True)
password = db.StringField(required=True)
long_list_of_thing_1s = db.EmbeddedDocumentListField("Thing1")
long_list_of_thing_2s = db.EmbeddedDocumentListField("Thing2")
class Thing1(gj.EmbeddedDocument):
some_string = db.StringField()
class Thing2(gj.EmbeddedDocument):
some_string = db.StringField()
Trying to append a new EmbeddedDocument to the EmbeddedDocumentListField in my User class in the Thing2 Resource endpoint:
class Thing2(Resource):
def post(self):
try:
body = request.get_json()
user_id = body["user_id"]
user = UserModel.objects.only("long_list_of_thing_2s").get(id=user_id)
some_string = body["some_string"]
new_thing_2 = Thing2Model()
new_thing_2.some_string = some_string
user.long_list_of_thing_2s.append(new_thing_2)
user.save()
return 201
except Exception as exception:
raise InternalServerError
On hitting this endpoint I get the following error on the user.save()
mongoengine.errors.ValidationError: ValidationError (User:603e39e7097f3e9a6829f422) (Field is required: ['email', 'password'])
I think this is because of the .only("long_list_of_thing_2s")
But I am specifically using UserModel.objects.only("long_list_of_thing_2s") because I don't want to be inefficient in bringing the entire UserModel into memory when I only want to append something the long_list_of_thing_2s
Is there a different way I should be going about this? I am relatively new to Flask and Mongoengine so I am not sure what all the best practices are when going about this process.
You are correct, this is due to the .only and is a known "bug" in MongoEngine.
Unless your Model is really large, using .only() will not make a big difference so I'd recommend to use it only if you observe performance issues.
If you do have to keep the .only() for whatever reason, you should be able to make use of the push atomic operator. An advantage of using the push operator is that in case of race conditions (concurrent requests), it will gracefully deal with the different updates, this is not the case with regular .save() which will overwrite the list.
I am trying to add some data into table made in Django.
Here is my code:
class Prefixe(models.Model):
remote_as = models.CharField(max_length=250)
prefix_list = models.CharField(max_length=250, unique=True)
def insert():
p = Prefixe(remote_as='Apress', prefix_list='Berkeley')
p.save()
I'm creating records using ORM and it fails with "duplicates" exception (see unique option in prefix_list definition).
But there is a way I can avoid this exception if I make insert as raw SQL with specific PG option:
INSERT INTO "peer_table" ("remote_as","prefix_list")
VALUES('{}','{}')
ON CONFLICT DO NOTHING
Can I avoid this exception caused by duplicates when working via ORM?
Your code could benefit from the use of the method .get_or_create():
def insert(remote_as, prefix_list):
p, was_created = Prefixe.objects.get_or_create(
remote_as=remote_as, prefix_list=prefix_list)
if was_create:
print('a new instance was created.')
else:
print('the instance already exists.')
I have this small project to create my bills through Django and Latex which worked flawlessly until today. Now when I try to add another costumer, Django throws
duplicate key value violates unique constraint "kunden_kundearbeitsamt_pkey"
DETAIL: Key (id)=(4) already exists.
These are the model definitions in question:
class Kunde(models.Model):
name = models.CharField('Name', max_length = 200)
vorname = models.CharField('Vorname', max_length = 200)
geburtsdatum = models.DateField('Geburtsdatum', max_length = 200)
untersuchungsdatum = models.DateField('Untersuchungsdatum', max_length = 200)
class Meta:
abstract = True
class KundeArbeitsamt(Kunde):
kundennummer = models.CharField('Kundennummer', max_length = 100)
bglnummer = models.CharField('BGL-Nummer', max_length = 100)
empfaenger = models.ForeignKey('rechnungen.NumberToEmpfaenger', blank = True, null = True)
class Meta:
verbose_name = "Proband Arbeitsamt"
verbose_name_plural = "Proband Arbeitsamt"
def __str__(self):
return '{}, {}'.format(self.name, self.vorname)
The admin part where the object is created (nothing special, I guess):
from django.contrib import admin
from .models import KundeArbeitsamt
class KundeArbeitsamtAdmin(admin.ModelAdmin):
ordering = ('name',)
admin.site.register(KundeArbeitsamt, KundeArbeitsamtAdmin)
I swear, I did not make any migrations or other changes to the database (Postgres) whatsoever. Django is handling the creation of the objects. What is causing this error and how to fix it?
This error is raised by your database, because django wants to add an new column with an ID (=4) already in use.
To investigate further you need to find the part of your app responsible for creating the IDs. Django usually delegates this task to your database. In case of postgres the datatype serial is used. Postgres uses so called sequences for this purpose and generates and executes the following SQL for you:
CREATE SEQUENCE tablename_colname_seq;
CREATE TABLE tablename (
colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
);
ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
I would now start with checking the database sanity like that:
-- views contents of the table
SELECT * FROM kunden_kundearbeitsamt;
-- check the sequence
select currval('kunden_kundearbeitsamt_id_seq');
If the first shows 4 records with IDs 1, 2, 3 and 4 and the sequence answers with 4 everything is alright. I would proceed with the django sources to figure out why they pass an ID on object creating without relying on the sequence. The django shell might be a good place to start with in that case.
Otherwise I would fix the sequence and ask myself how this happend as it is barely the case that postgres makes mistakes at this point.
SELECT setval('kunden_kundearbeitsamt_id_seq', (SELECT max(id) FROM kunden_kundearbeitsamt));
if is_admin == True:
admin_users = Group(name = 'Admin')
try:
admin_users.save()
except:
log.info("Admin Group already exists")
pass
group_id = Group.objects.get(name='Admin').id
If in the data that I get 'is_admin' is true then I will create the group 'Admin' if not existed then save it and fetches the id of that group-'Admin'. This id will be saved in the userinfo with Group as a foreign key.
The following query should give me the id of that group.
group_id = Group.objects.get(name='admin').id
Instead it is saying
current transaction is aborted, commands ignored until end of transaction block
I am using the postgresql database I don't know why it is giving me the error while executing this query. Please tell me how to write the query.
What are you trying to achieve is already in django: get_or_create. Using this your code should look like
group, created = Group.objects.get_or_create(name='Admin')
if created:
log.info("Admin Group already exists")
group_id = group.pk
pk is a convenience property on all django models that always point to a primary key field, no matter if it's autocreated or specified explicitly.
I have written code for managing conditional insert/update/delete to
multiple tables from single form in 'web2py'.
I agree, the code is in very raw form & may not be ‘pythonic’.
There are code repeatitions.
But at least I have something to go ahead & build a refined
structure.
MODELS:
db.define_table('mdlmst',
Field('mdlmstid','id'),
Field('mdlmstcd'),
Field('mdlmstnm'),
migrate=False,
format='%(mdlmstnm)s'
)
db.define_table('wrmst',
Field('wrmstid','id'),
Field('wrmstcd'),
Field('wrmstnm'),
migrate=False,
format='%(wrmstnm)s'
)
db.define_table('extwrmst',
Field('extwrmstid','id'),
Field('extwrmstcd'),
Field('extwrmstnm'),
migrate=False,
format='%(extwrmstnm)s'
)
from the FORM, data will be populated in the following two tables
db.define_table('mdlwr',
Field('mdlwrid','id'),
Field('mdlmstid',db.mdlmst),
Field('wrmstid',db.wrmst),
migrate=False
)
db.define_table('mdlextwr',
Field('mdlextwrid','id'),
Field('mdlmstid',db.mdlmst),
Field('extwrmstid',db.extwrmst),
migrate=False
)
CONTROLLERS:
‘modelwar’ controller will render the records from ‘mdlmst’ table
def modelwar():
models = db(db.mdlmst.mdlmstid>0).select(orderby=db.mdlmst.mdlmstnm)
return dict(models=models)
after clicking a particular record, ‘war_edit’ controller will
manage the tables – ‘mdlwr’ & ‘mdlextwr’
def war_edit():
mdl_id = request.args(0)
mdl_id is a variable identifying the ‘mdlmstid’ (which record to be modified)
mdl_nm = request.args(1)
mdl_nm is a variable for getting the ‘mdlmstnm’
warset = db(db.mdlwr.mdlmstid==mdl_id) # fetch a set
extwarset = db(db.mdlextwr.mdlmstid==mdl_id) # fetch a set
warlist = db(db.mdlwr.mdlmstid==mdl_id).select() # get a ROW object
extwarlist = db(db.mdlextwr.mdlmstid==mdl_id).select() # get a ROW object
form_war=FORM(TABLE(TR("Basic Warranty",
SELECT(_type="select",_name="baswar",*[OPTION(x.wrmstnm,_value=x.wrmstid)
for x in db().select(db.wrmst.ALL)]),
TR("Extended Warranty",
SELECT(_type="select",_name="extwar",*[OPTION(x.extwrmstnm,_value=x.extwrmstid)
for x in db().select(db.extwrmst.ALL)]),
TR("", INPUT(_type='submit',_value='Save')), ))))
pre-populate the fields in‘form_war’
if len(warlist)>0:
form_war.vars.baswar = warlist[0].wrmstid
if len(extwarlist)>0:
form_war.vars.extwar = extwarlist[0].extwrmstid
after successful form submission, manage the table 'mdlwr'
if form_war.accepts(request.vars, session):
if there was any record in the list fetched from database & sent to FORM,
if len(warlist)>0:
delete if value returned from FORM field is blank, else update
if form_war.vars.baswar==''
warset.delete()
else:
warset.update(wrmstid=form_war.vars.baswar)
else insert
else:
db.mdlwr.insert(mdlmstid=mdl_id, wrmstid=form_war.vars.baswar)
Similarly, manage the table 'mdlextwr'
if len(extwarlist)>0:
if form_war.vars.extwar=='':
extwarset.delete()
else:
extwarset.update(extwrmstid=form_war.vars.extwar)
else:
db.mdlextwr.insert(mdlmstid=mdl_id, extwrmstid=form_war.vars.extwar)
response.flash = 'Warranty definition saved'
return dict(form_war=form_war,mdlnm=mdl_nm)
VIEW for 'mdlmst' table
{{response.files.append(URL(r=request,c='static',f='jquery.dataTables.min.j
s'))}}
{{response.files.append(URL(r=request,c='static',f='demo_table.css'))}}
{{extend 'layout.html'}}
jQuery(document).ready(function()
{ jQuery('.smarttable').dataTable();});
Modelwise Warranty Master
Model IDModel CodeModel Name
{{for model in models:}}
{{=model.mdlmstid}}
{{=model.mdlmstcd}}
{{=model.mdlmstnm}}
{{=A('edit
warranty',_href=URL('war_edit',args=[model.mdlmstid,model.mdlmstnm]))}}
{{pass}}
Pl. tell me if I have coded anything stupid here.
I would highly welcome any ideas/suggestions for improvements.
Thanks,
Vineet
Your database design looks strange to me.
In each table you have a field of type 'id'. This will replace the id field automatically generated by web2py - a bad idea. From the web2py book: "Do not declare a field called "id", because one is created by web2py anyway. Every table has a field called "id" by default. It is an auto-increment integer field (starting at 1) used for cross-reference and for making every record unique, so "id" is a primary key"
You have created a many to many relationship between table 'mdlmst' and 'wrmst' and another many to many relationship between 'mdlmst' and 'extwrmst'. While this is not necessarily wrong, it strikes me as extremely unlikely this is what you want.
My feeling is that your database design needs work. This should be sorted out before you start designing forms.