I am trying to add some data into table made in Django.
Here is my code:
class Prefixe(models.Model):
remote_as = models.CharField(max_length=250)
prefix_list = models.CharField(max_length=250, unique=True)
def insert():
p = Prefixe(remote_as='Apress', prefix_list='Berkeley')
p.save()
I'm creating records using ORM and it fails with "duplicates" exception (see unique option in prefix_list definition).
But there is a way I can avoid this exception if I make insert as raw SQL with specific PG option:
INSERT INTO "peer_table" ("remote_as","prefix_list")
VALUES('{}','{}')
ON CONFLICT DO NOTHING
Can I avoid this exception caused by duplicates when working via ORM?
Your code could benefit from the use of the method .get_or_create():
def insert(remote_as, prefix_list):
p, was_created = Prefixe.objects.get_or_create(
remote_as=remote_as, prefix_list=prefix_list)
if was_create:
print('a new instance was created.')
else:
print('the instance already exists.')
Related
Problem: I get a ValidationError when trying to perform a .save() when appending a value to an EmbeddedDocumentListField because I am missing required fields that already exist on the document.
Note that at this point the User document has already been created as part of the signup process, so it already has an email and password in the DB.
My classes:
class User(gj.Document):
email = db.EmailField(required=True, unique=True)
password = db.StringField(required=True)
long_list_of_thing_1s = db.EmbeddedDocumentListField("Thing1")
long_list_of_thing_2s = db.EmbeddedDocumentListField("Thing2")
class Thing1(gj.EmbeddedDocument):
some_string = db.StringField()
class Thing2(gj.EmbeddedDocument):
some_string = db.StringField()
Trying to append a new EmbeddedDocument to the EmbeddedDocumentListField in my User class in the Thing2 Resource endpoint:
class Thing2(Resource):
def post(self):
try:
body = request.get_json()
user_id = body["user_id"]
user = UserModel.objects.only("long_list_of_thing_2s").get(id=user_id)
some_string = body["some_string"]
new_thing_2 = Thing2Model()
new_thing_2.some_string = some_string
user.long_list_of_thing_2s.append(new_thing_2)
user.save()
return 201
except Exception as exception:
raise InternalServerError
On hitting this endpoint I get the following error on the user.save()
mongoengine.errors.ValidationError: ValidationError (User:603e39e7097f3e9a6829f422) (Field is required: ['email', 'password'])
I think this is because of the .only("long_list_of_thing_2s")
But I am specifically using UserModel.objects.only("long_list_of_thing_2s") because I don't want to be inefficient in bringing the entire UserModel into memory when I only want to append something the long_list_of_thing_2s
Is there a different way I should be going about this? I am relatively new to Flask and Mongoengine so I am not sure what all the best practices are when going about this process.
You are correct, this is due to the .only and is a known "bug" in MongoEngine.
Unless your Model is really large, using .only() will not make a big difference so I'd recommend to use it only if you observe performance issues.
If you do have to keep the .only() for whatever reason, you should be able to make use of the push atomic operator. An advantage of using the push operator is that in case of race conditions (concurrent requests), it will gracefully deal with the different updates, this is not the case with regular .save() which will overwrite the list.
Despite numerous recipes and examples in peewee's documentation; I have not been able to find how to accomplish the following:
For finer-grained control, check out the Using context manager / decorator. This allows you to specify the database to use with a given list of models for the duration of the wrapped block.
I assume it would go something like...
db = MySQLDatabase(None)
class BaseModelThing(Model):
class Meta:
database = db
class SubModelThing(BaseModelThing):
'''imagine all the fields'''
class Meta:
db_table = 'table_name'
runtime_db = MySQLDatabase('database_name.db', fields={'''imagine field mappings here''', **extra_stuff)
#Using(runtime_db, [SubModelThing])
#runtime_db.execution_context()
def some_kind_of_query():
'''imagine the queries here'''
but I have not found examples, so an example would be the answer to this question.
Yeah, there's not a great example of using Using or the execution_context decorators, so the first thing is: don't use the two together. It doesn't appear to break anything, just seems to be redundant. Logically that makes sense as both of the decorators cause the specified model calls in the block to run in a single connection/transaction.
The only(/biggest) difference between the two is that Using allows you to specify the particular database that the connection will be using - useful for master/slave (though the Read slaves extension is probably a cleaner solution).
If you run with two databases and try using execution_context on the 'second' database (in your example, runtime_db) nothing will happen with the data. A connection will be opened at the start of the block and closed and the end, but no queries will be executed on it because the models are still using their original database.
The code below is an example. Every run should result in only 1 row being added to each database.
from peewee import *
db = SqliteDatabase('other_db')
db.connect()
runtime_db = SqliteDatabase('cmp_v0.db')
runtime_db.connect()
class BaseModelThing(Model):
class Meta:
database = db
class SubModelThing(Model):
first_name = CharField()
class Meta:
db_table = 'table_name'
db.create_tables([SubModelThing], safe=True)
SubModelThing.delete().where(True).execute() # Cleaning out previous runs
with Using(runtime_db, [SubModelThing]):
runtime_db.create_tables([SubModelThing], safe=True)
SubModelThing.delete().where(True).execute()
#Using(runtime_db, [SubModelThing], with_transaction=True)
def execute_in_runtime(throw):
SubModelThing(first_name='asdfasdfasdf').save()
if throw: # to demo transaction handling in Using
raise Exception()
# Create an instance in the 'normal' database
SubModelThing.create(first_name='name')
try: # Try to create but throw during the transaction
execute_in_runtime(throw=True)
except:
pass # Failure is expected, no row should be added
execute_in_runtime(throw=False) # Create a row in the runtime_db
print 'db row count: {}'.format(len(SubModelThing.select()))
with Using(runtime_db, [SubModelThing]):
print 'Runtime DB count: {}'.format(len(SubModelThing.select()))
I have this small project to create my bills through Django and Latex which worked flawlessly until today. Now when I try to add another costumer, Django throws
duplicate key value violates unique constraint "kunden_kundearbeitsamt_pkey"
DETAIL: Key (id)=(4) already exists.
These are the model definitions in question:
class Kunde(models.Model):
name = models.CharField('Name', max_length = 200)
vorname = models.CharField('Vorname', max_length = 200)
geburtsdatum = models.DateField('Geburtsdatum', max_length = 200)
untersuchungsdatum = models.DateField('Untersuchungsdatum', max_length = 200)
class Meta:
abstract = True
class KundeArbeitsamt(Kunde):
kundennummer = models.CharField('Kundennummer', max_length = 100)
bglnummer = models.CharField('BGL-Nummer', max_length = 100)
empfaenger = models.ForeignKey('rechnungen.NumberToEmpfaenger', blank = True, null = True)
class Meta:
verbose_name = "Proband Arbeitsamt"
verbose_name_plural = "Proband Arbeitsamt"
def __str__(self):
return '{}, {}'.format(self.name, self.vorname)
The admin part where the object is created (nothing special, I guess):
from django.contrib import admin
from .models import KundeArbeitsamt
class KundeArbeitsamtAdmin(admin.ModelAdmin):
ordering = ('name',)
admin.site.register(KundeArbeitsamt, KundeArbeitsamtAdmin)
I swear, I did not make any migrations or other changes to the database (Postgres) whatsoever. Django is handling the creation of the objects. What is causing this error and how to fix it?
This error is raised by your database, because django wants to add an new column with an ID (=4) already in use.
To investigate further you need to find the part of your app responsible for creating the IDs. Django usually delegates this task to your database. In case of postgres the datatype serial is used. Postgres uses so called sequences for this purpose and generates and executes the following SQL for you:
CREATE SEQUENCE tablename_colname_seq;
CREATE TABLE tablename (
colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
);
ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
I would now start with checking the database sanity like that:
-- views contents of the table
SELECT * FROM kunden_kundearbeitsamt;
-- check the sequence
select currval('kunden_kundearbeitsamt_id_seq');
If the first shows 4 records with IDs 1, 2, 3 and 4 and the sequence answers with 4 everything is alright. I would proceed with the django sources to figure out why they pass an ID on object creating without relying on the sequence. The django shell might be a good place to start with in that case.
Otherwise I would fix the sequence and ask myself how this happend as it is barely the case that postgres makes mistakes at this point.
SELECT setval('kunden_kundearbeitsamt_id_seq', (SELECT max(id) FROM kunden_kundearbeitsamt));
I wrote a view to update my draft object , before updating my draft I need to see if any draft exists for package(draft.package) in db or not .
If any draft available, i need to update that draft's fields.
I am using get queryset to look into db to check draft availability.
I want to know that using get queryset here is good way or not and using pass into except.
My View
def save_draft(draft, document_list):
"""
"""
try:
draft = Draft.objects.get(package=draft.package)
except Draft.DoesNotExist as exc:
pass
except Draft.MultipleObjectsReturned as exc:
raise CustomException
else:
draft.draft_document_list.filter().delete()
draft.draft_document_list.add(*document_list)
draft.save()
Extra Information :
models.py
class Package(models.Model):
name = models.CharField(max_length=100)
# -- fields
class Document(models.Model):
# -- fields
Class Draft(models.Model):
# --- fields
package = models.ForeignKey(Package)
draft_document_list = models.ManyToManyField(Document)
My Algorithm :
# first check to see if draft exists for package
# if exists
# overwrite draft_document_list with existed draft and save
# if none exists
# update passed draft object with draft_document_list
Input variables
save_draft(draft, document_list)
draft --> latest draft object
document_list --> list of documents mapped with Draft as M2M.
Yes, for for you models and method signature you use get right. To simplify things you can get rid of delete()/add() methods by direct assign document_list to M2M relation.
def save_draft(draft, document_list):
try:
draft = Draft.objects.get(package=draft.package)
except Draft.DoesNotExist:
pass
except Draft.MultipleObjectsReturned:
raise CustomException
draft.draft_document_list = document_list
draft.save()
EDIT: If there can be only one draft per package then why you use ForeignKey(Package)? With OneToOne relation your code will be much simpler:
def save_draft(draft, document_list):
draft.draft_document_list = document_list
draft.save()
class PersonSite(models.Model):
vps_id = models.AutoField(primary_key=True)
person = models.ForeignKey(CanonPerson, db_column='p_id',null=True)
site = models.ForeignKey(CanonSite, db_column='s_id',null=True)
person_sites = PersonSite.objects.filter(person=cp)
for person_site in person_sites:
if person_site and person_site.site_id and person_site.site.s_id:
# crashes for some records
We have a problem with the data, where PersonSite may point to a site that no longer exists.
In the debugger I can see that person_site.site_id has a value of 5579, however that id doesn't exist in the database:
select * from tbl_vpd_sites where s_id = 5579
Hence person_site.site_id is not null, yet just accessing person_site.site within the conditional crashes the app with the message:
DoesNotExist: CanonSite matching query does not exist.
This is a very difficult situation, I can't even check for this case to bypass it.
PersonSite.site has null=True, so it makes sense that you have to check if a object exists before accessing it.
In stead of doing all those checks if person_site and person_site.site_id and person_site.site.s_id: you can just query the db and filter the empty sites out.
person_sites = PersonSite.objects.filter(person=cp).filter(site__isnull=False)
This will return only the PersonSite objects where site IS NOT NULL and therefore have a pk.