I am having trouble saving documents to a new collection and then deleting them from the old one.
I create a new object like so:
class Test(mongo.Document):
field = mongo.StringField()
t = Test(field="test")
t.switch_collection('default')
t.save()
t.switch_collection('switched')
t.save()
t.switch_collection('default')
t.delete()
It seems only to perform the first save to the default collection and then performs nothing after that. I have played around with a bunch of difference options such as reloading the object after every switch/save and from mongoengine context managers:
with switch_collection(Test, 'mongoswitch') as test:
test(field="switch").save()
My mongo settings look like (called first):
app.config["MONGODB_SETTINGS"] = {'db': 'TestDB'}
mongo = MongoEngine(app)
Using mongoengine 0.10 and pymongo 2.8.1 with Python 3.4 .
Anyone have an idea?
Much Thanks.
I see it's old question but maybe there's someone with the same problem.. I think it's because when you have a document that has id set, by calling t.save() you only update existing document in the collection. To really save it you need to call t.save(force_insert=True)
Are you getting any errors? It had worked for me. Check records in your mongo db collections.
One possible reason from mongoengine docs:
"Make sure any aliases have been registered with register_connection() or connect() before using the context manager."
Related
I am using MongoDB for application setting and in my Python code, I read the setting document from db using mongoengine that is a high-level wrapper for pymongo.
I'm able to reload the document each time accessing its attributes like this (assuming that there is a document in Setting collection in MongoDB):
import time
import mongoengine
class Setting(mongoengine.Document):
log_level = mongoengine.StringField(default='info')
setting = Setting.objects[0]
while True:
time.sleep(1)
setting.reload()
print(setting.log_level)
But, I'm interested in a method to make Setting document automatically reloaded each time accessing log_level attribute. Is there a clean way to do this in mongoengine? I prefer to have a code like this and my Setting be always sync with db:
...
while True:
time.sleep(1)
print(setting.log_level)
I've read mongoengine doc a bit more to find a nice way and I've found no_cache() method that can be used to force the Setting class to returned a non-caching queryset. With this, I'm a bit closer:
def get_setting():
return Setting.objects.no_cache()[0]
while True:
time.sleep(1)
print(get_setting().log_level)
Any idea?
Thank you
You can write a custom method which does that.
Pseudocode:
def get_reloaded_attr(obj,attr):
obj.reload()
return getattr(obj,attr)
The documentation (https://cloud.google.com/appengine/docs/python/ndb/) states that
NDB uses Memcache as a cache service for "hot spots" in the data
I am now using memcache only as follows:
memcache.set(key=(id), value=params, time=0)
That expires (auto flushes) pretty often and so I would like to use NDB Datastore.
I thought I would have to always put the key-value in both NDB and Memcache, then check both.
Is this being done automatically by NDB?
Ie.
ancestor_key = ndb.Key("Book", guestbook_name or "*notitle*")
greetings = Greeting.query_book(ancestor_key).fetch(20)
Would that implicitly set Memcache ?
And when I read from NDB, would it implicitly try a memcache.get(key) first?
Thanks for your patience.
EDIT - What I tried:
As a test I tried something like this:
class Book(ndb.Model):
content = ndb.StringProperty()
class update(webapp2.RequestHandler):
def post(self):
p1='1'
p2='2'
p3='3'
p4='4'
p5='5'
id='test'
paramarray = (p1,p2,p3,p4,p5)
book = Book(name=id,value=paramarray)
# OR likes this - book = Book(ndb.Key(id),value=paramarray)
book.put()
Both versions error out.
Trying to get a key of the var id with the values of paramarray
EDIT 2 Daniel, Thank you for everything.
Have follow up formatting questions, will ask a new question.
Yes; see the full documentation on ndb caching. Basically, every write is cached both in a request-local in-context cache, and in the main memcached store; a get by key will look up in both caches first before falling back to the real datastore.
Edit I can't understand why you think your example would work. You defined a model with a content property, but then try to set name and value properties on it; naturally that will fail.
You should go through the ndb documentation, which gives a good introduction to using the model class.
I am trying to delete an entry from my table. This is my code for the delete function.
#app.route("/delete_link/<link_id>", methods=['GET', 'POST'])
def delete_link(link_id):
link = models.Link.query.filter(models.Link.l_id == link_id).first()
db.session.delete(link)
db.session.commit()
return flask.redirect(flask.url_for('links'))
the line: db.session.delete(link) returns me this error:
InvalidRequestError: Object '' is already attached to session '1' (this is '2')
I've tried this code as well:
#app.route("/delete_link/<link_id>", methods=['GET', 'POST'])
def delete_link(link_id):
link = models.Link.query.filter(models.Link.l_id == link_id)
link.delete()
db.session.commit()
return flask.redirect(flask.url_for('links'))
which does not update the database. Link must not be in the session I guess, but I don't know how to check that, and how to fix it.
I am new to sqlalchemy.
EDIT:
I use this to create my db variable which probably creates the session at this stage (this is at the top of the code). It comes from the flask documentation
from yourapplication import db
You are creating 2 instances of the db object, inherently creating 2 different sessions.
In models.py:
...
5. from config import app
6.
7. db = SQLAlchemy(app)
In erika.py:
...
16. from config import app
...
23. db = SQLAlchemy(app)
then when you try to delete the element:
link = models.Link.query.filter(models.Link.l_id == link_id).first()
db.session.delete(link)
db.session.commit()
the following happens:
models.Link.query uses the database session created by models.py to get the record.
db.session.delete uses the session created by erika.py.
link is attached to the models.py session and you can't use another session (erikas.py) to delete it. Hence:
InvalidRequestError: Object '' is already attached to session '1' (this is '2')
Solution
The solution it's simple. Have only one instance of a db object at any time and reuse that instance whenever you need db operations.
erika.py
from models import db
This way you are always using the same session that was used to fetch your records.
It appears to be a similar problem to the one described at http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvi-debugging-testing-and-profiling
It's a good in-depth description of the problem and how he solved it. The author of that article made a fix that's available as a fork.
The Fix
To address this problem we need to find an alternative way of attaching Flask-WhooshAlchemy's query object to the model.
The documentation for Flask-SQLAlchemy mentions there is a model.query_class attribute that contains the class to use for queries. This is actually a much cleaner way to make Flask-SQLAlchemy use a custom query class than what Flask-WhooshAlchemy is doing. If we configure Flask-SQLAlchemy to create queries using the Whoosh enabled query class (which is already a subclass of Flask-SQLAlchemy's BaseQuery), then we should have the same result as before, but without the bug.
I have created a fork of the Flask-WhooshAlchemy project on github where I have implemented these changes. If you want to see the changes you can see the github diff for my commit, or you can also download the fixed extension and install it in place of your original flask_whooshalchemy.py file.
I googled for a while but I could not find reference on how to retrieve a Colander Schema from a config file or from a database. I think this is not difficult to implement but I might have overlooked something. Maybe somebody has done or seen something like that and might share some insights.
Here a sample for a Colander Schema:
class PageSchema(colander.MappingSchema):
title = SchemaNode(String(),
title='Page title',
description='The title of the page',
)
description = SchemaNode(String(),
title='A short description',
description='Keep it under 60 characters or so',
missing = u'',
validator=colander.Length(max=79)
)
body = colander.SchemaNode(colander.String(),
description='Tell the world',
missing = u'')
As micheal said, it might not be supported. If you really need it. Here is some pointers.
Save your schema in a database by name for example: "PageSchema". Save all of its record in the database with all the needed parameters.
You'd have to do something like that:
for row in rows:
attrinbutes[row['name']] = build_attribute(row)
schemas[schema_name] = type(schema_name, (colander.MappingSchema,), attributes)
exec('%s = schemas[schema_name]' % schema_name)
In other words, it loads all attributes and build a class using the type operator. That kind of task is pretty simple and should work as good as the habitual class syntax. The exec call is just to push the name in locals. You could probably use locals()[schema_name] = schmea or even other scopes.
That way you can load schemas from anywhere if needed. You could build yourself a factory like:
schemas.get('PageSchema') that would return a schema if possible or None if not present.
That's pretty much it!
This is not supported by colander. The one thing I know of in this area is the "limone" package which does the opposite. It is able to generate arbitrary python objects from a colander schema.
ColanderAlchemy may do what you need. It takes SQLAlchemy objects and generates a Colander schema from them. However, generating from an SQLAlchemy object isn't exactly "from a database".
To empty a database table, I use this SQL Query:
TRUNCATE TABLE `books`
How to I truncate a table using Django's models and ORM?
I've tried this, but it doesn't work:
Book.objects.truncate()
The closest you'll get with the ORM is Book.objects.all().delete().
There are differences though: truncate will likely be faster, but the ORM will also chase down foreign key references and delete objects in other tables.
You can do this in a fast and lightweight way, but not using Django's ORM. You may execute raw SQL with a Django connection cursor:
from django.db import connection
cursor = connection.cursor()
cursor.execute("TRUNCATE TABLE `books`")
You can use the model's _meta property to fill in the database table name:
from django.db import connection
cursor = connection.cursor()
cursor.execute('TRUNCATE TABLE "{0}"'.format(MyModel._meta.db_table))
Important: This does not work for inherited models as they span multiple tables!
In addition to Ned Batchelder's answer and refering to Bernhard Kircher's comment:
In my case I needed to empty a very large database using the webapp:
Book.objects.all().delete()
Which, in the development SQLlite environment, returned:
too many SQL variables
So I added a little workaround. It maybe not the neatest, but at least it works until the truncate table option is build into Django's ORM:
countdata = Book.objects.all().count()
logger.debug("Before deleting: %s data records" % countdata)
while countdata > 0:
if countdata > 999:
objects_to_keep = Book.objects.all()[999:]
Book.objects.all().exclude(pk__in=objects_to_keep).delete()
countdata = Book.objects.all().count()
else:
Book.objects.all().delete()
countdata = Book.objects.all().count()
By the way, some of my code was based on "Django Delete all but last five of queryset".
I added this while being aware the answer was already answered, but hopefully this addition will help some other people.
I know this is a very old Question and few corrects answer is in here is as well but I can't resist myself to share the most elegant and fastest way to serve the purpose of this question.
class Book(models.Model):
# Your Model Declaration
#classmethod
def truncate(cls):
with connection.cursor() as cursor:
cursor.execute('TRUNCATE TABLE {} CASCADE'.format(cls._meta.db_table))
And now to truncate all data from Book table just call
Book.truncate()
Since this is directly interact with your Database it will perform much faster than doing this
Book.objects.all().delete()
Now there's a library to help you truncate a specific TABLE in your Django project Database, It called django-truncate.
It's simple just run python manage.py truncate --apps myapp --models Model1 and all of the data in that TABLE will be deleted!
Learn more about it here: https://github.com/KhaledElAnsari/django-truncate
For me the to truncate my local sqllite database I end up with python manage.py flush.
What I have initial tried is to iterate over the models and delete all to rows one by one:
models = [m for c in apps.get_app_configs() for m in c.get_models(include_auto_created=False)]
for m in models:
m.objects.all().delete()
But becuse I have Protected foreign key the success of the operation depended on the order of the models.
So, I am using te flush command to truncate my local test database and it is working for me
https://docs.djangoproject.com/en/3.0/ref/django-admin/#django-admin-flush
This code uses PosgreSQL dialect. Leave out the cascade bits to use standard SQL.
Following up on Shubho Shaha's answer, you could also create a model manager for this.
class TruncateManager(models.Manager):
def truncate(self, cascade=False):
appendix = " CASCADE;" if cascade else ";"
raw_sql = f"TRUNCATE TABLE {self.model._meta.db_table}{appendix}"
cursor = connection.cursor()
cursor.execute(raw_sql)
class Truncatable(models.Model):
class Meta:
abstract = True
objects = TruncateManager()
Then, you can extend the Truncatable to create truncatable objects:
class Book(Truncatable):
...
That will allow you to call truncate on all models that extend from Truncatable.
Book.objects.truncate()
I added a flag to use cascade as well, which (danger zone) will also: "Automatically truncate all tables that have foreign-key references to any of the named tables, or to any tables added to the group due to CASCADE.", which is obviously more destructive, but will allow the code to run inside an atomic transaction.
This is doesn't directly answer the OP's question, but is nevertheless a solution one might use to achieve the same thing - differently.
Well, for some strange reason (while attempting to use the suggested RAW methods in the other answers here), I failed to truncate my Django database cache table until I did something like this:
import commands
cmd = ['psql', DATABASE, 'postgres', '-c', '"TRUNCATE %s;"' % TABLE]
commands.getstatusoutput(' '.join(cmd))
Basically, I had to resort to issuing the truncate command via the database's utility commands - psql in this case since am using Postgres. So, automating the command line might handle such corner cases.
Might save someone else some time...