There is a way to define MongoDB collection schema using mongoose in NodeJS. Mongoose verifies the schema at the time of running the queries.
I have been unable to find a similar thing for Motor in Python/Tornado. Is there a way to achieve a similar effect in Motor, or is there a package which can do that for me?
No there isn't. Motor is a MongoDB driver, it does basic operations but doesn't provide many conveniences. An Object Document Mapper (ODM) library like MongoTor, built on Motor, provides higher-level features like schema validation.
I don't vouch for MongoTor. Proceed with caution. Consider whether you really need an ODM: mongodb's raw data format is close enough to Python types that most applications don't need a layer between their code and the driver.
Currently (2019) this project Umongo https://github.com/Scille/umongo seems the more active and usefull if you need a sync/async Python MongoDB ODM. It work with multiple drivers like PyMongo or Motor for async.
Doc is here: http://umongo.readthedocs.io
Also you can use ODMantic as it has the best documentation and the engine supports motor client.
Currently (2023), Beanie [Github Link] is the best ODM I have ever used. It works beautifully with FASTAPI and is deeply integrated with Pydantic making it very easy to model data models.
They have a very nice documentation given in here
Related
What is the preferred way of properly building InfluxDB queries in Python?
(I have been looking for a proper InfluxDB query builder. The ones I've found are either old/outdated or can't really generate valid queries for InfluxDB.)
you might be able to use Pinform which is some kind of ORM/OSTM (Object time series mapping) for InfluxDB.
It can help with designing the schema and building normal or aggregation queries.
cli.get_fields_as_series(OHLC,
field_aggregations={'close': [AggregationMode.MEAN]},
tags={'symbol': 'AAPL'},
group_by_time_interval='10d')
Disclaimer: I am the author of this library
Now I want to use mongodb as my Python website backend storage, but I am wondering whether it's necessary to use an ODM such as MongoEngine? Or just use mongodb python driver directly?
Any good advice?
Is it strictly necessary? no - you can use the python driver directly without an ODM in the middle. If you prefer defining schemas and models to crafting/modifying your own schema via normal database operations, then an ODM is probably something you should look into.
A lot of people got used to using this kind of solution when mapping their development data model into a relational database (in that case an ORM). Because the MongoDB document model more closely maps to an object in your code (for example), you may feel you no longer need this mapping.
It can still be convenient though (as you can see from the popularity of mongoengine, mongoid, morphia and others) - the choice, in the end, is yours.
can anyone tell me and describe what oodbms implementation can be used for productive web application that stores all persistent data on it.
Maybe you should give ZODB a look. It is easy pretty easy to use, even if the user community is not as big as those of some other DBMSs.
After some research and benchmark. i found the MongoDB productive, because:
MongoDB (from "humongous") is a scalable, high-performance, open source NoSQL database. Written in C++, MongoDB features:
Document-oriented storage »
JSON-style documents with dynamic schemas offer simplicity and power.
Full Index Support »
Index on any attribute, just like you're used to.
Replication & High Availability »
Mirror across LANs and WANs for scale and peace of mind.
Auto-Sharding »
Scale horizontally without compromising functionality.
Querying »
Rich, document-based queries.
Fast In-Place Updates »
Atomic modifiers for contention-free performance.
Map/Reduce »
Flexible aggregation and data processing.
GridFS »
Store files of any size without complicating your stack.
Commercial Support »
Enterprise class support, training, and consulting available.
EDIT 1:
Packages that provides mongodb in python:
Humongolus
Humongolus is a lightweight ORM framework for Python and MongoDB. The name comes from the combination of MongoDB and Homunculus (the concept of a miniature though fully formed human body). Humongolus allows you to create models/schemas with robust validation. It attempts to be as pythonic as possible and exposes the pymongo cursor objects whenever possible. The code is available for download at github. Tutorials and usage examples are also available at GitHub.
MongoKit
The MongoKit framework is an ORM-like layer on top of PyMongo. There is also a MongoKit google group.
Ming
Ming (the Merciless) is a library that allows you to enforce schemas on a MongoDB database in your Python application. It was developed by SourceForge in the course of their migration to MongoDB. See the introductory blog post for more details.
MongoAlchemy
MongoAlchemy is another ORM-like layer on top of PyMongo. Its API is inspired by SQLAlchemy. The code is available on github; for more information, see the tutorial.
MongoEngine
MongoEngine is another ORM-like layer on top of PyMongo. It allows you to define schemas for documents and query collections using syntax inspired by the Django ORM. The code is available on github; for more information, see the tutorial.
Minimongo
minimongo is a lightweight, pythonic interface to MongoDB. It retains pymongo’s query and update API, and provides a number of additional features, including a simple document-oriented interface, connection pooling, index management, and collection & database naming helpers. The source is on github.
We're trying to set up a Pyramid project that will use MySQL instead of SQLAlchemy.
My experience with Pyramid/Python is limited, so I was hoping to find a guide online. Unfortunately, I haven't been able to find anything to push us in the right direction. Most search results were for people trying to use raw SQL/MySQL commands with SQLAlchemy (many were re-posted links).
Anyone have a useful tutorial on this?
Pyramid at its base does not assume that you will use any one specific library to help you with your persistence. In order to make things easier, then, for people who DO wish to use libraries such as SQLALchemy, the Pyramid library contains Scaffolding, which is essentially some auto-generated code for a basic site, with some additions to set up items like SQLAlchemy or a specific routing strategy. The pyramid documentation should be able to lead you through creating a new project using the "pyramid_starter" scaffolding, which sets up the basic site without SQLAlchemy.
This will give you the basics you need to set up your views, but next you will need to add code to allow you to connect to a database. Luckily, since your site is just python code, learning how to use MySQL in Pyramid is simply learning how to use MySQL in Python, and then doing the exact same steps within your Pyramid project.
Keep in mind that even if you'd rather use raw SQL queries, you might still find some usefulness in SQLAlchemy. At it's base level, SQLAlchemy simply wraps around the DBAPI calls and adds in useful features like connection pooling. The ORM functionality is actually a large addition to the tight lower-level SQLAlchemy toolset.
sqlalchemy does not make any assumption that you will be using it's orm. If you wish to use plain sql, you can do so, with nothing more than what sqlalchemy already provides. For instance, if you followed the recipe in the cookbook, you would have access to the sqlalchemy session object as request.db, your handler would look something like this:
def someHandler(request):
rows = request.db.execute("SELECT * FROM foo").fetchall()
The Quick Tutorial shows a Pyramid application that uses SQL but not SQLAlchemy. It uses SQLite, but should be reasonably easy to adapt for MySQL.
For one of my projects I prefered using Django+Mongo.
Why should I use MongoEngine, but not just PyMongo? What are advantages? Querying with PyMongo gives results that are allready objects, aren't they? So what is the purpose of MongoEngine?
This is an old question but stumbling across it, I don't think the accepted answer answers the question. The question wasn't "What is MongoEngine?" - it was "Why should I use MongoEngine?" And the advantages of such an approach. This goes beyond Django to Python/Mongo in general. My two cents:
While both PyMongo and MongoEngine do both return objects (which is not wrong), PyMongo returns dictionaries that need to have their keys referenced by string. MongoEngine allows you to define a schema via classes for your document data. It will then map the documents into those classes for you and allow you to manipulate them. Why define a schema for schema-less data? Because in my opinion, its clear, explicit, and much easier to program against. You don't end up with dictionaries scattered about your code where you can't tell what's in them without actually looking at the data or running the program. In the case of MongoEngine and a decent IDE like PyCharm, typing a simple "." after the object will tell you all you need to know via auto-complete. It's also much easier for other developers coming in to examine and learn the data model as they work and will make anybody who hasn't seen the code in a while more productive, quicker.
Additionally, to me, the syntax used to manipulate documents with PyMongo (which is essentially the same as the mongo console) is ugly, error prone, and difficult to maintain.
Here is a basic example of updating a document in MongoEngine, which to me, is very elegant:
BlogPost.objects(id=post.id).update(title='Example Post')
Why use PyMongo? MongoEngine is a layer between you and the bare metal, so it's probably slower, although I don't have any benchmarks. PyMongo is lower level, so naturally you have more control. For simple projects, MongoEngine might be unnecessary. If you're already fluent in Mongo syntax, you may find PyMongo much more intuitive than I do and have no problem writing complex queries and updates. Perhaps you enjoy working directly with dictionaries on that lower level and aren't interested in an additional layer of abstraction. Maybe you're writing a script that isn't part of a big system, and you need it to be as lean and as fast as possible.
There's more to the argument, but I think that's pretty good for the basics.
I assume you have not read the MongoEngine claim.
MongoEngine is a Document-Object
Mapper (think ORM, but for document
databases) for working with MongoDB
from Python.
This basically say it all.
In addition: your claim that Pymongo would deliver objects is wrong....well in Python everything is an object - even a dict is an object...so you are true but not in the sense of having a custom class defined on the application level.
PyMongo is the low-level driver wrapping the MongoDB API into Python and delivering JSON in and out.
MongoEngine or other layers like MongoKit map your MongoDB-based data to objects similar to native Python database drivers + SQLAlchemy as ORM.
Probably way too late, but for anyone else attempting Django+Mongo, Django-nonrel is worth considering.
mongoengine will use the pymongo driver to connect to mongodb.
If you are familiar with django.. use mongoengine