Insert document to a collection with mongoengine - python

I need to insert a document into mongodb without using the models, just raw documents without following a model. I'm currently doing the following:
db.analyticsNew.insert(documents_list)
But this throws an error saying
AttributeError: 'MongoEngine' object has no attribute 'analyticsNew'
From what I know, I think it can't find the mentioned collection. But the collection exists in the db. Plus, even if this collection isn't there, it should create one.
Any ideas as to whats wrong and how to fix it?

MongoEngine is a Document-Object Mapper (think ORM, but for document
databases) for working with MongoDB from Python.
That means you have to work with models which are python classes each represents a mongo collection.
If you need to run raw queries you may think to use pymongo.

MongoEngine is built on top of pymongo. It exposes pymongo and therefore you can use it for raw queries as well.
For instance have a look at this.

Related

Can one use mongoengine to define a document first, then use pymongo to insert many documents?

MongoEngine is good for defining a document for data validation. Raw pymongo is lacking in this area. Can I use MongoEngine to define a document first, then use pymongo to insertMany documents into an empty collection? If yes, will pymongo's insertMany() do data validation based on the document definition set by mongoengine?
Can pymongo and mongoengine code be mixed together in the same python script?
I am using python 3.7, mongodb v4.2.7.
Let me breakdown your question into three parts.
Can i use mongoengine validation with pymongo ?
If used insertmany then how do validate ?
Can pymongo and mongoengine code be mixed together in the same python script?
Answers -:
No you cannot use mongoengine validation with pymongo. Because validation is define in mongoengine module like -:
snap from mongoengine lib -- mongoengine -> common.py:
So, when you use validation in mongoenine these function is called and validate the data. If you want to validate you data for pymongo, then you have to write custom validation code for that.
Insertmany() is seperate function, there is no relation with mongoengine. So mongoenine define is not worked with Insertmany().
Yes, You can write pymongo and mongoengine code together in same python script. [In some scenario we need to do that].
Note -: If you Use both pymongo and mongoenine together. Be careful, Because mongoengine have default validation function. So they check some validation before perform operations. But pymongo not have any validation, They just insert data and create collection [ If not exist ] and then insert data. So may be datatype conflict occur and problem like this.
Your three questions:
'Can I use MongoEngine to define a document first, then use pymongo to insertMany documents into an empty collection? '
Yes.
'If yes, will pymongo's insertMany() do data validation based on the document definition set by mongoengine?'
No.
'Can pymongo and mongoengine code be mixed together in the same python script?'
Yes.
Mongoengine based on pymongo. If you do not need very complex function that mongoengine not had. I suggest you just use mongoengine to complete your work. This will save your time most of the time.
I'm a novice when it comes to Python and Mongo. I believe you can validate each document with MongoEngine like so:
for my_obj in my_objs:
my_obj.validate()
Then, bulk insert like so:
dicts = map(lambda my_obj: my_obj.to_mongo().to_dict(), my_objs)
collection.insert_many(dicts)
However, you can use MongoEngine to also bulk insert if you don't need to use PyMongo directly.

Pickle database independent query in Django

First of all, I know that my question duplicate this question. But I supose it's not the same.
I need to save user "search filter". As I understand Django ORM create specific SQL query for different DB. So if I save SQL query I can't migrate on other database with different SQL syntax.
Am I wrong? If no, how can I save Django side of query, without accsesing to DB?
The short answer is that you're correct -- mostly. If the SQL dialect that Django compiled the query for isn't compatible with a different backend, it wouldn't work or might work unpredictably.
To save the Django side of the query, why not just save the actual filter() statement that you're using or a representation of it that you can convert back on the fly?
Edit: Okay in that case I think you're on the right track based on comments and above answer. If you're parsing a query string already save that in the database as a CharField and then just use it to build a Django QuerySet when you retrieve it. If I'm understanding.
If you can suggest better sulution I open for conversation
So... Pickle the function .filter() is not the best idea so as saving SQL string for specific DB. I think the best solution for this problem is saving search parameters. In my case it's GET string. I get it:
request.META["QUERY_STRING"]
And save to DB.
If I need to get it, i just parse:
from django.http import QueryDict
QueryDict(request.META["QUERY_STRING"])
Aditionally I use different form for validate this values (optional) SearchTrustedForm(), because if data structure has changed I can save backwards compatibility.

MongoEngine & serverStatus

I am switching my python script from using pymongo to using mongoengine. Previously, I had this call:
client_count = db.command("serverStatus")["connections"]['current'] - 1
but I want to get the number of current connections using mongoengine's wrapper instead. I looked at server-status and the mongoengine docs and was unable to find an equivalent call for pulling server variables.
I want to swap over to mongoengine because we're using its ODM features and we would like to cut out redundancy.
Any pointers for this? What is the correct syntax, if at all? Thanks.
In the meantime, we are just going to use pymongo for getting the correct data, while using mongoengine for everything else.
There is no Mongoengine equivalent, so you have to call this manually. As we wrap pymongo you can do the call via getting the database from the connection register. The easiest way to do this is to use the document class eg:
db = Document._get_db()
client_count = db.command("serverStatus")["connections"]['current'] - 1
Caveat: I don't know either of these APIs well.
However, if you look at the dependencies of mongoengine, you will see that it depends on pymongo, since it is in fact implemented in terms of pymongo.
Now have a look at mongoengines implementation of its connection logic: https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/connection.py#L113: the connection objects it returns are actually pymongo connections. So, in principle, you should be able to make pymongo calls on the returned connection. So you could in fact continue to use the above call. Of course, you are violating the encapsulation of mongoengine at this point, since mongoengine could change its internal implementation type for connections.
However, looking through their documentation, I don't see any other way to access this aspect of the database, so perhaps this is the easiest way forward.

Is it necessary to use an ODM framework for Mongodb in Python?

Now I want to use mongodb as my Python website backend storage, but I am wondering whether it's necessary to use an ODM such as MongoEngine? Or just use mongodb python driver directly?
Any good advice?
Is it strictly necessary? no - you can use the python driver directly without an ODM in the middle. If you prefer defining schemas and models to crafting/modifying your own schema via normal database operations, then an ODM is probably something you should look into.
A lot of people got used to using this kind of solution when mapping their development data model into a relational database (in that case an ORM). Because the MongoDB document model more closely maps to an object in your code (for example), you may feel you no longer need this mapping.
It can still be convenient though (as you can see from the popularity of mongoengine, mongoid, morphia and others) - the choice, in the end, is yours.

PyMongo vs MongoEngine for Django

For one of my projects I prefered using Django+Mongo.
Why should I use MongoEngine, but not just PyMongo? What are advantages? Querying with PyMongo gives results that are allready objects, aren't they? So what is the purpose of MongoEngine?
This is an old question but stumbling across it, I don't think the accepted answer answers the question. The question wasn't "What is MongoEngine?" - it was "Why should I use MongoEngine?" And the advantages of such an approach. This goes beyond Django to Python/Mongo in general. My two cents:
While both PyMongo and MongoEngine do both return objects (which is not wrong), PyMongo returns dictionaries that need to have their keys referenced by string. MongoEngine allows you to define a schema via classes for your document data. It will then map the documents into those classes for you and allow you to manipulate them. Why define a schema for schema-less data? Because in my opinion, its clear, explicit, and much easier to program against. You don't end up with dictionaries scattered about your code where you can't tell what's in them without actually looking at the data or running the program. In the case of MongoEngine and a decent IDE like PyCharm, typing a simple "." after the object will tell you all you need to know via auto-complete. It's also much easier for other developers coming in to examine and learn the data model as they work and will make anybody who hasn't seen the code in a while more productive, quicker.
Additionally, to me, the syntax used to manipulate documents with PyMongo (which is essentially the same as the mongo console) is ugly, error prone, and difficult to maintain.
Here is a basic example of updating a document in MongoEngine, which to me, is very elegant:
BlogPost.objects(id=post.id).update(title='Example Post')
Why use PyMongo? MongoEngine is a layer between you and the bare metal, so it's probably slower, although I don't have any benchmarks. PyMongo is lower level, so naturally you have more control. For simple projects, MongoEngine might be unnecessary. If you're already fluent in Mongo syntax, you may find PyMongo much more intuitive than I do and have no problem writing complex queries and updates. Perhaps you enjoy working directly with dictionaries on that lower level and aren't interested in an additional layer of abstraction. Maybe you're writing a script that isn't part of a big system, and you need it to be as lean and as fast as possible.
There's more to the argument, but I think that's pretty good for the basics.
I assume you have not read the MongoEngine claim.
MongoEngine is a Document-Object
Mapper (think ORM, but for document
databases) for working with MongoDB
from Python.
This basically say it all.
In addition: your claim that Pymongo would deliver objects is wrong....well in Python everything is an object - even a dict is an object...so you are true but not in the sense of having a custom class defined on the application level.
PyMongo is the low-level driver wrapping the MongoDB API into Python and delivering JSON in and out.
MongoEngine or other layers like MongoKit map your MongoDB-based data to objects similar to native Python database drivers + SQLAlchemy as ORM.
Probably way too late, but for anyone else attempting Django+Mongo, Django-nonrel is worth considering.
mongoengine will use the pymongo driver to connect to mongodb.
If you are familiar with django.. use mongoengine

Categories

Resources