I am switching my python script from using pymongo to using mongoengine. Previously, I had this call:
client_count = db.command("serverStatus")["connections"]['current'] - 1
but I want to get the number of current connections using mongoengine's wrapper instead. I looked at server-status and the mongoengine docs and was unable to find an equivalent call for pulling server variables.
I want to swap over to mongoengine because we're using its ODM features and we would like to cut out redundancy.
Any pointers for this? What is the correct syntax, if at all? Thanks.
In the meantime, we are just going to use pymongo for getting the correct data, while using mongoengine for everything else.
There is no Mongoengine equivalent, so you have to call this manually. As we wrap pymongo you can do the call via getting the database from the connection register. The easiest way to do this is to use the document class eg:
db = Document._get_db()
client_count = db.command("serverStatus")["connections"]['current'] - 1
Caveat: I don't know either of these APIs well.
However, if you look at the dependencies of mongoengine, you will see that it depends on pymongo, since it is in fact implemented in terms of pymongo.
Now have a look at mongoengines implementation of its connection logic: https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/connection.py#L113: the connection objects it returns are actually pymongo connections. So, in principle, you should be able to make pymongo calls on the returned connection. So you could in fact continue to use the above call. Of course, you are violating the encapsulation of mongoengine at this point, since mongoengine could change its internal implementation type for connections.
However, looking through their documentation, I don't see any other way to access this aspect of the database, so perhaps this is the easiest way forward.
Related
I need to insert a document into mongodb without using the models, just raw documents without following a model. I'm currently doing the following:
db.analyticsNew.insert(documents_list)
But this throws an error saying
AttributeError: 'MongoEngine' object has no attribute 'analyticsNew'
From what I know, I think it can't find the mentioned collection. But the collection exists in the db. Plus, even if this collection isn't there, it should create one.
Any ideas as to whats wrong and how to fix it?
MongoEngine is a Document-Object Mapper (think ORM, but for document
databases) for working with MongoDB from Python.
That means you have to work with models which are python classes each represents a mongo collection.
If you need to run raw queries you may think to use pymongo.
MongoEngine is built on top of pymongo. It exposes pymongo and therefore you can use it for raw queries as well.
For instance have a look at this.
i wonder wether there is a solution (or a need for) an ORM with Graph-Database (f.e. Neo4j). I'm tracking relationships (A is related to B which is related to A via C etc., thus constructing a large graph) of entities (including additional attributes for those entities) and need to store them in a DB, and i think a graph database would fit this task perfectly.
Now, with sql-like DBs, i use sqlalchemyś ORM to store my objects, especially because of the fact that i can retrieve objects from the db and work with them in a pythonic style (use their methods etc.).
Is there any object-mapping solution for Neo4j or other Graph-DB, so that i can store and retrieve python objects into and from the Graph-DB and work with them easily?
Or would you write some functions or adapters like in the python sqlite documentation (http://docs.python.org/library/sqlite3.html#letting-your-object-adapt-itself) to retrieve and store objects?
Shameless plug... there is also my own ORM which you may also want to checkout: https://github.com/robinedwards/neomodel
It's built on top of py2neo, using cypher and rest API calls under hood, i.e no dependency on gremlin.
There are a couple choices in Python out there right now, based on databases' REST interfaces.
As I mentioned in the link #Peter provided, we're working on neo4django, which updates the old Neo4j/Django integration. It's a good choice if you need complex queries and want an ORM that will manage node indexing as well- or if you're already using Django. It works very similarly to the native Django ORM. Find it on PyPi or GitHub.
There's also a more general solution called Bulbflow that is supposed to work with any graph database supported by Blueprints. I haven't used it, but from what I've seen it focuses on domain modeling - Bulbflow already has working relationship models, for example, which we're still working on- but doesn't much support complex querying (as we do with Django querysets + index use). It also lets you work a bit closer to the graph.
Maybe you could take a look on Bulbflow, that allows to create models in Django, Flask or Pyramid. However, it works over a REST client instead of the python-binding provided by Neo4j, so perhaps it's not as fast as the native binding is.
We're trying to set up a Pyramid project that will use MySQL instead of SQLAlchemy.
My experience with Pyramid/Python is limited, so I was hoping to find a guide online. Unfortunately, I haven't been able to find anything to push us in the right direction. Most search results were for people trying to use raw SQL/MySQL commands with SQLAlchemy (many were re-posted links).
Anyone have a useful tutorial on this?
Pyramid at its base does not assume that you will use any one specific library to help you with your persistence. In order to make things easier, then, for people who DO wish to use libraries such as SQLALchemy, the Pyramid library contains Scaffolding, which is essentially some auto-generated code for a basic site, with some additions to set up items like SQLAlchemy or a specific routing strategy. The pyramid documentation should be able to lead you through creating a new project using the "pyramid_starter" scaffolding, which sets up the basic site without SQLAlchemy.
This will give you the basics you need to set up your views, but next you will need to add code to allow you to connect to a database. Luckily, since your site is just python code, learning how to use MySQL in Pyramid is simply learning how to use MySQL in Python, and then doing the exact same steps within your Pyramid project.
Keep in mind that even if you'd rather use raw SQL queries, you might still find some usefulness in SQLAlchemy. At it's base level, SQLAlchemy simply wraps around the DBAPI calls and adds in useful features like connection pooling. The ORM functionality is actually a large addition to the tight lower-level SQLAlchemy toolset.
sqlalchemy does not make any assumption that you will be using it's orm. If you wish to use plain sql, you can do so, with nothing more than what sqlalchemy already provides. For instance, if you followed the recipe in the cookbook, you would have access to the sqlalchemy session object as request.db, your handler would look something like this:
def someHandler(request):
rows = request.db.execute("SELECT * FROM foo").fetchall()
The Quick Tutorial shows a Pyramid application that uses SQL but not SQLAlchemy. It uses SQLite, but should be reasonably easy to adapt for MySQL.
For one of my projects I prefered using Django+Mongo.
Why should I use MongoEngine, but not just PyMongo? What are advantages? Querying with PyMongo gives results that are allready objects, aren't they? So what is the purpose of MongoEngine?
This is an old question but stumbling across it, I don't think the accepted answer answers the question. The question wasn't "What is MongoEngine?" - it was "Why should I use MongoEngine?" And the advantages of such an approach. This goes beyond Django to Python/Mongo in general. My two cents:
While both PyMongo and MongoEngine do both return objects (which is not wrong), PyMongo returns dictionaries that need to have their keys referenced by string. MongoEngine allows you to define a schema via classes for your document data. It will then map the documents into those classes for you and allow you to manipulate them. Why define a schema for schema-less data? Because in my opinion, its clear, explicit, and much easier to program against. You don't end up with dictionaries scattered about your code where you can't tell what's in them without actually looking at the data or running the program. In the case of MongoEngine and a decent IDE like PyCharm, typing a simple "." after the object will tell you all you need to know via auto-complete. It's also much easier for other developers coming in to examine and learn the data model as they work and will make anybody who hasn't seen the code in a while more productive, quicker.
Additionally, to me, the syntax used to manipulate documents with PyMongo (which is essentially the same as the mongo console) is ugly, error prone, and difficult to maintain.
Here is a basic example of updating a document in MongoEngine, which to me, is very elegant:
BlogPost.objects(id=post.id).update(title='Example Post')
Why use PyMongo? MongoEngine is a layer between you and the bare metal, so it's probably slower, although I don't have any benchmarks. PyMongo is lower level, so naturally you have more control. For simple projects, MongoEngine might be unnecessary. If you're already fluent in Mongo syntax, you may find PyMongo much more intuitive than I do and have no problem writing complex queries and updates. Perhaps you enjoy working directly with dictionaries on that lower level and aren't interested in an additional layer of abstraction. Maybe you're writing a script that isn't part of a big system, and you need it to be as lean and as fast as possible.
There's more to the argument, but I think that's pretty good for the basics.
I assume you have not read the MongoEngine claim.
MongoEngine is a Document-Object
Mapper (think ORM, but for document
databases) for working with MongoDB
from Python.
This basically say it all.
In addition: your claim that Pymongo would deliver objects is wrong....well in Python everything is an object - even a dict is an object...so you are true but not in the sense of having a custom class defined on the application level.
PyMongo is the low-level driver wrapping the MongoDB API into Python and delivering JSON in and out.
MongoEngine or other layers like MongoKit map your MongoDB-based data to objects similar to native Python database drivers + SQLAlchemy as ORM.
Probably way too late, but for anyone else attempting Django+Mongo, Django-nonrel is worth considering.
mongoengine will use the pymongo driver to connect to mongodb.
If you are familiar with django.. use mongoengine
There is a Java paradigm for database access implemented in the Java DataSource. This object create a useful abstraction around the creation of database connections. The DataSource object keeps database configuration, but will only create database connections on request. This is allows you to keep all database configuration and initialization code in one place, and makes it easy to change database implementation, or use a mock database for testing.
I currently working on a Python project which uses cx_Oracle. In cx_Oracle, one gets a connection directly from the module:
import cx_Oracle as dbapi
connection = dbapi.connect(connection_string)
# At this point I am assuming that a real connection has been made to the database.
# Is this true?
I am trying to find a parallel to the DataSource in cx_Oracle. I can easily create this by creating a new class and wrapping cx_Oracle, but I was wondering if this is the right way to do it in Python.
You'll find relevant information of how to access databases in Python by looking at PEP-249: Python Database API Specification v2.0. cx_Oracle conforms to this specification, as do many database drivers for Python.
In this specification a Connection object represents a database connection, but there is no built-in pooling. Tools such as SQLAlchemy do provide pooling facilities, and although SQLAlchemy is often billed as an ORM, it does not have to be used as such and offers nice abstractions for use on top of SQL engines.
If you do want to do object-relational-mapping, then SQLAlchemy does the business, and you can consider either its own declarative syntax or another layer such as Elixir which sits on top of SQLAlchemy and provides increased ease of use for more common use cases.
I don't think there is a "right" way to do this in Python, except maybe to go one step further and use another layer between yourself and the database.
Depending on the reason for wanting to use the DataSource concept (which I've only ever come across in Java), SQLAlchemy (or something similar) might solve the problems for you, without you having to write something from scratch.
If that doesn't fit the bill, writing your own wrapper sounds like a reasonable solution.
Yes, Python has a similar abstraction.
This is from our local build regression test, where we assure that we can talk to all of our databases whenever we build a new python.
if database == SYBASE:
import Sybase
conn = Sybase.connect('sybasetestdb','mh','secret')
elif database == POSTRESQL:
import pgdb
conn = pgdb.connect('pgtestdb:mh:secret')
elif database == ORACLE:
import cx_Oracle
conn = cx_Oracle.connect("mh/secret#oracletestdb")
curs=conn.cursor()
curs.execute('select a,b from testtable')
for row in curs.fetchall():
print row
(note, this is the simple version, in our multidb-aware code we have a dbconnection class that has this logic inside.)
I just sucked it up and wrote my own. It allowed me to add things like abstracting the database (Oracle/MySQL/Access/etc), adding logging, error handling with transaction rollbacks, etc.