I have pretty simple model. User defines url and database name for his own Postgres server. My django backend fetches some info from client DB to make some calculations, analytics and draw some graphs.
How to handle connections? Create new one when client opens a page, or keep connections alive all the time?(about 250-300 possible clients)
Can I use Django ORM or smth like SQLAlchemy? Or even psycopg library?
Does anyone tackle such a problem before?
Thanks
In your case, I would rather go with Django internal implementation and follow Django ORM as you will not need to worry about handling connection and different exceptions that may arise during your own implementation of DAO model in your code.
As per your requirement, you need to access user database, there still exists overhead for individual users to create db and setup something to connect with your codebase. So, I thinking sticking with Django will be more profound.
Related
So I'm starting a new Django project that essentially requires the login & registration process be routed through an EXTERNAL & ALREADY created database.
Is it possible to have the User model use an EXTERNAL database table ONLY when Django is:
Logging in a user, to check if the login is valid
Registering a user, inserting data for that user in the external database
I would like for the rest of the Django server to use a local database.
If so, could someone either provide examples or guide me to documentation on the subject?
Easiest way to use multiple database with Django is to use a database routing. By default Django stick to single database, however, if you want to implement more interesting database routing system, you can define and install your own database routers.
Database routers are installed using the DATABASE_ROUTERS setting. You have to specify this setting in your settings.py file
What you have to do is write one AuthRouter as described Django documentation Django Multiple Database
"Yes, but"
What you are looking for in the docs is called "database router".
There is even an example for the auth app in the docs there.
But, there is s serious drawback to consider with this approach:
We cannot have cross-database relationships in the models. If auth tables are in a separate database, this means that any otehr app that needs a foreign key to User model is going to run into problems. You might be able to "fake" the relationships using a db that doesn't enforce relationship checks (SQLite or MyISAM/MySQL).
Out of the box, such apps are: session, authtoken, and admin (and probably more).
Alternatively, a single-sign-on solution might do a better job: django-sso, or django-mama-cas + django-cas-ng, or the commercial Stormpath.
I'm planning to develop web UI something like iSQL*Plus-oracle but, for most common databases. Just take input query from user and return result with save options,etc.
So, for connecting to external data bases, what is advisable way,
1. Using django model and raw sql or,
2. with modules outside django - sqlalchemy+mysqldb,psycopg..?
Going through django documentation my understanding is db connections has to be in settings.py and I could not add from user input. Is my understanding is true or not?
I'm new to django not to python.
An ORM (something like Django's models or sqlalchemy) is a really helpful abstraction to help map tabular data in the database to the objects its being used to model in your code. It won't help with connecting to databases provided by the user since you won't know what the schema of the database is you're connecting to, nor what you are going to receive back from a query.
With django, the database defined in settings.py is used to store information related to your app such as user credentials, migrations as well as whatever else you define in your models.py files. So definitely don't try to change that dynamically as it is being used to store the state of your application for all users.
If you need to connect to external databases and run user-supplied queries, you can do that inside a view using the appropriate database driver. So psycopg2 for postgres would be fine.
We have our own custom NoSQL storage system (sort of a document database), and we want a Django app above it.
I want to write a backend that will allow me to connect to our database. I don't care about the Django ORM because we have our own library for querying our database.
What would be the simplest & quickest way of doing this? Rewriting
& overriding the django.db.backends.base classes seems too much of
a hassle since we don't care about Django's ORM. What we basically
need is a way to keep a connection alive, and have the queries run
against the database from our views.py file
If we want to keep the Django auth system, should we rewrite that part so that Django creates the User model etc. in our DB? Perhaps we should have two databases, a simple RDBMS for the authentication part, and the rest of the data in our custom database?
Thanks in advance!
I have a Django app that has several database backends - all connected to different instances of Postgresql database. One of them is not guaranteed to be always online. It even can be offline when application starts up.
Can I somehow configure Django to use lazy connections? I would like to:
Try querying
return "sorry, try again later" if database is offline
or return the results if database is online
Is this possible?
The original confusion is that Django tries to connect to its databases on startup. This is actually not true. Django does not connect to database, until some app tries to access the database.
Since my web application uses auth and site apps, it looks like it tries to connect on startup. But its not tied to startup, its tied to the fact that those app access the database "early".
If one defines second database backend (non-default), then Django will not try connecting to it unless application tries to query it.
So the solution was very trivial - originally I had one database that hosted both auth/site data and also "real" data that I've exposed to users. I wanted to make "real" database connection to be volatile. So I've defined separate psql backend for it and switched default backend to sqlite.
Now when trying to access "real" database through Query, I can easily wrap it with try/except and handle "Sorry, try again later" over to the user.
If you use Django you can simply create an instance of one of your models, fill it with data and call save() on it and it will be saved to the database. You don't have to pass in a "connection" parameter or do anything special. Also, your view are just simple callables so there seems to be no magic hidden. I.e. this works:
from django.http import HttpResponse
from models import MyModel
def a_simple_view(request):
instance = MyModel(some_field="Foobar")
instance.save()
return HttpResponse(<html><body>Jep, just saved</body></html>)
So the question is: How does my freshly created model instance get a database connection so save itself? And as a followup: Is this is a sensible way to do it?
How does my freshly created model instance get a database connection
so save itself?
Essentially, each model has a Manager that knows the database connection. In reality it is a bit more complicated, because the manager delegates the connection creation and management (to database routers and connection managers).
Is this is a sensible way to do it?
Well, that's a question that cannot be answered without context, really. In the context of what a Django model is, this is the sensible approach because as a developer you do not have to concern yourself with connection management.
If you're asking whether Django takes a sensible approach to connection management, and you are worried it may not, here's what the Django documentation has to say about it:
Django opens a connection to the database when it first makes a
database query. It keeps this connection open and reuses it in
subsequent requests. Django closes the connection once it exceeds the
maximum age defined by CONN_MAX_AGE or when it isn’t usable any
longer.
and:
Since each thread maintains its own connection, your database must
support at least as many simultaneous connections as you have worker
threads.
So now the question is: when and how many threads are created? This depends on the server used. E.g. the development server starts a new thread for every request, whereas gunicorn reuses threads across requests.
It is hard to describe how django does this but you can probably learn more about this by looking at the source code of django and specifically on django.db module.
There's a DB abstraction layer so that Django works with many databases such as sqlite, mysql and postgresql. There's connection pooling so that django reuses the connection on subsequent queries. All these things are used by a django model when a db query is to be run. In the end it is not simple and you should check the source code to find detailed answers.