Develop Reference

Customize the Django ContentTypes and AdminLog and Migration model

I am working on a Django project and it requires 3 different database servers with different engines like MySql, PostgreSQL and SQLite.
SQLite has all the client side setting table, tables with not shared data [app has a feature to not upload some data to the server] and configuration tables and it is on the client machine through a tweaked C# app to create a django server environment and show the django site in a browser control like a native application.
PostgreSQL has tables that can be accessed by different clients on the network and has different tables with values of shared data. They maintain integrity with SQLite data through ContentType.
Django provides the contentType table that is created in both SQLite and PostgreSQL. Due to heavy use of other database managing tables we require them to be in a MySQL database on a separate server. So it would be great to have django_content_types, django_admin_log and django_migration in that MySQL server.
ContentTypes table is also used a lot and it is difficult to maintain contentTypes form two different database tables so we require it to be in one separate one with values form both the databases. And when one goes there why not the other two: django_migration, django_admin_log.
Also there is requirement to store raw SQL DDL queries in the database to track changes outside of the Django ORM and so it makes sense to add rows to django_migration and a new column to store raw SQL queries.
Also, this is the last thing for sure, django_admin_log only logs admin-site changes and also lacks data in message columns when new row is added or deleted. We really require it to store the JSON of values added to the particular table when adding and when deleting, the instance [json format of row] to message column about the record deleted. And require it to log all transactions on instances of both the databases.
I am looking to [customize/extend/create my own and override] these models in a Django App to gain the above expected behavior and have a portable app to use it in future such projects.
Regards,
    Aagam Sheth

Using Apache Ignite with Django

We have an existing Apache Ignite in-memory DB with lots of stored data. I want to connect my existing Django web application to the Ignite DB in order to query the existing data. Are there any examples of how to connect a Django application to an Apache Ignite DB and how to query the DB using the Django ORM?

You can use JDBC drivers with Django ORM: How to write custom database adapter for django using JDBC drivers?
Then, you can use Ignite JDBC driver: https://ignite.apache.org/docs/latest/SQL/JDBC/jdbc-driver

Django ORM requires a special adapter for each kind of database server. Such an adapter is called “database backend” in Django terminology. There is no Django database backend for Ignite. There is also no universal, middle-ground backend I know of, that can be used with Ignite.
You can implement your own custom backend (guidelines) or try to pick up the latest attempt in creating a more-or-less server-independent ODBC backend. But in your place I'd probably not get involved in any of this, since it implies a lot of work not directly related to your task.
I'd just convert the data with one-off script (or Django management command), using pyodbc or pyignite as a source and Django model as a target.

Should I deploy my web server with Django's default database?

I'm a noobie to Django and I'm almost at the stage of deploying a web server.
I was just having some doubts with Django's database. Currently I'm using the default sqlite3 database to store all the user models as well as the info models. I'm thinking of using AWS to deploy my web sever.
So when I get to that stage, should I continue with sqlite or should I switch to AWS's database or something like Firebase. If I continue with sqlite, where and how exactly will the information be stored? And what if I switch to something like PostgreSQL, where will the information be stored and will it be secure/fast (even if I manage to get thousands of users)?
Thanks so much, this question might be really basic but I'm super confused.

sqlite is a flat file database, it uses an exposed file in your project to save your data, this is fine in local environment, but when deploying you need to consider that the server and the database are in the same machine and using the same disk. that means if you accidentally remove the machine -and its disk space- used to serve the application, then the database itself will be deleted with all records.
Plus you will face problems if you tried to scale your servers, that is every server will have his own copy of the database and syncing all those files will be huge headache.
If your data is not that important then you can keep using sqlite, but if you are expecting high traffic and complex db structure, then I would recommend you consider a db engine like Mysql or maybe look up the databases offered by amazon here:
https://aws.amazon.com/products/databases/
For django, you will need to change the adapter when using a different db like mysql, sqlite or anything else.
https://docs.djangoproject.com/en/3.0/ref/databases/

Django app as database web based UI

I'm planning to develop web UI something like iSQL*Plus-oracle but, for most common databases. Just take input query from user and return result with save options,etc.
So, for connecting to external data bases, what is advisable way,
1. Using django model and raw sql or,
2. with modules outside django - sqlalchemy+mysqldb,psycopg..?
Going through django documentation my understanding is db connections has to be in settings.py and I could not add from user input. Is my understanding is true or not?
I'm new to django not to python.

An ORM (something like Django's models or sqlalchemy) is a really helpful abstraction to help map tabular data in the database to the objects its being used to model in your code. It won't help with connecting to databases provided by the user since you won't know what the schema of the database is you're connecting to, nor what you are going to receive back from a query.
With django, the database defined in settings.py is used to store information related to your app such as user credentials, migrations as well as whatever else you define in your models.py files. So definitely don't try to change that dynamically as it is being used to store the state of your application for all users.
If you need to connect to external databases and run user-supplied queries, you can do that inside a view using the appropriate database driver. So psycopg2 for postgres would be fine.

Lazy psql connection with Django

I have a Django app that has several database backends - all connected to different instances of Postgresql database. One of them is not guaranteed to be always online. It even can be offline when application starts up.
Can I somehow configure Django to use lazy connections? I would like to:
Try querying
return "sorry, try again later" if database is offline
or return the results if database is online
Is this possible?

The original confusion is that Django tries to connect to its databases on startup. This is actually not true. Django does not connect to database, until some app tries to access the database.
Since my web application uses auth and site apps, it looks like it tries to connect on startup. But its not tied to startup, its tied to the fact that those app access the database "early".
If one defines second database backend (non-default), then Django will not try connecting to it unless application tries to query it.
So the solution was very trivial - originally I had one database that hosted both auth/site data and also "real" data that I've exposed to users. I wanted to make "real" database connection to be volatile. So I've defined separate psql backend for it and switched default backend to sqlite.
Now when trying to access "real" database through Query, I can easily wrap it with try/except and handle "Sorry, try again later" over to the user.