Extract metadata from database using python? - python

I have looked for an answer to this question, but very was able to find very little. I want to extract the names of the tables, references between them, column names so I can graphically visualize that information. I need to this in a Django project.
Since I am a newbie to python I would like to know if there is some kind of API to do this type of thing.
Edit
I have created a model which consists Node, Attribute and Link. Node has attributes, while Link has fields parent_node and child_node. What I want is to connect to a database, read the metadata by which I mean: Table names, Column names and Foreign key constraints. Then I could properly put this data in the model I have created.

You can use inspectdbcommand, with this django reads your database and create models for each table and if your database has relations you get also in django. You can see more info here.
python manage.py inspectdb

Related

Can we create a table in django without models class?

I need to dynamically create database tables depending on user requirements. so apart from a few predefined databases, all other databases should be created at runtime after taking table characteristics(like no of cols, primary key etc.) from user.
I read a bit of docs, and know about django.db.connection but all examples there are only for adding data to a database, not creating tables. (ref: https://docs.djangoproject.com/en/4.0/topics/db/sql/#executing-custom-sql-directly)
So is there anyway to create tables without models in django, this condition is a must, so if not possible with django, which other framework should I look at?
note: I am not good at writing questions, ask if any other info is needed.
Thanks!
You can use inspectdb to automatically generate the models from the legacy database. You can check about it in here.
Or you can use SQL directly. Although, you will have to process the tables in python. Check it here.

How to populate my django database with json that I scraped from a website

I have scraped data from a website using their API on a Django application. The data is JSON (a Python dictionary when I retrieve it on my end). The data has many, many fields. I want to store them in a database, so that I can create endpoints that will allow for lookup and modifications (updates). I need to use their fields to create the structure of my database. Any help on this issue or on how to tackle it would be greatly appreciated. I apologize if my question is not concise enough, please let me know if there is anything I need to specify.
I have seen many, many people saying to just populate it, such as this example How to populate a Django sqlite3 database. The issue is, there are so many fields that I cannot go and actually create the django model fields myself. From what I have read, it seems like I may be able to use serializers.ModelSerializer, although that seems to just populate a pre-existing db with already defined model.
Tricky to answer without details, but I would consider doing this in two steps - first, convert your json data to a database schema, for example using a tool like sqlify: https://sqlify.io/convert/json/to/sqlite
Then, create a database from the generated schema file, and use inspectdb to generate your django models: https://docs.djangoproject.com/en/2.2/ref/django-admin/#inspectdb
You'll probably need to tweak the generated schema and/or models, but this should go a long way towards automating the process.
I would go for a document database, like Elasticsearch or MongoDB.
Those are made for this kind of situation, look it up.

Is there a way to standardize the business datatypes across all systems in organization?

Can we do a loosely coupled data access layer design in python?
Lets say,in a scenario i have an oracle table with column name ACTIVITY_ID with column datatype as Number(10). If this column is a foreign key in lot many tables,to hold the data of this column, can i create something like ACTID class (like a java object) and can be used across the code if i want to manipulate/hold the ACTIVITY_ID column data so that i could maintain consistency of business object columns. Is there any such possibility in python ?
Try Django
As I understand it, Python does not natively have any database functionality. There are many different libraries/frameworks/etc. that can be used to provide database functionality to Python. I recommend taking a look at Django. With Django, you create a class for each database table and Django hides a LOT of the details, including allowing using with multiple database engines such as MySQL and PostgreSQL. Django handles Foreign Key relationships very well. Every table normally has a primary key, by default an auto-incremented id field. If you add a field like activity = models.ForeignKey(Activity) then you now have a foreign key field activity_id in one table referencing the primary key field id in the Activity table. The Admin page will take care of cascading deletion of records if you delete an Activity record, and in general things "just work" the way you might expect them to.

Storing an Object in a Database

I have little to no experience with databases and i'm wondering how i would go about storing certain parts of an object.
Let's say I have an object like the following and steps can be an arbitrary length. How would I store these steps or list of steps into an sql database?
class Error:
name = "" #name of error
steps = [] #steps to take to attempt to solve error
For your example you would create a table called Errors with metadata about the error such as an error_ID as the primary key, a name, date created, etc... then you'd create another table called Steps with it's own id, lets say Step_ID and any fields related to the step. The important part is you'd create a field on the Steps table that relates back to the Error that the steps are for we'll call that field again error_ID, then you'd make that field a foreign key so the database enforces that constraint.
If you want to store your Python objects in a database (or any other language objects in a database) the place to start is a good ORM (Object-Relational Mapper). For example Django has a built-in ORM. This link has a comparison of some Python Object-Relational mappers.

Loading data from a (MySQL) database into Django without models

This might sound like a bit of an odd question - but is it possible to load data from a (in this case MySQL) table to be used in Django without the need for a model to be present?
I realise this isn't really the Django way, but given my current scenario, I don't really know how better to solve the problem.
I'm working on a site, which for one aspect makes use of a table of data which has been bought from a third party. The columns of interest are liklely to remain stable, however the structure of the table could change with subsequent updates to the data set. The table is also massive (in terms of columns) - so I'm not keen on typing out each field in the model one-by-one. I'd also like to leave the table intact - so coming up with a model which represents the set of columns I am interested in is not really an ideal solution.
Ideally, I want to have this table in a database somewhere (possibly separate to the main site database) and access its contents directly using SQL.
You can always execute raw SQL directly against the database: see the docs.
There is one feature called inspectdb in Django. for legacy databases like MySQL , it creates models automatically by inspecting your db tables. it stored in our app files as models.py. so we don't need to type all column manually.But read the documentation carefully before creating the models because it may affect the DB data ...i hope this will be useful for you.
I guess you can use any SQL library available for Python. For example : http://www.sqlalchemy.org/
You have just then to connect to your database, perform your request and use the datas at your will. I think you can't use Django without their model system, but nothing prevents you from using another library for this in parallel.

Categories

Resources