I am supporting a software that uses SQLAlchemy database on Python.
I need to add the column to this database. As I understand, I need to run migration on it to do this.
I am new to working with databases, so not sure how it would affect the database.
Will the people who currently use this software also need to run migration on their own environments to keep their database structure up-to-date?
Is there a way to get the user's darabases updated automatically?
Thank you
I have used Liquibase an a number of projects to solve this database migration problem, and found it to be highly effective.
Related
Its's nice to see mongodb is connected with django but doesn't support completely. I explain why so because when you install any third party packages to your project it could written for Django models not Djongo which will fall you in Database error while migrating you apps.
djongo.database.DatabaseError
Do we have any solution for this case?
Well, in the general case, the solutions you have are, in increasing order of difficulty:
Get rid of Djongo. Use Django with an SQL database, as that's what it's designed for. You can still use MongoDB for whatever data you might have in there, but the main Django database is better off being SQL.
Don't use packages that don't work with Djongo.
Fix the packages that don't work with Djongo to work with it.
I would recommend going for (1)...
Is there any library or tool to perform database migrations in python without SQLAlchemy? I am using a PostgreSQL database and the queries are raw SQL. I use psycopg2 for the database connection.
In short, "yes."
There are numerous possibilities. The one you choose will depend on doing some homework to determine which one best suits your needs.
https://github.com/amacneil/dbmate
https://pypi.org/project/yoyo-migrations/
https://pypi.org/project/alembic/
Suggest when asking questions that can be answered with a google search, to be more specific about your situation, to help answerers to understand why Google couldn't answer your question in the first page of results.
If you can bring yourself to do your db migrations outside of the CLR, I'd strongly recommend considering FlyWay. It has been around forever, it's supported by RedGate, and the community version is free and open source. It has built-in support for all the major db platforms and installs like any other utility you'd use. For a Python app you'd do all of your db migrations via shell scripts or process calls instead of directly in the code.
My team settled on this tech because:
Yoyo is pretty immature and buggy for us
We don't use SQLAlchemy's OR framework
Alembic's syntax and framework seems pretty heavyweight for us
Under FlyWay's model we just write SQL scripts and check them into a directory. It's really lightweight.
It's been proven to work nicely in clustered environments
Give it a look.
This is a common practice that people working on django project usually push migrations to the version control system along with other code.
My question is why this practice is so common? Why not just push the updated models and everyone generate migrations locally. This approach can reduce the effort for resolving migrations conflicts too.
If you didn't commit them to a VCS then what would happen is people would make potentially conflicting changes to the model.
When finally ready to deploy, you would still need django to make new migrations that would then merge everybodys changes together. And this just creates an additional unnecessary step that can introduce bugs.
You also are assuming everybody will always be able to work on an up to date version of the code which isn't always possible when you start working on branches that are not ready to be merged into mainline.
Migrations synchronize the state of your database with the state of your code. If you don't check in the migrations into version control, you lose the intermediate steps. You won't be able to go back in the version control history and just run the code, as the database won't match the models at that point in time.
Migrations, like any code, should be tested, at the very least on a basic level. Even though they are auto-generated, that's not a guarantee that they will work 100% of the time. So the safe path is to create the migrations in your development environment, test them, and then push them to the production environment to apply them there.
Firstly, migrations in version control allows you to run them in production.
Secondly, migrations are not always automatically generated. For example, if you add a new field to a model, you might write a migration to populate the field. That migration cannot be re-created from the models. If that migration is not in version control, then no-one else will be able to run it.
I'm currently exploring using python to develop my server-side implementation. I've decided to use SQLAlchemy for database stuff.
What I'm not currently to sure about is how it should be set up so that more than one developer can work on the project. For the code it is not a problem but how do I handle the database modifications? How do the users sync databases and how should potential data be set up? Should/can each developer use their own sqlite db for development?
For production postgresql will be used but the developers must be able to work offline.
You can use Alembic to manage your DB structure. Keep your migrations under version control in order to ensure that it's easy for developers to keep in sync with changes.
As regards data itself, you can either work with a set of fixtures, or you could write some Fabric commands to import/export Postgres dumps, which you could also keep under version control if necessary – depending on data size, you may not want to keep a large uncompressed SQL dump under version control (though it will compress well using git or hg), however. This aspect of your question depends on the size of your development team, and the size of your database.
Make sure you have a python programs or programs to fill databases with test data from scratch. It allows each developer to work from different starting points, but also test with the same environment.
i am new in python.before this i done some work in php with codeigniter framework.
now i want to develop website using python and sqlite3.
so which framework will you suggest me.
i also want to know IDE for sqlite.
Do you mean something similar to sqliteman or dbeaver? These are not not IDEs for SQLite, but instead, are GUI interfaces. There are more here: http://www.google.com/search?&q=sqlite+gui
3rd link down is another stack overflow question: What are good open source GUI SQLite database managers?
For Python web frameworks, a very popular full-stack framework is the Django Project
Check out web2py. It's a feature-rich full-stack framework, but it's also very easy to set up, learn, and use. It requires no installation or configuration, has no dependencies, and includes its own web server and web-based IDE/administrative interface (see demo). If you download the binary version for Windows or Mac, it even includes its own Python interpreter (so you don't need to have Python installed).
The database abstraction layer (DAL) works with 10 different database backends, including SQLite. It doesn't include its own SQLite IDE, though it does include a database administrative interface called appadmin, which allows you to view all database tables; view, insert, update, and delete records within each table; and import/export tables to CSV. You can also import and export the entire database to CSV. (Note, these features work for all database backends, not just SQLite.)
You will probably also find that you don't really need an IDE for SQLite when using web2py because the DAL takes care of everything for you. You specify all of your data models using the DAL syntax, and then the DAL automatically creates the SQLite database, creates all tables, and handles migrations when you changes your models.
If you have any questions, there's a friendly and responsive mailing list that will provide lots of help.