This is my model.
class Otp(models.Model):
user = models.OneToOneField('CustomUser', on_delete=models.CASCADE)
onetime = models.CharField(max_length=25, default=calculated, unique=True)
link = models.UUIDField(default=uuid.uuid4, unique=True)
created_at = models.DateTimeField(auto_now_add=True)
I just added the column 'link'. Workes great on my development environment.
When on my server I go inte venv and run 'makemigrations' see the changes. Then 'migrate' and everything applied.
Now when I access a page that use the table, I get 'column does not exist'.
I go inte postgres and check the table. No changes was applied. The new column is missing.
I try adding, removing forth and back. The migrations are flawless, but there are no changes to the table.
I even droppped the table. Re ran migrations to create it. It says no changes.
I'm strting to think the migrations are applying it to another database. But how can I check this. It should not be another. I only have one.
EDIT with update on my investigation.
My database connection is set in the settings in production.py for postgres. My development.py is sqlite.
So now I checked the db for sqlite on my server. And for sure, the migrations are applied on sqlite, not postgres.
how come it's choosing sqlite all of a sudden?
This is how I apply changes.
1. Make changes in models.py for app
2. Activate venv to be able to run migrations
3. Run 'python manage.py makemigrations APPNAME'
4. Run 'python manage.py migrate APPNAME'
So what is defining it to run migrations towards sqlite and not postgres?
Your description does suggest that you are interfacing with a different database than you expect on your server. I'd recommend checking the DATABASES setting in your Django settings file. The credentials you provide there will likely need to be different in your production and development environments.
In response to you most recent edit, make sure that you are running your Django instance on your server using your production settings. Based on what you have described, I assume your wsgi.py file includes a line resembling
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.development")
Django itself has no notion of development vs production settings. It only knows to load the setting module that you tell it to.
When running in your production environment, you need to explicitly specify the use of your "production settings". This can be accomplished most easily by setting the environment variable DJANGO_SETTINGS_MODULE to the Python path of your settings module. Do note that this must be accomplished by the web server that runs your Django app. Simply sshing in and running export DJANGO_SETTINGS_MODULE=myapp.production won't cut it.
A common practice is to configure your wsgi.py endpoint to default to the production settings, and to explicitly opt for the development settings on your local machine, e.g. via
$ ./manage.py --settings=myapp.development_settings <command>
Related
I'm currently deploying a project to Heroku, however it returns me the following error: ValueError: Related model 'store.user' cannot be resolved.
But there is a way to avoid such error locally. You simply do:
py manage.py migrate store
And then
py manage.py migrate
In other words, if I migrate separetely, I won't face such error. However, Heroku migrates all together, then the deploy fails because of this error.
If I proceed locally as Heroku does, i.e. run
py manage.py migrate
I can effortlessly click on and delete the db.sqlite3 file and makemigrations and migrate separetely. Then the issue would be resolved. However, that's not possible when deploying to Heroku. Thus, how can I delete this file only via terminal? I have been searching, but people only say you click on the file and the delete it, which for me it is not possible.
Thanks
However, Heroku migrates all together, then the deploy fails because of this error
Heroku does nothing of the sort.
Migrations only run on Heroku when you tell them to, either by running something like
heroku run python manage.py migrate
or because you have declared a release process that should run when you deploy a new version, e.g.:
web: gunicorn app.wsgi
release: python manage.py migrate
In both cases you are in charge of what that command is. If you don't want to run python manage.py migrate every time you deploy, remove the release process from your Procfile.
if I migrate separately, I won't face such error
This is concerning.
There could be several causes for this. One common issue is missing dependencies in your migration. If you write a migration in app1 that depends on certain models and tables from app2 existing, you need to add a dependency to the relevant migration in app1, e.g. something like this example from the documentation:
class Migration(migrations.Migration):
dependencies = [
('app1', '0001_initial'),
# added dependency to enable using models from app2 in move_m1
('app2', '0004_foobar'),
]
operations = [
migrations.RunPython(move_m1),
]
If migrations are written properly, such that they accurately reflect the code in each commit, build off each other, have dependencies declared, etc. you should always be able to safely apply them to your database.
Finally, you ask about deleting db.sqlite.
Heroku's filesystem is ephemeral. It gets baked into your application slug at build time, and any changes you make to it get reset every time your dyno restarts. This happens frequently (at least once per day).
This means you can't effectively delete your database file. But it also means you can't save data and expect it to be there when you look it up later! SQLite isn't a good fit for Heroku.
A client-server database like PostgreSQL is a much better choice. Heroku offers its own Postgres service with a free tier.
If you switch, I strongly urge you to switch in development as well. Django's ORM makes it relatively easy to change, but database engines aren't drop-in replacements for each other. I've seen real-world examples where developing on SQLite and deploying to Postgres or MySQL causes things to fail in production that worked in development.
I have added a simple model to my reporter app in my django project
class Municipalities(models.Model):
namelsad = models.CharField(max_length=100)
geom = gismodels.MultiPolygonField(srid=4326)
when I python3 manage.py makemigrations reporter
it says no changes detected in reporter
then when I python3 manage.py migrate reporter
Operations to perform:
Apply all migrations: reporter
Running migrations:
No migrations to apply.
but there is no postgresql database table of reporter_municipalities
reporter is included in installed apps
Municipalities model is in the model.py file in reporter app
I should add I had a counties table and manually deleted it in postgresql and tried adding the municipalities model to create a table
municipalities is also in the django_content_type
but there is no table of municipalities
update
Changed Class Municipalities to Class Muni
python3 manage.py makemigrations reporter
then it asked me
Did you rename the reporter.Municipalities model to Muni? [y/N]
if I click y
then run migrate
gives me
django.db.utils.ProgrammingError: table "reporter_municipalities" does not exist
but I am so confused that table does not and never existed!!
I cannot migrate at all to the DB now because of "reporter_municipalities"
When Django looks for migrations, it searches for the app reporter as is it configured in your INSTALLED_APPS. It uses AppConfig or application name to retrieve the full package name, ie my_project.reporter. This package must be available in your PYTHONPATH. It should be available only in you development project, but could happen that it is "installed" in your virtualenv. It can happen running pip install . (without -e) or (with some configurations) running tests (I have seen this happen with tox).
In this scenario you have two my_project.reporter available to python, but you edit/update the "second" one. When you run ./manage.py makemigrations, python first find the code that you did not change (the installed one in your virtualenv), so it does not find the updates.
To check if you have this duplication, you can:
uninstall your project (pip uninstall <django-project>) to see if it refers to a "symbolic link"
If this is the case you will see something like
Uninstalling <PROJECT>:
Would remove:
/data/PROGETTI/UNICEF/odk/.venv/lib/python3.6/site-packages/<PROJECT>.egg-link
Proceed (y/n)? y
Note the egg-link at the end of the line.
OR
open a shell and type import <full_path_of_reporter> AS pkg; print(pkg.__file__) and check the filepath
Sorry I can't give a very technical answer but I got round this my creating a table with the same name directly via pgAdmin. Making and running migrations. At this point the table was recognised but obviously didn't work correctly because it had no columns. I then commented out the model in models.py and made ran migrations which removed the table. I checked on pgAdmin that the table had been removed by this point. I then deleted the migrations files (no idea if this was necessary), uncommented the model in models.py and then made and ran migrations again. Worked fine after that.
I'm late to the question but for people facing this in future, there are files inside the pycache folder inside the migrations folder with .pyc extension. Just delete those files if they are already there and run from starting, i.e., start from makemigrations again and hopefully you'll get it. I got it the same way by deleting the files in migrations folder other than init and deleting the files inside pycache folder other than init.
ty running python ./manage.py makemigrations "app_name" as the last argument, then the migrate command. this worked for me when having the same issue.
I have flask-migrate (version 1.8.0) working well with a sqlite database in a development environment. Now I would like to migrate our data to MySQL and maintain all of our migration history (so it stays in sync with our Flask-SQLAlchemy models in our git repository).
I created an empty MySQL database, and after changing my SQLALCHEMY_DATABASE_URI, I tried running:
python manage.py db upgrade
That resulted in an error about not being able to drop the table migrate_version. (Which makes sense, since this is a new database, although sqlite actually contains the table 'alembic_version' not 'migrate_version'.)
So, I tried to initialize this new database:
python manage.py db init
Now I get an error: "Directory migrations already exists".
I can rename that folder and re-run the command with no problem, but then I lose all of my previous migrations. I think we would have the same issues when we also transition to our test and production environments.
I've seen in the docs Flask-Migrate has multiple database support, but I think that looks to be more for maintaining multiple databases in a single development environment. Is there a way to have Flask-Migrate track changes across multiple development environments?
To address the real issue in the OP's question, you need to use the --directory flag to initiate a migrations directory specific to your each environment's database.
From the flask-migrate documentation:
All commands also take a --directory DIRECTORY option that points to
the directory containing the migration scripts. If this argument is
omitted the directory used is migrations.
So:
flask db init --directory=[DIRECTORY NAME]
Flask-Migrate itself has no memory of your database, so when running migration commands with flask db, it will reference the specified migrations directory (by default, when the --directory flag is not used, this is called 'migrations').
flask db migrate --directory=[DIRECTORY_NAME]
etc.
It goes without saying that the flask command will reference the application context as configured by your config file or environment variables.
I typically create a migration directory for each environment with an explicit reference to the environment: e.g. development and staging, with something like 'migrations_dev' and 'migrations_stg'.
Hope this is helpful.
Here are the steps I took to transition from SQLite to MySQL and maintain all the migration history. I highly suspect there is a better way to do this, but it worked for me.
Initialize the new, blank database using another folder for your "new" migrations
python manage.py db init -d tmp
Create a migration
python manage.py db migrate -d tmp -m "Bring MySQL up to date"
Apply the migration
python maange.py db upgrade -d tmp
Now, you can delete the "tmp" migrations folder. You no longer need it. Find the HEAD migration. Look for 'Rev: your_revision_num (head)'
python manage.py db show
Run an update statement against your MySQL database
update alembic_version set version_num = 'your_revision_num'
Now your MySQL database schema should match your old SQLite schema and you'll have your full migration history as well.
The table migrate_version is used to track migrations by package sqlalchemy-migrate. Alembic, the package used by Flask-Migrate, uses a alembic_version table, as you know.
So my guess, is that this MySQL database that you want to use has been used before by an application that was under sqlalchemy-migrate control.
I recommend that you delete the MySQL database and make a brand new one.
I also had the same need on my side. I wanted to reproduce the command that exists in the laravel framework to make a migration in different environments:
php artisan migrate --env prod
With this kind of command, you can launch a migration in different environments. I have not found a directly equivalent command in flask.
THE "flask db upgrade --env prod" command does not exist. In particular, the --env argument .
As a workaround, I created a command that allows me to change the environment:
flask env --name prod
That command is a custom flask command that will copy the content of the .env.prod file to .env.
This way, the application is in the prod environment. And we can then launch the migration command on the prod environment.
How to use the custom env command to migrate to different environments?
To start the migration in the staging environment, just run these two commands:
flask env --name staging
flask db updgrade
Then if you want to start the migration in the production environment, just run these two commands:
flask env --name prod
flask db updgrade
How to create the custom command flask env?
First, you need to know how to create custom command in flask. Just follow the official falsk documentation
Here is the content of my custom command which allows to change the environment:
from flask.cli import with_appcontext
import shutil
#click.command('env')
#click.option("--name", is_flag=False, flag_value="Flag", default="")
#with_appcontext
def env(name):
if name == '':
print(os.getenv('FLASK_ENV'))
return
if name not in ['dev', 'prod', 'local', 'staging']:
print('That env does not exist')
return
shutil.copyfile('.env.' + name, '.env')
return
In my setup, I have 4 environments: local, dev, staging, prod.
And I have 4 corresponding .env files: .env.local, .env.staging, .env.prod, .env.dev
The custom flask env command also copies the contents of the environment files into the .env file that the flask application loads at start-up.
I was following the documentation to get python social auth on my django project
https://python-social-auth.readthedocs.org/en/latest/configuration/django.html
And after adding 'social.apps.django_app.default', to the INSTALLED_APPS in my settings.py I run this:
python manage.py makemigrations
I get this
No changes detected
Shouldn't this command be doing something. Because without this I can't migrate to create the tables that are needed for the auth.
EDIT:
I've also tried this command, and still ended up getting the same result
python manage.py makemigrations main
where 'main' is the name of my app
Today i ran into this problem. The error is in the documentation itself.
You should run $ python manage.py migrate directly. It creates tables in the database.
All the old tutorials used makemigrations, I think it was used in earlier versions of django.
My answer will be cover some basics so one can understand this types of errors easily.
Let me clear some basic terminology about migrations in the latest versions of Django(From 1.7 to under development ones).
In older versions of Django when One has to make changes in Models (Eventually in Database) then One has to use South app to apply changes in Database without affecting older database.
Django developer community has included this south app in and after Django 1.7 and also provided some simple commands to apply migrations.
When one install New app(Above question scenario) or one make changes in existing models and wish to apply changes in database then one has to tell database about what changes they want to make. To do so one has to make migrations and below is the command.
$ python manage.py makemigrations app_name
or If it is initial then no need to specify app_name, it will consider all apps.
This command will generate migrations files which will include instructions for database to make what tables and what are the attributes of that table and what will be the relationships between tables and what are changes in the current tables etc etc.
Now one has to run below command to apply this all changes in database.
$ python manage.py migrate app_name
or If it is initial then no need to specify app_name.
Please shoot any questions If you have any and please take a look at Django's official documentation on migrations for more information.
The possible reason is your project doesn't use any database models of 'social' application yet. Add a URL in your urls.py, link it to 'social' urls.
to Django < 1.8
INSTALLED_APPS ['social.apps.Django_appConfig',]
I want to revert my last migration (0157) by running its Migration.backwards() method. Since I am reverting the migration in production server I want to run it automatically during code deployment. Deployment script executes these steps:
Pull code changes
Run migrations: manage.py migrate <app>
Refresh Apache to use newest code: touch django.wsgi
If I could, I would create new migration file which would tell South to backward migrate to 0156:
migrations/0158_backward__migrate_to_0156.py
This commited migration would be deployed to production and executed during manage.py migrate <app> command. In this case I wouldn't have to execute backward migration by hand, like suggested in these answers.
Lets say, I have created two data migrations, first for user's Payment, second for User model. I have implemented backwards() methods for both migrations in case I'd have to revert these data migrations. I've deployed these two migrations to production. And suddenly find out that Payment migration contains an error. I want to revert my two last data migrations as fast as possible. What is the fastest safe way to do it?
Since I am reverting the migration in production server I want to run
it automatically during code deployment.
IMHO the safest path is
run manage.py migrate <app> (i.e. apply all existing migrations, i.e. up to 0156)
undo the changes in your model
run manage.py schemamigration <app> --auto
This will create a new migration 0157 that effectively reverts the previous migration 0156. Then simply apply the new migration by running manage.py migrate <app> again. As I understand, your code deployment will just do that.
Apparently the codeline has migrations up to #157 and now the developer decided that the last one was not a good idea after all. So the plan is to go back to #156.
Two scenarios:
(a) migration #157 was not released or deployed anywhere yet.
Simply revert the last change from models.py and delete migration #157.py from the source archive. Any deployment will take the system to level 156; "157 was never there".
(b) there have been deployments of the latest software with migration #157.
In this case the previous strategy will obviously not work. So you need to create a migration #158 to undo #157. Revert the change in models.py and run
django manage.py migrate <app> 0157
django manage.py schemamigration <app> --auto
This will auto-generate a new migration #158, which will contain the inverse schema migration compared to #157.
If schemamigration is giving trouble because of django Model validation (something that can happen if you have custom validators which check stuff outside the ORM box), I suggest the following workaround:
<django project>/<app>/management/commands/checkmigrations.py
from south.management.commands import schemamigration
class Command(schemamigration.Command):
requires_model_validation = False
help = "schemamigration without model validation"
This command becomes available in manage.py:
django manage.py checkmigrations <app> --auto
There's no silver bullet here. The simplest solution I can think of would be to - in your dev env of course - manually migrate back to 0156, manually update your migration's history table (sorry I can't remember the table's name now) to fool south in thinking you're still #0158, then run schemamigration again. Not garanteed to work but might be worth trying.