Develop Reference

UPDATE statement on table 'xxx' expected to update 1 row(s); 2 were matched [duplicate]

I am running a Pyramid + Zope transaction manager + SQLAlchemy + PostgreSQL. On some occasions, I have seen StaleDataError error on a Pyramid web application which should very trivial view of updating one row in a database. As the error happens outside the normal view boundary and is not repeatable, it is quite tricky to debug.
I guess this might have something to do with broken database connections or transaction lifecycle. However I don't know how to start debugging the system, so I am asking what could cause this and furthermore how one can pin down errors like this.
UPDATE statement on table 'xxx' expected to update 1 row(s); 0 were matched.
Stacktrace (most recent call last):
File "pyramid/tweens.py", line 20, in excview_tween
response = handler(request)
File "pyramid_tm/__init__.py", line 94, in tm_tween
reraise(*exc_info)
File "pyramid_tm/compat.py", line 15, in reraise
raise value
File "pyramid_tm/__init__.py", line 82, in tm_tween
manager.commit()
File "transaction/_manager.py", line 111, in commit
return self.get().commit()
File "transaction/_transaction.py", line 280, in commit
reraise(t, v, tb)
File "transaction/_compat.py", line 55, in reraise
raise value
File "transaction/_transaction.py", line 271, in commit
self._commitResources()
File "transaction/_transaction.py", line 417, in _commitResources
reraise(t, v, tb)
File "transaction/_compat.py", line 55, in reraise
raise value
File "transaction/_transaction.py", line 389, in _commitResources
rm.tpc_begin(self)
File "/srv/pyramid/trees/venv/lib/python3.4/site-packages/zope/sqlalchemy/datamanager.py", line 90, in tpc_begin
self.session.flush()
File "sqlalchemy/orm/session.py", line 2004, in flush
self._flush(objects)
File "sqlalchemy/orm/session.py", line 2122, in _flush
transaction.rollback(_capture_exception=True)
File "sqlalchemy/util/langhelpers.py", line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "sqlalchemy/util/compat.py", line 182, in reraise
raise value
File "sqlalchemy/orm/session.py", line 2086, in _flush
flush_context.execute()
File "sqlalchemy/orm/unitofwork.py", line 373, in execute
rec.execute(self)
File "sqlalchemy/orm/unitofwork.py", line 532, in execute
uow
File "sqlalchemy/orm/persistence.py", line 170, in save_obj
mapper, table, update)
File "sqlalchemy/orm/persistence.py", line 692, in _emit_update_statements
(table.description, len(records), rows))

This is most likely scenario:
You have 2 requests that first select an object and try to update/delete it in the datastore and you end up with a "race condition".
Lets say you want to do something like fetch an object and then update it.
If transaction takes some time and you do not select the object with "for update" thus locking the rows - if the object gets deleted in first request and 2nd transaction tries to issue update to row that is not present in the db anymore you can end up with this exception.
You can try doing some row locking to prevent this from happening - subsequent transaction will "wait" for the first operation to finish. Before it gets executed.
http://docs.sqlalchemy.org/en/rel_1_0/orm/query.html?highlight=for_update#sqlalchemy.orm.query.Query.with_for_update
and
http://docs.sqlalchemy.org/en/rel_1_0/orm/query.html?highlight=with_lockmode#sqlalchemy.orm.query.Query.with_lockmode
Describe some of the sqlalchemy machinery you can use to resolve this.

Another option:
TL,DR: if you have "first()" in some cases, you need to remove this in alchemy if you update multiple records
db.session.query(xxx).filter_by(field=value).first()
This command expects the update to affect only one line. And it should if your table has only one record with field=value. This is particularly the case if the field is your ID.
HOWEVER - if your ID is not unique, you might have multiple records with the same ID.
In this case, your can update all by removing "first()"
BTW, use the following to debug your SQL queries (which wouldn't have helped this time...)
import logging
logging.basicConfig()
logging.getLogger('sqlalchemy.engine').setLevel(logging.INFO)

how i can fix the errors of migrate command

i'm new in django
i want to create a businessplan application
i create 3 classes in my models file "models.py"
when i run python manage.py migrates
it show me this errors:
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "C:\Users\hp\PycharmProjects\business\lib\site-packages\django\core\management\__init__.py",
line 401, in execute_from_command_line
utility.execute()
File "C:\Users\hp\PycharmProjects\business\lib\site-packages\django\core\management\__init__.py",
line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\hp\PycharmProjects\business\lib\site-packages\django\core\management\base.py", line
328, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\hp\PycharmProjects\business\lib\site-packages\django\core\management\base.py", line
369, in execute
output = self.handle(*args, **options)
File "C:\Users\hp\PycharmProjects\business\lib\site-packages\django\core\management\base.py", line
83, in wrapped
res = handle_func(*args, **kwargs)
File "C:\Users\hp\PycharmProjects\business\lib\site-
packages\django\core\management\commands\migrate.py", line 89, in handle
executor.loader.check_consistent_history(connection)
File "C:\Users\hp\PycharmProjects\business\lib\site-packages\django\db\migrations\loader.py", line
295, in check_consistent_history
raise InconsistentMigrationHistory(
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration admin.0001_initial is applied
before its dependency businessplan.0001_initial on database 'de
fault'.
this is my models.py :
it contain 4 classes Entrepreneur ,Admin , User(abstract) and projet:
from django.db import models
from django.contrib.auth.models import *
#user
class CustomUser(AbstractUser):
user_type=((1,"admin"),(2,"staff"))
user_type=models.CharField(default=1,choices=user_type,max_length=10)
# Entrepreneur
class Entrepreneur(models.Model):
id_models= models.IntegerField(primary_key=True)
admin=models.OneToOneField(CustomUser,on_delete=models.CASCADE)
nom_et_prenom=models.CharField(max_length=50)
date_naissance=models.DateField()
adresse_entr=models.CharField(max_length=20)
telephone=models.IntegerField()
statut_social=(('ce','celébataire'),
('ma','marié'),
('di','divorcé'),
('ve','veuf'),
)
occupation=models.CharField(max_length=50)
niveau_scolaire=(
('pri','primaire'),
('sec','secondaire'),
('cap','certificat aptitude professionel'),
('btp','brevet technicien professionel'),
('bts','brevet technicien superieur'),
('lic','license'),
('mai','maitrise'),
('mas','mastere'),
('doc','doctorat'),
)
niveau_scolaire=models.CharField(default='ser',choices=niveau_scolaire,max_length=50)
annnee_exp=models.IntegerField()
email=models.CharField(max_length=255)
password=models.CharField(max_length=255)
#Projet
class Projet(models.Model):
id_models=models.IntegerField(primary_key=True)
admin = models.OneToOneField(CustomUser, on_delete=models.CASCADE)
nom_projet=models.CharField(max_length=30)
lieu_implantation=models.CharField(max_length=20)
type_projet=(
('ser','service'),
('com','commerce'),
('agr','agricuture'),
('ind','industrie'),
('IT','technologie information'),
('art','artisanat'),
('tor','tourisme'),
)
type_projet=models.CharField(default='ser',choices=type_projet,max_length=50)
produit=models.CharField(max_length=50)
id_promoteur=models.ForeignKey(Entrepreneur,on_delete=models.CASCADE)
#admin
class Admin(models.Model):
id_admin=models.AutoField(primary_key=True)
name=models.CharField(max_length=255)
email=models.CharField(max_length=255)
password=models.CharField(max_length=255)
objects=models.Manager()
first i run makemigration command and everythings is ok
but when i run migrate command it's not ok

First of all delete the migration files under the migrations folder of the app excluding the init file.If this will not work then do the same process and delete rows related to that app in django migrartions table. Then try to migrate. It will work. Happy CODING :D

You can delete the database then migration files, then make migrations and migrate to have a new consistent database. Don't forget to store any valuable data you have in the database.

solution is that delete all migration file from app/migrations folder.
hit python3 manage.py makemigration
It's create fresh migration file

django.db.utils.OperationalError: 1005, 'Can't create table `xyz`.`#sql-600_237` (errno: 150 "Foreign key constraint is incorrectly formed")

I am trying to add One-to-One keys into my Django app, but I always get that error when I try to "migrate" process (makemigrations works great).
Applying xyzapp.0007_personne_extended_foreign...Traceback (most recent call last):
File "manage.py", line 29, in <module>
execute_from_command_line(sys.argv)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
utility.execute()
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
output = self.handle(*args, **options)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
state = migration.apply(state, schema_editor)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 91, in __exit__
self.execute(sql)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 111, in execute
cursor.execute(sql, params)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/opt/xyz/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
django.db.utils.OperationalError: (1005, 'Can\'t create table `xyz`.`#sql-600_297` (errno: 150 "Foreign key constraint is incorrectly formed")')
That's what my models looks like :
class PersonVolunteer(models.Model):
person = models.OneToOneField(Person)
class Person(models.Model):
name = models.CharField(max_length=100)
And the migration process that cause the crash :
class Migration(migrations.Migration):
dependencies = [
('xyzapp', '0014_member_to_person'),
]
operations = [
migrations.CreateModel(
name='PersonVolunteer',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('personne', models.OneToOneField(to='xyzapp.Personne')),
],
),
]
However, all works correctly when I test it even after the migrate error. But if that error message appears during "migrate", that should not be something good happening. If there's no problem, it is possible to skip that last step which crash my migration ?
Can someone can say me why do I get that error message and how I can resolve it ?
Thank you very much and have a great day !

I have found a solution, may be not the cleaner, but god dammit it works, that's perfect for me.
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('cvmapp', '0006_person_extended'),
]
operations = [
migrations.AddField(
model_name='PersonVolunteer',
name='personne',
field=models.OneToOneField(related_name='info_volunteer', to='cvmapp.Person', db_constraint=False),
),
]
The trick was to add "db_constraint=False" as a parameter of the OneToOne field.

Please make sure all your MySQL tables are using the same storage engine (i.e. MyISM v. InnoDB). Especially tables that have ForeignKeys between them.
If you need more information about MySQL storage engines and how to know which one(s) are you using you will need to read the MySQL documentation, although the MySQL notes in our database docs also have some introductory information.
I suspect you had tables created with the MyISAM storage engine (the default for MySQL < 5.5) and since MySQL 5.5 defaults to InnoDB and you created new tables since then, you ended with a mix
You can change the table storage engine from phpMyadmin. Click on your teble name, go to the 'operations' tab, and change in the "Table Operation" box. Convert all or your storage engine to the same.

First thing to do should be checking table's storage engines. However, in my case storage engines were same (both INNODB). My issue was unmatched collation of tables. After setting default char-set of the Database, my problem has been solved.
You can check table details with;
SHOW TABLE STATUS WHERE Name = "yyyyyyy";
Once you figure out your table's (that will be ForeignKey) collation, you can set your database default char-set as shown here like this:
ALTER DATABASE my_db COLLATE = 'latin1_swedish_ci';

Look at the output of the command SHOW ENGINE INNODB STATUS in MySql or MariaDB console client (mariadb link, mysql link). It`s more informative than error message.
For example, it showed me this message:
...
------------------------
LATEST FOREIGN KEY ERROR
------------------------
2021-09-20 18:27:08 7faea3ad1700 Error in foreign key constraint of table `my_db`.`django_admin_log`:
Alter table `my_db`.`django_admin_log` with foreign key constraint failed. Referenced table `my_db`.`profiles_userprofile` not found in the data dictionary near ' FOREIGN KEY (`user_id`) REFERENCES `profiles_userprofile` (`id`)'.
...
It said to me that I forgot to create migrations.

Django with postgresql - manage.py syncdb returns errors

I'm starting with Django. I had some site set up with SQLite working but after changing DB engine to postgresql manage.py syncdb returns errors.I've been googling for 2 days but still nothing works for me.Postgres user 'joe' has superuser rights and local 'joe' db exists.
Postgresql is running:
/etc/init.d/postgresql status
Running clusters: 9.1/main
Here's my part of settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2', #
'NAME': 'joe', #
'USER': 'joe', #
'PASSWORD': 'asdf', #
'HOST': 'localhost', #
'PORT': '5432', .
}
}
And the errors:
$ python manage.py syncdb
Syncing...
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/south/management/commands/syncdb.py", line 90, in handle_noargs
syncdb.Command().execute(**options)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 59, in handle_noargs
tables = connection.introspection.table_names()
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 792, in table_names
return self.get_table_list(cursor)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/db/backends/postgresql/introspection.py", line 31, in get_table_list
AND pg_catalog.pg_table_is_visible(c.oid)""")
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/home/m/.virtualenvs/mayan/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
django.db.utils.DatabaseError: current transaction is aborted, commands ignored until end of transaction block
Thanks!

My suggestion would be to go to django.db.backends.postgresql_psycopg2.base and insert a print query so that CursorWrapper looks like.
class CursorWrapper:
...
def execute(self, query, args=None):
print query # New print statement here
try:
return self.cursor.execute(query, args)
That will print out every query hitting the database, and hopefully allow you to identify the offending SQL query when trying to run python manage.py syncdb

First, is your DB properly synced? Try running python manage.py syncdb. If this does not help, then read on...
This is what postgres does when a query produces an error and you try to run another query without first rolling back the transaction in case of error. So you should first rollback
the DB state when something goes wrong. Since its transactional based, it causes problems. To fix it, you'll want to figure out where in the code that bad query is being executed.
It might be helpful to use the log_statement option in your postgresql server for debugging.
The below solution is what I came up when I had same error.
from django.db import transaction
#transaction.commit_manually
def db_insert():
try:
YourModel(name='hello world').save() #trying to save
except: #any error rollback
transaction.rollback()
else: #if all fine, commit
transaction.commit()
Hope this helps.

In my case, I had to disable some apps, do syncdb, enable apps again and syncdb again.

MySQL 1045 exception

I use Django 1.3.1 & Python 2.7.2 and I'm trying to deploy a project locally with nginx.
Something wrong with manage.py after executing the first command. For example: the first command is:
$ python manage.py runfcgi host=127.0.0.1 port=7782
and it works correctly. But when I try to execute any command after that, like syncdb or anything else (runserver, validate, runfcgi, etc...), I get strange exceptions:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 88, in inner_run
self.validate(display_num_errors=True)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 249, in validate
num_errors = get_validation_errors(s, app)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/validation.py", line 102, in get_validation_errors
connection.validation.validate_field(e, opts, f)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/validation.py", line 14, in validate_field
db_version = self.connection.get_server_version()
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 338, in get_server_version
self.cursor()
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/__init__.py", line 250, in cursor
cursor = self.make_debug_cursor(self._cursor())
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 322, in _cursor
self.connection = Database.connect(**kwargs)
File "/usr/lib/pymodules/python2.7/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/pymodules/python2.7/MySQLdb/connections.py", line 187, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'root'#'localhost' (using password: NO)")
I don't know what the reason is. In settings.py all MySQL access parameters (user, pass, dbname & host) are written correctly (syncdb succeed).
Note: if I copy the project directory for example to "project2" and rename it to original "project", the problem disappears for the first manage.py command I exectue, after that, I see the exceptions again.
I have another django projects deployed in the same way, using same django & python, but they work without any problem.
Anybody knows what the problem is?

Is your other Django project is set up on the same server/computer?
If it is, check your commas when you giving your db credentials and compare with the one which is running.