Develop Reference

pymysql.err.InternalError: (1049, "Unknown database")

I want to connect MySQL RDS DB using python from raspberrypi.(i want to get seq from MySQL table 'face' using select query.)
and I have an error but i can not fix it.
This is rds mysql connection code:
import rds_config
import pymysql
rds_host = rds_config.rds_host
name = rds_config.rds_user
password = rds_config.rds_pwd
db_name = rds_config.rds_db
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name,
connect_timeout=10)
with conn.cursor() as cur:
cur.execute("select seq from face")
conn.commit()
rds_config:
rds_host='rds endpoint'
rds_port=3306
rds_user='user'
rds_pwd='password'
rds_db='db name'
and This is traceback:
Traceback (most recent call last):
File "getRds.py", line 18, in <module>
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=10)
File "/usr/local/lib/python2.7/dist-packages/pymysql/__init__.py", line 94, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 327, in __init__
self.connect()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 598, in connect
self._request_authentication()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 862, in _request_authentication
auth_packet = self._process_auth(plugin_name, auth_packet)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 933, in _process_auth
pkt = self._read_packet()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 683, in _read_packet
packet.check_error()
File "/usr/local/lib/python2.7/dist-packages/pymysql/protocol.py", line 220, in check_error
err.raise_mysql_exception(self._data)
File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.InternalError: (1049, u"Unknown database 'bsb-rds'")
i alread added ip address in vpc security group and public access is on.
it was possible to connect through mysql cli or workbench.
can anyone help me?

tl;dr you need to create bsb-rd. Execute this command: create database bsb-rds either with cur.execute() in python or in your favorite sql cli.
If you get this error, good news! You can connect to your RDS instance. This means you have the security group set up right, the host url, username, password and port are all correct! What this error is telling you is that your database bsb-rds does not exist on your RDS instance. If you have just made the RDS instance it is probably because you have not created the database yet. So create it!
Here are two ways to do this
mysql cli
In your terminal run
mysql --host=the_host_address user=your_user_name --password=your_password
Then inside the mysql shell execute
create database bsb-rd;
Now try your code again!
Python with pymysql
import rds_config
conn = pymysql.connect('hostname', user='username', passwd='password', connect_timeout=10)
with conn.cursor() as cur:
cur.execute('create database bsb-rd;')
I came to this page looking for a solution and didn't get it. I eventually found the answer and now share what helped me.

I ran into your exact issue and I believe I have the solution:
Context:
My tech stack looks like the following
Using AWS RDS (MySQL)
Using Flask Development on local host
RDS instance name: "test-rds"
Actual DB Name: test-database
Here is where my issue was and I believe your issue is in the same place:
You are using the AWS RDS NAME in your connection rather than using the true Database name.
Simply changing the DB name in my connection to the true DB name that I had setup via MySQL Workbench fixed the issue.
Other things things to note for readers:
Please ensure the following:
If you are connecting from outside your AWS VPC make sure you have public access enabled. This is a huge "gotcha". (Beware security risks)
Make sure your connection isn't being blocked by a NACL
Make sure your connection is allowed by a Security Group Rule

CommandError: Can't locate revision identified by '...' when migrating using Flask-Migrate

I started using Flask-Migrate today and installed it on a test project.
However i am getting following error:
alembic.util.exc.CommandError: Can't locate revision identified by
'e39d16e62810'
Steps to reproduce:
run "python create_db.py"
run "flask db init"
add column "name" to Entry-model
run "flask db migrate"
EDIT:
After removing migrations directory and repeating the process i am getting the same error after running "flask db migrate".
I also tried using a manage.py file with flask-script --> same issue
Error:
(venv_mentz) H:\Flask-API-Test>python manage.py db migrate
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
Traceback (most recent call last):
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 143, in _catch_revision_errors
yield
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 206, in get_revisions
return self.revision_map.get_revisions(id_)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 299, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 299, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 304, in get_revisions
for rev_id in resolved_id)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 304, in <genexpr>
for rev_id in resolved_id)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 362, in _revision_for_ident
resolved_id)
alembic.script.revision.ResolutionError: No such revision or branch 'e39d16e62810'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 14, in <module>
manager.run()
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\flask_script\__init__.py", line 417, in run
result = self.handle(argv[0], argv[1:])
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\flask_script\__init__.py", line 386, in handle
res = handle(*args, **config)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\flask_script\commands.py", line 216, in __call__
return self.run(*args, **kwargs)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\flask_migrate\__init__.py", line 197, in migrate
version_path=version_path, rev_id=rev_id)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\command.py", line 176, in revision
script_directory.run_env()
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 425, in run_env
util.load_python_file(self.dir, 'env.py')
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\util\pyfiles.py", line 81, in load_python_file
module = load_module_py(module_id, path)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\util\compat.py", line 83, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "migrations\env.py", line 87, in <module>
run_migrations_online()
File "migrations\env.py", line 80, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\runtime\environment.py", line 836, in run_migrations
self.get_context().run_migrations(**kw)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\runtime\migration.py", line 321, in run_migrations
for step in self._migrations_fn(heads, self):
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\command.py", line 156, in retrieve_migrations
revision_context.run_autogenerate(rev, context)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\autogenerate\api.py", line 415, in run_autogenerate
self._run_environment(rev, migration_context, True)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\autogenerate\api.py", line 425, in _run_environment
if set(self.script_directory.get_revisions(rev)) != \
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 206, in get_revisions
return self.revision_map.get_revisions(id_)
File "c:\users\marschall\appdata\local\programs\python\python36-32\Lib\contextlib.py", line 100, in __exit__
self.gen.throw(type, value, traceback)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 174, in _catch_revision_errors
compat.raise_from_cause(util.CommandError(resolution))
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\util\compat.py", line 194, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\util\compat.py", line 187, in reraise
raise value.with_traceback(tb)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 143, in _catch_revision_errors
yield
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\base.py", line 206, in get_revisions
return self.revision_map.get_revisions(id_)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 299, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 299, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 304, in get_revisions
for rev_id in resolved_id)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 304, in <genexpr>
for rev_id in resolved_id)
File "H:\Flask-API-Test\venv_mentz\lib\site-packages\alembic\script\revision.py", line 362, in _revision_for_ident
resolved_id)
alembic.util.exc.CommandError: Can't locate revision identified by 'e39d16e62810'
My file structure looks like this:
app
-- views
----- __init__.py
----- main.py
-- __init__.py
-- config.py
-- models.py
instance
-- __init__.py
-- config.py
create_db.py
dev.db
run.py
My app factory:
from flask import Flask
from instance.config import app_config
from flask_migrate import Migrate
def create_app(config_name):
""" Creates a runnable app.
This app will be using the config with name "config_name".
"""
app = Flask(__name__)
# Loading the the config from instance folder with name "config_name"
app.config.from_object(app_config[config_name])
# Loading generic config from 'config.py'
app.config.from_pyfile('config.py')
# Registering this app at db
from app.models import db
db.init_app(app)
migrate = Migrate(app, db)
from app import models, views
return app
My run.py:
""" This script runs a the app with the given configuration. """
from app import create_app
# Configuration used to run the app
config_name = 'dev'
# Creating the app by using the required configuration
app = create_app(config_name)
if __name__ == '__main__':
app.run()
And my create_db.py file to create the database using models.py:
""" This script creates the database defined in app.models. """
from app import create_app
app = create_app('dev')
from app.models import db
# Telling SQLAlchemy what app should be used as the database model
with app.app_context():
db.create_all()
This is my models.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Entry(db.Model):
__tablename__ = 'Entries'
layer_id = db.Column(db.Integer, primary_key=True)
def __repr__(self):
return "ID: {}; text: {}".format(self.layer_id, self.text)
EDIT: manage.py:
from flask_script import Manager
from flask_migrate import Migrate, MigrateCommand
from app import create_app
from app import models
from app.models import db
app = create_app('dev')
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
if __name__ == '__main__':
manager.run()

you delete the migration directory but the version has been saved in the database, so you have to delete the version info in the dabase,
run
delete from alembic_version;
in mysql shell.
As suggested by #mirekphd, If this is a developing environment or a single app for test, just delete it, Else BACKUP the data in the table.

I tried this on flask and it worked:
python app.py db revision --rev-id e39d16e62810
python app.py db migrate
python app.py db upgrade

In my case I have accidentally deleted the most recent migration file but the alembic version (alembic_version) table refers to the deleted version.
So instead of droping the entire database, you can change the version_num field in the alembic_version table.
Following steps worked for me:
Find the head using db history
Update the version_num field to the head version.
run migrate using db migrate
upgrade the database db upgrade

Most of the time this happens, if you are at the development stage where you don't have much data to loose, you can drop the database and create a fresh one, alternatively, you can delete the migrations and begin the whole cycle again, for sqlite, it is all about deleting the sqlite file in your application, and beginning this cycle again. I hope to find the best solution to this verry soon.

In my case, I have accidentally deleted the most recent migration file but the alembic version (alembic_version) table refers to the deleted version.
I tried the following:
flask db revision --rev-id [revision-id]
flask db migrate
flask db upgrade

My problem was that another flask app was already connected to the same database, so I just created a new database, and changed:
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://root:root#127.0.0.1:3306/newDataBaseName'

Delete the history entry from the database will solve the issue.
Error: alembic.util.exc.CommandError: Can't locate revision identified by 'e39d16e62810'
select * from alembic_version where version_num='e39d16e62810'
Option 1: Update the record with your latest revision id
update alembic_version set version_num='<your_version_num>' where version_num='e39d16e62810'
Option 2: Delete this record:
delete from alembic_version where version_num='e39d16e62810'

For my case reason for the error was the relative path to sqlite db.
Fixed it by specifying the absolute path
app.config['SQLALCHEMY_DATABASE_URI'] = f"sqlite:///{os.path.join(os.path.dirname(__file__), os.path.pardir, 'db.sqlite')}"

Yet another cause of problems - making two flask projects in the same virtual environment.
I tried to make fake twin project for performance testing.
copied crucial parts of the original project
created new database
made changes in configuration file to point to new database
flask db init,
flask db migrate
got this error at first migration.
Then I did only one thing - created new python environment for this twin project - and error disappeared.
PS. Maybe it is possible to create two flask projects in the same environment, but I was not able to find solution.

In my case, since I was in the initial phase, I had the luxury of deleting the database file directly i.e. in my project folder there was a .sqlite file, so I deleted it by right-clicking it.
But please keep in mind that I was in the initial phase, so I won't suggest this if you are
not in the starting phase.

I had the same error message but in my case it's because I used wrong database name in the connection string. Check your connection string
Error Message:
ERROR [root] Error: Can't locate revision identified by 'a80ab7ca5e1a'
Reason:
My connection string was:
mysql+pymysql://kaunda:kaunda#localhost:3306/smis
Instead of
mysql+pymysql://kaunda:kaunda#localhost:3306/WorkFolder
The revision identified by "a80ab7ca5e1a" belongs to the database "smis" not "WorkFolder"

Make sure you set FLASK_APP=app.py on windows or export FLASK_APP=app.py on MAC before running flask db init, then flask db migrate -m "message" and flask db upgrade. You can check this link for more info https://pypi.org/project/Flask-Migrate/

Connection problems with SQLAlchemy and multiple processes

I'm using PostgreSQL and SQLAlchemy in a project that consists of a main process which launches child processes. All of these processes access the database via SQLAlchemy.
I'm experiencing repeatable connection failures: The first few child processes work correctly, but after a while a connection error is raised. Here's an MWCE:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, create_engine
from sqlalchemy.orm import sessionmaker
DB_URL = 'postgresql://user:password#localhost/database'
Base = declarative_base()
class Dummy(Base):
__tablename__ = 'dummies'
id = Column(Integer, primary_key=True)
value = Column(Integer)
engine = None
Session = None
session = None
def init():
global engine, Session, session
engine = create_engine(DB_URL)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
def cleanup():
session.close()
engine.dispose()
def target(id):
init()
try:
dummy = session.query(Dummy).get(id)
dummy.value += 1
session.add(dummy)
session.commit()
finally:
cleanup()
def main():
init()
try:
dummy = Dummy(value=1)
session.add(dummy)
session.commit()
p = multiprocessing.Process(target=target, args=(dummy.id,))
p.start()
p.join()
session.refresh(dummy)
assert dummy.value == 2
finally:
cleanup()
if __name__ == '__main__':
i = 1
while True:
print(i)
main()
i += 1
On my system (PostgreSQL 9.6, SQLAlchemy 1.1.4, psycopg2 2.6.2, Python 2.7, Ubuntu 14.04) this yields
1
2
3
4
5
6
7
8
9
10
11
Traceback (most recent call last):
File "./fork_test.py", line 64, in <module>
main()
File "./fork_test.py", line 55, in main
session.refresh(dummy)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1422, in refresh
only_load_props=attribute_names) is None:
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", line 223, in load_on_ident
return q.one()
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2756, in one
ret = self.one_or_none()
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2726, in one_or_none
ret = list(self)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2797, in __iter__
return self._execute_and_instances(context)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2820, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception
exc_info
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/vagrant/latest-sqlalchemy/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 469, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: 'SELECT dummies.id AS dummies_id, dummies.value AS dummies_value \nFROM dummies \nWHERE dummies.id = %(param_1)s'] [parameters: {'param_1': 11074}]
This is repeatable and always crashes at the same iteration.
I'm creating a new engine and session after the fork as recommended by the SQLAlchemy documentation and elsewhere. Interestingly, the following slightly different approach does not crash:
import contextlib
import multiprocessing
import sqlalchemy
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, create_engine
from sqlalchemy.orm import sessionmaker
DB_URL = 'postgresql://user:password#localhost/database'
Base = declarative_base()
class Dummy(Base):
__tablename__ = 'dummies'
id = Column(Integer, primary_key=True)
value = Column(Integer)
#contextlib.contextmanager
def get_session():
engine = sqlalchemy.create_engine(DB_URL)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
try:
yield session
finally:
session.close()
engine.dispose()
def target(id):
with get_session() as session:
dummy = session.query(Dummy).get(id)
dummy.value += 1
session.add(dummy)
session.commit()
def main():
with get_session() as session:
dummy = Dummy(value=1)
session.add(dummy)
session.commit()
p = multiprocessing.Process(target=target, args=(dummy.id,))
p.start()
p.join()
session.refresh(dummy)
assert dummy.value == 2
if __name__ == '__main__':
i = 1
while True:
print(i)
main()
i += 1
Since the original code is more complex and cannot simply be switched over to the latter version I'd like to understand why one of these works and the other doesn't.
The only obvious difference is that the crashing code uses global variables for the engine and the session -- these are shared via copy-on-write with the child processes. However, since I reset them directly after the fork I don't understand how that could be a problem.
Update
I re-ran the two code pieces with the latest SQLAlchemy (1.1.5) using both Python 2.7 and Python 3.4. On both the results are basically as described above. However, on Python 2.7 the crash of the first code piece now happens in the 13th iteration (reproducibly) while on 3.4 it already happens in the third iteration (also reproducibly). The second code piece runs without problems on both versions. Here's the traceback from 3.4:
1
2
3
Traceback (most recent call last):
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
psycopg2.OperationalError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "fork_test.py", line 64, in <module>
main()
File "fork_test.py", line 55, in main
session.refresh(dummy)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 1424, in refresh
only_load_props=attribute_names) is None:
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 223, in load_on_ident
return q.one()
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2749, in one
ret = self.one_or_none()
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2719, in one_or_none
ret = list(self)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2790, in __iter__
return self._execute_and_instances(context)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2813, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception
exc_info
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/vagrant/latest-sqlalchemy-3.4/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: 'SELECT dummies.id AS dummies_id, dummies.value AS dummies_value \nFROM dummies \nWHERE dummies.id = %(param_1)s'] [parameters: {'param_1': 3397}]
Here's the PostgreSQL log (it's the same for 2.7 and 3.4):
2017-01-18 10:59:36 UTC [22429-1] LOG: database system was shut down at 2017-01-18 10:59:35 UTC
2017-01-18 10:59:36 UTC [22429-2] LOG: MultiXact member wraparound protections are now enabled
2017-01-18 10:59:36 UTC [22428-1] LOG: database system is ready to accept connections
2017-01-18 10:59:36 UTC [22433-1] LOG: autovacuum launcher started
2017-01-18 10:59:36 UTC [22435-1] [unknown]#[unknown] LOG: incomplete startup packet
2017-01-18 11:00:10 UTC [22466-1] user#db LOG: SSL error: decryption failed or bad record mac
2017-01-18 11:00:10 UTC [22466-2] user#db LOG: could not receive data from client: Connection reset by peer
(Note that the message about the incomplete startup packet is harmless)

Quoting "How do I use engines / connections / sessions with Python multiprocessing, or os.fork()?" with added emphasis:
The SQLAlchemy Engine object refers to a connection pool of existing database connections. So when this object is replicated to a child process, the goal is to ensure that no database connections are carried over.
and
However, for the case of a transaction-active Session or Connection being shared, there’s no automatic fix for this; an application needs to ensure a new child process only initiate new Connection objects and transactions, as well as ORM Session objects.
The issue stems from the forked child process inheriting the live global session, which is holding on to a Connection. When target calls init, it overwrites the global references to engine and session, thus decreasing their refcounts to 0 in the child, forcing them to finalize. If you for example one way or another create another reference to the inherited session in the child, you prevent it from being cleaned up – but don't do that. After main has joined and returns to business as usual it is trying to use the now potentially finalized – or otherwise out of sync – connection. As to why this causes an error only after some amount of iterations I'm not sure.
The only way to handle this situation using globals the way you do is to
Close all sessions
Call engine.dispose()
before forking. This will prevent connections from leaking to the child. For example:
def main():
global session
init()
try:
dummy = Dummy(value=1)
session.add(dummy)
session.commit()
dummy_id = dummy.id
# Return the Connection to the pool
session.close()
# Dispose of it!
engine.dispose()
# ...or call your cleanup() function, which does the same
p = multiprocessing.Process(target=target, args=(dummy_id,))
p.start()
p.join()
# Start a new session
session = Session()
dummy = session.query(Dummy).get(dummy_id)
assert dummy.value == 2
finally:
cleanup()
Your second example does not trigger finalization in the child, and so it only seems to work, though it might be as broken as the first, as it is still inheriting a copy of the session and its connection defined locally in main.

Heroku MySQL DB Error with Flask

I'm trying to host a local MySQL database using ClearDB on Heroku. The build is successful, but when I try to access the application, the application fails.
Here is my app.py code (it fails on the first line):
db = MySQLdb.connect(host="localhost", user="root", db="database")
cur = db.cursor()
query = """SELECT * from database"""
cur.execute(query)
data = cur.fetchall()
Here is the log output:
2016-06-01T14:48:48.720931+00:00 heroku[web.1]: Starting process with command `python app.py`
2016-06-01T14:48:51.790883+00:00 app[web.1]: File "app.py", line 30, in <module>
2016-06-01T14:48:51.790864+00:00 app[web.1]: Traceback (most recent call last):
2016-06-01T14:48:51.790910+00:00 app[web.1]: db = MySQLdb.connect(host="localhost", user="root", db="database")
2016-06-01T14:48:51.790981+00:00 app[web.1]: _mysql_exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")
2016-06-01T14:48:51.790911+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
2016-06-01T14:48:51.790912+00:00 app[web.1]: return Connection(*args, **kwargs)
2016-06-01T14:48:51.790912+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__
2016-06-01T14:48:51.790937+00:00 app[web.1]: super(Connection, self).__init__(*args, **kwargs2)
2016-06-01T14:48:52.491629+00:00 heroku[web.1]: Process exited with status 1
Here is what heroku config prints:
CLEARDB_DATABASE_URL: mysql://actualurl
DATABASE_URL: mysql://actualurl
MYSQL_DB: database
MYSQL_HOST: localhost
MYSQL_USER: root

There's no point in posting your Heroku config, because it's clear that your app isn't using it. You have hardcoded the database host as "localhost", rather than using the DATABASE_URL environment variable.

SqlAlchemy+Tornado: can't reconnect until invalid transaction is rolled back

i'm building a webapp with tornado+sqlalchemy and absolutely random i got this error
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1024, in _handle_dbapi_exception
exc_info
File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 187, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 182, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 822, in _execute_context
conn = self._revalidate_connection()
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 239, in _revalidate_connection
"Can't reconnect until invalid "
sqlalchemy.exc.StatementError: Can't reconnect until invalid transaction is rolled back
I can't figure out how to solve this. I've put all db.commit into a
try:
self.db.commit()
except Exception(e):
self.db.rollback()
That's my class Application.
class Application
[...]
engine = create_engine(options.db_path, convert_unicode=True, echo=options.debug)
models.init_db(engine)
self.db = scoped_session(sessionmaker(bind=engine))
tornado.web.Application.__init__(self, handlers, **settings)
but nothing.
What is the best way to configure sqlalchemy and tornado for a web app like mysql+php?

My way is do rollback on finish, add this to your BaseHandler:
def on_finish(self):
if self.get_status() == 500:
self.db_session.rollback()

I remember having same issuess a while ago. Seems there was some strange stuff related to connection pooling. Disabling pooling seemd to fixed it.
Not the best idea in general, but it worked.
Try passing poolclass=NullPool to create_engine
...
from sqlalchemy.pool import NullPool
...
engine = create_engine(options.db_path, convert_unicode=True, echo=options.debug, poolclass=NullPool)