I am trying to reflect an existing MS Access database into a new model. To link the database to SQLAlchemy, I am using
engine = create_engine("access+pyodbc://#db-dns")
Since the database already exists, I am following the basic use scheme from this article: https://docs.sqlalchemy.org/en/14/orm/extensions/automap.html
# reflect the table
Base.prepare(engine, reflect=True)
# mapped classes are now created with names by default
# matching that of the table name
Personen = Base.classes.ref_personen #here, 'ref_personen' is a table from my database
session = Session(engine)
When I try to access the table using Personen.query.all() I get an AttributeError (error log see below).
EDIT: To raise the error, I dont even have to query anything. I assume that the error is already caused at this line: Personen = Base.classes.ref_personen
Often, the reason for this error is due to not hanving defined a primary key in the external database. However, this is not the case for me. In my MS Access DB rel_personen table, a primary key is defined.
Thanks, I appreciate any help on this topic!
(appenv4) C:\Users\Code\webapps4>flask run
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
c:\users\code\webapps4\appenv4\lib\site-packages\sqlalchemy\engine\reflection.py:919: SAWarning: The Access ODBC driver does not support the ODBC "SQLPrimaryKeys" function. get_pk_constraint() is returning an empty list.
pk_cons = self.get_pk_constraint(
c:\users\code\webapps4\appenv4\lib\site-packages\sqlalchemy\engine\reflection.py:947: SAWarning: The Access ODBC driver does not support the ODBC "SQLForeignKeys" function. get_foreign_keys() is returning an empty list.
fkeys = self.get_foreign_keys(
Traceback (most recent call last):
File "c:\users\code\webapps4\appenv4\lib\site-packages\sqlalchemy\util\_collections.py", line 186, in __getattr__
return self._data[key]
KeyError: 'ref_personen'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Code\webapps4\appenv4\Scripts\flask.exe\__main__.py", line 7, in <module>
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 990, in main
cli.main(args=sys.argv[1:])
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 596, in main
return super().main(*args, **kwargs)
File "c:\users\code\webapps4\appenv4\lib\site-packages\click\core.py", line 1062, in main
rv = self.invoke(ctx)
File "c:\users\code\webapps4\appenv4\lib\site-packages\click\core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\code\webapps4\appenv4\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\code\webapps4\appenv4\lib\site-packages\click\core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "c:\users\code\webapps4\appenv4\lib\site-packages\click\decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "c:\users\code\webapps4\appenv4\lib\site-packages\click\core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 845, in run_command
app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 321, in __init__
self._load_unlocked()
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 346, in _load_unlocked
self._app = rv = self.loader()
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 406, in load_app
app = locate_app(self, import_name, None, raise_if_not_found=False)
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 273, in locate_app
return find_best_app(script_info, module)
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 68, in find_best_app
app = call_factory(script_info, app_factory)
File "c:\users\code\webapps4\appenv4\lib\site-packages\flask\cli.py", line 119, in call_factory
return app_factory(*args, **kwargs)
File "C:\Users\Code\webapps4\app\__init__.py", line 19, in create_app
from .zkl import zkl as zkl_bp
File "C:\Users\Code\webapps4\app\zkl\__init__.py", line 6, in <module>
from . import zkl_forms, zkl_views
File "C:\Users\Code\webapps4\app\zkl\zkl_views.py", line 3, in <module>
from .zkl_database import db_session, init_db
File "C:\Users\Code\webapps4\app\zkl\zkl_database.py", line 12, in <module>
Personen = Base.classes.ref_personen
File "c:\users\code\webapps4\appenv4\lib\site-packages\sqlalchemy\util\_collections.py", line 188, in __getattr__
raise AttributeError(key)
AttributeError: ref_personen
Update: sqlalchemy-access version 1.1.0 now supports reflection of Primary Key (PK) and Foreign Key (FK) constraints.
(previous answer)
As noted by one of the warning messages in your stack trace:
SAWarning: The Access ODBC driver does not support the ODBC "SQLPrimaryKeys" function. get_pk_constraint() is returning an empty list.
This is a known issue with the Access ODBC driver. It is also discussed here
https://github.com/gordthompson/sqlalchemy-access/issues/9
and here
pyodbc - read primary keys from MS Access (MDB) database
Related
I am new to airflow. I created a virtual environment and followed the steps in https://airflow.apache.org/docs/apache-airflow/stable/start.html. In the end I gave "airflow standalone" and got "ValueError: Unable to configure handler 'processor'"
(venv) hgovea155#INSML-CPXX7WW AFDocProj % airflow standalone
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/usr/local/opt/python#3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py", line 563, in configure
handler = self.configure_handler(handlers[name])
File "/usr/local/opt/python#3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py", line 736, in configure_handler
result = factory(**kwargs)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/utils/log/file_processor_handler.py", line 49, in __init__
Path(self._get_log_directory()).mkdir(parents=True, exist_ok=True)
File "/usr/local/opt/python#3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 1273, in mkdir
self._accessor.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/Users/hgovea155/airflow/logs/scheduler/2023-01-02'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/__init__.py", line 46, in <module>
settings.initialize()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/settings.py", line 569, in initialize
LOGGING_CLASS_PATH = configure_logging()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/logging_config.py", line 74, in configure_logging
raise e
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/logging_config.py", line 69, in configure_logging
dictConfig(logging_config)
File "/usr/local/opt/python#3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py", line 800, in dictConfig
dictConfigClass(config).configure()
File "/usr/local/opt/python#3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py", line 571, in configure
'%r' % name) from e
ValueError: Unable to configure handler 'processor'
I then gave export AIRFLOW_HOME=. after which I gave "airflow standalone" again and I received "airflow.exceptions.AirflowConfigException: Cannot use relative path: sqlite:///./airflow.db to connect to sqlite. Please use absolute path such as sqlite:////tmp/airflow.db."
standalone | Database ready
Traceback (most recent call last):
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/bin/airflow", line 8, in <module>
sys.exit(main())
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/__main__.py", line 39, in main
args.func(args)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 52, in command
return func(*args, **kwargs)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/commands/standalone_command.py", line 48, in entrypoint
StandaloneCommand().run()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/commands/standalone_command.py", line 64, in run
self.initialize_database()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/commands/standalone_command.py", line 183, in initialize_database
appbuilder = cached_app().appbuilder
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/www/app.py", line 167, in cached_app
app = create_app(config=config, testing=testing)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/www/app.py", line 90, in create_app
f'Cannot use relative path: `{conf.get("database", "SQL_ALCHEMY_CONN")}` to connect to sqlite. '
airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///./airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`.
I then tried the fix provided by #kulasangar
I navigated to user -> airflow and changed the "logs" folder to read write and execute for all users. The "ValueError: Unable to configure handler 'processor'" error didn't occur but I received another error.
(venv) hgovea155#INSML-CPXX7WW AFDocProj % airflow standalone
standalone | Starting Airflow Standalone
standalone | Checking database is initialized
Traceback (most recent call last):
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1901, in _execute_context
cursor, statement, parameters, context
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: attempt to write a readonly database
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/bin/airflow", line 8, in
sys.exit(main())
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/main.py", line 39, in main
args.func(args)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 52, in command
return func(*args, **kwargs)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/commands/standalone_command.py", line 48, in entrypoint
StandaloneCommand().run()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/commands/standalone_command.py", line 64, in run
self.initialize_database()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/cli/commands/standalone_command.py", line 175, in initialize_database
db.initdb()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/utils/session.py", line 75, in wrapper
return func(*args, session=session, **kwargs)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/utils/db.py", line 697, in initdb
_create_db_from_orm(session=session)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/utils/db.py", line 682, in _create_db_from_orm
_create_flask_session_tbl()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/airflow/utils/db.py", line 677, in _create_flask_session_tbl
db.create_all()
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 1094, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 1086, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4931, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 3228, in _run_ddl_visitor
conn._run_ddl_visitor(visitorcallable, element, **kwargs)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2211, in _run_ddl_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
return meth(obj, **kw)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 855, in visit_metadata
_is_metadata_operation=True,
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
return meth(obj, **kw)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 900, in visit_table
include_foreign_key_constraints, # noqa
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 81, in _execute_on_connection
self, multiparams, params, execution_options
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1478, in _execute_ddl
compiled,
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1944, in execute_context
e, statement, parameters, cursor, context
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2125, in handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from=e
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 211, in raise
raise exception
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1901, in _execute_context
cursor, statement, parameters, context
File "/Users/hgovea155/PycharmProjects/AFDocProj/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) attempt to write a readonly database
[SQL:
CREATE TABLE session (
id INTEGER NOT NULL,
session_id VARCHAR(255),
data BLOB,
expiry DATETIME,
PRIMARY KEY (id),
UNIQUE (session_id)
)
]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Is there a way to fix this? I believe this problem is because of some mistake in the initial setup or configuration. Can a more proper fix be found rather than a temporary one.
It seems like the user doesn't have the privilege to write logs under your airflow distribution folder.
Could you try granting write permission to the folder, so that airflow instance can write logs
sudo chmod -R 777 /home/user/airflow_logs
Please change the logs directory accordingly.
I am deploying a Flask-based app based on this amazing tutorial. I deploy my DB through:
flask db init
flask db migrate
flask db upgrade
sometimes it checks for older version of the DB inside migrations directory and I get this error (full log below, here just what I think are the most important parts):
# ...
alembic.util.exc.CommandError: Directory migrations already exists
# ...
alembic.script.revision.ResolutionError: No such revision or branch 'f05567e712eb'
# ...
alembic.util.exc.CommandError: Can't locate revision identified by 'f05567e712eb'
# ...
I know how to fix this in several ways (as example recreate DB or drop the revision ID from the table etc.). But is there any way to prevent this "revision-ID check" alembic behavior?
Here the full error log:
Traceback (most recent call last):
File "/home/diagnosticator/venv/bin/flask", line 8, in <module>
sys.exit(main())
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask/cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask/cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask/cli.py", line 412, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask_migrate/cli.py", line 31, in init
_init(directory, multidb)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 118, in init
command.init(config, directory, 'flask')
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/command.py", line 42, in init
raise util.CommandError("Directory %s already exists" % directory)
alembic.util.exc.CommandError: Directory migrations already exists
[2022-01-04 18:30:59,314] INFO in __init__: Diagnosticator-local startup
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
20
Traceback (most recent call last):
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
yield
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 206, in get_revisions
return self.revision_map.get_revisions(id_)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 299, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 299, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 302, in get_revisions
return tuple(
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 303, in <genexpr>
self._revision_for_ident(rev_id, branch_label)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 360, in _revision_for_ident
raise ResolutionError(
alembic.script.revision.ResolutionError: No such revision or branch 'f05567e712eb'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/diagnosticator/venv/bin/flask", line 8, in <module>
sys.exit(main())
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask/cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask/cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask/cli.py", line 412, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask_migrate/cli.py", line 89, in migrate
_migrate(directory, message, sql, head, splice, branch_label, version_path,
File "/home/diagnosticator/venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 195, in migrate
command.revision(config, message, autogenerate=True, sql=sql,
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/command.py", line 176, in revision
script_directory.run_env()
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 425, in run_env
util.load_python_file(self.dir, 'env.py')
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
module = load_module_py(module_id, path)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 83, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "migrations/env.py", line 87, in <module>
run_migrations_online()
File "migrations/env.py", line 80, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/runtime/environment.py", line 836, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/runtime/migration.py", line 321, in run_migrations
for step in self._migrations_fn(heads, self):
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/command.py", line 156, in retrieve_migrations
revision_context.run_autogenerate(rev, context)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 415, in run_autogenerate
self._run_environment(rev, migration_context, True)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 425, in _run_environment
if set(self.script_directory.get_revisions(rev)) != \
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 206, in get_revisions
return self.revision_map.get_revisions(id_)
File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 174, in _catch_revision_errors
compat.raise_from_cause(util.CommandError(resolution))
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 194, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 187, in reraise
raise value.with_traceback(tb)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 143, in _catch_revision_errors
yield
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/base.py", line 206, in get_revisions
return self.revision_map.get_revisions(id_)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 299, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 299, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 302, in get_revisions
return tuple(
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 303, in <genexpr>
self._revision_for_ident(rev_id, branch_label)
File "/home/diagnosticator/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 360, in _revision_for_ident
raise ResolutionError(
alembic.util.exc.CommandError: Can't locate revision identified by 'f05567e712eb'
Luckily, it's not possible. Such behaviour could make a lot harm - when we command alembic to do some operation it creates list of migrations from upper to lower in some topological order. One of them is given at command (e.g. head in alembic upgrade head), second is took from that table. Then it can safely apply migrations one by one.
Even if we come with some hack, there is no good way to answer what should be done in such case. And whatever we do, it would mean that some migration was applied but will never be downgraded.
Saying that, if you really want act some logical way instead of throwing error, it's a workaround I invented. It replaces current version of alembic table with current head. Use it at your own risk.
# It's alembic env.py file
from sqlalchemy.sql.elements import literal_column
from alembic import context
from alembic.util import CommandError
from alembic.script import ScriptDirectory
def run_migrations_online():
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
# Until now everything is just defaults generated by alembic
try:
context.run_migrations()
except CommandError: # in case run_migrations failed
c = context.get_context()
c.impl._exec(c._version.delete()) # delete current version from alembic table
script = ScriptDirectory.from_config(config)
head = script.revision_map.get_current_head() # get last migration in topological order
c.impl._exec(
c._version.insert().values(
version_num=literal_column("'%s'" % head),
)
) # insert it into alembic table
context.run_migrations() # run again as usual
this is what might be happening
You create a migration and run "flask db migrate -m 'msg'". This will look through your model and create a migrations file. If you open the migrations file, among other things you will see "revision=REV1" and "down_revision=None". The very first migration will have "down_revision = None" as there is nothing to downgrade do.
You then deploy the migration by "flask db upgrade". At this point, the migration (only migration you have created in step.1) gets applied. If you were to go into the db and do "select * from alembic_version", you will see the same version that was there in the revision field..
You now make some changes to the model and create another migration. The migration file will now have "revision=REV2", "down_revision=REV1"
You do "flask db upgrade" and if all goes well alembic_version will show "REV2"
Now - if for some reason you decide that you had made a mistake in the model and delete the migrations/version/REV2-migration-created-in-step-3 and say make some DB fixes and create another migration (REV3).
If you try to do "flask db upgrade" the alembic version (REV2) does not match any of the files you have in migrations/versions and you will see "Can't locate the revision identified by REV2".
If you create a migration and apply it, the way to un-apply it is to do "flask db downgrade" and then delete the migrations file - if you have to.
If you happen to delete the migrations file after doing flask db upgrade (and its not in source control for you to rever the deletion), then you have to manually deal with the situation that will require you to undo the db changes and also "update alembic version set version_num=version" where version is the previous version to which you manually reverted to.
While at this, I'd like to point out another common pitfall..
When multiple developers are on a project and there is parallel development going on in 2 branches. Let's say main branch was on REV10 and 2 developers pull a private-branch. They both have "revision=REV10, down_revision=REV9".. At the end of their development let's say
DEVELOPER1: has created 2 migrations thus having "revision=REV12, down_revision=REV11" and "revision=REV11, down_revision=REV10" in the 2 migration files he has checked-in to his private-branch [now - we can debate why not just one revision but I am taking an example here]
DEVELOPER2: has created 1 migration thus having "revision=REV11, down_revision=REV10" in his private-branch
Say Developer1 merges to main ..
Now when developer2 merges main to his private branch (before sending out a pull) - the fun (or the trouble) begins for him.
This will require developer2 to manually edit his migrations file(s) change the down_revisions so the migrations chain is in-tact at the end of combining the migration files.
I set up a database with mongolab where the db had the same name as the collection. I decided I didnt like this deleted it and made a better naming scheme. I added a new user/password to for the new db and tried to authenticated. It keeps failing. Not sure why. I have double check my credentials and they are correct. I have checked the URI that mongolab provides and that is correct also. This code worked well for the first db.
this is my login code
def __init__(self, user_id, password, database, collection):
# mongodb://<dbuser>:<dbpassword>#ds017205.mlab.com:17205/words
mongodb_uri = "mongodb://" + user_id + ":" + password + "#ds017205.mlab.com:17205/" + database
client = pymongo.MongoClient(mongodb_uri)
db = client[database]
self.collection = db[collection] # is declared in class
this generates the following errors
Error
Traceback (most recent call last):
File "C:\Users\Austin\PycharmProjects\Words\test_mongodb.py", line 15, in testUpdate
results = self.mongodb_obj.update(test)
File "C:\Users\Austin\PycharmProjects\Words\mongodb.py", line 38, in update
upsert=True
File "C:\Python33\lib\site-packages\pymongo\collection.py", line 2235, in update
with self._socket_for_writes() as sock_info:
File "C:\Python33\lib\contextlib.py", line 48, in __enter__
return next(self.gen)
File "C:\Python33\lib\site-packages\pymongo\mongo_client.py", line 718, in _get_socket
with server.get_socket(self.__all_credentials) as sock_info:
File "C:\Python33\lib\contextlib.py", line 48, in __enter__
return next(self.gen)
File "C:\Python33\lib\site-packages\pymongo\server.py", line 152, in get_socket
with self.pool.get_socket(all_credentials, checkout) as sock_info:
File "C:\Python33\lib\contextlib.py", line 48, in __enter__
return next(self.gen)
File "C:\Python33\lib\site-packages\pymongo\pool.py", line 541, in get_socket
sock_info.check_auth(all_credentials)
File "C:\Python33\lib\site-packages\pymongo\pool.py", line 306, in check_auth
auth.authenticate(credentials, self)
File "C:\Python33\lib\site-packages\pymongo\auth.py", line 436, in authenticate
auth_func(credentials, sock_info)
File "C:\Python33\lib\site-packages\pymongo\auth.py", line 416, in _authenticate_default
return _authenticate_scram_sha1(credentials, sock_info)
File "C:\Python33\lib\site-packages\pymongo\auth.py", line 216, in _authenticate_scram_sha1
res = sock_info.command(source, cmd)
File "C:\Python33\lib\site-packages\pymongo\pool.py", line 213, in command
read_concern)
File "C:\Python33\lib\site-packages\pymongo\network.py", line 99, in command
helpers._check_command_response(response_doc, None, allowable_errors)
File "C:\Python33\lib\site-packages\pymongo\helpers.py", line 196, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.
Help with this would be appreciated. Thanks
Nothing wrong with the code... the db user set up entered the incorrect password the same twice in the setup box so it was accepted then when the code was trying to use the actual correct answer it wouldn't work for obvious reasons. thanks all for looking
the following code:
class CreateSurveyFromCsvTextTests(TestCase):
def test_parses_survey_passed_in_as_csv_and_returns_xml_representation(self):
self.assertTrue(True)
throws the following error:
ERROR: test_parses_survey_passed_in_as_csv_and_returns_xml_representation (dkobo.formbuilder.tests.CreateSurveyFromCsvTextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/test/testcases.py", line 178, in __call__
self._pre_setup()
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/test/testcases.py", line 749, in _pre_setup
self._fixture_setup()
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/test/testcases.py", line 861, in _fixture_setup
if not connections_support_transactions():
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/test/testcases.py", line 848, in connections_support_transactions
for conn in connections.all())
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/test/testcases.py", line 848, in <genexpr>
for conn in connections.all())
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/utils/functional.py", line 49, in __get__
res = instance.__dict__[self.func.__name__] = self.func(instance)
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 664, in supports_transactions
self.connection.leave_transaction_management()
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 317, in leave_transaction_management
if managed == self.get_autocommit():
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 324, in get_autocommit
self.ensure_connection()
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 124, in ensure_connection
self.connect()
File "/home/nico/.virtualenvs/kf/local/lib/python2.7/site-packages/django/db/utils.py", line 86, in __exit__
db_exc_type = getattr(self.wrapper.Database, dj_exc_type.__name__)
AttributeError: 'DatabaseWrapper' object has no attribute 'Database'
----------------------------------------------------------------------
I know this doesn't tell you much about the problem, and it's probably an environment problem, but I'm new to Linux/Python (switched from Windows/.Net 7 days ago) and wouldn't really know where to start gathering information, let alone diagnosing the error.
The problem was that the database was not configured. Adding the default entry for DATABASE in settings.py and the running python manage.py syncdb fixed it.
I'm almost there!! in deploying EveryBlock's ebcode in my mac OSX - here is my settings.py:
http://pastebin.com/ndQ57LCH
This is the entire error:
Traceback (most recent call last):
File "/Library/Python/2.6/site-packages/django/core/servers/basehttp.py", line 280, in run
self.result = application(self.environ, self.start_response)
File "/Library/Python/2.6/site-packages/django/core/servers/basehttp.py", line 674, in __call__
return self.application(environ, start_response)
File "/Library/Python/2.6/site-packages/django/core/handlers/wsgi.py", line 234, in __call__
signals.request_started.send(sender=self.__class__)
File "/Library/Python/2.6/site-packages/django/dispatch/dispatcher.py", line 172, in send
response = receiver(signal=self, sender=sender, **named)
File "/Library/Python/2.6/site-packages/django/db/__init__.py", line 90, in reset_queries
for conn in connections.all():
File "/Library/Python/2.6/site-packages/django/db/utils.py", line 101, in all
return [self[alias] for alias in self]
File "/Library/Python/2.6/site-packages/django/db/utils.py", line 93, in __getitem__
conn = backend.DatabaseWrapper(db, alias)
File "/Library/Python/2.6/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 11, in __init__
self.ops = PostGISOperations(self)
File "/Library/Python/2.6/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 91, in __init__
vtup = self.postgis_version_tuple()
File "/Library/Python/2.6/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 445, in postgis_version_tuple
version = self.postgis_lib_version()
File "/Library/Python/2.6/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 425, in postgis_lib_version
return self._get_postgis_func('postgis_lib_version')
File "/Library/Python/2.6/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 406, in _get_postgis_func
cursor = self.connection._cursor()
File "/Library/Python/2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 121, in _cursor
raise ImproperlyConfigured("You need to specify NAME in your Django settings file.")
ImproperlyConfigured: You need to specify NAME in your Django settings file.
Please advise.
Use either the one-database setup (DATABASE_xxx = ...) or the multiple-database (DATABASES dict) setup
Don't prepend "DATABASE_" to the dictionary keys, e.g. in DATABASES["users"]). This is where the error comes from.
Don't share your database password on SO
I think you might have forgotten to set the NAME property in your DATABASES setting:
DATABASES setting