Related
After updating superset to 2.0.0 I cannot access sqllab nor view databases (Oracle DB).
I'm running superset on openshift with custom image built from apache/superset:2.0.0
Installed dependencies:
cx_Oracle certifi flask-oidc==1.4.0 flask_openid gevent psycopg2 psycopg2-binary==2.9.1 redis==3.5.3 itsdangerous==2.0.1
I tried running superset db upgrade with following result:
$ superset db upgrade
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
logging was configured successfully
2022-11-21 14:21:41,835:INFO:superset.utils.logging_configurator:logging was configured successfully
2022-11-21 14:21:41,841:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
and superset init throws error:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column dbs.allow_multi_schema_metadata_fetch does not exist
LINE 1: ..., dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.allow_...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/superset", line 33, in \<module\>
sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(\*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 601, in main
return super().main(\*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return \_process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, \*\*ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return \__callback(\*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), \*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 445, in decorator
return \__ctx.invoke(f, \*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return \__callback(\*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), \*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 445, in decorator
return \__ctx.invoke(f, \*args, \*\*kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return _callback(\*args, \*\*kwargs)
File "/app/superset/cli/main.py", line 62, in init
security_manager.sync_role_definitions()
File "/app/superset/security/manager.py", line 731, in sync_role_definitions
self.create_missing_perms()
File "/app/superset/security/manager.py", line 679, in create_missing_perms
merge_pv("datasource_access", datasource.get_perm())
File "/app/superset/connectors/sqla/models.py", line 826, in get_perm
if self.database is None:
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 294, in __get__
return self.impl.get(instance_state(instance), dict_)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 730, in get
value = self.callable(state, passive)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 759, in \_load_for_state
return self.\_emit_lazyload(
File "\<string\>", line 1, in \<lambda\>
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 847, in \_emit_lazyload
q(session)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/baked.py", line 615, in \_load_on_pk_identity
result = list(bq.for_session(self.session).params(\*\*params))
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/baked.py", line 444, in __iter__
return q.\_execute_and_instances(context)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3560, in \_execute_and_instances
result = conn.execute(querycontext.statement, self.\_params)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in \_execute_on_connection
return connection.\_execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in \_execute_clauseelement
ret = self.\_execute_context(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in \_execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column dbs.allow_multi_schema_metadata_fetch does not exist
LINE 1: ..., dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.allow_...
^
\[SQL: SELECT dbs.uuid AS dbs_uuid, dbs.created_on AS dbs_created_on, dbs.changed_on AS dbs_changed_on, dbs.id AS dbs_id, dbs.verbose_name AS dbs_verbose_name, dbs.database_name AS dbs_database_name, dbs.sqlalchemy_uri AS dbs_sqlalchemy_uri, dbs.password AS dbs_password, dbs.cache_timeout AS dbs_cache_timeout, dbs.select_as_create_table_as AS dbs_select_as_create_table_as, dbs.expose_in_sqllab AS dbs_expose_in_sqllab, dbs.configuration_method AS dbs_configuration_method, dbs.allow_run_async AS dbs_allow_run_async, dbs.allow_file_upload AS dbs_allow_file_upload, dbs.allow_ctas AS dbs_allow_ctas, dbs.allow_cvas AS dbs_allow_cvas, dbs.allow_dml AS dbs_allow_dml, dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.allow_multi_schema_metadata_fetch AS dbs_allow_multi_schema_metadata_fetch, dbs.extra AS dbs_extra, dbs.encrypted_extra AS dbs_encrypted_extra, dbs.impersonate_user AS dbs_impersonate_user, dbs.server_cert AS dbs_server_cert, dbs.is_managed_externally AS dbs_is_managed_externally, dbs.external_url AS dbs_external_url, dbs.created_by_fk AS dbs_created_by_fk, dbs.changed_by_fk AS dbs_changed_by_fk
FROM dbs
WHERE dbs.id = %(param_1)s\]
\[parameters: {'param_1': 1}\]
(Background on this error at: http://sqlalche.me/e/13/f405)
Anyone found solution for this error?
Tried:
superset db upgrade
superset init
updating dependencies
restarting superset
Expect:
successfully migrate to 2.0.0 version
When I run this basic code to access my AWS redshift data warehouse, I get this error:
Traceback (most recent call last):
File "<ipython-input-40-dcebeb0708d9>", line 25, in <module>
engine = sqlalchemy.create_engine(URL)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\__init__.py", line 479, in create_engine
return strategy.create(*args, **kwargs)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\strategies.py", line 56, in create
plugins = u._instantiate_plugins(kwargs)
TypeError: _instantiate_plugins() missing 1 required positional argument: 'kwargs'
This is the code I'm running. I originally planned on using psycopg2 to access the database but psycopg2 returns the sql query as a list with funky formatting; I'd have to do too much work to extract the data into a dataframe. I'm hoping sqlalchemy produces and output that's easier to work with.
import sqlalchemy
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
import pandas as pd
from sqlalchemy.engine.url import URL
config = dict(
drivername='driver',
username= username_u,
password= password_u,
host='rs-data.prod-lde.bsp.gsa.gov',
port='5439',
database='edw',
query={'encoding': 'utf-8'}
)
url = URL(**config)
engine = sqlalchemy.create_engine(url) # Having "URL" instead of "url" caused the original problem
Thanks in advance.
>>>>>Update<<<<<:
I fixed my silly mistake and the new code is this:
import sqlalchemy
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
import pandas as pd
from sqlalchemy.engine.url import URL
config = dict(
drivername='postgresql',
username= username_u,
password= password_u,
host='rs-data.prod-lde.bsp.gsa.gov',
port='5439',
database='edw',
query={'encoding': 'utf-8'}
)
url = URL(**config)
engine = sqlalchemy.create_engine(url)
data_frame = pd.read_sql('select date_signed from edw.fpds.fpds_atom limit 5;', engine)
Now I'm getting this error:
ProgrammingError: (psycopg2.ProgrammingError) invalid dsn: invalid connection option "encoding"
(Background on this error at: http://sqlalche.me/e/f405)
I think the problem lies in the config dict values. I first had it as drivername='driver' and that caused this error:
Traceback (most recent call last):
File "<ipython-input-48-dcb4b19c1bd3>", line 25, in <module>
engine = sqlalchemy.create_engine(url)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\__init__.py", line 479, in create_engine
return strategy.create(*args, **kwargs)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\strategies.py", line 61, in create
entrypoint = u._get_entrypoint()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\url.py", line 172, in _get_entrypoint
cls = registry.load(name)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\langhelpers.py", line 240, in load
"Can't load plugin: %s:%s" % (self.group, name)
NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:driver
so I changed it to drivername='postgresql' and that produced this error:
Traceback (most recent call last):
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 2276, in _wrap_pool_connect
return fn()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 363, in connect
return _ConnectionFairy._checkout(self)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\impl.py", line 139, in _do_get
self._dec_overflow()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\impl.py", line 136, in _do_get
return self._create_connection()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\default.py", line 489, in connect
return self.dbapi.connect(*cargs, **cparams)
File "C:\Anaconda3\lib\site-packages\psycopg2\__init__.py", line 126, in connect
dsn = _ext.make_dsn(dsn, **kwargs)
File "C:\Anaconda3\lib\site-packages\psycopg2\extensions.py", line 175, in make_dsn
parse_dsn(dsn)
ProgrammingError: invalid dsn: invalid connection option "encoding"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<ipython-input-53-3b57714b0ad8>", line 28, in <module>
data_frame = pd.read_sql('select date_signed from edw.fpds.fpds_atom limit 5;', engine)
File "C:\Anaconda3\lib\site-packages\pandas\io\sql.py", line 438, in read_sql
chunksize=chunksize,
File "C:\Anaconda3\lib\site-packages\pandas\io\sql.py", line 1218, in read_query
result = self.execute(*args)
File "C:\Anaconda3\lib\site-packages\pandas\io\sql.py", line 1087, in execute
return self.connectable.execute(*args, **kwargs)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 2181, in execute
connection = self._contextual_connect(close_with_result=True)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 2242, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 2280, in _wrap_pool_connect
e, dialect, self
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 1547, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\base.py", line 2276, in _wrap_pool_connect
return fn()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 363, in connect
return _ConnectionFairy._checkout(self)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\impl.py", line 139, in _do_get
self._dec_overflow()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\impl.py", line 136, in _do_get
return self._create_connection()
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "C:\Anaconda3\lib\site-packages\sqlalchemy\engine\default.py", line 489, in connect
return self.dbapi.connect(*cargs, **cparams)
File "C:\Anaconda3\lib\site-packages\psycopg2\__init__.py", line 126, in connect
dsn = _ext.make_dsn(dsn, **kwargs)
File "C:\Anaconda3\lib\site-packages\psycopg2\extensions.py", line 175, in make_dsn
parse_dsn(dsn)
ProgrammingError: (psycopg2.ProgrammingError) invalid dsn: invalid connection option "encoding"
(Background on this error at: http://sqlalche.me/e/f405)
The key question I guess is what should I be using for the drivername parameter?
I want to add columns to a table in my database dynamically since I don't want to have to specify all columns when I set up the table in the SQLalchemy class
In order to solve this I am using alembic to add columns to the table but I am having problems.
In my python script I have defined a class as below.
DATABASE_URL_dev = "postgresql+psycopg2://user:password#localhost:5432/testdb"
engine = create_engine(DATABASE_URL_dev)
Base = declarative_base(engine)
class my_class1(Base):
__tablename__ = "test1"
id = Column(Integer, primary_key=True)
list = Column(String)
def loadsession():
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
return session
session = loadsession()
Now I want to add a new column ("new_column") to the table test1 in the database.
Using Alembic I then followed the instructions at https://alembic.sqlalchemy.org/en/latest/tutorial.html and did the following:
Installed alembic by running pip install alembic
Created an environment using alembic init alembic
Edited the .ini file by inputting "postgresql+psycopg2://user:password#localhost:5432/testdb" at sqlalchemy.url
Created a migration script by running alembic revision -m "add a column "
I then edited the script by adding the following "op.add_column('test1', sa.Column('new_column', sa.String))" in upgrade()
Running first migration with "alembic upgrade head"
This gave me the following error:
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38-32\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\user\appdata\local\programs\python\python38-32\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\Scripts\alembic.exe\__main__.py", line 9, in <module>
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 577, in main
CommandLine(prog=prog).main(argv=argv)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 571, in main
self.run_cmd(cfg, options)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\config.py", line 548, in run_cmd
fn(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\command.py", line 298, in upgrade
script.run_env()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\script\base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\util\pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\alembic\util\compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "y\env.py", line 77, in <module>
run_migrations_online()
File "y\env.py", line 65, in run_migrations_online
with connectable.connect() as connection:
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2218, in connect
return self._connection_cls(self, **kwargs)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 103, in __init__
else engine.raw_connection()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2317, in raw_connection
return self._wrap_pool_connect(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2288, in _wrap_pool_connect
Connection._handle_dbapi_exception_noconnection(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 1554, in _handle_dbapi_exception_noconnection
util.raise_(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\base.py", line 2285, in _wrap_pool_connect
return fn()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\impl.py", line 238, in _do_get
return self._create_connection()
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.raise_(
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
raise exception
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\pool\base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "c:\users\user\appdata\local\programs\python\python38-32\lib\site-packages\sqlalchemy\engine\default.py", line 488, in connect
return self.dbapi.connect(*cargs, **cparams)
File "c:\users\c\appdata\local\programs\python\python38-32\lib\site-packages\psycopg2\__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError)
(Background on this error at: http://sqlalche.me/e/e3q8)
Questions
Is this the right way to proceed in order to add columns dynamically or should it be done in another way?
How do I resolve the sqlalchemy operationalerror?
Thanks
Try this in step 5
op.add_column('test1', sa.Column('new_column', sa.String()))
I am trying to run my python code in pycharmie, which connects to the database, although it succeeds from the command line, PyCharm returns the following error.
It seems to me that I added all the necessary changes in the pycharma settings to run the Oracle Client.
My_file.py
import sqlalchemy as db
engine = db.create_engine('dialect+driver://user:pass#host:port/db')
connection = enigne.connect()
Error from PyCharm
/home/tymoteusz/Desktop/test/venv/bin/python /home/tymoteusz/Desktop/my_task_sqlalchemy/test_document.py
Traceback (most recent call last):
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2221, in _wrap_pool_connect
return fn()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 356, in unique_connection
return _ConnectionFairy._checkout(self)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 811, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 543, in checkout
rec = pool._do_get()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 1239, in _do_get
self._dec_overflow()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 277, in reraise
raise value
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 1236, in _do_get
return self._create_connection()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 361, in _create_connection
return _ConnectionRecord(self)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 488, in __init__
self.__connect(first_connect_check=True)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 690, in __connect
connection = pool._invoke_creator(self)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 437, in connect
return self.dbapi.connect(*cargs, **cparams)
cx_Oracle.DatabaseError: DPI-1047: 64-bit Oracle Client library cannot be loaded: "libclntsh.so: cannot open shared object file: No such file or directory". See https://oracle.github.io/odpi/doc/installation.html#linux for help
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/tymoteusz/Desktop/my_task_sqlalchemy/test_document.py", line 6, in <module>
connection = engine.connect()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2164, in connect
return self._connection_cls(self, **kwargs)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 103, in __init__
else engine.raw_connection()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2252, in raw_connection
self.pool.unique_connection, _connection
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2225, in _wrap_pool_connect
e, dialect, self
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1522, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 276, in reraise
raise value.with_traceback(tb)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2221, in _wrap_pool_connect
return fn()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 356, in unique_connection
return _ConnectionFairy._checkout(self)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 811, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 543, in checkout
rec = pool._do_get()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 1239, in _do_get
self._dec_overflow()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 277, in reraise
raise value
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 1236, in _do_get
return self._create_connection()
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 361, in _create_connection
return _ConnectionRecord(self)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 488, in __init__
self.__connect(first_connect_check=True)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 690, in __connect
connection = pool._invoke_creator(self)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/home/tymoteusz/.local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 437, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) DPI-1047: 64-bit Oracle Client library cannot be loaded: "libclntsh.so: cannot open shared object file: No such file or directory". See https://oracle.github.io/odpi/doc/installation.html#linux for help (Background on this error at: http://sqlalche.me/e/4xp6)
Process finished with exit code 1
How can I solve my problem? Why does the Terminal start the code without returning the error, while PyCharm returns the error related to the Oracle Client Library.
Any help will be appreciated.
If you're doing this on macOS (which I'm guessing from the way you have set DYLD_LIBRARY_PATH) then you need to put the Oracle Instant Client in ~/lib or as described in the cx_Oracle installation instructions https://cx-oracle.readthedocs.io/en/latest/installation.html#install-oracle-instant-client because SIP security on macOS now stops DYLD_LIBRARY_PATH working in sub-shells.
I would also recommend using Instant Client 12.2 unless you actually need to connect to very old DB versions. With 12.2 client you can connect to Oracle DB 10.2 or later.
I have been looking for this error for a while but nothing seems to solve my problem, the thing is..
I was using sqlite3 with flask, but it seems like sqlite3 does not support migrations, so I decided to change to Mysql. I have installed LAMPP, and I created a symbolik link from:
/opt/lampp/var/mysql/mysql.sock
to:
/var/run/mysqld/mysql.sock
but still I get this error (it is a little long but the trouble relapse on the last line):
Traceback (most recent call last):
File "db_create.py", line 5, in <module>
db.create_all()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/flask_sqlalchemy/__init__.py", line 895, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/flask_sqlalchemy/__init__.py", line 887, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 3634, in create_all
tables=tables)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1851, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1844, in _optional_conn_ctx_manager
with self.contextual_connect() as conn:
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2035, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2074, in _wrap_pool_connect
e, dialect, self)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception_noconnection
exc_info
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2070, in _wrap_pool_connect
return fn()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 376, in connect
return _ConnectionFairy._checkout(self)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 708, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 480, in checkout
rec = pool._do_get()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1055, in _do_get
self._dec_overflow()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1052, in _do_get
return self._create_connection()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 323, in _create_connection
return _ConnectionRecord(self)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 449, in __init__
self.connection = self.__connect()
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 602, in __connect
connection = self.__pool._invoke_creator(self)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 97, in connect
return dialect.connect(*cargs, **cparams)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 385, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/home/gerardo/Documentos/python_web_dev/flask-intro/venv/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__
super(Connection, self).__init__(*args, **kwargs2)
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")
On my research, I found that I have to change the line:
socket = /var/run/mysqld/mysqld.sock
placed on /etc/mysql/my.cnf to:
socket = /opt/lampp/var/mysql/mysql.sock
(The path were "mysql.sock" is located), but still it does not solve my problem. Thank you!
In the database configuration. Replace localhost with 127.0.0.1:
environ.get('APP_DATABASE_HOST', 'localhost')
to
environ.get('APP_DATABASE_HOST', '127.0.0.1')