Develop Reference

Accessing Oracle via SQLAlchemy's `create_engine()` requires more settings, is not it?

I am developing an API by means of the FastAPI. It is intended to get an input, retrieving data from a database (Oracle), and providing a processed output.
Here is a function that executes queries with Oracle engine:
from sqlalchemy import exc, create_engine, inspect
def db_connection_sqlalchemy(your_query):
db_config = config['database'] # configurations from a yml-file
try:
engine = create_engine("{0}+{1}://{2}:{3}#{4}".format(db_config['dialect'] #'oracle',
db_config['sql_driver'] #'cx_oracle',
db_config['username'],
db_config['password'],
db_config['network_alias']),
max_identifier_length=30,
encoding="utf8")
inspector = inspect(engine)
conn = engine.connect()
rows = conn.execute(your_query)
rows = rows.fetchmany()
result = [dict(row.items()) for row in rows]
conn.close()
return result
except exc.SQLAlchemyError as e:
error = str(e.__dict__['orig'])
return error
It works okay. However, to set it up I faced several issues.
Issue 1. After installing the SQLAlchemy via pip install SQLAlchemy, I was encountering this error:
NoSuchModuleError: Can't load plugin:
sqlalchemy.dialects:oracle.cx_oracle
Even though I thought that "I do not need to import cx_Oracle anymore", based on this answer. I decided to install pip install cx_Oracle and than I got a next problem
Issue 2. Every time when I tried to execute an SQL-query, I was seeing this error:
"DPI-1047: Cannot locate a 64-bit Oracle Client library:
"C:\oracle\12.2.0\x86\bin\oci.dll is not the correct
architecture". See
https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html
for help"
So, to overcome this problem, this answer helped me.
Actually everything works till now, but is it still possible to avoid those additional steps (installing cx-Oracle library, getting dll-files) when dealing with SQLAlchemy or these are typical pitfalls when working with the Oracle DB in a virtual environment?
I have a virtual environment via Anaconda and Python 3.7.10 on Windows 10.

I always install cx_Oracle separately - adding one word "cx_Oracle" to my pip install isn't a big deal. Regarding the quoted wording: importing (in a running script) is different to installing (in a system).
You will always need to install Instant Client and set PATH to it, or call cx_Oracle.init_oracle_client(lib_dir= r"c:\path_to_libraries") in your code (see here).
When fetching a large number of rows, tune arraysize and prefetchrows, see Tuning Fetch Performance.

Sqlalchemy shows "Code 516 Authentication failed" when trying to connect to clickhouse db

I have connected to a clickhouse db with dbeaver and installed sqlalchemy v1.3.13 and clickhouse-sqlalchemy 0.1.3 for python 3.7.
When I tried to connect with
from sqlalchemy import create_engine
engine_clickhouse = create_engine('clickhouse://use:pass#host:port/db')
engine_clickhouse.raw_connection()
I got
Exception: Code: 516, e.displayText() = DB::Exception: default: Authentication failed: password is incorrect or there is no user with such name (version 20.3.4.10 (official build))
Does anybody know why?
I didn't find a similar issue.

This issue is well-known - look at sqlalchemy-clickhouse Issue-45 or sqlalchemy-clickhouse Issue-49.
To fix it need to explicitly downgrade the package infi.clickhouse_orm up to version 1.0.4:
requirements.txt
...
infi.clickhouse_orm==1.0.4
It allows using the _build_params-function with the behavior expected by sqlalchemy-clickhouse (following versions of infi.clickhouse_orm aren't passed the password that leads to 'Authentication failed..'-error).
Or alternatively can be used the fork where fixed this error, for example adaiboy fork.
I would avoid using the official sqlalchemy-clickhouse because:
fixes not committed to master ("Latest commit was on 23 Jan 2019" !!)
new featured not adding
there are some issues with pandas etc
clickhouse-driver is the greatest alternative for sqlalchemy-clickhouse.

According to #vladimir reply.I tried
pip uninstall infi.clickhouse_orm
pip install infi.clickhouse_orm==1.0.4
It worked for me.
Thanks

Trying to connect oracle db from python using cx_oracle and found "ppcx_Oracle.i386 : Python interface to Oracle" using yum list installed

When tried to import cx_Oracle getting error module not found.
Not sure what exactly "ppcx_Oracle.i386" is for and the way to utilize it.
=================================================================== Matched: ppcx_Oracle ====================================================================
ppcx_Oracle.i386 : Python interface to Oracle
is "ppcx_Oracle.i386" anything related to "cx_Oracle" package or i have to look for other options to install cx_Oracle to connect oracle database?
Environment : RHEL 5.4 and python 2.4,2.6
PS: Not having any issues in cx_Oracle since i have already used it in one of my servers.
Thanks in advance!!!

You should have Oracle Instant Client installed on the machine (or Oracle DB). The cx_oracle will use its shared libraries.
Then run this command to install cx_oracle itself:
python -m pip install cx_Oracle --upgrade
More on how to configure it and a code examples can be found here.
https://cx-oracle.readthedocs.io/en/latest/installation.html
ppcx_oracle.i386 looks like some non-official interface, I don't know it and could not give any recommendations about it.

Python - pysqlite or sqlite3 must be installed

I have python 2.7.12 installed on my server. I'm using PuTTY to connect to my server. When running my python script I get the following.
File "home/myuser/python/lib/python2.7/site-packages/peewee.py", line 3657, in _connect
raise ImproperlyConfigured('pysqlite or sqlite3 must be installed.')
peewee.ImproperlyConfigured: pysqlite or sqlite3 must be installed.
I thought sqlite was installed with python 2.7.12, so I'm assuming the issue is something else. Haven't managed to find any posts on here yet that have been helpful.
I am missing something?
Thanks in advance

Peewee will use either the standard library sqlite3 module or, if you did not compile Python with SQLite, Peewee will look for pysqlite2.
The problem is most definitely not with Peewee on this one, as Peewee requires a SQLite driver to use the SqliteDatabase class... If that driver does not exist, then you need to install it.

Accessing a MySQL database from python

I have been trying for the past several hours to find a working method of accessing a mysql database in python. The only thing that I've managed to get to compile and install is pyodbc but the necessary driver is not available for ppc leopard.
I already know about this.
UPDATE:
I've gotten setuptools to install, but now MySQL-python won't build.
UPDATE:
Now I've gotten sqlalchemy to install but while it will show up when called by the command line it won't import when used in my cgi script.

Try SQL Alchemy.
It is awesome.

Install fink. It includes the MySQLdb package.

UPDATE: Now I've gotten sqlalchemy to
install but while it will show up when
called by the command line it won't
import when used in my cgi script.
Can you verify that the Python being invoked from your CGI script is the same as the one you get when you run Python interactively? Check which python and compare it to your webserver CGI settings. That's the only thing I can think of that would cause this - getting it installed in one Python but not the other.
What OS are you on? If you're on something like Ubuntu, sudo apt-get install python-mysqldb is much more reliable than trying to build it yourself.
Also, unless I'm mistaken, SQLAlchemy won't actually help you make the connection itself if you don't have a DB-API2 module (like python-mysqldb) installed - SQLAlchemy sits at the next level up, using the DB-API2 connection and making access to it more Pythonic.