Develop Reference

Python to MS SQL Error: Error when connecting to SQL using sqlalchemy.create_engine() using pypyodbc

Scenario:
I am trying to Convert the SQL output directly to Table using dataframe.to_sql, so for that i am using sqlalchemy.create_engine() and its throwing error when trying to createngine()
sqlchemyparams= urllib.parse.quote_plus(ConnectionString)
sqlchemy_conn_str = 'mssql+pypyodbc:///?odbc_connect={}'.format(sqlchemyparams)
engine_azure = sqlalchemy.create_engine(sqlchemy_conn_str,echo=True,fast_executemany =
True, poolclass=NullPool)
df_top_features.to_sql('Topdata', engine_azure,schema='dbo', index = False, if_exists =
'replace')
2.It will work fine if i use:pyodbc
sqlchemy_conn_str = 'mssql+pyodbc:///?odbc_connect={}'.format(sqlchemyparams)
So is there any way i can using pypyodbc in sqlchem_conn_str

SQLAlchemy does not have a pypyodbc driver defined for the mssql dialect, so
mssql+pypyodbc:// …
simply will not work. There may be some way to "fool" your code into using pypyodbc when you specify mssql+pyodbc://, similar to doing
import pypyodbc as pyodbc
in plain Python, but it is not recommended.
In cases where pyodbc cannot be used, the recommended alternative would be mssql+pymssql://.

Here's what I do
import sqlalchemy as sa
from sqlalchemy import create_engine, event
from sqlalchemy.engine.url import URL
Then create varaibles to holder the server, database, username and password and pass it to...
params = urllib.parse.quote_plus("DRIVER={SQL Server};"
"SERVER="+server+";"
"DATABASE="+database+";"
"UID="+username+";"
"PWD="+password+";")
engine = sa.create_engine("mssql+pyodbc:///?odbc_connect={}".format(params))
then upload data to sql using.
dfc.to_sql('jobber',con=engine,index=False, if_exists='append')
Using https://www.dataquest.io/blog/sql-insert-tutorial/ as a source.

How to connect to Oracle-DB ODBC connection string using SQLAlchemy?

I am trying to connect to oracle-db using the odbc connection string, I am able to make the connection using pyodbc
import pyodbc
import pandas as pd
connection_string = 'DRIVER={Oracle};DBQ=X.X.X.X/YY/dbname;UID=someuser;PWD=XXXXXX'
cnxn = pyodbc.connect(connection_string)
but I am not able to connect using SQLAlchemy.
from sqlalchemy.engine import create_engine
params = urllib.parse.quote_plus(connection_string)
db_engine = create_engine(f"cx-Oracle+pyodbc:///?odbc_connect={params}")

Python Database connection for informix DB using sqlalchemy

I'm trying to connect to remote informix DB as follows using python3 sqlalchemy but it fails to connect
sqlalchemy.create_engine("informix://usr1:pwd1#XXX:23300/DB_NAME;SERVER=dsinfmx").connect()
I get the below ERROR while connecting.
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:informix
Can someone please provide some help on this.. From Dbeaver, DB server is accessible.

I assume you are using Informix Python drivers. If not please install Informix Python driver i.e IfxPy. Details to install Informix Python drivers are at this link https://github.com/OpenInformix/IfxPy/blob/master/README.md
Try out below code.
from sqlalchemy import create_engine
from sqlalchemy.dialects import registry
from sqlalchemy.orm import sessionmaker
registry.register("informix", "IfxAlchemy.IfxPy", "IfxDialect_IfxPy")
registry.register("informix.IfxPy", "IfxAlchemy.IfxPy", "IfxDialect_IfxPy")
registry.register("informix.pyodbc", "IfxAlchemy.pyodbc", "IfxDialect_pyodbc")
from sqlalchemy import Table, Column, Integer
ConStr = 'informix://<username>:<password>#<machine name>:<port number>/<database name>;SERVER=<server name>'
engine = create_engine(ConStr)
connection = engine.connect()
connection.close()
print( "Done2" )

impala connection via sqlalchemy

I'm new to hadoop and impala. I managed to connect to impala by installing impyla and executing the following code. This is connection by LDAP:
from impala.dbapi import connect
from impala.util import as_pandas
conn = connect(host="server.lrd.com",port=21050, database='tcad',auth_mechanism='PLAIN', user="alexcj", use_ssl=True,timeout=20, password="secret1pass")
I'm then able to grab a cursor and execute queries as:
cursor = conn.cursor()
cursor.execute('SELECT * FROM tab_2014_m LIMIT 10')
df = as_pandas(cursor)
I'd like to be able use sqlalchemy to connect to impala and be able to use some nice sqlalchemy functions. I found a test file in imyla source code that illustrates how to create an sqlalchemy engine with impala driver like:
engine = create_engine('impala://localhost')
I'd like to be able to do that but I'm not able to because my call to the connect function above has a lot more parameters; and I do not know how to pass those to sqlalchemy's create_engine to get a successful connection. Has anyone done this? Thanks.

As explained at https://github.com/cloudera/impyla/issues/214
import sqlalchemy
def conn():
return connect(host='some_host',
port=21050,
database='default',
timeout=20,
use_ssl=True,
ca_cert='some_pem',
user=user, password=pwd,
auth_mechanism='PLAIN')
engine = sqlalchemy.create_engine('impala://', creator=conn)

If your Impala is secured by Kerberos below script works (due to some reason I need to use hive:// instead of impala://)
import sqlalchemy
from sqlalchemy.engine import create_engine
connect_args={'auth': 'KERBEROS', 'kerberos_service_name': 'impala'}
engine = create_engine('hive://impalad-host:21050', connect_args=connect_args)
conn = engine.connect()
ResultProxy = conn.execute("SELECT * FROM db1.table1 LIMIT 5")
print(ResultProxy.fetchall())

import time
from sqlalchemy import create_engine, MetaData, Table, select, and_
ENGINE = create_engine(
'impala://{host}:{port}/{database}'.format(
host=host, # your host
port=port,
database=database,
)
)
METADATA = MetaData(ENGINE)
TABLES = {
'table': Table('table_name', METADATA, autoload=True),
}

pyodbc autocommit does not appear to work with sybase and sqlalchemy

I am connecting to a sybase ASE 15 database from Python 3.4 using pyodbc and executing a stored procedure.
All works as expected if I use native pyodbc:
import pd
import pyodbc
con = pyodbc.connect('DSN=dsn_name;UID=username;PWD=password', autocommit=True)
df = pd.read_sql("exec p_procecure #GroupName='GROUP'", con)
[Driver is Adaptive Server Enterprise].
I have to have autocommit=True and if I do no I get the following error:
DatabaseError: Execution failed on sql 'exec ....': ('ZZZZZ', "[ZZZZZ]
[SAP][ASE ODBC Driver][Adaptive Server Enterprise]Stored procedure
'p_procedure' may be run only in unchained transaction mode. The 'SET
CHAINED OFF' command will cause the current session to use unchained
transaction mode.\n (7713) (SQLExecDirectW)")
I attempt to achieve the same using SQLAlchemy (1.0.9):
from sqlalchemy import create_engine, engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.sql import text
url = r'sybase+pyodbc://username:password#dsn'
engine = create_engine(url, echo=True)
sess = sessionmaker(bind=engine).Session()
df = pd.read_sql(text("exec p_procedure #GroupName='GROUP'"),conn.execution_options(autocommit=True))
The error message is the same despite the fact I have specified autocommit=True on the connection. (I have also tested this at the session level but should not be necessary and made no difference).
DBAPIError: (pyodbc.Error) ('ZZZZZ', "[ZZZZZ] [SAP][ASE ODBC
Driver][Adaptive Server Enterprise]....
Can you see anything wrong here?
As always, any help would be much appreciated.

Passing the autocommit=True argument as an item in the connect_args argument dictionary does work:
connect_args = {'autocommit': True}
create_engine(url, connect_args=connect_args)
connect_args – a dictionary of options which will be passed directly
to the DBAPI’s connect() method as additional keyword arguments.

I had some problems with autocommit option. The only thing that worked for me was to change this option to True after establishing connection.
ConnString = 'Driver=%SQL_DRIVER%;Server=%SQL_SERVER%;Uid=%SQL_LOGIN%;Pwd=%SQL_PASSWORD%;'
SQL_CONNECTION = pyodbc.connect(ConnString)
SQL_CONNECTION.autocommit = True