Develop Reference

Connect with Oracle Database via LDAP Authentication

I am using python's pyodbc library to connect with the Oracle Database. The authentication to my non-prod servers were set-up as Basic Authentication so my connection string worked well.
It was tough to understand and set up the right drivers initially, But then did manage to get through. Here is what the code looks like for connection and worked well for my other servers.
import textwrap
import pyodbc
connection_string = textwrap.dedent('''Driver={driver};
DBQ={hostname}:{port}/{sid};
UID={username};
PWD={password};
Connection Timeout=30;
Trusted_Connection="yes"
'''.format(driver = 'Oracle in instantclient_11_2',
hostname = <hostname>,
port = <port>,
sid = <sid>,
username = <username>,
password = <passwd>
))
connection = pyodbc.connect(connection_string)
To my surprise PROD is authenticated via LDAP only. To which I have the LDAP Server, Context, and DB Service along with the credentials.
I tried creating my connection string as
connection_string = textwrap.dedent('''Driver={driver};
DBQ={hostname}:{port}/{sid};
UID={username};
PWD={password};
Connection Timeout=30;
Authentication=LDAP;
Trusted_Connection="yes"
'''.format(driver = 'Oracle in instantclient_11_2',
hostname = <ldap_server>,
port = 389,
sid = <db_service>,
username = <username>,
password = <passwd>
))
To which I was not surprised to know it wont work. Tried going through many links but could not get through. Has anyone tried this before or please let me know what I must be missing.
I could definitely connect in SQL Developer using the LDAP Authentication. And since TNS Listener is not configured on this server I cannot use Basic Authentication to connect.
And help would be welcome on this.

Setting up Oracle Drivers - (This is a pre-requisite if exist please ignore)
Download Oracle Instant Client for Microsoft Windows (x64) 64-bit. Alternatively download available for your 32 bit system.
Choose your version of Oracle Database. My version was 11.2.0.4.0
Download below files -
i. Instant Client Package - Basic
ii. Instant Client Package - ODBC
Unzip both the packages into the same directory such as C:\oracle\instantclient_19_3.
Execute odbc_install.exe from the Instant Client directory. If Instant Client is 11g or lower, start the command prompt with the Administrator privilege.
Create C:\oracle\instantclient_19_3\network\admin folder to place tnsnames.ora, sqlnet.ora.
Set Environment variable - TNS_ADMIN to above folder location.
Add C:\oracle\instantclient_19_3 to PATH Environment variable.
Add C:\oracle\instantclient_19_3 to ORACLE_HOME (optional, if connection doesn't work try this.)
LDAP Authentication
Create two files: sqlnet.ora and ldap.ora
ldap.ora
# Place this file in the network/admin subdirectory or your
# $ORACLE_HOME location.
# LDAP Server name should be added here. Rest all the values remains unchanged.
DIRECTORY_SERVERS = (your-server.your-organization:389:636)
DEFAULT_ADMIN_CONTEXT = "ldap-ou-designation"
DIRECTORY_SERVER_TYPE = OID
sqlnet.ora
# Place this file in the network/admin subdirectory or your
# $ORACLE_HOME location.
SQLNET.AUTHENTICATION_SERVICES=(NTS)
NAMES.DIRECTORY_PATH = (LDAP)
Verify C:\oracle\instantclient_19_3 is present in the Path else try updating the path in the Python Code as below
import os
lib_dir=r"C:\oracle\instantclient_19_3"
os.environ["PATH"] = lib_dir + ";" + os.environ["PATH"]
With all this set-up in place you should have
username, password and DB Service name lets call it as db_service here
with cx_Oracle
import cx_Oracle
con = cx_Oracle.connect('{0}/{1}#{2}'.format(user, password, db_service))
version_script = "SELECT * FROM v$version"
cursor = con.cursor()
cursor.execute(version_script)
version = cursor.fetchall()
print(version[0][0])
Ouput
-----
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
with pyodbc
import pyodbc
con = pyodbc.connect('Driver={0};DBQ={1};UID={2};PWD={3}'.format(driver, db_service, user, password))
version_script = "SELECT * FROM v$version"
cursor = con.cursor()
cursor.execute(version_script)
version = cursor.fetchall()
print(version[0][0])
Ouput
-----
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
#Drivers for pyodbc
pyodbc.drivers()
Output
------
['ODBC Driver 17 for SQL Server',
'Oracle in instantclient_11_2']
References :
https://www.oracle.com/database/technologies/instant-client/winx64-64-downloads.html
https://www.oracle.com/database/technologies/releasenote-odbc-ic.html
https://eikonomega.medium.com/connecting-to-oracle-database-with-cx-oracle-and-ldap-5da7925a305c

Need help in fixing sql connection issue using service account though pyodbc

When I try to run the below code I am getting following error. Tried referring other posts but it does not seem to help.
The server principal "{Myuser}" is not able to access the database "{database}" under the current security context. (916) (SQLExecDirectW)'
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server};"
'Server=XXXXXXX;'
'Port=xxxx'
'Username={service_account}'
'Password={password of service_account}'
'Database={detabase};'
'Trusted_Connection=yes;')
results = cnxn.cursor()
results.execute('SELECT * FROM {database}.dbo.{tablename}')
for row in results:
print(row)

If you are going to use Microsoft's ODBC driver for SQL Server and Trusted_Connection=yes (SQL Server Windows authentication) then you don't pass the Windows credentials in the connection string. Instead, you run your Python app as the Windows user. Ways to do that include
launching the Python app from a Windows command prompt using the RUNAS command, or
Shiftright_clicking your app's icon, then choose "Run as different user".
Another option might be to use the FreeTDS ODBC driver. It supports the older NTLM authentication protocol and allows you to specify the DOMAIN, UID, and PWD (for Windows authentication) in the connection string.

This worked for me:
import pyodbc
password = "<password>"
conn = pyodbc.connect("Driver={SQL Server};Server=<host>;Port=<port>;UID=
<username>;PWD=" + password + ";Database=<dbname>;")
cursor = conn.cursor()
cursor.execute('SELECT * FROM <table>')
for row in cursor:
print(row)

Python DB2 SSL connection

I have been running SQL queries (client side) from DB2 databases using ibm_db & ibm_db_dbi with pandas. However our company implemented new security standards and I would need a way to secure the connection as well.
Running Python3.7 and DB2 10.5
Below is my current connection string:
import ibm_db
import ibm_db_dbi
import pandas as pd
driver = 'IBM DB2 ODBC DRIVER'
database = 'DB0001'
hostname = 'my.host.com'
port = '1234'
protocol = 'TCPIP'
uid = 'user'
pwd = 'password'
security = 'SSL'
dsn = (
f'DRIVER={driver};'
f'DATABASE={database};'
f'HOSTNAME={hostname};'
f'PORT={port};'
f'PROTOCOL={protocol};'
f'UID={uid};'
f'PWD={pwd};'
f'SECURITY={security};'
)
test_query = 'SELECT 1 FROM SYSIBM.SYSDUMMY1'
conn_engine = ibm_db.connect(dsn, '', '')
db_conn = ibm_db_dbi.Connection(conn_engine)
df = pd.read_sql(test_query, db_conn)
Is there any way to incorporate SSL for this code?

This is, unfortunately, a little complicated, and (hopefully) your DBA can help with some of this.
If you're using a Db2 10.5 Fixpack 5 (or newer) client, then you just need to add a couple of parameters in your DSN string:
Security=ssl;
SslServerCertificate=/path/to/file.arm;
Your DBA should be able to provide you with the SslServerCertificate file (or contents).
If you installed a Db2 client separately from the python ibm_db package, then depending on which Db2 client you have installed (i.e. providing the native libraries for ibm_db, you might need to install some additional libraries (the IBM GSKit libraries, which provide the SSL functionality for the Db2 client).

Try to set the following properties:
Security=SSL, SSLClientKeystoredb, SSLClientKeystoreDBPassword or SSLClientKeystash as described at the link.

With some newer versions of the Db2 driver and to connect to Db2 on Cloud and Db2 Warehouse on Cloud, addint just Security=SSL; to the list of parameters works.

Authenticate from Linux to Windows SQL Server with pyodbc

I am trying to connect from a linux machine to a windows SQL Server with pyodbc.
I do have a couple of constraints:
Need to log on with a windows domain account
Need to use python3
Need to do it from Linux to Windows
Need to connect to a specific instance
I set up the environment as described by microsoft and have it working (I can import pyodbc and use the configured mussel driver).
I am not familiar with Windows domain authentication and what not, so there is where my problem is.
My connection string:
DRIVER={ODBC Driver 17 for SQL Server};SERVER=myserver.mydomain.com;PORT=1433;DATABASE=MyDatabase;Domain=MyCompanyDomain;Instance=MyInstance;UID=myDomainUser;PWD=XXXXXXXX;Trusted_Connection=yes;Integrated_Security=SSPI
Supposedly one should use "Trusted_Connection" to use the Windows domain authentication instead of directly authenticating with the SQL server.
The error I get when running pyodbc.connect(connString):
pyodbc.Error: ('HY000', '[HY000] [unixODBC][Microsoft][ODBC Driver 17 for SQL Server]SSPI Provider: No Kerberos credentials available (851968) (SQLDriverConnect)')
From other sources I read this should work on Windows as this code would use the credentials of the currently logged in user.
My question is how can I connect to a Windows SQL Server instance from Linux using Windows Domain credentials.

You must obtain a Kerberos ticket for this to work. Your example doesn't specify whether your Linux system is set up to authenticate via Kerberos or whether you have previously obtained a Kerberos ticket before your code hits your connection string.
If your Linux system is set up to authenticate via Kerberos, then as a proof of concept you can obtain a Kerberos ticket using kinit from the command line. Here's what works for me in python3 running in Ubuntu on Windows via the WSL. The python code:
#!/usr/bin/env python
# minimal example using Kerberos auth
import sys
import re
import pyodbc
driver='{ODBC Driver 17 for SQL Server}'
server = sys.argv[1]
database = sys.argv[2]
# trusted_connection uses kerberos ticket and ignores UID and PASSWORD in connection string
# https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/using-integrated-authentication?view=sql-server-ver15
try:
cnxn = pyodbc.connect(driver=driver, server=server, database=database, trusted_connection='yes')
cursor = cnxn.cursor()
except pyodbc.Error as ex:
msg = ex.args[1]
if re.search('No Kerberos', msg):
print('You must login using kinit before using this script.')
exit(1)
else:
raise
# Sample select query
cursor.execute("SELECT ##version;")
row = cursor.fetchone()
while row:
print(row[0])
row = cursor.fetchone()
print('success')
This tells you if you don't have a ticket. Since it uses a ticket you don't have to specify a user or password in the script. It will ignore both.
Now we run it:
user#localhost:~# kdestroy # make sure there are no active tickets
kdestroy: No credentials cache found while destroying cache
user#localhost:~# python pyodbc_sql_server_test.py tcp:dbserver.example.com mydatabase
You must login using kinit before using this script.
user#localhost:~# kinit
Password for user#DOMAIN.LOCAL:
user#localhost:~# python pyodbc_sql_server_test.py tcp:dbserver.example.com mydatabase
Microsoft SQL Server 2016 (SP2-GDR) (KB4505220) - 13.0.5101.9 (X64)
Jun 15 2019 23:15:58
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows Server 2016 Datacenter 10.0 <X64> (Build 14393: )
success
user#localhost:~#
You may also have success obtaining a Kerberos ticket from python code that runs before you make this connection but that is beyond the scope of this answer. A search for python Kerberos modules might point you toward a solution.
It also appears possible to set up the Linux system so that as soon as a user logs in it automatically obtains a Kerberos ticket that can be passed to other processes. That is also outside of the scope of this answer but a search for automatic Kerberos ticket upon Linux login may yield some clues.

I ended up using the pymssql library which basically is pyodbc on top of the FreeTDS driver. It worked out of the box.
Weird how I had such a hard time discovering this library..

I find two ways for same task. I have MSSQL server with AD auth.
You can use JVM.
Load and install JAVA https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html. Also install JPype1 version 0.6.3 pip install JPype==0.6.3. Version above 0.6.3 won't work correct
import jaydebeapi
import pandas as pd
driver_name = "net.sourceforge.jtds.jdbc.Driver"
connection_url="jdbc:jtds:sqlserver://<server>:<port>/<database name>"
connection_properties = {
"domain": "<domain name>",
"user": "<username>",
"password": "<pwd>"}
jar_path = <path to jsds>"/jtds-1.3.1.jar"
CONN = jaydebeapi.connect(driver_name, connection_url, connection_properties, jar_path)
sql = "SELECT * FROM INFORMATION_SCHEMA.COLUMNS"
df = pd.read_sql(sql, CONN)
This version was too slow for me.
Also You can use pyodbc via FreeTDS. To create a FreeTDS connection Install FreeTDS on your Linux apt-get install tdsodbc freetds-bin, configure FreeTDS /etc/odbcinst.ini like this:
[FreeTDS]
Description=FreeTDS
Driver=/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup=/usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
and turn it on odbcinst -i -d -f /etc/odbcinst.ini
After that, you can use pyodbc
import pandas as pd
import pyodbc
CONN =pyodbc.connect('DRIVER={FreeTDS};'
'Server=<server>;'
'Database=<database>;'
'UID=<domain name>\\<username>;'
'PWD=<password>;'
'TDS_Version=8.0;'
'Port=1433;')
sql = "SELECT * FROM INFORMATION_SCHEMA.COLUMNS"
df = pd.read_sql(sql, CONN)
It's works much faster

I was trying to do the same thing and after reading the OPs answer I tested out pymssql and noticed that it worked with just the below:
pymssql.connect(server='myserver', user='domain\username', password='password', database='mydb')
After realizing that that was all pymssql needed I went back to pyodbc and was able to get it working with:
pyodbc.connect("DRIVER={FreeTDS};SERVER=myserver;PORT=1433;DATABASE=mydb;UID=domain\username;PWD=password;TDS_Version=8.0")
I just wanted to thank you for posting this as it helped me so greatly!!!! :)

Generating windows authentication via Linux is complex. EasySoftDB (commercial) used to be able to handle this, and FreeTDS has some convoluted support.
https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/using-integrated-authentication
My suggestion is to move away from Windows Authentication and use SQL authentication. Really there is no security difference, except that you are providing a username and password in the connection string. But this would make your life a lot easier.

I had the same issue and got the docker container for airflow using windows authentication by adding a few things to my airflow build. The apt install needs to be run as root.
USER root
RUN apt install -y krb5-config
RUN apt-get install -y krb5-user
COPY krb5.conf /etc/krb5.conf
In the krb5.conf file
[appdefaults]
default_lifetime = 52hrs
krb4_convert = false
krb4_convert_524 = false
ksu = {
forwardable = false
}
pam = {
minimum_uid = 100
forwardable = true
}
pam-afs-session = {
minimum_uid = 100
}
[libdefaults]
default_realm = DEFAULT_DOMAIN
ticket_lifetime = 52h
renew_lifetime = 90d
forwardable = true
noaddresses = true
allow_weak_crypto = true
rdns = false
[realms]
MY.COMPANY.LOCAL = {
kdc = SERVER.DEFAULT_DOMAIN
default_domain = DEFAULT_DOMAIN
}
[domain_realm]
my.company.local = DEFAULT_DOMAIN
[logging]
kdc = SYSLOG:NOTICE
admin_server = SYSLOG:NOTICE
default = SYSLOG:NOTICE
DEFAULT_DOMAIN for me is DOMAIN.COMPANY.COM. Others have .LOCAL at the end. Make sure it is all caps in the file. I had an error the first time I tried to authenticate.
Rebuild and then launch the shell for the airflow worker.
Run kinit USER
It will prompt for a password. Running klist afterwards to confirm you have a ticket. Once you get this working you should be able to authenticate to the server from python.

DB2 using pyodbc on Mac OSX

I'm trying to connect to a DB2 database using pyodbc on Mac OS X and I can't seem to get this going. I have had success using FreeTDS to connect to MS SQL Server under similar conditions.
First, I installed db2exc_client_952_MAC_x86_64 from IBM. After successfully installing that, I went to add the driver via ODBC Administrator.
Then I've tried running this python:
import pyodbc
cnxn = pyodbc.connect('Driver={IBM DB2 ODBC Driver}; Hostname=myhost.com; Port=50300; Protocol=TCPIP; Database=DB2ET; CurrentSchema=SCHM1; UID=MYUID; PWD=MYPWD;');
And I receive this error:
Traceback (most recent call last):
File "", line 1, in
pyodbc.Error: ('HY000', '[] \xf8\x92\x90\x81\x9b\xf8\x93\x90\x81\x82\xf8\x96\xb0\x81\x9d\xf8\x93\x80\x81\x83\xf8\x88\x80\x81\x89\xf8\x9c\xa0\x81\x84\xf8\x9d\xa0\x81\xa9\xf8\x9c\xa0\x81\xa5\xf8\x88\x80\x81\x9d\xf8\x94\x90\x81\x93\xf8\x8c\x90\x81\x8c\xf8\x8c\xa0\x80\xb0\xf8\x93\xa0\x80\xb4\xf8\x88\x80\x80\xa0\xf8\x88\x80\x81\x81\xf8\x98\x90\x81\xa4\xf8\x98\x90\x81\xb4\xf8\x98\x90\x81\xa2\xf8\x99\x90\x81\xb3\xf8\x98\xb0\x80\xa0\xf8\x9b\xa0\x81\xaf\xf8\x99\x90\x81\xae\xf8\x9d\x80\x81\xa3\xf8\x9b\xb0\x81\xa9\xf8\x88\x80\x81\xae\xf8\x9b\xb0\x81\xa4\xf8\x9c\xb0\x81\xa5\xf8\x9b\xa0\x80\xa0\xf8\x9d\x80\x81\xaf\xf8\x99\x90\x80\xa0\xf8\x9a\x90\x81\xb8\xf8\x9d\x80\x81\xb3\xf8\x88\x80\x80\xae\xf8\x94\xb0\x80\xa0\xf8\x93\x80\x81\x91\xf8\x95\x80\x81\x93\xf8\x95\x80\x81\x81\xf8\x8f\x90\x81\x85\xf8\x8e\x80\x80\xb0\xf8\x8c\x80\x80\xb0\xf2\xa0\x80\xb3\xfc\xa0\x9c\xb0\x80\x80\xfd\xa1\x99\x82\x81\xa1\xfd\xa1\x98\xa6\x85\xb4\xfd\xa9\x88\x86\x95\xb3\xfd\x82\x91\x82\x81\xae\xfd\xaf\x99\xa2\x80\xb2\xfd\x96\x93\x92\x81\xb2\xfd\x93\x82\xa2\xb9\x93\xff\x7f (-1024) (SQLDriverConnect)')
I gathered from searching that HY000 may be a connectionString problem, but I'm not sure how to decipher the rest of the error.
Any ideas?

It appears connection strings vary by database and the connection string you are using is not compatible with DB2.
Here's some DB2 example connection strings:
http://www.connectionstrings.com/ibm-db2
Here's the best reference I could find as to what is valid in a DB2 connection string:
http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=%2Fcom.ibm.swg.im.dbclient.adonet.ref.doc%2Fdoc%2FDB2ConnectionClass.html
Based on that, I'd convert your connection string from:
Driver={IBM DB2 ODBC Driver}; Hostname=myhost.com; Port=50300; Protocol=TCPIP; Database=DB2ET; CurrentSchema=SCHM1; UID=MYUID; PWD=MYPWD;
To:
Driver={IBM DB2 ODBC Driver}; Server=myhost.com:50300; Database=DB2ET; CurrentSchema=SCHM1; UID=MYUID; PWD=MYPWD;

Here's what my connection string looks like in PHP (I know you're using Python but I think they're very similar):
$this->db_connection = new PDO("odbc:DRIVER={iSeries Access ODBC Driver};SYSTEM=10.xxx.xxx.xxx;PROTOCOL=TCPIP", $temp_username, $temp_password);
Notice that I say "SYSTEM=..." and you say "Hostname=..." - I think that could be important.
Is your DSN set up completely? On my Linux machines I have two files that contain all of our configuration information for DSN's.
odbc.ini
[primary]
Description = primary
Driver = iSeries Access ODBC Driver
System = xxx.xxx.xxx.xxx
UserID = xxxxxxxxxx
Password = xxxxxxxxxx
Naming = 0
DefaultLibraries = QGPL
Database = xxxxxxxxxx
ConnectionType = 0
CommitMode = 2
ExtendedDynamic = 0
DefaultPkgLibrary = QGPL
DefaultPackage = A/DEFAULT(IBM),2,0,1,0,512
AllowDataCompression = 1
LibraryView = 0
AllowUnsupportedChar = 0
ForceTranslation = 0
Trace = 0
and odbcinst.ini
[iSeries Access ODBC Driver]
Description = iSeries Access for Linux ODBC Driver
Driver = /usr/lib/libcwbodbc.so
Setup = /usr/lib/libcwbodbcs.so
NOTE1 = If using unixODBC 2.2.11 or later and you want the 32 and 64-bit ODBC drivers to share DSN's,
NOTE2 = the following Driver64/Setup64 keywords will provide that support.
Driver64 = /usr/lib/lib64/libcwbodbc.so
Setup64 = /usr/lib/lib64/libcwbodbcs.so
Threading = 2
DontDLClose = 1
UsageCount = 1
I believe you enter similar information in the dialog you show, probably under the DSN tab.
Have you enabled ODBC tracing? To do this in Linux I have to add this to the odbcinst.ini file:
[ODBC]
Trace = Yes
TraceFile = /tmp/odbc.log
This dumped a LOT of information for me and helped to diagnose some problems I've had in the past.
According to this page (http://code.google.com/p/pyodbc/wiki/ConnectionStrings), "The most important thing to know is that pyodbc does not even look at the connection string -- it is passed directly to SQLDriverConnect unmodified." If that's true then changing your "Hostname" to "SYSTEM" should do the trick.
EDIT
Any chance this could be a problem with encoding? Unicode trying to talk to a UTF-8 server (or something similar)? Have you tried setting the CCSID? When you configure the driver do you have options such as "BinAsChar", "CCSID" or "Host CCSID"?
By default I believe the CCSID is 37 (US/Canada) and I think Unicode is 1208.

Why not define a User DSN using a driver and then use the DSN in the connect() method?
pyodbc.connect( 'DSN=MYDSN;UID=MYUSER;PWD=MYPASS' )
And by the way using any driver other than the "iSeries Access ODBC Driver" might raise those DB2 licensing errors. Maybe this is what the message is saying.
The driver is installed when you install "IBM i Access for Windows 7.1"