Develop Reference

Connect to remote MSSQL 2016 using isql and python on Mac?

Goal: Connect to remote MSSQL 2016 server via Python.
Main approach: Closely followed tutorial in https://github.com/mkleehammer/pyodbc/wiki/Connecting-to-SQL-Server-from-Mac-OSX .
Problem: Able to connect via tsql, but isql is not working. Errors
[S1000][unixODBC][FreeTDS][SQL Server]Unable to connect to data source
[37000][unixODBC][FreeTDS][SQL Server]Login failed for user 'DOMAIN\user-p'
[37000][unixODBC][FreeTDS][SQL Server]Cannot open database "TIT_BI_OPERATIONS" requested by the login. The login failed.
Things tried:
Different ODBC drivers 13.1, 17, FreeTDS
Inclusion/exclusion of escape character in the user name.
Host name vs host ip.
Settings:
odbc.ini
[ODS_DSN]
Description = Connection to ODS MS_SQL 2016
Driver = FreeTDS
Servername = ODS_DSN
Port = 40000
Database = TIT_BI_OPERATIONS
odbcinst.ini
[FreeTDS]
Driver=/usr/local/lib/libtdsodbc.so
Setup=/usr/local/lib/libtdsodbc.so
UsageCount=1
freetds.conf
[ODS_DSN]
host = 164.10.17.77
port = 40000
tds version = 7.4
client charset = UTF-8
Notes:
Even though, its not very promising to run python without connecting through tsql and isql first, i still tried without success. Using pyodbc, pypodbc, sqlalchemy.
Most errors in the form: Login failed for user 'DOMAIN\user-p'
For ODBC driver 13: Can't open lib '/usr/local/lib/libmsodbcsql.13.dylib'
I am able to connect via SQL PRO STUDIO, using exact same credentials.
If you have any thoughts which direction to go to climb out of this connection problem, it would be greatly appreciated. Thank you!

If you're using Windows domain auth, you'll have to use FreeTDS. Oddly enough, Windows domain auth isn't supported by the Microsoft ODBC Driver, only FreeTDS.
Since you can connect with the tsql command, that means FreeTDS is working. I'd recommend connecting directly from Python explicitly. Try a connection string like this:
import pyodbc
con = pyodbc.connect(
r"DRIVER={FreeTDS};"
r"SERVER=164.10.17.77;"
r"PORT=40000;"
r"DATABASE=TIT_BI_OPERATIONS;"
f"UID=DOMAIN\\user-p;"
f"PWD=yourpassword;"
r"TDS_Version=7.3;"
)
cursor = con.cursor();
cursor.execute("SELECT 'this' AS that")
for row in cursor.fetchall():
print(row)
Note that you do need two backslashes in the UID field to connect with Windows domain auth; that is not a typo!

Python DB2 SSL connection

I have been running SQL queries (client side) from DB2 databases using ibm_db & ibm_db_dbi with pandas. However our company implemented new security standards and I would need a way to secure the connection as well.
Running Python3.7 and DB2 10.5
Below is my current connection string:
import ibm_db
import ibm_db_dbi
import pandas as pd
driver = 'IBM DB2 ODBC DRIVER'
database = 'DB0001'
hostname = 'my.host.com'
port = '1234'
protocol = 'TCPIP'
uid = 'user'
pwd = 'password'
security = 'SSL'
dsn = (
f'DRIVER={driver};'
f'DATABASE={database};'
f'HOSTNAME={hostname};'
f'PORT={port};'
f'PROTOCOL={protocol};'
f'UID={uid};'
f'PWD={pwd};'
f'SECURITY={security};'
)
test_query = 'SELECT 1 FROM SYSIBM.SYSDUMMY1'
conn_engine = ibm_db.connect(dsn, '', '')
db_conn = ibm_db_dbi.Connection(conn_engine)
df = pd.read_sql(test_query, db_conn)
Is there any way to incorporate SSL for this code?

This is, unfortunately, a little complicated, and (hopefully) your DBA can help with some of this.
If you're using a Db2 10.5 Fixpack 5 (or newer) client, then you just need to add a couple of parameters in your DSN string:
Security=ssl;
SslServerCertificate=/path/to/file.arm;
Your DBA should be able to provide you with the SslServerCertificate file (or contents).
If you installed a Db2 client separately from the python ibm_db package, then depending on which Db2 client you have installed (i.e. providing the native libraries for ibm_db, you might need to install some additional libraries (the IBM GSKit libraries, which provide the SSL functionality for the Db2 client).

Try to set the following properties:
Security=SSL, SSLClientKeystoredb, SSLClientKeystoreDBPassword or SSLClientKeystash as described at the link.

With some newer versions of the Db2 driver and to connect to Db2 on Cloud and Db2 Warehouse on Cloud, addint just Security=SSL; to the list of parameters works.

Authenticate from Linux to Windows SQL Server with pyodbc

I am trying to connect from a linux machine to a windows SQL Server with pyodbc.
I do have a couple of constraints:
Need to log on with a windows domain account
Need to use python3
Need to do it from Linux to Windows
Need to connect to a specific instance
I set up the environment as described by microsoft and have it working (I can import pyodbc and use the configured mussel driver).
I am not familiar with Windows domain authentication and what not, so there is where my problem is.
My connection string:
DRIVER={ODBC Driver 17 for SQL Server};SERVER=myserver.mydomain.com;PORT=1433;DATABASE=MyDatabase;Domain=MyCompanyDomain;Instance=MyInstance;UID=myDomainUser;PWD=XXXXXXXX;Trusted_Connection=yes;Integrated_Security=SSPI
Supposedly one should use "Trusted_Connection" to use the Windows domain authentication instead of directly authenticating with the SQL server.
The error I get when running pyodbc.connect(connString):
pyodbc.Error: ('HY000', '[HY000] [unixODBC][Microsoft][ODBC Driver 17 for SQL Server]SSPI Provider: No Kerberos credentials available (851968) (SQLDriverConnect)')
From other sources I read this should work on Windows as this code would use the credentials of the currently logged in user.
My question is how can I connect to a Windows SQL Server instance from Linux using Windows Domain credentials.

You must obtain a Kerberos ticket for this to work. Your example doesn't specify whether your Linux system is set up to authenticate via Kerberos or whether you have previously obtained a Kerberos ticket before your code hits your connection string.
If your Linux system is set up to authenticate via Kerberos, then as a proof of concept you can obtain a Kerberos ticket using kinit from the command line. Here's what works for me in python3 running in Ubuntu on Windows via the WSL. The python code:
#!/usr/bin/env python
# minimal example using Kerberos auth
import sys
import re
import pyodbc
driver='{ODBC Driver 17 for SQL Server}'
server = sys.argv[1]
database = sys.argv[2]
# trusted_connection uses kerberos ticket and ignores UID and PASSWORD in connection string
# https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/using-integrated-authentication?view=sql-server-ver15
try:
cnxn = pyodbc.connect(driver=driver, server=server, database=database, trusted_connection='yes')
cursor = cnxn.cursor()
except pyodbc.Error as ex:
msg = ex.args[1]
if re.search('No Kerberos', msg):
print('You must login using kinit before using this script.')
exit(1)
else:
raise
# Sample select query
cursor.execute("SELECT ##version;")
row = cursor.fetchone()
while row:
print(row[0])
row = cursor.fetchone()
print('success')
This tells you if you don't have a ticket. Since it uses a ticket you don't have to specify a user or password in the script. It will ignore both.
Now we run it:
user#localhost:~# kdestroy # make sure there are no active tickets
kdestroy: No credentials cache found while destroying cache
user#localhost:~# python pyodbc_sql_server_test.py tcp:dbserver.example.com mydatabase
You must login using kinit before using this script.
user#localhost:~# kinit
Password for user#DOMAIN.LOCAL:
user#localhost:~# python pyodbc_sql_server_test.py tcp:dbserver.example.com mydatabase
Microsoft SQL Server 2016 (SP2-GDR) (KB4505220) - 13.0.5101.9 (X64)
Jun 15 2019 23:15:58
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows Server 2016 Datacenter 10.0 <X64> (Build 14393: )
success
user#localhost:~#
You may also have success obtaining a Kerberos ticket from python code that runs before you make this connection but that is beyond the scope of this answer. A search for python Kerberos modules might point you toward a solution.
It also appears possible to set up the Linux system so that as soon as a user logs in it automatically obtains a Kerberos ticket that can be passed to other processes. That is also outside of the scope of this answer but a search for automatic Kerberos ticket upon Linux login may yield some clues.

I ended up using the pymssql library which basically is pyodbc on top of the FreeTDS driver. It worked out of the box.
Weird how I had such a hard time discovering this library..

I find two ways for same task. I have MSSQL server with AD auth.
You can use JVM.
Load and install JAVA https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html. Also install JPype1 version 0.6.3 pip install JPype==0.6.3. Version above 0.6.3 won't work correct
import jaydebeapi
import pandas as pd
driver_name = "net.sourceforge.jtds.jdbc.Driver"
connection_url="jdbc:jtds:sqlserver://<server>:<port>/<database name>"
connection_properties = {
"domain": "<domain name>",
"user": "<username>",
"password": "<pwd>"}
jar_path = <path to jsds>"/jtds-1.3.1.jar"
CONN = jaydebeapi.connect(driver_name, connection_url, connection_properties, jar_path)
sql = "SELECT * FROM INFORMATION_SCHEMA.COLUMNS"
df = pd.read_sql(sql, CONN)
This version was too slow for me.
Also You can use pyodbc via FreeTDS. To create a FreeTDS connection Install FreeTDS on your Linux apt-get install tdsodbc freetds-bin, configure FreeTDS /etc/odbcinst.ini like this:
[FreeTDS]
Description=FreeTDS
Driver=/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup=/usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
and turn it on odbcinst -i -d -f /etc/odbcinst.ini
After that, you can use pyodbc
import pandas as pd
import pyodbc
CONN =pyodbc.connect('DRIVER={FreeTDS};'
'Server=<server>;'
'Database=<database>;'
'UID=<domain name>\\<username>;'
'PWD=<password>;'
'TDS_Version=8.0;'
'Port=1433;')
sql = "SELECT * FROM INFORMATION_SCHEMA.COLUMNS"
df = pd.read_sql(sql, CONN)
It's works much faster

I was trying to do the same thing and after reading the OPs answer I tested out pymssql and noticed that it worked with just the below:
pymssql.connect(server='myserver', user='domain\username', password='password', database='mydb')
After realizing that that was all pymssql needed I went back to pyodbc and was able to get it working with:
pyodbc.connect("DRIVER={FreeTDS};SERVER=myserver;PORT=1433;DATABASE=mydb;UID=domain\username;PWD=password;TDS_Version=8.0")
I just wanted to thank you for posting this as it helped me so greatly!!!! :)

Generating windows authentication via Linux is complex. EasySoftDB (commercial) used to be able to handle this, and FreeTDS has some convoluted support.
https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/using-integrated-authentication
My suggestion is to move away from Windows Authentication and use SQL authentication. Really there is no security difference, except that you are providing a username and password in the connection string. But this would make your life a lot easier.

I had the same issue and got the docker container for airflow using windows authentication by adding a few things to my airflow build. The apt install needs to be run as root.
USER root
RUN apt install -y krb5-config
RUN apt-get install -y krb5-user
COPY krb5.conf /etc/krb5.conf
In the krb5.conf file
[appdefaults]
default_lifetime = 52hrs
krb4_convert = false
krb4_convert_524 = false
ksu = {
forwardable = false
}
pam = {
minimum_uid = 100
forwardable = true
}
pam-afs-session = {
minimum_uid = 100
}
[libdefaults]
default_realm = DEFAULT_DOMAIN
ticket_lifetime = 52h
renew_lifetime = 90d
forwardable = true
noaddresses = true
allow_weak_crypto = true
rdns = false
[realms]
MY.COMPANY.LOCAL = {
kdc = SERVER.DEFAULT_DOMAIN
default_domain = DEFAULT_DOMAIN
}
[domain_realm]
my.company.local = DEFAULT_DOMAIN
[logging]
kdc = SYSLOG:NOTICE
admin_server = SYSLOG:NOTICE
default = SYSLOG:NOTICE
DEFAULT_DOMAIN for me is DOMAIN.COMPANY.COM. Others have .LOCAL at the end. Make sure it is all caps in the file. I had an error the first time I tried to authenticate.
Rebuild and then launch the shell for the airflow worker.
Run kinit USER
It will prompt for a password. Running klist afterwards to confirm you have a ticket. Once you get this working you should be able to authenticate to the server from python.

pyodbc connection to Teradata not working after upgrade to Teradata 16.00 driver

I have a Windows application that utilizes pyodbc to connect to Teradata. Currently, the clients have either the 14.10 or 15.00 drivers installed to perform this connection. The connection is made using this (simplified) code:
import pyodbc
constr = 'DRIVER={Teradata};DBCNAME='+dbname+';UID='+uid+';PWD='+pwd+';QUIETMODE=YES;'
pyodbc.pooling = False
pyodbc.connect(constr, ANSI=True, autocommit=True)
After upgrading to the 16.00 driver, this no longer works. Instead it throws the following error on the same code:
pyodbc.Error: ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
I've tried a few variations of the connection string but all return the same error:
constr = """DRIVER={Teradata};DSN='+dbname+';UID='+uid+';PWD='+pwd+';QUIETMODE=YES;"""
constr = """DRIVER={Teradata};DSN='+dbname+';DATABASE='+dbname+';UID='+uid+';'+pwd+';QUIETMODE=YES;"""
constr = """Provider=Teradata;DBCNAME='+dbname+';DATABASE='+dbname+';UID='+uid+';PWD='+pwd+';QUIETMODE=YES;"""
What do I need to do to utilize 16.00 teradata drivers and pyodbc?

I had the exact same issue. This recommendation may sound silly, but my server admins changed the name of the connection string and Windows SQLDriverConnect was bombing.
On my workstation, in the DSN connection list, the DSN name is "TDPROD" and the string description is "Teradata". However, on the server, the DSN name is "TDPROD" and the string description is "Teradata Database ODBC Driver 16.10". So, I had to update my python function from this (Please note that I'm using Python 3.6 so I can use "f" strings; if you're using an earlier version of Python make sure you perform string interpolation by using supported methodologies):
pyodbc.connect(f"""DRIVER=Teradata;
DBCNAME=TDPROD;
UID={user};
PWD={password};
QUIETMODE=YES""",
autocommit=True,
unicode_results=True)
to this:
pyodbc.connect(f"""DRIVER=Teradata Database ODBC Driver 16.10;
DBCNAME=TDPROD;
UID={user};
PWD={password};
QUIETMODE=YES""",
autocommit=True,
unicode_results=True)
Hope this helps to at least give you something else to check that wasn't immediately obvious to me.

Python - Linux - Connecting to MS SQL with Windows Credentials - FreeTDS+UnixODBC + pyodbc or pymssql

There doesn't seem to be any great instructions for setting this up. Does anyone have any good instructions? I am a linux noob so be gentle. I did see another post that is similar, but no real answer.
I have a couple of problems.
FreeTDS doesn't "seem" to be working. I am trying to connect and I get the following message using the "tsql" command: "Default database being set to databaseName
There was a problem connecting to the server" but it doesn't mention what the problem is.
The error I get when I try to connect using pyodbc is: "pyodbc.Error: ('08S01', '[08S01] [unixODBC][FreeTDS][SQL Server]Unable to connect: Adaptive Server is unavailable or does not exist (20009) (SQLDriverConnectW)')"
I tried something similar with pymssql, but I ran into similar issues. I keep getting errors that I can't connect, but it doesn't tell me why.

The following works if you configure the MS SQL server to allow remote TCP/IP connections and have an appropriate user to connect as.
You also need to be careful to set up the correct hostname for the db as reported by MS SQL.
import pymssql
connection = pymssql.connect(
user = 'username',
password = 'password',
host = 'server',
database = 'database',
)
cursor = connection.cursor()
cursor.execute('select * from db;')
rows = cursor.fetchall()

When building FreeTDS (http://www.freetds.org/userguide/config.htm):
./configure --with-tdsver=8.0 --enable-msdblib

That error suggests that the TDS version is not set right. You can set that in the configuration setting for FreeTDS. You don't mention which MSSQL version you are using. But, if you are using say 2005, setting 8.0 as the TDS version will work.