Develop Reference

Error: `pyodbc.drivers()` returns empty list inside VSCode

I followed Microsoft instructions for installing its ODBC driver and prepared a conda environment with the necessary packages to work with azure SQL database with Python, including pyodbc. I used the instructions for Ubuntu because I'm on Linux Mint 20 and well, that option isn't available (and Linux Mint is based on Ubuntu).
Then I prepared a small script to connect to a test azure database that I created:
import pyodbc
configString = (
f'DRIVER={ODBC Driver 17 for SQL Server};'
f'SERVER={server};'
'PORT=1433;'
f'DATABASE={database};'
f'UID={username};'
f'PWD={password}'
)
with pyodbc.connect(configString) as conn:
with conn.cursor() as cursor:
cursor.execute('SELECT TOP 3 name, collation_name FROM sys.databases')
row = cursor.fetchone()
while row:
print (str(row[0]) + ' ' + str(row[1]))
row = cursor.fetchone()
This fails with the error ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)").
But the file exists, and if I run this from just a terminal (instead of from within VSCode) it works fine.
Also, pyodbc.drivers() resturns [] from within VSCode but it returns the correct value ['ODBC Driver 17 for SQL Server'] when called from a Python shell within a terminal.
So the question is: why pyodbc fails to get the drivers from within VSCode but gets the correct values from a terminal?

InterfaceError with sqlalchemy and Microsoft Access Driver in Python

I want to connect to a MS Acces database that is on my host system.
I am using Python 3.7 in jupyter notebook. When I connect to the engine, I get the exception InterfaceError.
My Code:
import urllib
from sqlalchemy import create_engine
connection_string = (
r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};'
r'DBQ=./Datenbank1.accdb;'
)
connect_str = f"access+pyodbc:///?odbc_connect={urllib.parse.quote_plus(connection_string)}"
engine = create_engine(connect_str, echo=True)
engine.connect()
InterfaceError: (pyodbc.InterfaceError) ('IM002', '[IM002]
[Microsoft][ODBC Driver Manager] Der Datenquellenname wurde nicht
gefunden, und es wurde kein Standardtreiber angegeben (0)
(SQLDriverConnect)') (Background on this error at:
http://sqlalche.me/e/13/rvf5)
The data source name was not found and no default driver was specified (0) (SQLDriverConnect)')
Can you help me to find the error?
Maybe I am missing the right driver but I don't know how to install it

On Windows, one way to tell if a Python script is running as 32-bit or 64-bit is to check the output from pyodbc.drivers(). In 32-bit mode, the list will include the older "Jet" driver …
Microsoft Access Driver (*.mdb)
… (note: no mention of *.accdb) which ships with Windows and is only available as 32-bit.
In this case the list returned by pyodbc.drivers() did not include that driver so we can deduce that the script is running as 64-bit. Therefore the solution is to download the 64-bit version of the newer "ACE" ODBC driver from here.
For more details on connecting to Access via pyodbc, see the pyodbc wiki.

Sym linking issue connecting to Azure Database from macOS

I am trying to get the data from my SQL Azure Database. It seems like my python code throws the error because of linking confusion with ODBC Drivers.
Here is my Python code.
from urllib import parse
from sqlalchemy import create_engine
connecting_string = 'Driver={ODBC Driver 13 for SQL Server};Server=tcp:mftaccountinghost.database.windows.net,1433;Database=mft_accounting;Uid=localhost;Pwd=######;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30'
params = parse.quote_plus(connecting_string)
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
result = connection.execute("select 1+1 as res")
for row in result:
print("res:", row['res'])
connection.close()
Here is the error I get:
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libmsodbcsql.13.dylib' : file not found (0) (SQLDriverConnect)")
When I check my terminal I get the following results. Cannot resolve this issue...
Here is my connection string for the server:

Here is complete guide to install the ODBC driver in UNIX machine:
https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15
If you installed the v17 msodbcsql package that was briefly available, you should remove it before installing the msodbcsql17 package. This will avoid conflicts.
Try uninstalling the driver and install it again it should work.
Also check below thread for additional reference:
Can't open lib 'ODBC Driver 13 for SQL Server'? Sym linking issue?
Hope it helps.

pyodbc connection with Advantage ODBC Driver (Linux)

I am trying to connect to an existing Sybase Advantage Database Server via the ODBC driver on a LOCAL instance. I currently have unixodbc, unixodbc-dev, and unixodbc-bin installed.
When I attempt the following:
import pyodbc
str='DRIVER={Advantage ODBC Driver};DataDirectory=/var/lib/advantage/.../dbfile.add;User ID=...;Password=...;ServerTypes=1;'
connection = pyodbc.connect(str)
I get the following error:
pyodbc.Error: ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')
Here's my /etc/odbc.ini (and /etc/odbcinst.ini) file:
;
; odbc.ini
;
[ODBC Data Sources]
Odie = Advantage ODBC Driver
[Odie]
Driver=/opt/ads/odbc/redistribute/libadsodbc.so.11.10.0.24
DataDirectory=/var/lib/advantage/.../dbfile.add
Description=Advantage ODBC driver
Rows=False
MemoBlockSize=64
DefaultType=Advantage
MaxTableCloseCache=0
LOCKING=Record
CharSet=OEM
ADVANTAGELOCKING=OFF
ServerTypes=1
TableExtension=
I see three potential issues here - either my connection string is wrong, my odbc.ini file is incorrectly setup, or my unixodbc hasn't reloaded the odbc.ini since I modified it (if there is such a thing). I have attempted the solution proposed here, without avail.
Thanks for your help!

How to access Hive on remote server using python client

Case: I have Hive on a cloudera platform. There is a database on Hive that I want to access using python client from my computer. I read a similar SO question but its using pyhs2 which I am unable to install on the remote server. And this SO question too uses Thrift but I cant seem to install it either.
Code: After following the documentation, when I execute the following program it gives me an error.
import pyodbc, sys, os
pyodbc.autocommit=True
con = pyodbc.connect("DSN=default",driver='SQLDriverConnect',autocommit=True)
cursor = con.cursor()
cursor.execute("select * from fb_mpsp")
Error: ssh://ashish#ServerIPAddress/home/ashish/anaconda/bin/python2.7 -u /home/ashish/PyCharm_proj/hdfsConnect/home/ashish/PyCharm_proj/hdfsConnect/Hive_connect/hive_connect.py
Traceback (most recent call last):
File "/home/ashish/PyCharm_proj/hdfsConnect/home/ashish/PyCharm_proj/hdfsConnect/Hive_connect/hive_connect.py", line 5, in
con = pyodbc.connect("DSN=default", driver='SQLDriverConnect',autocommit=True)
pyodbc.Error: ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')
Process finished with exit code 1
Please suggest how can I solve this problem? Also I am not sure why do I have to specify the driver as SQLDriverConnect when the code will be executed using hadoop hive?
Thanks

This worked for me
oODBC = pyodbc.connect("DSN=Cloudera Hive DSN 64;", autocommit = True, ansi = True )
And now everything works fine.
Be sure anything is fine with you DSN using:
isql -v "Cloudera Hive DSN 64"
and replace "Cloudera Hive DSN 64" with the name you used in your odbc.ini
Also, currently I'm not able to use the kerberos authentication unless I make a ticket by hand. Impala works smoothly using kerberos keytab files
Any help about how to have hive odbc working with keytab files is appreciated.

If you do decide to revisit pyhs2 note that it doesn't need to be installed on the remote server, it's installed on your local client.
If you continue with pyodbc, you need to install the ODBC driver for Hive, which you can get from Cloudera's site.
You don't need to specify the driver in your connection, it should be part of your DSN. The specifics of creating the DSN depend on your OS, but essentially you will create it using Administrative Tools -> Data Sources (Windows), install ODBC and edit /Library/ODBC/odbc.ini (Mac), or edit /etc/odbc.ini (Linux).
Conceptually, think of the DSN as a specification that represents all the information about the connection - it will contain the host, port, and driver information. That way in your code you don't have to specify these things and you can switch details about the database without changing your code.
# Note only the DSN name specifies the connection
import pyodbc
conn = pyodbc.connect("DSN=Hive1")
cursor = conn.cursor()
cursor.execute("select * from YYY")
Alternatively, I've updated the other question you referenced with information about how to install the thrift libraries. I think that's the way to go, if you have that option.

Try this method also to conenct and get data remotely from hive server:
connect remote server with ssh and give the cli command to access data from remote server:
ssh -o UserKnownHostsFile=/dev/null -o ConnectTimeout=90 -o StrictHostKeyChecking=no shashanks#remote_host 'hive -e "select * from DB.testtable limit 5;" >/home/shashanks/testfile'