Develop Reference

Unable to connect to SQL using pyodbc-Python

I try to connect to SQL using the pyodbc package, but get the error:
SystemError:
built-in function connect returned NULL without setting an error
What could be the reason for this?
When I run the code from another computer - I do manage to connect to SQL.
The connection string I use:
conn_str = 'DRIVER={SQL Server};SERVER=' + server + ';DATABASE=' + database + ';'
conn = pyodbc.connect(conn_str)
I have ODBC and pyodbc installed on my computer.
Of course I set values for the database and server fields
Before I do connect I use a function that does logon with the username and password

Because of your question do not have a reproductible exemple i can't be 100% of what I will say;
This is probably a driver problem, if you have acces to db2dsdriver.cfg file or db2cli.ini you shoyld try to look in those files to find what is the exact name of the driver.
Following this could help you to have more information on the problem.

Connect to db2 database with Python

I'm working on an app that needs to connect to an ibm db2 database. Using DBeaver I can successfully connect to the database (I provide him the db2cc.jar and db2cc4.jar files).
It looks to me as DBeaver is using my Window's credentials to login, because I didn't need to input any login or password to connect.
Now, I've been trying to connect to the same database using python 3.7 and pypi's latest version of the ibm_db package. I didn't install anything else.
import ibm_db
# ...
connection_string = "DATABASE=" + self.params['schema'] + ";" + \
"HOSTNAME=" + self.params['host'] + ";" + \
"PORT=" + self.params['port'] + ";" + \
"PROTOCOL=TCPIP;" + \
"SECURITYMECHANISM=4;" + \
"UID=" + self.params['user'] + ";" + \
"PWD=" + self.params['password'] + ";"
try:
self.connection = ibm_db.connect(connection_string, "", "")
# ...
Using my Windows credentials in the parameters, I get the following error message:
Connection error
Bad credentials
SQLCODE=-30082
08001
From what I've seen on stack overflow connecting to a db2 database is complicated...
Does someone know how to connect ? Using the windows credentials or otherwise...
Thanks !

Db2 works fine with Python.
You need to be aware of some basics before you start such as:
what operating-system runs the target Db2-database and
what kind of client is being used (java, odbc/cli, .net etc), and
what kind of authentication/encrpytion is in place for the database
(ssl, server based authentication/+/-/encryption etc.).
is the remote database rdbms Apache DERBY or Db2.
Find out these basics before you start. You have to speak with people who run the Db2-server.
Note: in your question you mention (SecurityMechanism=4) com.ibm.db2.jcc.DB2BaseDataSource.USER_ONLY_SECURITY - this is not relevant for non-JAVA clients, it is relevant is the database manager is DERBY .
For python, the ibm_db package is not a java application.
DBeaver is a java application (hence it uses db2jcc.jar or db2jcc4.jar and a licence-file to connect to the remote database).
You can only use your Windows credentials for connecting to a Db2-database when that Db2-database run on Microsoft-Windows, and the credentials work on the hostname running the Db2-server. For any other combinations, the administrator must issue you a userid/password that is relevant for the target hostname.
The ibm_db package needs a Db2-client to be installed. The Db2-client is a separate installable. There are different kinds of Db2-client depending both on which operating-system runs your Db2-server and how much functionality you need to have in your Db2-client. If your remote Db2-server runs on Linux, Unix, Windows or Z/OS then you can use the "IBM Data Server Runtime Client" which you can either download from IBM's passport advantage website, or get from your internal IT folks. If your Db2-server runs on i-Series (AS/400) you should get its drivers from your i-Series administrator. For either Z/OS or i-Series you will additionally need a license file (which costs money) and you should get that from your administrator, unless your company uses a gateway product called Db2-connect in which case you don't need a separate license file on your workstation.

Try the following connection string, If your db2 client and server are on the same host.
Change the 'mydb' and 'DB2' (db2 instance name, you can get it with db2ilist utility) constants according your case.
ibm_db.connect('DATABASE=mydb;Instance=DB2;PROTOCOL=IPC;', '', '')

add ibm_db library and import ibm_db on top in the .py file.
def get_db_connection():
"""
This will help to get db2 connection for query execution
:return: conn
"""
dsn_driver = "{IBM DB2 ODBC DRIVER}"
dsn_database = "BLUDB"
dsn_hostname = "your_hostname"
dsn_port = "50000"
dsn_protocol = "TCPIP"
dsn_uid = "your_userid"
dsn_pwd = "your_pwd"
dsn = (
"DRIVER={0};"
"DATABASE={1};"
"HOSTNAME={2};"
"PORT={3};"
"PROTOCOL={4};"
"UID={5};"
"PWD={6};").format(dsn_driver, dsn_database, dsn_hostname, dsn_port, dsn_protocol, dsn_uid, dsn_pwd)
try:
conn = ibm_db.connect(dsn, "", "")
print("Connected!")
return conn
except Exception:
print("\nERROR: Unable to connect to the \'" + dsn_database + "\' server.")
print("error: ", ibm_db.conn_errormsg())
exit(-1)
function get_db_connection() will return the connection object. On that connection object you can perform operation like:
conn = get_db_connection()
list_results = []
select_query = 'SELECT a.STATUS, a.ID FROM "TABLE_NAME" AS a'
print(select_query)
selectStmt = ibm_db.exec_immediate(conn, select_query)
while ibm_db.fetch_row(selectStmt) != False:
list_results.append(ibm_db.result(selectStmt, 'ID'))
ibm_db.close(conn)

How to connect to a cluster in Amazon Redshift using SQLAlchemy?

In Amazon Redshift's Getting Started Guide, it's mentioned that you can utilize SQL client tools that are compatible with PostgreSQL to connect to your Amazon Redshift Cluster.
In the tutorial, they utilize SQL Workbench/J client, but I'd like to utilize python (in particular SQLAlchemy). I've found a related question, but the issue is that it does not go into the detail or the python script that connects to the Redshift Cluster.
I've been able to connect to the cluster via SQL Workbench/J, since I have the JDBC URL, as well as my username and password, but I'm not sure how to connect with SQLAlchemy.
Based on this documentation, I've tried the following:
from sqlalchemy import create_engine
engine = create_engine('jdbc:redshift://shippy.cx6x1vnxlk55.us-west-2.redshift.amazonaws.com:5439/shippy')
ERROR:
Could not parse rfc1738 URL from string 'jdbc:redshift://shippy.cx6x1vnxlk55.us-west-2.redshift.amazonaws.com:5439/shippy'

I don't think SQL Alchemy "natively" knows about Redshift. You need to change the JDBC "URL" string to use postgres.
jdbc:postgres://shippy.cx6x1vnxlk55.us-west-2.redshift.amazonaws.com:5439/shippy
Alternatively, you may want to try using sqlalchemy-redshift using the instructions they provide.

I was running into the exact same issue, and then I remembered to include my Redshift credentials:
eng = create_engine('postgresql://[LOGIN]:[PASSWORD]#shippy.cx6x1vnxlk55.us-west-2.redshift.amazonaws.com:5439/shippy')

sqlalchemy-redshift is works for me, but after few days of reserch
packages (python3.4):
SQLAlchemy==1.0.14 sqlalchemy-redshift==0.5.0 psycopg2==2.6.2
First of all, I checked, that my query is working workbench (http://www.sql-workbench.net), then I force it work in sqlalchemy (this https://stackoverflow.com/a/33438115/2837890 helps to know that auto_commit or session.commit() must be):
db_credentials = (
'redshift+psycopg2://{p[redshift_user]}:{p[redshift_password]}#{p[redshift_host]}:{p[redshift_port]}/{p[redshift_database]}'
.format(p=config['Amazon_Redshift_parameters']))
engine = create_engine(db_credentials, connect_args={'sslmode': 'prefer'})
connection = engine.connect()
result = connection.execute(text(
"COPY assets FROM 's3://xx/xx/hello.csv' WITH CREDENTIALS "
"'aws_access_key_id=xxx_id;aws_secret_access_key=xxx'"
" FORMAT csv DELIMITER ',' IGNOREHEADER 1 ENCODING UTF8;").execution_options(autocommit=True))
result = connection.execute("select * from assets;")
print(result, type(result))
print(result.rowcount)
connection.close()
And after that, I forced to work sqlalchemy_redshift CopyCommand perhaps bad way, looks little tricky:
import sqlalchemy as sa
tbl2 = sa.Table(TableAssets, sa.MetaData())
copy = dialect_rs.CopyCommand(
assets,
data_location='s3://xx/xx/hello.csv',
access_key_id=access_key_id,
secret_access_key=secret_access_key,
truncate_columns=True,
delimiter=',',
format='CSV',
ignore_header=1,
# empty_as_null=True,
# blanks_as_null=True,
)
print(str(copy.compile(dialect=RedshiftDialect(), compile_kwargs={'literal_binds': True})))
print(dir(copy))
connection = engine.connect()
connection.execute(copy.execution_options(autocommit=True))
connection.close()
We make just that I made with sqlalchemy, excute query, except comine query by CopyCommand. I have not see some profit :(.

The following works for me with Databricks on all kinds of SQLs
import sqlalchemy as SA
import psycopg2
host = 'your_host_url'
username = 'your_user'
password = 'your_passw'
port = 5439
url = "{d}+{driver}://{u}:{p}#{h}:{port}/{db}".\
format(d="redshift",
driver='psycopg2',
u=username,
p=password,
h=host,
port=port,
db=db)
engine = SA.create_engine(url)
cnn = engine.connect()
strSQL = "your_SQL ..."
try:
cnn.execute(strSQL)
except:
raise

import sqlalchemy as db
engine = db.create_engine('postgres://username:password#url:5439/db_name')
This worked for me

pyodbc can't connect to database

I'm using pyodbc library from here and I'm connecting this way:
conn = pyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};Server=(localdb)\MSSQLLocalDB;Integrated Security=true; database = online_banking; autocommit = True')
I use MSSQLLocalDBbecause it's the default instance name for SQL Server 2014. And this last version of Python 2.7.
However I cant run any simple query, every if them raise the error, saying that there is no such object or in that particular case database:
cursor.execute('use online_banking;')
The full error:
pyodbc.Error: ('08004', "[08004] [Microsoft][SQL Server Native Client 11.0][SQL Server]Database 'online_banking' does not exist. Make sure that the name is entered correctly. (911) (SQLExecDirectW)")
So what is wrong here?
There is only 1 instance installed and such databases(.mdf)
As you can see only 1 engine:
Selecting that engine will allow me to see online_banking db
upd1 Database've been created this way:
CREATE DATABASE [online_banking]
ON PRIMARY
( NAME = N'online_banking', FILENAME = N'C:\...\online_banking.mdf' ,
SIZE = 512000KB , MAXSIZE = UNLIMITED, FILEGROWTH = 30%)
LOG ON
( NAME = N'online_banking_log', FILENAME = N'C:\...\online_banking_log.ldf' ,
SIZE = 1024KB , MAXSIZE = 20GB , FILEGROWTH = 10%)
GO
upd2 I've used built-in tool sqlcmd.
So this sqlcmd -S (LocalDB)\MSSQLLocalDB -i C:\Users\1.sql -E have shown, that
MSSQLLocalDB doesn't have my database.
However sqlcmd -S localhost -i C:\Users\1.sql -E performed successfully.
I'm totally confused, I' ve installed only one server, moreover SQL Management studio sees only one local server with my online_banking DB. This is look really weird to me.
Trying to use this connection string in Python
conn = pyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};Server=localhost;Integrated Security=true; database = online_banking; autocommit = True')
causes the error below:
pyodbc.Error: ('28000', '[28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]\x... "". (18456) (SQLDriverConnect); [01S00] [Microsoft][SQL Server Native Client 11.0]\xcd\xe5\xe....xe8\xff (0); [28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]\xce...ff "". (18456); [01S00] [Microsoft][SQL Server Native Client 11.0]\xcd\xe.... (0)'
upd3: Specified mdf should be attached, got it:
Tried several ways, always errors (with database specified or not in connection string):
conn = pyodbc.connect(
r'Driver={SQL Server Native Client 11.0};Server=(localdb)\MSSQLLocalDB; database =online_banking; AttachDbFilename=C:\Program Files\Microsoft SQL Server\MSSQL12.SQLSERVERINSAF\MSSQL\DATA\online_banking.mdf;Trusted_Connection=Yes; Integrated Security=true; database = online_banking;')
error: A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
I found out, that may be related with parent server which already have attached this db, but failed to solve this.
upd4
I tried simple code from here to see if "online_banking" shows up in the list of databases for that instance. But faced another error:
pyodbc.Error: ('08001', '[08001] [Microsoft][SQL Server Native Client 11.0]\ - unreadable error
In addition that database according to SSMS seems have already attached by online_banking DB

As it turns out, the database in question was already attached to the default instance of SQL Server on the local machine, so all that was needed to connect was
import pyodbc
conn_str = (
r"Driver={SQL Server Native Client 11.0};"
r"Server=(local);"
r"Database=online_banking;"
r"Trusted_Connection=yes;"
)
conn = pyodbc.connect(conn_str)
There were two main points of confusion:
Q: What is the name of a SQL Server "default instance"?
A: It doesn't have one.
When referring to a SQL Server instance by name, a default instance simply goes by the name of the machine, while a named instance is identified by MachineName\InstanceName. So, on a server named PANORAMA
If we install a "default instance" of SQL Server we refer to it as PANORAMA.
If we install a "named instance" called "SQLEXPRESS" we refer to it as PANORAMA\SQLEXPRESS.
If we are referring to a SQL server instance on the local machine we can use (local) instead of PANORAMA.
Q: Do (local) and (localdb) mean the same thing?
A: NO.
(local) and (local)\InstanceName refer to "real" server-based instances of SQL Server. These are the instances that have been around since SQL Server was first released. They run as a service and are able to accept network connections and do all of the the things we expect a database server to do.
(localdb) and (localdb)\InstanceName references – with (localdb) usually capitalized as (LocalDB) for clarity – are used to connect to "SQL Server LocalDB" instances. These are temporary local SQL Server instances primarily intended for developers. For details see the following MSDN blog post:
SQL Express v LocalDB v SQL Compact Edition

It could possibly be a security issue. You are using integrated security so it will use the security credentials of the windows login that the client program is running. If that user or a group that the user belongs to does not have at least public access to the database, it will appear as if the database does not exist. Either ensure that the user or a group that the user is a member of is set up with a login and that it has at least public access to your database, or use SQL server authentication and send a username and password in your connection string.

How do I connect to SQL Server via sqlalchemy using Windows Authentication?

sqlalchemy, a db connection module for Python, uses SQL Authentication (database-defined user accounts) by default. If you want to use your Windows (domain or local) credentials to authenticate to the SQL Server, the connection string must be changed.
By default, as defined by sqlalchemy, the connection string to connect to the SQL Server is as follows:
sqlalchemy.create_engine('mssql://*username*:*password*#*server_name*/*database_name*')
This, if used using your Windows credentials, would throw an error similar to this:
sqlalchemy.exc.DBAPIError: (Error) ('28000', "[28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for us
er '***S\\username'. (18456) (SQLDriverConnect); [28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for us
er '***S\\username'. (18456)") None None
In this error message, the code 18456 identifies the error message thrown by the SQL Server itself. This error signifies that the credentials are incorrect.

In order to use Windows Authentication with sqlalchemy and mssql, the following connection string is required:
ODBC Driver:
engine = sqlalchemy.create_engine('mssql://*server_name*/*database_name*?trusted_connection=yes')
SQL Express Instance:
engine = sqlalchemy.create_engine('mssql://*server_name*\\SQLEXPRESS/*database_name*?trusted_connection=yes')

If you're using a trusted connection/AD and not using username/password, or otherwise see the following:
SAWarning: No driver name specified; this is expected by PyODBC when using >DSN-less connections
"No driver name specified; "
Then this method should work:
from sqlalchemy import create_engine
server = <your_server_name>
database = <your_database_name>
engine = create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver=ODBC+Driver+13+for+SQL+Server')

A more recent response if you want to connect to the MSSQL DB from a different user than the one you're logged with on Windows. It works as well if you are connecting from a Linux machine with FreeTDS installed.
The following worked for me from both Windows 10 and Ubuntu 18.04 using Python 3.6 & 3.7:
import getpass
from sqlalchemy import create_engine
password = getpass.getpass()
eng_str = fr'mssql+pymssql://{domain}\{username}:{password}#{hostip}/{db}'
engine = create_engine(eng_str)
What changed was to add the Windows domain before \username.
You'll need to install the pymssql package.

Create Your SqlAlchemy Connection URL      From Your pyodbc Connection String      OR Your Known Connection Parameters
I found all the other answers to be educational, and I found the SqlAlchemy Docs on connection strings helpful too, but I kept failing to connect to MS SQL Server Express 19 where I was using no username or password and trusted_connection='yes' (just doing development at this point).
Then I found THIS method in the SqlAlchemy Docs on Connection URLs built from a pyodbc connection string (or just a connection string), which is also built from known connection parameters (i.e. this can simply be thought of as a connection string that is not necessarily used in pyodbc). Since I knew my pyodbc connection string was working, this seemed like it would work for me, and it did!
This method takes the guesswork out of creating the correct format for what you feed to the SqlAlchemy create_engine method. If you know your connection parameters, you put those into a simple string per the documentation exemplified by the code below, and the create method in the URL class of the sqlalchemy.engine module does the correct formatting for you.
The example code below runs as is and assumes a database named master and an existing table named table_one with the schema shown below. Also, I am using pandas to import my table data. Otherwise, we'd want to use a context manager to manage connecting to the database and then closing the connection like HERE in the SqlAlchemy docs.
import pandas as pd
import sqlalchemy
from sqlalchemy.engine import URL
# table_one dictionary:
table_one = {'name': 'table_one',
'columns': ['ident int IDENTITY(1,1) PRIMARY KEY',
'value_1 int NOT NULL',
'value_2 int NOT NULL']}
# pyodbc stuff for MS SQL Server Express
driver='{SQL Server}'
server='localhost\SQLEXPRESS'
database='master'
trusted_connection='yes'
# pyodbc connection string
connection_string = f'DRIVER={driver};SERVER={server};'
connection_string += f'DATABASE={database};'
connection_string += f'TRUSTED_CONNECTION={trusted_connection}'
# create sqlalchemy engine connection URL
connection_url = URL.create(
"mssql+pyodbc", query={"odbc_connect": connection_string})
""" more code not shown that uses pyodbc without sqlalchemy """
engine = sqlalchemy.create_engine(connection_url)
d = {'value_1': [1, 2], 'value_2': [3, 4]}
df = pd.DataFrame(data=d)
df.to_sql('table_one', engine, if_exists="append", index=False)
Update
Let's say you've installed SQL Server Express on your linux machine. You can use the following commands to make sure you're using the correct strings for the following:
For the driver: odbcinst -q -d
For the server: sqlcmd -S localhost -U <username> -P <password> -Q 'select ##SERVERNAME'

pyodbc
I think that you need to put:
"+pyodbc" after mssql
try this:
from sqlalchemy import create_engine
engine = create_engine("mssql+pyodbc://user:password#host:port/databasename?driver=ODBC+Driver+17+for+SQL+Server")
cnxn = engine.connect()
It works for me
Luck!

If you are attempting to connect:
DNS-less
Windows Authentication for a server not locally hosted.
Without using ODBC connections.
Try the following:
import sqlalchemy
engine = sqlalchemy.create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver=SQL+Server')
This avoids using ODBC connections and thus avoids pyobdc interface errors from DPAPI2 vs DBAPI3 conflicts.
I would recommend using the URL creation tool instead of creating the url from scratch.
connection_url = sqlalchemy.engine.URL.create("mssql+pyodbc",database=databasename, host=servername, query = {'driver':'SQL Server'})
engine = sqlalchemy.create_engine(connection_url)
See this link for creating a connection string with SQL Server Authentication (non-domain, uses username and password)