I'm working on an app that needs to connect to an ibm db2 database. Using DBeaver I can successfully connect to the database (I provide him the db2cc.jar and db2cc4.jar files).
It looks to me as DBeaver is using my Window's credentials to login, because I didn't need to input any login or password to connect.
Now, I've been trying to connect to the same database using python 3.7 and pypi's latest version of the ibm_db package. I didn't install anything else.
import ibm_db
# ...
connection_string = "DATABASE=" + self.params['schema'] + ";" + \
"HOSTNAME=" + self.params['host'] + ";" + \
"PORT=" + self.params['port'] + ";" + \
"PROTOCOL=TCPIP;" + \
"SECURITYMECHANISM=4;" + \
"UID=" + self.params['user'] + ";" + \
"PWD=" + self.params['password'] + ";"
try:
self.connection = ibm_db.connect(connection_string, "", "")
# ...
Using my Windows credentials in the parameters, I get the following error message:
Connection error
Bad credentials
SQLCODE=-30082
08001
From what I've seen on stack overflow connecting to a db2 database is complicated...
Does someone know how to connect ? Using the windows credentials or otherwise...
Thanks !
Db2 works fine with Python.
You need to be aware of some basics before you start such as:
what operating-system runs the target Db2-database and
what kind of client is being used (java, odbc/cli, .net etc), and
what kind of authentication/encrpytion is in place for the database
(ssl, server based authentication/+/-/encryption etc.).
is the remote database rdbms Apache DERBY or Db2.
Find out these basics before you start. You have to speak with people who run the Db2-server.
Note: in your question you mention (SecurityMechanism=4) com.ibm.db2.jcc.DB2BaseDataSource.USER_ONLY_SECURITY - this is not relevant for non-JAVA clients, it is relevant is the database manager is DERBY .
For python, the ibm_db package is not a java application.
DBeaver is a java application (hence it uses db2jcc.jar or db2jcc4.jar and a licence-file to connect to the remote database).
You can only use your Windows credentials for connecting to a Db2-database when that Db2-database run on Microsoft-Windows, and the credentials work on the hostname running the Db2-server. For any other combinations, the administrator must issue you a userid/password that is relevant for the target hostname.
The ibm_db package needs a Db2-client to be installed. The Db2-client is a separate installable. There are different kinds of Db2-client depending both on which operating-system runs your Db2-server and how much functionality you need to have in your Db2-client. If your remote Db2-server runs on Linux, Unix, Windows or Z/OS then you can use the "IBM Data Server Runtime Client" which you can either download from IBM's passport advantage website, or get from your internal IT folks. If your Db2-server runs on i-Series (AS/400) you should get its drivers from your i-Series administrator. For either Z/OS or i-Series you will additionally need a license file (which costs money) and you should get that from your administrator, unless your company uses a gateway product called Db2-connect in which case you don't need a separate license file on your workstation.
Try the following connection string, If your db2 client and server are on the same host.
Change the 'mydb' and 'DB2' (db2 instance name, you can get it with db2ilist utility) constants according your case.
ibm_db.connect('DATABASE=mydb;Instance=DB2;PROTOCOL=IPC;', '', '')
add ibm_db library and import ibm_db on top in the .py file.
def get_db_connection():
"""
This will help to get db2 connection for query execution
:return: conn
"""
dsn_driver = "{IBM DB2 ODBC DRIVER}"
dsn_database = "BLUDB"
dsn_hostname = "your_hostname"
dsn_port = "50000"
dsn_protocol = "TCPIP"
dsn_uid = "your_userid"
dsn_pwd = "your_pwd"
dsn = (
"DRIVER={0};"
"DATABASE={1};"
"HOSTNAME={2};"
"PORT={3};"
"PROTOCOL={4};"
"UID={5};"
"PWD={6};").format(dsn_driver, dsn_database, dsn_hostname, dsn_port, dsn_protocol, dsn_uid, dsn_pwd)
try:
conn = ibm_db.connect(dsn, "", "")
print("Connected!")
return conn
except Exception:
print("\nERROR: Unable to connect to the \'" + dsn_database + "\' server.")
print("error: ", ibm_db.conn_errormsg())
exit(-1)
function get_db_connection() will return the connection object. On that connection object you can perform operation like:
conn = get_db_connection()
list_results = []
select_query = 'SELECT a.STATUS, a.ID FROM "TABLE_NAME" AS a'
print(select_query)
selectStmt = ibm_db.exec_immediate(conn, select_query)
while ibm_db.fetch_row(selectStmt) != False:
list_results.append(ibm_db.result(selectStmt, 'ID'))
ibm_db.close(conn)
Related
we have been using sqlalchemy successfully to connect to Oracle. Just now our organization is moving to encrypted Oracle database and we have been asked to switch to the encrypted database.
The sample code given to me by the database engineering team is which uses cx_Oracle directly is:
import cx_Oracle
dsn = """(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=tcps)(HOST=test_host)(PORT=1531)))
(CONNECT_DATA=(SERVICE_NAME=test_service)))"""
connection = cx_Oracle.connect(user="test", password="test", dsn=dsn, encoding="UTF-8")
However, when I try to connect to the database using sqlalchemy using :
oracle+cx_oracle://test:test#test_host:1531/test_service
I get an error : (cx_Oracle.DatabaseError) ORA-12547: TNS:lost contact\n(Background on this error at: http://sqlalche.me/e/13/4xp6)
I suspect that it is the protocol tcps that needs to be set.
I tried the following connection string :
protocol_url = 'oracle+cx_oracle://test:test#test_host:1531?service_name=test_service&protocol=tcps'
I get the following error :
ValueError: invalid literal for int() with base 10: '1531?service_name=test_service&protocol=tcps'
Is there a way to use sqlalchemy to connect to Oracle on an encrypted port?
EDIT : I went through the steps listed in Python connect to Oracle database with TCPS
I still get the error : sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-12547: TNS:lost contact
NOTE: I know that I can successfully connect to Oracle with encryption using a direct cx_Oracle connection.
Since you're using Python/SQLAlchemy, you can use create_engine. My Python
code constructs the connection string then calls create_engine like this:
dsnStr = cx_Oracle.makedsn(config[args.tier]['oracle_host'], config[args.tier]['oracle_port'],config[args.tier]['oracle_sid'],
config[args.tier]['oracle_service'])
dsnStr = dsnStr.replace('TCP', config[args.tier]['oracle_protocol'])
oracle_conn_str = "oracle://" + config[args.tier]['username'] + ":" + config[args.tier]['password'] + "#" + dsnStr
engine = create_engine(oracle_conn_str, connect_args={ "encoding": "UTF-8", "nencoding": "UTF-8"})
In my code, I read a config file (via configparser) that specified the oracle_protocol to be TCPS.
dsnStr for me was:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCPS)(HOST=REDACTED)(PORT=1923))(CONNECT_DATA=(SID=REDACTED)(SERVICE_NAME=)))
In the end, oracle_conn_str for me was:
oracle://rumali:REDACTED#(DESCRIPTION=(ADDRESS=(PROTOCOL=TCPS)(HOST=REDACTED)(PORT=1923))(CONNECT_DATA=(SID=REDACTED)(SERVICE_NAME=)))
In order to use oracle encryption with sql alchemy the following needs to be done :
a) Follow the instructions to set up your client machine as described in the answer to the question : Python connect to Oracle database with TCPS
b) Then use the DSN as specified in the answer to this question : Using Oracle Service Names with SQLAlchemy
For my part, I needed to do both a) and b). Some of you using sqlalchemy may already have specified the DSN in the needed format.
engine = create_engine("oracle+cx_oracle://:#(DESCRIPTION = (LOAD_BALANCE=on) (FAILOVER=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = )(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = devdb)))")
The main thing here is that the DSN format I was using :
oracle+cx_oracle://:#:/test_db
does not allow the protocol to be specified as TCPS which is also required.
I try to connect to SQL using the pyodbc package, but get the error:
SystemError:
built-in function connect returned NULL without setting an error
What could be the reason for this?
When I run the code from another computer - I do manage to connect to SQL.
The connection string I use:
conn_str = 'DRIVER={SQL Server};SERVER=' + server + ';DATABASE=' + database + ';'
conn = pyodbc.connect(conn_str)
I have ODBC and pyodbc installed on my computer.
Of course I set values for the database and server fields
Before I do connect I use a function that does logon with the username and password
Because of your question do not have a reproductible exemple i can't be 100% of what I will say;
This is probably a driver problem, if you have acces to db2dsdriver.cfg file or db2cli.ini you shoyld try to look in those files to find what is the exact name of the driver.
Following this could help you to have more information on the problem.
I am using python's pyodbc library to connect with the Oracle Database. The authentication to my non-prod servers were set-up as Basic Authentication so my connection string worked well.
It was tough to understand and set up the right drivers initially, But then did manage to get through. Here is what the code looks like for connection and worked well for my other servers.
import textwrap
import pyodbc
connection_string = textwrap.dedent('''Driver={driver};
DBQ={hostname}:{port}/{sid};
UID={username};
PWD={password};
Connection Timeout=30;
Trusted_Connection="yes"
'''.format(driver = 'Oracle in instantclient_11_2',
hostname = <hostname>,
port = <port>,
sid = <sid>,
username = <username>,
password = <passwd>
))
connection = pyodbc.connect(connection_string)
To my surprise PROD is authenticated via LDAP only. To which I have the LDAP Server, Context, and DB Service along with the credentials.
I tried creating my connection string as
connection_string = textwrap.dedent('''Driver={driver};
DBQ={hostname}:{port}/{sid};
UID={username};
PWD={password};
Connection Timeout=30;
Authentication=LDAP;
Trusted_Connection="yes"
'''.format(driver = 'Oracle in instantclient_11_2',
hostname = <ldap_server>,
port = 389,
sid = <db_service>,
username = <username>,
password = <passwd>
))
To which I was not surprised to know it wont work. Tried going through many links but could not get through. Has anyone tried this before or please let me know what I must be missing.
I could definitely connect in SQL Developer using the LDAP Authentication. And since TNS Listener is not configured on this server I cannot use Basic Authentication to connect.
And help would be welcome on this.
Setting up Oracle Drivers - (This is a pre-requisite if exist please ignore)
Download Oracle Instant Client for Microsoft Windows (x64) 64-bit. Alternatively download available for your 32 bit system.
Choose your version of Oracle Database. My version was 11.2.0.4.0
Download below files -
i. Instant Client Package - Basic
ii. Instant Client Package - ODBC
Unzip both the packages into the same directory such as C:\oracle\instantclient_19_3.
Execute odbc_install.exe from the Instant Client directory. If Instant Client is 11g or lower, start the command prompt with the Administrator privilege.
Create C:\oracle\instantclient_19_3\network\admin folder to place tnsnames.ora, sqlnet.ora.
Set Environment variable - TNS_ADMIN to above folder location.
Add C:\oracle\instantclient_19_3 to PATH Environment variable.
Add C:\oracle\instantclient_19_3 to ORACLE_HOME (optional, if connection doesn't work try this.)
LDAP Authentication
Create two files: sqlnet.ora and ldap.ora
ldap.ora
# Place this file in the network/admin subdirectory or your
# $ORACLE_HOME location.
# LDAP Server name should be added here. Rest all the values remains unchanged.
DIRECTORY_SERVERS = (your-server.your-organization:389:636)
DEFAULT_ADMIN_CONTEXT = "ldap-ou-designation"
DIRECTORY_SERVER_TYPE = OID
sqlnet.ora
# Place this file in the network/admin subdirectory or your
# $ORACLE_HOME location.
SQLNET.AUTHENTICATION_SERVICES=(NTS)
NAMES.DIRECTORY_PATH = (LDAP)
Verify C:\oracle\instantclient_19_3 is present in the Path else try updating the path in the Python Code as below
import os
lib_dir=r"C:\oracle\instantclient_19_3"
os.environ["PATH"] = lib_dir + ";" + os.environ["PATH"]
With all this set-up in place you should have
username, password and DB Service name lets call it as db_service here
with cx_Oracle
import cx_Oracle
con = cx_Oracle.connect('{0}/{1}#{2}'.format(user, password, db_service))
version_script = "SELECT * FROM v$version"
cursor = con.cursor()
cursor.execute(version_script)
version = cursor.fetchall()
print(version[0][0])
Ouput
-----
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
with pyodbc
import pyodbc
con = pyodbc.connect('Driver={0};DBQ={1};UID={2};PWD={3}'.format(driver, db_service, user, password))
version_script = "SELECT * FROM v$version"
cursor = con.cursor()
cursor.execute(version_script)
version = cursor.fetchall()
print(version[0][0])
Ouput
-----
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
#Drivers for pyodbc
pyodbc.drivers()
Output
------
['ODBC Driver 17 for SQL Server',
'Oracle in instantclient_11_2']
References :
https://www.oracle.com/database/technologies/instant-client/winx64-64-downloads.html
https://www.oracle.com/database/technologies/releasenote-odbc-ic.html
https://eikonomega.medium.com/connecting-to-oracle-database-with-cx-oracle-and-ldap-5da7925a305c
I'm testing an API using Fast API and Docker. When I load up the webpage it attempts to run a query using an AS400 connection that is setup with pyodbc.
I started with the error: pyodbc.InterfaceError: ('28000', '[28000] [IBM][System i Access ODBC Driver]Communication link failure. comm rc=8051 - CWBSY1011 - Kerberos client credentials not found
So I installed krb5 and created a ticket. Then re-loaded the webpage and the new error I am stuck at is pyodbc.InterfaceError: ('28000', '[28000] [IBM][System i Access ODBC Driver]Communication link failure. comm rc=8052 - CWBSY1012 - Kerberos service principal not found for system SYSTEM where SYSTEM is my AS400 system's name.
Has anyone else run into this before? It seems like AS400 is requiring me to use Kerberos on connecting but then I set it up and the system does not have a Kerberos service principal.
Do I possibly need to join the domain fully using the instructions found here? I started this process however I run into an error: Failed to join domain: Not enough storage is available to process this command. when trying to run net ads join which I am assuming is due to the Docker image being so small; so I assume this is not possible.
Here is my ODBC.ini config file:
[QDSN_ASW]
Description=ASW
Driver=iSeries Access ODBC Driver
System=192.168.100.1
UserID=user
Password=pass
Naming=0
DefaultLibraries=QGPL
Database=1492BFDD
ConnectionType=2
CommitMode=2
ExtendedDynamic=0
DefaultPkgLibrary=QGPL
DefaultPackage=A/DEFAULT(IBM),2,0,1,0,512
AllowDataCompression=1
LibraryView=0
AllowUnsupportedChar=0
ForceTranslation=0
Trace=0
Trusted_Connection=no
AuthenticationType=No Authentication
I was able to solve the issue thanks to this post on stackoverflow.
Basically, it appears my original construction of the database connection (shown below) was too convoluted and unnecessary.
class CommitMode:
NONE = 0 # Commit immediate (*NONE) --> QSQCLIPKGN
CS = 1 # Read committed (*CS) --> QSQCLIPKGS
CHG = 2 # Read uncommitted (*CHG) --> QSQCLIPKGC
ALL = 3 # Repeatable read (*ALL) --> QSQCLIPKGA
RR = 4 # Serializable (*RR) --> QSQCLIPKGL
class ConnectionType:
ReadWrite = 0 # Read/Write (all SQL statements allowed)
ReadCall = 1 # Read/Call (SELECT and CALL statements allowed)
Readonly = 2 # Read-only (SELECT statements only)
def connstr(system, commitmode=None, connectiontype=ConnectionType.Readonly):
_connstr = 'DRIVER=iSeries Access ODBC Driver;'+ \
'SYSTEM='+system+';'+\
'SIGNON=4;CCSID=1208;TRANSLATE=1;'
if commitmode is not None:
_connstr = _connstr + 'CommitMode=' + str(commitmode) + ';'
if connectiontype is not None:
_connstr = _connstr +'ConnectionType=' + str(connectiontype) + ';'
return _connstr
and I was able to simplify it to:
system = f"SYSTEM=AS400;DRIVER=iSeries Access ODBC Driver;SERVER=192.168.100.0;PORT=446;DATABASE=AS400;UID={user};PWD={pw}"
Now when I build the Docker image and load the web page, it instantly queries the database with no Kerberos authentication required.
Using python 2.7 and the MS odbc driver through pyodbc. My connection string looks like this:
mssql+pyodbc://myuser:mypass#serverip/instancename?driver=ODBC+Driver+11+for+SQL+Server
I am getting login failed. However if I use the same credentials and "serverip\instancename" in the microsoft sql server management studio, I can connect.
The thing that is driving me crazy is a couple of days ago, this same connection string worked for me but to a different sql server instance on the same machine.
So what I am trying to figure out is how to go about troubleshooting it.
Thanks for any pointers.
When learning sqlalchemy I had the same issue. This method worked for me:
import urllib
params = urllib.quote_plus("DRIVER={SQL Server Native Client 10.0};SERVER=dagger;DATABASE=test;UID=user;PWD=password")
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
Which comes from the sqlalchemy documentation. It does not seem related to the login credentials, but the error message was the same for me and this worked.
ref http://docs.sqlalchemy.org/en/latest/dialects/mssql.html#pass-through-exact-pyodbc-string
I just went through this, the reason it fails is the port, each instance listens in a different port, so you need to point which port is for that instance.
my code is:
DBserver='host.abc.nt'
DBengine='devenv'
DBport='####'
DBname='mydb'
DBDriver='ODBC Driver 13 for SQL Server'
DBuser='user'
DBpwd='pass'
DBuserpass = DBuser if len(DBuser) > 0 else ''
DBuserpass = DBuserpass + ':' + DBpwd if len(DBuser) > 0 else ''
if len(DBengine) > 0:
DBserver = DBserver + '\\' + DBengine
engine = sa.create_engine(f'''mssql+pyodbc://{DBuserpass}#{DBserver}:{DBport}/{DBname}?driver={DBDriver}''')
query=engine.execute('SELECT DB_NAME(), ##SERVERNAME')
query.fetchall()
[('mydb', 'host\\devenv')]
With that you create the engine, there is no need for other libraries besides sqlalchemy
but if you want to know the port of the instance, you can use:
Github repository from gordthompson/sqlserverport
or in Windows you connect with SQL Studio (or anything else) do some selects the the DB and in the meanwhile in CMD run a netstat to get the port:
C:\Users\pla>netstat -qfa | findstr host
TCP 10.0.0.1:##87 host.abc.nt:4096 ESTABLISHED