Develop Reference

Login to Azure SQL with JWT token in python - [SQL Server]Login failed for user '<token-identified principal>

I try to connect to Azure SQL using SQLAlchemy in python, authenticating with JWT generated for App registration. Despite checking almost any combination of parameters and granting all potentially meaningful permissions, I still get following error:
[28000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Login failed for user ''. (18456)
I did the following:
Created App registration 'test-app'.
Executed on the db:
CREATE USER [test-app] FROM EXTERNAL PROVIDER;
EXEC sp_addrolemember N'db_datareader', N'test-app';
GRANT SELECT ON test_vw to [test-app];
Added 'test-app' to Contributor and Reader roles for the SQL Server.
Whitelisted IP in the portal - I'm able to log in through SSMS and 'AAD Universal with MFA' using personal account.
Generated token with (tenant ID matches the one used by the organization):
POST https://login.microsoftonline.com/9ff8761-1be4-4729-b88df-e421e19d92f/oauth2/v2.0/token HTTP/1.1
Content-Type: application/x-www-form-urlencoded
client_id=[client_id]
client_secret=[client_secret]
scope=https://database.windows.net/.default
grant_type=client_credentials
Used generated JWT token in following python code:
from sqlalchemy import create_engine
import urllib
import struct
server = "test-server.database.windows.net"
database = "test-db"
driver = "{ODBC Driver 17 for SQL Server}"
jwt_token = "<jwt_token>"
SQL_COPT_SS_ACCESS_TOKEN = 1256
exptoken = b""
for i in bytes(jwt_token, "UTF-8"):
exptoken += bytes({i})
exptoken += bytes(1)
tokenstruct = struct.pack("=i", len(exptoken)) + exptoken
params = urllib.parse.quote_plus(
f"Driver={driver};Server={server};Database={database};Encrypt=yes"
)
conn_str = f"mssql+pyodbc:///?odbc_connect={params}"
engine_azure = create_engine(
conn_str,
connect_args={"attrs_before": {SQL_COPT_SS_ACCESS_TOKEN: tokenstruct}},
echo=True,
)
with engine_azure.connect() as con:
rs = con.execute("SELECT TOP 100 * FROM test_vw")
for row in rs:
print(row)
Any idea what I've missed?

I registered an app and copied ClientID, TenantId, generated client secret and copied ClientSecret value.I created azure SQL database and copied servername, databasename in azure portal.
I tried to connect azure SQL database by using token by replacing ClientID, TenantId, ClientSecret:
I installed SQLALchemy and adal packages using
pip install sqlalchemy
pip install adal
Connecting to Azure SQL database using below code:
from sqlalchemy import create_engine
import adal
server = '<servername>'
database = '<databasename>'
client_id = '<ClientId>'
tenant_id = '<TenantId>'
authority_host_uri = 'https://login.microsoftonline.com'
authority_uri = authority_host_uri + '/' + tenant_id
resource_uri = 'https://database.windows.net/'
client_secret = 'NbH8Q~x8PsGqD4C~8HUGzLmql_iRS7dxOMv2Bcjj'
context = adal.AuthenticationContext(authority_uri)
token = context.acquire_token_with_client_credentials(resource_uri, client_id, client_secret)
conn_str = (
'Driver={ODBC Driver 17 for SQL Server};'
f'Server={server};'
f'Database={database};'
'Encrypt=yes;'
'TrustServerCertificate=yes;'
f'AccessToken={token["accessToken"]};'
)
engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(conn_str))
with engine.connect() as conn:
result = conn.execute('SELECT * FROM your-table')
for row in result:
print(row)
I got below error:
I created new group and added to registered app by following below procedure:
Goto-->Azure Active directory-->Enterprise Applications
Select the app and go to--> Group/Users-->Add User/Group select the user/group and click on assign.
Set that user/group as admin of SQL server by clicking Set admin:
I tried to connect SQL database again:
It worked successfully for me kindly check from your end.

Connect to SQL Server from Python

What I have in SQLite at the moment in the JSON configuration file:
"db": "sqlite:///\\\\F\\Evaluations\\data\\abc.db"
What I want is to connect to a database in SQL Server. How I do that?
db is used in the pipeline numerous times in different python files as follows:
...
self.db = settings['db']
...
engine = sa.create_engine( self.db, echo=False)
conn = engine.connect()

First, try to install pyodbc with the following command:
pip install pyodbc
Then, you can simply add the MS SQL URI connection to the JSON config file as follows:
"db_mssql":"mssql+pyodbc://username:password#host:port/database"
Then, in your python script:
self.db = settings['db_mssql']
engine = sa.create_engine( self.db, echo=False)
conn = engine.connect()

It looks like you are connecting through sqlalchemy. Your connection string for SQL server will need to include more information such as:
host name/server name
database name
credentials (or specify a trusted connection)
driver name
It will be good to test in a different script first.
import sqlalchemy as db
import pyodbc # you may need to install pyodbc (or use a different supported driver)
server_name = 'localhost'
database_name = 'AdventureWorks2019'
# directly specify your driver
driver = 'ODBC Driver 17 for SQL Server'
# or use pyodbc to find it
# driver = [x for x in pyodbc.drivers() if x.endswith(' for SQL Server')]
# driver = driver[0]
# specifying a username/password
engine = db.create_engine('mssql+pyodbc://USERNAME:PASSWORD#'+server_name+'/'+database_name+'?driver='+driver)
# or using a trusted connection
# sqlalchemy will add "trusted_connection=yes" if username/password isn't specified
engine = db.create_engine('mssql+pyodbc://#'+server_name+'/'+database_name+'?driver='+driver)
Then your JSON configuration will end up being something like:
"db": "mssql+pyodbc://USERNAME:PASSWORD#localhost/AdventureWorks2019?driver=ODBC Driver 17 for SQL Server"
OR
"db": "mssql+pyodbc://#localhost/AdventureWorks2019?driver=ODBC Driver 17 for SQL Server"
Side note, there is an extra parameter to speed up inserts into SQL server, if you find yourself needing it.
db.create_engine('mssql+pyodbc://USERNAME:PASSWORD#'+server_name+'/'+database_name+'?driver='+driver, fast_executemany=True)
Reference Links:
https://docs.sqlalchemy.org/en/14/core/engines.html
https://docs.sqlalchemy.org/en/14/dialects/mssql.html

Teradata and sqlachemy connection

I wish to use sqlachemy with teradata dialect to push some csv into a table.
So far I wrote this :
import pandas as pd
from sqlalchemy import create_engine
user = '******'
pasw = '******'
host = 'FTGPRDTD'
DATABASE = 'DB_FTG_SRS_DATALAB'
# connect
td_engine = create_engine('teradata://'+ user +':' + pasw + '#'+ DBCNAME + ':1025/')
print ('ok step one')
print(td_engine)
# execute sql
df = pd.read_csv(r'C:/Users/c92434/Desktop/Load.csv')
print('df chargé')
df.to_sql(name= 'mdc_load', con = td_engine, index=False, schema = DATABASE,
if_exists='replace')
print ('ok step two')
This is the error message I get :
DatabaseError: (teradata.api.DatabaseError) (0, '[08001] [TPT][ODBC SQL Server Wire Protocol driver]Invalid Connection Data., [TPT][ODBC SQL Server Wire Protocol driver]Invalid attribute in connection string: DBCNAME.')
(Background on this error at: http://sqlalche.me/e/4xp6)
What I can I do ?

Hopefully you've solved this by now, but I had success with this. Looking at what you provided, it looks like the host information you set is not being used in the connection string. My example includes the dtype parameter, which I use to define the data type for each column so they don't show up as CLOB.
database = "database_name"
table = "mdc_load"
user = "user"
password = "password"
host = 'FTGPRDTD:1025'
td_engine = create_engine(f'teradata://{user}:{password}#{host}/?database={database}&driver=Teradata&authentication=LDAP')
conn = td_engine.connect()
data.to_sql(name=table, con=conn, index=False, if_exists='replace', dtype=destType)
conn.close()

The "teradata" dialect (sqlalchemy-teradata module) relies on a Teradata ODBC driver being separately installed on the client platform. If you have multiple ODBC drivers installed that include the word Teradata in the name (for example, because you installed TPT with the Teradata-branded drivers for other database platforms), you may need to explicitly specify the one to be used by appending an optional parameter to your connection string, e.g.
td_engine = create_engine('teradata://'+ user +':' + pasw + '#'+ DBCNAME + ':1025/?driver=Teradata Database ODBC Driver 16.20')
Alternatively, you could use the "teradatasql" dialect (teradatasqlalchemy module) which does not require ODBC.

How do I use pyodbc.connect() with authorization=ActiveDirectoryIntegrated?

Trying to access Azure SQL through a python function in VS code, with Authentication set to Active Directory Integrated. Using pyodbc to connect.
Works fine when run locally but get an error after deploying to Azure. Also works fine if I use SQL login but I want to use Active Directory Integrated. I have already set myself as an AD admin.
What I am trying:
cnxn = pyodbc.connect("Driver={ODBC Driver 17 for SQL Server};Server=tcp:khawajaserver1.database.windows.net,1433;Database=KhawajaDB1;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryIntegrated")
Error I get:
Result: Failure
Exception: Error: ('HY000', '[HY000] [Microsoft][ODBC Driver 17 for SQL Server]MAX_PROVS: Error code 0x57 (87) (SQLDriverConnect)')
Stack: File "/usr/local/lib/python3.6/site-packages/azure/functions_worker/dispatcher.py", line 308, in _handle__invocation_request

I know Active Directory Password as authentication type works.
db_list = [TEST_DB1, TEST_DB2]
sql_conn = None
for db in db_list:
try:
conn_string = 'DRIVER={ODBC Driver 17 for SQL Server};' \
'SERVER=' + <db_url> + \
';DATABASE=' + <db_name> + \
';UID=' + <db_username> + \
';PWD=' + <db_password> + \
';Authentication=ActiveDirectoryPassword'
print conn_string
sql_conn = pyodbc.connect(conn_string)
except Exception as e:
print "Exception:::", e
print 'Cannot connect to DB' + str(sys.exc_info()[0])
return None
sql_conn.cursor().execute(<some SQL Query>)
sql_conn.close()

when using the ODBC Driver 17 for SQL Server, the following works when you are using some form of Managed Identity to connect to an Azure SQL Instance;
conn_str = 'Driver={};SERVER=tcp:{},1433;DATABASE=CustomerProfiling;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryMsi;'.format("{ODBC Driver 17 for SQL Server}", os.environ["SQL_SERVER"])
conn = pyodbc.connect(conn_str)
The key is to use the ActiveDirectoryMsi Authentication attribute.

The following code worked for me
# Connection to SQL Server using AADIntegrated
import pyodbc
server = 'data1.database.windows.net'
database = 'MyTestDB'
authentication = 'ActiveDirectoryIntegrated'
kpi_server_connection = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';Authentication='+authentication+';TrustServerCertificate='+ 'no')
query_string = '''
select top 10 * from [SomeTable]
'''
df = pd.read_sql(query_string, kpi_server_connection)
df

How do I connect to SQL Server via sqlalchemy using Windows Authentication?

sqlalchemy, a db connection module for Python, uses SQL Authentication (database-defined user accounts) by default. If you want to use your Windows (domain or local) credentials to authenticate to the SQL Server, the connection string must be changed.
By default, as defined by sqlalchemy, the connection string to connect to the SQL Server is as follows:
sqlalchemy.create_engine('mssql://*username*:*password*#*server_name*/*database_name*')
This, if used using your Windows credentials, would throw an error similar to this:
sqlalchemy.exc.DBAPIError: (Error) ('28000', "[28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for us
er '***S\\username'. (18456) (SQLDriverConnect); [28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for us
er '***S\\username'. (18456)") None None
In this error message, the code 18456 identifies the error message thrown by the SQL Server itself. This error signifies that the credentials are incorrect.

In order to use Windows Authentication with sqlalchemy and mssql, the following connection string is required:
ODBC Driver:
engine = sqlalchemy.create_engine('mssql://*server_name*/*database_name*?trusted_connection=yes')
SQL Express Instance:
engine = sqlalchemy.create_engine('mssql://*server_name*\\SQLEXPRESS/*database_name*?trusted_connection=yes')

If you're using a trusted connection/AD and not using username/password, or otherwise see the following:
SAWarning: No driver name specified; this is expected by PyODBC when using >DSN-less connections
"No driver name specified; "
Then this method should work:
from sqlalchemy import create_engine
server = <your_server_name>
database = <your_database_name>
engine = create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver=ODBC+Driver+13+for+SQL+Server')

A more recent response if you want to connect to the MSSQL DB from a different user than the one you're logged with on Windows. It works as well if you are connecting from a Linux machine with FreeTDS installed.
The following worked for me from both Windows 10 and Ubuntu 18.04 using Python 3.6 & 3.7:
import getpass
from sqlalchemy import create_engine
password = getpass.getpass()
eng_str = fr'mssql+pymssql://{domain}\{username}:{password}#{hostip}/{db}'
engine = create_engine(eng_str)
What changed was to add the Windows domain before \username.
You'll need to install the pymssql package.

Create Your SqlAlchemy Connection URL      From Your pyodbc Connection String      OR Your Known Connection Parameters
I found all the other answers to be educational, and I found the SqlAlchemy Docs on connection strings helpful too, but I kept failing to connect to MS SQL Server Express 19 where I was using no username or password and trusted_connection='yes' (just doing development at this point).
Then I found THIS method in the SqlAlchemy Docs on Connection URLs built from a pyodbc connection string (or just a connection string), which is also built from known connection parameters (i.e. this can simply be thought of as a connection string that is not necessarily used in pyodbc). Since I knew my pyodbc connection string was working, this seemed like it would work for me, and it did!
This method takes the guesswork out of creating the correct format for what you feed to the SqlAlchemy create_engine method. If you know your connection parameters, you put those into a simple string per the documentation exemplified by the code below, and the create method in the URL class of the sqlalchemy.engine module does the correct formatting for you.
The example code below runs as is and assumes a database named master and an existing table named table_one with the schema shown below. Also, I am using pandas to import my table data. Otherwise, we'd want to use a context manager to manage connecting to the database and then closing the connection like HERE in the SqlAlchemy docs.
import pandas as pd
import sqlalchemy
from sqlalchemy.engine import URL
# table_one dictionary:
table_one = {'name': 'table_one',
'columns': ['ident int IDENTITY(1,1) PRIMARY KEY',
'value_1 int NOT NULL',
'value_2 int NOT NULL']}
# pyodbc stuff for MS SQL Server Express
driver='{SQL Server}'
server='localhost\SQLEXPRESS'
database='master'
trusted_connection='yes'
# pyodbc connection string
connection_string = f'DRIVER={driver};SERVER={server};'
connection_string += f'DATABASE={database};'
connection_string += f'TRUSTED_CONNECTION={trusted_connection}'
# create sqlalchemy engine connection URL
connection_url = URL.create(
"mssql+pyodbc", query={"odbc_connect": connection_string})
""" more code not shown that uses pyodbc without sqlalchemy """
engine = sqlalchemy.create_engine(connection_url)
d = {'value_1': [1, 2], 'value_2': [3, 4]}
df = pd.DataFrame(data=d)
df.to_sql('table_one', engine, if_exists="append", index=False)
Update
Let's say you've installed SQL Server Express on your linux machine. You can use the following commands to make sure you're using the correct strings for the following:
For the driver: odbcinst -q -d
For the server: sqlcmd -S localhost -U <username> -P <password> -Q 'select ##SERVERNAME'

pyodbc
I think that you need to put:
"+pyodbc" after mssql
try this:
from sqlalchemy import create_engine
engine = create_engine("mssql+pyodbc://user:password#host:port/databasename?driver=ODBC+Driver+17+for+SQL+Server")
cnxn = engine.connect()
It works for me
Luck!

If you are attempting to connect:
DNS-less
Windows Authentication for a server not locally hosted.
Without using ODBC connections.
Try the following:
import sqlalchemy
engine = sqlalchemy.create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver=SQL+Server')
This avoids using ODBC connections and thus avoids pyobdc interface errors from DPAPI2 vs DBAPI3 conflicts.
I would recommend using the URL creation tool instead of creating the url from scratch.
connection_url = sqlalchemy.engine.URL.create("mssql+pyodbc",database=databasename, host=servername, query = {'driver':'SQL Server'})
engine = sqlalchemy.create_engine(connection_url)
See this link for creating a connection string with SQL Server Authentication (non-domain, uses username and password)