Develop Reference

How can i access Azure Sql Database with Python function App?

I need to create connection with Azure Sql Database(Azure-Server:1) with Azure function which is hosted in Azure-Server:2. Basically both accounts are different but i need to fetch some data from Azure Sql Database which is hosted in (Azure-Server:1).
Is it even possible?
I tried:
import pandas as pd
import pyodbc
from sqlalchemy import create_engine
server = 'server.database.windows.net'
database = 'A'
username = 'B'
password = '###'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
driver = 'ODBC Driver 17 for SQL Server'
DATABASE_CONNECTION = f'mssql://{username}:{password}#{server}/{database}?driver={driver}'
engine = create_engine(DATABASE_CONNECTION,fast_executemany = True)
connection = engine.connect()
The above code is working for within same Azure server Sql DB, when i am creating connection through Azure Function but not working for other Azure linked azure database?

The way you follow the python Azure function to connect azure Database is correct.
conn = "DRIVER={ODBC Driver 17 for SQL Server};SERVER=IP_ADDRESS;DATABASE=DataLake;UID=USERID;PWD=PASSWORD"
quotedConnection = quote_plus(conn)
db_con = 'mssql+pyodbc:///?odbc_connect={}'.format(quotedConnection )
engine = create_engine(db_con)
#event.listens_for(engine, 'before_cursor_execute')
def receive_before_cursor_execute(conn, cursor, statement, params, context, executemany):
if executemany:
cursor.fast_executemany = True
If you want to call the two different Database connections, you can make use of different calls. So that we can avoid the timeout issue.
For that either you can use One by one call or two different calls of the database connection.
Refer here to use different calls to connect databases.

azure function using pyodbc works fine on local machine, but not on azure cloud

I developed a simple python Azure function app using pyodbc to select a few rows from a public IP MS SQL server. My function app runs fine on my laptop, but it doesn't work when I publish it on Azure cloud (I used Consumption - serverless plan, linux environment). Thru the logging, I knows that it always gets stuck at the pyodbc.connect(...) command and time-out.
#...
conn_str = f'Driver={driver};Server={server},{port};Database={database};Uid={user};Pwd={password};Encrypted=yes;TrustServerCertificate=no;Connection Timeout=30'
sql_query = f'SELECT * FROM {table_name}'
try:
conn = pyodbc.connect(conn_str) # always time-out here if running on Azure cloud!!!
logging.info(f'Inventory API - connected to {server}, {port}, {user}.')
except Exception as error:
logging.info(f'Inventory API - connection error: {repr(error)}.')
else:
with conn.cursor() as cursor:
cursor.execute(sql_query)
logging.info(f'Inventory API - executed query: {sql_query}.')
data = []
for row in cursor:
data.append({'Sku' : row.Sku, 'InventoryId' : row.InventoryId, 'LocationId' : row.LocationId, 'AvailableQuantity' : row.AvailableQuantity})
#...
The logging captured:
Inventory API - connection error: OperationalError('HYT00', '[HYT00] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0) (SQLDriverConnect)').
I already include the pyodbc in the requirements.txt file. I also allows all outboundIpAddresses and possibleOutboundIpAddresses of my function app on my SQL server firewall. My function app does not have any network restriction on Azure cloud (or at least it said so on the network settings).
my config file:
driver={ODBC Driver 17 for SQL Server}
server=I tried both IP and full internet host name, both didn't work
Could someone give me a hint? Thanks.

Workarounds to fix the PYODBC connection Error
Try to use a below format
import pyodbc
server = 'tcp:myserver.database.windows.net'
database = 'mydb'
username = 'myusername'
password = 'mypassword'
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
If you are using Domain Name add TCP with domain name server = 'tcp:myserver.database.windows.net' otherwise use the IP server = '129.0.0.1’
If you are using port use like this 'tcp:myserver.database.windows.net,1233’ or '129.0.0.1,1233'
Try to remove additional properties like Connection_Timeout, Trusted_certificate and all and check now.
Refer here

I put the following snippet into my function to check the outbound IP, and found out that Azure use a few outbound IPs that are not listed in the [outboundIpAddresses] and [possibleOutboundIpAddresses] (documented in this MS link)
import requests
#...
outbound_ip_response = requests.request('GET', 'https://checkip.amazonaws.com')
logging.info(f'Inventory API - main()- outbound ip = {outbound_ip_response.text}')
So, I followed the instructions in this link to setup a static outbound IP for my function app and allowed this IP to access my SQL server. It worked.

Cannot connect to SQL Server from python with a specific user

I have to users to connect to a SQL Server. On the server, I need to read from SQL Server into python and write from python to SQL Server.
When I login with one one the users, everything goes fine with connection, whether I use Windows authentication or SQL Server authentication, with this code:
SQL Server authentication:
import sqlalchemy
engine = sqlalchemy.create_engine("mssql+pyodbc://myservername/mydatabasename/driver=SQL+Server+Native+Client+11.0?Trusted_Connection = no/UID = sa/PWD = mypassword")
conn = engine.connect()
df.to_sql(name = 'TestTable1', schema='dbo', con = conn)
Windows authentication:
import sqlalchemy
engine = sqlalchemy.create_engine("mssql+pyodbc://myservername/mydatabasename/driver=SQL+Server+Native+Client+11.0?Trusted_Connection = yes")
conn = engine.connect()
df.to_sql(name = 'TestTable1', schema='dbo', con = conn)
but with another user, I got this error:
Data source name too long State:IM010,Native:0,Origin:[Microsoft][ODBC Driver Manager]
Does this error do something with the other user?

For every login SQL Server needs credentials setup for Windows authentication or SQL Server authentication. You can test the credentials using SSMS before you use them in the Python program.

How to connect to Azure sql database with python SQL alchemy using Active directory integrated authentication

I am trying to connect to an Azure SQL Database using SQL Alchemy in python. The database was recently moved to Azure from on-prem and as I understand that azure does not support Windows Auth.
I can connect to the DB from SSMS using Active Directory Integrated Auth.
When the Db was on prem i would use the following connection string and it worked:
"mssql+pyodbc://#*Server*/*DB*?driver=SQL Server"
I have tried a few other connection string and cannot get it to work.
"mssql+pyodbc://#*Server*/*DB*?driver=SQL Server?Integrated Security=true"
"mssql+pyodbc://#*Server*/*DB*?driver=SQL Server?Trusted_Connection=true"
i keep getting the following error, it seems like sql alchemy is trying to connect via windows auth by default is there anyway i can fix this ?
(pyodbc.Error) ('HY000', '[HY000] [Microsoft][ODBC SQL Server Driver][SQL Server]Windows logins are not supported in this version of SQL Server. (40607) (SQLDriverConnect); [HY000] [Microsoft][ODBC SQL Server Driver][SQL Server]Windows logins are not supported in this version of SQL Server. (40607)')
(Background on this error at: http://sqlalche.me/e/dbapi)

As I known, all of your need are in the offical document Using Azure Active Directory with the ODBC Driver.
First, only the odbc driver 17 version for MS SQL Server supports Active Directory Integrated Authentication if you want to connect to Azure SQL Database by pyodbc. So please make sure you have installed the lastest odbc driver for SQL Server, or you can download from https://learn.microsoft.com/en-us/sql/connect/odbc/microsoft-odbc-driver-for-sql-server?view=sql-server-2017.
Second, please follow the section UI Additions for Azure Active Directory (Windows driver only) to configure the DSN for Azure Active Directory Integrated authentication to SQL Azure.
Then, you can follow the code below to connect to SQL Azure by SQL Alchemy with pyodbc.
from urllib import parse
from sqlalchemy import create_engine
connecting_string = 'Driver={ODBC Driver 17 for SQL Server};Server=tcp:<your sql azure server name>.database.windows.net,1433;Database=<your database name>;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryIntegrated'
params = parse.quote_plus(connecting_string)
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
result = connection.execute("select 1+1 as res")
for row in result:
print("res:", row['res'])
connection.close()
Note: The value of connection string above, you can copy it from the `` tab on Azure portal, but please notice to change the odbc driver version and remove the UID and PWD options.
To connect using Windows Integrated or Active Directory Integrated
(Windows driver only) authentication, specify
Authentication=ActiveDirectoryIntegrated in the connection string. The
driver will choose the correct authentication mode automatically. UID
and PWD must not be specified.
Or you can consider to use Authentication=ActiveDirectoryPassword which be easier than Authentication=ActiveDirectoryIntegrated and the code as below.
from urllib import parse
from sqlalchemy import create_engine
your_user_name = '<your AAD user or configured in SQL Azure Server as the figure below>'
your_password_here = '<your AAD account password>'
#connecting_string = 'Driver={ODBC Driver 17 for SQL Server};Server=tcp:sqlserverleon.database.windows.net,1433;Database=dbleon;Uid='+your_user_name+';Pwd='+your_password_here+';Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryPassword'
connecting_string = 'Driver={ODBC Driver 17 for SQL Server};Server=tcp:sqlserverleon.database.windows.net,1433;Database=dbleon;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryIntegrated'
params = parse.quote_plus(connecting_string)
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
result = connection.execute("select 1+1 as res")
for row in result:
print("res:", row['res'])
connection.close()
Hope it helps.

How do I connect to SQL Server via sqlalchemy using Windows Authentication?

sqlalchemy, a db connection module for Python, uses SQL Authentication (database-defined user accounts) by default. If you want to use your Windows (domain or local) credentials to authenticate to the SQL Server, the connection string must be changed.
By default, as defined by sqlalchemy, the connection string to connect to the SQL Server is as follows:
sqlalchemy.create_engine('mssql://*username*:*password*#*server_name*/*database_name*')
This, if used using your Windows credentials, would throw an error similar to this:
sqlalchemy.exc.DBAPIError: (Error) ('28000', "[28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for us
er '***S\\username'. (18456) (SQLDriverConnect); [28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for us
er '***S\\username'. (18456)") None None
In this error message, the code 18456 identifies the error message thrown by the SQL Server itself. This error signifies that the credentials are incorrect.

In order to use Windows Authentication with sqlalchemy and mssql, the following connection string is required:
ODBC Driver:
engine = sqlalchemy.create_engine('mssql://*server_name*/*database_name*?trusted_connection=yes')
SQL Express Instance:
engine = sqlalchemy.create_engine('mssql://*server_name*\\SQLEXPRESS/*database_name*?trusted_connection=yes')

If you're using a trusted connection/AD and not using username/password, or otherwise see the following:
SAWarning: No driver name specified; this is expected by PyODBC when using >DSN-less connections
"No driver name specified; "
Then this method should work:
from sqlalchemy import create_engine
server = <your_server_name>
database = <your_database_name>
engine = create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver=ODBC+Driver+13+for+SQL+Server')

A more recent response if you want to connect to the MSSQL DB from a different user than the one you're logged with on Windows. It works as well if you are connecting from a Linux machine with FreeTDS installed.
The following worked for me from both Windows 10 and Ubuntu 18.04 using Python 3.6 & 3.7:
import getpass
from sqlalchemy import create_engine
password = getpass.getpass()
eng_str = fr'mssql+pymssql://{domain}\{username}:{password}#{hostip}/{db}'
engine = create_engine(eng_str)
What changed was to add the Windows domain before \username.
You'll need to install the pymssql package.

Create Your SqlAlchemy Connection URL      From Your pyodbc Connection String      OR Your Known Connection Parameters
I found all the other answers to be educational, and I found the SqlAlchemy Docs on connection strings helpful too, but I kept failing to connect to MS SQL Server Express 19 where I was using no username or password and trusted_connection='yes' (just doing development at this point).
Then I found THIS method in the SqlAlchemy Docs on Connection URLs built from a pyodbc connection string (or just a connection string), which is also built from known connection parameters (i.e. this can simply be thought of as a connection string that is not necessarily used in pyodbc). Since I knew my pyodbc connection string was working, this seemed like it would work for me, and it did!
This method takes the guesswork out of creating the correct format for what you feed to the SqlAlchemy create_engine method. If you know your connection parameters, you put those into a simple string per the documentation exemplified by the code below, and the create method in the URL class of the sqlalchemy.engine module does the correct formatting for you.
The example code below runs as is and assumes a database named master and an existing table named table_one with the schema shown below. Also, I am using pandas to import my table data. Otherwise, we'd want to use a context manager to manage connecting to the database and then closing the connection like HERE in the SqlAlchemy docs.
import pandas as pd
import sqlalchemy
from sqlalchemy.engine import URL
# table_one dictionary:
table_one = {'name': 'table_one',
'columns': ['ident int IDENTITY(1,1) PRIMARY KEY',
'value_1 int NOT NULL',
'value_2 int NOT NULL']}
# pyodbc stuff for MS SQL Server Express
driver='{SQL Server}'
server='localhost\SQLEXPRESS'
database='master'
trusted_connection='yes'
# pyodbc connection string
connection_string = f'DRIVER={driver};SERVER={server};'
connection_string += f'DATABASE={database};'
connection_string += f'TRUSTED_CONNECTION={trusted_connection}'
# create sqlalchemy engine connection URL
connection_url = URL.create(
"mssql+pyodbc", query={"odbc_connect": connection_string})
""" more code not shown that uses pyodbc without sqlalchemy """
engine = sqlalchemy.create_engine(connection_url)
d = {'value_1': [1, 2], 'value_2': [3, 4]}
df = pd.DataFrame(data=d)
df.to_sql('table_one', engine, if_exists="append", index=False)
Update
Let's say you've installed SQL Server Express on your linux machine. You can use the following commands to make sure you're using the correct strings for the following:
For the driver: odbcinst -q -d
For the server: sqlcmd -S localhost -U <username> -P <password> -Q 'select ##SERVERNAME'

pyodbc
I think that you need to put:
"+pyodbc" after mssql
try this:
from sqlalchemy import create_engine
engine = create_engine("mssql+pyodbc://user:password#host:port/databasename?driver=ODBC+Driver+17+for+SQL+Server")
cnxn = engine.connect()
It works for me
Luck!

If you are attempting to connect:
DNS-less
Windows Authentication for a server not locally hosted.
Without using ODBC connections.
Try the following:
import sqlalchemy
engine = sqlalchemy.create_engine('mssql+pyodbc://' + server + '/' + database + '?trusted_connection=yes&driver=SQL+Server')
This avoids using ODBC connections and thus avoids pyobdc interface errors from DPAPI2 vs DBAPI3 conflicts.
I would recommend using the URL creation tool instead of creating the url from scratch.
connection_url = sqlalchemy.engine.URL.create("mssql+pyodbc",database=databasename, host=servername, query = {'driver':'SQL Server'})
engine = sqlalchemy.create_engine(connection_url)
See this link for creating a connection string with SQL Server Authentication (non-domain, uses username and password)