Develop Reference

azure function using pyodbc works fine on local machine, but not on azure cloud

I developed a simple python Azure function app using pyodbc to select a few rows from a public IP MS SQL server. My function app runs fine on my laptop, but it doesn't work when I publish it on Azure cloud (I used Consumption - serverless plan, linux environment). Thru the logging, I knows that it always gets stuck at the pyodbc.connect(...) command and time-out.
#...
conn_str = f'Driver={driver};Server={server},{port};Database={database};Uid={user};Pwd={password};Encrypted=yes;TrustServerCertificate=no;Connection Timeout=30'
sql_query = f'SELECT * FROM {table_name}'
try:
conn = pyodbc.connect(conn_str) # always time-out here if running on Azure cloud!!!
logging.info(f'Inventory API - connected to {server}, {port}, {user}.')
except Exception as error:
logging.info(f'Inventory API - connection error: {repr(error)}.')
else:
with conn.cursor() as cursor:
cursor.execute(sql_query)
logging.info(f'Inventory API - executed query: {sql_query}.')
data = []
for row in cursor:
data.append({'Sku' : row.Sku, 'InventoryId' : row.InventoryId, 'LocationId' : row.LocationId, 'AvailableQuantity' : row.AvailableQuantity})
#...
The logging captured:
Inventory API - connection error: OperationalError('HYT00', '[HYT00] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0) (SQLDriverConnect)').
I already include the pyodbc in the requirements.txt file. I also allows all outboundIpAddresses and possibleOutboundIpAddresses of my function app on my SQL server firewall. My function app does not have any network restriction on Azure cloud (or at least it said so on the network settings).
my config file:
driver={ODBC Driver 17 for SQL Server}
server=I tried both IP and full internet host name, both didn't work
Could someone give me a hint? Thanks.

Workarounds to fix the PYODBC connection Error
Try to use a below format
import pyodbc
server = 'tcp:myserver.database.windows.net'
database = 'mydb'
username = 'myusername'
password = 'mypassword'
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
If you are using Domain Name add TCP with domain name server = 'tcp:myserver.database.windows.net' otherwise use the IP server = '129.0.0.1’
If you are using port use like this 'tcp:myserver.database.windows.net,1233’ or '129.0.0.1,1233'
Try to remove additional properties like Connection_Timeout, Trusted_certificate and all and check now.
Refer here

I put the following snippet into my function to check the outbound IP, and found out that Azure use a few outbound IPs that are not listed in the [outboundIpAddresses] and [possibleOutboundIpAddresses] (documented in this MS link)
import requests
#...
outbound_ip_response = requests.request('GET', 'https://checkip.amazonaws.com')
logging.info(f'Inventory API - main()- outbound ip = {outbound_ip_response.text}')
So, I followed the instructions in this link to setup a static outbound IP for my function app and allowed this IP to access my SQL server. It worked.

pyodbc can't connect to database

I'm using pyodbc library from here and I'm connecting this way:
conn = pyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};Server=(localdb)\MSSQLLocalDB;Integrated Security=true; database = online_banking; autocommit = True')
I use MSSQLLocalDBbecause it's the default instance name for SQL Server 2014. And this last version of Python 2.7.
However I cant run any simple query, every if them raise the error, saying that there is no such object or in that particular case database:
cursor.execute('use online_banking;')
The full error:
pyodbc.Error: ('08004', "[08004] [Microsoft][SQL Server Native Client 11.0][SQL Server]Database 'online_banking' does not exist. Make sure that the name is entered correctly. (911) (SQLExecDirectW)")
So what is wrong here?
There is only 1 instance installed and such databases(.mdf)
As you can see only 1 engine:
Selecting that engine will allow me to see online_banking db
upd1 Database've been created this way:
CREATE DATABASE [online_banking]
ON PRIMARY
( NAME = N'online_banking', FILENAME = N'C:\...\online_banking.mdf' ,
SIZE = 512000KB , MAXSIZE = UNLIMITED, FILEGROWTH = 30%)
LOG ON
( NAME = N'online_banking_log', FILENAME = N'C:\...\online_banking_log.ldf' ,
SIZE = 1024KB , MAXSIZE = 20GB , FILEGROWTH = 10%)
GO
upd2 I've used built-in tool sqlcmd.
So this sqlcmd -S (LocalDB)\MSSQLLocalDB -i C:\Users\1.sql -E have shown, that
MSSQLLocalDB doesn't have my database.
However sqlcmd -S localhost -i C:\Users\1.sql -E performed successfully.
I'm totally confused, I' ve installed only one server, moreover SQL Management studio sees only one local server with my online_banking DB. This is look really weird to me.
Trying to use this connection string in Python
conn = pyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};Server=localhost;Integrated Security=true; database = online_banking; autocommit = True')
causes the error below:
pyodbc.Error: ('28000', '[28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]\x... "". (18456) (SQLDriverConnect); [01S00] [Microsoft][SQL Server Native Client 11.0]\xcd\xe5\xe....xe8\xff (0); [28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]\xce...ff "". (18456); [01S00] [Microsoft][SQL Server Native Client 11.0]\xcd\xe.... (0)'
upd3: Specified mdf should be attached, got it:
Tried several ways, always errors (with database specified or not in connection string):
conn = pyodbc.connect(
r'Driver={SQL Server Native Client 11.0};Server=(localdb)\MSSQLLocalDB; database =online_banking; AttachDbFilename=C:\Program Files\Microsoft SQL Server\MSSQL12.SQLSERVERINSAF\MSSQL\DATA\online_banking.mdf;Trusted_Connection=Yes; Integrated Security=true; database = online_banking;')
error: A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
I found out, that may be related with parent server which already have attached this db, but failed to solve this.
upd4
I tried simple code from here to see if "online_banking" shows up in the list of databases for that instance. But faced another error:
pyodbc.Error: ('08001', '[08001] [Microsoft][SQL Server Native Client 11.0]\ - unreadable error
In addition that database according to SSMS seems have already attached by online_banking DB

As it turns out, the database in question was already attached to the default instance of SQL Server on the local machine, so all that was needed to connect was
import pyodbc
conn_str = (
r"Driver={SQL Server Native Client 11.0};"
r"Server=(local);"
r"Database=online_banking;"
r"Trusted_Connection=yes;"
)
conn = pyodbc.connect(conn_str)
There were two main points of confusion:
Q: What is the name of a SQL Server "default instance"?
A: It doesn't have one.
When referring to a SQL Server instance by name, a default instance simply goes by the name of the machine, while a named instance is identified by MachineName\InstanceName. So, on a server named PANORAMA
If we install a "default instance" of SQL Server we refer to it as PANORAMA.
If we install a "named instance" called "SQLEXPRESS" we refer to it as PANORAMA\SQLEXPRESS.
If we are referring to a SQL server instance on the local machine we can use (local) instead of PANORAMA.
Q: Do (local) and (localdb) mean the same thing?
A: NO.
(local) and (local)\InstanceName refer to "real" server-based instances of SQL Server. These are the instances that have been around since SQL Server was first released. They run as a service and are able to accept network connections and do all of the the things we expect a database server to do.
(localdb) and (localdb)\InstanceName references – with (localdb) usually capitalized as (LocalDB) for clarity – are used to connect to "SQL Server LocalDB" instances. These are temporary local SQL Server instances primarily intended for developers. For details see the following MSDN blog post:
SQL Express v LocalDB v SQL Compact Edition

It could possibly be a security issue. You are using integrated security so it will use the security credentials of the windows login that the client program is running. If that user or a group that the user belongs to does not have at least public access to the database, it will appear as if the database does not exist. Either ensure that the user or a group that the user is a member of is set up with a login and that it has at least public access to your database, or use SQL server authentication and send a username and password in your connection string.

Python connect to MySQL database on web server

I wanted to know the process of connecting to a MySQL database that is hosted on a web server.
I have a basic free webserver for testing on 000webhost on which I created a MySQL database.
I have the credentials for the database which I will pretend are
host - mysql.webhost000.com
user - dummy_user
password - dummy_password
database - dummy_database
and I have a python script executing from my local computer with internet access
import MySQLdb
db = MySQLdb.connect(host="mysql.webhost000.com",
port=3306,
user="dummy_user",
passwd="dummy_password",
db="dummy_database")
I was hoping it would connect as long as I have the right credentials but when I execute the script it just hangs and once I quit it I see the error
Can't connect to MySQL server on 'mysql.webhost000.com' (4)
Am I missing some steps?

There are two possible problems and im not able to recreate the first one. One is the
host="mysql.webhost000.com"
is incorrect and throwing an error. The connection could be listed as another way. The other I noticed is this is usually how I set up my connection script.
import MySQLdb
def connect():
db = MySQLdb.connect(host="mysql.webhost000.com",
port=3306,
user="dummy_user",
passwd="dummy_password",
db="dummy_database")
c = conn.cursor()
return c, db

Connection to external MySQL instance using Python

I have tried the following code:
MySQLdb.connect(host='xxx', port=3306, user='yyy')
But I get the following error:
(2005, "Unknown MySQL server host ...
I have tried to remove all firewall restrictions on the external MySQL instance, as a test. I am able to connect to the instance from my developing machine.
I believe this should be possible now that the App Engine supports sockets, or am I wrong?

I think this connection is not allowed (no external socket support in MySQLdb) :
https://developers.google.com/appengine/docs/python/cloud-sql/?hl=en#Python_Using_a_local_MySQL_instance_during_development
You have to use localhost/127.0.0.1 or CloudSQL socket (unix_socket='/cloudsql/'):
if (os.getenv('SERVER_SOFTWARE') and
os.getenv('SERVER_SOFTWARE').startswith('Google App Engine/')):
db = MySQLdb.connect(unix_socket='/cloudsql/' + _INSTANCE_NAME, db='guestbook', user='root')
else:
db = MySQLdb.connect(host='127.0.0.1', port=3306, user='root')
# Alternately, connect to a Google Cloud SQL instance using:
# db = MySQLdb.connect(host='ip-address-of-google-cloud-sql-instance', port=3306, user='root')

Connecting to MS-SQL from pyodbc using windows authentication

I am trying to connect to MSSQL server using pyodbc. I can connect to the server and query it using the basic authentication mode as:
connection = pyodbc.connect("DRIVER={Easysoft ODBC-SQL Server};SERVER=192.168.2.119;DATABASE=dbame;UID=**;PWD=****")
Connection to MSSQL can also be done using Windows Authentication where it takes the parameters
DOMAIN
USERNAME
PASSWORD
I don't know how to use this sort of credential from pyodbc to connect to the MSSQL Server.
Furthermore, the ODBC driver I am using (Easysoft ODBC-SQL Server) needs licensing. Don't we get such drivers for free?

The Easysoft SQL Server driver parameters to use NTLM authentication are
Trusted_Domain=<domain name>
NTLMv2=Yes|No
Trusted_Connection=Yes|No
And UID, PWD as usual.
NTLM can also be triggered simply using a UID that looks like
DOMAIN\USER
If you want to use Kerberous, the following can be set
ServerSPN=SPN
Its all the the user guide for the driver

connection = pyodbc.connect("DRIVER={Easysoft ODBC-SQL Server};SERVER=192.168.2.119;DATABASE=dbame;UID=;PWD=**")
The string part of the connection is what is known as a DSN-Less connection so you can pass in any of the attributes required for example :-
connection = pyodbc.connect("DRIVER={Easysoft ODBC-SQL Server};SERVER=192.168.2.119;DATABASE=dbame;UID=MyWindowsUserName;PWD=MyPassword;Trusted_Domain=MyWindowsDomainName;Trusted_Connection=1")
Trusted_Connection = 1 tells the Easysoft driver that you intend to use the Trusted_Domain, user ( UID ) and password ( PWD ) to login to SQL Server.
For a full list of all the attributes available within the Easysoft ODBC-SQL Server Driver please read the Attributes section of the manual.