Develop Reference

How to do df.to_sql using SQL Server in Azure

I can do a df.to_slq on my local instance of SQL Server just fine. I am getting stuck when trying to do the same df.to_sll using Python and Azure SQL Server. I thought it would essentially be done like this.
import urllib.parse
params = urllib.parse.quote_plus(
'Driver=%s;' % '{ODBC Driver 17 for SQL Server}' +
'Server=%s,1433;' % 'ryan-server.database.windows.net' +
'Database=%s;' % 'ryan_sql_db' +
'Uid=%s;' % 'UN' +
'Pwd={%s};' % 'PW' +
'Encrypt=no;' +
'TrustServerCertificate=no;'
)
from sqlalchemy.engine import create_engine
conn_str = 'mssql+pyodbc:///?odbc_connect=' + params
engine = create_engine(conn_str)
connection = engine.connect()
connection
all_data.to_sql('health', engine, if_exists='append', chunksize=100000, method=None,index=False)
That is giving me this error.
OperationalError: (pyodbc.OperationalError) ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\r\n (10060) (SQLExecDirectW); [08S01] [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure (10060)')
[SQL: INSERT INTO health ([0], [Facility_BU_ID], [Code_Type], [Code], [Description], [UB_Revenue_Code], [UB_Revenue_Description], [Gross_Charge], [Cash_Charge], [Min_Negotiated_Rate], [Max_Negotiated_Rate], etc., etc., etc.
I found this link today:
https://learn.microsoft.com/en-us/sql/machine-learning/data-exploration/python-dataframe-sql-server?view=sql-server-ver15
I tried to do something similar, like this.
import pyodbc
import pandas as pd
df = all_data
# server = 'myserver,port' # to specify an alternate port
server = 'ryan-server.database.windows.net'
database = 'ryan_sql_db'
username = 'UN'
password = 'PW'
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
# Insert Dataframe into SQL Server:
for index, row in df.iterrows():
cursor.execute(all_data.to_sql('health', cnxn, if_exists='append', chunksize=100000, method=None,index=False))
cnxn.commit()
cursor.close()
When I run that, I get this error.
DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'sqlite_master'. (208) (SQLExecDirectW); [42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared. (8180)")
What I'm really hoping to to is df.to_sql, not Insert Into. I am working in Spyder and trying to send the data from my local machine to the cloud.

I read the two links below, and got it working.
https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-set-database-firewall-rule-azure-sql-database?view=azuresqldb-current
https://www.virtual-dba.com/blog/firewalls-database-level-azure-sql/
Basically, you need to open your command window on your local machine, enter 'ipconfig', and grab two IP addresses. Then, enter those into SQL Server in Azure.
EXECUTE sp_set_database_firewall_rule
N'health',
'192.0.1.1',
'192.0.0.5';
Finally, run the small script below, in SQL Server, to confirm that the changes were made correctly.
USE [ryan_sql_db]
GO
SELECT * FROM sys.database_firewall_rules
ORDER BY modify_date DESC

Microsoft SQL Server Won't Connect Through Python

I've looked through some other things but haven't been able to find a working solution.
Here is my code:
conn = db.connect("Driver={SQL Server}; Server='Server';Database='Database_DW'; uid='uid'; pwd = 'pwd'")
I run this code and I get the following error:
DatabaseError: ('08001', '[08001] [Microsoft][ODBC SQL Server
Driver][DBNETLIB]SQL Server does not exist or access denied.')
I'm really at a loss here. I can log in fine through the SQL Server Client with the exact some credentials.

Consider adjusting connection strings as parameter values are not quoted. Right now, pypyodbc is attempted to find the 'Server' (quotes included) server.
conn = pypyodbc.connect("DRIVER={SQL Server};server=servername;database=databasename;" + \
"UID=username;PWD=***")
Alternatively, use keyword arguments:
conn = pypyodbc.connect(driver="{SQL Server}", host="servername", database="database",
uid="username", pwd="***")

Connect to SQL Server Database using Pycharm (pyodbc)

I can access my SQL Server via SQL Server Mgmt Studio located at
<IP_ADDRESS> with credentials:
<USERID>
<PWD>
Using pycharm with Anaconda 2.7 as the interpreter my code is:
import pyodbc as p
server = '<IP_ADDRESS>'
database = 'listOfDBs/DB'
username = '<USERID>'
password = '<PWD>'
print ("DB CONNECT ATTEMPT")
try:
cnxn = p.connect(
'DRIVER={SQL Server};SERVER=' + server + ';DATABASE=' + database + ';UID=' + username + ';PWD=' + password)
print ("SUCCESS")
except Exception as e:
print ("Error: " + str(e))
My SQL Server is setup as:
ListOfDBs/DB
Folder1
Folder2
Folder3
Folder3
How do I connect to DB in ListOfDBs?
If I do:
Server = "XX.XXX.XXX/listOfDbs"
Database = "DB"
I get :
('08001', u'[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect); [08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()). (53)')
If I do:
Server = "XX.XXX.XXX"
Database = "listOfDBs/DB"
I get:
('42000', u'[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Cannot open database "listOfDBs/DB" requested by the login. The login failed. (4060) (SQLDriverConnect); [42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Cannot open database "listOfDBs/DB" requested by the login. The login failed. (4060)')
How do I do this? I am using Pycharm community version.

pyodbc- connection failure to SQL Server

I have been trying to connect to Microsoft SQL Server. I have an ODBC connection set up and the test is successful. I am not using Windows Authentication to connect to SQL Server but it keep getting this error:
Cannot be used with Windows authentication
InterfaceError: ('28000', '[28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed. The login is from an untrusted domain and cannot be used with Windows authentication. (18452) (SQLDriverConnect); [28000] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute (0); [28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed. The login is from an untrusted domain and cannot be used with Windows authentication. (18452); [28000] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute (0)')
Here is my code:
import pyodbc
cnxn = pyodbc.connect(Driver='{SQL Server}',
Server='servername.abc.xyz.co.com',
username = 'user_xyz',
password = 'abcdfgh')
I am using Windows 7. Please help me debug this problem
Thanks

I was able to solve this by defining the dsn connection as below:
dsn="DRIVER={SQL
SERVER};server=ip_address_here;database=db_name_here;uid=user;pwd=password"
This worked and I was able to connect and query the sql server.

This is how I do it and it works:
import pyodbc
server_name = "server_name"
db_name = "db_name"
server = "Server="+str(server_name)
db = "Database="+str(db_name)
key = "Driver={SQL Server Native Client 11.0};"+server+";"+db+";"+"Trusted_Connection=yes;"
cnxn = pyodbc.connect(key)

pyodbc connection error when trying to connect to DB on localhost

I have a local DB on my machine called 'Test' which contains a table called 'Tags'. I am able to access this DB and query from this table through SQL Server management studio 2008.
However, when using pyodbc I keep running into problems.
Using this:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost:1433;DATABASE=Test')
yields the error:
pyodbc.Error: ('08001', '[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]Invalid connection. (14) (SQLDriverConnectW); [01000] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Invalid Instance()). (14)')
(with or without specifying the port)
Trying an alternative connection string:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost\Test,1433')
yields no error, but then:
cur = conn.cursor()
cur.execute("SELECT * FROM Tags")
yields the error:
pyodbc.ProgrammingError: ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'Tags'. (208) (SQLExecDirectW)")
Why could this be?

I tried changing your query to
SELECT * FROM Test.dbo.Tags
and it worked.

I don't see any authentication attributes in your connection strings. Try this (I'm using Windows authentication):
conn = pyodbc.connect('Trusted_Connection=yes', driver = '{SQL Server}',
server = 'localhost', database = 'Test')
cursor = conn.cursor()
# assuming that Tags table is in dbo schema
cursor.execute("SELECT * FROM dbo.Tags")

For me, apart from maintaining the connection details (user, server, driver, correct table name etc.),
I took these steps:
Checked the ODBC version here (Windows 10) ->
(search for) ODBC ->
Select 32/64 bit version ->
Drivers ->
Verify that the ODBC driver version is present there. If it is not, use this link to download the relevant driver: here
Reference Link: here

conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost:1433;DATABASE=Test')
This connection lack of instance name and the port shouldn't be writen like this.
my connection is this:
cn=pyodbc.connect('DRIVER={SQL Server};SERVER=localhost\SQLEXPRESS;PORT=1433;DATABASE=ybdb;UID=sa;PWD=*****')
enter image description here

Try replacing 'localhost' with either '(local)' or '.'. This solution fixed the problem for me.