Develop Reference

How to do df.to_sql using SQL Server in Azure

I can do a df.to_slq on my local instance of SQL Server just fine. I am getting stuck when trying to do the same df.to_sll using Python and Azure SQL Server. I thought it would essentially be done like this.
import urllib.parse
params = urllib.parse.quote_plus(
'Driver=%s;' % '{ODBC Driver 17 for SQL Server}' +
'Server=%s,1433;' % 'ryan-server.database.windows.net' +
'Database=%s;' % 'ryan_sql_db' +
'Uid=%s;' % 'UN' +
'Pwd={%s};' % 'PW' +
'Encrypt=no;' +
'TrustServerCertificate=no;'
)
from sqlalchemy.engine import create_engine
conn_str = 'mssql+pyodbc:///?odbc_connect=' + params
engine = create_engine(conn_str)
connection = engine.connect()
connection
all_data.to_sql('health', engine, if_exists='append', chunksize=100000, method=None,index=False)
That is giving me this error.
OperationalError: (pyodbc.OperationalError) ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\r\n (10060) (SQLExecDirectW); [08S01] [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure (10060)')
[SQL: INSERT INTO health ([0], [Facility_BU_ID], [Code_Type], [Code], [Description], [UB_Revenue_Code], [UB_Revenue_Description], [Gross_Charge], [Cash_Charge], [Min_Negotiated_Rate], [Max_Negotiated_Rate], etc., etc., etc.
I found this link today:
https://learn.microsoft.com/en-us/sql/machine-learning/data-exploration/python-dataframe-sql-server?view=sql-server-ver15
I tried to do something similar, like this.
import pyodbc
import pandas as pd
df = all_data
# server = 'myserver,port' # to specify an alternate port
server = 'ryan-server.database.windows.net'
database = 'ryan_sql_db'
username = 'UN'
password = 'PW'
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
# Insert Dataframe into SQL Server:
for index, row in df.iterrows():
cursor.execute(all_data.to_sql('health', cnxn, if_exists='append', chunksize=100000, method=None,index=False))
cnxn.commit()
cursor.close()
When I run that, I get this error.
DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'sqlite_master'. (208) (SQLExecDirectW); [42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared. (8180)")
What I'm really hoping to to is df.to_sql, not Insert Into. I am working in Spyder and trying to send the data from my local machine to the cloud.

I read the two links below, and got it working.
https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-set-database-firewall-rule-azure-sql-database?view=azuresqldb-current
https://www.virtual-dba.com/blog/firewalls-database-level-azure-sql/
Basically, you need to open your command window on your local machine, enter 'ipconfig', and grab two IP addresses. Then, enter those into SQL Server in Azure.
EXECUTE sp_set_database_firewall_rule
N'health',
'192.0.1.1',
'192.0.0.5';
Finally, run the small script below, in SQL Server, to confirm that the changes were made correctly.
USE [ryan_sql_db]
GO
SELECT * FROM sys.database_firewall_rules
ORDER BY modify_date DESC

The SQL contains 0 parameter markers, but 1 parameters were supplied | pyodbc

I am trying to send a data from my code to SQL Server, but i got this error:
ProgrammingError:
('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near 'emotion'. (102) (SQLExecDirectW); [42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared. (8180)")
My Code:
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};'
'Server=DESKTOP-T7OFQV6\SQLEXPRESS1;'
'Database=VidgaEmotionRecognition;'
'Trusted_Connection=yes;')
cursor = conn.cursor()
cursor.execute('''INSERT INTO FaceEmotion emotion ?''', index_pred_as_int)
index_pred is a value from previous code snippet.
Structure of FaceEmotion table is like this:
Column: emotion - nvarchar(50) - allows nulls
Column: _id - int - key identity

Python to Access connection Database Error?

I am looking to connect python to an Access database with the following code:
connStr = (
r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=O:\Architecture\DAART\Data Analytics Team\DAART.accdb;"
)
cnxn = pyodbc.connect(connStr)
cursor = cnxn.cursor()
df = pd.read_sql("select * from APMS SV-8 Report", cnxn)
For the last line of code, I am receiving the following error message:
DatabaseError: Execution failed on sql 'select * from APMS SV-8 Report': ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in FROM clause. (-3506) (SQLExecDirectW)')

Access SQL requires you bracket table names if they contain spaces, keywords or special characters:
select * from [APMS SV-8 Report]

Unable to write to SQL Server even when using parameters

The SQL Server has these properties.
I want to write df to a SQL Server table but I get an error "table does not exist".
What am I doing wrong?
conn = pyodbc.connect('Driver={SQL Server};'
'Server=DESKTOP-FCBC9GE/NEW_INSTANCE;'
'Database=master;'
'Trusted_Connection=yes;')
cursor = conn.cursor()
cursor.execute('SELECT * FROM df')

Creating stored procedure in MS SQL through pyodbc

I am trying to create a store procedure in master in mssql through python code. The following is my code:
import pyodbc
conn = pyodbc.connect("driver={SQL Server};server=localhost; database=master; trusted_connection=true",
autocommit=True)
cursor = conn.cursor()
sqlcommand = """
USE master
GO
CREATE PROCEDURE sp_myCustomSystemProc
AS
BEGIN
PRINT 'myCustomCode'
END
GO
EXEC sp_ms_marksystemobject 'sp_myCustomSystemProc'
"""
cursor.execute(sqlcommand)
cursor.commit()
conn.commit()
After running this python code, I am getting this error:
Traceback (most recent call last):
File "auto_complete.py", line 27, in <module>
cursor.execute(sqlcommand)
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]
Incorrect syntax near 'GO'. (102) (SQLExecDirectW);
[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]
'CREATE/ALTER PROCEDURE' must be the first statement in a query batch. (111);
[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]
Incorrect syntax near 'GO'. (102);
[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]
Incorrect syntax near 'sp_myCustomSystemProc'. (102)")
Can anyone please help me to resolve this?

Since your connection string already specifies the master database (i.e. database=master;), simply remove
USE master
GO
from your query.