Why does this workc(I get a result set back):
sql_server = 'myserver.database.windows.net'
sql_database = 'pv'
sql_username = 'sqladmin'
sql_password = 'password1'
sql_driver= '{ODBC Driver 17 for SQL Server}'
with pyodbc.connect('DRIVER='+sql_driver+';SERVER=tcp:'+sql_server+';DATABASE='+sql_database+';UID='+sql_username+';PWD='+ sql_password) as conn:
with conn.cursor() as cursor:
cursor.execute("SELECT TOP 3 SAPPHIRE_CASE_ID FROM PV_ALL_SUBMISSIONS_SL")
row = cursor.fetchone()
while row:
print (str(row[0]))
row = cursor.fetchone()
But this fails:
import pyodbc
sql_engine = sqlalchemy.create_engine(f'mssql+pyodbc://{sql_username}:{sql_password}#{sql_server}/{sql_database}?driver=ODBC+Driver+17+for+SQL+Server')
df.to_sql('PV_ALL_CLOSED_CASES_SL', con=sql_engine, if_exists='append')
Error is:
OperationalError: (pyodbc.OperationalError) ('08001', '[08001]
[Microsoft][ODBC Driver 17 for SQL Server]Named Pipes Provider: Could
not open a connection to SQL Server [53]. (53) (SQLDriverConnect);
[08001] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout
expired (0); [08001] [Microsoft][ODBC Driver 17 for SQL Server]A
network-related or instance-specific error has occurred while
establishing a connection to SQL Server. Server is not found or not
accessible. Check if instance name is correct and if SQL Server is
configured to allow remote connections. For more information see SQL
Server Books Online. (53)') (Background on this error at:
https://sqlalche.me/e/14/e3q8)
While I know one is doing a read and the other a write, my issue seems to be just establishing a connection one way vs another, when using the same connection details. It isn't an Azure firewall issue as I am able to connect and run a select statment via the first method, but when using create_engine() of sqlalchemy, it fails to make the connection - but I am pretty sure the connection string is correct.
It is the same variables for server, user name and password being used in both connections.
I think the issue is that the real password as an "#" symbol in it, and so this interferes with the latter connection string.
Thanks to #Larnu, this worked:
from sqlalchemy.engine import URL
connection_string = f"DRIVER={sql_driver};SERVER={sql_server};DATABASE={sql_database};UID={sql_username};PWD={sql_password}"
connection_url = URL.create("mssql+pyodbc", query={"odbc_connect": connection_string})
sql_engine = sqlalchemy.create_engine(connection_url)
I dont have to url encode when I use a cx_Oracle connection, but hey it works now.
Related
I can do a df.to_slq on my local instance of SQL Server just fine. I am getting stuck when trying to do the same df.to_sll using Python and Azure SQL Server. I thought it would essentially be done like this.
import urllib.parse
params = urllib.parse.quote_plus(
'Driver=%s;' % '{ODBC Driver 17 for SQL Server}' +
'Server=%s,1433;' % 'ryan-server.database.windows.net' +
'Database=%s;' % 'ryan_sql_db' +
'Uid=%s;' % 'UN' +
'Pwd={%s};' % 'PW' +
'Encrypt=no;' +
'TrustServerCertificate=no;'
)
from sqlalchemy.engine import create_engine
conn_str = 'mssql+pyodbc:///?odbc_connect=' + params
engine = create_engine(conn_str)
connection = engine.connect()
connection
all_data.to_sql('health', engine, if_exists='append', chunksize=100000, method=None,index=False)
That is giving me this error.
OperationalError: (pyodbc.OperationalError) ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\r\n (10060) (SQLExecDirectW); [08S01] [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure (10060)')
[SQL: INSERT INTO health ([0], [Facility_BU_ID], [Code_Type], [Code], [Description], [UB_Revenue_Code], [UB_Revenue_Description], [Gross_Charge], [Cash_Charge], [Min_Negotiated_Rate], [Max_Negotiated_Rate], etc., etc., etc.
I found this link today:
https://learn.microsoft.com/en-us/sql/machine-learning/data-exploration/python-dataframe-sql-server?view=sql-server-ver15
I tried to do something similar, like this.
import pyodbc
import pandas as pd
df = all_data
# server = 'myserver,port' # to specify an alternate port
server = 'ryan-server.database.windows.net'
database = 'ryan_sql_db'
username = 'UN'
password = 'PW'
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
# Insert Dataframe into SQL Server:
for index, row in df.iterrows():
cursor.execute(all_data.to_sql('health', cnxn, if_exists='append', chunksize=100000, method=None,index=False))
cnxn.commit()
cursor.close()
When I run that, I get this error.
DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'sqlite_master'. (208) (SQLExecDirectW); [42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared. (8180)")
What I'm really hoping to to is df.to_sql, not Insert Into. I am working in Spyder and trying to send the data from my local machine to the cloud.
I read the two links below, and got it working.
https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-set-database-firewall-rule-azure-sql-database?view=azuresqldb-current
https://www.virtual-dba.com/blog/firewalls-database-level-azure-sql/
Basically, you need to open your command window on your local machine, enter 'ipconfig', and grab two IP addresses. Then, enter those into SQL Server in Azure.
EXECUTE sp_set_database_firewall_rule
N'health',
'192.0.1.1',
'192.0.0.5';
Finally, run the small script below, in SQL Server, to confirm that the changes were made correctly.
USE [ryan_sql_db]
GO
SELECT * FROM sys.database_firewall_rules
ORDER BY modify_date DESC
I'm having a problem establishing a connection to the MySQL database No matter what I try or change I seem to be getting the same error. I want to be able to use python to modify MySQL database using ODBC but being new to this I'm not sure how to go about it. This is the error message:
Traceback (most recent call last):
File "C:\Users\Lutho\PycharmProjects\pyODBC\main.py", line 17, in
conx = pyodbc.connect(f'''DRIVER={driver};
pyodbc.OperationalError: ('08001', '[08001] [Microsoft][ODBC Driver 17 for SQL Server]Named Pipes Provider: Could not open a connection to SQL Server [53]. (53) (SQLDriverConnect); [08001] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0); [08001] [Microsoft][ODBC Driver 17 for SQL Server]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. (53)')
Code I used:
import pyodbc
# define the server and database name
driver = '{ODBC Driver 17 for SQL Server}'
server = 'Local instance MySQL80'
database = 'tarsdb'
username = 'root'
password = '1234'
# define connection string
conx = pyodbc.connect(f'''DRIVER={driver};
SERVER={server};
DATABASE={database};
Uid={username};
Pwd={password};''')
# create connection cursor
cursor = conx.cursor()
I also tried using Trusted_Connection=yes and moved my MySQL file into the same folder as my python file but nothing has worked. Is there something I'm missing that I don't know about? If you can solve my problem that would be much appreciated. (If the question looks messy, just know I had a problem formating it.)
The issue was trying to use the ODBC driver for Microsoft SQL Server when the target database was MySQL. ODBC drivers are not interchangeable between different database products.
Connecting to MySQL from Python is possible using pyodbc and "MySQL Connector/ODBC" but there are better alternatives, specifically mysqlclient or pymysql. Both are solid choices, although in many circumstances mysqlclient is significantly faster than pymysql.
I wasn't sure what to title my post, if you have a better idea, feel free to edit the title.
I have not used SQL Alchemy before and the documentation that I have looked at located in the following places, is not helpful:
Connecting to SQL Database Using SQL Alchemy in Python
Tutorial Point
Here is the code I am using:
import sqlalchemy as sal
from sqlalchemy import create_engine
#Here are the parameters I am using:
- server = 'Q-20/fake_example'
- database = 'AdventureWorks2017'
- driver = 'ODBC Driver 17 for SQL Server'
- trusted_connection='yes'
DATABASE_CONNECTION = 'mssql+pyodbc://#server = ' + server + '/database = ' + database + '?trusted_connection = ' + trusted_connection + '&driver=' + driver
engine = sal.create_engine(DATABASE_CONNECTION)
All of that seems to work fine without any problems; however, when I add this line:
connection=engine.connect()
I get the following error message:
sqlalchemy.exc.OperationalError: (pyodbc.OperationalError) ('08001',
'[08001] [Microsoft][ODBC Driver 17 for SQL Server]Named Pipes
Provider: Could not open a connection to SQL Server [53]. (53)
(SQLDriverConnect); [08001] [Microsoft][ODBC Driver 17 for SQL
Server]Login timeout expired (0); [08001] [Microsoft][ODBC Driver 17
for SQL Server]Invalid connection string attribute (0); [08001]
[Microsoft][ODBC Driver 17 for SQL Server]A network-related or
instance-specific error has occurred while establishing a connection
to SQL Server. Server is not found or not accessible. Check if
instance name is correct and if SQL Server is configured to allow
remote connections. For more information see SQL Server Books Online.
(53)')
I am not sure what is wrong with what I am doing, does anyone have any suggestions?
What I have tried so far:
I have confirmed that SQL Server is configured to allow remote connections. I did this check by following the instructions here
Removing the "#" sign before the server, but this just generated the same error message.
I figured out part of what I needed to do. I needed to change my parameters.
Old Parameters:
server = 'Q-20/fake_example'
database = 'AdventureWorks2017'
driver = 'ODBC Driver 17 for SQL Server'
trusted_connection='yes'
New Parameters:
server = 'Q-20'
database = 'AdventureWorks2017'
driver = 'SQL+SERVER+NATIVE+CLIENT+11.0'
trusted_connection='yes'
This is what my code ultimately looked like:
database_connection = 'mssql+pyodbc://Q-20/AdventureWorks2017?trusted_connection=yes&driver=SQL+SERVER+NATIVE+CLIENT+11.0'
I have been trying to connect to Microsoft SQL Server. I have an ODBC connection set up and the test is successful. I am not using Windows Authentication to connect to SQL Server but it keep getting this error:
Cannot be used with Windows authentication
InterfaceError: ('28000', '[28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed. The login is from an untrusted domain and cannot be used with Windows authentication. (18452) (SQLDriverConnect); [28000] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute (0); [28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed. The login is from an untrusted domain and cannot be used with Windows authentication. (18452); [28000] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute (0)')
Here is my code:
import pyodbc
cnxn = pyodbc.connect(Driver='{SQL Server}',
Server='servername.abc.xyz.co.com',
username = 'user_xyz',
password = 'abcdfgh')
I am using Windows 7. Please help me debug this problem
Thanks
I was able to solve this by defining the dsn connection as below:
dsn="DRIVER={SQL
SERVER};server=ip_address_here;database=db_name_here;uid=user;pwd=password"
This worked and I was able to connect and query the sql server.
This is how I do it and it works:
import pyodbc
server_name = "server_name"
db_name = "db_name"
server = "Server="+str(server_name)
db = "Database="+str(db_name)
key = "Driver={SQL Server Native Client 11.0};"+server+";"+db+";"+"Trusted_Connection=yes;"
cnxn = pyodbc.connect(key)
I have a local DB on my machine called 'Test' which contains a table called 'Tags'. I am able to access this DB and query from this table through SQL Server management studio 2008.
However, when using pyodbc I keep running into problems.
Using this:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost:1433;DATABASE=Test')
yields the error:
pyodbc.Error: ('08001', '[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]Invalid connection. (14) (SQLDriverConnectW); [01000] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Invalid Instance()). (14)')
(with or without specifying the port)
Trying an alternative connection string:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost\Test,1433')
yields no error, but then:
cur = conn.cursor()
cur.execute("SELECT * FROM Tags")
yields the error:
pyodbc.ProgrammingError: ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'Tags'. (208) (SQLExecDirectW)")
Why could this be?
I tried changing your query to
SELECT * FROM Test.dbo.Tags
and it worked.
I don't see any authentication attributes in your connection strings. Try this (I'm using Windows authentication):
conn = pyodbc.connect('Trusted_Connection=yes', driver = '{SQL Server}',
server = 'localhost', database = 'Test')
cursor = conn.cursor()
# assuming that Tags table is in dbo schema
cursor.execute("SELECT * FROM dbo.Tags")
For me, apart from maintaining the connection details (user, server, driver, correct table name etc.),
I took these steps:
Checked the ODBC version here (Windows 10) ->
(search for) ODBC ->
Select 32/64 bit version ->
Drivers ->
Verify that the ODBC driver version is present there. If it is not, use this link to download the relevant driver: here
Reference Link: here
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost:1433;DATABASE=Test')
This connection lack of instance name and the port shouldn't be writen like this.
my connection is this:
cn=pyodbc.connect('DRIVER={SQL Server};SERVER=localhost\SQLEXPRESS;PORT=1433;DATABASE=ybdb;UID=sa;PWD=*****')
enter image description here
Try replacing 'localhost' with either '(local)' or '.'. This solution fixed the problem for me.