I am trying to reduce the loading times to extract a database from SQL SERVER. I have read a lot about this library but no matter how hard I have tried I have not been able to connect, can someone tell me if this is possible or guide me to configure it, thanks.
try using pyodbc config
per this discussion, at least some support has been added
I used sqlalchemy's urlib to make urls: and since url objects do not have a split() method, I cast to string.
connection_url = str(sqlalchemy.engine.URL.create("mssql",database=database, host=servername,
username=username,
password=password,
query = {'driver':'SQL Server'}))
I then used connectorx.read_sql(connection_url, 'select * from sys.databases') to validate the connection worked.
It worked for my system's sql server authentication database connections but failed to login for Windows Domain Authentication.
Related
Moving this question from DevOps Stack Exchange where it got only 5 views in 2 days:
I would like to query an Azure Database for MySQL Single Server.
I normally interact with this database using a universal database tool (dBeaver) installed onto an Azure VM. Now I would like to interact with this database using Python from outside Azure. Ultimately I would like to write an API (FastAPI) allowing multiple users to connect to the database.
I ran a simple test from a Jupyter notebook, using SQLAlchemy as my ORM and specifying the pem certificate as a connection argument:
import pandas as pd
from sqlalchemy import create_engine
cnx = create_engine('mysql://XXX', connect_args={"ssl": {"ssl_ca": "mycertificate.pem"}})
I then tried reading data from a specific table (e.g. mytable):
df = pd.read_sql('SELECT * FROM mytable', cnx)
Alas I ran into the following error:
'Client with IP address 'XX.XX.XXX.XXX' is not allowed to connect to
this MySQL server'.
According to my colleagues, a way to fix this issue would be to whitelist my IP address.
While this may be an option for a couple of users with static IP addresses I am not sure whether it is a valid solution in the long run.
Is there a better way to access an Azure Database for MySQL Single Server from outside Azure?
As mentioned in comments, you need to whitelist the IP address ranges(s) in the Azure portal for your MySQL database resource. This is a well accepted and secure approach.
I was looking at the documentation listed sqlalchemy engine interface dialect and I can't see where it says how to do this.
I know I need to use trusted_connection = 'yes' but I want to know where it says that that is what we need to do.
Can someone point me in the correct direction?
I figured it out.
What to do:
1) Use the DBAPI to connect to the database, in my case, it is pyodbc.
2) The connection string you will pass will be based upon the driver being used, in my case, ODBC Driver 11 for SQL Server.
3) In the syntax used for my driver, it specifically says to use trusted_connection='yes' for windows authentication.
4) The values you need to pass to create_engine are the same values you would send to the driver you are using.
Details:
1) What I am doing:
I am using pyodbc to access SQL Server. I am then using sqlalchemy to work with the database.
2) Trusted_connection = 'yes' is not found on the documentation for SQLAlchemy because it is part of the connection string, not the syntax for create_engine.
3) Documentation for pyodbc and Connection Strings. Pyodbc does not look at the connection string. The connection string is passed to the databse driver unmodified and is driver-specific. Documentation for Connecting to databases
4) Reference for all connection strings can tell you specifically what the connection string needs to look like for each database you are trying to connect to.
5) Not part of my original question, but was curious about this too: Port number is only required if connecting outside of the network SQL Server is on. When is Port Number Required
6) In the create engine documentation it says that the URL is the first positional argument, but when you look at the parameters, it does not specify this. I'd love for someone to prove me wrong, but I'm convinced this is true.
I am using Pycharm and try to link to a mssql server. When I link to a server that requires SQL authentication, the connect is created successfully. However, when I try to link to a server that requires my Windows Authentication, even though I use my username or password of windows log in, I cannot connect successfully. May I know what should be a proper way to setup if it is windows authentication.
I am using the below code:
import pymssql
conn=pymssql.connect(host="10.xx.xx.xx",user="CORPORATE/mywindowsloginname",password="mypassword",database="BIC_reference")
cur=conn.cursor()y
cur.execute('SELECT top 10 * FROM dbo.hi_invoiceline')
print (cur.fetchall())
in order to use Windows Authentication you have to add the property trusted_connection='yes' to your connection string. In this case you can omit user and password:
conn=pymssql.connect(host="10.xx.xx.xx",database="BIC_reference",trusted_connection='yes')
When using Windows Authentication, you should not specify any user credentials. The following should work assuming your Windows account has the relevant permissions:
conn=pymssql.connect(host="10.xx.xx.xx",database="BIC_reference")
I have tested this using pymssql-2.1.3. Using this version there was no need to specify trusted_connection='yes' (see apomene's answer), however, you may want to try that as well in case the above snippet doesn't work.
I have a python application designed to pull data from a remote database server using pyodbc, then organize and display the data in a spreadsheet. I've had it working fine for several months now, with multiple coworkers in my department using it through a shared network folder.
My connection:
pyodbc.connect('DRIVER={SQL Server};
SERVER=<myServer_name>;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword')
A different employee within our same network recently tried to use the program and got this error:
pyodbc.Error: ('08001','[08001][Microsoft][ODBC SQL Server Driver]
[DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect)')
It looked like a simple permissions issue to me, so to confirm I replaced the userID and password with my own hardcoded in, but it gave the same error. Furthermore the same employee can log in and execute queries through SQL Server Management Studio without issue.
Since everyone else in our department can still use the application fine, I know it must be a client-side issue, but I just can't pinpoint the problem. Any input would be greatly appreciated, Thanks!
Updates:
Per flipperPA's answer below, I updated my connection string to include the port:
con = pyodbc.connect('''DRIVER={SQL Server};
SERVER=<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword;''')
Unfortunately we still got the same error.
He is running 32-bit Windows 7 on an HP machine, the same setup as the rest of the group so it shouldn't to be an os-level issue.
He does operate SSMS on the same machine, but I ran through the telnet check just be sure - no issue there.
I've taught myself the pyodbc API and basic SQL, but I'm still relatively new to the underlying concepts of databases and remote connections. Could you explain the TDS driver a little more?
When including SERVER, I've found you often need to include the PORT as well; this is the most likely problem:
pyodbc.connect('DRIVER={SQL Server};
SERVER=<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword')
I connect mostly from Linux, however. Could it be the other person is connecting from Mac OS/X or Linux? If so, they'll need to use the FreeTDS driver (MS provides one as well, but it is flaky, at best). If you continue to have problems, from the coworkers machine, make sure you can connect from the machine you're having issues with (unless its the same machine they can connect SSMS from):
telnet <myServer_name> 1433
If it connects, you're good, if it hangs on connecting, you're most likely looking at a firewall issue. Good luck!
After talking with a knowledgeable friend I was finally able to figure out my issue!
For some reason, the user's system was configured to connect using named pipes, but the server I was connecting to only had TCP/IP protocol enabled. The solution was to force the application to use TCP/IP by adding "tcp:" to the front of the server name.
The fixed connection string:
pyodbc.connect('''DRIVER={SQL Server};
SERVER=tcp:<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword
''')
If for any of you still doesn't work you can try to refer the Localdb (if that's the case) by its pipe address.
If the DB name is LocalDBA, in cmd type
SqlLocalDB LocalDBA v
Get the instance pipe name and then put it on the server name:
conn_str = (
r'DRIVER={SQL Server};'
r'SERVER=np:\\.\pipe\LOCALDB#ECE5B7EE\tsql\query;'
r'PORT=1433;'
r'DATABASE=VT-DE;'
r'trusted_connection=yes;'
)
I am using SQLalchemy with pyodbc to connect to a SQL server. I can connect using "Windows Authentication":
create_engine('mssql+pyodbc://[ServerName]/[DatabaseName]',echo=True)
That is fine but when I try to login (SQL server authentication) it fails:
create_engine('mssql+pyodbc://[User]:{Password]#[ServerName]/[DatabaseName]',echo=True)
Is my code correct? Am I missing a setting?
Is there a way of listing database users to check the names?
Yes, your connection string is correct. Make sure the user/pwd combinations are correct. Also note that user/pwd can only be used for users which use "SQL Server authentication" (in the code below SQL_LOGIN) and not "Windows authentication"
Given you connect with enough permissions, you can execute the following to see all logins:
for x in session.execute("SELECT name, type, type_desc FROM sys.server_principals"):
print x