Python SQL Alchemy: Writing Python Dataframe to MSSQL Server - python

I am trying to write my python dataframe to a MSSQL server. The table design has already been created in the server.
Below is the code I am attempting to use. But I get an error.
import sqlalchemy as sa
import pyodbc
engine = sa.create_engine('mssql+pyodbc://username:password#server/database', echo = True)
# engine
df_EVENT5_14.to_sql("MODREPORT", engine)
This is the error I'm getting:
DBAPIError: (pyodbc.Error) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')

You have to use correct connection string as below
engine = sqlalchemy.create_engine('mssql+pyodbc://'+user+':'+password+'#'+host+':'+port+'/'+database+'?'\
+'driver=SQL+Server+Native+Client+11.0')
Please make sure that, put correct details into variables that used in connection string. You may use different driver as well according to your please change the driver, in my case I put SQL+Server+Native+Client+11.0.

Related

I'm trying to connect to a SQL Express Server on my machine using sqlalchemy but I'm getting an error

I'm trying to connect to a SQL Express Server on my machine using sqlalchemy but I'm getting an error. Here is the code I have:
import sqlalchemy as sal
import pandas as pd
sqlcon=sal.create_engine('mssql+pyodbc://#' + 'DESKTOP-A1BUCDR\SQLEXPRESS' + '/' + 'Northwind' + '?driver=ODBC+Driver+13+for+SQL+Server')
df = pd.read_sql_query('select * from Orders', sqlcon)
And here is the error I'm getting:
InterfaceError: (pyodbc.InterfaceError) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
(Background on this error at: https://sqlalche.me/e/14/rvf5)
Does anyone have any ideas on what I may be getting wrong here?
Your URL evaluates to:
mssql+pyodbc://#DESKTOP-A1BUCDR\\SQLEXPRESS/Northwind?driver=ODBC+Driver+13
+for+SQL+Server
The errors points to not finding the data source name (DSN), which I think you must not want to use since you give what looks like a hostname and a driver parameter.
That's fine too:
mssql+pyodbc://<UID>:<PWD>#DESKTOP-A1BUCDR:1433/Northwind?driver=ODBC+Driver+17+for+SQL+Server
If your port is the default (1433), you don’t really need to add it.
Why is your hostname DESKTOP-A1BUCDR\SQLEXPRESS ? and why are you using ODBC Driver 13 for SQL Server and not a more recent version ?
You could instead try using a URL built for your hostname or directly the pyODBC string.

Connect to SQLite3 with pyodbc for fast_executemany

We currently use a program that creates and writes large datasets to databases, but the process can take a long time. We are trying to incorporate cursor.fast_executemany = True from sqlalchemy to improve the write times to these databases. My code errors out when I try to create an engine using SQLite3 and pyodbc here:
import pandas as pd
from sqlite3 import connect
import pyodbc
from sqlalchemy import create_engine
engine = create_engine('SQLite3 ODBC Driver+pyodbc:///C:\\Users\\Documents\\PythonScripts\\FLR.sosat')
conn = engine.connect()
c = conn.cursor()
We have tried numerous ways where we specify the driver and server and things like that like the following:
# conn = pyodbc.connect('DRIVER={SQL Server};'
# 'SERVER=localhost;'
# 'DATABASE=C:\\Users\\Documents\\PythonScripts\\FLR.sosat')
The single engine line seems to be the closest to working due to us receiving driver and server errors from the commented out code above. We have downloaded the ODBC driver from http://www.ch-werner.de/sqliteodbc/
We receive the ArgumentError: Could not parse rfc1738 URL from string.
We would appreciate any help or ideas on how to get the SQLite3 database to pyodbc and how to improve the write speed. Thanks!
Note the .sosat file is a database file that uses sqlite3, it should work like any .db file
We tried the fix from here: Connect to SQLite3 server using PyODBC, Python and that did not work for us, we received the driver error:
InterfaceError: ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
What you are trying to accomplish will not work for two reasons:
Reason 1:
SQLAlchemy does not support pyodbc as a DBAPI layer for SQLite.
Reason 2:
Even if SQLAlchemy did support sqlite+pyodbc:// the SQLite ODBC Driver would have to support "parameter arrays", an optional ODBC feature that fast_executemany = True uses to do its magic. Not all ODBC drivers support fast_executemany = True as shown here. A quick test with vanilla pyodbc shows that "SQLite3 ODBC Driver" doesn't support it, in fact it crashes the Python interpreter:
crsr.fast_executemany = True
crsr.executemany(
f"INSERT INTO {table_name} (txt) VALUES (?)", [("foo",), ("bar",)]
)
# Process finished with exit code -1073741819 (0xC0000005)
(Error 0xC0000005 is "Access Violation".)
Have you tried
import sqlite3
db = r'C:\Users\Documents\PythonScripts\FLR.sosat'
conn = sqlite3.connect(db)
print('connection established')

Python: pyodbc can't connect to SQL Server

I am trying to connect to SQL Server 2008 R2 using pyodbc.
This is my code snippet:
conn_strng = ['Driver={SQL Server Native Client
11},'Database=db_name','uid=user','pwd=password','trusted_source=yes']
cnxn = pyodbc.connect(';'.join(conn_strng))
I get the following error:
('IM002', '[IM002] [Microsoft] [ODBC Driver Manager] Data Source name not found or no default driver specified (0) (SQLDriverConnect)')
I have installed the SQL Server ODBC Driver 11. I am using Windows Sever 2012. I have searched in Stackoverflow and tried by changing the Driver string and other string formatting (i.e. no space etc.) but no luck.
Do I need to specify port as well?
Please help.
Ok. So I have found out a solution. Below is the correct connection string
Driver={SQL Server} #<--Note that it is different than earlier
Database=db_name
uid=user
pwd=password
trusted_source=no #<--if we set this 'yes' then it will try to connect the DB with windows login. This was causing error
I think it is better to use Driver={SQL Server} for a safer side.

pyodbc connection to Teradata not working after upgrade to Teradata 16.00 driver

I have a Windows application that utilizes pyodbc to connect to Teradata. Currently, the clients have either the 14.10 or 15.00 drivers installed to perform this connection. The connection is made using this (simplified) code:
import pyodbc
constr = 'DRIVER={Teradata};DBCNAME='+dbname+';UID='+uid+';PWD='+pwd+';QUIETMODE=YES;'
pyodbc.pooling = False
pyodbc.connect(constr, ANSI=True, autocommit=True)
After upgrading to the 16.00 driver, this no longer works. Instead it throws the following error on the same code:
pyodbc.Error: ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
I've tried a few variations of the connection string but all return the same error:
constr = """DRIVER={Teradata};DSN='+dbname+';UID='+uid+';PWD='+pwd+';QUIETMODE=YES;"""
constr = """DRIVER={Teradata};DSN='+dbname+';DATABASE='+dbname+';UID='+uid+';'+pwd+';QUIETMODE=YES;"""
constr = """Provider=Teradata;DBCNAME='+dbname+';DATABASE='+dbname+';UID='+uid+';PWD='+pwd+';QUIETMODE=YES;"""
What do I need to do to utilize 16.00 teradata drivers and pyodbc?
I had the exact same issue. This recommendation may sound silly, but my server admins changed the name of the connection string and Windows SQLDriverConnect was bombing.
On my workstation, in the DSN connection list, the DSN name is "TDPROD" and the string description is "Teradata". However, on the server, the DSN name is "TDPROD" and the string description is "Teradata Database ODBC Driver 16.10". So, I had to update my python function from this (Please note that I'm using Python 3.6 so I can use "f" strings; if you're using an earlier version of Python make sure you perform string interpolation by using supported methodologies):
pyodbc.connect(f"""DRIVER=Teradata;
DBCNAME=TDPROD;
UID={user};
PWD={password};
QUIETMODE=YES""",
autocommit=True,
unicode_results=True)
to this:
pyodbc.connect(f"""DRIVER=Teradata Database ODBC Driver 16.10;
DBCNAME=TDPROD;
UID={user};
PWD={password};
QUIETMODE=YES""",
autocommit=True,
unicode_results=True)
Hope this helps to at least give you something else to check that wasn't immediately obvious to me.

Issue Connecting to Teradata using Python

I need to connect to the Teradata database using python. I have used the below code:
import pyodbc
import teradata
cnxn = pyodbc.connect('DRIVER={Teradata};SERVER=<*ServerName*>;DATABASE=<*Database Name*>;UID=<*User ID*>;PWD=<*Password*>',ansi=True, autocommit=True)
cur = cnxn.cursor()
But on executing, I am getting the error as :
Error: ('28000', '[28000] [Teradata][ODBC Teradata Driver] Not enough
information to log on (0) (SQLDriverConnect); [28000] [Teradata][ODBC
Teradata Driver] Not enough information to log on (0)')
What I am missing here ? What else needs to be included to set up the connection ?
Also, is there any other way to set up the connection. While looking, I have come across teradata.UdaExec(). Can this also be used?
The following works in CentOS Linux server.
create a file with the below contents in any file (say odbc.ini)
[ODBC Data Sources]
my_data_source=tdata.so
[my_data_source]
Driver=/path/to/teradata/drivers/tdata.so
DBCName=<td_hostname>
LastUser=<user_name>
Username=<user_name>
Password=<password>
Database=<default_database>
DefaultDatabase=<default_database>
TDMSTPortNumber=<teradata_port>
set ODBCINI variable to the path of the odbc file
export ODBCINI=/file/to/path/of/odbc.ini
note: you can skip the setting of ODBCINI env variable by creating the odbc.ini file in the home directory i.e. /home/user/.odbc.ini (note that the .odbc.ini is a hidden file with a dot prefix in the file name)
now to connect to Teradata use the below snippet.
import pyodbc
pyodbc.pooling = False
conn = pyodbc.connect('DSN=my_data_source',ansi=True, autocommit=True)

Categories

Resources