I'm trying to connect to Teradata without using Teradata driver. so installed the teradatasql package and written the below code to connect.
import teradatasql
import pandas as pd
with teradatasql.connect(host='abc.abc.net', user='abcabc', password='abce123') as connect;
query = "select * from abc.emp;"
df = pd.read_sql(query, connect)
print (df.head())
when executing the above code I'm getting the Invalid syntax error at the below line
with teradatasql.connect(host='abc.abc.net', user='abcabc', password='abce123') as connect;
^
can you help me with the code to connect to teradata
Thanks in Advance.
Replace the a semicolon (;) with a colon (:)
The correct syntax for with is:
with expression [as variable]:
with-block
In your case:
with teradatasql.connect(host='abc.abc.net', user='abcabc', password='abce123') as connect:
query = ...
Related
I am using python to connect to DB2 Database
I have installed ibm_db and ibm_dbi packages and imported in to the code
import ibm_db
import ibm_db_dbi
1)created a connection string as conn_str
conn_str='database=pydev;hostname=host.test.com;port=portno;protocol=tcpip;uid=db2inst1;pwd=secret'
ibm_db_conn = ibm_db.connect(conn_str,'','')
conn = ibm_db_dbi.Connection(ibm_db_conn)
2)Now i need to read a DB2 table which is in under schemas called as "BRUD" into python pandas
could any one please help me in getting the connection for this
I'm not sure about sql syntax, but resolution looks like:
df = pd.read_sql('SELECT * FROM BRUD.table_name', conn)
I'm trying to upload a dataframe in SQL server using pandas (to_sql) function, I get the below error
[SQL Server Native Client 11.0]Invalid character value for cast
specification (0) (SQLExecDirectW)')
I checked for variables' names and types and they are exactly the same in the SQL database and pandas dataframe.
How can I fix this?
Thanks
df.to_sql(raw_table, connDB, if_exists='append', index=False )
plz try this , this code use to juypter note book and SQL workbench
import mysql.connector
from mysql.connector import Error
from sqlalchemy import create_engine
import pandas as pd
mydata = pd.read_csv("E:\\Hourly_Format\\upload.csv")
engine = create_engine("mysql://root:admin#localhost/pythondb", pool_size=10, max_overflow=20)
mydata.to_sql(name='emp',con=engine,if_exists='append', index=False)
jupyter :-
workbench :-
Using Python: when connecting to SQL Server using pyodbc, everything works fine, but when I switch to sqlalchemy, the connection fails, giving me the error message:
('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
My code:
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=servername;DATABASE=dbname;UID=username;PWD=password')
engine = sqlalchemy.create_engine("mssql+pyodbc://username:password#servername/dbname")
I can't find the error in my code, and don't understand why the first options works, but the second doesn't.
Help is highly appreciated!
Ran into this problem as well, appending a driver query string to the end of my connection path worked:
"mssql+pyodbc://" + uname + ":" + pword + "#" + server + "/" + dbname + "?driver=SQL+Server"
Update (July 2021) – As above, just modernized (Python 3.6+):
f"mssql+pyodbc://{uname}:{pword}#{server}:{port}/{dbname}?driver=ODBC+Driver+17+for+SQL+Server"
Note that driver= must be all lowercase.
It works using pymssql, instead of pyodbc.
Install pymssql using pip, then change your code to:
engine = sqlalchemy.create_engine("mssql+pymssql://username:password#servername/dbname")
Very late, but experienced the same such problem myself recently. Turned out it was a problem with the latest SQLAlchemy version. Had to rollback my version from 1.4.17 to 1.4.12 (unsure of in-between versions, just went with a version I knew worked).
pip install sqlalchemy==1.4.12
I had original poster's problem with a trusted connection to the Microsoft SQL Server database (pandas 1.5.3, SQLAlchemy 2.0.4). Using answers from this question, this did the trick for me:
import sqlalchemy
import pandas as pd
server = "servername"
database = "dbname"
driver = "ODBC+Driver+17+for+SQL+Server"
url = f"mssql+pyodbc://{server}/{database}?trusted_connection=yes&driver={driver}"
engine = sqlalchemy.create_engine(url)
query = """
SELECT [column1]
,[column2] as some_other_name
FROM [server].[dbo].[table]"""
with engine.begin() as conn:
sqla_query = sqlalchemy.text(query)
df = pd.read_sql(sqla_query, conn)
It should be noted that pandas is not yet fully compatible with SQLAlchemy 2.0: https://pandas.pydata.org/docs/whatsnew/v1.5.3.html
I have a need to access data that resides in a remote db2 database via a sql statement and convert it to a Pandas DataFrame. All from my Mac. I looked at using Pandas' read_sql with the ibm_db_sa adapter, but it looks like the prerequisite client side software is not supported on the Mac
I came up with an jdbc option, which I'm posting, but I'm curious to know if anyone else has any ideas
Here's an option using jdbc, the pip installable JayDeBeApi and the appropriate db jar file
Note: this could be used for other jdbc/jaydebeapi compliant databases like Oracle, MS Sql Server, etc
import jaydebeapi
import pandas as pd
def read_jdbc(sql, jclassname, driver_args, jars=None, libs=None):
'''
Reads jdbc compliant data sources and returns a Pandas DataFrame
uses jaydebeapi.connect and doc strings :-)
https://pypi.python.org/pypi/JayDeBeApi/
:param sql: select statement
:param jclassname: Full qualified Java class name of the JDBC driver.
e.g. org.postgresql.Driver or com.ibm.db2.jcc.DB2Driver
:param driver_args: Argument or sequence of arguments to be passed to the
Java DriverManager.getConnection method. Usually the
database URL. See
http://docs.oracle.com/javase/6/docs/api/java/sql/DriverManager.html
for more details
:param jars: Jar filename or sequence of filenames for the JDBC driver
:param libs: Dll/so filenames or sequence of dlls/sos used as
shared library by the JDBC driver
:return: Pandas DataFrame
'''
try:
conn = jaydebeapi.connect(jclassname, driver_args, jars, libs)
except jaydebeapi.DatabaseError as de:
raise
try:
curs = conn.cursor()
curs.execute(sql)
columns = [desc[0] for desc in curs.description] #getting column headers
#convert the list of tuples from fetchall() to a df
return pd.DataFrame(curs.fetchall(), columns=columns)
except jaydebeapi.DatabaseError as de:
raise
finally:
curs.close()
conn.close()
Some examples
#DB2
conn = 'jdbc:db2://<host>:5032/<db>:currentSchema=<schema>;'
class_name = 'com.ibm.db2.jcc.DB2Driver'
sql = 'SELECT name FROM table_name FETCH FIRST 5 ROWS ONLY'
df = read_jdbc(sql, class_name, [conn, 'myname', 'mypwd'])
#PostgreSQL
conn = 'jdbc:postgresql://<host>:5432/<db>?currentSchema=<schema>'
class_name = 'org.postgresql.Driver'
jar = '/path/to/jar/postgresql-9.4.1212.jar'
sql = 'SELECT name FROM table_name LIMIT 5'
df = read_jdbc(sql, class_name, [conn, 'myname', 'mypwd'], jars=jar)
I got a simpler answer from https://stackoverflow.com/a/33805547/914967 where it uses pip module ibm_db only:
import ibm_db
import ibm_db_dbi
import pandas as pd
conn_handle = ibm_db.connect('DATABASE={};HOSTNAME={};PORT={};PROTOCOL=TCPIP;UID={};PWD={};'.format(db_name, hostname, port_number, user, password), '', '')
conn = ibm_db_dbi.Connection(conn_handle)
df = pd.read_sql(sql, conn)
Bob, you should check out ibmdbpy (https://pypi.python.org/pypi/ibmdbpy). It is a pandas data frame style API to DB2 and dashDB tables. It supports both underlying DB2 client drivers, ODBC and JDBC.
So as prerequisites you need to set up the DB2 client driver package for Mac that you can find here: http://www-01.ibm.com/support/docview.wss?uid=swg21385217
After #IanBjorhovde commented on my question I investigated another solution that allows me to use sqlalchemy and pandas' read_sql()
Here are the steps I took. Note: I got this working on OSX Yosemite (10.10.4) for python 3.4 and 3.5
1) Download IBM DB2 Express-C (no-cost community edition of DB2)
https://www-01.ibm.com/marketing/iwm/iwm/web/pick.do?source=swg-db2expressc&S_TACT=000000VR&lang=en_US&S_OFF_CD=10000761
2) After navigating to the unzipped dir
sudo ./db2_install
I accepted the default location of /opt/IBM/db2/V10.1
3) Install ibm_db and ibm_db_sa
pip install ibm_db
I built ibm_db_sa from source because the pip installed failed
python setup.py install
That should do it. You might get an error like 'Reason: image not found' when you try to connect to your db so read this for the fix. Note: might require a reboot
Example usage:
import ibm_db_sa
import pandas as pd
from sqlalchemy import select, create_engine
eng = create_engine('ibm_db_sa://<user_name>:<pwd>#<host>:5032/<db name>')
sql = 'SELECT name FROM table_name FETCH FIRST 5 ROWS ONLY'
df = pd.read_sql(sql, eng)
I've been at this for many hours, and cannot figure out what's wrong with my approach. I'm trying to read a table into pandas using sqlalchemy (from a SQL server 2012 instance) and getting the following error:
DBAPIError: (Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)") None None
I'm using the following code:
import sqlalchemy as sql
from sqlalchemy import create_engine
import pyodbc
pyodbc.connect('DSN=MYDSN;UID=User;PWD=Password')
Which returns:
<pyodbc.Connection at 0x10a26a420>
Which I think is good. Then when I run the following:
connectionString = 'mssql+pyodbc://User:Password#IPAdress/Database'
engine = sql.create_engine(connectionString)
pd.read_sql("ecodata", engine)
I get the following error mentioned above.
Is there something wrong with my driver setup? I've been wrestling with the driver setup for days and thought I had it beat.
Any help is greatly appreciated.
For the record, the answer to my question was figured out by joris:
My syntax for the connection string was wrong, and when I changed it from this
connectionString = 'mssql+pyodbc://User:Password#IPAdress/Database'
to this
connectionString = 'mssql+pyodbc://User:Password#IPAddress:Port/Database?driver=FreeTDS'
It worked!
Thanks again for the help!
Try forming your connection like this. You need a few more parameters.
con = pyodbc.connect('DRIVER={FreeTDS};SERVER="yourserver";PORT=1433;DATABASE=yourdb;UID=youruser;PWD=yourpassword;TDS_Version=7.2;')
To figure out which TDS Version to use:
http://www.freetds.org/userguide/choosingtdsprotocol.htm
Hopefully, this helps!