Develop Reference

connecting teradata from python without terada driver

I'm trying to connect to Teradata without using Teradata driver. so installed the teradatasql package and written the below code to connect.
import teradatasql
import pandas as pd
with teradatasql.connect(host='abc.abc.net', user='abcabc', password='abce123') as connect;
query = "select * from abc.emp;"
df = pd.read_sql(query, connect)
print (df.head())
when executing the above code I'm getting the Invalid syntax error at the below line
with teradatasql.connect(host='abc.abc.net', user='abcabc', password='abce123') as connect;
^
can you help me with the code to connect to teradata
Thanks in Advance.

Replace the a semicolon (;) with a colon (:)
The correct syntax for with is:
with expression [as variable]:
with-block
In your case:
with teradatasql.connect(host='abc.abc.net', user='abcabc', password='abce123') as connect:
query = ...

Dremio ODBC with Python

Getting below error while running this code in Python, If anyone could advise me on this that would be appreciated. Thanks
dataframe = pandas.read_sql(sql,cnxn)
DatabaseError: Execution failed on sql 'SELECT * FROM train_data': ('HY000', "[HY000] [Dremio][Connector] (1040) Dremio failed to execute the query: SELECT * FROM train_data\n[30038]Query execution error. Details:[ \nVALIDATION ERROR: Table 'train_data' not found\n\nSQL Query SELECT * FROM train_data\nstartLine 1\nstartColumn 15\nendLine 1\nendColumn 24\n\n[Error Id: 24c7de0e-6e23-44c6-8cb6-b0a110bbd2fd on user:31010]\n\n (org.apache.calcite.runtime.CalciteContextException) From line 1, column 15 to line 1, column 24: ...[see log] (1040) (SQLExecDirectW)")

You only need to provide your Space name, just before your table name.
for example:
SELECT * FROM
SpaceName.train_data
This is a query to fetch data from Dremio Space, Dremio source cannot be used for data ingestion. Dremio Source only be used to establish a connection between database and Dremio.

this is being solved, it says that table does not exist, should give a valid table, in dremio it can be inside a specific space

Error when using fast_executemany = True in sql alchemy with driver='SQL+Server'

I am trying to use fast_executemany to speed up my df.to_sql insert.
I read the documentation and added it to my code like this:
import pandas as pd
import sqlalchemy
import numpy as np
import random
#connect to database
server = 'Test'
database = 'Test'
driver = 'SQL+Server'
driver1 = 'ODBC+Driver+13+for+SQL+Server'
engine_stmt = ("mssql+pyodbc://#%s/%s?driver=%s" % (server, database, driver))
engine = sqlalchemy.create_engine(engine_stmt, fast_executemany=True)
connection = engine.connect()
When I run this code without the fast_executemany it works, but it takes fairly long for the insert.
Therefore I wanted to use the command, but I get an error when using it with the 'SQL+Server' driver. Therefore I tried to change the driver according to the documentation to 'ODBC+Driver+13+for+SQL+Server' I get the following error:
def create_connect_args(self, url):
InterfaceError: (pyodbc.InterfaceError) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
So I guess this driver is not working for me? I tested a few different ones, but the only one that is working is the 'SQL+Server'

It looks like you're missing the ODBC 13 driver on your computer.
Try installing that and run your script again.
Alternatively, try swapping:
driver1 = 'ODBC+Driver+13+for+SQL+Server'
for
driver1 = 'ODBC+Driver+17+for+SQL+Server'

Reading table into pandas using sqlalchemy from SQL Server

I've been at this for many hours, and cannot figure out what's wrong with my approach. I'm trying to read a table into pandas using sqlalchemy (from a SQL server 2012 instance) and getting the following error:
DBAPIError: (Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)") None None
I'm using the following code:
import sqlalchemy as sql
from sqlalchemy import create_engine
import pyodbc
pyodbc.connect('DSN=MYDSN;UID=User;PWD=Password')
Which returns:
<pyodbc.Connection at 0x10a26a420>
Which I think is good. Then when I run the following:
connectionString = 'mssql+pyodbc://User:Password#IPAdress/Database'
engine = sql.create_engine(connectionString)
pd.read_sql("ecodata", engine)
I get the following error mentioned above.
Is there something wrong with my driver setup? I've been wrestling with the driver setup for days and thought I had it beat.
Any help is greatly appreciated.

For the record, the answer to my question was figured out by joris:
My syntax for the connection string was wrong, and when I changed it from this
connectionString = 'mssql+pyodbc://User:Password#IPAdress/Database'
to this
connectionString = 'mssql+pyodbc://User:Password#IPAddress:Port/Database?driver=FreeTDS'
It worked!
Thanks again for the help!

Try forming your connection like this. You need a few more parameters.
con = pyodbc.connect('DRIVER={FreeTDS};SERVER="yourserver";PORT=1433;DATABASE=yourdb;UID=youruser;PWD=yourpassword;TDS_Version=7.2;')
To figure out which TDS Version to use:
http://www.freetds.org/userguide/choosingtdsprotocol.htm
Hopefully, this helps!

Does SQLAlchemy support H2DB?

Does SQLAlchemy support H2 db? I'm using pyramid and would like to connect to H2 db database. If using postgres dialect, I'm getting error like the following:
File "/Users/homecamera/gotocamera/hcadmin/env/lib/python2.7/site-packages/SQLAlchemy-0.7.3-py2.7-macosx-10.4-x86_64.egg/sqlalchemy/dialects/postgresql/base.py", line 871, in initialize
super(PGDialect, self).initialize(connection)
File "/Users/homecamera/gotocamera/hcadmin/env/lib/python2.7/site-packages/SQLAlchemy-0.7.3-py2.7-macosx-10.4-x86_64.egg/sqlalchemy/engine/default.py", line 181, in initialize
self.get_isolation_level(connection.connection)
File "/Users/homecamera/gotocamera/hcadmin/env/lib/python2.7/site-packages/SQLAlchemy-0.7.3-py2.7-macosx-10.4-x86_64.egg/sqlalchemy/dialects/postgresql/base.py", line 910, in get_isolation_level
cursor.execute('show transaction isolation level')
ProgrammingError: Syntax error in SQL statement "SELECT "; expected "TOP, LIMIT, DISTINCT, ALL, *, NOT, EXISTS"; SQL statement:
show transaction isolation level [42001-140]
DETAIL: org.h2.jdbc.JdbcSQLException: Syntax error in SQL statement "SELECT "; expected "TOP, LIMIT, DISTINCT, ALL, *, NOT, EXISTS"; SQL statement:
show transaction isolation level [42001-140]

AFAIK there's no official support for either HSQLDB dialects or native H2 dialect.
Using Postgres dialect with H2 (without using HSQLDB) would definitely result in error you're getting.
You might have better luck trying sqlalchemy-jython and using H2 dialect.

Just in case anyone stumbles over this again, I tried to get this running (as an alternative to sqlite) and it only partially works, and the only driver that works is pg8000.
With the server running using:
nohup java -cp /opt/h2/bin/h2*.jar org.h2.tools.Server -pg -pgAllowOthers -pgPort 5435 -baseDir /opt/h2-data &
This code works in sqlalchemy:
from sqlalchemy import create_engine
engine = create_engine('postgresql+pg8000://sa:sa#localhost:5435/main')
engine.execute("SELECT 1")
However this code throws an exception:
from sqlalchemy_utils import create_database
create_database('postgresql+pg8000://sa:sa#localhost:5435/main')
Exception:
sqlalchemy.exc.ProgrammingError: (pg8000.core.ProgrammingError) ('ERROR',
'HY000', 'General error: "java.lang.IllegalStateException: output binary
format is undefined" [50000-196]', 'org.h2.jdbc.JdbcSQLException: General
error: "java.lang.IllegalStateException: output binary format is undefined"
[50000-196]') [SQL: 'select version()']