Develop Reference

Using Python to connect to Impala database (thriftpy error)

What I'm trying to do is very basic: connect to an Impala db using Python:
from impala.dbapi import connect
conn = connect(host='impala', port=21050, auth_mechanism='PLAIN')
I'm using Impyla package to do so. I got this error:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/thriftpy/transport/socket.py", line 96, in open
self.sock.connect(addr)
socket.gaierror: [Errno -3] Temporary failure in name resolution
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/alaaeddine/PycharmProjects/test/data_test.py", line 3, in <module>
conn = connect(host='impala', port=21050, auth_mechanism='PLAIN')
File "/usr/local/lib/python3.6/dist-packages/impala/dbapi.py", line 147, in connect
auth_mechanism=auth_mechanism)
File "/usr/local/lib/python3.6/dist-packages/impala/hiveserver2.py", line 758, in connect
transport.open()
File "/usr/local/lib/python3.6/dist-packages/thrift_sasl/__init__.py", line 61, in open
self._trans.open()
File "/usr/local/lib/python3.6/dist-packages/thriftpy/transport/socket.py", line 104, in open
message="Could not connect to %s" % str(addr))
thriftpy.transport.TTransportException: TTransportException(type=1, message="Could not connect to ('impala', 21050)")
Tried also the Ibis package but failed with the same thriftpy related error.
In Windows using Dbeaver, I could connect to the database using the official Cloudera JDBC connector. My questions are:
Should pass my JDBC connector as parameter in my connect code? I have made some search I could not find something pointing at this direction.
Should I try something else than Ibis and Impyla packages? I had experienced a lot of version related issues and dependencies when using them. If yes, what would you recommend as alternatives?
Thanks!

Solved:
I used pyhive package instead of Ibis/Impyla. Here's an example:
#import hive from pyhive
from pyhive import hive
#establish the connection to the db
conn = hive.Connection(host='host_IP_addr', port='conn_port', auth='auth_type', database='my_db')
#prepare the cursor for the queries
cursor = conn.cursor()
#execute a query
cursor.execute("SHOW TABLES")
#navigate and display the results
for table in cursor.fetchall():
print(table)

Your impala domain name must not be resolving. Are you able to do nslookup impala in command prompt? If you're using Docker, you need to have the docker service name in docker-compose as "impala" or have "extra_hosts" option. Or you can always add it to /etc/hosts (Windows/Drivers/etc/hosts) as impala 127.0.0.1
Also try 'NOSASL' instead of PLAIN sometimes that works better with security turned off.

This is the simple method, connecting impala through impala shell using python.
import commands
import re
query1 = "select * from table_name limit 10"
impalad = str('hostname')
port = str('21000')
database = str('database_name')
result_string = 'impala-shell -i "'+ impalad+':'+port +'" -k -B --delimited -q "'+query1+'"'
status, output = commands.getstatusoutput(result_string)
print output
if status == 0:
print output
else:
print "Error encountered while executing HiveQL queries."

Unable to connect to server with python and pymssql

I'm developing a script that it's supossed to read data from a Microsoft SQL database and display it in a nice format. Also, It's supossed to write into the database as well. The issue is that I'm not able to connect to the server.
I'm using this code:
import pymssql
server = "serverIpAddress"
user = "username"
password = "pass"
db = "databaseName"
port = 1433
db = pymssql.connect(server,user,password,port= port)
# prepare a cursor object using cursor() method
cursor = db.cursor()
# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")
# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print "Database version : %s " % data
# disconnect from server
db.close()
And I'm getting this traceback:
Traceback (most recent call last):
File ".\dbtest.py", line 9, in <module>
db = pymssql.connect(server,user,password,port= port)
File "pymssql.pyx", line 641, in pymssql.connect (pymssql.c:10824)
pymssql.OperationalError: (18452, 'Login failed. The login is from an untrusted domain and cannot be used with Windows authentication.DB-Lib error message 20018, severity 14:\nGeneral SQL Server e
rror: Check messages from the SQL Server\nDB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (serverip:1433)\n')
I've changed some data to keep privacy.
This give me some clues about what it's going on:
The login is from an untrusted domain and cannot be used with Windows authentication
But I don't know how to fix it. I've seen that some people uses
integratedSecurity=true
But I don't know if there is something like this on pymssql or even if that it's a good idea.
Also, I don't need to use pymssql at all. If you know any other library that can perform what I need, I don't mind changing it.
Thanks and greetings.
--EDIT--
I've also tested this code:
import pyodbc
server = "serverIpAddress"
user = "username"
password = "pass"
db = "databaseName"
connectString = "Driver={SQL Server};server="+serverIP+";database="+db+";uid="+user+";pwd="+password
con = pyodbc.connect(connectString)
cur = con.cursor()
cur.close()
con.close()
and I'm getting this traceback:
Traceback (most recent call last):
  File ".\pyodbc_test.py", line 9, in <module>
    con = pyodbc.connect(connectString)
pyodbc.Error: ('28000', "[28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user '.\\sa'. (18456) (SQLDriverConnect); [28000] [Microsoft][ODBC SQL Server Driver][SQL Server]Lo
gin failed for user '.\\sa'. (18456)")

Python pyodbc cursor execution fails on Teradata

I have a Python script which runs successfully from my Windows workstation and I am trying to migrate it to a Unix server. The script connects to a Teradata database using pyodbc package and executes a bunch of queries. When it is execute from the server, it triggers the following error message:
Error: ('HY000', 'The driver did not supply an error!')
I am able to consistently reproduce the error with the following code snippet executed on the server:
import pyodbc
oConnexion = pyodbc.connect("Driver={Teradata};DBCNAME=myserver;UID=myuser;PWD=mypassword", autocommit=True)
print("Connected")
oCursor = oConnexion.cursor()
oCursor.execute("select 1")
print("Success")
Configuration:
Python 3.5.2
Pyodbc 3.1.2b2
UnixODBC Driver Manager
Teradata 15.10
After enabling ODBC logging and running a simple SELECT query, I have noticed the following Invalid cursor GeTypeInfo errors:
Data Type = SQL_VARCHAR
[ODBC][57920][1481847636.278776][SQLGetTypeInfo.c][190]Error: 24000
[ODBC][57920][1481847636.278815][SQLGetTypeInfo.c][168]
Entry:
Statement = 0x1bc69e0
Data Type = Unknown(-9)
[ODBC][57920][1481847636.278839][SQLGetTypeInfo.c][190]Error: 24000
[ODBC][57920][1481847636.278873][SQLGetTypeInfo.c][168]
Entry:
Statement = 0x1bc69e0
Data Type = SQL_BINARY
[ODBC][57920][1481847636.278896][SQLGetTypeInfo.c][190]Error: 24000
Also, trying to list the connection attributes using the following code:
for attr in vars(pyodbc):
print (attr)
value = oConnexion.getinfo(getattr(pyodbc, attr))
print('{:<40s} | {}'.format(attr, value))
Fails with:
SQL_DESCRIBE_PARAMETER
Traceback (most recent call last):
File "test.py", line 28, in <module>
value = oConnexion.getinfo(getattr(pyodbc, attr))
pyodbc.Error: ('IM001', '[IM001] [unixODBC][Driver Manager]Driver does not support this function (0) (SQLGetInfo)')

Upgrading to the last (unreleased) version of pyodbc (v4) solved the issue.
https://github.com/mkleehammer/pyodbc/tree/v4

Mysql OperationalError 1044, connect to mysql database

f_dbI am trying to connect to mysql database using a python script. My code to do so is the following:
`db = MySQLdb.connect(host="xxx.xx.xx.xx", # your host, usually localhost
port = xxxx,
user="chrathan", # your username
passwd="...", # your password
db="f_db") # name of the data base`
I ve open the database from mysql workbench. However, I am not able to open it from python I am getting the following error:
_mysql_exceptions.OperationalError: (1044, "Access denied for user 'chrathan'#'%' to database 'f_db'"). Basically I am receiving `Traceback (most recent call last):
File "D:\_WORKSPACE\mysql\fashion_db.py", line 7, in <module>
db = 'f_db') mysql_exceptions.OperationalError: (1044, "Access denied for user 'chrathan'#'%' to database 'f_db'")`

enter this command on mysql terminal
$> mysql
GRANT ALL PRIVILEGES ON f_db.* TO 'chrathan'#'%' identified by 'secret'
for add user to database in mysqlworkbench look at this page How to Create a MySQL Database with MySQL Workbench

maybe you should check db name whether is same, the same problem occour on me, finally i found mysqlclient db name is not same as i use in python

Cannot establish connection to sql-server using pyodbc on Windows 7

I'm using ActivePython 2.7.2.5 on Windows 7.
While trying to connect to a sql-server database with the pyodbc module using the below code, I receive the subsequent Traceback. Any ideas on what I'm doing wrong?
CODE:
import pyodbc
driver = 'SQL Server'
server = '**server-name**'
db1 = 'CorpApps'
tcon = 'yes'
uname = 'jnichol3'
pword = '**my-password**'
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=server;DATABASE=db1;UID=uname;PWD=pword;Trusted_Connection=yes')
cursor = cnxn.cursor()
cursor.execute("select * from appaudit_q32013")
rows = cursor.fetchall()
for row in rows:
print row
TRACEBACK:
Traceback (most recent call last):
File "pyodbc_test.py", line 9, in <module>
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=server;DATABASE=db1;UID=uname;PWD=pword;Trusted_Connection=yes')
pyodbc.Error: ('08001', '[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect); [01000] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()). (53)')

You're using a connection string of 'DRIVER={SQL Server};SERVER=server;DATABASE=db1;UID=uname;PWD=pword;Trusted_Connection=yes', you're trying to connect to a server called server, a database called db1, etc. It doesn't use the variables you set before, they're not used.
It's possible to pass the connection string parameters as keyword arguments to the connect function, so you could use:
cnxn = pyodbc.connect(driver='{SQL Server}', host=server, database=db1,
trusted_connection=tcon, user=uname, password=pword)

I had the same error message and in my case the issue was the [SQL Server] drivers required TLS 1.0 which is disabled on my server. Changing to the newer version of the SNAC, SQL Server Native Client 11.0 fixed the problem.
So my connection string looks like:
cnxn = pyodbc.connect(driver='{SQL Server Native Client 11.0}',
host=server, database=db1, trusted_connection=tcon,
user=uname, password=pword)

I had faced this error due to another reason.
It was because my server had a "port" apart from the address.
I could fix that by assigning the following value to "Server" parameter of the connection string.
"...;Server=<server_name>,<port#>;..."
Note that it is a 'comma' and not 'colon'/'period'

cnxn = pyodbc.connect(driver='{SQL Server}', host=server, database=db1,
user=uname, password=pword)
print(cnxn)
I removed "Trusted_Connection" part and it worked for me.

Different security risks exist with either method. If you use Sql Server authentication you expose your userid/password in the code. But at least you process with the same credentials. If you use Windows authentication you have to insure all the possible users are setup with the right permission in the Sql server. With Sql authentication you can setup just one user but multiple people can use that one Sql User permissions wise.

I had the same issue today. I was using localhost in the connectionstring. Got rid of the issue by replacing localhost woth 'server name',. My db and application are running in the same machine.
If you don't have server name
go to Sql server management studio and execute below query, which will give you the server name.
SELECT ##SERVERNAME
The connection string look as below
conn = pyodbc.connect('Driver={SQL Server};'
'Server=myServerName;'
'Database=mydb;'
'Trusted_Connection=yes;')