I have a trouble connecting to hive running on remote server through my python script.
I'm using the same script (With different server details, of course) to connect to hive running on my localhost & am able to connect.
I'm starting the server on local host from command line with a command:
hive —service hiveserver2
That start the server and I run the python script
Script to connect to Hive running on local host:
import pyhs2
conn = pyhs2.connect(host='localhost', port=10000, authMechanism='PLAIN', user='hive', password ='', database='default')
with conn.cursor() as cur:
cur.execute("show databases")
for i in cur.fetch():
print i
Using above code, am able to access db # Hive on local host.
I'm using below code to connect to remote server, here I'm not doing anything on command line to start the remote server.
Script to connect to Hive running on the remote server:
conn = pyhs2.connect(host='<my remote server Ip>', port=<port no>, authMechanism='PLAIN', user='<usernameToConnectToRemoteServer>', password ="<remoteServerPassword>" database='default')
with conn.cursor() as cur:
cur.execute("show databases")
for i in cur.fetch():
print i
and this returns me a message:
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes.
I've tried to google & find the solution, as much as i can, but all I see are the examples to connect to local host. please help me resolve this.
Try doing SSH to your remote machine and then connect to hive like below-
import paramiko
import traceback
def hive_query_executor():
dns_name = ''
conn_obj = paramiko.SSHClient()
conn_obj.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
conn_obj.connect(dns_name, username="hadoop",
key_filename='')# or password
Hive_query="select * from abc limit 10;"
query_execute_command = 'ihive -e "' + impala_query + '"'
std_in, std_out, std_err = conn_obj.exec_command(query_execute_command)
conn_obj.close()
except:
print "Error :" + str(traceback.format_exc())
exit(0)
hive_query_executor()
Related
I am writing an online application. Rented hosting on TimeWeb, created a database on the hosting. I successfully connected to it using Putty, but the problem is that I need to connect using pycharm. Don't be surprised that I'm making an ssh connection here, and not in the ssh terminal of pycharm, I'm just planning to create an .exe file.I tried to specify port 3306 and utf8 encoding, but it still didn't help. So what's my mistake?
import paramiko
import pymysql
host = "x.x.x.x"
port = 22
username ="cv86943"
password = "xxxxxx"
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(host,port,username,password)
print("1") #1 writes, so the connection to the hosting is successful
conn = pymysql.connect(host = "localhost",user = "cv86943_letter", password = "12345678", db = "cv86943_letter")
print("2") #But 2 does not write anymore
ssh.close()
UPD: returns an error
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'localhost' ([WinError 10061]...)
I have a web app running on AWS Ubuntu and I am unable to connect to a database via ssh tunneling. I am able to access the db via MySQL workbench's SSH tunneling option but access via SSH tunneling on Python fails to work. Kindly suggest ways to solve it or of accessing a Phpmyadmin MySQL database other than via SSH tunneling.
from os.path import expanduser
pkeyfilepath = '/html/fbads/phpmyadmin-ec2.pem'
home = expanduser('~')
print(pkeyfilepath)
print("hello")
print(home)
mypkey = paramiko.RSAKey.from_private_key_file(home + pkeyfilepath)
print(mypkey)
server = SSHTunnelForwarder(
('ec2-52-91-169-11.compute-1.amazonaws.com', 22),
ssh_username="ec2-user",
ssh_pkey="~/html/fbads/phpmyadmin-ec2.pem",
remote_bind_address=('phpmyadmin.cpsisadn02rn.us-east-1.rds.amazonaws.com', 3306)
)
server.start()
print(server)
local_port = str(server.local_bind_port)
engine = create_engine('mysql://{}:{}#{}:{}/{}'.format("root", "passwd", "127.0.0.1", local_port, "passwd"))
dataDF = pd.read_sql("SELECT * FROM test;", engine) #testing teh connection
#server.stop() #use this command to stop the connection
I am new to Python and I am trying to make a script that connects to a remote windows machine and change mdb file there. I can’t figure out how to work with the WMI library.
When I need to change mdb file (Access database) on my machine, I just use pyodbc connect. For example like this:
connStr = pyodbc.connect(r'DRIVER={Microsoft Access Driver (*.mdb)};'
r'DBQ=C:\Komax\Data\Topwin\DatabaseServer.mdb;')
cursor = connStr.cursor()
But now I need to run code on my machine so that it modifies the file (mdb) on the other machine. It seems like I have to use this code below to connect to remote machine, but I don't know what to do next, how to use connection. Please, help me to solve the problem :)
ip = "192.168.18.74"
username = "admin"
password = "komax"
from socket import *
try:
print("Establishing connection to %s" % ip)
connection = wmi.WMI(ip, user=username, password=password)
print("Connection established")
except wmi.x_wmi:
print("Your Username and Password of "+getfqdn(ip)+" are wrong.")
I have a database available on a remote host. When I use putty and SSH, I am able to access it. The database itself has no password. Only, the SSH connection has a password. But, when I try to connect to it using Python, it asks for password. I'm new to postgres and paramiko.
Here's what I've tried:
import psycopg2
import paramiko
import time
t = paramiko.Transport(('xxx.com', 22))
t.connect(username="xxx", password='xxx')
c = paramiko.Channel(t)
conn = psycopg2.connect("dbname='xxx'")
curs = conn.cursor()
sql = "select * from xxx"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
Other method that I tried was:
import os, psycopg2
os.system("ssh xxx#xxx.com -fNL 5432:localhost:5432 -p 22")
while True:
try:
conn = psycopg2.connect("dbname='xxx'")
curs = conn.cursor()
sql = "select * from xxx"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
except:
print "I am unable to connect to the database"
This gives me a 'Could not request local forwarding' error.
Is there some other way to go about this?
I have a Windows 7 (x64) machine with Python 2.7. Please help me. Thanks.
You should connect to the remote server and make port forwarding of remote PostgreSQL to a local port.
Without paramiko, it's something like this:
# start port forwarding
$ ssh -L PGSQL_LOCAL_PORT:localhost:PGSQL_REMOTE_PORT user#xxx.com
# in python
psycopg.connect("dbname='xxx' host='localhost' port='PGSQL_LOCAL_PORT'")
Here is an example of doing this with paramiko
https://code.ros.org/trac/wg-ros-pkg/browser/pkg/trunk/paramiko/demos/forward.py?rev=30
Note: Port forwarding is blocking operation. So, you have to start port forwarding in separate thread/process.
Hi i have a requirement where i need to connect to remote mysql server. My application shall be running on local machine and my mysql will be running on remote server.I have tried the following code:
DB = 'gts'
DB_HOST = 'ps95074.dreamhost.com'
DB_USER = 'root'
DB_PASSWORD = 'dbadminpassword'
conn = MySQLdb.Connection(db=DB, host=DB_HOST, user=DB_USER,passwd=DB_PASSWORD)
cursor = conn.cursor()
But i am getting the following error
OperationalError: (2005, "Unknown MySQL server host 'ps95074.dreamhost.com' (1)")
Instead if i use
DB_HOST='localhost'
Everything works fine. How can same be possible with remote host.Any help shall be appreciated.
Check your firewall. That server is online and available from any machines:
> mysql -h ps95074.dreamhost.com
ERROR 1045 (28000): Access denied for user 'myuser'#'myhost' (using password: NO)
However, even if you can connect chances are good that your database user only allows local connections.
Update: I just tried it again and now it also fails using the commandline client. So clearly something is wrong with your server.