I have a url I use to connect to Hadoop. I'm prompted for a username and password when entering the url, port etc.
Out of curiosity, is it possible to connect to a remote hive database using python with the set up I have ? Any examples of code ?
Thanks in advance!
No sure this fit your needs, since I don't have the details of your setup, but it is possible to connect to hive remotely with Python using HiverServer2.
To get more details, you may reference this page.
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Overview
Example of python code are provided here:
https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-PythonClientDriver
Hope this help!
Related
I've been trying to use Continuous Query Notification (CQN) in python script to get notification from database about changes that were made to a specific table.
I have followed tutorial from this link here
https://python-oracledb.readthedocs.io/en/latest/user_guide/cqn.html
Connection to oracle database was successful and I can query the table to get results but I can't get any message from callback function that looks like this
def cqn_callback(message):
print("Notification:")
for query in message.queries:
for tab in query.tables:
print("Table:", tab.name)
print("Operation:", tab.operation)
for row in tab.rows:
if row.operation & oracledb.OPCODE_INSERT:
print("INSERT of rowid:", row.rowid)
if row.operation & oracledb.OPCODE_DELETE:
print("DELETE of rowid:", row.rowid)
subscr = connection.subscribe(callback=cqn_callback,
operations=oracledb.OPCODE_INSERT | oracledb.OPCODE_DELETE,
qos=oracledb.SUBSCR_QOS_QUERY | oracledb.SUBSCR_QOS_ROWIDS)
subscr.registerquery("select * from regions")
input("Hit enter to stop CQN demo\n")
I can see that the registration was created in the database after I run the script but I just don't receive any message about insert or delete after I perform any of those operations through SQL* Plus or SQL Developer.
I am reading other questions and blogs about this functionality but currently without success, so if anyone has any recommendations or has encountered similar problem, please comment or answer here.
Oracle database 12C from docker
Python version is 3.10.7
I am running it in thick mode and for oracle client libraries I am using this command
oracledb.init_oracle_client(lib_dir = ".../instantclient_21_3"
P.S This is my first time posting a question here so if I didn't correctly follow a structure or rules of asking a question please correct me, thanks in advance :)
Please take a look at the requirements for CQN in the documentation. Note in particular the fact that the database needs to connect back to the application. If this cannot be done no notifications will take place even though the registration is successful with the database. With Oracle Database 19.4 a new mode was introduced which eliminates this requirement, but since you are still using 12c that won't work for you. You will need to ensure that the database can connect back to the application -- opening up any ports, ensuring that an IP address is directly specified in the parameters or an IP address can be looked up from the name of the client machine connecting to the database, etc.
I'm currently trying to use the dockerpy sdk to connect to my remote ubuntu server so i can manage my docker containers via python.
I am getting a few issues when attempting to do this.
docker.APIClient(base_url="ssh://user#ip")
When doing the following I am getting the error:
paramiko.ssh_exception.PasswordRequiredException: private key file is encrypted
I can resolve this issue by adding the kwarg: use_ssh_client, but then i am forced to input a password, which limits the potential for automation.
docker.APIClient(base_url="ssh://user:#ip", use_ssh_client=True)
When using the above code, I have also tried to enter my ssh key password into the base_url such as:
docker.APIClient(base_url="ssh://user:pass#ip", use_ssh_client=True)
However, this then greets me with the following error:
docker.errors.DockerException: Invalid bind address format: ssh://root:pass#ip
I have run out of ideas and am confused as to how I am supposed to get around this?
Many thanks in advance...
It's possible to make a connection as Mr. Piere answered here. Even with that question about docker.client.DockerClient which uses docker.api.client.APIClient under the hood.
You are trying to establish a connection using Password Authentication that's why you asked to prompt a password.
I guess you need to configure the Key-Based SSH Login as said in docker's docs
Steps to fix:
configure SSH Login on a remote server and fill ~/.ssh/config on your local machine
connect from the local terminal using the ssh command to ensure a connection is established without asking password ssh user#ip
connect using library client = docker. APIClient(base_url="ssh://user#ip", use_ssh_client=True)
I had a similar Problem. Your Problem is, that you Key is encrypted. The Docker Clients doesn't have a Passphrase option by default. I wrote some Code based on this Post. It works for me :)
import os
from docker import APIClient
from docker.transport import SSHHTTPAdapter
class MySSHHTTPAdapter(SSHHTTPAdapter):
def _connect(self):
if self.ssh_client:
self.ssh_params["key_filename"] = os.environ.get("SSH_KEY_FILENAME")
self.ssh_params["passphrase"] = os.environ.get("SSH_PASSPHRASE")
self.ssh_client.connect(**self.ssh_params)
client = APIClient('ssh://ip:22', use_ssh_client=True, version='1.41')
ssh_adapter = MySSHHTTPAdapter('ssh://user#ip:22')
client.mount('http+docker://ssh', ssh_adapter)
print(client.version())
Anyone use the Python Connector to connect to Snowflake, specifically using https://<your_okta_account_name>.okta.com (i.e. the URL endpoint for Okta) to authenticate through native Okta.? I keep getting the error below, but I know it is the correct password/username. I am able to login fine via 'external browser" (but not through cloud9, nor do I want to continue this way)
Failed to connect to DB: appfolio.snowflakecomputing.com:443. Incorrect username or password was specified.
I have the following variables set up and I know I have correct password/username so going a little crazy , anyone here have this issue?
authenticator = 'https://company.okta.com'
user='user.name#company.com',
password= "password",
account='company'
People are having the same error (with & without MFA) but no solution was posted on snowflakes community board: https://community.snowflake.com/s/question/0D50Z00009JJKZGSA5/unable-to-connect-to-snowflake-using-aws-lambda
Hi all I need to write a custom script in Python which will fetch data from one server and insert to another physical server even DB may be different like one will be Netezza other one will be Postgres/VoltDB .Everything is working fine except the VoltDB part.I am not able to find the driver for VoltDB to use with Python .I am using pyodbc for Netezza and Native Python library for Postgres .Can anyone help me to sort out things .VoltDB official site saying about Fastserializer I didn't understand the same
Thanks
Anoop
Here is one:
https://github.com/VoltDB/voltdb-client-python
It does not need compilation.
I am using Tally.ERP9, I haven't found anything about Tally ODBC connection on net. I want a python script to connect with Tally database. Can I get some links for that and some python scripts to connect tally database using Tally ODBC?
Since it supports ODBC, you might try pyodbc: http://pyodbc.sourceforge.net/
I am not aware of direct connection engine is available or not but you can have a look at this post may it will help
Specification for Tally SOAP API