I have a R code which connect to the Vertica database using RJDBC driver. The code is following:
library(RJDBC)
#for vertica save
user = "myuser"
pwd = "mypwd"
driver<- JDBC(driverClass="com.vertica.jdbc.Driver", classPath=Pathto thedriver")
connection<- dbConnect(driver, databasewithport, user, pwd)
sql_code = paste("SELECT .....")
mydata= dbGetQuery(connection, sql_code )
I am searching for a solution that helps do the same thing but using Python. I found the following link, but do not understand which example to use and what else to do. As I understood here no need to connect to the RJDBC driver. Could you help to find the solution which gives the same output as R version.
The code below works well, however, data is retrieved as one value, to get another I need to change ....cur.fetchone()[ANYNUMBER]). How can I get a data frame of the SQL code?
import vertica_python
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'vdb',
'connection_load_balance': True}
# Server enables load balancing
with vertica_python.connect(**conn_info) as conn:
cur = conn.cursor()
cur.execute("SELECT NODE_NAME FROM V_MONITOR.CURRENT_SESSION")
print("Client connects to primary node:", cur.fetchone()[0])
cur.execute("SELECT SET_LOAD_BALANCE_POLICY('ROUNDROBIN')")
First of all you will need to install the vertica-python package:
pip install vertica-python
Next, you need to create a connection and perform the query. When retrieving the query results, you can (1) load them all or (2) process them one by one.
import vertica_python
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'myuser',
'password': 'mypass',
'database': 'vdb',
'connection_load_balance': True}
with vertica_python.connect(**conn_info) as connection:
cur = conn.cursor()
cur.execute("SELECT NODE_NAME FROM V_MONITOR.CURRENT_SESSION")
# (1) If you want to load all the results in-memory
data = cur.fetchall()
print(data)
# (2) If you want to process one by one
for row in cur.iterate():
print(row)
Related
I use cx_Oracle module to connect to standalone Oracle server as follows
import cx_Oracle
CONN_INFO = {
'host': 'xxx.xx.xxx.x',
'port': 12345,
'user': 'user_name',
'psw': 'your_password',
'service': 'abc.xyz.com',
}
CONN_STR = '{user}/{psw}#{host}:{port}/{service}'.format(**CONN_INFO)
connection = cx_Oracle.connect(CONN_STR)
but as scan IP doesn not have machine and its own username passoword, How do we connect?
Es described in the documentation, you can simple use the name defined in tnsnames.ora.
Say your RAC tnsnames entry is called MAXIMIR than you can connect with
con = cx_Oracle.connect("my_usr", "my_pwd", "MAXIMIR", encoding="UTF-8")
alternatively you may pass the whole connection string in a dns variable
dsn = """(DESCRIPTION=
(FAILOVER=on)
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=tcp)(HOST=scan1)(PORT=1521))
(ADDRESS=(PROTOCOL=tcp)(HOST=scan2)(PORT=1521)))
(CONNECT_DATA=(SERVICE_NAME=MAXIMIR)))"""
connection = cx_Oracle.connect("my_usr", "my_pwd", dsn, encoding="UTF-8")
I have collection of objects and each object I would like to populate with some data from MySQL (could be SQL Server). I would like to do in parallel fashion. When establishing connection to MySQL I have two items
connection
cursor
Hence, the question, when making calls in parallel, should I pass
connection only
cursor only
connection and cursor only
do not pass anything and let each object's method establish connection on its own.
---------------code formatting does not work without this line -------
import mysql.connector
my_collection = [My_Object(), My_Object()]
config = {'user': 'someuser', 'password': 'somepassword', 'host': 'localhost', 'port': '3306',
'database': 'somedb','raise_on_warnings': True}
connection = mysql.connector.connect(**config)
cursor = connection.cursor(dictionary=True)
#some parallel loop
for el in my_collection:
el.pull_data_from(WHAT TO PASS???)
I have an XAMPP DB I am trying to connect to with mysql.connector on a Red Hat Linux server. The issue is the DB is only accessible from /opt/lampp/bin/mysql. I cannot find a way to specify path with the mysql.connector module.
My code is:
import mysql.connector
config = {
'user': 'user',
'password': '*****',
'host': '127.0.0.1',
'database': 'test',
'raise_on_warnings': True,
}
cnx = mysql.connector.connect(**config)
cursor = cnx.cursor()
query = "show columns from Table1"
cursor.execute(query)
lst = []
for line in cursor:
lst.append(line[0])
query2 = "select * from Table1 limit 5"
lst2 = []
cursor.execute(query2)
for line in cursor:
lst2.append(dict(zip(lst, line)))
print(lst2)
cnx.close()
Right now I am getting an error when running it mysql.connector.errors.InterfaceError: Failed parsing handshake; end byte not present in buffer but I am assuming that it is probably because I am not specifying path to database.
Thanks
I am finding evidence which suggests that this problem is related to MySQL version 5.5.8. What version of MySQL are you running?
I have a code that connect to oracle using connection string:
conn = cx_Oracle.connect('username/password#server:port/services')
But the problem is my password contain # character so it may become
conn = cx_Oracle.connect('username/p#ssword#server:port/services')
it return
DatabaseError: ORA-12154: TNS:could not resolve the connect identifier
specified
I use Django with Oracle with this settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'Services',
'USER': 'user',
'PASSWORD': 'p#ssword',
'HOST': 'ip',
'PORT': 'port',
}
}
I cant change password :( Does anyone know this problem?
I haven't tried cx_Oracle, but you might be able to connect by specifying the individual parameters -
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn='server:port/services')
OR
dsn_tns = cx_Oracle.makedsn('server', 'port', 'services')
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn=dsn_tns)
You can use any of the following way based on Service Name or SID whatever you have.
With SID:
dsn_tns = cx_Oracle.makedsn('server', 'port', 'sid')
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn=dsn_tns)
OR
With Service Name:
dsn_tns = cx_Oracle.makedsn('server', 'port', service_name='service_name')
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn=dsn_tns)
Does this work?
conn = cx_Oracle.connect('username/"p#ssword"#server:port/services')
FYI: This was a long-standing bug in Django. The first stable version containing the fix is v2.1
All the answers mentioned here did not work for SQLAlchemy.
Parsing the characters to create a valid URL worked:
from sqlalchemy.engine import create_engine
import urllib.parse
url_password = urllib.parse.quote_plus(password)
connection_string = f"oracle+cx_oracle://{username}:{url_password}#{server}:{port}/?service_name={service}"
connection = create_engine(connection_string)
Should probably do the same with the other components of the connection string.
I'm getting a "decryption failed or bad record mac" error in this code-fragment:
conn = psycopg2.connect(...)
cursor = conn.cursor()
cursor.execute("SELECT id, ip FROM schema.table;")
rows = cursor.fetchall()
cursor.close()
conn.commit()
conn.close()
This is called in the run() method of a Thread, several times in a while(True) loop.
I'm just opening a connection to my PostgreSQL database using the psycopg2 driver.
Any idea of how safe is opening db connections into Threads in Python?
I don't know what is raising this error.
Ok, looks like I've fixed the problem. I was creating too many connections and seems I was running out of memory or something.
I gathered all the queries and do cursor.execute(...) once with a huge query, instead performing hundreds of small queries/connections.
conn = psycopg2.connect(...)
cursor = conn.cursor()
cursor.execute("SELECT id, ip FROM schema.table;")
rows = cursor.fetchall()
cursor.close()
conn.commit()
conn.close()
conn = None
The cause of this issue could be, there were too many processes(multi) were trying to access PostGres, it was not able to handle that. I was using Django & PostGres in BeanStalk.
Adding 'OPTIONS': {'sslmode': 'disable'} in the database config helped.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': ...
'USER': ....
'PASSWORD': ...
'HOST': ....
'PORT': '5432',
'OPTIONS': {
'sslmode': 'disable',
}
}
}