Develop Reference

Python alternative to R RJDBC connection to SQL

I have a R code which connect to the Vertica database using RJDBC driver. The code is following:
library(RJDBC)
#for vertica save
user = "myuser"
pwd = "mypwd"
driver<- JDBC(driverClass="com.vertica.jdbc.Driver", classPath=Pathto thedriver")
connection<- dbConnect(driver, databasewithport, user, pwd)
sql_code = paste("SELECT .....")
mydata= dbGetQuery(connection, sql_code )
I am searching for a solution that helps do the same thing but using Python. I found the following link, but do not understand which example to use and what else to do. As I understood here no need to connect to the RJDBC driver. Could you help to find the solution which gives the same output as R version.
The code below works well, however, data is retrieved as one value, to get another I need to change ....cur.fetchone()[ANYNUMBER]). How can I get a data frame of the SQL code?
import vertica_python
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'vdb',
'connection_load_balance': True}
# Server enables load balancing
with vertica_python.connect(**conn_info) as conn:
cur = conn.cursor()
cur.execute("SELECT NODE_NAME FROM V_MONITOR.CURRENT_SESSION")
print("Client connects to primary node:", cur.fetchone()[0])
cur.execute("SELECT SET_LOAD_BALANCE_POLICY('ROUNDROBIN')")

First of all you will need to install the vertica-python package:
pip install vertica-python
Next, you need to create a connection and perform the query. When retrieving the query results, you can (1) load them all or (2) process them one by one.
import vertica_python
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'myuser',
'password': 'mypass',
'database': 'vdb',
'connection_load_balance': True}
with vertica_python.connect(**conn_info) as connection:
cur = conn.cursor()
cur.execute("SELECT NODE_NAME FROM V_MONITOR.CURRENT_SESSION")
# (1) If you want to load all the results in-memory
data = cur.fetchall()
print(data)
# (2) If you want to process one by one
for row in cur.iterate():
print(row)

How do I connect to my database without hard coding it in flask

I’m currently connecting to my database through my hard coding link in my config file, but I want to be able to dynamically generate this connection so I will be able to use multiple connections in the future.
SQLALCHEMY_DATABASE_URI = ‘postgresql+psycopg2://<blah>/database’

You could put the database connection parameters in an external file, eg connections_settings.ini
[credentials]
host=localhost
dbname=test
username=me
password=secret
and then read them with the configparser module, and create the connection url with string interpolation
import configparser
config = configparser.ConfigParser('connection_settings')['credentials']
connection_settings = {
'host': config['host'],
'dbname': config['dbname'],
'user': config['username'],
'password': config['password']
}
SQLALCHEMY_DATABASE_URI = f'postgresql+psycopg2://{host}/{dbname}'

Query MySQL (SQL server) in parallel from python

I have collection of objects and each object I would like to populate with some data from MySQL (could be SQL Server). I would like to do in parallel fashion. When establishing connection to MySQL I have two items
connection
cursor
Hence, the question, when making calls in parallel, should I pass
connection only
cursor only
connection and cursor only
do not pass anything and let each object's method establish connection on its own.
---------------code formatting does not work without this line -------
import mysql.connector
my_collection = [My_Object(), My_Object()]
config = {'user': 'someuser', 'password': 'somepassword', 'host': 'localhost', 'port': '3306',
'database': 'somedb','raise_on_warnings': True}
connection = mysql.connector.connect(**config)
cursor = connection.cursor(dictionary=True)
#some parallel loop
for el in my_collection:
el.pull_data_from(WHAT TO PASS???)

Oracle connection string with at sign # in pasword

I have a code that connect to oracle using connection string:
conn = cx_Oracle.connect('username/password#server:port/services')
But the problem is my password contain # character so it may become
conn = cx_Oracle.connect('username/p#ssword#server:port/services')
it return
DatabaseError: ORA-12154: TNS:could not resolve the connect identifier
specified
I use Django with Oracle with this settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'Services',
'USER': 'user',
'PASSWORD': 'p#ssword',
'HOST': 'ip',
'PORT': 'port',
}
}
I cant change password :( Does anyone know this problem?

I haven't tried cx_Oracle, but you might be able to connect by specifying the individual parameters -
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn='server:port/services')
OR
dsn_tns = cx_Oracle.makedsn('server', 'port', 'services')
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn=dsn_tns)

You can use any of the following way based on Service Name or SID whatever you have.
With SID:
dsn_tns = cx_Oracle.makedsn('server', 'port', 'sid')
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn=dsn_tns)
OR
With Service Name:
dsn_tns = cx_Oracle.makedsn('server', 'port', service_name='service_name')
conn = cx_Oracle.connect(user='username', password='p#ssword', dsn=dsn_tns)

Does this work?
conn = cx_Oracle.connect('username/"p#ssword"#server:port/services')

FYI: This was a long-standing bug in Django. The first stable version containing the fix is v2.1

All the answers mentioned here did not work for SQLAlchemy.
Parsing the characters to create a valid URL worked:
from sqlalchemy.engine import create_engine
import urllib.parse
url_password = urllib.parse.quote_plus(password)
connection_string = f"oracle+cx_oracle://{username}:{url_password}#{server}:{port}/?service_name={service}"
connection = create_engine(connection_string)
Should probably do the same with the other components of the connection string.

MySQL Connector for Python

I am using mysql.connector for Python. Below are my connection parameters. How can I set a timeout or increase the default timeout?
class Config(object):
"""Configure me so examples work
Use me like this:
mysql.connector.Connect(**Config.dbinfo())
"""
HOST = dbhost
DATABASE = dbdatabase
USER = dbusername
PASSWORD = dbpassword
PORT = 3306
CHARSET = 'utf8'
UNICODE = True
WARNINGS = True
#classmethod
def dbinfo(cls):
return {
'host': cls.HOST,
'port': cls.PORT,
'database': cls.DATABASE,
'user': cls.USER,
'password': cls.PASSWORD,
'charset': cls.CHARSET,
'use_unicode': cls.UNICODE,
'get_warnings': cls.WARNINGS,
}

According to MySQL Connector/Python :: 6 Connector/Python Connection Arguments in the official documentation:
To set a timeout value for connections, use connection_timeout.

If you are using mysql.connector.connect to connect with DB you can simply use connection_timeout parameter to define a timeout externally make sure value given will be in seconds instead of milliseconds.
conn = mysql.connector.connect(
pool_name=pool['name'],
pool_size=pool['size'],
host=config['host'],
port=config['port'],
user=config['user'],
passwd=config['passwd'],
db=config['db'],
charset=config['charset'],
use_unicode=True,
connection_timeout=10
)

I found something here: http://dev.mysql.com/doc/refman/5.5/en/connector-python-connectargs.html
I think you are looking for the parameter
connection_timeout (connect_timeout*)

This works for me:
conn = mysql.connector.connect(
pool_name=pool['name'],
pool_size=pool['size'],
host=config['host'],
port=config['port'],
user=config['user'],
passwd=config['passwd'],
db=config['db'],
charset=config['charset'],
use_unicode=True,
connect_timeout=1000
)
If you use MySQLdb:
conn = MySQLdb.connect(
host=config['host'],
port=config['port'],
user=config['user'],
passwd=config['passwd'],
db=config['db'],
charset=config['charset'],
use_unicode=True,
connect_timeout=1000
)