Develop Reference

How to pass variable values for oracle connection in python

Im able to connect to Oracle Db successfully when i hardcode the Db details like " connection = cx_Oracle.connect("uname/pass#192.168.xxx.yyy:port/db")" but how to pass the variable values to connect to the db?
I tried some like this.
connection = cx_Oracle.connect("{}/{}#{}:{}/{}".format(uname,password,IP,port,db))
which is not working as expected so please share some thought on this to make it work.
def knowcobrand(request):
value_type = request.POST.get('CobranSelection')
cobrand_value = request.POST.get('cobrand')
env = request.POST.get('NewEnvName')
print(value_type)
print(cobrand_value)
print(env)
# feed = Environments.objects.get(Q(Env_name__icontains=env))
# print(feed.user_name)
connection = cx_Oracle.connect("app/app#192.168.xxx.yy:port/db")
I want to use the variable's value of value_type and env for Db connection

EDIT: You should probably run through the Django tutorial, it will explain the basics of Django and using the ORM
You should configure your database connection in your settings
There is a specific example for oracle
You can now use the Django ORM (after running migrations)
If you want a raw cursor for the database you can still use Django for this like so
from django.db import connection
with connection.cursor() as c:
c.execute(...)

QueuePool limit reached for Flask application

I am running an application with Flask and Flask-SQLAlchemy.
from config import FlaskDatabaseConfig
from flask import Flask
from flask import request
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy
application = Flask(__name__)
application.config.from_object(FlaskDatabaseConfig())
db = SQLAlchemy(application)
#application.route("/queue/request", methods=["POST"])
def handle_queued_request():
stuff()
return ""
def stuff():
# Includes database queries and updates and a call to db.session.commit()
# db.session.begin() and db.session.close() are not called
pass
if __name__ == "__main__":
application.run(debug=False, port=5001)
Now, from my understanding, by using Flask-SQLAlchemy I do not need to manage sessions on my own. So why am I getting the following error if I run several requests turn by turn to my endpoint?
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/3o7r)
I've tried using db.session.close() but then, instead of this error, my database updates are not committed properly. What am I doing incorrectly? Do I need to manually close connections with the database once a request has been handled?

I have found a solution to this. The issue was that I had a lot of processes that were "idle in transaction" because I did not call db.session.commit() after making certain database SELECT statements using Query.first()
To investigate this, I queried my (development) PostgreSQL database directly using:
SELECT * FROM pg_stat_activity

Just remove connection every time you make a query session to db.
products = db.session.query(Product).limit(20).all()
db.session.remove()

How to fix Firebird error Database already opened with engine instance, incompatible with current'

I have a Flask app with using flask_sqlalchemy:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config.from_pyfile(filename='settings.py', silent=True)
db = SQLAlchemy(app=app)
I want connect to same database from daemon. In daemon I just import db and use db.engine.execute for SQLAlchemy queries.
But when daemon starts main app can't connect to database.
In log I see that:
fdb.fbcore.DatabaseError: ('Error while connecting to database:\n- SQLCODE:
-902\n- I/O error during "lock" operation for file "main.fdb"\n- Database
already opened with engine instance, incompatible with current', -902,
335544344)
I trying use isolation level:
from fdb.fbcore import ISOLATION_LEVEL_READ_COMMITED_LEGACY
class TPBAlchemy(SQLAlchemy):
def apply_driver_hacks(self, app_, info, options):
if 'isolation_level' not in options:
options['isolation_level'] = ISOLATION_LEVEL_READ_COMMITED_LEGACY
return super(TPBAlchemy, self).apply_driver_hacks(app_, info, options)
And replace this:
db = SQLAlchemy()
To:
db = TPBAlchemy()
But this only make another error:
TypeError: Invalid argument(s) 'isolation_level' sent to create_engine(),
using configuration FBDialect_fdb/QueuePool/Engine. Please check that the
keyword arguments are appropriate for this combination of components.
I would appreciate the full example to address my issue.

Your connection string is for an embedded database. You're only allowed to have one 'connection' to an embedded database at a time.
If you have the Loopback provider enabled you can change your connection string to something like:
localhost:/var/www/main.fdb
or if you have the Remote provider enabled you will have to access your database from another remote node, and assuming your Firebird server lives on 192.168.1.100 change your connection string to
192.168.1.100:/var/www/main.fdb
If you're intending to use the Engine12 provider (the embedded provider), then you have to stop whatever is already connected to that database because you just can't do two simultaneously users with this provider.
Also, try to set up some database aliases so you aren't specifying a database explicitly like that. In Firebird 3.0.3 check out databases.conf, where you can do something like:
mydatabasealias=/var/www/main.fdb
and your connection string would now be mydatabasealias instead of the whole path.

Why does the python Cassandra driver fail when declared as a class field?

I'm currently working with a server process that is meant to connect to a Cassandra database and forward information to it. This process is written as a class, and my goal is to create a single Cassandra session for this class that it can use to send the information. However, I'm running into a problem; when I create a Cassandra session in the classes init method, and then later try to use the session in another method, I get the following error:
errors={}, last_host=<server IP address>. I can currently get around this problem by creating a new Cassandra session each time the method is called, but this is obviously not a good way to go about this. So, how can I make a Cassandra session that I can use consistently throughout the class?
This code does NOT work:
from cassandra.cluster import Cluster
from multiprocessing import Process
class DataProcess(Process):
def __init__(self):
super(DataProcess,self).__init__()
# Do a few other irrelevant things ...
# Set up the Cassandra connection
self.cluster = Cluster(contact_points=[CASSANDRA_IP])
self.session = self.cluster.connect('some keyspace')
print "Connected to cassandra."
def callback(self,ch,method,props,body):
prepared_statement = self.session.prepare("Some CQL statement...")
bound_statement = prepared_statement.bind(some values)
self.session.execute(bound_statement)
Output:
"Connected to cassandra."
errors={}, last_host=<server IP address>
This code DOES work, but it's a silly way to do it:
from cassandra.cluster import Cluster
from multiprocessing import Process
class DataProcess(Process):
def __init__(self):
super(DataProcess,self).__init__()
# Do a few irrelevant things ...
def callback(self,ch,method,props,body):
# Set up the Cassandra connection
cluster = Cluster(contact_points=[CASSANDRA_IP])
session = cluster.connect('some keyspace')
prepared_statement = session.prepare("Some CQL statement...")
bound_statement = prepared_statement.bind(some values)
session.execute(bound_statement)
Other relevant info:
Using python cassandra-driver version 2.5.1
Cassandra database version 2.1.8
EDIT: ANSWER
The following code solved the issue:
from cassandra.cluster import Cluster
from multiprocessing import Process
class DataProcess(Process):
def __init__(self):
super(DataProcess,self).__init__()
self.cluster = None
self.session = None
# Do a few irrelevant things ...
def callback(self,ch,method,props,body):
# Set up the Cassandra connection
cluster = Cluster(contact_points=[CASSANDRA_IP])
session = cluster.connect('some keyspace')
prepared_statement = session.prepare("Some CQL statement...")
bound_statement = prepared_statement.bind(some values)
session.execute(bound_statement)
def run(self):
self.cluster = Cluster(contact_points=[CASSANDRA_IP])
self.session = self.cluster.connect('some keyspace')

Are you creating your cluster and sessions before the fork? That could cause issues like what you are seeing. There is an amazing writeup of how to distribute work using pools using the python driver here here. This will likely be exactly what you are looking for.
If it's not please leave more context on how you are running your process, as it's tough to reproduce without knowing your process lifecycle.

Pylons: Sharing SQLAlchemy MySQL connection with external library

I am running Pylons using SQLAlchemy to connect to MySQL, so when I want to use a database connection in a controller, I can do this:
from myapp.model.meta import Session
class SomeController(BaseController):
def index(self):
conn = Session.connection()
rows = conn.execute('SELECT whatever')
...
Say my controller needs to call up an external library, that also needs a database connection, and I want to provide the connection for it from the SQLAlchemy MySQL connection that is already established:
from myapp.model.meta import Session
import mymodule
class SomeController(BaseController):
def index(self):
conn = Session.connection()
myobject = mymodule.someobject(DATABASE_OBJECT)
...
conn.close()
What should DATABSE_OBJECT be? Possibilities:
Pass Session -- and then open and close Session.connection() in the module code
Pass conn, and then call conn.close() in the controller
Just pass the connection parameters, and have the module code set up its own connection
There is another wrinkle, which is that I need to instantiate some objects in app_globals.py, and these objects need a database connection as well. It seems that app_globals.py cannot use Session's SQLAlchemy connection yet -- it's not bound yet.
Is my architecture fundamentally unsounds? Should I not be trying to share connections between Pylons and external libraries this way? Thanks!

You should not manage connections yourself - it's all done by SQLAlchemy. Just use scoped session object everywhere, and you will be fine.
def init_model(engine):
sm = orm.sessionmaker(autoflush=False, autocommit=False, expire_on_commit=False, bind=engine)
meta.engine = engine
meta.Session = orm.scoped_session(sm)
def index(self):
rows = Session.execute('SELECT ...')
You can pass Session object to your external library and do queries there as you wish. There is no need to call .close() on it.
Regarding app_globals, I solved that by adding other method in globals class which is called after db initialization from environment.py
class Globals(...):
def init_model(self, config):
self.some_persistent_db_object = Session.execute('...')
def load_environment(...):
...
config['pylons.app_globals'].init_model(config)
return config

What should DATABSE_OBJECT be? Possibilities:
4. pass a "proxy" or "helper" object with higher level of abstraction interface
Unless the external library really needs direct access to SQLAlchemy session, you could provide it with object that has methods like "get_account(account_no)" instead of "execute(sql)". Doing so would keep SQLAlchemy-specific code more isolated, and the code would be also easier to test.
Sorry that this is not so much an answer to your original question, more a design suggestion.