For my repository class, I am creating an connection to the database when a function needs it (if None). I am using the __del__ to close the connection if not None. The object is guaranteed to be short lived.
I realized I am dependent on Ref counting to close the connection as soon as possible. I need the connection to be a field in the class because I am doing a select for update. The default python gc I read does ref counting at least the CPython implementation. But the pypy implementation has many different types of garbage collectors. Is it okay to rely on ref counting gc to close the connection as soon as possible? Or should I write a separate function and ensure that the connection is closed using a try finally block?
The class only consists of the connection field and two functions one to lock the table in database and select values and the other to update them.
This is my current implementation:
class ReserveRepository:
def __init__(self):
self.conn = None
def select(self):
cur = None
try:
self.conn = psycopg2.connect(config.DATABASE_URL)
cur = self.conn.cursor()
cur.execute('lock table sometable in ACCESS EXCLUSIVE mode')
cur.execute('select id from sometable where condition')
return list(map(lambda x: x[0]))
finally:
if cur is not None:
cur.close()
def update_quantities(self, id):
cur = None
try:
if self.conn is None:
self.conn = psycopg2.connect(config.DATABASE_URL)
cur = self.conn.cursor()
cur.execute('update sometable set column1 = value where id = %s', (id,))
self.conn.commit()
return reservation_id
finally:
if cur is not None:
cur.close()
def __del__(self):
if self.conn is not None:
self.conn.close()
self.conn = None
tldr; The answer is no. Pypy does not implement reference counting as far as I understand. To demonstrate this, I made this simple program:
class Test:
def __init__(self):
print('__init__')
def __del__(self):
print('__del__')
t = Test()
print(t)
The output on CPython 3.8.2 is:
__init__
<__main__.Test object at 0x7f6e3bc13dc0>
__del__
However, the output on Pypy 7.3.1 (Python 3.6.9) is
__init__
<__main__.Test object at 0x00007f58325dfe88>
The __del__ function is not called.
Even on CPython, the answer is no. If the object is attached to a long-lived cached instance, __del__ will never be called. A better design pattern is to use a context manager for your object and do clean up at __exit__:
with make_a_connection(parameter) as conn:
useconn(conn)
Related
I am new to Python and coding in general and after watching a four hour lecture, I decided to give writing a small functional application a shot.
In my application I am working with a tkinter GUI and a SQlite3 database. ClassA handles the GUI and functions that the GUI calls, while ClassB handles the database.
To insert some data the user has entered into the GUI I have defined the function add() in ClassA. The function simply calls method .insert from ClassB and then passes the data the GUI has gathered as a parameter into the .insert function in ClassB. Or so I thought..
Whenever I try I get the following traceback:
self.cur.execute("INSERT TO example(column1, column2, column3, column4) VALUES (?, ?, ?, ?)",
AttributeError: 'ClassA' object has no attribute 'cur'
Here's the code I am trying to run:
class ClassA:
def __init__(self, master):
self.master = master
master.title = ("Example Program")
...
# REST OF GUI
def add(self):
ClassB.insert(self, self.entry_1.get())
class ClassB:
def __init_(self):
self.con = sqlite3.connect("example.db")
self.cur = self.con.cursor()
self.cur.execute("CREATE TABLE IF NOT EXISTS example (row text)")
self.con.commit()
def insert(self, data):
self.cur.execute("INSERT INTO TABLE example (row) VALUES (?)", (data))
self.con.commit()
On the first line of ClassA.add, don't pass in self. self is passed in by python as the object the method was called on. If you have my_object.method(), the instance of my_object would be passed into my_object method as self. An example
Something else that I'm noticing is that calling ClassB.insert should not work since
in order to use a class's methods, you need to instantiate an instance. For e.g.:
b = ClassB()
b.insert(a1, a2,a3, a4)
I'm writing a Python class to manage a Postgres database connection using pyscopg2.
I'd like the class to set up a connection to the database upon initialisation (I feel like this might be a terrible idea, but I can't think of a good reason why). I'm trying to make this work with a property, which I've never used before. In other words, I want the getter to be called from within the __init__ method.
My class looks something like this:
class MyDatabase:
connection_string = "host='<host_ip>' db_name='<db_name>'"
def __init__(self):
# *
self._connection = connection
#property
def connection(self):
# Check for an existing connection
if self._connection:
self._connection.close()
self._connection = psycopg2.connect(connection_string)
return self._connection
...
In this version, the check for an existing connection throws AttributeError: Elefriends instance has no attribute '_connection', which makes sense. I can get around this by simply adding a line that says self._connection = None at the place I've marked with # *, but this feels clunky. Is this the price I pay for the convenience? Am I just being fussy? Or is there a better way?
Thanks!
Instead of the if ... statement, you could use:
try:
self._connection.close()
except AttributeError:
pass
self._connection = psycopg2.connect(connection_string)
return self._connection
i wrote a python program like this that should run in multithreading mode:
def Func(host,cursor,db):
cursor.execute('''SELECT If_index, Username, Version, Community, Ip_traff FROM HOST WHERE
Hostname = ?''',(host,))
#do something
#--- Main ---
db = sqlite3.connect(os.getcwd()+'\HOST', check_same_thread = False) #opendatabase
cursor = db.cursor() #generate a cursor
for ii in range(len(host)): #host is a list of ipaddress
#for each host i want generate a thread
thr = threading.Thread(target = Func, args=(host[ii],cursor,db)
thr.start()
i receive the sqlite3.ProgrammingError: Recursive use of cursors not allowed. How can i manage the recursive cursor for sqlite3 in this case?
thanks a lot
Paolo
Well, the thing is the sqlite3 module doesn't likes multithread cases, you can see that in the sqlite3 module's documentation
...the Python module disallows sharing connections and cursors between threads[1]
What I would do is to use some sort of synchronization in the Func function, for example, a threading.Lock[2]. Your Func will look like this:
# Define the lock globally
lock = threading.Lock()
def Func(host,cursor,db):
try:
lock.acquire(True)
res = cursor.execute('''...''',(host,))
# do something
finally:
lock.release()
The previous code will synchronize the execution of the cursor.execute by letting just one thread take the lock, the other threads will wait until it's released, when the thread with the lock is done, it releases the lock for the others to take it.
That should fix the problem.
[1] https://docs.python.org/2/library/sqlite3.html#multithreading
[2] https://docs.python.org/2/library/threading.html?highlight=threading#rlock-objects
One way is to fix it just block sqlite before writing
import sqlite3
con = sqlite3.connect(...)
...
with con:
# Database is locked here
cur = conn.cursor()
res = cur.execute('''...''',(host,))
I'm currently writing a script that uses sqlite3. I recently ran into a problem with the database being in use by another program due to my code exiting early with an error.
With similar problems, one usually uses:
conn = sqlite3.connect(...)
try:
#Do stuff
finally:
conn.close()
But this won't work in my case. In a nutshell, this is my code:
import sqlite3
class Thingamadoodle:
def __init__(self, ...):
self.conn = sqlite3.connect(...)
...
#Methods and stuff
def __del__(self):
self.conn.close()
poop = Thingamadoodle(...)
poop.do_stuff(...)
poop.throw_irritating_exception_that_you_cant_track_down(irritatingness=11)
After the program exits without closing the connection, I get errors when I try to modify the database.
Is there a way to safely close the connection, even on an unclean exit?
To be honest, i don't understand the question much, but why not just wrap the poop.do_stuff() in a try/except block?
try:
poop.do_stuff()
except:
poop.__del__()
finally:
poop.__del__()
Or to be a bit cleaner, use a context manager:
class Thingamadoodle:
def __init__(self, ...):
...
#Methods and stuff
def __enter__(self):
self.conn = sqlite3.connect(...)
return self
def __exit__(self, errorType, errorValue, errorTrace):
self.conn.close()
And just execute it as:
with Thingmadoodle(args) as poop:
#do things
After all the code is done, or after an exception happened in the statement, __exit__ will be executed, and you can safely close it.
Hope this helps!
I would like to write a class that would contain all my MySQL operations.
Right now, I cant even get the class instantiated.
Traceback (most recent call last):
File "./compare.py", line 71, in <module>
main()
File "./compare.py", line 67, in main
db = Table.mysqlconnect()
TypeError: unbound method mysqlconnect() must be called with Table instance as first argument (got nothing instead)
code:
import MySQLdb
class Table(object):
""" Using Databases """
def __init__(self, db, name ):
self.db = db
self.name = name
self.cur = self.db.cursor()
def mysqlconnect():
conn = MySQLdb.connect (host = "mysql.blah.com",
user = "user",
passwd = "password",
db = "database")
cursor = conn.cursor ()
cursor.execute ("SELECT VERSION()")
row = cursor.fetchone ()
print "server version:", row[0]
cursor.close ()
conn.close ()
def main():
db = Table.mysqlconnect()
pass
if __name__ == '__main__':
main()
You should read those docs, but what you're looking for is:
db = Table()
db.mysqlconnect()
Short explanation: mysqlconnect is a instance method on your Table class.
Long explanation: Table is an abstract concept right now -- you've told the Python interpreter about it and what it should do, but you haven't actually made one yet. It's like the blueprint, if you will, for your class. Before you use it, or use any method defined as part of it* you'll need to actually "build it" first.
This is what you do when you do: db = Table() This tells the Python interpreter that I've got this variable called db now and I want it to be an instance of Table(). Now that you've got your instance, you can call the instance method (since the instance method only works on an instance) and get your result.
*There are things called class methods that you can use without instantiating the class first, but you'll see that when you read the documentation.
The error is exactly right:
TypeError: unbound method mysqlconnect() must be called with Table instance as first argument (got nothing instead)
In other words, you are using a class name (Table) and not an instance of the class Table.
Read these examples, they do exactly what you want. If you want to read up on classes and objects, the Python manual chapter on this is excellent.
Also, mysqlconnect needs to use self to access the class attributes. For example:
import MySQLdb
class Table:
def __init__(self, host, user, passwd, name)
self.db = MySQLdb.connect (host = host,
user = user,
passwd = passwd,
db = name)
self.cursor = self.db.cursor()
def mysqlconnect(self):
self.cursor.execute ("SELECT VERSION()")
row = cursor.fetchone()
print "server version:", row[0]
self.cursor.close ()
self.db.close ()
# Main must be outside the table class
def main():
tableInstance = Table("mysql.blah.com", "user", "password", "database")
tableInstance.mysqlconnect()
if __name__ == '__main__':
main()