Develop Reference

Incremental ID shared between classes

I'm coding a virtual assistant in Python, and I want to store each request, response and possible error in a database.
I'm using one class for request, another class for response and another class for error.
How can I create an ID variable that is shared for the respectives classes instances, for example:
First run of the program (the normal and correct running of the program):
request_id = 1
response_id = 1
Second run (an error occurred and stopped the program to proceed to the response class):
request_id = 2
error_id = 2
Third run (the program ran fine and the response class skipped the id 2 -
that is the behavior that I want):
request_id = 3
response_id = 3
Note that in the third run, that response_id received the id 3 and the response_id = 2 will never exist, cause in the second run the proccess started with request and stopped in the error.
The ID variable must be always unique, even when my program crashes and I must restart him. I know I could grab the last id in the database when my program runs, but there's a way to do it without envolving the database?

Since you are using database to store the request and response why don't you use database to generate this id for you.
This can be done by creating the table with primary key int auto increment. Every request/response should be inserted into database, and the database will generate an unique id for each record inserted.

a possible solution would be to use Pyro4 instead of a DB if you don't want it. You can use the following code:
Tracker.py
import Pyro4
#Pyro4.expose
#Pyro4.behavior(instance_mode="single")
class Tracker(object):
def __init__(self):
self._id = None
def setId(self, value):
print "set well :)", value
self._id = value
print self._id
def getId(self):
print "returning", self._id
return self._id
daemon = Pyro4.Daemon()
uri = daemon.register(Tracker)
print("URI: ", uri)
daemon.requestLoop()
Status.py
import Pyro4
class Status(object):
def __init__(self, id):
self._id = id
self._pyro = None
def connect(self, target):
self._pyro = Pyro4.Proxy(target)
def updateId(self):
if ( not self._pyro is None ):
self._pyro.setId(self._id)
print "updated"
else:
print "please connect"
def getId(self):
if ( not self._pyro is None ):
return self._pyro.getId()
else:
print "please connect"
Success.py
from Status import *
class Success(Status):
def __init__(self):
super(Success,self).__init__(1)
Wait.py
from Status import *
class Wait(Status):
def __init__(self):
super(Wait,self).__init__(1)
Error.py
from Status import *
class Error(Status):
def __init__(self):
super(Error,self).__init__(3)
run.py
from Success import *
from Wait import *
from Error import *
#just an example
s = Success()
w = Wait()
e = Error()
s.connect("PYRO:obj_c98931f8b95d486a9b52cf0edc61b9d6#localhost:51464")
s.updateId()
print s.getId()
w.connect("PYRO:obj_c98931f8b95d486a9b52cf0edc61b9d6#localhost:51464")
w.updateId()
print s.getId()
e.connect("PYRO:obj_c98931f8b95d486a9b52cf0edc61b9d6#localhost:51464")
e.updateId()
print s.getId()
Of course you would need to use a different URI but you should have a good idea now. using Pyro you could also specify a static URI name if needed.
The output should be:
$ c:\Python27\python.exe run.py
updated
1
updated
2
updated
3
HTH

Pyqt and general python, can this be considered a correct approach for coding?

I have a dialog window containing check-boxes, when each of them is checked a particular class needs to be instantiated and a run a a task on a separated thread (one for each check box). I have 14 check-boxes to check the .isChecked() property and is comprehensible checking the returned Boolean for each of them is not efficient and requires a lot more coding.
Hence I decided to get all the children items corresponding to check-box element, get just those that are checked, appending their names to list and loop through them matching their name to d dictionary which key is the name of the check box and the value is the corresponding class to instantiate.
EXAMPLE:
# class dictionary
self.summary_runnables = {'dupStreetCheckBox': [DupStreetDesc(),0],
'notStreetEsuCheckBox': [StreetsNoEsuDesc(),1],
'notType3CheckBox': [Type3Desc(False),2],
'incFootPathCheckBox': [Type3Desc(True),2],
'dupEsuRefCheckBox': [DupEsuRef(True),3],
'notEsuStreetCheckBox': [NoLinkEsuStreets(),4],
'invCrossRefCheckBox': [InvalidCrossReferences()],
'startEndCheckBox': [CheckStartEnd(tol=10),8],
'tinyEsuCheckBox': [CheckTinyEsus("esu",1)],
'notMaintReinsCheckBox': [CheckMaintReins()],
'asdStartEndCheckBox': [CheckAsdCoords()],
'notMaintPolysCheckBox': [MaintNoPoly(),16],
'notPolysMaintCheckBox': [PolyNoMaint()],
'tinyPolysCheckBox': [CheckTinyEsus("rd_poly",1)]}
# looping through list
self.long_task = QThreadPool(None).globalInstance()
self.long_task.setMaxThreadCount(1)
start_report = StartReport(val_file_path)
end_report = EndReport()
# start_report.setAutoDelete(False)
# end_report.setAutoDelete(False)
end_report.signals.result.connect(self.log_progress)
end_report.signals.finished.connect(self.show_finished)
# end_report.setAutoDelete(False)
start_report.signals.result.connect(self.log_progress)
self.long_task.start(start_report)
# print str(self.check_boxes_names)
for check_box_name in self.check_boxes_names:
run_class = self.summary_runnables[check_box_name]
if run_class[0].__class__.__name__ is 'CheckStartEnd':
run_class[0].tolerance = tolerance
runnable = run_class[0]()
runnable.signals.result.connect(self.log_progress)
self.long_task.start(runnable)
self.long_task.start(end_report)
example of a runnable (even if some of them use different global functions)
I can't post the global functions that write content to file as they are too many and not all 14 tasks execute the same type function. arguments of these functions are int keys to other dictionaries that contain the report static content and the SQL queries to return report main dynamic contents.
class StartReport(QRunnable):
def __init__(self, file_path):
super(StartReport,self).__init__()
# open the db connection in thread
db.open()
self.signals = GeneralSignals()
# self.simple_signal = SimpleSignal()
# print self.signals.result
self.file_path = file_path
self.task = "Starting Report"
self.progress = 1
self.org_name = org_name
self.user = user
self.report_title = "Validation Report"
print "instantiation of start report "
def run(self):
self.signals.result.emit(self.task, self.progress)
if self.file_path is None:
print "I started and found file none "
return
else:
global report_file
# create the file and prints the header
report_file = open(self.file_path, 'wb')
report_file.write(str(self.report_title) + ' for {0} \n'.format(self.org_name))
report_file.write('Created on : {0} at {1} By : {2} \n'.format(datetime.today().strftime("%d/%m/%Y"),
datetime.now().strftime("%H:%M"),
str(self.user)))
report_file.write(
"------------------------------------------------------------------------------------------ \n \n \n \n")
report_file.flush()
os.fsync(report_file.fileno())
class EndReport(QRunnable):
def __init__(self):
super(EndReport,self).__init__()
self.signals = GeneralSignals()
self.task = "Finishing report"
self.progress = 100
def run(self):
self.signals.result.emit(self.task, self.progress)
if report_file is not None:
# write footer and close file
report_file.write("\n \n \n")
report_file.write("---------- End of Report -----------")
report_file.flush()
os.fsync(report_file.fileno())
report_file.close()
self.signals.finished.emit()
# TODO: checking whether opening a db connection in thread might affect the db on the GUI
# if db.isOpen():
# db.close()
else:
return
class DupStreetDesc(QRunnable):
"""
duplicate street description report section creation
:return: void if the report is to text
list[string] if the report is to screen
"""
def __init__(self):
super(DupStreetDesc,self).__init__()
self.signals = GeneralSignals()
self.task = "Checking duplicate street descriptions..."
self.progress = 16.6
def run(self):
self.signals.result.emit(self.task,self.progress)
if report_file is None:
print "report file is none "
# items_list = write_content(0, 0, 0, 0)
# for item in items_list:
# self.signals.list.emit(item)
else:
write_content(0, 0, 0, 0)
Now, I used this approach before and it has always worked fine without using multiprocessing. In this case it works good to some extent, I can run the tasks the first time but if I try to run for the second time I get the following Python Error :
self.long_task.start(run_class[0])
RuntimeError: wrapped C/C++ object of type DupStreetDesc has been deleted
I tried to use run_class[0].setAutoDelete(False) before running them in the loop but pyQt crashes with a minidump error (I am running the code in QGIS) and I the programs exists with few chances to understand what has happened.
On the other hand, if I run my classes separately, checking with an if else statement each check-box, then it works fine, I can run the tasks again and the C++ classes are not deleted, but it isn't a nice coding approach, at least from my very little experience.
Is there anyone else out there who can advise a different approach in order to make this run smoothly without using too many lines of code? Or knows whether there is a more efficient pattern to handle this problem, which I think must be quite common?

It seems that you should create a new instance of each runnable, and allow Qt to automatically delete it. So your dictionary entries could look like this:
'dupStreetCheckBox': [lambda: DupStreetDesc(), 0],
and then you can do:
for check_box_name in self.check_boxes_names:
run_class = self.summary_runnables[check_box_name]
runnable = run_class[0]()
runnable.signals.result.connect(self.log_progress)
self.long_task.start(runnable)
I don't know why setAutoDelete does not work (assuming you are calling it before starting the threadpool). I suppose there might be a bug, but it's impossible to be sure without having a fully-working example to test.

How to use the DICT protocol client in Python?

I use the command line DICT client named dict like this : dict <some word>, which will show me the meaning from the dictionary servers which I have configured.
I'd like to interact with the dict servers from Python, for example reimplement that command line client in Python.
I found a Python module for the dict protocol in the Ubuntu repositories that I've installed (apt-get install python-dictclient), but unfortunately I couldn't find any documentation for this module. I tried to understand the modules's builtin help help('dictclient') but didn't succeed. I could only make a connection to a dict server and was able to see apartial definition, here's my attempt :
import dictclient
c = dictclient.Connection('localhost', 2628)
If anyone has experience with this module please explain to me how to use it.

dict-like definition fetching using dictclient in Python:
from dictclient import Connection, Database
from sys import argv
con = Connection("dict.org") #or whatever your server is
db = Database(con, "*") #replace * with ! to get only the first result
def_list = db.define(argv[1]) #list containing Definition objects
for x in def_list:
print x.getdefstr() + '\n'

Here is a Python3 version of dictlient:
# -*- coding: UTF-8 -*-
# Client for the DICT protocol (RFC2229)
#
# Copyright (C) 2002 John Goerzen
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
# A few small hacks to make it work on Python3 - KrisvanderMerwe 25 Aoril 2015
import socket, re
version = '1.0.2.1'
def dequote(teks):
"""Will remove single or double quotes from the start and end of a string
and return the result."""
quotechars = "'\""
while len(teks) and teks[0] in quotechars:
teks = teks[1:]
while len(teks) and teks[-1] in quotechars:
teks = teks[0:-1]
return teks
def enquote(teks):
"""This function will put a string in double quotes, properly
escaping any existing double quotes with a backslash. It will
return the result."""
return '"' + teks.replace('"', "\\\"") + '"'
class Connection:
"""This class is used to establish a connection to a database server.
You will usually use this as the first call into the dictclient library.
Instantiating it takes two optional arguments: a hostname (a string)
and a port (an int). The hostname defaults to localhost
and the port to 2628, the port specified in RFC."""
def __init__(self, hostname = 'localhost', port = 2628):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.connect((hostname, port))
self.rfile = self.sock.makefile()
self.wfile = self.sock.makefile("wb", 0)
self.saveconnectioninfo()
def getresultcode(self):
"""Generic function to get a result code. It will return a list
consisting of two items: the integer result code and the text
following. You will not usually use this function directly."""
line = self.rfile.readline().strip()
code, text = line.split(' ', 1)
return [int(code), text]
def get200result(self):
"""Used when expecting a single line of text -- a 200-class
result. Returns [intcode, remaindertext]"""
code, text = self.getresultcode()
if code < 200 or code >= 300:
raise Exception ("Got '%s' when 200-class response expected %s " % (code,text) )
return [code, text]
def get100block(self):
"""Used when expecting multiple lines of text -- gets the block
part only. Does not get any codes or anything! Returns a string."""
data = []
while 1:
line = self.rfile.readline().strip()
if line == '.':
break
data.append(line)
return "\n".join(data)
def get100result(self):
"""Used when expecting multiple lines of text, terminated by a period
and a 200 code. Returns: [initialcode, [bodytext_1lineperentry],
finalcode]"""
code, text = self.getresultcode()
if code < 100 or code >= 200:
raise Exception ("Got '%s' when 100-class response expected" % code )
bodylines = self.get100block().split("\n")
code2 = self.get200result()[0]
return [code, bodylines, code2]
def get100dict(self):
"""Used when expecting a dictionary of results. Will read from
the initial 100 code, to a period and the 200 code."""
dicl = {}
for line in self.get100result()[1]:
key, val = line.split(' ', 1)
dicl[key] = dequote(val)
return dicl
def saveconnectioninfo(self):
"""Called by __init__ to handle the initial connection. Will
save off the capabilities and messageid."""
code, string = self.get200result()
assert code == 220
capstr, msgid = re.search('<(.*)> (<.*>)$', string).groups()
self.capabilities = capstr.split('.')
self.messageid = msgid
def getcapabilities(self):
"""Returns a list of the capabilities advertised by the server."""
return self.capabilities
def getmessageid(self):
"""Returns the message id, including angle brackets."""
return self.messageid
def getdbdescs(self):
"""Gets a dict of available databases. The key is the db name
and the value is the db description. This command may generate
network traffic!"""
if hasattr(self, 'dbdescs'):
return self.dbdescs
self.sendcommand("SHOW DB")
self.dbdescs = self.get100dict()
return self.dbdescs
def getstratdescs(self):
"""Gets a dict of available strategies. The key is the strat
name and the value is the strat description. This call may
generate network traffic!"""
if hasattr(self, 'stratdescs'):
return self.stratdescs
self.sendcommand("SHOW STRAT")
self.stratdescs = self.get100dict()
return self.stratdescs
def getdbobj(self, dbname):
"""Gets a Database object corresponding to the database name passed
in. This function explicitly will *not* generate network traffic.
If you have not yet run getdbdescs(), it will fail."""
if not hasattr(self, 'dbobjs'):
self.dbobjs = {}
if dbname in self.dbobjs:
return self.dbobjs[dbname]
# We use self.dbdescs explicitly since we don't want to
# generate net traffic with this request!
if dbname != '*' and dbname != '!' and \
not dbname in self.dbdescs.keys():
raise Exception( "Invalid database name '%s'" % dbname )
self.dbobjs[dbname] = Database(self, dbname)
return self.dbobjs[dbname]
def sendcommand(self, command):
"""Takes a command, without a newline character, and sends it to
the server."""
self.wfile.write(command.encode() + b"\n")
def define(self, database, word):
"""Returns a list of Definition objects for each matching
definition. Parameters are the database name and the word
to look up. This is one of the main functions you will use
to interact with the server. Returns a list of Definition
objects. If there are no matches, an empty list is returned.
Note: database may be '*' which means to search all databases,
or '!' which means to return matches from the first database that
has a match."""
self.getdbdescs() # Prime the cache
if database != '*' and database != '!' and \
not database in self.getdbdescs():
raise Exception ( "Invalid database '%s' specified" % database )
self.sendcommand("DEFINE " + enquote(database) + " " + enquote(word))
code = self.getresultcode()[0]
retval = []
if code == 552:
# No definitions.
return []
if code != 150:
raise Exception ("Unknown code %d" % code )
while 1:
code, text = self.getresultcode()
if code != 151:
break
resultword, resultdb = re.search('^"(.+)" (\S+)', text).groups()
defstr = self.get100block()
retval.append(Definition(self, self.getdbobj(resultdb),
resultword, defstr))
return retval
def match(self, database, strategy, word):
"""Gets matches for a query. Arguments are database name,
the strategy (see available ones in getstratdescs()), and the
pattern/word to look for. Returns a list of Definition objects.
If there is no match, an empty list is returned.
Note: database may be '*' which means to search all databases,
or '!' which means to return matches from the first database that
has a match."""
self.getstratdescs() # Prime the cache
self.getdbdescs() # Prime the cache
if not strategy in self.getstratdescs().keys():
raise Exception ( "Invalid strategy '%s'" % strategy )
if database != '*' and database != '!' and not database in self.getdbdescs().keys():
raise Exception ( "Invalid database name '%s'" % database )
self.sendcommand("MATCH %s %s %s" % (enquote(database),
enquote(strategy),
enquote(word)))
code = self.getresultcode()[0]
if code == 552:
# No Matches
return []
if code != 152:
raise Exception ( "Unexpected code %d" % code )
retval = []
for matchline in self.get100block().split("\n"):
matchdict, matchword = matchline.split(" ", 1)
retval.append(Definition(self, self.getdbobj(matchdict),
dequote(matchword)))
if self.getresultcode()[0] != 250:
raise Exception ( "Unexpected end-of-list code %d" % code )
return retval
class Database:
"""An object corresponding to a particular database in a server."""
def __init__(self, dictconn, dbname):
"""Initialize the object -- requires a Connection object and
a database name."""
self.conn = dictconn
self.name = dbname
def getname(self):
"""Returns the short name for this database."""
return self.name
def getdescription(self):
if hasattr(self, 'description'):
return self.description
if self.getname() == '*':
self.description = 'All Databases'
elif self.getname() == '!':
self.description = 'First matching database'
else:
self.description = self.conn.getdbdescs()[self.getname()]
return self.description
def getinfo(self):
"""Returns a string of info describing this database."""
if hasattr(self, 'info'):
return self.info
if self.getname() == '*':
self.info = "This special database will search all databases on the system."
elif self.getname() == '!':
self.info = "This special database will return matches from the first matching database."
else:
self.conn.sendcommand("SHOW INFO " + self.name)
self.info = "\n".join(self.conn.get100result()[1])
return self.info
def define(self, word):
"""Get a definition from within this database.
The argument, word, is the word to look up. The return value is the
same as from Connection.define()."""
return self.conn.define(self.getname(), word)
def match(self, strategy, word):
"""Get a match from within this database.
The argument, word, is the word to look up. The return value is
the same as from Connection.define()."""
return self.conn.match(self.getname(), strategy, word)
class Definition:
"""An object corresponding to a single definition."""
def __init__(self, dictconn, db, word, defstr = None):
"""Instantiate the object. Requires: a Connection object,
a Database object (NOT corresponding to '*' or '!' databases),
a word. Optional: a definition string. If not supplied,
it will be fetched if/when it is requested."""
self.conn = dictconn
self.db = db
self.word = word
self.defstr = defstr
def getdb(self):
"""Get the Database object corresponding to this definition."""
return self.db
def getdefstr(self):
"""Get the definition string (the actual content) of this
definition."""
if not self.defstr:
self.defstr = self.conn.define(self.getdb().getname(), self.word)[0].getdefstr()
return self.defstr
def getword(self):
"""Get the word this object describes."""
return self.word

Passing variable to another Python Script [duplicate]

This question already has answers here:
Python Unbound Method TypeError
(3 answers)
Closed 9 years ago.
I am having difficulty passing a variable from one function to another function in another python script. I have read the other answers but they have not really helped on this subject.
This is the first file I want to send the variable to( some code omitted for clarity )
# TestGUI.py
from Tkinter import *
import serial
import os
class Testgui:
def __init__(self, master):
def writetoBOT(self,instruct):
ser = serial.Serial(6)
ser.baudrate = 9600
ser.parity = serial.PARITY_NONE #set parity check: no parity
ser.timeout = 1 #non-block read
ser.writeTimeout = 2 #timeout for writ
if(ser.isOpen() == False):
ser.open()
print ser.portstr # check which port was really used
ser.write(instruct)
else :
ser.write(instruct)
This is the sceond file:
# TestGUI_2.py
from TestGUI import Testgui
class Tracker:
def __init__(self):
pass
def drive(self,cords, cords1):
while( cords >= 320):
l='l'
Testgui.writetoBOT(l) # This is the problem line
TypeError: unbound method writetoBOT() must be called with TestGUI instance as first argument (got str instance instead)

writetoBOT takes 2 arguments: self and instruct.
call it with a Testgui instance:
tgui=Testgui(your_master)
tgui.writetoBOT(l)
If you want to call it with Testgui class, you still need to pass an instance of Testgui:
tgui=Testgui(your_master)
Testgui.writetoBOT(tgui, l)

Alternatively, you can make common space for this two scripts, it acn by database - sqllite
For example,
# file1.py
import sqlite3
con = sqlite3.connect('messages.db')
cur = con.cursor()
#cur.execute('CREATE TABLE message (id INTEGER PRIMARY KEY, name TEXT, content TEXT, read INTEGER)')
#con.commit()
for x in range(100000):
if x in range(1, 500):
cur.execute('INSERT INTO message (id, name, content, read) VALUES(NULL, "Guido", "van Rossum", 0)')
con.commit()
# file2.py
import sqlite3
import time
con = sqlite3.connect('messages.db')
cur = con.cursor()
def listen():
messages = cur.execute('SELECT * FROM message WHERE read=0')
if not messages:
return False
for m in messages:
print 'get message ', m
cur.execute("UPDATE message SET read=1 WHERE id=?", m[0])
con.commit()
print 'update db'
return True
while True:
listen()
time.sleep(5)

You declared Testgui as a class. This is to be understood as a skeleton or wireframe (beware, this is a shortcut, not the reality). You need to first create a "real" object out from this skeleton in order to use it.
testgui=Testgui(amaster)
It is possible in classes to have methods (bound functions) that apply at class level. These are called static methods or class methods. They have to be decorated in python.
See http://en.wikipedia.org/wiki/Object_oriented_programming for more information.

Proper way to do session handling in Python + Pylons for a php programmer

I'm a php programmer who's just getting started with Python. I'm trying to get Python to handle login/logout via database-stored sessions. Things work, but seem inconsistent. For example, sometimes a user isn't logged out. Sometimes users "switch" logins. I'm guessing this has something to do with thread-safety, but I'm just not sure where to begin on how to fix this. Any help would be appreciated. Here's what I have now:
#lib/base.py
def authenticate():
#Confirm login
try:
if user['authenticated'] != True:
redirect_to(controller='login', action='index')
except KeyError:
redirect_to(controller='login', action='index')
#Global variables
user = {}
connection = {}
class BaseController(WSGIController):
#Read if there is a cookie set
try:
session = request.cookies['session']
#Create a session object from the session id
session_logged_in = Session(session)
#If the session is valid, retrieve the user info
if session_logged_in.isValid(remote_addr):
#Set global variables about the logged in user
user_logged_in = User(session_logged_in.user_id)
user['name'] = c.name = user_logged_in.name
user['name_url'] = c.name_url = user_logged_in.name_url
user['first_name'] = c.first_name = user_logged_in.first_name
user['last_name'] = c.last_name = user_logged_in.last_name
user['email'] = c.email = user_logged_in.email
user['about'] = c.about = user_logged_in.about
user['authenticated'] = c.authenticated = True
user['profile_url'] = c.profile_url = user_logged_in.profile_url
user['user_thumb'] = c.user_thumb = user_logged_in.user_thumb
user['image_id'] = c.image_id = user_logged_in.image_id
user['id'] = c.user_id = user_logged_in.id
#Update the session
session_logged_in.current_uri = requested_url
session_logged_in.update()
#If no session has been set, do nothing
except KeyError:
user['authenticated'] = False
I can then access the user{} global from my controllers:
#controllers/profile.py
from project.lib.base import BaseController, user
class ProfileController(BaseController):
def index(self, id=None, name_url=None):
#If this is you
if user['id'] == 1
print 'this is you'
Is there a better way to do this? Thanks for your help.

Pylons has a 'sessions' object that exists to handle this kind of situation. The example on the Pylons website seems to match what you want.
I think you are seeing problems because of the globals 'user' and 'connection'. Pylons has a globals object that is designed to share information between all controllers and is not reset on each request.