Note: I see that I need to more clearly work out what it is that I want each property/descriptor/class/method to do before I ask how to do it! I don't think my question can be answered at this time. Thanks all for helping me out.
Thanks to icktoofay and BrenBarn, I'm starting to understand discriptors and properties, but now I have a slightly harder question to ask:
I see now how these work:
class Blub(object):
def __get__(self, instance, owner):
print('Blub gets ' + instance._blub)
return instance._blub
def __set__(self, instance, value):
print('Blub becomes ' + value)
instance._blub = value
class Quish(object):
blub = Blub()
def __init__(self, value):
self.blub = value
And how a = Quish('one') works (produces "Blub becomes one") but take a gander at this code:
import os
import glob
class Index(object):
def __init__(self, dir=os.getcwd()):
self.name = dir #index name is directory of indexes
# index is the list of indexes
self.index = glob.glob(os.path.join(self.name, 'BatchStarted*'))
# which is the pointer to the index (index[which] == BatchStarted_12312013_115959.txt)
self.which = 0
# self.file = self.File(self.index[self.which])
def get(self):
return self.index[self.which]
def next(self):
self.which += 1
if self.which < len(self.index):
return self.get()
else:
# loop back to the first
self.which = 0
return None
def back(self):
if self.which > 0:
self.which -= 1
return self.get()
class File(object):
def __init__(self, file):
# if the file exists, we'll use it.
if os.path.isfile(file):
self.name = file
# otherwise, our name is none and we return.
else:
self.name = None
return None
# 'file' attribute is the actual file object
self.file = open(self.name, 'r')
self.line = Lines(self.file)
class Lines(object):
# pass through the actual file object (not filename)
def __init__(self, file):
self.file = file
# line is the list if this file's lines
self.line = self.file.readlines()
self.which = 0
self.extension = Extension(self.line[self.which])
def __get__(self):
return self.line[self.which]
def __set__(self, value):
self.which = value
def next(self):
self.which += 1
return self.__get__()
def back(self):
self.which -= 1
return self.__get__()
class Extension(object):
def __init__(self, lineStr):
# check to make sure a string is passed
if lineStr:
self.lineStr = lineStr
self.line = self.lineStr.split('|')
self.pathStr = self.line[0]
self.path = self.pathStr.split('\\')
self.fileStr = self.path[-1]
self.file = self.fileStr.split('.')
else:
self.lineStr = None
def __get__(self):
self.line = self.lineStr.split('|')
self.pathStr = self.line[0]
self.path = self.pathStr.split('\\')
self.fileStr = self.path[-1]
self.file = self.fileStr.split('.')
return self.file[-1]
def __set__(self, ext):
self.file[-1] = ext
self.fileStr = '.'.join(self.file)
self.path[-1] = fileStr
self.pathStr = '\\'.join(self.path)
self.line[0] = self.pathStr
self.lineStr = '|'.join(self.line)
Firstly, there may be some typos in here because I've been working on it and leaving it half-arsed. That's not my point. My point is that in icktoofay's example, nothing gets passed to Blub(). Is there any way to do what I'm doing here, that is set some "self" attributes and after doing some processing, taking that and passing it to the next class? Would this be better suited for a property?
I would like to have it so that:
>>> i = Index() # i contains list of index files
>>> f = File(i.get()) # f is now one of those files
>>> f.line
'\\\\server\\share\\folder\\file0.txt|Name|Sean|Date|10-20-2000|Type|1'
>>> f.line.extension
'txt'
>>> f.line.extension = 'rtf'
>>> f.line
'\\\\server\\share\\folder\\file0.rtf|Name|Sean|Date|10-20-2000|Type|1'
You can do that, but the issue there is less about properties/descriptors and more about creating classes that give the behavior you want.
So, when you do f.line, that is some object. When you do f.line.extension, that is doing (f.line).extension --- that is, it first evalautes f.line and then gets the extension attribute of whatever f.line is.
The important thing here is that f.line cannot know whether you are later going to try to access its extension. So you can't have f.line do one thing for "plain" f.line and another thing for f.line.extension. The f.line part has to be the same in both, and the extension part can't change that.
The solution for what you seem to want to do is to make f.line return some kind of object that in some way looks or works like a string, but also allows setting attributes and updating itself accordingly. Exactly how you do this depends on how much you need f.lines to behave like a string and how much you need it to do other stuff. Basically you need f.line to be a "gatekeeper" object that handles some operations by acting like a string (e.g., you apparently want it to display as a string), and handles other objects in custom ways (e.g., you apparently want to be able to set an extension attribute on it and have that update its contents).
Here's a simplistic example:
class Line(object):
def __init__(self, txt):
self.base, self.extension = txt.split('.')
def __str__(self):
return self.base + "." + self.extension
Now you can do:
>>> line = Line('file.txt')
>>> print line
file.txt
>>> line.extension
'txt'
>>> line.extension = 'foo'
>>> print line
file.foo
However, notice that I did print line, not just line. By writing a __str__ method, I defined the behavior that happens when you do print line. But if you evaluate it "raw" without printing it, you'll see it's not really a string:
>>> line
<__main__.Line object at 0x000000000233D278>
You could override this behavior as well (by defining __repr__), but do you want to? That depends on how you want to use line. The point is that you need to decide what you want your line to do in what situations, and then craft a class that does that.
Related
I have a class which I want to use to extract data from a text file (already parsed) and I want do so using dynamically created class methods, because otherwise there would be a lot of repetitive code. Each created class method shall be asociated with a specific line of the text file, e.g. '.get_name()' --> read a part of 0th line of text file.
My idea was to use a dictionary for the 'to-be-created' method names and corresponding line.
import sys
import inspect
test_file = [['Name=Jon Hancock'],
['Date=16.08.2020'],
['Author=Donald Duck']]
# intented method names
fn_names = {'get_name': 0, 'get_date': 1, 'get_author': 2}
class Filer():
def __init__(self, file):
self.file = file
def __get_line(cls):
name = sys._getframe().f_code.co_name
line = fn_names[name] # <-- causes error because __get_line is not in fn_names
print(sys._getframe().f_code.co_name) # <-- '__get_line'
print(inspect.currentframe().f_code.co_name) # <-- '__get_line'
return print(cls.file[line][0].split('=')[1])
for key, val in fn_names.items():
setattr(Filer, key, __get_line)
f = Filer(test_file)
f.get_author()
f.get_date()
When I try to access the method name to link the method to the designated line in the text file, I do get an error because the method name is always '__get_line' instead of e.g. 'get_author' (what I had hoped for).
Another way how I thought to solve this was to make '__get_line' accepting an additional argument (line) and set it by passing the val during 'the setattr()' as shown below:
def __get_line(cls, line):
return print(cls.file[line][0].split('=')[1])
and
for key, val in fn_names.items():
setattr(Filer, key, __get_line(val))
however, then Python complains that 1 argument (line) is missing.
Any ideas how to solve that?
I would propose a much simpler solution, based on some assumptions. Your file appears to consist of key-value pairs. You are choosing to map the line number to a function that processes the right hand side of the line past the = symbol. Python does not conventionally use getters. Attributes are much nicer and easier to use. You can have getter-like functionality by using property objects, but you really don't need that here.
class Filer():
def __init__(self, file):
self.file = file
for line in file:
name, value = line[0].split('=', 1)
setattr(self, name.lower(), value)
That's all you need. Now you can use the result:
>>> f = Filer(test_file)
>>> f.author
'Donald Duck'
If you want to have callable methods exactly like the one you propose for each attribute, I would one-up your proposal and not even have a method to begin with. You can actually generate the methods on the fly in __getattr__:
class Filer():
def __init__(self, file):
self.file = file
def __getattr__(self, name):
if name in fn_names:
index = fn_names[name]
def func(self):
print(self.file[index][0].split('=', 1)[1])
func.__name__ = func.__qualname__ = name
return func.__get__(self, type(self))
return super().__getattr__(name)
Calling __get__ is an extra step that makes the function behave as if it were a method of the class all along. It binds the function object to the instance, even through the function is not part of the class.
For example:
>>> f = Filer(test_file)
>>> f.get_author
<bound method get_author of <__main__.Filer object at 0x0000023E7A247748>>
>>> f.get_author()
'Donald Duck'
Consider closing over your keys and values -- note that you can see the below code running at https://ideone.com/qmoZCJ:
import sys
import inspect
test_file = [['Name=Jon Hancock'],
['Date=16.08.2020'],
['Author=Donald Duck']]
# intented method names
fn_names = {'get_name': 0, 'get_date': 1, 'get_author': 2}
class Filer():
def __init__(self, file):
self.file = file
def getter(key, val):
def _get_line(self):
return self.file[val][0].split('=')[1]
return _get_line
for key, val in fn_names.items():
setattr(Filer, key, getter(key, val))
f = Filer(test_file)
print("Author: ", f.get_author())
print("Date: ", f.get_date())
I'm not sure what's happening when I print my dictionary.
In Python 3, I have a dictionary of parse_blast objects called transSwiss. Each object's proteinID is the key with the entire object as the value.
I can print transSwiss in it's entirety and I can also print blasto.protein, but not when I combine them to get a dictionary value. I'm not sure what is happening when I use:
print(transSwiss[blasto.protein])
<__main__.parse_blast object at 0x000000373C5666A0>
Here is the code
class parse_blast(object):
def __init__(self, line):
#Strip end-of-line and split on tabs
self.fields = line.strip("\n").split("\t")
self.transcriptId, self.isoform = self.fields[0].split("|")
self.swissStuff = self.fields[1].split("|")
self.swissProtId = self.swissStuff[3]
self.percentId = self.fields[2]
def filterblast(self):
return float(self.percentId) > 95
class parse_matrix(object):
#Consider __init__ as a Constructor
def __init__(self, matrix_lines):
(self.protein,
self.Sp_ds,
self.Sp_hs,
self.Sp_log,
self.Sp_plat) = matrix_lines.strip("\n").split("\t")
def separate_tuples(one_tuple):
return "\t".join(one_tuple)
blastmap = map(parse_blast, blast_output.readlines())
filtered = filter(parse_blast.filterblast, blastmap)
matrixmap = map(parse_matrix, matrix_output.readlines()[1:])
transSwiss = {blasto.transcriptId:blasto for blasto in filtered}
for matrixo in matrixmap:
print(transSwiss[matrixo.protein])
Because your object is defined by you, you also need to tell python how you want it to print. You can do this by defining a function called "__str__" that returns how you want to print your object.
https://en.wikibooks.org/wiki/Python_Programming/Classes#str
There is large python project where one attribute of one class just have wrong value in some place.
It should be sqlalchemy.orm.attributes.InstrumentedAttribute, but when I run tests it is constant value, let's say string.
There is some way to run python program in debug mode, and run some check (if variable changed type) after each step throught line of code automatically?
P.S. I know how to log changes of attribute of class instance with help of inspect and property decorator. Possibly here I can use this method with metaclasses...
But sometimes I need more general and powerfull solution...
Thank you.
P.P.S. I need something like there: https://stackoverflow.com/a/7669165/816449, but may be with more explanation of what is going on in that code.
Well, here is a sort of slow approach. It can be modified for watching for local variable change (just by name). Here is how it works: we do sys.settrace and analyse the value of obj.attr each step. The tricky part is that we receive 'line' events (that some line was executed) before line is executed. So, when we notice that obj.attr has changed, we are already on the next line and we can't get the previous line frame (because frames aren't copied for each line, they are modified ). So on each line event I save traceback.format_stack to watcher.prev_st and if on the next call of trace_command value has changed, we print the saved stack trace to file. Saving traceback on each line is quite an expensive operation, so you'd have to set include keyword to a list of your projects directories (or just the root of your project) in order not to watch how other libraries are doing their stuff and waste cpu.
watcher.py
import traceback
class Watcher(object):
def __init__(self, obj=None, attr=None, log_file='log.txt', include=[], enabled=False):
"""
Debugger that watches for changes in object attributes
obj - object to be watched
attr - string, name of attribute
log_file - string, where to write output
include - list of strings, debug files only in these directories.
Set it to path of your project otherwise it will take long time
to run on big libraries import and usage.
"""
self.log_file=log_file
with open(self.log_file, 'wb'): pass
self.prev_st = None
self.include = [incl.replace('\\','/') for incl in include]
if obj:
self.value = getattr(obj, attr)
self.obj = obj
self.attr = attr
self.enabled = enabled # Important, must be last line on __init__.
def __call__(self, *args, **kwargs):
kwargs['enabled'] = True
self.__init__(*args, **kwargs)
def check_condition(self):
tmp = getattr(self.obj, self.attr)
result = tmp != self.value
self.value = tmp
return result
def trace_command(self, frame, event, arg):
if event!='line' or not self.enabled:
return self.trace_command
if self.check_condition():
if self.prev_st:
with open(self.log_file, 'ab') as f:
print >>f, "Value of",self.obj,".",self.attr,"changed!"
print >>f,"###### Line:"
print >>f,''.join(self.prev_st)
if self.include:
fname = frame.f_code.co_filename.replace('\\','/')
to_include = False
for incl in self.include:
if fname.startswith(incl):
to_include = True
break
if not to_include:
return self.trace_command
self.prev_st = traceback.format_stack(frame)
return self.trace_command
import sys
watcher = Watcher()
sys.settrace(watcher.trace_command)
testwatcher.py
from watcher import watcher
import numpy as np
import urllib2
class X(object):
def __init__(self, foo):
self.foo = foo
class Y(object):
def __init__(self, x):
self.xoo = x
def boom(self):
self.xoo.foo = "xoo foo!"
def main():
x = X(50)
watcher(x, 'foo', log_file='log.txt', include =['C:/Users/j/PycharmProjects/hello'])
x.foo = 500
x.goo = 300
y = Y(x)
y.boom()
arr = np.arange(0,100,0.1)
arr = arr**2
for i in xrange(3):
print 'a'
x.foo = i
for i in xrange(1):
i = i+1
main()
There's a very simple way to do this: use watchpoints.
Basically you only need to do
from watchpoints import watch
watch(your_object.attr)
That's it. Whenever the attribute is changed, it will print out the line that changed it and how it's changed. Super easy to use.
It also has more advanced features, for example, you can call pdb when the variable is changed, or use your own callback functions instead of print it to stdout.
A simpler way to watch for an object's attribute change (which can also be a module-level variable or anything accessible with getattr) would be to leverage hunter library, a flexible code tracing toolkit. To detect state changes we need a predicate which can look like the following:
import traceback
class MutationWatcher:
def __init__(self, target, attrs):
self.target = target
self.state = {k: getattr(target, k) for k in attrs}
def __call__(self, event):
result = False
for k, v in self.state.items():
current_value = getattr(self.target, k)
if v != current_value:
result = True
self.state[k] = current_value
print('Value of attribute {} has chaned from {!r} to {!r}'.format(
k, v, current_value))
if result:
traceback.print_stack(event.frame)
return result
Then given a sample code:
class TargetThatChangesWeirdly:
attr_name = 1
def some_nested_function_that_does_the_nasty_mutation(obj):
obj.attr_name = 2
def some_public_api(obj):
some_nested_function_that_does_the_nasty_mutation(obj)
We can instrument it with hunter like:
# or any other entry point that calls the public API of interest
if __name__ == '__main__':
obj = TargetThatChangesWeirdly()
import hunter
watcher = MutationWatcher(obj, ['attr_name'])
hunter.trace(watcher, stdlib=False, action=hunter.CodePrinter)
some_public_api(obj)
Running the module produces:
Value of attribute attr_name has chaned from 1 to 2
File "test.py", line 44, in <module>
some_public_api(obj)
File "test.py", line 10, in some_public_api
some_nested_function_that_does_the_nasty_mutation(obj)
File "test.py", line 6, in some_nested_function_that_does_the_nasty_mutation
obj.attr_name = 2
test.py:6 return obj.attr_name = 2
... return value: None
You can also use other actions that hunter supports. For instance, Debugger which breaks into pdb (debugger on an attribute change).
Try using __setattr__ to override the function that is called when an attribute assignment is attempted. Documentation for __setattr__
You can use the python debugger module (part of the standard library)
To use, just import pdb at the top of your source file:
import pdb
and then set a trace wherever you want to start inspecting the code:
pdb.set_trace()
You can then step through the code with n, and investigate the current state by running python commands.
def __setattr__(self, name, value):
if name=="xxx":
util.output_stack('xxxxx')
super(XXX, self).__setattr__(name, value)
This sample code helped me.
I am having some problem accessing class instances. I am calling the class from a procedure, name of instance is defined in some variable. I want the instance name to be created of that value and then want to access it, but when i access it is giving error. Can some one please help to resolve this issue.
class myclass:
def __init__(self,object):
self.name = object
def mydef():
global a1
b = "a1"
b = myclass(b)
mydef()
print a1.name
Second Problem:
In my actual script, I have to create a large number of such instances from this function (around 100). So defining their name as global would be painful, is there a way i could access those instances outside function without having to declare them as global.
Modification:
class myclass:
def __init__(self,object,typename):
self.name = object
self.typeid = typename
def mydef():
file_han = open(file,"r")
while True:
line = file_han.readline()
if not line:
break
start = line.find('"')
end = line.find('"',start+1)
string_f = line[start+1:end]
myclass(string_f,'a11')
mydef(file)
print def.name
print def.typeid
File Contents are :
a11 "def"
a11 "ghi"
a11 "eff"
Here's how I'd do it. I don't know why you're messing around with globals, if you'd care to explain, I'll update my answer.
class Myclass(object):
def __init__(self, name):
self.name = name
def mydef():
return Myclass("a1")
a1 = mydef()
print a1.name
Gather your instances in a list:
instances = []
for x in range(1000):
instances.append(Myclass("Instance {0}".format(x)))
print instance[42].name
Note the changes:
Class names should be capitalized
Use object as the base class of your classes (since python 2.2, but no longer necessary in 3.x)
Don't shadow the built-in object with your parameter name
Just use the string "a1" directly as a parameter instead of assigning it to a variable
Return something from the function instead of passing the result by global variable
RE: Comment
You haven't said anything about the format of these files, so I'll just give an example where the file to be read contains one class name per line, and nothing else:
def mydef(filename):
ret = []
with open(filename) as f:
for line in f:
# Call `strip` on line to remove newline and surrounding whitespace
ret.append(Myclass(line.strip()))
return ret
So if you have several files and wish to add all your instances from all your files to a large list, do it like this:
instances = []
for filename in ["myfile1", "myfile2", "myfile3"]:
instances.extend(mydef(filename))
RE: OP Edit
def mydef(filename):
ret = []
with open(filename, "r") as file_han:
for line in file_han:
string_f = line.split('"')[1]
ret.append(Myclass(string_f))
return ret
i = mydef("name_of_file")
RE: Comment
Oh, you want to access them by name. Then return a dict instead:
def mydef(filename):
ret = {}
with open(filename, "r") as file_han:
for line in file_han:
string_f = line.split('"')[1]
ret[string_f] = Myclass(string_f)
return ret
i = mydef("name_of_file")
print i["ghi"].name # should print "ghi"
RE: Comment
If I understand you correctly, you want to have it both ways -- index by both line number and name. Well then why don't you return both a list and a dictionary?
def mydef(filename):
d = {}
L = []
with open(filename, "r") as file_han:
for line in file_han:
string_f = line.split('"')[1]
instance = Myclass(string_f)
d[string_f] = instance
L.append(instance)
return L, d
L, d = mydef("name_of_file")
print d["ghi"].name
print L[3]
print L.index(d["ghi"])
You could use class as repository for your instances, for example
class Named(object):
def __init__(self,name):
self.name = name
def __new__(cls,name):
instance = super(type,cls).__new__(cls,name)
setattr(cls,name,instance)
return instance
def __repr__(self):
return 'Named[%s]'%self.name
Named('hello')
Named('x123')
Named('this is not valid attribute name, but also working')
print(Named.hello,Named.x123,getattr(Named,'this is not valid attribute name, but also working'))
I've got a piece of code which contains a for loop to draw things from an XML file;
for evoNode in node.getElementsByTagName('evolution'):
evoName = getText(evoNode.getElementsByTagName( "type")[0].childNodes)
evoId = getText(evoNode.getElementsByTagName( "typeid")[0].childNodes)
evoLevel = getText(evoNode.getElementsByTagName( "level")[0].childNodes)
evoCost = getText(evoNode.getElementsByTagName("costperlevel")[0].childNodes)
evolutions.append("%s x %s" % (evoLevel, evoName))
Currently it outputs into a list called evolutions as it says in the last line of that code, for this and several other for functions with very similar functionality I need it to output into a class instead.
class evolutions:
def __init__(self, evoName, evoId, evoLevel, evoCost)
self.evoName = evoName
self.evoId = evoId
self.evoLevel = evoLevel
self.evoCost = evoCost
How to create a series of instances of this class, each of which is a response from that for function? Or what is a core practical solution? This one doesn't really need the class but one of the others really does.
A list comprehension might be a little cleaner. I'd also move the parsing logic to the constructor to clean up the implemenation:
class Evolution:
def __init__(self, node):
self.node = node
self.type = property("type")
self.typeid = property("typeid")
self.level = property("level")
self.costperlevel = property("costperlevel")
def property(self, prop):
return getText(self.node.getElementsByTagName(prop)[0].childNodes)
evolutionList = [Evolution(evoNode) for evoNode in node.getElementsByTagName('evolution')]
Alternatively, you could use map:
evolutionList = map(Evolution, node.getElementsByTagName('evolution'))
for evoNode in node.getElementsByTagName('evolution'):
evoName = getText(evoNode.getElementsByTagName("type")[0].childNodes)
evoId = getText(evoNode.getElementsByTagName("typeid")[0].childNodes)
evoLevel = getText(evoNode.getElementsByTagName("level")[0].childNodes)
evoCost = getText(evoNode.getElementsByTagName("costperlevel")[0].childNodes)
temporaryEvo = Evolutions(evoName, evoId, evoLevel, evoCost)
evolutionList.append(temporaryEvo)
# Or you can go with the 1 liner
evolutionList.append(Evolutions(evoName, evoId, evoLevel, evoCost))
I renamed your list because it shared the same name as your class and was confusing.