I have a class which I want to use to extract data from a text file (already parsed) and I want do so using dynamically created class methods, because otherwise there would be a lot of repetitive code. Each created class method shall be asociated with a specific line of the text file, e.g. '.get_name()' --> read a part of 0th line of text file.
My idea was to use a dictionary for the 'to-be-created' method names and corresponding line.
import sys
import inspect
test_file = [['Name=Jon Hancock'],
['Date=16.08.2020'],
['Author=Donald Duck']]
# intented method names
fn_names = {'get_name': 0, 'get_date': 1, 'get_author': 2}
class Filer():
def __init__(self, file):
self.file = file
def __get_line(cls):
name = sys._getframe().f_code.co_name
line = fn_names[name] # <-- causes error because __get_line is not in fn_names
print(sys._getframe().f_code.co_name) # <-- '__get_line'
print(inspect.currentframe().f_code.co_name) # <-- '__get_line'
return print(cls.file[line][0].split('=')[1])
for key, val in fn_names.items():
setattr(Filer, key, __get_line)
f = Filer(test_file)
f.get_author()
f.get_date()
When I try to access the method name to link the method to the designated line in the text file, I do get an error because the method name is always '__get_line' instead of e.g. 'get_author' (what I had hoped for).
Another way how I thought to solve this was to make '__get_line' accepting an additional argument (line) and set it by passing the val during 'the setattr()' as shown below:
def __get_line(cls, line):
return print(cls.file[line][0].split('=')[1])
and
for key, val in fn_names.items():
setattr(Filer, key, __get_line(val))
however, then Python complains that 1 argument (line) is missing.
Any ideas how to solve that?
I would propose a much simpler solution, based on some assumptions. Your file appears to consist of key-value pairs. You are choosing to map the line number to a function that processes the right hand side of the line past the = symbol. Python does not conventionally use getters. Attributes are much nicer and easier to use. You can have getter-like functionality by using property objects, but you really don't need that here.
class Filer():
def __init__(self, file):
self.file = file
for line in file:
name, value = line[0].split('=', 1)
setattr(self, name.lower(), value)
That's all you need. Now you can use the result:
>>> f = Filer(test_file)
>>> f.author
'Donald Duck'
If you want to have callable methods exactly like the one you propose for each attribute, I would one-up your proposal and not even have a method to begin with. You can actually generate the methods on the fly in __getattr__:
class Filer():
def __init__(self, file):
self.file = file
def __getattr__(self, name):
if name in fn_names:
index = fn_names[name]
def func(self):
print(self.file[index][0].split('=', 1)[1])
func.__name__ = func.__qualname__ = name
return func.__get__(self, type(self))
return super().__getattr__(name)
Calling __get__ is an extra step that makes the function behave as if it were a method of the class all along. It binds the function object to the instance, even through the function is not part of the class.
For example:
>>> f = Filer(test_file)
>>> f.get_author
<bound method get_author of <__main__.Filer object at 0x0000023E7A247748>>
>>> f.get_author()
'Donald Duck'
Consider closing over your keys and values -- note that you can see the below code running at https://ideone.com/qmoZCJ:
import sys
import inspect
test_file = [['Name=Jon Hancock'],
['Date=16.08.2020'],
['Author=Donald Duck']]
# intented method names
fn_names = {'get_name': 0, 'get_date': 1, 'get_author': 2}
class Filer():
def __init__(self, file):
self.file = file
def getter(key, val):
def _get_line(self):
return self.file[val][0].split('=')[1]
return _get_line
for key, val in fn_names.items():
setattr(Filer, key, getter(key, val))
f = Filer(test_file)
print("Author: ", f.get_author())
print("Date: ", f.get_date())
Related
Will use following example to explain.
Existing python file (a.py) contains one class:
class A:
def method1(self, par1, par2='e'):
# some code here
pass
def method2(self, parA):
# some code here
pass
def method3(self, a, b, c):
# lots of code here
pass
def anothermethod(self):
pass
if __name__ == '__main__':
A().anothermethod()
Now, there is a need to create another py file (b.py), which would contain subclass (class B) of class A.
And there is a need to have all the methods included (all inherited from parent class), but without
implementation in it. Result might look like:
class B(A):
def method1(self, par1, par2='e'):
# empty here; ready to override
pass
def method2(self, parA):
# empty here; ready to override
pass
def method3(self, a, b, c):
# empty here; ready to override
pass
def anothermethod(self):
# empty here; ready to override
pass
if __name__ == '__main__':
B().anothermethod()
Having described the example, the question is: how could one generate last mentioned (skeleton-like) py file? So that after generating you can just open generated file and start right away with filling specific implementation.
There must be a shorter way, 1-2 line solution. Maybe it is already solvable by some existing functionality within modules already provided by Python (Python 3)?
Edit (2018 Mar 14). Thank you https://stackoverflow.com/a/49152537/4958287 (though was looking for short and already existing solution here). Will have to settle with longer solution for now -- will include its rough version here, maybe it would be helpful to someone else:
import inspect
from a import A
def construct_skeleton_subclass_from_parent(subcl_name, parent_cl_obj):
"""
subcl_name : str
Name for subclass and
file to be generated.
parent_cl_obj : obj (of any class to create subclass for)
Object of parent class.
"""
lines = []
subcl_name = subcl_name.capitalize()
parent_cl_module_name = parent_cl_obj.__class__.__module__
parent_cl_name = parent_cl_obj.__class__.__name__
lines.append('from {} import {}'.format(parent_cl_module_name, parent_cl_name))
lines.append('')
lines.append('class {}({}):'.format(subcl_name, parent_cl_name))
for name, method in inspect.getmembers(parent_cl_obj, predicate=inspect.ismethod):
args = inspect.signature(method)
args_others = str(args).strip('()').strip()
if len(args_others) == 0:
lines.append(' def {}(self):'.format(name))
else:
lines.append(' def {}(self, {}):'.format(name, str(args).strip('()')))
lines.append(' pass')
lines.append('')
#...
#lines.append('if __name__ == \'__main__\':')
#lines.append(' ' + subcl_name + '().anothermethod()')
#...
with open(subcl_name.lower() + '.py', 'w') as f:
for c in lines:
f.write(c + '\n')
a_obj = A()
construct_skeleton_subclass_from_parent('B', a_obj)
Get the list of methods and each of their signatures using the inspect module:
import a
import inspect
for name, method in inspect.getmembers(a.A, predicate=inspect.ismethod):
args = inspect.signature(method)
print(" def {}({}):".format(name, args))
print(" pass")
print()
I'm not sure what's happening when I print my dictionary.
In Python 3, I have a dictionary of parse_blast objects called transSwiss. Each object's proteinID is the key with the entire object as the value.
I can print transSwiss in it's entirety and I can also print blasto.protein, but not when I combine them to get a dictionary value. I'm not sure what is happening when I use:
print(transSwiss[blasto.protein])
<__main__.parse_blast object at 0x000000373C5666A0>
Here is the code
class parse_blast(object):
def __init__(self, line):
#Strip end-of-line and split on tabs
self.fields = line.strip("\n").split("\t")
self.transcriptId, self.isoform = self.fields[0].split("|")
self.swissStuff = self.fields[1].split("|")
self.swissProtId = self.swissStuff[3]
self.percentId = self.fields[2]
def filterblast(self):
return float(self.percentId) > 95
class parse_matrix(object):
#Consider __init__ as a Constructor
def __init__(self, matrix_lines):
(self.protein,
self.Sp_ds,
self.Sp_hs,
self.Sp_log,
self.Sp_plat) = matrix_lines.strip("\n").split("\t")
def separate_tuples(one_tuple):
return "\t".join(one_tuple)
blastmap = map(parse_blast, blast_output.readlines())
filtered = filter(parse_blast.filterblast, blastmap)
matrixmap = map(parse_matrix, matrix_output.readlines()[1:])
transSwiss = {blasto.transcriptId:blasto for blasto in filtered}
for matrixo in matrixmap:
print(transSwiss[matrixo.protein])
Because your object is defined by you, you also need to tell python how you want it to print. You can do this by defining a function called "__str__" that returns how you want to print your object.
https://en.wikibooks.org/wiki/Python_Programming/Classes#str
Hi folks I am experimenting with Python (I found pygame.org and wanted to play around) and I am trying to read some settings from a configuration file. I want to be able to change stats on the fly. (So if I wanted to change how hard a fighter hits or how fast a wizard runs then I'd be able to do that.) I was hoping to be able to read from a list and create an attribute for each instance in the list basically this:
for stat in Character.stats:
self.stat = parser.get(self.char_class, stat)
What ends up happening is there is an object with an attribute names 'stat' that contains the last value assigned. What I would LIKE to happen is to have an attribute created for each item in the list, and then get assigned the related value from the config file.
here is more code for context:
class Character(object):
stats = ["level_mod",
"power",
"speed",
"hit",
"evade",
"magic",
"stamina",
"magic_defense",
"intelligence"]
def __init__(self, name, rpg_id):
self.name = name
self.rpg_id = rpg_id
self.__setStats()
def __setStats(self):
parser = SafeConfigParser()
parser.read('char_config.cfg')
for stat in Character.stats:
self.stat = parser.get(self.char_class, stat)
Thanks for your time!
You can use, setattr:
for stat in Character.stats:
setattr(self, stat, parser.get(self.char_class, stat))
Or manually access dict
for stat in Character.stats:
self.__dict__[stat] = parser.get(self.char_class, stat))
You want setattr(obj, attrname, value)
You better re-design that part of the game by adding a Stats class.
class Stats:
STATS = ["level_mod",
"power",
"speed",
"hit",
"evade",
"magic",
"stamina",
"magic_defense",
"intelligence"]
def __init__(self, conf_file=None):
self.__stats = {}
if conf_file is not None:
self.loads_stats_from_file(conf_file)
def load_stats_from_file(self, conf_file):
"""
Here add the pairs <stat_name>:<value>
to the self.__stats dict. For that just parse the config
file like before.
"""
pass
def get_stat(self, stat_name):
return self.__stats[stat_name]
def set_stat(self, stat_name, value):
self.__stats[stat_name] = value
Then you can add a Stats instance to your Character.
class Character(object):
def __init__(self, name, rpg_id):
self.stats = Stats("char_config.cfg")
self.name = name
self.rpg_id = rpg_id
This way you improve usability and decouple the Stats and Character logics. And besides, your problem is reduced from "Adding attributes to an object" to "Adding items to a dictionary".
Note: I see that I need to more clearly work out what it is that I want each property/descriptor/class/method to do before I ask how to do it! I don't think my question can be answered at this time. Thanks all for helping me out.
Thanks to icktoofay and BrenBarn, I'm starting to understand discriptors and properties, but now I have a slightly harder question to ask:
I see now how these work:
class Blub(object):
def __get__(self, instance, owner):
print('Blub gets ' + instance._blub)
return instance._blub
def __set__(self, instance, value):
print('Blub becomes ' + value)
instance._blub = value
class Quish(object):
blub = Blub()
def __init__(self, value):
self.blub = value
And how a = Quish('one') works (produces "Blub becomes one") but take a gander at this code:
import os
import glob
class Index(object):
def __init__(self, dir=os.getcwd()):
self.name = dir #index name is directory of indexes
# index is the list of indexes
self.index = glob.glob(os.path.join(self.name, 'BatchStarted*'))
# which is the pointer to the index (index[which] == BatchStarted_12312013_115959.txt)
self.which = 0
# self.file = self.File(self.index[self.which])
def get(self):
return self.index[self.which]
def next(self):
self.which += 1
if self.which < len(self.index):
return self.get()
else:
# loop back to the first
self.which = 0
return None
def back(self):
if self.which > 0:
self.which -= 1
return self.get()
class File(object):
def __init__(self, file):
# if the file exists, we'll use it.
if os.path.isfile(file):
self.name = file
# otherwise, our name is none and we return.
else:
self.name = None
return None
# 'file' attribute is the actual file object
self.file = open(self.name, 'r')
self.line = Lines(self.file)
class Lines(object):
# pass through the actual file object (not filename)
def __init__(self, file):
self.file = file
# line is the list if this file's lines
self.line = self.file.readlines()
self.which = 0
self.extension = Extension(self.line[self.which])
def __get__(self):
return self.line[self.which]
def __set__(self, value):
self.which = value
def next(self):
self.which += 1
return self.__get__()
def back(self):
self.which -= 1
return self.__get__()
class Extension(object):
def __init__(self, lineStr):
# check to make sure a string is passed
if lineStr:
self.lineStr = lineStr
self.line = self.lineStr.split('|')
self.pathStr = self.line[0]
self.path = self.pathStr.split('\\')
self.fileStr = self.path[-1]
self.file = self.fileStr.split('.')
else:
self.lineStr = None
def __get__(self):
self.line = self.lineStr.split('|')
self.pathStr = self.line[0]
self.path = self.pathStr.split('\\')
self.fileStr = self.path[-1]
self.file = self.fileStr.split('.')
return self.file[-1]
def __set__(self, ext):
self.file[-1] = ext
self.fileStr = '.'.join(self.file)
self.path[-1] = fileStr
self.pathStr = '\\'.join(self.path)
self.line[0] = self.pathStr
self.lineStr = '|'.join(self.line)
Firstly, there may be some typos in here because I've been working on it and leaving it half-arsed. That's not my point. My point is that in icktoofay's example, nothing gets passed to Blub(). Is there any way to do what I'm doing here, that is set some "self" attributes and after doing some processing, taking that and passing it to the next class? Would this be better suited for a property?
I would like to have it so that:
>>> i = Index() # i contains list of index files
>>> f = File(i.get()) # f is now one of those files
>>> f.line
'\\\\server\\share\\folder\\file0.txt|Name|Sean|Date|10-20-2000|Type|1'
>>> f.line.extension
'txt'
>>> f.line.extension = 'rtf'
>>> f.line
'\\\\server\\share\\folder\\file0.rtf|Name|Sean|Date|10-20-2000|Type|1'
You can do that, but the issue there is less about properties/descriptors and more about creating classes that give the behavior you want.
So, when you do f.line, that is some object. When you do f.line.extension, that is doing (f.line).extension --- that is, it first evalautes f.line and then gets the extension attribute of whatever f.line is.
The important thing here is that f.line cannot know whether you are later going to try to access its extension. So you can't have f.line do one thing for "plain" f.line and another thing for f.line.extension. The f.line part has to be the same in both, and the extension part can't change that.
The solution for what you seem to want to do is to make f.line return some kind of object that in some way looks or works like a string, but also allows setting attributes and updating itself accordingly. Exactly how you do this depends on how much you need f.lines to behave like a string and how much you need it to do other stuff. Basically you need f.line to be a "gatekeeper" object that handles some operations by acting like a string (e.g., you apparently want it to display as a string), and handles other objects in custom ways (e.g., you apparently want to be able to set an extension attribute on it and have that update its contents).
Here's a simplistic example:
class Line(object):
def __init__(self, txt):
self.base, self.extension = txt.split('.')
def __str__(self):
return self.base + "." + self.extension
Now you can do:
>>> line = Line('file.txt')
>>> print line
file.txt
>>> line.extension
'txt'
>>> line.extension = 'foo'
>>> print line
file.foo
However, notice that I did print line, not just line. By writing a __str__ method, I defined the behavior that happens when you do print line. But if you evaluate it "raw" without printing it, you'll see it's not really a string:
>>> line
<__main__.Line object at 0x000000000233D278>
You could override this behavior as well (by defining __repr__), but do you want to? That depends on how you want to use line. The point is that you need to decide what you want your line to do in what situations, and then craft a class that does that.
There is large python project where one attribute of one class just have wrong value in some place.
It should be sqlalchemy.orm.attributes.InstrumentedAttribute, but when I run tests it is constant value, let's say string.
There is some way to run python program in debug mode, and run some check (if variable changed type) after each step throught line of code automatically?
P.S. I know how to log changes of attribute of class instance with help of inspect and property decorator. Possibly here I can use this method with metaclasses...
But sometimes I need more general and powerfull solution...
Thank you.
P.P.S. I need something like there: https://stackoverflow.com/a/7669165/816449, but may be with more explanation of what is going on in that code.
Well, here is a sort of slow approach. It can be modified for watching for local variable change (just by name). Here is how it works: we do sys.settrace and analyse the value of obj.attr each step. The tricky part is that we receive 'line' events (that some line was executed) before line is executed. So, when we notice that obj.attr has changed, we are already on the next line and we can't get the previous line frame (because frames aren't copied for each line, they are modified ). So on each line event I save traceback.format_stack to watcher.prev_st and if on the next call of trace_command value has changed, we print the saved stack trace to file. Saving traceback on each line is quite an expensive operation, so you'd have to set include keyword to a list of your projects directories (or just the root of your project) in order not to watch how other libraries are doing their stuff and waste cpu.
watcher.py
import traceback
class Watcher(object):
def __init__(self, obj=None, attr=None, log_file='log.txt', include=[], enabled=False):
"""
Debugger that watches for changes in object attributes
obj - object to be watched
attr - string, name of attribute
log_file - string, where to write output
include - list of strings, debug files only in these directories.
Set it to path of your project otherwise it will take long time
to run on big libraries import and usage.
"""
self.log_file=log_file
with open(self.log_file, 'wb'): pass
self.prev_st = None
self.include = [incl.replace('\\','/') for incl in include]
if obj:
self.value = getattr(obj, attr)
self.obj = obj
self.attr = attr
self.enabled = enabled # Important, must be last line on __init__.
def __call__(self, *args, **kwargs):
kwargs['enabled'] = True
self.__init__(*args, **kwargs)
def check_condition(self):
tmp = getattr(self.obj, self.attr)
result = tmp != self.value
self.value = tmp
return result
def trace_command(self, frame, event, arg):
if event!='line' or not self.enabled:
return self.trace_command
if self.check_condition():
if self.prev_st:
with open(self.log_file, 'ab') as f:
print >>f, "Value of",self.obj,".",self.attr,"changed!"
print >>f,"###### Line:"
print >>f,''.join(self.prev_st)
if self.include:
fname = frame.f_code.co_filename.replace('\\','/')
to_include = False
for incl in self.include:
if fname.startswith(incl):
to_include = True
break
if not to_include:
return self.trace_command
self.prev_st = traceback.format_stack(frame)
return self.trace_command
import sys
watcher = Watcher()
sys.settrace(watcher.trace_command)
testwatcher.py
from watcher import watcher
import numpy as np
import urllib2
class X(object):
def __init__(self, foo):
self.foo = foo
class Y(object):
def __init__(self, x):
self.xoo = x
def boom(self):
self.xoo.foo = "xoo foo!"
def main():
x = X(50)
watcher(x, 'foo', log_file='log.txt', include =['C:/Users/j/PycharmProjects/hello'])
x.foo = 500
x.goo = 300
y = Y(x)
y.boom()
arr = np.arange(0,100,0.1)
arr = arr**2
for i in xrange(3):
print 'a'
x.foo = i
for i in xrange(1):
i = i+1
main()
There's a very simple way to do this: use watchpoints.
Basically you only need to do
from watchpoints import watch
watch(your_object.attr)
That's it. Whenever the attribute is changed, it will print out the line that changed it and how it's changed. Super easy to use.
It also has more advanced features, for example, you can call pdb when the variable is changed, or use your own callback functions instead of print it to stdout.
A simpler way to watch for an object's attribute change (which can also be a module-level variable or anything accessible with getattr) would be to leverage hunter library, a flexible code tracing toolkit. To detect state changes we need a predicate which can look like the following:
import traceback
class MutationWatcher:
def __init__(self, target, attrs):
self.target = target
self.state = {k: getattr(target, k) for k in attrs}
def __call__(self, event):
result = False
for k, v in self.state.items():
current_value = getattr(self.target, k)
if v != current_value:
result = True
self.state[k] = current_value
print('Value of attribute {} has chaned from {!r} to {!r}'.format(
k, v, current_value))
if result:
traceback.print_stack(event.frame)
return result
Then given a sample code:
class TargetThatChangesWeirdly:
attr_name = 1
def some_nested_function_that_does_the_nasty_mutation(obj):
obj.attr_name = 2
def some_public_api(obj):
some_nested_function_that_does_the_nasty_mutation(obj)
We can instrument it with hunter like:
# or any other entry point that calls the public API of interest
if __name__ == '__main__':
obj = TargetThatChangesWeirdly()
import hunter
watcher = MutationWatcher(obj, ['attr_name'])
hunter.trace(watcher, stdlib=False, action=hunter.CodePrinter)
some_public_api(obj)
Running the module produces:
Value of attribute attr_name has chaned from 1 to 2
File "test.py", line 44, in <module>
some_public_api(obj)
File "test.py", line 10, in some_public_api
some_nested_function_that_does_the_nasty_mutation(obj)
File "test.py", line 6, in some_nested_function_that_does_the_nasty_mutation
obj.attr_name = 2
test.py:6 return obj.attr_name = 2
... return value: None
You can also use other actions that hunter supports. For instance, Debugger which breaks into pdb (debugger on an attribute change).
Try using __setattr__ to override the function that is called when an attribute assignment is attempted. Documentation for __setattr__
You can use the python debugger module (part of the standard library)
To use, just import pdb at the top of your source file:
import pdb
and then set a trace wherever you want to start inspecting the code:
pdb.set_trace()
You can then step through the code with n, and investigate the current state by running python commands.
def __setattr__(self, name, value):
if name=="xxx":
util.output_stack('xxxxx')
super(XXX, self).__setattr__(name, value)
This sample code helped me.