Related
Thanks everyone for your help so far. I've narrowed it down a bit. If you look at HERE in both the script and the class, and run the script, you'll see what is going on.
The ADD line print "789 789"
when it should be printing "456 789"
What appears to be happening, is in new the class is detecting the type of the incoming argument. However if the incoming object, has the same type as the constructor it appears to be paging the incoming object, into itself (at the class level) instead of returning the old object. That is the only thing I can think of that would cause 456 to get creamed.
So how do you detect something that is the same type of a class, within a constructor and decide NOT to page that data into the class memory space, but instead return the previously constructed object?
import sys
import math
class Foo():
# class level property
num = int(0)
#
# Python Instantiation Customs:
#
# Processing polymorphic input new() MUST return something or
# an object?, but init() cannot return anything. During runtime
# __new__ is running at the class level, while init is running
# at the instance level.
#
def __new__(self,*arg):
print ("arg type: ", type(arg[0]).__name__)
### functionally the same as isinstance() below
#
# if (type(arg[0]).__name__) == "type":
# if arg[0].__name__ == "Foo":
# print ("\tinput was a Foo")
# return arg[0] # objects of same type intercede
### HERE <-------------------------------------
#
# this creams ALL instances, because since we are a class
# the properties of the incoming object, seem to overwride
# the class, rather than exist as a separate data structure.
if (isinstance(arg[0], Foo)):
print ("\tinput was a Foo")
return arg[0] # objects of same type intercede
elif (type(arg[0]).__name__) == "int":
print ("\tinput was an int")
self.inum = int(arg[0]) # integers store
return self
elif (type(arg[0]).__name__) == "str":
print ("\tinput was a str")
self.inum = int(arg[0]) # strings become integers
return self
return self
def __init__(self,*arg):
pass
#
# because if I can do collision avoidance, I can instantiate
# inside overloaded operators:
#
def __add__(self,*arg):
print ("add operator overload")
# no argument returns self
if not arg:
return self
# add to None or zero return self
if not arg[0]:
return self
knowntype = Foo.Foo(arg[0])
# add to unknown type returns False
if not knowntype:
return knowntype
# both values are calculable, calculate and return a Foo
typedresult = (self.inum + knowntype.inum)
return Foo.Foo(typedresult)
def __str__(self): # return a stringified int or empty string
# since integers don't have character length,
# this tests the value, not the existence of:
if self.inum:
return str(self.inum)
# so the property could still be zero and we have to
# test again for no reason.
elif self.inum == 0:
return str(self.inum)
# return an empty str if nothing is defined.
return str("")
testfoo.py:
#! /usr/bin/python
import sys
import Foo
# A python class is not transparent like in perl, it is an object
# with unconditional inheritance forced on all instances that share
# the same name.
classhandle = Foo.Foo
# The distinction between the special class object, and instance
# objects is implicitly defined by whether there is a passed value at
# constructor time. The following therefore does not work.
# classhandle = Foo.Foo()
# but we can still write and print from the class, and see it propagate,
# without having any "object" memory allocated.
print ("\nclasshandle: ", classhandle)
print ("classhandle classname: ", classhandle.__name__) # print the classname
print ("class level num: ", classhandle.num) # print the default num
classhandle.classstring = "fdsa" # define an involuntary value for all instances
print ("\n")
# so now we can create some instances with passed properties.
instance1 = Foo.Foo(int(123)) #
print ("\ninstance1: ", instance1)
print ("involuntary property derived from special class memory space: ", instance1.classstring)
print ("instance property from int: ", instance1.inum)
print ("\n")
instance2 = Foo.Foo(str("456"))
print ("\ninstance2: ", instance2)
print ("instance2 property from int: ", instance2.inum)
#
# instance3 stands for (shall we assume) some math that happened a
# thousand lines ago in a class far far away. We REALLY don't
# want to go chasing around to figure out what type it could possibly
# be, because it could be polymorphic itself. Providing a black box so
# that you don't have to do that, is after all, the whole point OOP.
#
print ("\npretend instance3 is unknowningly already a Foo")
instance3 = Foo.Foo(str("789"))
## So our class should be able to handle str,int,Foo types at constructor time.
print ("\ninstance4 should be a handle to the same memory location as instance3")
instance4 = Foo.Foo(instance3) # SHOULD return instance3 on type collision
# because if it does, we should be able to hand all kinds of garbage to
# overloaded operators, and they should remain type safe.
# HERE <-----------------------------
#
# the creation of instance4, changes the instance properties of instance2:
# below, the instance properties inum, are now both "789".
print ("ADDING: ", instance2.inum, " ", instance4.inum)
# instance6 = instance2 + instance4 # also should be a Foo object
# instance5 = instance4 + int(549) # instance5 should be a Foo object.
How do I, at constructor time, return a non-new object?
By overriding the constructor method, __new__, not the initializer method, __init__.
The __new__ method constructs an instance—normally by calling the super's __new__, which eventually gets up to object.__new__, which does the actual allocation and other under-the-covers stuff, but you can override that to return a pre-existing value.
The __init__ method is handed a value that's already been constructed by __new__, so it's too late for it to not construct that value.
Notice that if Foo.__new__ returns a Foo instance (whether a newly-created one or an existing one), Foo.__init__ will be called on it. So, classes that override __new__ to return references to existing objects generally need an idempotent __init__—typically, you just don't override __init__ at all, and do all of your initialization inside __new__.
There are lots of examples of trivial __new__ methods out there, but let's show one that actually does a simplified version of what you're asking for:
class Spam:
_instances = {}
def __new__(cls, value):
if value not in cls._instances:
cls._instances[value] = super().__new__(cls)
cls._instances[value].value = value
return cls._instances[value]
Now:
>>> s1 = Spam(1)
>>> s2 = Spam(2)
>>> s3 = Spam(1)
>>> s1 is s2
False
>>> s1 is s3
True
Notice that I made sure to use super rather than object, and cls._instances1 rather than Spam._instances. So:
>>> class Eggs(Spam):
... pass
>>> e4 = Eggs(4)
>>> Spam(4)
<__main__.Eggs at 0x12650d208>
>>> Spam(4) is e4
True
>>> class Cheese(Spam):
... _instances = {}
>>> c5 = Cheese(5)
>>> Spam(5)
<__main__.Spam at 0x126c28748>
>>> Spam(5) is c5
False
However, it may be a better option to use a classmethod alternate constructor, or even a separate factory function, rather than hiding this inside the __new__ method.
For some types—like, say, a simple immutable container like tuple—the user has no reason to care whether tuple(…) returns a new tuple or an existing one, so it makes sense to override the constructor. But for some other types, especially mutable ones, it can lead to confusion.
The best test is to ask yourself whether this (or similar) would be confusing to your users:
>>> f1 = Foo(x)
>>> f2 = Foo(x)
>>> f1.spam = 1
>>> f2.spam = 2
>>> f1.spam
2
If that can't happen (e.g., because Foo is immutable), override __new__.
If that exactly what users would expect (e.g., because Foo is a proxy to some object that has the actual spam, and two proxies to the same object had better see the same spam), probably override __new__.
If it would be confusing, probably don't override __new__.
For example, with a classmethod:
>>> f1 = Foo.from_x(x)
>>> f2 = Foo.from_x(x)
… it's a lot less likely to be surprising if f1 is f2 turns out to be true.
1. Even though you define __new__ like an instance method, and its body looks like a class method, it's actually a static method, that gets passed the class you're trying to construct (which will be Spam or a subclass of Spam) as an ordinary first parameter, with the constructor arguments (and keyword arguments) passed after that.
Thanks everyone who helped! This answer was saught out to understand how to refactor an existing program that was already written, but that was having scalability problems. The following is the completed working example. What it demonstrates is:
The ability to test incoming types and avoid unneccessary object duplication at constructor time, given incoming types that are both user-defined and built-in. The ability to construct on the fly from a redefined operator or method. These capabilities are neccessary for writing scalable supportable API code. YMMV.
Foo.py
import sys
import math
class Foo():
# class level property
num = int(0)
#
# Python Instantiation Customs:
#
# Processing polymorphic input new() MUST return something or
# an object, but init() MAYNOT return anything. During runtime
# __new__ is running at the class level, while __init__ is
# running at the instance level.
#
def __new__(cls,*arg):
print ("arg type: ", type(arg[0]).__name__)
# since we are functioning at the class level, type()
# is reaching down into a non-public namespace,
# called "type" which is presumably something that
# all objects are ultimately derived from.
# functionally this is the same as isinstance()
if (type(arg[0]).__name__) == "Foo":
fooid = id(arg[0])
print ("\tinput was a Foo: ", fooid)
return arg[0] # objects of same type intercede
# at the class level here, we are calling into super() for
# the constructor. This is presumably derived from the type()
# namespace, which when handed a classname, makes one of
# whatever it was asked for, rather than one of itself.
elif (type(arg[0]).__name__) == "int":
self = super().__new__(cls)
self.inum = int(arg[0]) # integers store
fooid = id(self)
print ("\tinput was an int: ", fooid)
return (self)
elif (type(arg[0]).__name__) == "str":
self = super().__new__(cls)
self.inum = int(arg[0]) # strings become integers
fooid = id(self)
print ("\tinput was a str: ", fooid)
return (self)
# def __init__(self,*arg):
# pass
#
# because if I can do collision avoidance, I can instantiate
# inside overloaded operators:
#
def __add__(self,*arg):
argtype = type(arg[0]).__name__
print ("add overload in class:", self.__class__)
if argtype == "Foo" or argtype == "str" or argtype == "int":
print ("\tfrom a supported type")
# early exit for zero
if not arg[0]:
return self
# localized = Foo.Foo(arg[0])
# FAILS: AttributeError: type object 'Foo' has no attribute 'Foo'
# You can't call a constructor the same way from inside and outside
localized = Foo(arg[0])
print ("\tself class: ", self.__class__)
print ("\tself number: ", self.inum)
print ()
print ("\tlocalized class: ", localized.__class__)
print ("\tlocalized number: ", localized.inum)
print ()
answer = (self.inum + localized.inum)
answer = Foo(answer)
print ("\tanswer class:", answer.__class__)
print ("\tanswer sum result:", answer.inum)
return answer
assert(0), "Foo: cannot add an unsupported type"
def __str__(self): # return a stringified int or empty string
# Allow the class to stringify as if it were an int.
if self.inum >= 0:
return str(self.inum)
testfoo.py
#! /usr/bin/python
import sys
import Foo
# A python class is not transparent like in perl, it is an object
# with unconditional inheritance forced on all instances that share
# the same name.
classhandle = Foo.Foo
# The distinction between the special class object, and instance
# objects is implicitly defined by whether there is a passed value at
# constructor time. The following therefore does not work.
# classhandle = Foo.Foo()
# but we can still write and print from the class, and see it propagate,
# without having any "object" memory allocated.
print ("\nclasshandle: ", classhandle)
print ("classhandle classname: ", classhandle.__name__) # print the classname
print ("class level num: ", classhandle.num) # print the default num
classhandle.classstring = "fdsa" # define an involuntary value for all instances
print ("\n")
# so now we can create some instances with passed properties.
instance1 = Foo.Foo(int(123)) #
print ("\ninstance1: ", instance1)
print ("involuntary property derived from special class memory space: ", instance1.classstring)
print ("instance property from int: ", instance1.inum)
print ("\n")
instance2 = Foo.Foo(str("456"))
print ("\ninstance2: ", instance2)
print ("instance2 property from int: ", instance2.inum)
#
# instance3 stands for (shall we assume) some math that happened a
# thousand lines ago in a class far far away. We REALLY don't
# want to go chasing around to figure out what type it could possibly
# be, because it could be polymorphic itself. Providing a black box so
# that you don't have to do that, is after all, the whole point OOP.
#
print ("\npretend instance3 is unknowningly already a Foo\n")
instance3 = Foo.Foo(str("789"))
## So our class should be able to handle str,int,Foo types at constructor time.
print ("\ninstance4 should be a handle to the same memory location as instance3\n")
instance4 = Foo.Foo(instance3) # SHOULD return instance3 on type collision
print ("instance4: ", instance4)
# because if it does, we should be able to hand all kinds of garbage to
# overloaded operators, and they should remain type safe.
# since we are now different instances these are now different:
print ("\nADDING:_____________________\n", instance2.inum, " ", instance4.inum)
instance5 = instance4 + int(549) # instance5 should be a Foo object.
print ("\n\tAdd instance4, 549, instance5: ", instance4, " ", int(549), " ", instance5, "\n")
instance6 = instance2 + instance4 # also should be a Foo object
print ("\n\tAdd instance2, instance4, instance6: ", instance2, " ", instance4, " ", instance6, "\n")
print ("stringified instance6: ", str(instance6))
The following code:
class ParentModel(models.Model):
pass
class ChildA(ChildB):
pass
class ChildB(ParentModel):
pass
Obviously fails with the message.
NameError: name "ChildB" is not defined
Is there anyway to get around this issue, without actually reordering the class definitions? (The code is auto-generated, about 45K lines, and the order of classes is random).
Perfectionists look away!!
This is a workaround (hack); the solution would be to solve the incorrect declaration order.
WARNING: This is extremely daft.
Concept:
Imagine a namespace where anything can exist. Literally anything that is asked of it. Not the smartest thing usually but out-of-order declaration isn't smart either, so why not?
The key problem of out-of-sequence classes is that dependent classes were being defined before their dependencies, the base classes. At that point of evaluation, the base classes are undefined resulting in a NameError.
Wrapping each class in try except statements would take as much effort as rewriting the module anyway, so that can be dismissed out of hand.
A more efficient (in terms of programmer time) means of suppressing NameError must be used. This can be achieved by making the namespace totally permissible, as in, if a lookup object doesn't exist, it should be created thereby avoiding a NameError. This is the obvious danger of this approach as a lookup becomes a creation.
Implementation:
Namespaces in Python are dictionaries, I believe, and dictionaries methods can be overloaded, including the lookup function: __getitem__. So mr_agreeable is a dictionary subclass with an overloaded __getitem__ method which automatically creates a blank class when a lookup key doesn't exist. An instance of mr_agreeable is passed to execfile as the namespace for the classes.py script. The objects (aside from the builtins) created execfile call are merged with the globals() dict of the calling script: hack.py.
This works because Python doesn't care if class' base classes are changed after the fact.
This may be implementation dependent, I don't know. Tested on: Python 2.7.3 64bit on Win7 64bit.
Assuming your out-of-order classes are defined in classes.py:
class ParentModel(object):
name = "parentmodel"
class ChildC(ChildA):
name = "childc"
class ChildA(ChildB):
name = "childa"
class ChildB(ParentModel):
name = "childb"
The loader script, lets call it hack.py:
from random import randint
from codecs import encode
class mr_agreeable(dict):
sin_counter = 0
nutty_factor = 0
rdict = {0 : (0, 9), 200 : (10, 14), 500 : (15, 16), 550 : (17, 22)}
def __getitem__(self, key):
class tmp(object):
pass
tmp.__name__ = key
if(not key in self.keys()):
self.prognosis()
print self.insanity()
return self.setdefault(key, tmp)
def prognosis(self):
self.sin_counter += 1
self.nutty_factor = max(filter(lambda x: x < self.sin_counter, self.rdict.keys()))
def insanity(self):
insane_strs = \
[
"Nofbyhgryl", "Fher, jul abg?", "Sbe fher", "Fbhaqf terng", "Qrsvangryl", "Pbhyqa'g nterr zber",
"Jung pbhyq tb jebat?", "Bxl Qbnxl", "Lrc", "V srry gur fnzr jnl", "Zneel zl qnhtugre",
"Znlor lbh fubhyq svk gung", "1 AnzrReebe vf bar gbb znal naq n 1000'f abg rabhtu", "V'ir qbar qvegvre guvatf",
"Gur ebbz vf fgnegvat gb fcva", "Cebonoyl abg", "Npghnyyl, ab ..... nyevtug gura", "ZNXR VG FGBC",
"BU TBQ AB", "CYRNFR AB", "LBH'ER OERNXVAT CLGUBA", "GUVF VF ABG PBAFRAGHNY", "V'Z GRYYVAT THVQB!!"
]
return encode("ze_nterrnoyr: " + insane_strs[randint(*self.rdict[self.nutty_factor])], "rot13")
def the_act():
ns = mr_agreeable()
execfile("classes.py", ns)
hostages = list(set(ns.keys()) - set(["__builtins__", "object"]))
globals().update([(key, ns[key]) for key in hostages])
the_act()
mr_agreeable acts as the permissible namespace to the complied classes.py. He reminds you this is bad form.
My previous answer showed a loader script that executed the out of order script in execfile but provided a dynamic name space that created placeholder classes (these are typically base classes sourced before they are defined). It then loaded the changes from this name space in the loader's global namespace.
This approach has two problems:
1) Its a hack
2) The assumed class of the placeholders is the object class. So when:
class ChildC(ChildA):
name = "childc"
is evaluated, the namespace detects ChildA is undefined and so creates a placeholder class (an object subclass). When ChildA is actually defined (in the out-of-order script), it might be of a different base class than object and so rebasing ChildC to the new ChildA will fail if ChildA's base is not the object class (what ChildC was originally created with). See this for more info.
So I created a new script, which actually rewrites the input out-of-order script using a similar concept to the previous hack and this script. The new script is used by calling:
python mr_agreeable.py -i out_of_order.py -o ordered.py
mr_agreeable.py:
import os
import sys
from codecs import encode
from random import randint
import getopt
import inspect
import types
__doc__ = \
'''
A python script that re-orders out of sequence class defintions
'''
class rebase_meta(type):
'''
Rebase metaclass
Automatically rebases classes created with this metaclass upon
modification of classes base classes
'''
org_base_classes = {}
org_base_classes_subs = {}
base_classes = {}
base_classes_subs = {}
mod_loaded = False
mod_name = ""
mod_name_space = {}
def __init__(cls, cls_name, cls_bases, cls_dct):
#print "Making class: %s" % cls_name
super(rebase_meta, cls).__init__(cls_name, cls_bases, cls_dct)
# Remove the old base sub class listings
bases = rebase_meta.base_classes_subs.items()
for (base_cls_name, sub_dict) in bases:
sub_dict.pop(cls_name, None)
# Add class to bases' sub class listings
for cls_base in cls_bases:
if(not rebase_meta.base_classes_subs.has_key(cls_base.__name__)):
rebase_meta.base_classes_subs[cls_base.__name__] = {}
rebase_meta.base_classes[cls_base.__name__] = cls_base
rebase_meta.base_classes_subs[cls_base.__name__][cls_name] = cls
# Rebase the sub classes to the new base
if(rebase_meta.base_classes.has_key(cls_name)): # Is class a base class
subs = rebase_meta.base_classes_subs[cls_name]
rebase_meta.base_classes[cls_name] = cls # Update base class dictionary to new class
for (sub_cls_name, sub_cls) in subs.items():
if(cls_name == sub_cls_name):
continue
sub_bases_names = [x.__name__ for x in sub_cls.__bases__]
sub_bases = tuple([rebase_meta.base_classes[x] for x in sub_bases_names])
try:
# Attempt to rebase sub class
sub_cls.__bases__ = sub_bases
#print "Rebased class: %s" % sub_cls_name
except TypeError:
# The old sub class is incompatible with the new base class, so remake the sub
if(rebase_meta.mod_loaded):
new_sub_cls = rebase_meta(sub_cls_name, sub_bases, dict(sub_cls.__dict__.items() + [("__module__", rebase_meta.mod_name)]))
rebase_meta.mod_name_space[sub_cls_name] = new_sub_cls
else:
new_sub_cls = rebase_meta(sub_cls_name, sub_bases, dict(sub_cls.__dict__.items()))
subs[sub_cls_name] = new_sub_cls
#classmethod
def register_mod(self, imod_name, imod_name_space):
if(not self.mod_loaded):
self.org_base_classes = self.base_classes.copy()
self.org_base_classes_subs = self.base_classes_subs.copy()
self.mod_loaded = True
else:
self.base_classes = self.org_base_classes
self.base_classes_subs = self.org_base_classes_subs
self.mod_name = imod_name
self.mod_name_space = imod_name_space
# Can't subclass these classes
forbidden_subs = \
[
"bool",
"buffer",
"memoryview",
"slice",
"type",
"xrange",
]
# Builtin, sub-classable classes
org_class_types = filter(lambda x: isinstance(x, type) and (not x.__name__ in forbidden_subs) and x.__module__ == "__builtin__", types.__builtins__.values())
# Builtin classes recreated with Rebasing metaclass
class_types = [(cls.__name__, rebase_meta(cls.__name__, (cls,), {})) for cls in org_class_types]
# Overwrite builtin classes
globals().update(class_types)
class mr_quiet(dict):
'''
A namespace class that creates placeholder classes upon
a non existant lookup. mr_quiet doesnt say much.
'''
def __getitem__(self, key):
if(not key in self.keys()):
if(hasattr(__builtins__, key)):
return getattr(__builtins__, key)
else:
if(not key in self.keys()):
self.sanity_check()
return self.setdefault(key, rebase_meta(key, (object,), {}))
else:
return dict.__getitem__(self, key)
def sanity_check(self):
pass
class mr_agreeable(mr_quiet):
'''
A talkative cousin of mr_quiet.
'''
sin_counter = 0
nutty_factor = 0
rdict = {0 : (0, 9), 200 : (10, 14), 500 : (15, 16), 550 : (17, 22)}
def sanity_check(self):
self.prognogsis()
print self.insanity()
def prognogsis(self):
self.sin_counter += 1
self.nutty_factor = max(filter(lambda x: x < self.sin_counter, self.rdict.keys()))
def insanity(self):
insane_strs = \
[
"Nofbyhgryl", "Fher, jul abg?", "Sbe fher", "Fbhaqf terng", "Qrsvangryl", "Pbhyqa'g nterr zber",
"Jung pbhyq tb jebat?", "Bxl Qbnxl", "Lrc", "V srry gur fnzr jnl", "Zneel zl qnhtugre",
"Znlor lbh fubhyq svk gung", "1 AnzrReebe vf bar gbb znal naq n 1000'f abg rabhtu", "V'ir qbar qvegvre guvatf",
"Gur ebbz vf fgnegvat gb fcva", "Cebonoyl abg", "Npghnyyl, ab ..... nyevtug gura", "ZNXR VG FGBC",
"BU TBQ AB", "CYRNFR AB", "LBH'ER OERNXVAT CLGUBA", "GUVF VF ABG PBAFRAGHNY", "V'Z GRYYVAT THVQB!!"
]
return encode("ze_nterrnoyr: " + insane_strs[randint(*self.rdict[self.nutty_factor])], "rot13")
def coll_up(ilist, base = 0, count = 0):
'''
Recursively collapse nested lists at depth base and above
'''
tlist = []
if(isinstance(ilist, __builtins__.list) or isinstance(ilist, __builtins__.tuple)):
for q in ilist:
tlist += coll_up(q, base, count + 1)
else:
if(base > count):
tlist = ilist
else:
tlist = [ilist]
return [tlist] if((count != 0) and (base > count)) else tlist
def build_base_dict(ilist):
'''
Creates a dictionary of class : class bases pairs
'''
base_dict = {}
def build_base_dict_helper(iclass, idict):
idict[iclass] = list(iclass.__bases__)
for x in iclass.__bases__:
build_base_dict_helper(x, idict)
for cur_class in ilist:
build_base_dict_helper(cur_class, base_dict)
return base_dict
def transform_base_to_sub(idict):
'''
Transforms a base dict into dictionary of class : sub classes pairs
'''
sub_dict = {}
classes = idict.keys()
for cur_class in idict:
sub_dict[cur_class] = filter(lambda cls: cur_class in idict[cls], classes)
return sub_dict
recur_class_helper = lambda idict, ilist = []: [[key, recur_class_helper(idict, idict[key])] for key in ilist]
recur_class = lambda idict: recur_class_helper(idict, idict.keys())
class proc_func(list):
'''
Cmdline processing class
'''
def __init__(self, name = "", *args, **kwargs):
self.name = name
super(list, self).__init__(*args, **kwargs)
def get_args(self, *args):
self.extend(filter(lambda x: x, args))
def __call__(self, *args):
print self.name
print self
class proc_inputs(proc_func):
def get_args(self, *args):
self.extend(filter(os.path.isfile, args))
class proc_outputs(proc_func):
pass
class proc_helper(proc_func):
'''
Help function
Print help information
'''
def get_args(self, *args):
self()
def __call__(self, *args):
print __file__
print __doc__
print "Help:\n\t%s -h -i inputfile -o ouputfile" % sys.argv[0]
print "\t\t-h or --help\tPrint this help message"
print "\t\t-i or --input\tSpecifies the input script"
print "\t\t-o or --output\tSpecifies the output script"
sys.exit()
if __name__ == "__main__":
proc_input = proc_inputs("input")
proc_output = proc_outputs("output")
proc_help = proc_helper("help")
cmd_line_map = \
{
"-i" : proc_input,
"--input" : proc_input,
"-o" : proc_output,
"--ouput" : proc_output,
"-h" : proc_help,
"--help" : proc_help
}
try:
optlist, args = getopt.getopt(sys.argv[1:], "hi:o:", ["help", "input=", "output="])
for (key, value) in optlist:
cmd_line_map[key].get_args(value)
except getopt.GetoptError:
proc_help()
if(len(proc_input) != len(proc_output)):
print "Input files must have a matching output file"
proc_help()
elif(not proc_input):
proc_help()
else:
in_out_pairs = zip(proc_input, proc_output)
for (in_file, out_file) in in_out_pairs:
dodgy_module_name = os.path.splitext(in_file)[0]
sys.modules[dodgy_module_name] = types.ModuleType(dodgy_module_name)
sys.modules[dodgy_module_name].__file__ = in_file
# Make a fake space post haste
name_space = mr_agreeable\
(
[
("__name__", dodgy_module_name), # Needed for the created classes to identify with the fake module
("__module__", dodgy_module_name), # Needed to fool the inspect module
] + \
class_types
)
# Exclude these from returning
exclusions = name_space.keys()
# Associate the fake name space to the rebasing metaclass
rebase_meta.register_mod(dodgy_module_name, name_space)
# Run dodgy code
execfile(in_file, name_space)
# Bring back dodgy classes
import_classes = [cls if(isinstance(cls, type) and not cls_name in exclusions) else None for (cls_name, cls) in name_space.items()]
dodgy_import_classes = filter(lambda x: x, import_classes)
# Create base and sub class dictionaries
base_dict = build_base_dict(dodgy_import_classes)
sub_dict = transform_base_to_sub(base_dict)
# Create sets of base and sub classes
base_set = reduce(lambda x, y: x | y, map(set, base_dict.values()), set([]))
sub_set = reduce(lambda x, y: x | y, map(set, sub_dict.values()), set([]))
kings = list(base_set - sub_set) # A list of bases which are not subs
kingdoms = recur_class_helper(sub_dict, kings) # A subclass tree of lists
lineages = coll_up(kingdoms, 2) # Flatten the tree branches at and below 2nd level
# Filter only for the clases created in the dodgy module
inbred_lines = [filter(lambda x: x.__module__ == dodgy_module_name, lineage) for lineage in lineages]
# Load Source
for lineage in inbred_lines:
for cls in lineage:
setattr(cls, "_source", inspect.getsource(cls))
# Write Source
with open(out_file, "w") as file_h:
for lineage in inbred_lines:
for cls in lineage:
file_h.write(cls._source + "\n")
This is just a fun question about some thing you can do with python syntax.
When I moved from matlab to python, I made a class that worked similarly to matlab's struct
class DynStruct(AbstractPrintable):
' dynamical add and remove members '
def __init__(self, child_exclude_list=[]):
super(DynStruct, self).__init__(child_exclude_list)
it is just an object where you can dynamically add members without having to
resort a dictionary (because I hate typing quotes)
I also made a cool helper class that printed out the members of the class nicely so you can
see what you're doing while working in IPython. (I'm leaving out my imports, but its just some standard stuff as well as some numpy)
class AbstractPrintable(object):
'A base class that prints its attributes instead of the memory address'
def __init__(self, child_print_exclude=[]):
self._printable_exclude = ['_printable_exclude'] + child_print_exclude
def __str__(self):
head = printableType(self)
body = self.printable_attributes()
body = re.sub('\n *\n *\n','\n\n',body)
return head+('\n'+body).replace('\n','\n ')
def printable_attributes(self, type_bit=True):
body = ''
attri_list = []
for key in self.__dict__.iterkeys():
if key in self._printable_exclude: continue
val = self.__dict__[key]
namestr = str(key)
valstr = printableVal(val)
typestr = printableType(val)
max_valstr = 10000
if len(valstr) > max_valstr:
valstr = valstr[0:max_valstr/2]+valstr[-max_valstr/2:-1]
attri_list.append( (typestr, namestr, valstr) )
attri_list.sort()
for (typestr, namestr, valstr) in attri_list:
entrytail = '\n' if valstr.count('\n') <= 1 else '\n\n'
typestr2 = typestr+' ' if type_bit else ''
body += typestr2 + namestr + ' = ' + valstr + entrytail
return body
#---------------
def printableType(val):
if type(val) == numpy.ndarray:
info = npArrInfo(val)
_typestr = info.dtypestr
elif isinstance(val, object):
_typestr = val.__class__.__name__
else:
_typestr = str(type(val))
_typestr = _typestr.replace('type','')
_typestr = re.sub('[\'><]','',_typestr)
_typestr = re.sub(' *',' ',_typestr)
_typestr = _typestr.strip()
return _typestr
I then had a case where I needed to get a bunch of elements from my DynStruct, so I added a function which returns a tuple of the elements I wanted.
# I added this function to DynStruct
def getprops(self, *prop_list):
return tuple([self.__dict__[prop_name] for prop_name in prop_list])
EG:
>> point = DynStruct()
>> point.x = 3
>> point.y = 1
>> point.z = 60
>> print point
DynStruct
int x = 3
int y = 1
int z = 60
>> # Now I want to get the points
>> (x,y,z) = point.getprops('x','y','z')
Now, this works great, and it makes debugging really easy. But I came on a case where I wanted to set a bunch of properties at once (sort of like above). And I realize there are other ways to do this, but I'd really like to have a setprop where the syntax works like this:
point.setprops('x','y','z') = (14, 22, 30)
I'm not sure, but I feel like there might be a way to do this because of the #someobj.setter decorator. But I don't know how to overload the equals operator to use it in this way, or if its even possible.
I guess in the meantime I'll just write it like this
point.setprops('x','y','z', 14, 22, 30)
Right off the bat, you don't need this - since you can do:
point.x, point.y, point.z = (14, 22, 30)
# Tuple unpacking ... is there nothing it cannot do?
However, let's say that this is not clear enough and you really need to be able to set multiple fields at once. Then you can use __setitem__:
def __setitem__(self, key, value):
if isinstance(key, tuple):
for k, v in zip(key, value):
setattr(self, k, v)
else:
setattr(self, key, value)
Then you could do as follows:
point['x', 'y', 'z'] = (14, 22, 30)
Then you could also replace your getprops method with the __getitem__ method, similarly implemented and be able to do:
x, y, z = point['x', 'y', 'z']
it is just an object where you can dynamically add members without having to resort a dictionary (because I hate typing quotes)
Maybe I missed something in your explanation, but can't you already do this? For example
class Foo(Object):
def __init__(self):
pass
f = Foo()
f.x = 1
f.y = 2
f.z = 3
print f.x, f.y, f.z # outputs 1 2 3
As for the second part of your question, about getting this syntax,
point.setprops('x','y','z') = (14, 22, 30)
Unfortunately I'm not sure if this is possible. The reason is that
point.setprops('x', 'y', 'z')
is a 'call' expression. Due to the way python is parsed, I don't think you can have a call expression on the left hand side of an assignment statement. From what I know, you can only have a list of variable names or slice/index expressions. That being said, I'm not 100% sure. There might be some crazy python ninja stuff you can do to make that work.
This has come up several times recently and I'd like to deal with it better than I have been: I have a series of attributes that I'm cross referencing between an object and a dictionary. If the value is different between them, I want to set the object.attribute to the dictionary['attribute'] value. I also want to keep track of what's getting changed.
Now, my first thought is to just use an if else statement for every attribute, but after writing a few of these it's apparent that I'm re-writing the same code again and again. There has to be a DRY way to do this, where I specify only the parts that are changing every time, and then loop through all the attributes.
In production code, there are 15 different attributes, but my example below will just use 2 for simplicity. I have some idea about how to do this in a clever way, but I'm missing the final step of actually setting the object.attribute equal to the dictionary['attribute'] value.
# Simulated data setup - not under my control IRL
class someClass:
def __init__(self, name, version):
self.name = name
self.version = version
objA = someClass('Test1','1.1')
dictA = {'name':'Test1','revision':'1.2'}
# My code below
# option 1 - a series of for loops
def updateAttributesSimple(obj, adict, msg):
if obj.name == adict['name']:
msg.append('Name is the same')
else:
msg.append('Name was updated from %s to %s' % (obj.name, adict['name']))
obj.name = adict['name']
if obj.version == adict['revision']:
msg.append('Version is the same')
else:
msg.append('Version was updated from %s to %s' % (obj.version, adict['revision']))
obj.version = adict['revision']
# option 2 - trying to be clever about this
def updateAttributesClever(obj, adict, msg):
attributeList = (('Name', obj.name, adict['name']),
('Version', obj.version, adict['revision']))
for valTuple in attributeList:
if valTuple[1] == valTuple[2]:
msg.append('%s is the same' % (valTuple[0]))
else:
msg.append('%s was updated from %s to %s' % (valTuple[0], valTuple[1], valTuple[2]))
# code to set valTuple[1] = valTuple[2] goes here, but what is it?
# valTuple[1] = valTuple[2] attempts to set the desired value to a string, rather than the attribute of obj itself
msg = ['Updating Attributes simple way:']
updateAttributesSimple(objA, dictA, msg)
print '\n\t'.join(msg)
#reset data
objA = someClass('Test1','1.1')
dictA = {'name':'Test1','revision':'1.2'}
msg = ['Updating Attributes clever way:']
updateAttributesClever(objB, dictB, msg)
print '\n\t'.join(msg)
The idea being that this way, whenever I need to add another attribute, I can just update the list of attributes being inspected and the rest of the code is already written. What's the Pythonic way to accomplish this?
setattr() is what you're looking for:
attributeList = (('Name', 'name', 'name'),
('Version', 'version', 'revision'))
for title, obj_attribute, dict_key in attributeList:
obj_value = getattr(obj, obj_attribute)
adict_value = adict[dict_key]
if obj_value == adict_value:
msg.append('%s is the same' % (obj_value))
else:
msg.append('%s was updated from %s to %s' % (title, obj_value, adict_value))
setattr(obj, obj_attribute, adict_value)
This should work for your:
class X(object):
def __init__(self):
self.a = 1
self.b = 2
x = X()
d = dict()
d['a'] = 1
d['b'] = 3
def updateAttributes(obj,dic):
def update(name):
val = dic[name]
if getattr(obj,name)==val:
print name,"was equal"
else:
print "setting %s to %s" % (name,val)
setattr(obj,name,val)
for name in ['a','b']:
update(name)
updateAttributes(x,d)
print x.a
print x.b
You might want to think about creating a function which can take an arbitrary object and convert the dictionary of name/value pairs into something more meaningful. It's not strictly a "Python" strategy but something that is fairly easy to do in Python because of its support of closures and how it treats objects under the hood:
def checkUpdates( obj ):
def updated( dictionaryPrevious, msg ):
for name, value in dictionaryPrevious.items():
if( obj.__dict__[name] == value ):
msg.append('Name is the same')
else:
msg.append(name + 'has been changed!')
obj.__dict__[name] = value
return updated
I am making one assumption, the names in the dictionary always correspond to the object variables. If they're not the same you'll need to make a mapping.
edit:
() => [] and object => obj. thanks guys. Sometimes you go from one language to a few others and it all gets muddled.
A couple of answers are close, but to handle that fact that the name of the key in the dict don't match the corresponding object's attribute name, you'll need some way to handle that. This can be easily done by adding yet another dictionary mapping the names of keys in the dict to the names of the object's attributes.
class someClass:
def __init__(self, name, version):
self.name = name
self.version = version
objA = someClass('Test1','1.1')
dictA = {'name':'Test1','revision':'1.2'}
keymap = {'name':'name', 'revision':'version'}
def updateAttributesGeneric(obj, adict, key2attr, msg):
for key, value in adict.iteritems():
attrname = key2attr[key]
if getattr(obj, attrname) == value:
msg.append('%s is the same' % attrname)
else:
msg.append('%s has been changed' % attrname)
setattr(obj, attrname, adict[key])
msg = ['Updating Attributes:']
updateAttributesGeneric(objA, dictA, keymap, msg)
print '\n\t'.join(msg)
# Updating Attributes:
# name is the same
# version has been changed
This question already has answers here:
How can I represent an 'Enum' in Python?
(43 answers)
Closed 9 years ago.
Just looking at ways of getting named constants in python.
class constant_list:
(A_CONSTANT, B_CONSTANT, C_CONSTANT) = range(3)
Then of course you can refer to it like so:
constant_list.A_CONSTANT
I suppose you could use a dictionary, using strings:
constant_dic = {
"A_CONSTANT" : 1,
"B_CONSTANT" : 2,
"C_CONSTANT" : 3,}
and refer to it like this:
constant_dic["A_CONSTANT"]
My question, then, is simple. Is there any better ways of doing this? Not saying that these are inadequate or anything, just curious - any other common idioms that I've missed?
Thanks in advance.
For 2.3 or after:
class Enumerate(object):
def __init__(self, names):
for number, name in enumerate(names.split()):
setattr(self, name, number)
To use:
codes = Enumerate('FOO BAR BAZ')
codes.BAZ will be 2 and so on.
If you only have 2.2, precede this with:
from __future__ import generators
def enumerate(iterable):
number = 0
for name in iterable:
yield number, name
number += 1
(This was taken from here)
I find the enumeration class recipe (Active State, Python Cookbook) to be very effective.
Plus it has a lookup function which is nice.
Pev
An alternative construction for constant_dic:
constants = ["A_CONSTANT", "B_CONSTANT", "C_CONSTANT"]
constant_dic = dict([(c,i) for i, c in enumerate(constants)])
The following acts like a classisc "written in stone" C enum -- once defined, you can't change it, you can only read its values. Neither can you instantiate it. All you have to do is "import enum.py" and derive from class Enum.
# this is enum.py
class EnumException( Exception ):
pass
class Enum( object ):
class __metaclass__( type ):
def __setattr__( cls, name, value ):
raise EnumException("Can't set Enum class attribute!")
def __delattr__( cls, name ):
raise EnumException("Can't delete Enum class attribute!")
def __init__( self ):
raise EnumException("Enum cannot be instantiated!")
This is the test code:
# this is testenum.py
from enum import *
class ExampleEnum( Enum ):
A=1
B=22
C=333
if __name__ == '__main__' :
print "ExampleEnum.A |%s|" % ExampleEnum.A
print "ExampleEnum.B |%s|" % ExampleEnum.B
print "ExampleEnum.C |%s|" % ExampleEnum.C
z = ExampleEnum.A
if z == ExampleEnum.A:
print "z is A"
try:
ExampleEnum.A = 4
print "ExampleEnum.A |%s| FAIL!" % ExampleEnum.A
except EnumException:
print "Can't change Enum.A (pass...)"
try:
del ExampleEnum.A
except EnumException:
print "Can't delete Enum.A (pass...)"
try:
bad = ExampleEnum()
except EnumException:
print "Can't instantiate Enum (pass...)"
This is the best one I have seen: "First Class Enums in Python"
http://code.activestate.com/recipes/413486/
It gives you a class, and the class contains all the enums. The enums can be compared to each other, but don't have any particular value; you can't use them as an integer value. (I resisted this at first because I am used to C enums, which are integer values. But if you can't use it as an integer, you can't use it as an integer by mistake so overall I think it is a win.) Each enum is a unique object. You can print enums, you can iterate over them, you can test that an enum value is "in" the enum. It's pretty complete and slick.
In Python, strings are immutable and so they are better for constants than numbers. The best approach, in my opinion, is to make an object that keeps constants as strings:
class Enumeration(object):
def __init__(self, possibilities):
self.possibilities = set(possibilities.split())
def all(self):
return sorted(self.possibilities)
def __getattr__(self, name):
if name in self.possibilities:
return name
raise AttributeError("Invalid constant: %s" % name)
You could then use it like this:
>>> enum = Enumeration("FOO BAR")
>>> print enum.all()
['BAR', 'FOO']
>>> print enum.FOO
FOO
>>> print enum.FOOBAR
Traceback (most recent call last):
File "enum.py", line 17, in <module>
print enum.FOOBAR
File "enum.py", line 11, in __getattr__
raise AttributeError("Invalid constant: %s" % name)
AttributeError: Invalid constant: FOOBAR