I'd like to have FactoryBoy create a Django model instance for me, and then also create a temporary directory.
I've tried a few things:
class OrganizationFactory(factory.DjangoModelFactory):
class Meta:
model = Organization
some_directory = tempfile.mkdtemp
and
some_directory = tempfile.mkdtemp()
and
#factory.lazy_attribute
def some_directory(self):
return tempfile.mkdtemp()
When I try to use some_directory in my tests, the value is always None.
In order to understand what happens, the best will be to dive into factory_boy's internals.
factory_boy internals
There are separate phases in a Factory's life:
The declaration
When Python imports the module containing a class Factory declaration, that class's definition is handled to a special metaclass in factory_boy's source code.
The metaclass reads all defined class attributes and methods, analyses them, and stores them in a specific way.
Using the factory
When your code uses MyFactory(), the class starts building an instance, according to the following steps:
Evaluate the declarations: each LazyAttribute, SubFactory, etc. is asked to generate its value;
Pass the resulting parameters to the Meta.model class constructor — this provides an instance of said class;
Ask all post-generation declarations (RelatedFactory and co) to work on the generated object — note that any value returned from those declarations is thrown away
Return the instance.
Solving your issue
In your case, I believe the issue might be that some_directory is either not a valid kwarg for your model's __init__, or replaced by some magic on your model class.
The simplest way would be to work on the post-generation level:
class OrganizationFactory(factory.django.DjangoModelFactory):
class Meta:
model = Organization
#factory.post_generation
def some_directory(org, *args, **kwargs):
tmpdir = tempfile.mkdtemp()
# Attach the generated temporary directory to the generated organization.
org.some_directory = tmpdir
Related
I'm currently working on a CLI abstraction layer, which abstracts CLI programs as classes in Python. Such CLI programs offer a structured way to enable and configure CLI parameters. It helps checking for faulty inputs and generated properly escaped arguments (e.g. adding double quotes).
Note: The following example is using Git, while in my target application, it will be commercial tools, that don't offer any Python API or similar.
Basic Ideas:
An abstraction of tool Git declares a Git class, which derives from class Program.
This parent class implements common methods to all programs.
CLI options are represented as nested class definitions on the Git class.
Nested classes are marked with a class-based decorator CLIOption derived from Attribute
(see https://github.com/pyTooling/pyAttributes for more details)
CLI options can be enabled / modified via indexed syntax.
An instance of Git is used to enabled / configure CLI parameters and helps to assemble a list of correctly encoded strings that can be used e.g. in subprocess.Popen(...)
tool = Git()
tool[tool.FlagVersion] = True
print(tool.ToArgumentList())
Some Python Code:
from pyAttributes import Attribute
class CLIOption(Attribute): ... # see pyAttributes for more details
class Argument:
_name: ClassVar[str]
def __init_subclass__(cls, *args, name: str = "", **kwargs):
super().__init_subclass__(*args, **kwargs)
cls._name = name
class FlagArgument(Argument): ...
class CommandArgument(Argument): ...
class Program:
__cliOptions__: Dict[Type[Argument], Optional[Argument]]
def __init_subclass__(cls, *args, **kwargs):
"""Hook into subclass creation to register all marked CLI options in ``__cliOptions__``.
super().__init_subclass__(*args, **kwargs)
# get all marked options and
cls.__cliOptions__ = {}
for option in CLIOption.GetClasses():
cls.__cliOptions__[option] = None
class Git(Program):
#CLIOption()
class FlagVersion(FlagArgument, name="--version"): ...
#CLIOption()
class FlagHelp(FlagArgument, name="--help"): ...
#CLIOption()
class CmdCommit(CommandArgument, name="commit"): ...
Observations:
As #martineau pointed out in a comment, the CLIOption decorator has no access to the outer scope. So the scope can't be annotated to the nested classes.
The nested classes are used because of some nice effects in Python not demonstrated here. Also to keep their scope local to a program. Imagine there might be multiple programs offering a FlagVersion flag. Some as -v, but others as --version.
Primary Questions:
How can I check if class FlagVersion is a nested class of class Git?
What I investigated so far:
There is no helper function to achieve this goal like functions isinstance(...) or issubclass(...) are offering.
While root-level classes have a __module__ reference to the outer scope, nested classes have no "pointer" to the next outer scope.
Actually, nested classes have the same __module__ values.
Which makes sense.
A class' __qualname__ includes the names of parent classes.
Unfortunately this is a string like Git.FlagVersion
So I see a possible "ugly" solution using __qualname__ and string operations to check if a class is nested and if it's nested in a certain outer scope.
Algorithm:
Assemble fully qualified name from __module__ and __qualname__.
Check element by element from left to right for matches.
This gets even more complicated if one nested class is defined in a parent class and another nested class is defined in a derived class. Then I also need to look into MRO ... oOo
Secondary Questions:
Is there a better way than using string operations?
Shouldn't Pythons data model offer a better way to get this information?
Because I don’t understand English very well, I understand by translating is that you need to find out how to get the embedded class decorated by CLIOption in the subclass of Program (Git here). If so, the following methods may help you.
I read some codes of some pyAttributes
from pyAttributes import Attribute
class Program(object):
__cliOptions__: Dict[Type[Argument], Optional[Argument]]
def __init_subclass__(cls, *args, **kwargs):
cls.__cliOptions__ = {}
for obj in cls.__dict__.values():
if hasattr(obj, Attribute.__AttributesMemberName__):
print(obj)
# for option in CLIOption.GetClasses():
# cls.__cliOptions__[option] = None
class Git(Program):
a = 1
b = 2
#CLIOption()
class FlagVersion(FlagArgument, name="--version"):
...
#CLIOption()
class FlagHelp(FlagArgument, name="--help"):
...
Of course, the above can’t work directly. Later I found that there was of course an error in the Attribute._AppendAttribute method, as follows, I modified it
class CLIOption(Attribute):
... # see pyAttributes for more details
#staticmethod
def _AppendAttribute(func: Callable, attribute: 'Attribute') -> None:
# inherit attributes and prepend myself or create a new attributes list
if Attribute.__AttributesMemberName__ in func.__dict__:
func.__dict__[Attribute.__AttributesMemberName__].insert(0, attribute)
else:
# The original func.__setattr__(Attribute.__AttributesMemberName__, [attribute]) has an error
# Because __setattr__ of class FlagVersion is object.__setattr__
setattr(func, Attribute.__AttributesMemberName__, [attribute])
# or object.__setattr__(func, Attribute.__AttributesMemberName__, [attribute])
Following the proposed approaches by iterating __dict__ works quite good.
So this was the first solution developed based on the given ideas:
def isnestedclass(cls: Type, scope: Type) -> bool:
for memberName in scope.__dict__:
member = getattr(scope, memberName)
if type(member) is type:
if cls is member:
return True
return False
That solution doesn't work on members inherited from parent classes.
So I extended it with searching through the inheritance graph via mro().
This is my current and final solution for a isnestedclass helper function.
def isnestedclass(cls: Type, scope: Type) -> bool:
for mroClass in scope.mro():
for memberName in mroClass.__dict__:
member = getattr(mroClass, memberName)
if type(member) is type:
if cls is member:
return True
return False
The function is available within the pyTooling package.
Note: although my particular use is Flask related, I think the question is more general.
I am building a Flask web application meant to be customized by the user. For example, the user is expected to provide a concrete subclass of a DatabaseInterface and may add to the list of certain ModelObjects that the application knows how to handle.
What is the best way to expose the various hooks to users, and indicate required and optional status? 'Best' here primarily means most 'pythonic', or "easiest for python users to grasp", but other criteria like not causing headaches down the road are certainly worth mentioning.
Some approaches I've considered:
Rely solely on documentation
Create a template file with documented overrides, much like default config files for many servers. E.g.
app = mycode.get_app()
##Add your list of extra foo classes here
#app.extra_foos = []
Create a UserOverrides class with an attr/method for each of the hooks; possibly split into RequiredOverrides and OptionalOverrides
Create an empty class with unimplemented methods that the user must subclass into a concrete instance
One method is by using abstract base classes (abc module). For example, you can define an ABC with abstract methods that must be overridden by child classes like this:
from abc import ABC
class MyClass(ABC): # inherit from ABC
def __init__(self):
pass
#abstractmethod
def some_method(self, args):
# must be overridden by child class
pass
You would then implement a child class like:
class MyChild(MyClass):
# uses parent's __init__ by default
def some_method(self, args):
# overrides the abstract method
You can specify what everything needs to do in the overridden methods with documentation. There are also decorators for abstract properties, class methods, and static methods. Attempting to instantiate an ABC that does not have all of its abstract methods/properties overridden will result in an error.
Inheritance. Is. Bad.
This is especially true in Python, which gives you a nice precedent to avoid the issue. Consider the following code:
len({1,2,3}) # set with length 3
len([1,2,3]) # list with length 3
len((1,2,3)) # tuple with length 3
Which is cool and all for the built-in data structures, but what if you want to make your own data structure and have it work with Python's len? Simple:
class Duple(object):
def __init__(self, fst, snd):
super(Duple, self).__init__()
self.fst = fst
self.snd = snd
def __len__():
return 2
A Duple is a two-element (only) data structure (calling it with more or fewer arguments raises) and now works with len:
len(Duple(1,2)) # 2
Which is exactly how you should do this:
def foo(arg):
return arg.__foo__()
Any class that wants to work with your foo function just implements the __foo__ magic method, which is how len works under the hood.
I am reading this Genshi Tutorial and see there the following example:
from formencode import Schema, validators
class LinkForm(Schema):
username = validators.UnicodeString(not_empty=True)
url = validators.URL(not_empty=True, add_http=True, check_exists=False)
title = validators.UnicodeString(not_empty=True)
As far as I understand this example, we create a new class that inherits Schema class and this class contain three methods: username, url, title. However, I am not sure about the last because before I only saw methods created with def.
Anyway, my question is not about that. I would like to know if it is possible to make the definition of the class dynamic. For example, sometimes I do not want url or title to be in the class. It seems to be doable (I just use if and assign a value to url only if-statement is satisfied.
But what if I do not know in advance what fields I would like to have in the form? For example, now I have username, url and title. But what if later I would like to have city or age. Can I do something like that:
from formencode import Schema, validators
class LinkForm(Schema):
__init__(self, fields):
for field in fields:
condition = fields[field]
field = validators.UnicodeString(condition)
I think it will not work. Is there a work around in this case?
Yes, you can add methods to an instance dynamically. No, you can't do what you want.
You can bind methods to the instance in the initializer. Unfortunately what you have there are descriptors and those must be bound to the class.
I would go the other way round—first define all form fields that might be used, and delete unneeded ones later.
Provided that you have:
from formencode import Schema, validators
class LinkForm(Schema):
username = validators.UnicodeString(not_empty=True)
url = validators.URL(not_empty=True, add_http=True, check_exists=False)
title = validators.UnicodeString(not_empty=True)
you could do either this:
def xy():
my_form = LinkForm()
del my_form.url
…
… or this:
def xy():
class CustomLinkForm(LinkForm):
pass
if …:
del CustomLinkForm.url
…
Disclaimer: I am not familiar with FormEncode, so it might depend on its inner workings which of these two versions actually works.
of course you can have a constructor with some arguments after self and these arguments will be the value for some members of your class if you have for instance
__init__(self, fields):
self.fields = []
for field in fields:
self.fields = self.fields + field
see this in Dive into Python
class FileInfo(UserDict):
"store file metadata"
def __init__(self, filename=None):
UserDict.__init__(self)
self["name"] = filename
Classes can (and should) have doc strings too, just like modules and
functions.
init is called immediately after an instance of the
class is created. It would be tempting but incorrect to call this the
constructor of the class. It's tempting, because it looks like a
constructor (by convention, init is the first method defined for
the class), acts like one (it's the first piece of code executed in a
newly created instance of the class), and even sounds like one (“init”
certainly suggests a constructor-ish nature). Incorrect, because the
object has already been constructed by the time init is called,
and you already have a valid reference to the new instance of the
class. But init is the closest thing you're going to get to a
constructor in Python, and it fills much the same role.
The first
argument of every class method, including init, is always a
reference to the current instance of the class. By convention, this
argument is always named self. In the init method, self refers to
the newly created object; in other class methods, it refers to the
instance whose method was called. Although you need to specify self
explicitly when defining the method, you do not specify it when
calling the method; Python will add it for you automatically.
init methods can take any number of arguments, and just like
functions, the arguments can be defined with default values, making
them optional to the caller. In this case, filename has a default
value of None, which is the Python null value.
Note that in the later example you learn how to deal with inherited class, calling __init()__ for this inherited class.
To answer your not-a-question about class or instance variables, see this
Variables defined in the class definition are class variables; they
are shared by all instances. To create instance variables, they can be
set in a method with self.name = value. Both class and instance
variables are accessible through the notation “self.name”, and an
instance variable hides a class variable with the same name when
accessed in this way. Class variables can be used as defaults for
instance variables, but using mutable values there can lead to
unexpected results. For new-style classes, descriptors can be used to
create instance variables with different implementation details.
So I can create Django model like this:
from django.db import models
class Something(models.Model):
title = models.TextField(max_length=200)
and I can work with it like this:
thing = Something()
#set title
thing.title = "First thing"
#get title
thing.title
All works as it should but I'd like to understand HOW it works.
title = models.TextField(max_length=200)
in non-Django Python code above line defines class variable title of type models.TextField and I could access it also like this: thing.__class__.title(link)
But in Django when I create instance of Something I suddenly have a title attribute where I can get/set text. And cannot access it with thing.__class__.title So clearly when doing thing.title I'm not accessing class variable "title" but some generated attribute/property, or?
I know that fields ended up in thing._meta.fields but how? What's going on and how?
1, Does Django create property "title" behind the scenes?
2, What happened to class variable "title"?
I think its hard to beat what Django documentation has to say on this.
The Model class (see base.py) has a metaclass attribute that defines ModelBase (also in base.py) as the class to use for creating new classes. So ModelBase.new is called to create this new Example class. It is important to realise that we are creating the class object here, not an instance of it. In other words, Python is creating the thing that will eventually be bound to the Example name in our current namespace.
Basically a metaclass defines how a class itself will be created. During creation, additional attributes/methods/anything can be bound to that class. The example this stackoverflow answer gives, capitalizes all the attributes of a class
# remember that `type` is actually a class like `str` and `int`
# so you can inherit from it
class UpperAttrMetaclass(type):
# __new__ is the method called before __init__
# it's the method that creates the object and returns it
# while __init__ just initializes the object passed as parameter
# you rarely use __new__, except when you want to control how the object
# is created.
# here the created object is the class, and we want to customize it
# so we override __new__
# you can do some stuff in __init__ too if you wish
# some advanced use involves overriding __call__ as well, but we won't
# see this
def __new__(upperattr_metaclass, future_class_name,
future_class_parents, future_class_attr):
attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__'))
uppercase_attr = dict((name.upper(), value) for name, value in attrs)
return type(future_class_name, future_class_parents, uppercase_attr)
In a similar way, Django's metaclass for Models can digest the attributes you've applied to the class and add various useful attributes for validation/etc, including even methods and what-not.
python is extremely powerfull and permit the developer to use intrespection.
django use a lot of metaclass. and it seem that models.Model use it too. see in
django\db\models\base.py
class Model(object):
__metaclass__ = ModelBase
i think the metaclass just take the classes attributes such a the Field and for all new instance for these Model subclass,create the apropriate variable.
1) yes, django create the instance variable of property "title" automaticaly
2) in the same way, the metaclass move the fields into the meta class...
I have a number of atomic classes (Components/Mixins, not really sure what to call them) in a library I'm developing, which are meant to be subclassed by applications. This atomicity was created so that applications can only use the features that they need, and combine the components through multiple inheritance.
However, sometimes this atomicity cannot be ensured because some component may depend on another one. For example, imagine I have a component that gives a graphical representation to an object, and another component which uses this graphical representation to perform some collision checking. The first is purely atomic, however the latter requires that the current object already subclassed this graphical representation component, so that its methods are available to it. This is a problem, because we have to somehow tell the users of this library, that in order to use a certain Component, they also have to subclass this other one. We could make this collision component sub class the visual component, but if the user also subclasses this visual component, it wouldn't work because the class is not on the same level (unlike a simple diamond relationship, which is desired), and would give the cryptic meta class errors which are hard to understand for the programmer.
Therefore, I would like to know if there is any cool way, through maybe metaclass redefinition or using class decorators, to mark these unatomic components, and when they are subclassed, the additional dependency would be injected into the current object, if its not yet available. Example:
class AtomicComponent(object):
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent(object):
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
Can someone give me an hint on how I can do this? or if it is even possible...
edit:
Since it is debatable that the meta class solution is the best one, I'll leave this unaccepted for 2 days.
Other solutions might be to improve error messages, for example, doing something like UserClass2 would give an error saying that UnAtomicComponent already extends this component. This however creates the problem that it is impossible to use two UnAtomicComponents, given that they would subclass object on different levels.
"Metaclasses"
This is what they are for! At time of class creation, the class parameters run through the
metaclass code, where you can check the bases and change then, for example.
This runs without error - though it does not preserve the order of needed classes
marked with the "depends" decorator:
class AutoSubclass(type):
def __new__(metacls, name, bases, dct):
new_bases = set()
for base in bases:
if hasattr(base, "_depends"):
for dependence in base._depends:
if not dependence in bases:
new_bases.add(dependence)
bases = bases + tuple(new_bases)
return type.__new__(metacls, name, bases, dct)
__metaclass__ = AutoSubclass
def depends(*args):
def decorator(cls):
cls._depends = args
return cls
return decorator
class AtomicComponent:
pass
#depends(AtomicComponent) # <- something like this?
class UnAtomicComponent:
pass
class UserClass(UnAtomicComponent): #automatically includes AtomicComponent
pass
class UserClass2(AtomicComponent, UnAtomicComponent): #also works without problem
pass
(I removed inheritance from "object", as I declared a global __metaclass__ variable. All classs will still be new style class and have this metaclass. Inheriting from object or another class does override the global __metaclass__variable, and a class level __metclass__ will have to be declared)
-- edit --
Without metaclasses, the way to go is to have your classes to properly inherit from their dependencies. Tehy will no longer be that "atomic", but, since they could not work being that atomic, it may be no matter.
In the example bellow, classes C and D would be your User classes:
>>> class A(object): pass
...
>>> class B(A, object): pass
...
>>>
>>> class C(B): pass
...
>>> class D(B,A): pass
...
>>>