Related
I have created my own class that inherits from python's default list class. A simplified version is the following, that contains the __abs__ method so I can use python's abs function.
class DataSet(list):
def __abs__(self):
result = []
for i in self:
result.append(abs(i))
return result
Suppose have a DataSet that sometimes contains a NoneType value, for example
>>> dataset = DataSet([1, 2, 3, None, -1, -2, -3])
If I want to know the absolute value of this DataSet, I use the function
>>> abs_dataset = abs(dataset)
The result that I want to get is
[1, 2, 3, None, 1, 2, 3]
but because there is a value of type NoneType in the dataset, I get the error
TypeError: bad operand type for abs(): 'NoneType'
For this one case it can be fixed by modifying the DataSet's __abs__ function and to check for None in the individual elements of the DataSet, but in my case I have more cases where a None value can occur and I also want to implement more builtin functions than only abs.
Is there a method to set this default behaviour of default python functions like abs to None values?
You can do something like this.
class DataSet(list):
def __abs__(self):
# Here if i is 0 then it'll be 0.
# No need to check for `None`.
return [abs(i) if i else i for i in self]
dataset = DataSet([1, 2, 3, None, -1, -2, -3])
print(abs(dataset))
# [1, 2, 3, None, 1, 2, 3]
Edits:
As mentioned by #juanpa.arrivillaga, if you want to filter the None type elements then you can do something like [abs(i) for i in self if i is not None] inside list comprehension.
There isn't. And for a good reason. Other code can rely on None type bahaviour.
What you can do is provide method on DataSet which provide filtered list without None values and use it for your methods.
you can edit those methods in order to make sure there will be no None in your dataset.
__setattr__
__setitem__
append
extend
insert
just override them with a check for None, if there is change the value to 0/cancel the operation.
I've the following tests:
#pytest.mark.parametrize(
"nums",
[[3, 1, 5, 4, 2], [2, 6, 4, 3, 1, 5], [1, 5, 6, 4, 3, 2]]
)
def test_cyclic_sort(nums):
pass
#pytest.mark.parametrize(
"nums, missing",
[([4, 0, 3, 1], 2)]
)
def test_find_missing_number(nums, missing):
pass
I'd like to customize the test names to include the input array. I've read the pytest docs, and this question and this question, but none answer the following questions:
What is passed to the id func? In my code above, the first test takes one parameter, the second takes two.
pytest docs use a top-level function for id, whereas I'd like to put my tests in a class and use a #staticmethod. Trying to reference the static method with TestClass.static_method from inside TestClass gives an error in PyCharm; what is the correct syntax for doing this?
Edit:
Created https://github.com/pytest-dev/pytest/issues/8448.
When using a callable for the ids keyword, it will be called with a single argument: the value of the test parameter being parametrized. The callable ids return a string, which will be used in square brackets as the test name suffix.
If the test is parametrizing over multiple values, the function will still be called with a single argument, but it will be called multiple times per test. The generated name will be joined with dashes, something like
"-".join([idfunc(val) for val in parameters])
For example:
test_something[val1-val2-val3]
Here is the join in the pytest source.
To use a static method, this syntax works:
class TestExample:
#staticmethod
def idfunc(val):
return f"foo{val}"
#pytest.mark.parametrize(
"x, y",
[
[1, 2],
["a", "b"],
],
ids=idfunc.__func__,
)
def test_vals(self, x, y):
assert x
assert y
This will generate two tests, calling idfunc four times as described above.
TestExample::test_vals[foo1-foo2]
TestExample::test_vals[fooa-foob]
I like wims answer, and this is intended as a comment to his answer (I dont have the points to make a comment). This seems more pythonic to me. It also helps avoid using a static method.
class TestExample:
#pytest.mark.parametrize(
"x, y",
[
[1, 2],
["a", "b"],
],
ids= lamba val : f"foo{val}"
)
def test_vals(self, x, y):
assert x
assert y
This will have the same output:
TestExample::test_vals[foo1-foo2]
TestExample::test_vals[fooa-foob]
While going through some python codes, I found a program as follows
class py_solution:
def sub_sets(self, sset):
return self.subsetsRecur([], sorted(sset))
def subsetsRecur(self, current, sset):
if sset:
**return self.subsetsRecur(current, sset[1:]) + self.subsetsRecur(current + [sset[0]], sset[1:])**
return [current]
print(py_solution().sub_sets([4, 5, 6]))
How does the function call work for this function being called twice in the same return statement(highlighted in bold) and what will be the output of the given query?
It would be helpful if a detailed explanation is given.
This program outputs a set of all possible subsets of the given set.
For the given input it will give all subsets of [4, 5, 6] which are:
[[], [6], [5], [5, 6], [4], [4, 6], [4, 5], [4, 5, 6]]
The first defined function is a wrapper function implemented to pass the relevant parameters for the recursion function.
The recursion function has a stop condition, and that's if the given set is empty, and in that case, it returns the current built set of sets. If the set is not empty the recursion function splits returns two sets which are, a set that has the sset[0] element included and the other set doesn't include sset[0] element. In the recursion that builds every possible subset of the given set (every element can be included or not included).
I'm running Python 2.7.10.
I need to intercept changes in a list. By "change" I mean anything that modifies the list in the shallow sense (the list is not changed if it consists of the same objects in the same order, regardless of the state of those objects; otherwise, it is). I don't need to find out how the list has changed, only that it has. So I just make sure I can detect that, and let the base method do its work. This is my test program:
class List(list):
def __init__(self, data):
list.__init__(self, data)
print '__init__(', data, '):', self
def __getitem__(self, key):
print 'calling __getitem__(', self, ',', key, ')',
r = list.__getitem__(self, key)
print '-->', r
return r
def __setitem__(self, key, data):
print 'before __setitem__:', self
list.__setitem__(self, key, data)
print 'after __setitem__(', key, ',', data, '):', self
def __delitem__(self, key):
print 'before __delitem__:', self
list.__delitem__(self, key)
print 'after __delitem__(', key, '):', self
l = List([0,1,2,3,4,5,6,7]) #1
x = l[5] #2
l[3] = 33 #3
x = l[3:7] #4
del l[3] #5
l[0:4]=[55,66,77,88] #6
l.append(8) #7
Cases #1, #2, #3, and #5 work as I expected; #4, #6, and #7 don't. The program prints:
__init__( [0, 1, 2, 3, 4, 5, 6, 7] ): [0, 1, 2, 3, 4, 5, 6, 7]
calling __getitem__( [0, 1, 2, 3, 4, 5, 6, 7] , 5 ) --> 5
before __setitem__: [0, 1, 2, 3, 4, 5, 6, 7]
after __setitem__( 3 , 33 ): [0, 1, 2, 33, 4, 5, 6, 7]
before __delitem__: [0, 1, 2, 33, 4, 5, 6, 7]
after __delitem__( 3 ): [0, 1, 2, 4, 5, 6, 7]
I'm not terribly surprised by #7: append is probably implemented in an ad-hoc way. But for #4 and #6 I am confused. The __getitem__ documentation says: "Called to implement evaluation of self[key]. For sequence types, the accepted keys should be integers and slice objects." (my emphasys). And for __setitem__: " Same note as for __getitem__()", which I take to mean that key can also be a slice.
What's wrong with my reasoning? I'm prepared, if necessary, to override every list-modifying method (append, extend, insert, pop, etc.), but what should override to catch something like #6?
I am aware of the existence of __setslice__, etc. But those methods are deprecated since 2.0 ...
Hmmm. I read again the docs for __getslice__, __setslice__, etc., and I find this bone-chilling statement:
"(However, built-in types in CPython currently still implement __getslice__(). Therefore, you have to override it in derived classes when implementing slicing.)"
Is this the explanation? Is this saying "Well, the methods are deprecated, but in order to achieve the same functionality in 2.7.10 as you had in 2.0 you still have to override them"? Alas, then why did you deprecate them? How will things work in the future? Is there a "list" class - that I am not aware of - that I could extend and would not present this inconvenience? What do I really need to override to make sure I catch every list-modifying operation?
The problem is that you're subclassing a builtin, and so have to deal with a few wrinkles. Before I delve into that issue, I'll go straight to the "modern" way:
How will things work in the future? Is there a "list" class - that I am not aware of - that I could extend and would not present this inconvenience?
Yes, there's the stdlib Abstract Base Classes. You can avoid the ugly complications caused by subclassing builtin list by using the ABCs instead. For something list-like, try subclassing MutableSequence:
from collections import MutableSequence
class MyList(MutableSequence):
...
Now you should only need to deal with __getitem__ and friends for slicing behaviour.
If you want to push ahead with subclassing the builtin list, read on...
Your guess is correct, you will need to override __getslice__ and __setslice__. The language reference explains why and you already saw that:
However, built-in types in CPython currently still implement __getslice__(). Therefore, you have to override it in derived classes when implementing slicing.
Note that l[3:7] will hook into __getslice__, whereas the otherwise equivalent l[3:7:] will hook into __getitem__, so you have to handle the possibility of receiving slices in both methods... groan!
What I am looking for is a way to do that in python 2.7
oldlist = list
class list(oldlist):
def append(self, object):
super(list, self).append(object)
return self
def sort(self, cmp=None, key=None, reverse=False):
super(list, self).sort(cmp, key, reverse)
return self
__builtins__.list=list
print list([3, 4, 1, 2]).append(5)
print list([3, 4, 1, 2]).append(5).sort()
print list([3, 4, 1, 2]).append(5).sort(reverse=True)
print list([3, 4, 1, 2]).append(5).sort()[0]
print [3, 4, 1, 2].append(5)
print [3, 4, 1, 2].append(5).sort()
print [3, 4, 1, 2].append(5).sort(reverse=True)
print [3, 4, 1, 2].append(5).sort()[0]
Actually print :
[3, 4, 1, 2, 5]
[1, 2, 3, 4, 5]
[5, 4, 3, 2, 1]
1
None
...
AttributeError: 'NoneType' object has no attribute 'sort'
Should print :
[3, 4, 1, 2, 5]
[1, 2, 3, 4, 5]
[5, 4, 3, 2, 1]
1
[3, 4, 1, 2, 5]
[1, 2, 3, 4, 5]
[5, 4, 3, 2, 1]
1
I know it can be dangerous to edit builtins class, but some methods really return nothing, do a python script actually expect them to return something, so what the problem ?
For now I think that much simple to do :
filterfiles(myfilelist.sort())
than doing :
myfilelist.sort()
filterfiles(myfilelist)
And it permit to see the results when in interactive mode (instead of nothing)
One thing I don't understand is that when we put {1:1, 2:2}, python look for making the dict literal into a dict object, and I know I can't change python to make an instance of mydict, but is there a way to change the builtins dict directly, whereas it use somes hacky way?
No, it’s simply not possible. Literals, that means any literal (strings, numbers, lists, dicts), are part of the Python syntax. The objects they represent are created from the parser at a very low level, long before you can change anything with Python code.
There is another important thing though. The built-in objects are all implemented in native code; they don’t actually exist as Python objects in the Python environment. For that purpose, things like __builtins__.dict provides a reference to the native dictionary type. When the objects are created with literals, the real native type is used though, so __builtins__.dict is never accessed. As such, changing __builtins__.dict will not affect it at all. It will only change the environment, where these references actually matter.
You can imagine this situation like this:
# native code
class _InternalSpecialType:
pass
def makeSpecialType (): # this is our “literal” evaluator
return _InternalSpecialType()
# public interface
SpecialType = _InternalSpecialType
# then in the Python code
class NewSpecialType(SpecialType):
pass
SpecialType = NewSpecialType
# using a “literal”
x = makeSpecialType()
print(type(x)) # _InternalSpecialType
So no, you can’t change what the literal uses under the hood. It’s simply impossible. If you want to have an object of a different type, you will always have to create it explicitely. And then it’s best to name the type differently than the original type to avoid confusion (and incompatibility between the changed type and literals).
And finally, about methods of built-in types not allowing chaining: Just live with it. Guido knowingly decided against it, and usually, Guido has good reasons you can trust, so it’s likely for the better (see also this answer).
I'll explain how to solve the problem you have, rather than how to implement the solution you're after:
Write filterfiles(sorted(myfilelist)).
Methods that return None do so by design: In this case, to avoid inadvertently sorting a list in-place (and losing its current ordering) when you really wanted a sorted copy. Python already provides functional alternatives for such cases, like sorted() in this case, when it makes sense to. Note that sorted() does not modify its argument.
If you do find a use case for which no functional alternative is provided, I would recommend you get around it in the same way: Write a function (not method) that returns what you want to see. (But check out python's functools module first).