Alias for dictionary operation in Python - python

I want to do something like this:
f[frozenset((1,3,4))] = 5
f[frozenset((1,))] = 3
but it's just painful to type these all the time, is there anyway to have alias for this? I know in C++ it's possible to have a helper function which return a reference so you can just type:
F(1,3,4) = 5
F(1) = 3
with F as a helper function. Thanks very much!

I think that this can really only be achieved via a subclass:
class FrozenSetDict(dict):
def __setitem__(self,idx,value):
try:
dict.__setitem__(self,frozenset(idx),value)
except TypeError:
dict.__setitem__(self,frozenset((idx,)),value)
d = FrozenSetDict()
d[1,2,3] = 4
d[1] = 5
print d
yields:
{frozenset([1, 2, 3]): 4, frozenset([1]): 5}
This introduces an asymmetry between __getitem__ and __setitem__ which could easily be fixed by re-defining __getitem__ in the same way.
This might seem a little messy -- Indeed it is. Why require a subclass? That just makes it harder to put non-frozenset objects into your dictionary as keys. You could easily use this recipe though to create a proxy object which will do this with your dict:
#I don't like the name of this class -- I'm open to suggestions :)
class FrozenSetProxy(object):
def __init__(self,obj):
self.obj = obj
def __setitem__(self,idx,value):
try:
self.obj[frozenset(idx)] = value
except TypeError:
self.obj[frozenset((idx,))] = value
def __getitem__(self,idx):
try:
return self.obj[frozenset(idx)]
except TypeError:
return self.obj[frozenset((idx,))]
d = dict()
F = FrozenSetProxy(d)
F[1,2,3] = 4
F[1] = 5
print d
print F[1]

There's nothing like a C++ reference in Python, and the syntax you use is illegal to boot (in the words of the parser: can't assign to function call). You could emulate it with an object or subclass dict to customize its __getitem__. But there's a simpler and less intrusive way: Pass the value to the helper too, and let it handle the assignment:
def blah(f):
def F(*args, value):
f[frozenset(args)] = value
F(1, 3, 4, value=5)
F(1, value=3)
Note that this uses a Python 3 feature, keyword-only parameters. If you need it to work with Python 2, you can emulate the call syntax by accepting **kwdargs:
def F(*args, **kwds):
# optional: check that no other keyword arguments were passed
f[frozenset(args)] = kwds['value']

Related

equivalent to R's `do.call` in python

Is there an equivalent to R's do.call in python?
do.call(what = 'sum', args = list(1:10)) #[1] 55
do.call(what = 'mean', args = list(1:10)) #[1] 5.5
?do.call
# Description
# do.call constructs and executes a function call from a name or a function and a list of arguments to be passed to it.
There is no built-in for this, but it is easy enough to construct an equivalent.
You can look up any object from the built-ins namespace using the __builtin__ (Python 2) or builtins (Python 3) modules then apply arbitrary arguments to that with *args and **kwargs syntax:
try:
# Python 2
import __builtin__ as builtins
except ImportError:
# Python 3
import builtins
def do_call(what, *args, **kwargs):
return getattr(builtins, what)(*args, **kwargs)
do_call('sum', range(1, 11))
Generally speaking, we don't do this in Python. If you must translate strings into function objects, it is generally preferred to build a custom dictionary:
functions = {
'sum': sum,
'mean': lambda v: sum(v) / len(v),
}
then look up functions from that dictionary instead:
functions['sum'](range(1, 11))
This lets you strictly control what names are available to dynamic code, preventing a user from making a nuisance of themselves by calling built-ins for their destructive or disruptive effects.
do.call is pretty much the equivalent of the splat operator in Python:
def mysum(a, b, c):
return sum([a, b, c])
# normal call:
mysum(1, 2, 3)
# with a list of arguments:
mysum(*[1, 2, 3])
Note that I’ve had to define my own sum function since Python’s sum already expects a list as an argument, so your original code would just be
sum(range(1, 11))
R has another peculiarity: do.call internally performs a function lookup of its first argument. This means that it finds the function even if it’s a character string rather than an actual function. The Python equivalent above doesn’t do this — see Martijn’s answer for a solution to this.
Goes similar to previous answer, but why so complicated?
def do_call(what, args=[], kwargs = {}):
return what(*args, **kwargs)
(Which is more elegant than my previously posted definition:)
def do_call(which, args=None, kwargs = None):
if args is None and kwargs is not None:
return which(**kwargs)
elif args is not None and kwargs is None:
return which(*args)
else:
return which(*args, **kwargs)
Python's sum is different than R's sum (1 argument a list expected vs.
arbitraily many arguments expected in R). So we define our own sum (mysum)
which behaves similarly to R's sum. In a similar way we define mymean.
def mysum(*args):
return sum(args)
def mymean(*args):
return sum(args)/len(args)
Now we can recreate your example in Python - as a reasonable 1:1 translation of the R function call.
do_call(what = mymean, args=[1, 2, 3])
## 2.0
do_call(what = mysum, args=[1, 2, 3])
## 6
For functions with argument names, we use a dict for kwargs, where the parameter
names are keys of the dictionary (as strings) and their values the values.
def myfunc(a, b, c):
return a + b + c
do_call(what = myfunc, kwargs={"a": 1, "b": 2, "c": 3})
## 6
# we can even mix named and unnamed parts
do_call(what = myfunc, args = [1, 2], kwargs={"c": 3})
## 6

Python equivalent of C++ member pointer

What would be the equivalent of a C++ member pointer in Python? Basically, I would like to be able to replicate similar behavior in Python:
// Pointer to a member of MyClass
int (MyClass::*ptMember)(int) = &MyClass::member;
// Call member on some instance, e.g. inside a function to
// which the member pointer was passed
instance.*ptMember(3)
Follow up question, what if the member is a property instead of a method? Is it possible to store/pass a "pointer" to a property without specifying the instance?
One way would obviously be to pass a string and use eval. But is there a cleaner way?
EDIT: There are now several really good answers, each having something useful to offer depending on the context. I ended up using what is described in my answer, but I think that other answers will be very helpful for whoever comes here based on the topic of the question. So, I am not accepting any single one for now.
Assuming a Python class:
class MyClass:
def __init__(self):
self.x = 42
def fn(self):
return self.x
The equivalent of a C++ pointer-to-memberfunction is then this:
fn = MyClass.fn
You can take a method from a class (MyClass.fn above) and it becomes a plain function! The only difference between function and method is that the first parameter is customarily called self! So you can call this using an instance like in C++:
o = MyClass()
print(fn(o)) # prints 42
However, an often more interesting thing is the fact that you can also take the "address" of a bound member function, which doesn't work in C++:
o = MyClass()
bfn = o.fn
print(bfn()) # prints 42, too
Concerning the follow-up with the properties, there are plenty answers here already that address this issue, provided it still is one.
The closest fit would probably be operator.attrgetter:
from operator import attrgetter
foo_member = attrgetter('foo')
bar_member = attrgetter('bar')
baz_member = attrgetter('baz')
class Example(object):
def __init__(self):
self.foo = 1
#property
def bar(self):
return 2
def baz(self):
return 3
example_object = Example()
print foo_member(example_object) # prints 1
print bar_member(example_object) # prints 2
print baz_member(example_object)() # prints 3
attrgetter goes through the exact same mechanism normal dotted access goes through, so it works for anything at all you'd access with a dot. Instance fields, methods, module members, dynamically computed attributes, whatever. It doesn't matter what the type of the object is, either; for example, attrgetter('count') can retrieve the count attribute of a list, tuple, string, or anything else with a count attribute.
For certain types of attribute, there may be more specific member-pointer-like things. For example, for instance methods, you can retrieve the unbound method:
unbound_baz_method = Example.baz
print unbound_baz_method(example_object) # prints 3
This is either the specific function that implements the method, or a very thin wrapper around the function, depending on your Python version. It's type-specific; list.count won't work for tuples, and tuple.count won't work for lists.
For properties, you can retrieve the property object's fget, fset, and fdel, which are the functions that implement getting, retrieving, and deleting the attribute the property manages:
example_bar_member = Example.bar.fget
print example_bar_member(example_object) # prints 2
We didn't implement a setter or deleter for this property, so the fset and fdel are None. These are also type-specific; for example, if example_bar_member handled lists correctly, example_bar_member([]) would raise an AttributeError rather than returning 2, since lists don't have a bar attribute.
I was not satisfied with the string approach and did some testing. This seems to work pretty well and avoids passing strings around:
import types
# Our test class
class Class:
def __init__(self, val):
self._val = val
def method(self):
return self._val
#property
def prop(self):
return self._val
# Get the member pointer equivalents
m = Class.method
p = Class.prop
# Create an instance
c1 = Class(1)
# Bind the method and property getter to the instance
m1 = types.MethodType(m, c1)
p1 = types.MethodType(p.fget, c1)
# Use
m1() # Returns 1
p1() # Returns 1
# Alternatively, the instance can be passed to the function as self
m(c1) # Returns 1
p.fget(c1) # Returns 1
I'm not a C++ programmer, so maybe I'm missing some detail of method pointers here, but it sounds like you just want a reference to a function that's defined inside a class. (These were of type instancemethod in Python 2, but are just type function in Python 3.)
The syntax will be slightly different --- instead of calling it like a method with object.reference(args), you'll call it like a function: reference(object, args). object will be the argument to the self parameter --- pretty much what the compiler would have done for you.
Despite the more C-like syntax, I think it still does what you wanted... at least when applied to a callable member like in your example. It won't help with a non-callable instance field, though: they don't exist until after __init__ runs.
Here's a demonstration:
#!/usr/bin/env python3
import math
class Vector(object):
def __init__(self, x, y):
self.x = x
self.y = y
return
def __str__(self):
return '(' + str(self.x) + ', ' + str(self.y) + ')'
def __repr__(self):
return self.__class__.__name__ + str(self)
def magnitude(self):
return math.sqrt(self.x ** 2 + self.y ** 2)
def print_dict_getter_demo():
print('Demo of member references on a Python dict:')
dict_getter = dict.get
d = {'a': 1, 'b': 2, 'c': 3, 'z': 26}
print('Dictionary d : ' + str(d))
print("d.get('a') : " + str(d.get('a')))
print("Ref to get 'a' : " + str(dict_getter(d, 'a')))
print("Ref to get 'BOGUS': " + str(dict_getter(d, 'BOGUS')))
print('Ref to get default: ' + str(dict_getter(d, 'BOGUS', 'not None')))
return
def print_vector_magnitude_demo():
print('Demo of member references on a user-defined Vector:')
vector_magnitude = Vector.magnitude
v = Vector(3, 4)
print('Vector v : ' + str(v))
print('v.magnitude() : ' + str(v.magnitude()))
print('Ref to magnitude: ' + str(vector_magnitude(v)))
return
def print_vector_sorting_demo():
print('Demo of sorting Vectors using a member reference:')
vector_magnitude = Vector.magnitude
v0 = Vector(0, 0)
v1 = Vector(1, 1)
v5 = Vector(-3, -4)
v20 = Vector(-12, 16)
vector_list = [v20, v0, v5, v1]
print('Unsorted: ' + str(vector_list))
sorted_vector_list = sorted(vector_list, key=vector_magnitude)
print('Sorted: ' + str(sorted_vector_list))
return
def main():
print_dict_getter_demo()
print()
print_vector_magnitude_demo()
print()
print_vector_sorting_demo()
return
if '__main__' == __name__:
main()
When run with Python 3, this produces:
Demo of member references on a Python dict:
Dictionary d : {'a': 1, 'c': 3, 'b': 2, 'z': 26}
d.get('a') : 1
Ref to get 'a' : 1
Ref to get 'BOGUS': None
Ref to get default: not None
Demo of member references on a user-defined Vector:
Vector v : (3, 4)
v.magnitude() : 5.0
Ref to magnitude: 5.0
Demo of sorting Vectors using a member reference:
Unsorted: [Vector(-12, 16), Vector(0, 0), Vector(-3, -4), Vector(1, 1)]
Sorted: [Vector(0, 0), Vector(1, 1), Vector(-3, -4), Vector(-12, 16)]
As you can see, it works with both builtins and user-defined classes.
Edit:
The huge demo above was based on an assumption: that you had a reference to the class, and that your goal was to "hold on to" to one of the class's methods for use on whatever instances of that class showed up sometime later.
If you already have a reference to the instance, it's much simpler:
d = {'a': 1, 'b': 2, 'c': 3, 'z': 26}
d_getter = d.get
d_getter('z') # returns 26
This is basically the same thing as above, only after the transformation from a function into a method has "locked in" the argument to self, so you don't need to supply it.
The way I would approach this in python is to use __getattribute__. If you have the name of an attribute, which would be the analog of the c++ pointer-to-member, you could call a.__getattribute__(x) to get the attribute whose name is stored in x. It's strings and dicts instead of offsets & pointers, but that's python.

Python Class elements as Function parameters

class params:
a = 1
b = 2
c = 3
d = 4
I know it is not possible to pass all four elements of the class to a function, which takes a total of four parameters by just doing
function(class)
Is this the only way
function(a,b,c,d)
or is there an other solution?
I think 4 arguments is acceptable. But if you do want, define a function such as func(a, *args, **kargs) is a choice. Any of the three parameters is optional. I do think there is no need to create such a class to do something that Python has already done, and do much better. For more about function arguments, check here.
You could do something like this:
class params:
a = 1
b = 2
c = 3
d = 4
def __call__(self):
r = dict()
for attr in dir(self):
if not attr.startswith('_'):
r[attr] = getattr(self, attr)
return r
p = params()
function(**p())
However, this is quite hacky and probably not very maintainable: if for some reason the function call for function changes, you need to modify the class also.
(Perhaps using a tuple() instead of dict() for r might be a bit more flexible, but still not a solution I would highly recommend...)

Getting the block of commands that are to be executed in the with statement

In reading the specifications for the with statement (link), I have some things I'd like to play around with. This isn't for any production code or anything, I'm just exploring, so please don't be too harsh if this is a bad idea.
What I'd like to do is grab the piece called "BLOCK" in the linked docs above, and actually tinker around with it inside of the call to __enter__. (See the linked doc, just after the start of the motivation and summary section.)
The idea is to create my own sort of on-the-fly local namespace. Something like this:
with MyNameSpace(some_object):
print a #Should print some_object.a
x = 4 #Should set some_object.x=4
Basically, I want the statements inside of the with block to be subordinate to the local variables and assignment conventions of some_object.
In my specific case, some_object might be a special data array that has my own column-wise operations or something. In which case saying something like x = y + 5 if y > 4 else y - 2 might be some fancy NumPy vectorized operation under the hood, but I don't need to explicitly call some_object's interface to those methods. In the namespace, the expressions should "just work" (however I define them to be inferred in the MyNameSpace class.
My first idea is to somehow interrupt the with process and get a hold of the code that goes in the try block. Then interpret that code when __enter__ gets called, and replace the code in the try block with something else (perhaps pass if that would work, but possibly something that restores some_object back to the original variable scope with its new changed variables preserved).
A simple test case would be something like this:
my_dict = {'a':3, 'b':2}
with MyNameSpace(my_dict):
print a # Should print 3
x = 5 # When the block finishes, my_dict['x'] should now be 5
I'm interested if this idea exists somewhere already.
I am aware of best practices things for assigning variables. This is a pet project, so please assume that, just for the sake of this idea, we can ignore best practices. Even if you wouldn't like assigning variables this way, it could be useful in my current project.
Edit
To clarify the kinds of tricky stuff I might want to do, and to address the answer below claiming that it can't be done, consider the example file testLocals.py below:
my_dict = {'a':1, 'b':2}
m = locals()
print m["my_dict"]['a']
m["my_dict"]['c'] = 3
print my_dict
class some_other_scope(object):
def __init__(self, some_scope):
x = 5
g = locals()
some_scope.update(g)
some_scope["my_dict"]["d"] = 4
sos = some_other_scope(m)
print my_dict
print x
which gives the following when I run it non-interactively:
ely#AMDESK:~/Desktop/Programming/Python$ python testLocals.py
1
{'a': 1, 'c': 3, 'b': 2}
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
5
Try this.
import sys
class MyNameSpace(object):
def __init__(self,ns):
self.ns = ns
def __enter__(self):
globals().update(self.ns)
def __exit__(self, exc_type,exc_value,traceback):
self.ns.update(sys._getframe(1).f_locals)
my_dict = {'a':3, 'b':2}
with MyNameSpace(my_dict) as ns:
print(a) # Should print 3
x = 5 # When the block finishes, my_dict['x'] should now be 5
print(my_dict['x'])
Here is something similar I tried a few months ago:
import sys
import inspect
import collections
iscallable = lambda x: isinstance(x, collections.Callable)
class Namespace(object):
def __enter__(self):
"""store the pre-contextmanager scope"""
f = inspect.currentframe(1)
self.scope_before = dict(f.f_locals)
return self
def __exit__(self, exc_type, exc_value, traceback):
"""determine the locally declared objects"""
f = inspect.currentframe(1)
scope_after = dict(f.f_locals)
scope_context = set(scope_after) - set(self.scope_before)
# capture the local scope, ignoring the context manager itself
self.locals = dict(
(k, scope_after[k]) for k in scope_context if not isinstance(scope_after[k], self.__class__)
)
for name in self.locals:
obj = scope_after[name]
if iscallable(obj):
# closure around the func_code with the appropriate locals
_wrapper = type(lambda: 0)(obj.func_code, self.locals)
self.__dict__[name] = _wrapper
# update locals so the calling functions refer to the wrappers too
self.locals[name] = _wrapper
else:
self.__dict__[name] = obj
# remove from module scope
del sys.modules[__name__].__dict__[name]
return self
with Namespace() as Spam:
x = 1
def ham(a):
return x + a
def cheese(a):
return ham(a) * 10
It uses inspect to modify locals while within the context manager and then to re-assign back to the original values when done.
It's not perfect - I can't remember where it hits issues, but I'm sure it does - but it might help you get started.

Ignore python multiple return value

Say I have a Python function that returns multiple values in a tuple:
def func():
return 1, 2
Is there a nice way to ignore one of the results rather than just assigning to a temporary variable? Say if I was only interested in the first value, is there a better way than this:
x, temp = func()
You can use x = func()[0] to return the first value, x = func()[1] to return the second, and so on.
If you want to get multiple values at a time, use something like x, y = func()[2:4].
One common convention is to use a "_" as a variable name for the elements of the tuple you wish to ignore. For instance:
def f():
return 1, 2, 3
_, _, x = f()
If you're using Python 3, you can you use the star before a variable (on the left side of an assignment) to have it be a list in unpacking.
# Example 1: a is 1 and b is [2, 3]
a, *b = [1, 2, 3]
# Example 2: a is 1, b is [2, 3], and c is 4
a, *b, c = [1, 2, 3, 4]
# Example 3: b is [1, 2] and c is 3
*b, c = [1, 2, 3]
# Example 4: a is 1 and b is []
a, *b = [1]
The common practice is to use the dummy variable _ (single underscore), as many have indicated here before.
However, to avoid collisions with other uses of that variable name (see this response) it might be a better practice to use __ (double underscore) instead as a throwaway variable, as pointed by ncoghlan. E.g.:
x, __ = func()
Remember, when you return more than one item, you're really returning a tuple. So you can do things like this:
def func():
return 1, 2
print func()[0] # prints 1
print func()[1] # prints 2
The best solution probably is to name things instead of returning meaningless tuples (unless there is some logic behind the order of the returned items). You can for example use a dictionary:
def func():
return {'lat': 1, 'lng': 2}
latitude = func()['lat']
You could even use namedtuple if you want to add extra information about what you are returning (it's not just a dictionary, it's a pair of coordinates):
from collections import namedtuple
Coordinates = namedtuple('Coordinates', ['lat', 'lng'])
def func():
return Coordinates(lat=1, lng=2)
latitude = func().lat
If the objects within your dictionary/tuple are strongly tied together then it may be a good idea to even define a class for it. That way you'll also be able to define more complex operations. A natural question that follows is: When should I be using classes in Python?
Most recent versions of python (≥ 3.7) have dataclasses which you can use to define classes with very few lines of code:
from dataclasses import dataclass
#dataclass
class Coordinates:
lat: float = 0
lng: float = 0
def func():
return Coordinates(lat=1, lng=2)
latitude = func().lat
The primary advantage of dataclasses over namedtuple is that its easier to extend, but there are other differences. Note that by default, dataclasses are mutable, but you can use #dataclass(frozen=True) instead of #dataclass to force them being immutable.
Here is a video that might help you pick the right data class for your use case.
Three simple choices.
Obvious
x, _ = func()
x, junk = func()
Hideous
x = func()[0]
And there are ways to do this with a decorator.
def val0( aFunc ):
def pick0( *args, **kw ):
return aFunc(*args,**kw)[0]
return pick0
func0= val0(func)
This seems like the best choice to me:
val1, val2, ignored1, ignored2 = some_function()
It's not cryptic or ugly (like the func()[index] method), and clearly states your purpose.
If this is a function that you use all the time but always discard the second argument, I would argue that it is less messy to create an alias for the function without the second return value using lambda.
def func():
return 1, 2
func_ = lambda: func()[0]
func_() # Prints 1
This is not a direct answer to the question. Rather it answers this question: "How do I choose a specific function output from many possible options?".
If you are able to write the function (ie, it is not in a library you cannot modify), then add an input argument that indicates what you want out of the function. Make it a named argument with a default value so in the "common case" you don't even have to specify it.
def fancy_function( arg1, arg2, return_type=1 ):
ret_val = None
if( 1 == return_type ):
ret_val = arg1 + arg2
elif( 2 == return_type ):
ret_val = [ arg1, arg2, arg1 * arg2 ]
else:
ret_val = ( arg1, arg2, arg1 + arg2, arg1 * arg2 )
return( ret_val )
This method gives the function "advanced warning" regarding the desired output. Consequently it can skip unneeded processing and only do the work necessary to get your desired output. Also because Python does dynamic typing, the return type can change. Notice how the example returns a scalar, a list or a tuple... whatever you like!
When you have many output from a function and you don't want to call it multiple times, I think the clearest way for selecting the results would be :
results = fct()
a,b = [results[i] for i in list_of_index]
As a minimum working example, also demonstrating that the function is called only once :
def fct(a):
b=a*2
c=a+2
d=a+b
e=b*2
f=a*a
print("fct called")
return[a,b,c,d,e,f]
results=fct(3)
> fct called
x,y = [results[i] for i in [1,4]]
And the values are as expected :
results
> [3,6,5,9,12,9]
x
> 6
y
> 12
For convenience, Python list indexes can also be used :
x,y = [results[i] for i in [0,-2]]
Returns : a = 3 and b = 12
It is possible to ignore every variable except the first with less syntax if you like. If we take your example,
# The function you are calling.
def func():
return 1, 2
# You seem to only be interested in the first output.
x, temp = func()
I have found the following to works,
x, *_ = func()
This approach "unpacks" with * all other variables into a "throwaway" variable _. This has the benefit of assigning the one variable you want and ignoring all variables behind it.
However, in many cases you may want an output that is not the first output of the function. In these cases, it is probably best to indicate this by using the func()[i] where i is the index location of the output you desire. In your case,
# i == 0 because of zero-index.
x = func()[0]
As a side note, if you want to get fancy in Python 3, you could do something like this,
# This works the other way around.
*_, y = func()
Your function only outputs two potential variables, so this does not look too powerful until you have a case like this,
def func():
return 1, 2, 3, 4
# I only want the first and last.
x, *_, d = func()

Categories

Resources