I have this example code:
class objectTest():
def __init__(self, a):
self.value = a
def get_value(self):
return self.value
a = objectTest(1)
b = objectTest(1)
print(a == b)
print(a.get_value() == b.get_value)
print(a.get_value() == b.get_value())
print(a.get_value == b.get_value)
The displayed results are:
False
False
True
False
Why doesn't this cause an error? get_value is a method, so why can we use it like this without calling it first?
See also: In Python, what does '<function at ...>' mean?
As mentioned, functions and methods are first-class objects. You call them by throwing some parentheses (brackets) on the end. But it looks like you want some more motivation for why python even lets us do that. Why should we care if functions are first-class or not?
Sometimes you don't want to call them, you want to pass a reference to the callable itself.
from multiprocessing import Process
t = Process(target=my_long_running_function)
If you put brackets after the above, it runs your my_long_running_function in your main thread; hardly what you wanted! You wanted to give Process a reference to your callable that it will run itself in a new process.
Sometimes you just want to specify the callable and let something else...
def do_something(s):
return s[::-1].upper()
map(do_something,['hey','what up','yo'])
Out[3]: ['YEH', 'PU TAHW', 'OY']
(map in this case) fill in its arguments.
Maybe you just want to drop a bunch of callables into some collection, and fetch the one you want in a dynamic manner.
from operator import *
str_ops = {'<':lt,'>':gt,'==':eq} # etc
op = str_ops.get(my_operator)
if op:
result = op(lhs,rhs)
The above is one way to map string representations of operators onto their actual action.
Functions and methods in Python are also objects themselves. Thus you can compare them just as you would any other object.
>>> type(a.get_value)
<type 'instancemethod'>
>>> type(a.get_value())
<type 'int'>
Normally of course you wouldn't compare methods to each other or anything else, because it's not terribly useful. One place it's useful is when you want to pass a function into another function.
def mul(a, b):
return a * b
def add(a, b):
return a + b
def do(op, a, b):
return op(a, b)
do(add, 2, 3) # return 5
print(a.get_value() == b.get_value) # 1
print(a.get_value() == b.get_value()) # 2
print(a.get_value == b.get_value) # 3
1) Is return value of calling a.get_value() equal to the method b.get_value ?
2) Does a.get_value() return the same as b.get_value() ?
3) Is the method-reference a.get_value equal to the method-reference b.get_value ?
This is perfectly valid Python :)
Several commentators want an example of where this is useful. One application is in threading. We need to pass the target to the thread without using brackets. Otherwise the target is created in the main thread, which is what we are trying to avoid.
Example:
In test1.py I call ThreadTest without using brackets. test_thread starts in the thread and allows test1.py to continue running.
In test2.py, I pass ThreadTest() as the target. In this case the thread does not allow test2.py to continue running.
test1.py
import threading
from thread_test import ThreadTest
thread = threading.Thread(target=ThreadTest)
thread.start()
print('not blocked')
test2.py
import threading
from thread_test import ThreadTest
thread = threading.Thread(target=ThreadTest())
thread.start()
print('not blocked')
test_thread.py
from time import sleep
class ThreadTest():
def __init__(self):
print('thread_test started')
while True:
sleep(1)
print('test_thread')
output from test1.py:
thread_test started
not blocked
test_thread
test_thread
test_thread
output from test2.py:
thread_test started
test_thread
test_thread
test_thread
I am using python3.5 on Linux Mint.
Difference between function without parentheses and with parentheses is that when using parentheses you will get the output of that function and when you use the function without parentheses you create a copy of that function.
for example
def outerFunction(text):
text = text
def innerFunction():
print(text)
return innerFunction()
if __name__ == '__main__':
outerFunction('Hey!')
x = outerFunction
y = x
x('Hey i am busy can you call me later')
y('this is not a function')
here we copy the function outerFunction to x and then copy y to x.
Related
So I am not sure if this can be done at all, but was curious anyway. Let us suppose I have the following piece of code:
def statement1():
# Do something
def statement2():
# Do something
def statement3():
# Do something
statement1()
statement2()
statement3()
Now, as you can see here, I have to execute all the above functions one after the other. They are also named in a similar fashion, so that made me wonder- is there a way to loop over function calls?
I basically want to have something like a for loop which loops over 'statements', and I don't end up calling each one again and again with only a slight modification in the name.
A side note- I am new to the community so feel free to give me feedback on how to ask and comment on the answers, would go a long way in helping me.
Thanks in advance!
You can use globals() with fstring if the function name is always follow the specific pattern
def statement1():
print("s1")
def statement2():
print("s2")
def statement3():
print("s3")
for i in range(3):
globals()[f"statement{i+1}"]()
Output:
s1
s2
s3
Try this:
def st1():
return 'Hi 1'
def st2():
return 'Hi 2'
def st3():
return 'Hi 3'
list1 = [st1(), st2(), st3()]
for word in list1:
print(word)
Here is a way if you don't want to use globals:
import sys
def statement1():
print(1)
def statement2():
print(2)
def statement3():
print(3)
for i in range(1, 4):
f_name = 'statement{}'.format(i)
getattr(sys.modules[__name__], f_name)()
#deadshot has provided one option how it can be done. On the other hand, if you want to design it this way, I guess this is because you plan that the standard usage will require running all three functions in this exact order? In this case the more classic way would be to define a main function:
def statement1():
# Do something
def statement2():
# Do something
def statement3():
# Do something
def main():
statement1()
statement2()
statement3()
if __name__ == '__main__':
main()
If the functions to similar things, you usually do this via arguments:
def statement(arg):
# do something with arg
for i in range(3):
statement(i)
Consider using this one eval() built in function in python; you could even use eval() when you want to pass parameters to the functions as you can see bellow. However, you should be careful when using eval with the OS module imported and taking input from the user with input() since eval() runs and executes everything it receives.
def statement1(num):
print("Statement_{} -> number-{}".format(num, num))
def statement2(num):
print("Statement_{} -> number-{}".format(num, num))
def statement3(num):
print("Statement_{} -> number-{}".format(num, num))
for i in range(3):
eval("statement{}({})".format(i+1, i+1))
Output would look like:
Statement_1 -> number-1
Statement_2 -> number-2
Statement_3 -> number-3
You can choose the functions which have sign word,the run the choosed functions use vars() or globals() dict.
def statement1():
print("s1")
def statement2():
print("s2")
def statement3():
print("s3")
functionSign = 'statement'
for varName in list(vars().keys()):
if functionSign in varName:
vars()[varName]()
Output:
s1
s2
s3
Below is solution. It is inelegant but will work. Please wait for an elegant solution to appear.
import re
def statement1():
# Do something
print("Statement 1")
def statement2():
# Do something
print("Statement 2")
def statement3():
# Do something
print("Statement 3")
for i in dir():
if re.fullmatch(r'^__[a-zA-Z0-9_]*__$' , i):
continue
else:
if re.fullmatch(r'^statement[0-9]*$',i):
eval('%s()'%i)
The output is
Statement 1
Statement 2
Statement 3
Functions are objects as well so you can used them as variables. Something like:
fns = [statement1, statement2, statement3]
for fn in fns:
fn()
I made some tests about this setting, that appeared unexpectedly as a quick fix for my problem:
I want to call a multiprocessing.Pool.map() from inside a main function (that sets up the parameters). However it is simpler for me to give a locally defined function as one of the args. Since the latter can't be pickled, I tried the laziest solution of declaring it as global. Should I expect some weird results? Would you advise a different strategy?
Here is an example (dummy) code:
#!/usr/bin/env python3
import random
import multiprocessing as mp
def processfunc(arg_and_func):
arg, func = arg_and_func
return "%7.4f:%s" %(func(arg), arg)
def main(*args):
# the content of var depends of main:
var = random.random()
# Now I need to pass a func that uses `var`
global thisfunc
def thisfunc(x):
return x+var
# Test regular use
for x in range(-5,0):
print(processfunc((x, thisfunc)))
# Test parallel runs.
with mp.Pool(2) as pool:
for r in pool.imap_unordered(processfunc, [(x, thisfunc) for x in range(20)]):
print(r)
if __name__=='__main__':
main()
PS: I know I could define thisfunc at module level, and pass the var argument through processfunc, but since my actual processfunc in real life already takes a lot of arguments, it seemed more readable to pass a single object thisfunc instead of many parameters...
What you have now looks OK, but might be fragile for later changes.
I might use partial in order to simplify the explicit passing of var to a globally defined function.
import random
import multiprocessing as mp
from functools import partial
def processfunc(arg_and_func):
arg, func = arg_and_func
return "%7.4f:%s" %(func(arg), arg)
def thisfunc(var, x):
return x + var
def main(*args):
# the content of var depends of main:
var = random.random()
f = partial(thisfunc, var)
# Test regular use
for x in range(-5,0):
print(processfunc((x, thisfunc)))
# Test parallel runs.
with mp.Pool(2) as pool:
for r in pool.imap_unordered(processfunc, [(x, f) for x in range(20)]):
print(r)
if __name__=='__main__':
main()
I saw this particular piece of code:
def g(x,y):
return x+y
def g(x,y):
return x*y
x,y=6,7
print (g(x,y))
The output is obviously(but not to me) is 42. Can somebody please explain this behavior? This is method overriding I suppose, but I'm still not getting the flow here.
When you define a function, and you redefine it, it will use the last one you defined, even the parameter is different:
def g(x,y):
return x+y
def g(x,y):
return x*y
x,y=6,7
print (g(x,y))
def hello():
return 'hello'
def hello():
return 'bye'
print hello()
def withone(word):
return word
def withone():
return 1==1
print withone('ok')
Output:
42
bye
TypeError: withone() takes no arguments (1 given)
And function name in Python is more like simple variable:
def hello():
return 'hello'
iamhello = hello # bind to the old one
def hello():
return 'bye'
print hello() # here is the new guy
print iamhello()
OutPut:
bye
hello
The devil is in the order of function definitions.
This is not technically method overriding as that requires class inheritance, instead it's a result of how python declares and references functions.
When declaring a function, python stores a reference to that function in a variable named after the function definition. (e.g. variable would be "foo" for "def foo():")
By declaring the function twice, the value of that variable gets overwritten by the second definition.
A Python script is parsed from top till bottom.
So anytime the same name of a variable or function or class occurs, it overwrites any definitions that where associated with this name before.
def g(x,z):
print('first')
def g():
print('second')
g = 3
print g
print g()
So look at this example which will result in the printout of '3' and then in an Exception: 'TypeError: 'int' object is not callable'
The name g is at first a function with two parameters, then it gets redefined to be a function with no parameters, then it gets redefined to be an int.
Which cannot be called obviously :)
Everything in python is treated as object, whether it be a function name or class name. So, when we define a function using 'def', the memory allocation is done for that method. Then python points the name that we assign to the function, to this allocated memory location. So if we define a method :-
def demo():
print 'hi'
the memory is allocated for the method, and the name 'demo' is pointed to its memory location as follows :-
Now as described by zoosuck in his second example, when you assign the function name to another variable :-
demo2 = demo # bind to the old one
then in that case, the assigned memory location to demo, is assigned to demo2 as well. So both demo and demo2 points to same location 12506.
print id(demo) # will print 12506
print id(demo2) # will print 12506
Now if we modify the above piece of code and in the next line, define a new method with same name demo:-
def demo():
print 'hi'
demo2 = demo # bind to the old one
demo() # Will print hi
def demo():
print "hello"
demo() # Will print hello
demo2() # Will print hi
then a completely new memory location 12534 is allocated for this new method, and now demo will point to this new location 12534 instead of pointing to the old one i.e. to 12506. But demo2 is still pointing to the location 12506.
I hope this will give you a clear idea of what is going on and how the method name is over-written.
Order matters, if names are same,last function you defined is processing. In your case it's;
def g(x,y):
return x*y
g is just a variable. The fact that the object it refers to is a function doesn't make it special in Python, so you can assign and reassign it as you want. In this case, the second assignment (which is what a function definition is) simply replaces the object stored there with a different one.
Functions and methods are normal objects like any others. So in
def g(x, y):
return x + y
def g(x, y):
return x * y
the second object g will override(replace) the first one, just like object a does below:
a = 1
a = 2
The number, type or order of parameters does not make any difference, because Python does not support function/method override and does not allow two functions/methods to have the same name.
If you are familiar with lambda function, also often called anonymous\inline functions, this might clear things up a bit
These two code blocks are essentially equal
def g(x,y):
return x+y
def g(x,y):
return x*y
g = lambda x,y: x+y
g = lambda x,y: x*y
So i have a relatively convoluted setup for something I'm working on explained as follows:
This is is python. and more of a rough outline, but it covers everything I need. Though the process next function is the same so feel free to clean that up if you want.
#timer event that runs every .1 second and processes events in a queue
some_event_timer():
events.process_next()
class Event_queue:
def __init__(self):
self.events = []
def push(self, event, parameters):
self.events.insert(len(self.events), event, parameters)
def process_next(self):
event = self.pop(0)
event[0](event[1])
class Foo:
def __init__(self, start_value = 1):
self.value = start_value
def update_value(self, multiple):
self.value *= multiple
def return_bah(self)
return self.value + 3
class Bar:
def __init__(self, number1, number2):
self.init = number1
self.add = number2
def print_alt_value(self, in_value):
print in_value * (self.init + self.add)
That is a barebones of what I have, but it illustrates my problem:
Doing the below
events2 = Event_queue2()
foo1 = Foo(4) ----> foo1.value = 4 here
bar1 = Bar(4, 2)
events2.push(foo1.update_value,1.5)
events2.push(bar1.print_alt_value,foo1.value)
events2.push(bar.print_alt_value,foo1.return_bah())
events2.process_next() ----> should process update_value to change foo.value to 6
events2.process_next() ----> should process print_alt_value in bar class - expected 36
events2.process_next() ----> should process print_alt_value - expected 54
I initially expected my output to be 36 6 * (4 + 2)
I know why its not, foo1.value and foo1.return_bah() gets passed as an evaluated parameter (correct term?).
What I really want is to pass the reference to the variable or the reference to the method, rather than having it evaluate when I put it in my event queue.
Can anyone help me.
I tried searching, but I couldn't piece together what I wanted exactly.
TO get what I have now I initially looked at these threads:
Calling a function of a module from a string with the function's name in Python
Use a string to call function in Python
But I don't see how to support parameters from that properly or how to support passing another function or reference to a variable from those.
I suppose at least for the method call, I could perhaps pass the parameter as foo1.return.bah and evaluate in the process_next method, but I was hoping for a general way that would accept both standard variables and method calls, as the event_queue will take both.
Thank you for the help
Update edit:
So I following the suggestion below, and got really close, but:
Ok, so I followed your queue suggestion and got really close to what I want, but I don't completely understand the first part about multiple functions.
I want to be able to call a dictionary of objects with this as well.
for example:
names = ["test1", "test2"]
for name in names:
names_objs[name] = Foo(4)
Then when attempting to push via lambda
for name in names_list:
events2.push(lambda: names_objs[name].update_value(2))
doesn't work. When teh event actually gets processed it only runs on whatever name_objs[name] references, and if the name variable is no longer valid or has been modified outside the function, it is wrong.
This actually wasn't surprising, but adding a:
name_obj_hold = name_objs[name]
then pushing that didn't either. it again only operates on whatever name_obj_hold last referenced.
Can someone clarify the multiple funcs thing. I'm afraid I'm having trouble wrapping my head around it.
basically I need the initial method call evaluated, so something like:
names_objs[name].some_func(#something in here#)
gets the proper method and associated with the right class object instance, but the #something in here# doesn't get evaluated (whether it is a variable or another function) until it actually gets called from the event queue.
Instead of passing in the function to call func1 and the arguments that should be passed to the function, pass in a function func2 that calls func1 with the arguments that should be passed in.
d = {"a":1}
def p(val):
print val
def func1():
p(d["a"])
def call_it(func):
func()
call_it(func1)
d["a"] = 111
call_it(func1)
Within func1, d["a"] is not evaluated until func1 actually executes.
For your purposes, your queue would change to:
class EventQueue(object):
def __init__(self):
self.events = deque()
def push(self, callable):
self.events.append(callable)
def process_next(self):
self.events.popleft()()
collections.deque will be faster at popping from the front of the queue than a list.
And to use the EventQueue, you can use lambdas for quick anonymous function.
events2 = EventQueue()
foo1 = Foo(4)
bar1 = Bar(4, 2)
events2.push(lambda: foo1.update_value(1.5))
events2.push(lambda: bar1.print_alt_value(foo1.value))
events2.push(lambda: bar1.print_alt_value(foo1.return_bah()))
events2.process_next()
events2.process_next() # 36.0
events2.process_next() # 54.0
For Edit:
In this case you need to "capture" the value in a variable that is more tightly scoped than the loop. You can use a normal function and partial() to achieve this.
for name in names_list:
def update(name):
names_objs[name].update_value(2)
events2.push(partial(update, name))
I have an unknown number of functions in my python script (well, it is known, but not constant) that start with site_...
I was wondering if there's a way to go through all of these functions in some main function that calls for them.
something like:
foreach function_that_has_site_ as coolfunc
if coolfunc(blabla,yada) == true:
return coolfunc(blabla,yada)
so it would go through them all until it gets something that's true.
thanks!
The inspect module, already mentioned in other answers, is especially handy because you get to easily filter the names and values of objects you care about. inspect.getmembers takes two arguments: the object whose members you're exploring, and a predicate (a function returning bool) which will accept (return True for) only the objects you care about.
To get "the object that is this module" you need the following well-known idiom:
import sys
this_module = sys.modules[__name__]
In your predicate, you want to select only objects which are functions and have names that start with site_:
import inspect
def function_that_has_site(f):
return inspect.isfunction(f) and f.__name__.startswith('site_')
With these two items in hand, your loop becomes:
for n, coolfunc in inspect.getmembers(this_module, function_that_has_site):
result = coolfunc(blabla, yada)
if result: return result
I have also split the loop body so that each function is called only once (which both saves time and is a safer approach, avoiding possible side effects)... as well as rewording it in Python;-)
Have you tried using the inspect module?
http://docs.python.org/library/inspect.html
The following will return the methods:
inspect.getmembers
Then you could invoke with:
methodobjToInvoke = getattr(classObj, methodName)
methodobj("arguments")
This method goes through all properties of the current module and executes all functions it finds with a name starting with site_:
import sys
import types
for elm in dir():
f = getattr(sys.modules[__name__], elm)
if isinstance(f, types.FunctionType) and f.__name__[:5] == "site_":
f()
The function-type check is unnecessary if only functions are have names starting with site_.
def run():
for f_name, f in globals().iteritems():
if not f_name.startswith('site_'):
continue
x = f()
if x:
return x
It's best to use a decorator to enumerate the functions you care about:
_funcs = []
def enumfunc(func):
_funcs.append(func)
return func
#enumfunc
def a():
print 'foo'
#enumfunc
def b():
print 'bar'
#enumfunc
def c():
print 'baz'
if __name__ == '__main__':
for f in _funcs:
f()
Try dir(), globals() or locals(). Or inspect module (as mentioned above).
def site_foo():
pass
def site_bar():
pass
for name, f in globals().items():
if name.startswith("site_"):
print name, f()