I want to perform the following processing of a String variable received as a parameter of my API endpoint (FastAPI) as shown below, and I have 30 functions that need to be applied to the String variable.
I'm kind of lost and don't really know how can I do this, any hints, or concepts I should learn about that can help solve the problem.
Code:
def func1(data):
return True
def func2(data):
return True
def func3(data):
return True
...
def func30(data):
return True
#app.get("/process/{data}")
def process(data):
# Apply all the methods on the data variable
return response ```
----------
thank you
You can use map with a list of functions and turn it into a list, as map is lazily evaluated. You may ignore the list if not needed.
def apply(data, functions):
return list(map(lambda f: f(data), functions))
from functools import reduce
from typing import Callable, Iterable, Any
def apply(funcs, data):
return reduce(lambda tmp, func: func(tmp), funcs, data)
funcs = [
lambda x: x + 'a',
lambda x: x + 'b',
lambda x: x + 'c',
lambda x: x + 'd',
]
print(apply(funcs, '')) # Result is 'abcd'
>>> a='test1'
>>> check_for_test1=lambda x:a in x
>>> check_for_test1('test1')
True
>>> a='test2'
>>> check_for_test2=lambda x:a in x
>>> check_for_test2('test1')
False
>>> check_for_test2('test2')
True
>>> check_for_test1('test1')
False
Is there any way I can keep check_for_test1 to be on the "original" test variable , i.e. test1?
Sure, use a closure:
>>> a='test1'
>>> check_for_test1 = (lambda needle: lambda x: needle in x)(a)
>>> check_for_test1('test1')
True
>>> a = 'foo'
>>> check_for_test1('test1')
True
Note, you should avoid assigning the result of a lambda expression to a name. The only purpose of lambda is so a function is anonymous. This is actually explicitly against PEP8 style guidelines.
This is generally called a factory function.
I would say, a more pythonic way would be:
def make_checker(needle):
def checker(x):
return needle in x
return checker
Which is much clearer to me.
You can capture the original value in the keyword argument with default value:
a='test1'
check_for_test1=lambda x, a=a:a in x
print( check_for_test1('test1') )
a='test2'
check_for_test2=lambda x, a=a:a in x
print( check_for_test2('test1') )
print( check_for_test2('test2') )
print( check_for_test1('test1') )
Prints:
True
False
True
True
Note: everything bad about named lambdas in the comments apply. Don't use them it you don't have to.
If you don't want the functions to change when you update the value of a then use the string itself rather than a variable.
check_for_test1=lambda x: 'test1' in x
check_for_test2=lambda x: 'test2' in x
you can use functions with constants, cleaner code:
TEST1 = 'test1'
TEST2 = 'test2'
def check_for_test1(my_str):
return TEST1 in my_str
def check_for_test2(my_str):
return TEST2 in my_str
if you want to create dynamically the check methods:
def check_method_creator(my_str):
def check_method(other_str):
return my_str in other_str
return check_method
check_for_test1 = check_method_creator('test1')
check_for_test2 = check_method_creator('test2')
I am implementing this requirement:
As part of a data processing pipeline, complete the implementation of the pipeline method:
The method should accept a variable number of functions, and it
should return a new function that accepts one parameter arg.
The returned function should call the first function in the pipeline
with the parameter arg, and call the second function with the result
of the first function.
The returned function should continue calling each function in the
pipeline in order, following the same pattern, and return the value
from the last function.
For example, pipeline(lambda x: x * 3, lambda x: x + 1, lambda x: x / 2) then calling the returned function with 3 should return 5.0.
My code
def pipeline(*funcs):
def helper(arg):
argCount = len(funcs)
if argCount > 0:
# Iterate over all the arguments and call each lamba's function
res = []
for elem in funcs:
if(len(res) > 0):
helper = elem(res.pop())
else:
helper = elem(arg)
res.append(helper)
helper = res.pop()
else:
return helper
print('before returning, helper value is: ', helper)
return helper
fun = pipeline(lambda x: x * 3, lambda x: x + 1, lambda x: x / 2)
print('final result: ', fun(3)) #should print 5.0
Question
None is returned. Why?
before returning, helper value is: 5.0
final result: None
The problem is that you don't execute a return right after you print. You do have a return in the else branch just before it, but not in the if block. Also, the return helper you have further below does not belong to the def helper function block, so you need one more return helper. I would in fact omit the else block, and just always do the return, like this:
def pipeline(*funcs):
def helper(arg):
argCount = len(funcs)
if argCount > 0:
# Iterate over all the arguments and call each lamba's function
res = []
for elem in funcs:
if(len(res) > 0):
helper = elem(res.pop())
else:
helper = elem(arg)
res.append(helper)
helper = res.pop()
print('before returning, helper value is: ', helper)
return helper # <-------
return helper
It is not really clear why you have a list res, since there is only one value to pass from one function to the next. You could just use arg for this purpose. Furthermore, you use helper in two different senses (function & value) which is quite confusing. The code can be simplified to this:
def pipeline(*funcs):
def helper(arg):
for elem in funcs:
arg = elem(arg)
return arg
return helper
Do not invent what is already available in Python
from functools import reduce
pipeline = [lambda x: x * 3, lambda x: x + 1, lambda x: x / 2]
val = reduce(lambda x, f: f(x), pipeline, 3)
print(val) # 5.0
You do: print('before returning, helper value is: ', helper)... and then do not actually return anything from helper, so it implicitly returns None.
def pipeline(*args):
def helper(num):
for i in args:
total=i(num)
num=total
return total
return helper
fun = pipeline(lambda x: x * 3, lambda x: x + 1, lambda x: x / 2)
print(fun(3)) #should print 5.0
I have multiple expensive functions that return results. I want to return a tuple of the results of all the checks if all the checks succeed. However, if one check fails I don't want to call the later checks, like the short-circuiting behavior of and. I could nest if statements, but that will get out of hand if there are a lot of checks. How can I get the short-circuit behavior of and while also storing the results for later use?
def check_a():
# do something and return the result,
# for simplicity, just make it "A"
return "A"
def check_b():
# do something and return the result,
# for simplicity, just make it "B"
return "B"
...
This doesn't short-circuit:
a = check_a()
b = check_b()
c = check_c()
if a and b and c:
return a, b, c
This is messy if there are many checks:
if a:
b = check_b()
if b:
c = check_c()
if c:
return a, b, c
Is there a shorter way to do this?
Just use a plain old for loop:
results = {}
for function in [check_a, check_b, ...]:
results[function.__name__] = result = function()
if not result:
break
The results will be a mapping of the function name to their return values, and you can do what you want with the values after the loop breaks.
Use an else clause on the for loop if you want special handling for the case where all of the functions have returned truthy results.
Write a function that takes an iterable of functions to run. Call each one and append the result to a list, or return None if the result is False. Either the function will stop calling further checks after one fails, or it will return the results of all the checks.
def all_or_none(checks, *args, **kwargs):
out = []
for check in checks:
rv = check(*args, **kwargs)
if not rv:
return None
out.append(rv)
return out
rv = all_or_none((check_a, check_b, check_c))
# rv is a list if all checks passed, otherwise None
if rv is not None:
return rv
def check_a(obj):
...
def check_b(obj):
...
# pass arguments to each check, useful for writing reusable checks
rv = all_or_none((check_a, check_b), obj=my_object)
In other languages that did have assignments as expressions you would be able to use
if (a = check_a()) and (b = check_b()) and (c = check_c()):
but Python is no such language. Still, we can circumvent the restriction and emulate that behaviour:
result = []
def put(value):
result.append(value)
return value
if put(check_a()) and put(check_b()) and put(check_c()):
# if you need them as variables, you could do
# (a, b, c) = result
# but you just want
return tuple(result)
This might loosen the connection between the variables and function calls a bit too much, so if you want to do lots of separate things with the variables, instead of using the result elements in the order they were put in the list, I would rather avoid this approach. Still, it might be quicker and shorter than some loop.
You could use either a list or an OrderedDict, using a for loop would serve the purpose of emulating short circuiting.
from collections import OrderedDict
def check_a():
return "A"
def check_b():
return "B"
def check_c():
return "C"
def check_d():
return False
def method1(*args):
results = []
for i, f in enumerate(args):
value = f()
results.append(value)
if not value:
return None
return results
def method2(*args):
results = OrderedDict()
for f in args:
results[f.__name__] = result = f()
if not result:
return None
return results
# Case 1, it should return check_a, check_b, check_c
for m in [method1, method2]:
print(m(check_a, check_b, check_c))
# Case 1, it should return None
for m in [method1, method2]:
print(m(check_a, check_b, check_d, check_c))
There are lots of ways to do this! Here's another.
You can use a generator expression to defer the execution of the functions. Then you can use itertools.takewhile to implement the short-circuiting logic by consuming items from the generator until one of them is false.
from itertools import takewhile
functions = (check_a, check_b, check_c)
generator = (f() for f in functions)
results = tuple(takewhile(bool, generator))
if len(results) == len(functions):
return results
Another way to tackle this is using a generator, since generators use lazy evaluation. First put all checks into a generator:
def checks():
yield check_a()
yield check_b()
yield check_c()
Now you could force evaluation of everything by converting it to a list:
list(checks())
But the standard all function does proper short cut evaluation on the iterator returned from checks(), and returns whether all elements are truthy:
all(checks())
Last, if you want the results of succeeding checks up to the failure you can use itertools.takewhile to take the first run of truthy values only. Since the result of takewhile is lazy itself you'll need to convert it to a list to see the result in a REPL:
from itertools import takewhile
takewhile(lambda x: x, checks())
list(takewhile(lambda x: x, checks()))
main logic:
results = list(takewhile(lambda x: x, map(lambda x: x(), function_list)))
if len(results) == len(function_list):
return results
you can learn a lot about collection transformations if you look at all methods of an api like http://www.scala-lang.org/api/2.11.7/#scala.collection.immutable.List and search/implement python equivalents
logic with setup and alternatives:
import sys
if sys.version_info.major == 2:
from collections import imap
map = imap
def test(bool):
def inner():
print(bool)
return bool
return inner
def function_for_return():
function_list = [test(True),test(True),test(False),test(True)]
from itertools import takewhile
print("results:")
results = list(takewhile(lambda x:x,map(lambda x:x(),function_list)))
if len(results) == len(function_list):
return results
print(results)
#personally i prefer another syntax:
class Iterator(object):
def __init__(self,iterable):
self.iterator = iter(iterable)
def __next__(self):
return next(self.iterator)
def __iter__(self):
return self
def map(self,f):
return Iterator(map(f,self.iterator))
def takewhile(self,f):
return Iterator(takewhile(f,self.iterator))
print("results2:")
results2 = list(
Iterator(function_list)
.map(lambda x:x())
.takewhile(lambda x:x)
)
print(results2)
print("with additional information")
function_list2 = [(test(True),"a"),(test(True),"b"),(test(False),"c"),(test(True),"d")]
results3 = list(
Iterator(function_list2)
.map(lambda x:(x[0](),x[1]))
.takewhile(lambda x:x[0])
)
print(results3)
function_for_return()
If you don't need to take an arbitrary number of expressions at runtime (possibly wrapped in lambdas), you can expand your code directly into this pattern:
def f ():
try:
return (<a> or jump(),
<b> or jump(),
<c> or jump())
except NonLocalExit:
return None
Where those definitions apply:
class NonLocalExit(Exception):
pass
def jump():
raise NonLocalExit()
Flexible short circuiting is really best done with Exceptions. For a very simple prototype you could even just assert each check result:
try:
a = check_a()
assert a
b = check_b()
assert b
c = check_c()
assert c
return a, b, c
except AssertionException as e:
return None
You should probably raise a custom Exception instead. You could change your check_X functions to raise Exceptions themself, in an arbitrary nested way. Or you could wrap or decorate your check_X functions to raise errors on falsy return values.
In short, exception handling is very flexible and exactly what you are looking for, don't be afraid to use it. If you learned somewhere that exception handling is not to be used for your own flow control, this does not apply to python. Liberal use of exception handling is considered pythonic, as in EAFP.
You mentioned 'short-circuiting' in your answer, which can be done with the 'or' statement. Top answer basically does the same thing, but in case someone wants to know more about this behaviour you could do this;
class Container(object):
def __init__(self):
self.values = []
def check_and_cache(self, value, checking_function):
value_true = checking_function(value)
if value_true:
self.values.append(value)
return True
c = Container()
if not c.check_and_cache(a, check_a) or not c.check_and_cache(b, check_b) or not c.check_and_cache(c, check_c):
print 'done'
return tuple(c.values)
The 'not .. or' setup of the if statements will result in a 'True' if the check fails, so the overall if statement passes without evaluating the remaining values.
Since I can not comment "wim":s answer as guest, I'll just add an extra answer.
Since you want a tuple, you should collect the results in a list and then cast to tuple.
def short_eval(*checks):
result = []
for check in checks:
checked = check()
if not checked:
break
result.append(checked)
return tuple(result)
# Example
wished = short_eval(check_a, check_b, check_c)
You can try use #lazy_function decorator from lazy_python
package. Example of usage:
from lazy import lazy_function, strict
#lazy_function
def check(a, b):
strict(print('Call: {} {}'.format(a, b)))
if a + b > a * b:
return '{}, {}'.format(a, b)
a = check(-1, -2)
b = check(1, 2)
c = check(-1, 2)
print('First condition')
if c and a and b: print('Ok: {}'.format((a, b)))
print('Second condition')
if c and b: print('Ok: {}'.format((c, b)))
# Output:
# First condition
# Call: -1 2
# Call: -1 -2
# Second condition
# Call: 1 2
# Ok: ('-1, 2', '1, 2')
This is similar to Bergi's answer but I think that answer misses the point of wanting separate functions (check_a, check_b, check_c):
list1 = []
def check_a():
condition = True
a = 1
if (condition):
list1.append(a)
print ("checking a")
return True
else:
return False
def check_b():
condition = False
b = 2
if (condition):
list1.append(b)
print ("checking b")
return True
else:
return False
def check_c():
condition = True
c = 3
if (condition):
list1.append(c)
print ("checking c")
return True
else:
return False
if check_a() and check_b() and check_c():
# won't get here
tuple1 = tuple(list1)
print (tuple1)
# output is:
# checking a
# (1,)
Or, if you don't want to use the global list, pass a reference of a local list to each of the functions.
If the main objection is
This is messy if there are many checks:
if a:
b = check_b()
if b:
c = check_c()
if c:
return a, b, c
A fairly nice pattern is to reverse the condition and return early
if not a:
return # None, or some value, or however you want to handle this
b = check_b()
if not b:
return
c = check_c()
if not c:
return
# ok, they were all truthy
return a, b, c
I want to write a python function like this:
def foo():
if x:
return y
else:
return y, z
Is it possible? If it is possible then how can I detect the number of return values?
It may be easier to modify the function to always return a 2-tuple:
def foo():
if x:
return y, None
else:
return y, z
or what I'd do absent compelling reasons why not:
def foo( z_default = None):
# plug in whatever default makes sense in place of None
# and the caller can override the default if he wants to
if x:
return y, z_default
else:
return y, z
Yes, it is possible. But when your if condition is false your function will return a tuple with y at index 0 and z at index 1. You can catch the return values in a tuple variable and check the number of returned values using len() function for the tuple.
Yes, it is highly possible:
def foo():
if x:
return y
else:
return y, z
result = foo()
the_number_of_return_values_of_foo_function = \
1 if type(result) is not tuple else len(result)
Good nights.