I'm sorry if this is a question answered elsewhere. Searching through Google and Stackforum I didn't find anything from which I could extrapolate the answers; but I feel like part of that is me.
I'm trying to work out lambdas as a concept, and as part of that I'm kinda looking for ways to use it.
SO, if this is a colossally stupid thing to do with lambda from a function standpoint, feel free to let me know and explain. But either way, I still want to know the answer/still want to know how to do this with the python language.
So, for testing purposes I have:
my_test = 'test_name'
testlist = ['test_name', 'test_name_dup', 'test_name_dup_1', 'test_name_dup_3']
I'm looking to use lambda to create one function that loops through and returns the first test_name_# that isn't in the testlist. The functionality will eventually be applied to filenames, but for testing purposes I had to get away from actually reading the filenames--gave me too many more ways to mess something up.
But my_test has to be able to change, and the test list will be a list of filepaths.
So, I'm looking for a function like:
new_name = lambda x: my_test + '_' + str(x)
But the initial value should be x = 1, and it should continue until new_name is not in testlist. Seems like:
bool(new_name not in testlist)
might be something work with.
But I can't figure out a way to set the initial x to 1, and have it loop through with (x+1) until the bool is true.
I know this is possible as I've found some CRAZY lambda examples out there that are looping through lines in a file. I just couldn't quite make sense of them (and didn't have any way to play with them as they were dealing with things outside my programming level.
On a related note, could I add values to the beginning of this loop? (i.e. can I have it check for test_name, then test_name_dup, then test_name_dup_#)?
Thanks in advance for the help! Lambdas (while very cool) totally mess with my head.
Lambdas are just another way of defining a function
def foo(x):
return x + x
is the same as
foo = lambda x: x + x
So let's start with a function to do what you want:
def first_missing(items, base):
for number in itertools.count():
text = base + '_' + str(number)
if text not in items:
return text
The first thing to note is that you can't use loops inside a lambda. So we'll need to rewrite this without a loop. Instead, we'll use recursion:
def first_missing(items, base, number = 0):
text = base + '_' + str(number)
if text not in items:
return text
else:
return first_missing(items, base, number + 1)
Now, we also can't use an if/else block in a lambda. But we can use a ternary expression:
def first_missing(items, base, number = 0):
text = base + '_' + str(number)
return text if text not in items else first_missing(items, base, number + 1)
We can't have local variables in a lambda, so we'll use a trick, default arguments:
def first_missing(items, base, number = 0):
def inner(text = base + '_' + str(number)):
return text if text not in items else first_missing(items, base, number + 1)
return inner()
At this point we can rewrite inner as a lambda:
def first_missing(items, base, number = 0):
inner = lambda text = base + '_' + str(number): text if text not in items else first_missing(items, base, number + 1)
return inner()
We can combine two lines to get rid of the inner local variable:
def first_missing(items, base, number = 0):
return (lambda text = base + '_' + str(number): text if text not in items else first_missing(items, base, number + 1))()
And at long last, we can make the whole thing into a lambda:
first_missing = lambda: items, base, number = 0: (lambda text = base + '_' + str(number): text if text not in items else first_missing(items, base, number + 1))()
Hopefully that gives you some insight into what you can do. But don't ever do it because, as you can tell, lambdas can make your code really hard to read.
There's no need to use a lambda in this case, a simple for loop will do:
my_test = 'test_name_dup'
testlist = ['test_name', 'test_name_dup','test_name_dup_1', 'test_name_dup_3']
for i in xrange(1, len(testlist)):
if my_test + '_' + str(i) not in testlist:
break
print my_test + '_' + str(i)
> test_name_dup_2
If you really, really want to use a lambda for this problem, you'll also have to learn about itertools, iterators, filters, etc. I'm gonna build on thg435's answer, writing it in a more idiomatic fashion and explaining it:
import itertools as it
iterator = it.dropwhile(
lambda n: '{0}_{1}'.format(my_test, n) in testlist,
it.count(1))
print my_test + '_' + str(iterator.next())
> test_name_dup_2
The key to understanding the above solution lies in the dropwhile() procedure. It takes two parameters: a predicate and an iterable, and returns an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element.
For the iterable, I'm passing count(1), an iterator that produces an infinite number of integers starting from 1.
Then dropwhile() starts to consume the integers until the predicate is false; this is a good opportunity for passing an in-line defined function - and here's our lambda. It receives each generated integer in turn, checking to see if the string test_name_dup_# is present in the list.
When the predicate returns false, dropwhile() returns and we can retrieve the value that made it stop by calling next() on it.
You can combine a lambda with itertools.dropwhile:
import itertools
n = itertools.dropwhile(lambda n: 'test_name_dup_%d' % n in testlist, range(1, len(testlist))).next()
As to your last question, you can write a generator for names, like:
def possible_names(prefix):
yield prefix
yield prefix + '_dup'
n = 0
while True:
n += 1
yield '%s_dup_%d' % (prefix, n)
and then use this generator with dropwhile:
unique_name = itertools.dropwhile(lambda x: x in testlist, possible_names('test_name')).next()
print unique_name
You are a bit off the track. Lambdas are nothing but "simple" functions, often used for their fast syntax in functional programming. They are the perfect companion built-in functions "map", "reduce", "filter" but also for more complicated functions defined into itertools. Therefore the most useful thing to do with them is to generate/manipulate iterable objects (especially lists). Note that lambdas will slow down your code in most of the cases if compared to list comprehensions/normal loops and will make it harder to be read. Here is an example of what you want to do with lambdas.
>>> filter(lambda i: i!=(0 if len(testlist[i].split("_"))==3 else int(testlist[i].split("_")[-1])), range(len(testlist)))[0]
2
Or you could use more complicated functions with itertools. Anyhow I strongly suggest you not to use lambdas for this kind of assignments, as the readability is terrible. I'd rather use a well structured for loop, which is also faster.
[Edit]
To prove that lambdas+builtins are not faster than list comprehensions: consider a simple problem, for x in range(1000) create a list of x shifted by 5.
$ python -m timeit 'map(lambda x: x>>5, range(1000))' 1000 loops, best of 3: 225 usec per loop
$ python -m timeit '[x>>5 for x in range(1000)]'10000 loops, best of 3: 99.1 usec per loop
You have a >100% performance increase without lambdas.
I prefer the list comprehension or iterator method. Makes for easy one liners that I feel are pretty easy to read and maintain. Quite frankly, lambdas belong some places, here I believe its less elegant a solution.
my_test = 'test_name'
prefix = 'test_name_dup_'
testlist = ['test_name','test_name_dup','test_name_dup_1','test_name_dup_3']
from itertools import count
print next('%s%d' % (prefix, i) for i in count(1) if '%s%d' % (prefix, i) not in testlist)
This returns the first not-found instance in the sequence, which I think is the cleanest.
Of course, if you prefer a list from a definite range, you can modify it to become a list comprehension:
print ['%s%d' % (prefix, i) for i in xrange(0,5) if '%s%d' % (prefix, i) not in testlist]
returns:
['test_name_dup_0', 'test_name_dup_2', 'test_name_dup_4']
Related
Let's say I have a string
S = "qwertyu"
And I want to build a list using recursion so the list looks like
L = [u, y, t, r, e, w, q]
I tried to write code like this:
def rec (S):
if len(S) > 0:
return [S[-1]].append(rec(S[0:-1]))
Ideally I want to append the last element of a shrinking string until it reaches 0
but all I got as an output is None
I know I'm not doing it right, and I have absolutely no idea what to return when the length of S reaches 0, please show me how I can make this work
(sorry the answer has to use recursion, otherwise it won't bother me)
Thank you very much!!!
There are many simpler ways than using recursion, but here's one recursive way to do it:
def rec (S):
if not S:
return []
else:
temp = list(S[-1])
temp.extend(rec(S[:-1]))
return temp
EDIT:
Notice that the base case ensures that function also works with an empty string. I had to use temp, because you cannot return list(S[-1]).extend(rec(S[:-1])) due to it being a NoneType (it's a method call rather than an object). For the same reason you cannot assign to a variable (hence the two separate lines with temp). A workaround would be to use + to concatenate the two lists, like suggested in Aryerez's answer (however, I'd suggest against his advice to try to impress people with convoluted one liners):
def rec (S):
if not S:
return []
else:
return list(S[-1]) + rec(S[:-1])
In fact using + could be more efficient (although the improvement would most likely be negligible), see answers to this SO question for more details.
This is the simplest solution:
def rec(S):
if len(S) == 1:
return S
return S[-1] + rec(S[:-1])
Or in one-line, if you really want to impress someone :)
def rec(S):
return S if len(S) == 1 else S[-1] + rec(S[:-1])
Since append mutates the list, this is a bit difficult to express recursively. One way you could do this is by using a separate inner function that passes on the current L to the next recursive call.
def rec(S):
def go(S, L):
if len(S) > 0:
L.append(S[-1])
return go(S[0:-1], L)
else:
return L
return go(S, [])
L = [i for i in S[::-1]]
It should work.
I am trying to make a reverse function which takes an input (text) and outputs the reversed version. So "Polar" would print raloP.
def reverse(text):
list = []
text = str(text)
x = len(text) - 1
list.append("T" * x)
for i in text:
list.insert(x, i)
x -= 1
print "".join(list)
reverse("Something")
As others have mentioned, Python already provides a couple of ways to reverse a string. The simple way is to use extended slicing: s[::-1] creates a reversed version of string s. Another way is to use the reversed function: ''.join(reversed(s)). But I guess it can be instructive to try implementing it for yourself.
There are several problems with your code.
Firstly,
list = []
You shouldn't use list as a variable name because that shadows the built-in list type. It won't hurt here, but it makes the code confusing, and if you did try to use list() later on in the function it would raise an exception with a cryptic error message.
text = str(text)
is redundant. text is already a string. str(text) returns the original string object, so it doesn't hurt anything, but it's still pointless.
x = len(text) - 1
list.append("T" * x)
You have an off-by-one error here. You really want to fill the list with as many items as are in the original string, this is short by one. Also, this code appends the string as a single item to the list, not as x separate items of one char each.
list.insert(x, i)
The .insert method inserts new items into a list, the subsequent items after the insertion point get moved up to make room. We don't want that, we just want to overwrite the current item at the x position, and we can do that by indexing.
When your code doesn't behave the way you expect it to, it's a Good Idea to add print statements at strategic places to make sure that variables have the value that they're supposed to have. That makes it much easier to find where things are going wrong.
Anyway, here's a repaired version of your code.
def reverse(text):
lst = []
x = len(text)
lst.extend("T" * x)
for i in text:
x -= 1
lst[x] = i
print "".join(lst)
reverse("Something")
output
gnihtemoS
Here's an alternative approach, showing how to do it with .insert:
def reverse(text):
lst = []
for i in text:
lst.insert(0, i)
print "".join(lst)
Finally, instead of using a list we could use string concatenation. However, this approach is less efficient, especially with huge strings, but in modern versions of Python it's not as inefficient as it once was, as the str type has been optimised to handle this fairly common operation.
def reverse(text):
s = ''
for i in text:
s = i + s
print s
BTW, you really should be learning Python 3, Python 2 reaches its official End Of Life in 2020.
You can try :
def reverse(text):
return text[::-1]
print(reverse("Something")) # python 3
print reverse("Something") # python 2
Easier way to do so:
def reverse(text):
rev = ""
i = len(text) - 1
while i > -1:
rev += text[i]
i = i - 1
return rev
print(reverse("Something"))
result: gnihtemoS
You could simply do
print "something"[::-1]
i have a function which gets a string like "ABA?" the question-mark is a wildcard which can either be A or B the passed string can have multiple wildcards. It should give back multiple strings as an array with all possible solutions. My Code is far to slow. I am new to python so its a little difficult to find a good solution.
When "ABA?" is passed it should return ['ABAA', 'ABAB'].
from itertools import product
def possibilities(param):
result = []
for i in product([A,B], repeat=param.count('?')):
string = param
for p in [i]:
for val in p:
string = string.replace('?', str(val), 1)
result.append(string)
return result
You should use a generator so you can loop over the output. If you have a lot of wildcards, the complete list would require a lot of memory to store completely (it grows exponentially!)
import itertools
def possibilties(s):
n_wildcard = s.count('?')
for subs in itertools.product(['A','B'], repeat=n_wildcard):
subs = iter(subs)
yield ''.join([x if x != '?' else subs.next() for x in s])
for p in possibilties('A?BA?'):
print p
This gives:
AABAA
AABAB
ABBAA
ABBAB
I love python. However, one thing that bugs me a bit is that I don't know how to format functional activities in a fluid manner like a can in javascript.
example (randomly created on the spot): Can you help me convert this to python in a fluent looking manner?
var even_set = [1,2,3,4,5]
.filter(function(x){return x%2 === 0;})
.map(function(x){
console.log(x); // prints it for fun
return x;
})
.reduce(function(num_set, val) {
num_set[val] = true;
}, {});
I'd like to know if there are fluid options? Maybe a library.
In general, I've been using list comprehensions for most things but it's a real problem if I want to print
e.g., How can I print every even number between 1 - 5 in python 2.x using list comprehension (Python 3 print() as a function but Python 2 it doesn't). It's also a bit annoying that a list is constructed and returned. I'd rather just for loop.
Update Here's yet another library/option : one that I adapted from a gist and is available on pipy as infixpy:
from infixpy import *
a = (Seq(range(1,51))
.map(lambda x: x * 4)
.filter(lambda x: x <= 170)
.filter(lambda x: len(str(x)) == 2)
.filter( lambda x: x % 20 ==0)
.enumerate() Ï
.map(lambda x: 'Result[%d]=%s' %(x[0],x[1]))
.mkstring(' .. '))
print(a)
pip3 install infixpy
Older
I am looking now at an answer that strikes closer to the heart of the question:
fluentpy https://pypi.org/project/fluentpy/ :
Here is the kind of method chaining for collections that a streams programmer (in scala, java, others) will appreciate:
import fluentpy as _
(
_(range(1,50+1))
.map(_.each * 4)
.filter(_.each <= 170)
.filter(lambda each: len(str(each))==2)
.filter(lambda each: each % 20 == 0)
.enumerate()
.map(lambda each: 'Result[%d]=%s' %(each[0],each[1]))
.join(',')
.print()
)
And it works fine:
Result[0]=20,Result[1]=40,Result[2]=60,Result[3]=80
I am just now trying this out. It will be a very good day today if this were working as it is shown above.
Update: Look at this: maybe python can start to be more reasonable as one-line shell scripts:
python3 -m fluentpy "lib.sys.stdin.readlines().map(str.lower).map(print)"
Here is it in action on command line:
$echo -e "Hello World line1\nLine 2\Line 3\nGoodbye"
| python3 -m fluentpy "lib.sys.stdin.readlines().map(str.lower).map(print)"
hello world line1
line 2
line 3
goodbye
There is an extra newline that should be cleaned up - but the gist of it is useful (to me anyways).
Generators, iterators, and itertools give added powers to chaining and filtering actions. But rather than remember (or look up) rarely used things, I gravitate toward helper functions and comprehensions.
For example in this case, take care of the logging with a helper function:
def echo(x):
print(x)
return x
Selecting even values is easy with the if clause of a comprehension. And since the final output is a dictionary, use that kind of comprehension:
In [118]: d={echo(x):True for x in s if x%2==0}
2
4
In [119]: d
Out[119]: {2: True, 4: True}
or to add these values to an existing dictionary, use update.
new_set.update({echo(x):True for x in s if x%2==0})
another way to write this is with an intermediate generator:
{y:True for y in (echo(x) for x in s if x%2==0)}
Or combine the echo and filter in one generator
def even(s):
for x in s:
if x%2==0:
print(x)
yield(x)
followed by a dict comp using it:
{y:True for y in even(s)}
Comprehensions are the fluent python way of handling filter/map operations.
Your code would be something like:
def evenize(input_list):
return [x for x in input_list if x % 2 == 0]
Comprehensions don't work well with side effects like console logging, so do that in a separate loop. Chaining function calls isn't really that common an idiom in python. Don't expect that to be your bread and butter here. Python libraries tend to follow the "alter state or return a value, but not both" pattern. Some exceptions exist.
Edit: On the plus side, python provides several flavors of comprehensions, which are awesome:
List comprehension: [x for x in range(3)] == [0, 1, 2]
Set comprehension: {x for x in range(3)} == {0, 1, 2}
Dict comprehension: ` {x: x**2 for x in range(3)} == {0: 0, 1: 1, 2: 4}
Generator comprehension (or generator expression): (x for x in range(3)) == <generator object <genexpr> at 0x10fc7dfa0>
With the generator comprehension, nothing has been evaluated yet, so it is a great way to prevent blowing up memory usage when pipelining operations on large collections.
For instance, if you try to do the following, even with python3 semantics for range:
for number in [x**2 for x in range(10000000000000000)]:
print(number)
you will get a memory error trying to build the initial list. On the other hand, change the list comprehension into a generator comprehension:
for number in (x**2 for x in range(1e20)):
print(number)
and there is no memory issue (it just takes forever to run). What happens is the range object gets built (which only stores the start, stop and step values (0, 1e20, and 1)) the object gets built, and then the for-loop begins iterating over the genexp object. Effectively, the for-loop calls
GENEXP_ITERATOR = `iter(genexp)`
number = next(GENEXP_ITERATOR)
# run the loop one time
number = next(GENEXP_ITERATOR)
# run the loop one time
# etc.
(Note the GENEXP_ITERATOR object is not visible at the code level)
next(GENEXP_ITERATOR) tries to pull the first value out of genexp, which then starts iterating on the range object, pulls out one value, squares it, and yields out the value as the first number. The next time the for-loop calls next(GENEXP_ITERATOR), the generator expression pulls out the second value from the range object, squares it and yields it out for the second pass on the for-loop. The first set of numbers are no longer held in memory.
This means that no matter how many items in the generator comprehension, the memory usage remains constant. You can pass the generator expression to other generator expressions, and create long pipelines that never consume large amounts of memory.
def pipeline(filenames):
basepath = path.path('/usr/share/stories')
fullpaths = (basepath / fn for fn in filenames)
realfiles = (fn for fn in fullpaths if os.path.exists(fn))
openfiles = (open(fn) for fn in realfiles)
def read_and_close(file):
output = file.read(100)
file.close()
return output
prefixes = (read_and_close(file) for file in openfiles)
noncliches = (prefix for prefix in prefixes if not prefix.startswith('It was a dark and stormy night')
return {prefix[:32]: prefix for prefix in prefixes}
At any time, if you need a data structure for something, you can pass the generator comprehension to another comprehension type (as in the last line of this example), at which point, it will force the generators to evaluate all the data they have left, but unless you do that, the memory consumption will be limited to what happens in a single pass over the generators.
The biggest dealbreaker to the code you wrote is that Python doesn't support multiline anonymous functions. The return value of filter or map is a list, so you can continue to chain them if you so desire. However, you'll either have to define the functions ahead of time, or use a lambda.
Arguments against doing this notwithstanding, here is a translation into Python of your JS code.
from __future__ import print_function
from functools import reduce
def print_and_return(x):
print(x)
return x
def isodd(x):
return x % 2 == 0
def add_to_dict(d, x):
d[x] = True
return d
even_set = list(reduce(add_to_dict,
map(print_and_return,
filter(isodd, [1, 2, 3, 4, 5])), {}))
It should work on both Python 2 and Python 3.
There's a library that already does exactly what you are looking for, i.e. the fluid syntaxt, lazy evaluation and the order of operations is the same with how it's written, as well as many more other good stuff like multiprocess or multithreading Map/Reduce.
It's named pyxtension and it's prod ready and maintained on PyPi.
Your code would be rewritten in this form:
from pyxtension.strams import stream
def console_log(x):
print(x)
return x
even_set = stream([1,2,3,4,5])\
.filter(lambda x:x%2 === 0)\
.map(console_log)\
.reduce(lambda num_set, val: num_set.__setitem__(val,True))
Replace map with mpmap for multiprocessed map, or fastmap for multithreaded map.
We can use Pyterator for this (disclaimer: I am the author).
We define the function that prints and returns (which I believe you can omit completely however).
def print_and_return(x):
print(x)
return x
then
from pyterator import iterate
even_dict = (
iterate([1,2,3,4,5])
.filter(lambda x: x%2==0)
.map(print_and_return)
.map(lambda x: (x, True))
.to_dict()
)
# {2: True, 4: True}
where I have converted your reduce into a sequence of tuples that can be converted into a dictionary.
Working code
def not_double_cap_word(word):
cap_count = 0
for ch in word:
if str.isupper(ch):
cap_count += 1
not_double_cap = (cap_count < 2)
return not_double_cap
...
words_no_double_caps =list(filter(not_double_cap_word,words_alnum))
What would be a different solution, say maybe using lambdas or other patterns in Python? The above creates a new list of words with any word with more than two caps removed. one two TWo => one, two.
You can definitely simplify that not_double_cap_word function, but it's still going to be the same basic solution.
First, you can use ch.isupper() instead of str.isupper(ch). Calling a method the normal way is always easier than calling it as an unbound method and passing the self explicitly.
Next, we can replace the explicit for loop with sum over a generator expression:
cap_count = sum(ch.isupper() for ch in word)
And we don't really need to define not_double_cap, cap_count < 2 seems simple enough to return directly. So:
def not_double_cap_word(word):
cap_count = sum(ch.isupper() for ch in word)
return cap_count < 2
But really, this whole thing is probably simple enough to inline directly into the main expression. While you could do that by defining a function with lambda, there's no reason to. In general, map and filter are good when what you want to do to each thing is call a function that you already have lying around; comprehensions are better when what you want to do is an expression that you'd have to wrap in a function (lambda or otherwise) to pass to map or filter. Compare:
words_no_double_caps = [word for word in words_alnum
if sum(ch.isupper() for ch in word) < 2]
words_no_double_caps = list(filter((lambda word: sum(map(
lambda ch: ch.upper(), word)) < 2), words_alnum))
(I think I got the parens on the second version right. If not… well, if I wanted to program in Lisp, I would.:)
Either way, it's performing pretty much the exact same steps as your original code, but it's more concise. Is it more readable? That's for you to decide. But that's the most important reason to choose one or the other, or something intermediate between the two.
Well, that, and whether or not you need to reuse this logic; if you do, it should definitely be defined with a def statement and given a nice name.
You can rewrite your not_double_cap_word code using sum:
def not_double_cap_word(word):
return sum(x.isupper() for x in word) < 2
If you just want ti use a lambda with filter and not use the not_double_cap_word function:
print(list(filter(lambda x: sum(s.isupper() for s in x) < 2 ,["one", "two" ,"TWo"])))
['one', 'two']