Develop Reference

Why will the loop in my allSubStrings function not work?

I am trying to write a Python program that has two functions. The first finds all substrings of a word with a given length and adds them to a list (i.e. "hello" with x = 3 would return 'hel', 'ell', 'llo'). The second function uses this to find all possible substrings in a word. However, whenever I run it through a loop, the second function does not work. Can someone try to explain why?
def subString(string, x):
cutList = []
j = 0
i = 0
while(j < 1):
sliceText = slice(i, i + x)
cut = string[sliceText]
if (len(cut) == x):
cutList.append(cut)
else:
j += 5
i = i + 1
return cutList
def allSubStrings(string):
fullList = []
for k in range(len(string)):
tempList = subString(string, k)
fullList.extend(tempList)
print(k)
return fullList

The problem is that subString() goes into an infinite loop when x == 0, because len(cut) == 0 will always be true. Since you don't need zero-length substrings, the loop in allSubStrings() should use range(1, len(string)).
def subString(string, x):
return [string[i:i+x] for i in range(len(string)-x)]
def allSubStrings(string):
fullList = []
for k in range(1, len(string)):
tempList = subString(string, k)
fullList.extend(tempList)
print(k)
return fullList
print(allSubStrings("abcdefgh"))
Output is:
1
2
3
4
5
6
7
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'ab', 'bc', 'cd', 'de', 'ef', 'fg', 'abc', 'bcd', 'cde', 'def', 'efg', 'abcd', 'bcde', 'cdef', 'defg', 'abcde', 'bcdef', 'cdefg', 'abcdef', 'bcdefg', 'abcdefg']

Generating n-grams from a string

I need to make a list of all 𝑛 -grams beginning at the head of string for each integer 𝑛 from 1 to M. Then return a tuple of M such lists.
def letter_n_gram_tuple(s, M):
s = list(s)
output = []
for i in range(0, M+1):
output.append(s[i:])
return(tuple(output))
From letter_n_gram_tuple("abcd", 3) output should be:
(['a', 'b', 'c', 'd'], ['ab', 'bc', 'cd'], ['abc', 'bcd']))
However, my output is:
(['a', 'b', 'c', 'd'], ['b', 'c', 'd'], ['c', 'd'], ['d']).
Should I use string slicing and then saving slices into the list?

you can use nested for, first for about n-gram, second to slice the string
def letter_n_gram_tuple(s, M):
output = []
for i in range(1, M + 1):
gram = []
for j in range(0, len(s)-i+1):
gram.append(s[j:j+i])
output.append(gram)
return tuple(output)
or just one line by list comprehension:
output = [[s[j:j+i] for j in range(0, len(s)-i+1)] for i in range(1, M + 1)]
or use windowed in more_itertools:
import more_itertools
output = [list(more_itertools.windowed(s, i)) for i in range(1, M + 1)]
test and output:
print(letter_n_gram_tuple("abcd", 3))
(['a', 'b', 'c', 'd'], ['ab', 'bc', 'cd'], ['abc', 'bcd'])

You need one more for loop to iterate over letters or str :
def letter_n_gram_tuple(s, M):
output = []
for i in range(0, M):
vals = [s[j:j+i+1] for j in range(len(s)) if len(s[j:j+i+1]) == i+1]
output.append(vals)
return tuple(output)
print(letter_n_gram_tuple("abcd", 3))
Output:
(['a', 'b', 'c', 'd'], ['ab', 'bc', 'cd'], ['abc', 'bcd'])

Use the below fuction:
def letter_n_gram_tuple(s, M):
s = list(s)
output = [s]
for i in range(M + 1):
output.append([''.join(sorted(set(a + b), key=lambda x: (a + b).index(x))) for a, b in zip(output[-1], output[-1][1:])])
return tuple(filter(lambda x: len(x) > 1, output))
And now:
print(letter_n_gram_tuple('abcd',3))
Returns:
(['a', 'b', 'c', 'd'], ['ab', 'bc', 'cd'], ['abc', 'bcd'])

def n_grams(word,max_size):
i=1
output=[]
while i<= max_size:
index = 0
innerArray=[]
while index < len(word)-i+1:
innerArray.append(word[index:index+i])
index+=1
i+=1
output.append(innerArray)
innerArray=[]
return tuple(output)
print(n_grams("abcd",3))

Find all substrings in a string in python 3 with brute-force

I want to find all substrings 'A' to 'B' in L = ['C', 'A', 'B', 'A', 'A', 'X', 'B', 'Y', 'A'] with bruteforce, this is what i've done:
def find_substring(L):
t = 0
s = []
for i in range(len(L) - 1):
l = []
if ord(L[i]) == 65:
for j in range(i, len(L)):
l.append(L[j])
if ord(L[j]) == 66:
t = t + 1
s.append(l)
return s, t
Now I want the output:
[['A','B'], ['A','B','A','A','X','B'], ['A','A','X','B'], ['A','X','B']]
But i get:
[['A','B','A','A','X','B','Y','A'],['A','B','A','A','X','B','Y','A'],['A','A','X','B','Y','A'],['A','X','B','Y','A']]
Can someone tell me what I'm doing wrong?

The problem is that the list s, holds references to the l lists.
So even though you are appending the correct l lists to s, they are changed after being appended as the future iterations of the j loop modify the l lists.
You can fix this by appending a copy of the l list: l[:].
Also, you can compare strings directly, no need to convert to ASCII.
def find_substring(L):
s = []
for i in range(len(L) - 1):
l = []
if L[i] == 'A':
for j in range(i, len(L)):
l.append(L[j])
if L[j] == 'B':
s.append(l[:])
return s
which now works:
>>> find_substring(['C', 'A', 'B', 'A', 'A', 'X', 'B', 'Y', 'A'])
[['A', 'B'], ['A', 'B', 'A', 'A', 'X', 'B'], ['A', 'A', 'X', 'B'], ['A', 'X', 'B']]

When you append l to s, you are adding a reference to a list which you then continue to grow. You want to append a copy of the l list's contents at the time when you append, to keep it static.
s.append(l[:])
This is a common FAQ; this question should probably be closed as a duplicate.

You would be better first finding all indices of 'A' and 'B', then iterating over those, avoiding brute force.
def find_substrings(lst)
idx_A = [i for i, c in enumerate(lst) if c == 'A']
idx_B = [i for i, c in enumerate(lst) if c == 'B']
return [lst[i:j+1] for i in idx_A for j in idx_B if j > i]

You can reset l to a copy of the string after l is appended l = l[:] right after the last append.

So, you want all the substrings that start with 'A' and end with 'B'?
When you use #Joeidden's code you can change need the for i in range(len(L) - 1): to for i in range(len(L)): because only strings that end with 'B' will be appended to s.
def find_substring(L):
s = []
for i in range(len(L)):
l = []
if L[i] == 'A':
for j in range(i, len(L)):
l.append(L[j])
if L[j] == 'B':
s.append(l[:])
return s

Another slightly different approach would be this:
L = ['C', 'A', 'B', 'A', 'A', 'X', 'B', 'Y', 'A']
def find_substring(L):
output = []
# Start searching for A.
for i in range(len(L)):
# If you found one start searching all B's until you reach the end.
if L[i]=='A':
for j in range(i,len(L),1):
# If you found a B, append the sublist from i index to j+1 index (positions of A and B respectively).
if L[j]=='B':
output.append(L[i:j+1])
return output
result = find_substring(L)
print(result)
Output:
[['A', 'B'], ['A', 'B', 'A', 'A', 'X', 'B'], ['A', 'A', 'X', 'B'], ['A', 'X', 'B']]
In case you need a list comprehension of the above:
def find_substring(L):
output = [L[i:j+1] for i in range(len(L)) for j in range(i,len(L),1) if L[i]=='A' and L[j]=='B']
return output

Python all combinations of a list of lists

So I have a list of lists of strings
[['a','b'],['c','d'],['e','f']]
and I want to get all possible combinations, such that the result is
[['a','b'],['c','d'],['e','f'],
['a','b','c','d'],['a','b','e','f'],['c','d','e','f'],
['a','b','c','d','e','f']]
So far I have come up with this code snippet
input = [['a','b'],['c','d'],['e','f']]
combs = []
for i in xrange(1, len(input)+1):
els = [x for x in itertools.combinations(input, i)]
combs.extend(els)
print combs
largely following an answer in this post.
But that results in
[(['a','b'],),(['c','d'],),(['e','f'],),
(['a','b'],['c','d']),(['a','b'],['e','f']),(['c','d'],['e','f']),
(['a','b'],['c', 'd'],['e', 'f'])]
and I am currently stumped, trying to find an elegant, pythonic way to unpack those tuples.

You can use itertools.chain.from_iterable to flatten the tuple of lists into a list. Example -
import itertools
input = [['a','b'],['c','d'],['e','f']]
combs = []
for i in xrange(1, len(input)+1):
els = [list(itertools.chain.from_iterable(x)) for x in itertools.combinations(input, i)]
combs.extend(els)
Demo -
>>> import itertools
>>> input = [['a','b'],['c','d'],['e','f']]
>>> combs = []
>>> for i in range(1, len(input)+1):
... els = [list(itertools.chain.from_iterable(x)) for x in itertools.combinations(input, i)]
... combs.extend(els)
...
>>> import pprint
>>> pprint.pprint(combs)
[['a', 'b'],
['c', 'd'],
['e', 'f'],
['a', 'b', 'c', 'd'],
['a', 'b', 'e', 'f'],
['c', 'd', 'e', 'f'],
['a', 'b', 'c', 'd', 'e', 'f']]

One idea for such a goal is to map integers from [0..2**n-1] where n is the number of sublists to all your target element according to a very simple rule:
Take the element of index k if (2**k)&i!=0 where i runs over [0..2**n-1]. In other word, i has to be read bitwise, and for each bit set, the corresponding element from l is kept. From a mathematical point of view it is one of the cleanest way of achieving what you want to do since it follows very closely the definition of the parts of a set (where you have exactly 2**n parts for a set with n elements).
Not tried but something like that should work:
l = [['a','b'],['c','d'],['e','f']]
n = len(l)
output = []
for i in range(2**n):
s = []
for k in range(n):
if (2**k)&i: s = s + l[k]
output.append(s)
If you don't want the empty list, just replace the relevant line with:
for i in range(1,2**n):

If you want all combinations, you may consider this simple way:
import itertools
a = [['a','b'],['c','d'],['e','f']]
a = a + [i + j for i in a for j in a if i != j] + [list(itertools.chain.from_iterable(a))]

With comprehension lists :
combs=[sum(x,[]) for i in range(len(l)) for x in itertools.combinations(l,i+1)]

Pythonic way to combine (interleave, interlace, intertwine) two lists in an alternating fashion?

I have two lists, the first of which is guaranteed to contain exactly one more item than the second. I would like to know the most Pythonic way to create a new list whose even-index values come from the first list and whose odd-index values come from the second list.
# example inputs
list1 = ['f', 'o', 'o']
list2 = ['hello', 'world']
# desired output
['f', 'hello', 'o', 'world', 'o']
This works, but isn't pretty:
list3 = []
while True:
try:
list3.append(list1.pop(0))
list3.append(list2.pop(0))
except IndexError:
break
How else can this be achieved? What's the most Pythonic approach?
If you need to handle lists of mismatched length (e.g. the second list is longer, or the first has more than one element more than the second), some solutions here will work while others will require adjustment. For more specific answers, see How to interleave two lists of different length? to leave the excess elements at the end, or How to elegantly interleave two lists of uneven length? to try to intersperse elements evenly, or Insert element in Python list after every nth element for the case where a specific number of elements should come before each "added" element.

Here's one way to do it by slicing:
>>> list1 = ['f', 'o', 'o']
>>> list2 = ['hello', 'world']
>>> result = [None]*(len(list1)+len(list2))
>>> result[::2] = list1
>>> result[1::2] = list2
>>> result
['f', 'hello', 'o', 'world', 'o']

There's a recipe for this in the itertools documentation (note: for Python 3):
from itertools import cycle, islice
def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
# Recipe credited to George Sakkis
num_active = len(iterables)
nexts = cycle(iter(it).__next__ for it in iterables)
while num_active:
try:
for next in nexts:
yield next()
except StopIteration:
# Remove the iterator we just exhausted from the cycle.
num_active -= 1
nexts = cycle(islice(nexts, num_active))

import itertools
print([x for x in itertools.chain.from_iterable(itertools.zip_longest(list1,list2)) if x])
I think this is the most pythonic way of doing it.

In Python 2, this should do what you want:
>>> iters = [iter(list1), iter(list2)]
>>> print list(it.next() for it in itertools.cycle(iters))
['f', 'hello', 'o', 'world', 'o']

Without itertools and assuming l1 is 1 item longer than l2:
>>> sum(zip(l1, l2+[0]), ())[:-1]
('f', 'hello', 'o', 'world', 'o')
In python 2, using itertools and assuming that lists don't contain None:
>>> filter(None, sum(itertools.izip_longest(l1, l2), ()))
('f', 'hello', 'o', 'world', 'o')

If both lists have equal length, you can do:
[x for y in zip(list1, list2) for x in y]
As the first list has one more element, you can add it post hoc:
[x for y in zip(list1, list2) for x in y] + [list1[-1]]
Edit: To illustrate what is happening in that first list comprehension, this is how you would spell it out as a nested for loop:
result = []
for y in zip(list1, list2): # y is is a 2-tuple, containining one element from each list
for x in y: # iterate over the 2-tuple
result.append(x) # append each element individually

I know the questions asks about two lists with one having one item more than the other, but I figured I would put this for others who may find this question.
Here is Duncan's solution adapted to work with two lists of different sizes.
list1 = ['f', 'o', 'o', 'b', 'a', 'r']
list2 = ['hello', 'world']
num = min(len(list1), len(list2))
result = [None]*(num*2)
result[::2] = list1[:num]
result[1::2] = list2[:num]
result.extend(list1[num:])
result.extend(list2[num:])
result
This outputs:
['f', 'hello', 'o', 'world', 'o', 'b', 'a', 'r']

Here's a one liner that does it:
list3 = [ item for pair in zip(list1, list2 + [0]) for item in pair][:-1]

Here's a one liner using list comprehensions, w/o other libraries:
list3 = [sub[i] for i in range(len(list2)) for sub in [list1, list2]] + [list1[-1]]
Here is another approach, if you allow alteration of your initial list1 by side effect:
[list1.insert((i+1)*2-1, list2[i]) for i in range(len(list2))]

This one is based on Carlos Valiente's contribution above
with an option to alternate groups of multiple items and make sure that all items are present in the output :
A=["a","b","c","d"]
B=[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]
def cyclemix(xs, ys, n=1):
for p in range(0,int((len(ys)+len(xs))/n)):
for g in range(0,min(len(ys),n)):
yield ys[0]
ys.append(ys.pop(0))
for g in range(0,min(len(xs),n)):
yield xs[0]
xs.append(xs.pop(0))
print [x for x in cyclemix(A, B, 3)]
This will interlace lists A and B by groups of 3 values each:
['a', 'b', 'c', 1, 2, 3, 'd', 'a', 'b', 4, 5, 6, 'c', 'd', 'a', 7, 8, 9, 'b', 'c', 'd', 10, 11, 12, 'a', 'b', 'c', 13, 14, 15]

Might be a bit late buy yet another python one-liner. This works when the two lists have equal or unequal size. One thing worth nothing is it will modify a and b. If it's an issue, you need to use other solutions.
a = ['f', 'o', 'o']
b = ['hello', 'world']
sum([[a.pop(0), b.pop(0)] for i in range(min(len(a), len(b)))],[])+a+b
['f', 'hello', 'o', 'world', 'o']

from itertools import chain
list(chain(*zip('abc', 'def'))) # Note: this only works for lists of equal length
['a', 'd', 'b', 'e', 'c', 'f']

itertools.zip_longest returns an iterator of tuple pairs with any missing elements in one list replaced with fillvalue=None (passing fillvalue=object lets you use None as a value). If you flatten these pairs, then filter fillvalue in a list comprehension, this gives:
>>> from itertools import zip_longest
>>> def merge(a, b):
... return [
... x for y in zip_longest(a, b, fillvalue=object)
... for x in y if x is not object
... ]
...
>>> merge("abc", "defgh")
['a', 'd', 'b', 'e', 'c', 'f', 'g', 'h']
>>> merge([0, 1, 2], [4])
[0, 4, 1, 2]
>>> merge([0, 1, 2], [4, 5, 6, 7, 8])
[0, 4, 1, 5, 2, 6, 7, 8]
Generalized to arbitrary iterables:
>>> def merge(*its):
... return [
... x for y in zip_longest(*its, fillvalue=object)
... for x in y if x is not object
... ]
...
>>> merge("abc", "lmn1234", "xyz9", [None])
['a', 'l', 'x', None, 'b', 'm', 'y', 'c', 'n', 'z', '1', '9', '2', '3', '4']
>>> merge(*["abc", "x"]) # unpack an iterable
['a', 'x', 'b', 'c']
Finally, you may want to return a generator rather than a list comprehension:
>>> def merge(*its):
... return (
... x for y in zip_longest(*its, fillvalue=object)
... for x in y if x is not object
... )
...
>>> merge([1], [], [2, 3, 4])
<generator object merge.<locals>.<genexpr> at 0x000001996B466740>
>>> next(merge([1], [], [2, 3, 4]))
1
>>> list(merge([1], [], [2, 3, 4]))
[1, 2, 3, 4]
If you're OK with other packages, you can try more_itertools.roundrobin:
>>> list(roundrobin('ABC', 'D', 'EF'))
['A', 'D', 'E', 'B', 'F', 'C']

My take:
a = "hlowrd"
b = "el ol"
def func(xs, ys):
ys = iter(ys)
for x in xs:
yield x
yield ys.next()
print [x for x in func(a, b)]

def combine(list1, list2):
lst = []
len1 = len(list1)
len2 = len(list2)
for index in range( max(len1, len2) ):
if index+1 <= len1:
lst += [list1[index]]
if index+1 <= len2:
lst += [list2[index]]
return lst

How about numpy? It works with strings as well:
import numpy as np
np.array([[a,b] for a,b in zip([1,2,3],[2,3,4,5,6])]).ravel()
Result:
array([1, 2, 2, 3, 3, 4])

Stops on the shortest:
def interlace(*iters, next = next) -> collections.Iterable:
"""
interlace(i1, i2, ..., in) -> (
i1-0, i2-0, ..., in-0,
i1-1, i2-1, ..., in-1,
.
.
.
i1-n, i2-n, ..., in-n,
)
"""
return map(next, cycle([iter(x) for x in iters]))
Sure, resolving the next/__next__ method may be faster.

Multiple one-liners inspired by answers to another question:
import itertools
list(itertools.chain.from_iterable(itertools.izip_longest(list1, list2, fillvalue=object)))[:-1]
[i for l in itertools.izip_longest(list1, list2, fillvalue=object) for i in l if i is not object]
[item for sublist in map(None, list1, list2) for item in sublist][:-1]

An alternative in a functional & immutable way (Python 3):
from itertools import zip_longest
from functools import reduce
reduce(lambda lst, zipped: [*lst, *zipped] if zipped[1] != None else [*lst, zipped[0]], zip_longest(list1, list2),[])

using for loop also we can achive this easily:
list1 = ['f', 'o', 'o']
list2 = ['hello', 'world']
list3 = []
for i in range(len(list1)):
#print(list3)
list3.append(list1[i])
if i < len(list2):
list3.append(list2[i])
print(list3)
output :
['f', 'hello', 'o', 'world', 'o']
Further by using list comprehension this can be reduced. But for understanding this loop can be used.

My approach looks as follows:
from itertools import chain, zip_longest
def intersperse(*iterators):
# A random object not occurring in the iterators
filler = object()
r = (x for x in chain.from_iterable(zip_longest(*iterators, fillvalue=filler)) if x is not filler)
return r
list1 = ['f', 'o', 'o']
list2 = ['hello', 'world']
print(list(intersperse(list1, list2)))
It works for an arbitrary number of iterators and yields an iterator, so I applied list() in the print line.

def alternate_elements(small_list, big_list):
mew = []
count = 0
for i in range(len(small_list)):
mew.append(small_list[i])
mew.append(big_list[i])
count +=1
return mew+big_list[count:]
if len(l2)>len(l1):
res = alternate_elements(l1,l2)
else:
res = alternate_elements(l2,l1)
print(res)
Here we swap lists based on size and perform, can someone provide better solution with time complexity O(len(l1)+len(l2))

I'd do the simple:
chain.from_iterable( izip( list1, list2 ) )
It'll come up with an iterator without creating any additional storage needs.

This is nasty but works no matter the size of the lists:
list3 = [
element for element in
list(itertools.chain.from_iterable([
val for val in itertools.izip_longest(list1, list2)
]))
if element != None
]

Obviously late to the party, but here's a concise one for equal-length lists:
output = [e for sub in zip(list1,list2) for e in sub]
It generalizes for an arbitrary number of equal-length lists, too:
output = [e for sub in zip(list1,list2,list3) for e in sub]
etc.

I'm too old to be down with list comprehensions, so:
import operator
list3 = reduce(operator.add, zip(list1, list2))