How to use itertools to compute all combinations with repeating elements? [duplicate] - python

This question already has an answer here:
Which itertools generator doesn't skip any combinations?
(1 answer)
Closed 8 years ago.
I have tried to use itertools to compute all combinations of a list ['a', 'b', 'c'] using combinations_with_replacement with repeating elements. The problem is in the fact that the indices seem to be used to distinguish the elements:
Return r length subsequences of elements from the input iterable allowing individual elements to be repeated more than once.
Combinations are emitted in lexicographic sort order. So, if the input
iterable is sorted, the combination tuples will be produced in sorted
order.
Elements are treated as unique based on their position, not on their
value. So if the input elements are unique, the generated combinations
will also be unique.
Sot this code snippet:
import itertools
for item in itertools.combinations_with_replacement(['a','b','c'], 3):
print (item)
results in this output:
('a', 'a', 'a')
('a', 'a', 'b')
('a', 'a', 'c')
('a', 'b', 'b')
('a', 'b', 'c')
('a', 'c', 'c')
('b', 'b', 'b')
('b', 'b', 'c')
('b', 'c', 'c')
('c', 'c', 'c')
And what I need is the combination set to contain elements like: ('a', 'b', 'a') which seem to be missing. How to compute the complete combination set?


It sounds like you want itertools.product:
>>> from itertools import product
>>> for item in product(['a', 'b', 'c'], repeat=3):
... print item
...
('a', 'a', 'a')
('a', 'a', 'b')
('a', 'a', 'c')
('a', 'b', 'a')
('a', 'b', 'b')
('a', 'b', 'c')
('a', 'c', 'a')
('a', 'c', 'b')
('a', 'c', 'c')
('b', 'a', 'a')
('b', 'a', 'b')
('b', 'a', 'c')
('b', 'b', 'a')
('b', 'b', 'b')
('b', 'b', 'c')
('b', 'c', 'a')
('b', 'c', 'b')
('b', 'c', 'c')
('c', 'a', 'a')
('c', 'a', 'b')
('c', 'a', 'c')
('c', 'b', 'a')
('c', 'b', 'b')
('c', 'b', 'c')
('c', 'c', 'a')
('c', 'c', 'b')
('c', 'c', 'c')
>>>


For such small sequences you could use no itertools at all:
abc = ("a", "b", "c")
print [(x, y, z) for x in abc for y in abc for z in abc]
# output:
[('a', 'a', 'a'),
('a', 'a', 'b'),
('a', 'a', 'c'),
('a', 'b', 'a'),
('a', 'b', 'b'),
('a', 'b', 'c'),
('a', 'c', 'a'),
('a', 'c', 'b'),
('a', 'c', 'c'),
('b', 'a', 'a'),
('b', 'a', 'b'),
('b', 'a', 'c'),
('b', 'b', 'a'),
('b', 'b', 'b'),
('b', 'b', 'c'),
('b', 'c', 'a'),
('b', 'c', 'b'),
('b', 'c', 'c'),
('c', 'a', 'a'),
('c', 'a', 'b'),
('c', 'a', 'c'),
('c', 'b', 'a'),
('c', 'b', 'b'),
('c', 'b', 'c'),
('c', 'c', 'a'),
('c', 'c', 'b'),
('c', 'c', 'c')]

Related

Using itertools product but needing combination behavior

I am trying to generate a list of possible strings of characters from input, accounting for wildcards "?" and "*". The ordering of list items should not matter, meaning if ABC already exists in the list, then I do not want to add ACB, or any other ordering (processing speed issue). The code I am using is below:
import itertools
from itertools import permutations
#####################################################
def getWordsFromTiles(tiles, word):
#####################################################
return all(word.count(i) <= tiles.count(i) for i in word)
#####################################################
Main
#####################################################
chars = A?*
wilds = [
('A','B','C','D','E','F','G','H','I','J','K',
'L','M','N','O','P','Q','R','S','T','U','V',
'W','X','Y','Z')
if char == "?" or char == "*" else (char) for char in chars]
for p in itertools.product(*wilds):
x = ''.join(p)
hits.extend([word for word in data if getWordsFromTiles(x.upper(), word) and word not in hits])
This generates a list like the following:
A,A,A
A,A,B
A,A,C
.....
A,B,A
I actually do not care about the order of these list, so I would like to not have A,B,A when I have already generated "A,A,B. Any ideas how to implement this?

actually do not care about the order of these list, so I would like to not have A,B,A when I have already generated "A,A,B".
Is this what you are looking for?
from itertools import combinations_with_replacement
s = ['A','B','C','D']
list(combinations_with_replacement(s,3))
[('A', 'A', 'A'),
('A', 'A', 'B'),
('A', 'A', 'C'),
('A', 'A', 'D'),
('A', 'B', 'B'),
('A', 'B', 'C'),
('A', 'B', 'D'),
('A', 'C', 'C'),
('A', 'C', 'D'),
('A', 'D', 'D'),
('B', 'B', 'B'),
('B', 'B', 'C'),
('B', 'B', 'D'),
('B', 'C', 'C'),
('B', 'C', 'D'),
('B', 'D', 'D'),
('C', 'C', 'C'),
('C', 'C', 'D'),
('C', 'D', 'D'),
('D', 'D', 'D')]

How to pop item from list and push it back to list based on condition using python

I have list with urls for crawling.['http://domain1.com','http://domain1.com/page1','http://domain2.com']
Code:
prev_domain = ''
while urls:
url = urls.pop()
if base_url(url) == prev_domain: # base_url is custom function return domain of an url
urls.append(url) # is this is possible?
continue
else:
crawl(url)
Basically I dont want to crawl webpages of same domain continuously. Continuosly crawling a domain url, return http response status code with 429: Too Many Requests. The user has sent too many requests in a given amount of time ("rate limiting"). To by-pass this issue, I'm planning to go with below logic.
Loop through all items in the list and compare current element base url with previously processed element base url.
If base urls are different then process for next step, otherwise do not process current element, just append this element to the same list.
Note : If urls in list are of same domain, make delay in processing each element and then execute.
Please provide your thoughts.

Your algorithm is almost correct, but not the implementation:
>>> L = [1,2,3]
>>> L.pop()
3
>>> L.append(3)
>>> L
[1, 2, 3]
That's why your program loops forever: if the domain is the same as the previous domain, you just append then pop then append, then.... What you need is not a stack, it's a round robin:
>>> L.pop()
3
>>> L.insert(0, 3)
>>> L
[3, 1, 2]
Let's take a shuffled list of permutations of "abcd":
>>> L = [('b', 'c', 'd', 'a'), ('d', 'c', 'b', 'a'), ('a', 'c', 'd', 'b'), ('c', 'd', 'a', 'b'), ('b', 'd', 'a', 'c'), ('b', 'a', 'd', 'c'), ('b', 'c', 'a', 'd'), ('a', 'b', 'd', 'c'), ('d', 'a', 'b', 'c'), ('a', 'b', 'c', 'd'), ('d', 'c', 'a', 'b'), ('a', 'd', 'c', 'b'), ('d', 'a', 'c', 'b'), ('c', 'd', 'b', 'a'), ('d', 'b', 'c', 'a'), ('d', 'b', 'a', 'c'), ('a', 'd', 'b', 'c'), ('b', 'd', 'c', 'a'), ('c', 'b', 'd', 'a'), ('c', 'a', 'b', 'd'), ('b', 'a', 'c', 'd')]
The first letter is the domain. Here's a slightly modified version of your code:
>>> prev = None
>>> while L:
... e = L.pop()
... if L and e[0] == prev:
... L.insert(0, e)
... else:
... print(e)
... prev = e[0]
('b', 'a', 'c', 'd')
('c', 'a', 'b', 'd')
('b', 'd', 'c', 'a')
('a', 'd', 'b', 'c')
('d', 'b', 'a', 'c')
('c', 'd', 'b', 'a')
('d', 'a', 'c', 'b')
('a', 'd', 'c', 'b')
('d', 'c', 'a', 'b')
('a', 'b', 'c', 'd')
('d', 'a', 'b', 'c')
('a', 'b', 'd', 'c')
('b', 'c', 'a', 'd')
('c', 'd', 'a', 'b')
('a', 'c', 'd', 'b')
('d', 'c', 'b', 'a')
('b', 'c', 'd', 'a')
('c', 'b', 'd', 'a')
('d', 'b', 'c', 'a')
('b', 'a', 'd', 'c')
('b', 'd', 'a', 'c')
The modification is: if L and, because if the last element of the list domain is prev, then you'll loop forever with your one element list: pop, same as prev, insert, pop, ...(as with pop/append)
Here's another option: create a dict domain -> list of urls:
>>> d = {}
>>> for e in L:
... d.setdefault(e[0], []).append(e)
>>> d
{'b': [('b', 'c', 'd', 'a'), ('b', 'd', 'a', 'c'), ('b', 'a', 'd', 'c'), ('b', 'c', 'a', 'd'), ('b', 'd', 'c', 'a'), ('b', 'a', 'c', 'd')], 'd': [('d', 'c', 'b', 'a'), ('d', 'a', 'b', 'c'), ('d', 'c', 'a', 'b'), ('d', 'a', 'c', 'b'), ('d', 'b', 'c', 'a'), ('d', 'b', 'a', 'c')], 'a': [('a', 'c', 'd', 'b'), ('a', 'b', 'd', 'c'), ('a', 'b', 'c', 'd'), ('a', 'd', 'c', 'b'), ('a', 'd', 'b', 'c')], 'c': [('c', 'd', 'a', 'b'), ('c', 'd', 'b', 'a'), ('c', 'b', 'd', 'a'), ('c', 'a', 'b', 'd')]}
Now, take an element of every domain and clear the dict, then loop until the dict is empty:
>>> while d:
... for k, vs in d.items():
... e = vs.pop()
... print (e)
... d = {k: vs for k, vs in d.items() if vs} # clear the dict
...
('b', 'a', 'c', 'd')
('d', 'b', 'a', 'c')
('a', 'd', 'b', 'c')
('c', 'a', 'b', 'd')
('b', 'd', 'c', 'a')
('d', 'b', 'c', 'a')
('a', 'd', 'c', 'b')
('c', 'b', 'd', 'a')
('b', 'c', 'a', 'd')
('d', 'a', 'c', 'b')
('a', 'b', 'c', 'd')
('c', 'd', 'b', 'a')
('b', 'a', 'd', 'c')
('d', 'c', 'a', 'b')
('a', 'b', 'd', 'c')
('c', 'd', 'a', 'b')
('b', 'd', 'a', 'c')
('d', 'a', 'b', 'c')
('a', 'c', 'd', 'b')
('b', 'c', 'd', 'a')
('d', 'c', 'b', 'a')
The output is more uniform.

Check the following code snippet,
urls = ['http://domain1.com','http://domain1.com/page1','http://domain2.com']
crawl_for_urls = {}
for url in urls:
domain = base_url(url)
if domain not in crowl_for_urls:
crawl_for_urls.update({domain:url})
crawl(url)
crawl() will be called only for unique domain.
Or you can use:
urls = ['http://domain1.com','http://domain1.com/page1','http://domain2.com']
crawl_for_urls = {}
for url in urls:
domain = base_url(url)
if domain not in crowl_for_urls:
crawl_for_urls.update({domain:[url]})
crawl(url)
else:
crawl_for_urls.get(domain, []).append(url)
This way you can categories the URL's based on domain and also can use crawl() for unique domain.

Python - How to get all Combinations of names from a list: TypeError: 'list' object is not callable

I get an error when trying to print the permutation/combination of a user generated list of names.
I tried a couple of things from itertools, but can't get either permutations or combinations to work. Ran into some other errors along the way regarding concatenating strings, but currently getting a: TypeError: 'list' object not callable.
I know I'm making a simple mistake, but can't sort it out. Please help!
from itertools import combinations
name_list = []
for i in range(0,20):
name = input('Add up to 20 names.\nWhen finished, enter "Done" to see all first and middle name combinations.\nName: ')
name_list.append(name)
if name != 'Done':
print(name_list)
else:
name_list.remove('Done')
print(name_list(combinations))
I expect:
1) the user adds a name to list
2) the list prints showing user contents of list
3) when finished the user inputs "Done":
a) 'Done' is removed from the list
b) all the combinations of the remaining items on the list printed

Permutations and combinations are two different beasts. Look:
>>> from itertools import permutations,combinations
>>> from pprint import pprint
>>> l = ['a', 'b', 'c', 'd']
>>> pprint(list(combinations(l, 2)))
[('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'c'), ('b', 'd'), ('c', 'd')]
>>> pprint(list(permutations(l)))
[('a', 'b', 'c', 'd'),
('a', 'b', 'd', 'c'),
('a', 'c', 'b', 'd'),
('a', 'c', 'd', 'b'),
('a', 'd', 'b', 'c'),
('a', 'd', 'c', 'b'),
('b', 'a', 'c', 'd'),
('b', 'a', 'd', 'c'),
('b', 'c', 'a', 'd'),
('b', 'c', 'd', 'a'),
('b', 'd', 'a', 'c'),
('b', 'd', 'c', 'a'),
('c', 'a', 'b', 'd'),
('c', 'a', 'd', 'b'),
('c', 'b', 'a', 'd'),
('c', 'b', 'd', 'a'),
('c', 'd', 'a', 'b'),
('c', 'd', 'b', 'a'),
('d', 'a', 'b', 'c'),
('d', 'a', 'c', 'b'),
('d', 'b', 'a', 'c'),
('d', 'b', 'c', 'a'),
('d', 'c', 'a', 'b'),
('d', 'c', 'b', 'a')]
>>>

for use combinations , you need to give the r as argument.this code give all combinations for all numbers(0 to length of list),
from itertools import combinations
name_list = []
for i in range(0,20):
name = raw_input('Add up to 20 names.\nWhen finished, enter "Done" to see all first and middle name combinations.\nName: ')
name_list.append(name)
if name != 'Done':
print(name_list)
else:
name_list.remove('Done')
break
for i in range(len(name_list) + 1):
print(list(combinations(name_list, i)))
print("\n")

how to do a in depth permutation of some given letters in python?

So i have been trying to permutate some letters in python using the permutation library but i saw that it only return the given letters so that no letters are a duplicate, if i have the letters a, b and c the permutation will be as following.
abc
acb
bac
bca
cab
cba
Now i have been trying for a way for a sort of more in depth way i guess so i have it like.
aaa
aab
aac
aba
abc
abc
aca
Is there a proper way to do this in python?
It could be that i'm wrong but that's the way i saw it.
Edit after solve:
People who wondered what i tried, i used the permutation part of the itertools library:
>>>print([x for x in itertools.permutations('1234')])
>>>[('1', '2', '3', '4'), ('1', '2', '4', '3'), ('1', '3', '2', '4') ... ]

Perhaps you want something like itertools.product
from itertools import product
print(list(product('abc', repeat = 3)))
# [('a', 'a', 'a'), ('a', 'a', 'b'), ('a', 'a', 'c'), ('a', 'b', 'a'), ('a', 'b', 'b'), ('a', 'b', 'c'), ('a', 'c', 'a'), ('a', 'c', 'b'), ('a', 'c', 'c'), ('b', 'a', 'a'), ('b', 'a', 'b'), ('b', 'a', 'c'), ('b', 'b', 'a'), ('b', 'b', 'b'), ('b', 'b', 'c'), ('b', 'c', 'a'), ('b', 'c', 'b'), ('b', 'c', 'c'), ('c', 'a', 'a'), ('c', 'a', 'b'), ('c', 'a', 'c'), ('c', 'b', 'a'), ('c', 'b', 'b'), ('c', 'b', 'c'), ('c', 'c', 'a'), ('c', 'c', 'b'), ('c', 'c', 'c')]

Python loop (insert items in a list at dermined places)

I have this code:
number = 2
size = 5
list_b = [("b","b","b")]
list_a = [("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a")]
for i in range(number):
list_a.insert(size,list_b)
print list_a
it gives me this:
[('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('b', 'b', 'b'),
('b', 'b', 'b'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a')]
basically, it inserts 2 times the list_b in the position defined by size
I want a loop that repeats itself so that list_b is inserted the number of times defined in number but repeats size times. It difficult to explain, so here is the result that I want:
[('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('b', 'b', 'b'),
('b', 'b', 'b'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('b', 'b', 'b'),
('b', 'b', 'b'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('b', 'b', 'b'),
('b', 'b', 'b'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('b', 'b', 'b'),
('b', 'b', 'b'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('a', 'a', 'a'),
('b', 'b', 'b'),
('b', 'b', 'b'),...and so on]
EDIT
and if I had this:
list_a = [a, ] * 15
list_b = [b,]
s = 5
n = 2
I want to obtain this:
[b,b,a,a,a,a,a,b,b,b,b,a,a,a,a,a,b,b,b,b,a,a,a,a,a,b,b]
since this is an example and list_a, s and n will vary, how can I do this in one or two loops?
Thanks,
Favolas

For the sake of the argument, I'll call the ('a', 'a', 'a') => a and ('b', 'b', 'b') => b.
number=2
size=5
list_a=["a"]*20
list_b=["b"]
workfor=len(list_a)+(len(list_a)/size)*number*len(list_b)
i=0
while i<workfor:
i+=size
for times in range(number):
for elem in list_b:
list_a.insert(i,elem)
i+=len(list_b)
print list_a
Results in =>
['a', 'a', 'a', 'a', 'a', 'b', 'b', 'a', 'a', 'a', 'a', 'a', 'b', 'b', 'a', 'a', 'a', 'a', 'a', 'b', 'b', 'a', 'a', 'a', 'a', 'a', 'b', 'b']

#!/usr/bin/python
number = 2
size = 5
list_b = [("b","b","b")]
list_a = [("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a"),("a","a","a")]
if __name__ == '__main__':
insertion_count = len(list_a) / size
for j in xrange(insertion_count):
# compute insertion position
pos = (j+1)*size + j * number
for i in range(number):
list_a.insert(pos,list_b)
print list_a

from itertools import chain, izip, repeat
list_a = [('a', 'a', 'a')] * 15
list_b = [('b', 'b', 'b')]
a5b2s = [iter(list_a)] * 5 + [repeat(*list_b)] * 2
list_a[:] = chain.from_iterable(izip(*a5b2s))

>>> s,n=5,2
>>> a=[1,]*17
>>> b=2
>>> for i in range(len(a)//s*s,0,-s):
for j in range(n):
a.insert(i,b)
>>> a
[1, 1, 1, 1, 1, 2, 2, 1, 1, 1, 1, 1, 2, 2, 1, 1, 1, 1, 1, 2, 2, 1, 1]

Categories

Resources