Related
I'm having difficulty adding to a list iteratively.
Here's a MWE:
# Given a nested list of values, or sets
sets = [[1, 2, 3], [1, 2, 4], [1, 2, 5]]
# add a value to each sublist giving the number of that set in the list.
n_sets = len(sets)
for s in range(n_sets):
(sets[s]).insert(0, s)
# Now repeat those sets reps times
reps = 4
expanded_sets = [item for item in sets for i in range(reps)]
# then assign a repetition number to each occurance of a set.
rep_list = list(range(reps)) * n_sets
for i in range(n_sets * reps):
(expanded_sets[i]).insert(0, rep_list[i])
expanded_sets
which returns
[[3, 2, 1, 0, 0, 1, 2, 3],
[3, 2, 1, 0, 0, 1, 2, 3],
[3, 2, 1, 0, 0, 1, 2, 3],
[3, 2, 1, 0, 0, 1, 2, 3],
[3, 2, 1, 0, 1, 1, 2, 4],
[3, 2, 1, 0, 1, 1, 2, 4],
[3, 2, 1, 0, 1, 1, 2, 4],
[3, 2, 1, 0, 1, 1, 2, 4],
[3, 2, 1, 0, 2, 1, 2, 5],
[3, 2, 1, 0, 2, 1, 2, 5],
[3, 2, 1, 0, 2, 1, 2, 5],
[3, 2, 1, 0, 2, 1, 2, 5]]
instead of the desired
[[0, 0, 1, 2, 3],
[1, 0, 1, 2, 3],
[2, 0, 1, 2, 3],
[3, 0, 1, 2, 3],
[0, 1, 1, 2, 4],
[1, 1, 1, 2, 4],
[2, 1, 1, 2, 4],
[3, 1, 1, 2, 4],
[0, 2, 1, 2, 5],
[1, 2, 1, 2, 5],
[2, 2, 1, 2, 5],
[3, 2, 1, 2, 5]]
Just for fun, the first loop returns an expected value of sets
[[0, 1, 2, 3], [1, 1, 2, 4], [2, 1, 2, 5]]
but after the second loop sets changed to
[[3, 2, 1, 0, 0, 1, 2, 3], [3, 2, 1, 0, 1, 1, 2, 4], [3, 2, 1, 0, 2, 1, 2, 5]]
I suspect the issue has something to do with copies and references. I've tried adding .copy() and slices in various places, but with the indexed sublists I haven't come across a combo that works. I'm running Python 3.10.6.
Thanks for looking!
Per suggested solution, [list(range(reps)) for _ in range(n_sets)] doesn't correctly replace the list(range(reps)) * n_sets, since it gives [[0, 1, 2, 3], [0, 1, 2, 3], [0, 1, 2, 3]] instead of
the desired [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]. Do I need to flatten, or is there a syntax with the _ notation that gives me a single list?
Further update . . .
replacing
rep_list = list(range(reps)) * n_sets
with
rep_list_nest = [list(range(reps)) for _ in range(n_sets)]
rep_list = [i for sublist in rep_list_nest for i in sublist]
gives the same undesired result for expanded_sets.
The problem is here:
expanded_sets = [item
for item in sets
for i in range(reps)]
This list now contains the same element of sets four times in a row, followed by the next element repeated four times, and so on.
Creating copies of item fixes the issue:
expanded_sets = [item.copy()
for item in sets
for i in range(reps)]
Try it online
If you want a more pythonic approach, then recognize that the result is a product of two ranges, and your original sets all concatenated together:
from itertools import product
sets = [[1, 2, 3], [1, 2, 4], [1, 2, 5]]
expanded_sets = [[inner_counter, outer_counter] + sets_elem
for sets_elem, outer_counter, inner_counter in product(sets, range(len(sets)), range(4))]
Try it online
The original list is mutated after applying GA operators
I am using the function below where I copy part of rankedChromos to newpop (list of lists) and then use a while loop to add remaining lists to it until the length of newpop is equal to the predetermined popSize. But for some reason, the newpop list is mutated after the while loop is implemented. In the while loop, I am trying to implement some GA operators and all I do is append the new individuals to the newpop list, which should not mutate the newpop but it does. Not just that, even the rankedChromos is mutated for some reason.
def nextGenPopulation (population, rankedPop, num_robots, crossover_rate):
newpop = 0
fitnessScores = [item[-1] for item in rankedPop ] # extract fitness scores
rankedChromos = [item[0] for item in rankedPop ] # extract chromosomes
popSize = len(population)
newpop = []
print("\n fitness", fitnessScores)
newpop.extend(rankedChromos[:int(popSize*0.4)]) # elitism
print("newpop before operations", newpop)
while len(newpop) < popSize:
ind1, ind2 = selectFittest (fitnessScores, rankedChromos)
ind1, ind2 = breed (ind1, ind2, num_robots, crossover_rate)
newpop.append(ind1)
newpop.append(ind2)
print("\n newpop after operation", newpop)
return newpop
For example if I am using 5 for popSize, the first 3 lists of newpop should remain the same before and after the implementation of the loop but it's not the same for some reason (example below) and I cannot understand why. I would very much appreciate any help. Thank you!
newpop before operation = [[0, 3, 0, 3, 2, 0, 3, 3, 1, 1, 1, 1, 2, 0, 2, 0, 2, 3, 1, 1], [1, 3, 2, 3, 1, 1, 2, 0, 1, 3, 3, 1, 2, 2, 3, 2, 2, 2, 0, 2]]
newpop after operation = [[0, 3, 0, 3, 2, 0, 3, 3, 1, 1, 1, 1, 1, 1, 2, 0, 0, 3, 1, 1], [1, 3, 2, 3, 1, 1, 2, 0, 1, 3, 3, 1, 2, 2, 3, 2, 2, 2, 0, 2], [0, 3, 0, 3, 2, 0, 3, 3, 1, 1, 1, 1, 1, 1, 2, 0, 0, 3, 1, 1], [2, 0, 0, 1, 0, 1, 2, 2, 1, 0, 0, 2, 2, 0, 3, 3, 2, 0, 0, 0], [2, 0, 0, 1, 0, 1, 2, 2, 1, 0, 0, 2, 2, 0, 3, 3, 2, 0, 0, 0], [0, 3, 0, 3, 2, 0, 3, 3, 1, 1, 1, 1, 1, 1, 2, 0, 0, 3, 1, 1]]
newpop.extend(rankedChromos[:int(popSize*0.4)]) it is actually copying the references for the lists from rankedChromos for the first int(popSize*0.4) - 1 lists into newpop, this means that any modification for rankedChromos for those lists will be reflected in newpop, most probably the function selectFittest it is mutating rankedChromos and as a result, you see the mutation in your variable newpop, you could use copy.deepcopy to eliminate this behaviour
from copy import deepcopy
newpop = deepcopy(rankedChromos[:int(popSize*0.4)])
here is an example to understand your issue:
list_1 = [[1, 1], [2, 2], [3, 3], [4, 4]] # rankedChromos in your example
list_2 = [] # newpop in your example
list_2.extend(list_1[1: 3])
print('before operation',list_2)
# the function that mutates, selectFittest in your example
def some_fumction(l):
for i in l:
i[1:] = [-9]
some_fumction(list_1)
print('after operation',list_2)
output:
before operation [[2, 2], [3, 3]]
after operation [[2, -9], [3, -9]]
I have a numpy array like the following:
A = np.array([[1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 5, 1, 1, 1],
[1, 1, 1, 1, 3, 3, 1, 1],
[1, 1, 1, 1, 1, 1, 2, 1],
[1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 4, 1, 1]])
I am looking for the minimum indices in each column. I found this using numpy.argmin as follows:
I = np.zeros(A.shape[1], dtype=np.int64)
for j in range(A.shape[1]):
I[j] = np.argmin(A[:, j])
This gives me I = [0, 0, 0, 0, 0, 0, 0, 0]. Since there are ties, I could obtain the following: I = [0, 1, 2, 3, 4, 0, 5, 1], where I break the ties by the index that is least used (from the previous indices).
In more details:
For j=0, we have np.argmin(A[:, 0]) in [0, 1, 2, 3, 4, 5] and, say, we choose np.argmin(A[:, 0]) = 0.
For j=1, we have np.argmin(A[:, 1]) in [0, 1, 2, 3, 4, 5] and we have to choose the minimum index from [1, 2, 3, 4, 5] since these indices are the least used (we already choose np.argmin(A[:, 0]) = 0 for j=0). Say, we choose np.argmin(A[:, 1])=1.
For j=2, we have np.argmin(A[:, 2]) in [0, 1, 2, 3, 4, 5] and we have to choose the minimum index from [2, 3, 4, 5] since these indices are the least used.
We continue in this way...
For j=5, we have np.argmin(A[:, 5]) in [0, 1, 3, 4] and we have to choose the minimum index from [0, 1, 3, 4] since these indices are the least used. Say we choose np.argmin(A[:, 5])=0.
For j=6, we have np.argmin(A[:, 6]) in [0, 1, 2, 4, 5] and we have to choose from [5] since these indices are the least used. We choose np.argmin(A[:, 6])=5.
For j=7, we have np.argmin(A[:, 7]) in [0, 1, 2, 3, 4, 5] and we have to choose from [1, 2, 3, 4, 5] since these indices are the least used. Say we choose np.argmin(A[:, 7])=1.
I hope it is clear. My question is how to find the minimum indices and break ties by the least used indices in Python?
You could use min combined with a dictionary for keeping the counts of each index:
import numpy as np
A = np.array([[1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 5, 1, 1, 1],
[1, 1, 1, 1, 3, 3, 1, 1],
[1, 1, 1, 1, 1, 1, 2, 1],
[1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 4, 1, 1]])
counts = {}
I = np.zeros(A.shape[1], dtype=np.int64)
for j in range(A.shape[1]):
_, _, i = min([(v, counts.get(i, 0), i) for i, v in enumerate(A[:, j])])
counts[i] = counts.get(i, 0) + 1
I[j] = i
print(I)
Output
[0 1 2 3 4 0 5 1]
The idea is to create the following key: (value, count of index, index), and then use the normal comparison of tuples, so if the values are equal the one with less counts of the corresponding index will be selected, if both counts are equal get the one with lower index will be selected.
I would like to generate the following lists in Python:
[1, 1, 1, 2, 2]
[1, 1, 2, 1, 2]
... etc
[2, 1, 2, 1, 1]
[2, 2, 1, 1, 1]
There are always two "2"s and three "1"s in any list.
My intuition suggests that I will need to use the itertools module to do this. However, I am not sure where to begin, though I have read the documentation and looked at examples. Any suggestions?
You can notice that the number of such lists is equal to the number of ways to place two "2"s in a sequence of length 5. This suggests the following solution:
n = 5 # total length
n2 = 2 # number of "2"s
for idx in itertools.combinations( xrange(n), n2 ):
print [ 2 if i in idx else 1 for i in xrange(n) ]
It's easy to see that the answer using permutations is iterating over n! solutions, while my solution iterates over n!/( (n-n2)! * n2!). For example if the input list is [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2], the solution using permutations is ~90,000,000 times slower (10! * 4!)
You can use itertools.permutations and set (to eliminate duplicates):
>>> from itertools import permutations
>>> for combo in set(permutations([1, 1, 1, 2, 2])):
... print(list(combo))
...
[1, 2, 1, 1, 2]
[2, 1, 1, 1, 2]
[2, 1, 2, 1, 1]
[2, 1, 1, 2, 1]
[1, 1, 2, 1, 2]
[1, 1, 1, 2, 2]
[1, 2, 1, 2, 1]
[1, 1, 2, 2, 1]
[1, 2, 2, 1, 1]
[2, 2, 1, 1, 1]
>>>
If the combinations need to be in order, then you can use sorted:
>>> for combo in sorted(set(permutations([1, 1, 1, 2, 2]))):
... print(list(combo))
...
[1, 1, 1, 2, 2]
[1, 1, 2, 1, 2]
[1, 1, 2, 2, 1]
[1, 2, 1, 1, 2]
[1, 2, 1, 2, 1]
[1, 2, 2, 1, 1]
[2, 1, 1, 1, 2]
[2, 1, 1, 2, 1]
[2, 1, 2, 1, 1]
[2, 2, 1, 1, 1]
>>>
I know if I want to create a list like this:
[0 1 2 0 1 2 0 1 2 0 1 2]
I can use this command:
range(3) * 4
Is there a similar way to create a list like this:
[0 0 0 0 1 1 1 1 2 2 2 2]
I mean a way without loops
Integer division can help:
[x/4 for x in range(12)]
Same thing through map:
map(lambda x: x/4, range(12))
In python 3 integer division is done with //.
Beware that multiplication of a list will likely lead to a result you probably don't expect.
Yes, you can.
>>> [e for e in range(3) for _ in [0]*4]
[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]
itertools module is always an option:
>>> from itertools import chain, repeat
>>> list(chain(repeat(0, 4), repeat(1, 4), repeat(2, 4)))
[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]
More general way is:
def done(group_count, repeat_count):
return list(chain(*map(lambda i: repeat(i, repeat_count),
range(group_count))))
>>> done(3, 4)
[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]
Without any explicit "for" :)
>>> list(chain(*zip(*([range(5)] * 5))))
[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4]
What about this:
>>> sum([ [x]*4 for x in range(5)],[])
[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4]
>>>
or
>>> reduce(lambda x,y: x+y, [ [x]*4 for x in range(5)])
[0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4]
>>>
If you can't use a loop in your current method, create one in an other?
range(0,1)*4 + range(1,2)*4 + range(2,3)*4