Develop Reference

By changing values in one array it automatically changes values in another array. How to avoid that happening?

I have created a genetic algorithm to create children from parents.
At the start of the algorithm a random workload(arrays of sub-arrays) is created.
Workload L=2, population size N=30, InputsNumber=3 and mutation rate m=0.05. Then I do some score calculations for the population to pick the 2 workloads(the parents) with highest score. Now the new population are the parents only. After that form the 2 parents I create children with crossover function and mutation function and I add the children to the population with the parents. Now I do the same thing for 10 times and every time I pick the 2 best to be the parents from the population. Now the problem is that when I change the children values in mutation function, all of a sudden the parents change their values to children values. How to avoid that? The parents are correct BEFORE I call the mutation() function and AFTER the mutation function they change. I can't understand why this is happening. Please help me find why this is happening!
HERE IS AN EXAMPLE OUTPUT: https://ibb.co/m5T0hSq
Parent or child array example: [[0 0 0],[0 0 0]]
Parents or children array example: [ [[0 0 0],[0 0 0]], [[0 0 0],[0 0 0]] ]
def generateRandomWorkload(inputsNumber, L, N):
global population
individualWorkload = []
for n in range(N):
for i in range(L):
individual = [0 for _ in range(len(inputsNumber))]
individualWorkload.append(individual)
population.append(individualWorkload)
individualWorkload = []
def crossover(L):
global parents, children
children = []
for i in range(2):
C = random.randint(0, 1)
R = random.randint(0, L)
if C == 0:
child = parents[0][0:R] + parents[1][R:L]
children.append(child)
elif C == 1:
child = parents[1][0:R] + parents[0][R:L]
children.append(child)
return children
def mutation(mutation_rate):
global children
for i in range(len(children)):
for j in range(len(children[i])):
for k in range(len(children[i][j])):
r = random.uniform(0, 1)
if r <= mutation_rate:
children[i][j][k] = 1 - children[i][j][k]
return children
def geneticAlgorithm(inputsNumber, L, N):
global parents, children, population
generateRandomWorkload(inputsNumber, L, N)
print("SEED POPULATION: ", population, "\n \n")
for generation in range(10):
print(bcolors.OKGREEN + "MEASUREMENTS OF ", generation+1, " GENERATION" + bcolors.ENDC)
for individualWorkload in population:
### HERE I CALCULATE SOME SCORES (scoreI) ###
# Parents
print("PARENTS SELECTION... \n")
scoreI.sort(key=lambda x: x[1])
parents = [scoreI[-1][0], scoreI[-2][0]]
population = [parents[0], parents[1]]
print("SELECTED PARENTS: \n", parents, "\n")
print("PARENTS IN POPULATION:", population)
# Crossover
print("BEGIN CROSSOVER... \n")
print("PARENTS: ", parents)
children = crossover(L)
print("CROSSOVER CHILDREN:\n", children, "\n")
# Mutation
print("BEGIN MUTATION...\n")
print("PARENTS: ", parents)
children = mutation(0.05)
print("MUTATION CHILDREN:\n", children, "\n")
# New population
population.append(children[0])
population.append(children[1])
print("PARENTS: ", parents)
print("NEW POPULATION: \n", population, "\n")

If I understand correctly, this is due to something in python called mutability.
I haven't actually researched it, however I believe it stems from C pointers, as python is written in C and has very good integration with C code (look up CPython for example).
Basically, what it means is that when you say:
listB = listA
listA and listB are actually the same list, because they aren't actually a list,
but a number that points to a particular memory address.
You can get around this in many different ways
Deepcopy
from copy import deepcopy
listB = deepcopy(listA)
Array Slicing
listB = listA[::]
Iterate with a for loop (I wouldn't recommend it, but here it is)
listB = []
for element in listA:
listB.append(element)
You can also do a quick google search to find out more, these are just some options I can think of right now.

Perhaps confusion of copy by address and by values ?
Don't do direct affectation when working with arrays because they are linked.
"Using the copy() function is another way of copying an array in Python. In this case, a new array object is created from the original array and this type of copy is called deep copy. If any value is modified in the original or copied array, then it does not create any change on another array."

When copying an array to another array, simply by only using the equal sign it will not copy the values, but it copies the position this values are positioned therefore making the appearance of the array being copied but if tried to change the array, it will change both.
To avoid it happening go over every element in the array and copy the value individually

Python list first n entries in a custom base number system

I am sorry if the title is a misnomer and/or doesn't properly describe what this is all about, you are welcome to edit the title to make it clear once you understand what this is about.
The thing is very simple, but I find it hard to describe, this thing is sorta like a number system, except it is about lists of integers.
So we start with a list of integers with only zero, foreach iteration we add one to it, until a certain limit is reached, then we insert 1 at the start of the list, and set the second element to 0, then iterate over the second element until the limit is reached again, then we add 1 to the first element and set the second element 0, and when the first element reaches the limit, insert another element with value of 1 to the start of the list, and zero the two elements after it, et cetera.
And just like this, when a place reaches limit, zero the place and the places after it, increase the place before it by one, and when all available places reach limit, add 1 to the left, for example:
0
1
2
1, 0
1, 1
1, 2
2, 0
2, 1
2, 2
1, 0, 0
The limit doesn't have to be three.
This is what I currently have that does something similar to this:
array = []
for c in range(26):
for b in range(26):
for a in range(26):
array.append((c, b, a))
I don't want leading zeroes but I can remove them, but I can't figure out how to do this with a variable number of elements.
What I want is a function that takes two arguments, limit (or base) and number of tuples to be returned, and returns the first n such tuples in order.
This must be very simple, but I just can't figure it out, and Google returns completely irrelevant results, so I am asking for help here.
How can this be done? Any help will truly be appreciated!
Hmm, I was thinking about something like this, but very unfortunately I can't make it work, please help me figure out why it doesn't work and how to make it work:
array = []
numbers = [0]
for i in range(1000):
numbers[-1] += 1
while 26 in numbers:
index = numbers.index(26)
numbers[index:] = [0] * (len(numbers) - index)
if index != 0:
numbers[index - 1] += 1
else:
numbers.insert(0, 1)
array.append(numbers)
I don't quite understand it, my testing shows everything inside the loop work perfectly fine outside the loop, the results are correct, but it just simply magically will not work in a loop, I don't know the reason for this, it is very strange.
I discovered the fact that if I change the last line to print(numbers) then everything prints correctly, but if I use append only the last element will be added, how so?

from math import log
def number_to_base(n,base):
number=[]
for digit in range(int(log(n+0.500001,base)),-1,-1):
number.append(n//base**digit%base)
return number
def first_numbers_in_base(n,base):
numbers=[]
for i in range(n):
numbers.append(tuple(number_to_base(i,base)))
return numbers
#tests:
print(first_numbers_in_base(10,3))
print(number_to_base(1048,10))
print(number_to_base(int("10201122110212",3),3))
print(first_numbers_in_base(25,10))

I finally did it!
The logic is very simple, but the hard part is to figure out why it won't work in a loop, turns out I need to use .copy(), because for whatever reason, doing an in-place modification to a list directly modifies the data reside in its memory space, such behavior modifies the same memory space, and .append() method always appends the latest data in a memory space.
So here is the code:
def steps(base, num):
array = []
numbers = [0]
for i in range(num):
copy = numbers.copy()
copy[-1] += 1
while base in copy:
index = copy.index(base)
copy[index:] = [0] * (len(copy) - index)
if index != 0:
copy[index - 1] += 1
else:
copy.insert(0, 1)
array.append(copy)
numbers = copy
return array
Use it like this:
steps(26, 1000)
For the first 1000 lists in base 26.

Here is a a function, that will satisfy original requirements (returns list of tuples, first tuple represents 0) and is faster than other functions that have been posted to this thread:
def first_numbers_in_base(n,base):
if n<2:
if n:
return [(0,)]
return []
numbers=[(0,),(1,)]
base-=1
l=-1
num=[1]
for i in range(n-2):
if num[-1]==base:
num[-1]=0
for i in range(l,-1,-1):
if num[i]==base:
num[i]=0
else:
num[i]+=1
break
else:
num=[1]+num
l+=1
else:
num[-1]+=1
numbers.append(tuple(num))#replace tuple(num) with num.copy() if you want resutl to contain lists instead of tuples.
return numbers

Efficient algorithm to randomly find available places inside a list in Python

I need to randomly assign a place inside a list to an input. I need to check whether it is not occupied first and then use it. The best algorithm that I can come up with is the following:
def get_random_addr(input_arr):
while True:
addr = random.randrange(1, len(input_arr))
if input_arr[addr] is None:
break
return addr
This is obviously not efficient since as we occupy more slots, the loop takes longer to find an empty slot, and even it may take forever (suppose only one empty slot is left). Do you have any better solutions?
How I did it
Based on the chosen answer, this is how I ended up doing it. It is very fast and efficient compared to the solutions which search through the whole list and find the None elements and randomly choose from the retrieved set. I think the bottleneck was random.choice method which seems to be very slow.
# Create a list of indexes at the beginning when all the values are None
available_index = list(range(1, len(input_arr)))
random.shuffle(available_index)
# To get a random index simply pop from shuffled available index
random_index = available_index.pop()
While this method has extra O(n) memory complexity, in practice it is very efficient and fast.

If you can't use numpy I'd keep a set of indexes which are known to contain None. Every time None is added or removed this set of indexes will be updated

Your function can take arbitrarly long to return. In particular, you will get into an infinite loop if no item is None.
Instead, recover all indices which are None and use random.choices to randomly return k of them.
import random
def get_random_addr(input_arr, k=1, target=None):
return random.choices([i for i, v in enumerate(input_arr) if v is target], k=k)
Usage
l = [0, None, 2, 3, None, None]
for i in get_random_addr(l, k=2):
l[i] = i
print(l) # [0, None, 2, 3, 4, 5]

Similar to DeepSpace's idea, except with O(1) memory and O(n) time, but faster by a constant factor since it only iterates over half the slots in the array.
Keep track of the number of empty slots.
Iterate through the list.
If a slot is empty, return your new value with probability 1/number_empty_slots
If we did not return and the slot is empty, redistribute probability mass over other empty slots
Code:
def get_random_addr(input_arr, num_empty_slots):
# num_empty_slots contains the number of empty slots in input_arr
for index, elem in enumerate(arr):
if elem is None:
if random.random() < 1 / num_empty_slots:
return index
num_empty_slots -= 1

Simply use enumerate to index your list first, filter out those that are None, and then use random.choice to pick an available space.
from random import choice
def get_random_addr(input_arr):
return choice([index for index, value in enumerate(input_arr) if value is None])
print(get_random_addr([None, 1, None, 2]))
This outputs either 0 or 2 randomly, or None if there is no more space available.

In my approach, I pick an arbitrary address in the target array, and if it is free I add it to the output list, but if it is not, I map that address to an address which does contain None, nearest to the end of the list. All entries in the array beyond and including this mapped free address are dropped from this list, since they are either nonempty, or already are represented elsewhere in the list. I repeat this process, chopping away at the size of the target list, making it easier and easier to find new empty addresses as it proceeds. There's a few other minor details to make it all work, but I think the code below can explain those better than I can with words.
from random import random
def randint(max_val):
return int(random() * max_val)
def assign(values, target):
output = []
mapping = dict()
mmax = 0
size = len(target)
for val in values:
idx = randint(size)
while target[idx] != None:
if idx in mapping:
idx = mapping.pop(idx)
mmax = max(mapping or [0])
break
min_size = max(idx, mmax)
try:
size -= target[size-1:min_size:-1].index(None)
except:
size = min_size + 1
if target[size-1] == None:
size -= 1
mapping[idx] = size
if idx > mmax:
mmax = idx
elif size-1 in mapping:
size -= 1
mapping[idx] = mapping.pop(size)
mmax = max(mapping or [0])
idx = randint(size)
target[idx] = val
output.append(idx)
return output
Note that this modifies the target list passed to it. If you don't want to modify it, you really have two options: implement a bit of extra logic to check if the "free" address is already consumed, or copy the entire list (in which case, reverse it and patch the indices, so that the .index() can work on the list directly, which is the major time sink anyways.
I'd also recommend verifying that the solutions it produces are valid. I've done some testing on my part, but I very well could have missed something.

Recursive Selection sort to return a descending list in python

I'd been working through a problem, to sort the elements in a descending order through a recursive approach, the code is as follows..
import operator
def do_stuff(elem_list):
if not elem_list:
return None
max_index , max_element = max(enumerate(elem_list) , key = operator.itemgetter(1))
elem_list[max_index],elem_list[0] = elem_list[0],elem_list[max_index]
return do_stuff(elem_list[1:])
p_list = [4,2,3,5,1]
do_stuff(p_list)
print(p_list)
Output -
[5, 2, 3, 4, 1]
And I can't seem to figure wherein lies the problem and why won't I get the desired output ?

I was able to fix your problem by adding an extra parameter, since you seem to be using a recursive implementation of insertion sort, you need some way to track the next open place to swap values in the list.
import operator
def sortl(l, last):
# base case
if last + 1 >= len(l):
return l
# find the max index (mi) and max value in the list
mi, _ = max(enumerate(l[last:]), key = operator.itemgetter(1))
mi += last # caculate the offset in the sublist
l[mi], l[last] = l[last], l[mi] # swap the values
# recursive call
return sortl(l, last + 1)
By using "last + 1" every time, you are able to simulate using the underlying sublist since calling do_stuff([some_list[1:]) wont work

Python's slices are not true references to the underlying list. See: Can I create a "view" on a Python list?

how to make a list of functions execute in python?

I am trying to save the results of 10 function calls into a list. however, when I want to access this numerical data, I keep getting type errors, because the list is really a list of function calls. How do I make it do stuff so I can have numbers to do compare?
I tried setting temp variable to result of each spot in the list, but it still shows as a function.
Output should show the average for different # of dice, and return the best amount of dice to roll
def calculator(fn, num_samples=1000): # this is the function I'm trying to call
def print_and_return(*args):
total3 = 0
for _ in range(num_samples):
total3 += (fn(*args))
return float(total3)/num_samples
return print_and_return
def find_average(dice=six_sided):
avglist = []
k, spot = 0, 0
bob = roll_dice(k+1, dice)
passed_function = calculator(bob, 1000)
print(passed_function)
while k <= 10:
avglist.append((passed_function)) # <==trying to save the results when I check with 1-10 dice rolls
if k == 0:
spot = 1
else:
temp = 0
max = 0
i = 0
while i <= len(avglist):
temp = avglist[i]
if max > temp:
max = temp
i +=1
if (avglist[k] > temp):
spot = k
print(k, "dice scores", avglist[k], "on average")
k +=1
return spot

You are not calling passed_function. Calling a function with no arguments still reqires the ().

Your calculator function returns a function, not a plain value. You therefore need to call passed_function in order to get your result.
Based on the arguments to your call to roll_dice, it looks like you want the k argument to vary in the loop. As written now, it will not. I suspect you are tripping yourself up a bit. The code could be written a whole lot simpler without so many function references being passed around.