Related
I can't improve the performance of the following Sudoku Solver code. I know there are 3 loops here and they probably cause slow performance but I can't find a better/more efficient way. "board" is mutated with every iteration of recursion - if there are no zeros left, I just need to exit the recursion.
I tried to isolate "board" from mutation but it hasn't changed the performance. I also tried to use list comprehension for the top 2 "for" loops (i.e. only loop through rows and columns with zeros), tried to find coordinates of all zeros, and then use a single loop to go through them - hasn't helped.
I think I'm doing something fundamentally wrong here with recursion - any advice or recommendation on how to make the solution faster?
def box(board,row,column):
start_col = column - (column % 3)
finish_col = start_col + 3
start_row = row - (row % 3)
finish_row = start_row + 3
return [y for x in board[start_row:finish_row] for y in x[start_col:finish_col]]
def possible_values(board,row,column):
values = {1,2,3,4,5,6,7,8,9}
col_values = [v[column] for v in board]
row_values = board[row]
box_values = box(board, row, column)
return (values - set(row_values + col_values + box_values))
def solve(board, i_row = 0, i_col = 0):
for rn in range(i_row,len(board)):
if rn != i_row: i_col = 0
for cn in range(i_col,len(board)):
if board[rn][cn] == 0:
options = possible_values(board, rn, cn)
for board[rn][cn] in options:
if solve(board, rn, cn):
return board
board[rn][cn] = 0
#if no options left for the cell, go to previous cell and try next option
return False
#if no zeros left on the board, problem is solved
return True
problem = [
[9, 0, 0, 0, 8, 0, 0, 0, 1],
[0, 0, 0, 4, 0, 6, 0, 0, 0],
[0, 0, 5, 0, 7, 0, 3, 0, 0],
[0, 6, 0, 0, 0, 0, 0, 4, 0],
[4, 0, 1, 0, 6, 0, 5, 0, 8],
[0, 9, 0, 0, 0, 0, 0, 2, 0],
[0, 0, 7, 0, 3, 0, 2, 0, 0],
[0, 0, 0, 7, 0, 5, 0, 0, 0],
[1, 0, 0, 0, 4, 0, 0, 0, 7]
]
solve(problem)
Three things you can do to speed this up:
Maintain additional state using arrays of integers to keep track of row, col, and box candidates (or equivalently values already used) so that finding possible values is just possible_values = row_candidates[row] & col_candidates[col] & box_candidates[box]. This is a constant factors thing that will change very little in your approach.
As kosciej16 suggested use the min-remaining-values heuristic for selecting which cell to fill next. This will turn your algorithm into crypto-DPLL, giving you early conflict detection (cells with 0 candiates), constraint propagation (cells with 1 candidate), and a lower effective branching factor for the rest.
Add logic to detect hidden singles (like the Norvig solver does). This will make your solver a little slower for the simplest puzzles, but it will make a huge difference for puzzles where hidden singles are important (like 17 clue puzzles).
A result that worked at the end thanks to 53x15 and kosciej16. Not ideal or most optimal but passes the required performance test:
def solve(board, i_row = 0, i_col = 0):
cells_to_solve = [((rn, cn), possible_values(board,rn,cn)) for rn in range(len(board)) for cn in range(len(board)) if board[rn][cn] == 0]
if not cells_to_solve: return True
min_n_of_values = min([len(x[1]) for x in cells_to_solve])
if min_n_of_values == 0: return False
best_cells_to_try = [((rn,cn),options) for ((rn,cn),options) in cells_to_solve if len(options) == min_n_of_values]
for ((rn,cn),options) in best_cells_to_try:
for board[rn][cn] in options:
if solve(board, rn, cn):
return board
board[rn][cn] = 0
return False
I was trying out the backtracking algorithm with an easy example (sudoku). I first tried another approach where more possibilities are canceled, but after I got the same error I switched to an easier solution.
look for the first unsolved spot
fill in every number between 1 and 9 and backtrack the new field if it is valid
When I run it and output the non-valid fields I can see that when the algorithm goes out of a recursion call the spot that was in that recursion call is still a 9 (so the algorithm couldn't find anything for that spot)
e.g. the first two lines look something like this (it's trying to solve an empty field):
[1, 2, 3, 4, 6, 9, 9, 9, 9]
[9, 9, 9, 9, 9, 9, 9, 0, 0]
I thought it was a reference error and inserted
[e for e in field]
in the backtracking call so that the old field doesn't get altered although that didn't seem to help.
Here is my code:
for i in range(9):
a = [field[i][j] for j in range(9) if field[i][j] != 0]
if len(a) != len(set(a)):
return False
for i in range(9):
a = [field[j][i] for j in range(9) if field[j][i] != 0]
if len(a) != len(set(a)):
return False
for x in range(3):
for y in range(3):
a = []
for addX in range(3):
for addY in range(3):
spot = field[x * 3 + addX][y * 3 + addY]
if spot != 0:
a.append(spot)
if len(a) != len(set(a)):
return False
return True
def findEmpty(field):
for i in range(9):
for j in range(9):
if field[i][j] == 0:
return i, j
def backtracking(field):
find = findEmpty(field)
if not find:
return True, field
else:
x, y = find
for i in range(1, 10):
print(f"Trying {i} at {x} {y}")
field[x][y] = i
if isValid(field):
s = backtracking([e for e in field])
if s[0]:
return s
else:
print("Not valid")
for row in field:
print(row)
return False, None
field = [[0, 0, 0, 0, 1, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0]]
solution = backtracking(field)
if solution[0]:
print("There was a solution. The field is:")
for row in solution[1]:
print(row)
else:
print("No solution was found")
Okay, so based on what I can see in the logs, what happens is that when the code gets to 9 and still does not get an answer, it will backtrack, but keeps the value at 9.
So what happens is, every single time the program backtracks, it leaves the value at 9 and then go to the previous value, which might also go to 9, which is invalid as the value we backtracked from is already a 9. This causes a cycle where the program would backtrack straight to the start and make most slots 9, as you can see in your example.
So the solution would be to add a few lines to backtrack() as below. In short, that extra 2 lines checks if the invalid answer is a 9, if it is, it is resetted to a 0 and backtracks to the previous value until it gets a valid answer.
def backtracking(field):
find = findEmpty(field)
if not find:
return True, field
else:
x, y = find
for i in range(1, 10):
print(f"Trying {i} at {x} {y}")
field[x][y] = i
if isValid(field):
s = backtracking(field)
if s[0]:
return s
else:
print("Not valid")
if field[x][y] == 9:
field[x][y] = 0
for row in field:
print(row)
return False, None
Solution it gave:
[2, 3, 4, 5, 1, 6, 7, 8, 9]
[1, 5, 6, 7, 8, 9, 2, 3, 4]
[7, 8, 9, 2, 3, 4, 1, 5, 6]
[3, 1, 2, 4, 5, 7, 6, 9, 8]
[4, 6, 5, 1, 9, 8, 3, 2, 7]
[8, 9, 7, 3, 6, 2, 4, 1, 5]
[5, 2, 8, 6, 4, 1, 9, 7, 3]
[6, 7, 3, 9, 2, 5, 8, 4, 1]
[9, 4, 1, 8, 7, 3, 5, 6, 2]
I did some research and apparently it really is a reference error. For me importing pythons copy library and assigning each new field saying
f = copy.deepcopy(field)
fixes the issue (this also works for the complex example).
1- Create rows, cols, and boxes (3x3 units) array of dictionaries to store which indices of each row, col, and boxes have numbers.
2- Take a screenshot of the board. Run a for-loop and mark which points include numbers.
3- Call the recursive backtrack function.
4- Always in recursive functions define the base case to exit out of the recursion.
5- Start a for-loop to see which coordinate is ".". If you see ".", apply steps:[6,7,8,9]
6- Now we need to insert a valid number here. A valid number is a number that does not exist in the current row, col, and box.
7- If you find a valid number, insert it into the board and update rows, cols, and boxes.
8- After we inserted the valid point, we call backtrack function for the next ".".
9- When calling the backtrack, decide at which point you are. If you are in the last column, your next backtrack function will start from the next row and column 0. But if you are in the middle of row, you just increase the column parameter for next backtrack function.
10- If in step 5 your value is not ".", that means there is already a valid number here so call next backtracking depending on where position is. If you are in the last column, your next backtrack function will start from the next row and column 0. But if you are in the middle of row, you just increase the column parameter for next backtrack function.
class Solution:
def solveSudoku(self, board: List[List[str]]) -> None:
"""
Do not return anything, modify board in-place instead.
"""
n=len(board)
# create state variables,keep track of rows, cols and boxes
rows=[{} for _ in range(n)]
cols=[{} for _ in range(n)]
boxes=[{} for _ in range(n)]
# get the initial state of the grid
for r in range(n):
for c in range(n):
if board[r][c]!='.':
val=board[r][c]
box_id=self.get_box_id(r,c)
boxes[box_id][val]=True
rows[r][val]=True
cols[c][val]=True
# this backtracking just tells if shoul move to the next cell or not
self.backtrack(board,boxes,rows,cols,0,0)
def backtrack(self,board,boxes,rows,cols,r,c):
# base case. If I hit the last row or col, means all digits were correct so far
if r>=len(board) or c>=len(board[0]):
return True
# situation when cell is empty, fill it with correct value
if board[r][c]==".":
for num in range(1,10):
box_id=self.get_box_id(r,c)
box=boxes[box_id]
row=rows[r]
col=cols[c]
str_num=str(num)
# check rows, cols and boxes make sure str_num is not used before
if self.is_valid(box,col,row,str_num):
board[r][c]=str_num
boxes[box_id][str_num]=True
cols[c][str_num]=True
rows[r][str_num]=True
# if I am the last col and I placed the correct val, move to next row. So first col of the next row
if c==len(board)-1:
if self.backtrack(board,boxes,rows,cols,r+1,0):
return True
# if I am in col between 0-8, move to the col+1, in the same row
else:
if self.backtrack(board,boxes,rows,cols,r,c+1):
return True
# If I got a wrong value, then backtrack. So clear the state that you mutated
del box[str_num]
del row[str_num]
del col[str_num]
board[r][c]="."
# if cell is not empty just call the next backtracking
else:
if c==len(board)-1:
if self.backtrack(board,boxes,rows,cols,r+1,0):
return True
else:
if self.backtrack(board,boxes,rows,cols,r,c+1):
return True
return False
def is_valid(self,box,row,col,num):
if num in box or num in row or num in col:
return False
else:
return True
# a helper to get the id of the 3x3 sub grid, given row and column
def get_box_id(self,r,c):
row=(r//3)*3
col=c//3
return row+col
I have a series of functions that end up giving a list, with the first item containing a number, derived from the dictionaries, and the second and third items are dictionaries.
These dictionaries have been previously randomly generated.
The function I am using generates a given number of these dictionaries, trying to get the highest number possible as the first item. (It's designed to optimise dice rolls).
This all works fine, and I can print the value of the highest first item from all iterations. However, when I try and print the two dictionaries associated with this first number (bearing in mind they're all in a list together), it just seemingly randomly generates the two other dictionaries.
def repeat(type, times):
best = 0
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
print("The highest average success is", best)
return best
This works great. The last thing shown is:
BEST: (3.58, [{'strength': 4, 'intelligence': 1, 'charisma': 1, 'stamina': 4, 'willpower': 2, 'dexterity': 2, 'wits': 5, 'luck': 2}, {'agility': 1, 'brawl': 2, 'investigation': 3, 'larceny': 0, 'melee': 1, 'survival': 0, 'alchemy': 3, 'archery': 0, 'crafting': 0, 'drive': 1, 'magic': 0, 'medicine': 0, 'commercial': 0, 'esteem': 5, 'instruction': 2, 'intimidation': 2, 'persuasion': 0, 'seduction': 0}])
The highest average success is 3.58
But if I try something to store the list which gave this number:
def repeat(type, times):
best = 0
bestChar = []
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
bestChar = x
print("The highest average success is", best)
print("Therefore the best character is", bestChar)
return best, bestChar
I get this as the last result, which is fine:
BEST: (4.15, [{'strength': 2, 'intelligence': 3, 'charisma': 4, 'stamina': 4, 'willpower': 1, 'dexterity': 2, 'wits': 4, 'luck': 1}, {'agility': 1, 'brawl': 0, 'investigation': 5, 'larceny': 0, 'melee': 0, 'survival': 0, 'alchemy': 7, 'archery': 0, 'crafting': 0, 'drive': 0, 'magic': 0, 'medicine': 0, 'commercial': 1, 'esteem': 0, 'instruction': 3, 'intimidation': 0, 'persuasion': 0, 'seduction': 0}])
The highest average success is 4.15
but the last line is
Therefore the best character is (4.15, [{'strength': 1, 'intelligence': 3, 'charisma': 4, 'stamina': 4, 'willpower': 1, 'dexterity': 2, 'wits': 2, 'luck': 3}, {'agility': 1, 'brawl': 0, 'investigation': 1, 'larceny': 4, 'melee': 2, 'survival': 0, 'alchemy': 2, 'archery': 4, 'crafting': 0, 'drive': 0, 'magic': 0, 'medicine': 0, 'commercial': 1, 'esteem': 0, 'instruction': 0, 'intimidation': 2, 'persuasion': 1, 'seduction': 0}])
As you can see this doesn't match with what I want, and what is printed literally right above it.
Through a little bit of checking, I realised what it gives out as the "Best Character" is just the last one generated, which is not the best, just the most recent. However, it isn't that simple, because the first element IS the highest result that was recorded, just not from the character in the rest of the list. This is really confusing because it means the list is somehow being edited but at no point can I see where that would happen.
Am I doing something stupid whereby the character is randomly generated every time? I wouldn't think so since x[0] gives the correct result and is stored fine, so what changes when it's the whole list?
From the function rollForCharacter() it returns rollResult, character which is just the number and then the two dictionaries.
I would greatly appreciate it if anyone could figure out and explain where I'm going wrong and why it can print the correct answer to the console yet not store it correctly a line below!
EDIT:
Dictionary 1 Code:
attributes = {}
def assignRow(row, p): # p is the number of points you have to assign to each row
rowValues = {}
for i in range(0, len(row)-1):
val = randint(0, p)
rowValues[row[i]] = val + 1
p -= val
rowValues[row[-1]] = p + 1
return attributes.update(rowValues)
def getPoints():
points = [7, 5, 3]
shuffle(points)
row1 = ['strength', 'intelligence', 'charisma']
row2 = ['stamina', 'willpower']
row3 = ['dexterity', 'wits', 'luck']
for i in range(0, len(points)):
row = eval("row" + str(i+1))
assignRow(row, points[i])
Dictionary 2 Code:
skills = {}
def assignRow(row, p): # p is the number of points you have to assign to each row
rowValues = {}
for i in range(0, len(row) - 1):
val = randint(0, p)
rowValues[row[i]] = val
p -= val
rowValues[row[-1]] = p
return skills.update(rowValues)
def getPoints():
points = [11, 7, 4]
shuffle(points)
row1 = ['agility', 'brawl', 'investigation', 'larceny', 'melee', 'survival']
row2 = ['alchemy', 'archery', 'crafting', 'drive', 'magic', 'medicine']
row3 = ['commercial', 'esteem', 'instruction', 'intimidation', 'persuasion', 'seduction']
for i in range(0, len(points)):
row = eval("row" + str(i + 1))
assignRow(row, points[i])
It does look like the dictionary is being re-generated, which could easily happen if the function rollForCharacter returns either a generator or alternatively is overwriting a global variable which is being overwritten by a subsequent cycle of the loop.
A simple-but-hacky way to solve the problem would be to take a deep copy of the dictionary at the time of storing, so that you're sure you're keeping the values at that point:
def repeat(type, times):
best = 0
bestChar = []
for i in range(0, times):
x = rollForCharacter(type)
if x[0] > best:
print("BEST:", x)
best = x[0]
# Create a brand new tuple, containing a copy of the current dict
bestChar = (x[0], x[1].copy())
The correct answer would be however to pass a unique dictionary variable that is not affected by later code.
See this SO answer with a bit more context about how passing a reference to a dictionary can be risky as it's still a mutable object.
I have a list which consists of 0's and 1's. The list should ideally look like this 0,1,0,1,0,1,0,1,0,1,0,1,0,1.....
But due to some error in logging, my list looks like this: 0,1,0,1,1,1,0,1,0,0,0,1,0,1.... As one can clearly there are some missed 0's and 1's in middle. How can I fix this list to add those 0's and 1's in between the missing elements so as to get to the desired list values.
Here is the code used by me, this does the task for me but it is not the most pythonic way of writing scripts. So how can I improve on this script?
l1 = [0,1,0,1,1,1,0,1,0,0,0,1,0,1]
indices = []
for i in range(1,len(l1)):
if l1[i]!=l1[i-1]:
continue
else:
if l1[i]==0:
val=1
else:
val=0
l1.insert(i, val)
EDIT
As asked in the comments, Let me explain why is this important rather than generating 1's and 0's. I have TTL pulse coming i.e. a series of HIGH(1) and LOW(0) coming in and simultaneously time for each of these TTL pulse is logged on 2 machines with different clocks.
Now while machine I is extremely stable and logging each sequence of HIGH(1) and low(1) accurately, the other machine ends up missing a couple of them and as a result I don't have time information for those.
All I wanted was to merge the missing TTL pulse on one machine wrt to the other machine. This will now allow me to align time on both of them or log None for not received pulse.
Reason for doing this rather than correcting the logging thing (as asked in comments) is that this is an old collected data. We have now fixed the logging issue.
You can try something like this:
from itertools import chain
l1 = [0,1,0,1,1,1,0,1,0,0,0,1,0,1]
c = max(l1.count(0), l1.count(1))
print list(chain(*zip([0]*c,[1]*c)))
Output:
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1]
why would you have a list of 0,1,0,1,0,1? there is no good reason i can think of. oh well thats beyond the scope of this question i guess...
list(itertools.islice(itertools.cycle([0,1]),expected_length))
Just multiply a new list.
>>> l1 = [0,1,0,1,1,1,0,1,0,0,0,1,0,1]
>>> l1
[0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1]
>>> [0,1] * (len(l1)//2)
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1]
If the list has an odd number of elements, add the necessary 0:
>>> l2 = [0,1,0,1,1,1,0,1,0,0,0,1,0,1,0]
>>> l2_ = [0,1] * (len(l1)//2)
>>> if len(l2)%2: l2_.append(0)
...
>>> l2
[0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0]
>>> l2_
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0]
Given a list of data as follows:
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
I would like to create an algorithm that is able to offset the list of certain number of steps. For example, if the offset = -1:
def offsetFunc(inputList, offsetList):
#make something
return output
where:
output = [0,0,0,0,1,1,5,5,5,5,5,5,3,3,3,2,2]
Important Note: The elements of the list are float numbers and they are not in any progression. So I actually need to shift them, I cannot use any work-around for getting the result.
So basically, the algorithm should replace the first set of values (the 4 "1", basically) with the 0 and then it should:
Detect the lenght of the next range of values
Create a parallel output vectors with the values delayed by one set
The way I have roughly described the algorithm above is how I would do it. However I'm a newbie to Python (and even beginner in general programming) and I have figured out time by time that Python has a lot of built-in functions that could make the algorithm less heavy and iterating. Does anyone have any suggestion to better develop a script to make this kind of job? This is the code I have written so far (assuming a static offset at -1):
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
output = []
PrevVal = 0
NextVal = input[0]
i = 0
while input[i] == NextVal:
output.append(PrevVal)
i += 1
while i < len(input):
PrevVal = NextVal
NextVal = input[i]
while input[i] == NextVal:
output.append(PrevVal)
i += 1
if i >= len(input):
break
print output
Thanks in advance for any help!
BETTER DESCRIPTION
My list will always be composed of "sets" of values. They are usually float numbers, and they take values such as this short example below:
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
In this example, the first set (the one with value "1.236") is long 4 while the second one is long 6. What I would like to get as an output, when the offset = -1, is:
The value "0.000" in the first 4 elements;
The value "1.236" in the second 6 elements.
So basically, this "offset" function is creating the list with the same "structure" (ranges of lengths) but with the values delayed by "offset" times.
I hope it's clear now, unfortunately the problem itself is still a bit silly to me (plus I don't even speak good English :) )
Please don't hesitate to ask any additional info to complete the question and make it clearer.
How about this:
def generateOutput(input, value=0, offset=-1):
values = []
for i in range(len(input)):
if i < 1 or input[i] == input[i-1]:
yield value
else: # value change in input detected
values.append(input[i-1])
if len(values) >= -offset:
value = values.pop(0)
yield value
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
print list(generateOutput(input))
It will print this:
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
And in case you just want to iterate, you do not even need to build the list. Just use for i in generateOutput(input): … then.
For other offsets, use this:
print list(generateOutput(input, 0, -2))
prints:
[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 5, 5, 5, 3, 3]
Using deque as the queue, and using maxlen to define the shift length. Only holding unique values. pushing inn new values at the end, pushes out old values at the start of the queue, when the shift length has been reached.
from collections import deque
def shift(it, shift=1):
q = deque(maxlen=shift+1)
q.append(0)
for i in it:
if q[-1] != i:
q.append(i)
yield q[0]
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
print list(shift(Sample))
#[0, 0, 0, 0, 1.236, 1.236, 1.236, 1.236, 1.236, 1.236]
My try:
#Input
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
shift = -1
#Build service structures: for each 'set of data' store its length and its value
set_lengths = []
set_values = []
prev_value = None
set_length = 0
for value in input:
if prev_value is not None and value != prev_value:
set_lengths.append(set_length)
set_values.append(prev_value)
set_length = 0
set_length += 1
prev_value = value
else:
set_lengths.append(set_length)
set_values.append(prev_value)
#Output the result, shifting the values
output = []
for i, l in enumerate(set_lengths):
j = i + shift
if j < 0:
output += [0] * l
else:
output += [set_values[j]] * l
print input
print output
gives:
[1, 1, 1, 1, 5, 5, 3, 3, 3, 3, 3, 3, 2, 2, 2, 5, 5]
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
def x(list, offset):
return [el + offset for el in list]
A completely different approach than my first answer is this:
import itertools
First analyze the input:
values, amounts = zip(*((n, len(list(g))) for n, g in itertools.groupby(input)))
We now have (1, 5, 3, 2, 5) and (4, 2, 6, 3, 2). Now apply the offset:
values = (0,) * (-offset) + values # nevermind that it is longer now.
And synthesize it again:
output = sum([ [v] * a for v, a in zip(values, amounts) ], [])
This is way more elegant, way less understandable and probably way more expensive than my other answer, but I didn't want to hide it from you.