Develop Reference

How can I improve performance of Sudoku solver?

I can't improve the performance of the following Sudoku Solver code. I know there are 3 loops here and they probably cause slow performance but I can't find a better/more efficient way. "board" is mutated with every iteration of recursion - if there are no zeros left, I just need to exit the recursion.
I tried to isolate "board" from mutation but it hasn't changed the performance. I also tried to use list comprehension for the top 2 "for" loops (i.e. only loop through rows and columns with zeros), tried to find coordinates of all zeros, and then use a single loop to go through them - hasn't helped.
I think I'm doing something fundamentally wrong here with recursion - any advice or recommendation on how to make the solution faster?
def box(board,row,column):
start_col = column - (column % 3)
finish_col = start_col + 3
start_row = row - (row % 3)
finish_row = start_row + 3
return [y for x in board[start_row:finish_row] for y in x[start_col:finish_col]]
def possible_values(board,row,column):
values = {1,2,3,4,5,6,7,8,9}
col_values = [v[column] for v in board]
row_values = board[row]
box_values = box(board, row, column)
return (values - set(row_values + col_values + box_values))
def solve(board, i_row = 0, i_col = 0):
for rn in range(i_row,len(board)):
if rn != i_row: i_col = 0
for cn in range(i_col,len(board)):
if board[rn][cn] == 0:
options = possible_values(board, rn, cn)
for board[rn][cn] in options:
if solve(board, rn, cn):
return board
board[rn][cn] = 0
#if no options left for the cell, go to previous cell and try next option
return False
#if no zeros left on the board, problem is solved
return True
problem = [
[9, 0, 0, 0, 8, 0, 0, 0, 1],
[0, 0, 0, 4, 0, 6, 0, 0, 0],
[0, 0, 5, 0, 7, 0, 3, 0, 0],
[0, 6, 0, 0, 0, 0, 0, 4, 0],
[4, 0, 1, 0, 6, 0, 5, 0, 8],
[0, 9, 0, 0, 0, 0, 0, 2, 0],
[0, 0, 7, 0, 3, 0, 2, 0, 0],
[0, 0, 0, 7, 0, 5, 0, 0, 0],
[1, 0, 0, 0, 4, 0, 0, 0, 7]
]
solve(problem)

Three things you can do to speed this up:
Maintain additional state using arrays of integers to keep track of row, col, and box candidates (or equivalently values already used) so that finding possible values is just possible_values = row_candidates[row] & col_candidates[col] & box_candidates[box]. This is a constant factors thing that will change very little in your approach.
As kosciej16 suggested use the min-remaining-values heuristic for selecting which cell to fill next. This will turn your algorithm into crypto-DPLL, giving you early conflict detection (cells with 0 candiates), constraint propagation (cells with 1 candidate), and a lower effective branching factor for the rest.
Add logic to detect hidden singles (like the Norvig solver does). This will make your solver a little slower for the simplest puzzles, but it will make a huge difference for puzzles where hidden singles are important (like 17 clue puzzles).

A result that worked at the end thanks to 53x15 and kosciej16. Not ideal or most optimal but passes the required performance test:
def solve(board, i_row = 0, i_col = 0):
cells_to_solve = [((rn, cn), possible_values(board,rn,cn)) for rn in range(len(board)) for cn in range(len(board)) if board[rn][cn] == 0]
if not cells_to_solve: return True
min_n_of_values = min([len(x[1]) for x in cells_to_solve])
if min_n_of_values == 0: return False
best_cells_to_try = [((rn,cn),options) for ((rn,cn),options) in cells_to_solve if len(options) == min_n_of_values]
for ((rn,cn),options) in best_cells_to_try:
for board[rn][cn] in options:
if solve(board, rn, cn):
return board
board[rn][cn] = 0
return False

Python sudoku backtracking

I was trying out the backtracking algorithm with an easy example (sudoku). I first tried another approach where more possibilities are canceled, but after I got the same error I switched to an easier solution.
look for the first unsolved spot
fill in every number between 1 and 9 and backtrack the new field if it is valid
When I run it and output the non-valid fields I can see that when the algorithm goes out of a recursion call the spot that was in that recursion call is still a 9 (so the algorithm couldn't find anything for that spot)
e.g. the first two lines look something like this (it's trying to solve an empty field):
[1, 2, 3, 4, 6, 9, 9, 9, 9]
[9, 9, 9, 9, 9, 9, 9, 0, 0]
I thought it was a reference error and inserted
[e for e in field]
in the backtracking call so that the old field doesn't get altered although that didn't seem to help.
Here is my code:
for i in range(9):
a = [field[i][j] for j in range(9) if field[i][j] != 0]
if len(a) != len(set(a)):
return False
for i in range(9):
a = [field[j][i] for j in range(9) if field[j][i] != 0]
if len(a) != len(set(a)):
return False
for x in range(3):
for y in range(3):
a = []
for addX in range(3):
for addY in range(3):
spot = field[x * 3 + addX][y * 3 + addY]
if spot != 0:
a.append(spot)
if len(a) != len(set(a)):
return False
return True
def findEmpty(field):
for i in range(9):
for j in range(9):
if field[i][j] == 0:
return i, j
def backtracking(field):
find = findEmpty(field)
if not find:
return True, field
else:
x, y = find
for i in range(1, 10):
print(f"Trying {i} at {x} {y}")
field[x][y] = i
if isValid(field):
s = backtracking([e for e in field])
if s[0]:
return s
else:
print("Not valid")
for row in field:
print(row)
return False, None
field = [[0, 0, 0, 0, 1, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0]]
solution = backtracking(field)
if solution[0]:
print("There was a solution. The field is:")
for row in solution[1]:
print(row)
else:
print("No solution was found")

Okay, so based on what I can see in the logs, what happens is that when the code gets to 9 and still does not get an answer, it will backtrack, but keeps the value at 9.
So what happens is, every single time the program backtracks, it leaves the value at 9 and then go to the previous value, which might also go to 9, which is invalid as the value we backtracked from is already a 9. This causes a cycle where the program would backtrack straight to the start and make most slots 9, as you can see in your example.
So the solution would be to add a few lines to backtrack() as below. In short, that extra 2 lines checks if the invalid answer is a 9, if it is, it is resetted to a 0 and backtracks to the previous value until it gets a valid answer.
def backtracking(field):
find = findEmpty(field)
if not find:
return True, field
else:
x, y = find
for i in range(1, 10):
print(f"Trying {i} at {x} {y}")
field[x][y] = i
if isValid(field):
s = backtracking(field)
if s[0]:
return s
else:
print("Not valid")
if field[x][y] == 9:
field[x][y] = 0
for row in field:
print(row)
return False, None
Solution it gave:
[2, 3, 4, 5, 1, 6, 7, 8, 9]
[1, 5, 6, 7, 8, 9, 2, 3, 4]
[7, 8, 9, 2, 3, 4, 1, 5, 6]
[3, 1, 2, 4, 5, 7, 6, 9, 8]
[4, 6, 5, 1, 9, 8, 3, 2, 7]
[8, 9, 7, 3, 6, 2, 4, 1, 5]
[5, 2, 8, 6, 4, 1, 9, 7, 3]
[6, 7, 3, 9, 2, 5, 8, 4, 1]
[9, 4, 1, 8, 7, 3, 5, 6, 2]

I did some research and apparently it really is a reference error. For me importing pythons copy library and assigning each new field saying
f = copy.deepcopy(field)
fixes the issue (this also works for the complex example).

1- Create rows, cols, and boxes (3x3 units) array of dictionaries to store which indices of each row, col, and boxes have numbers.
2- Take a screenshot of the board. Run a for-loop and mark which points include numbers.
3- Call the recursive backtrack function.
4- Always in recursive functions define the base case to exit out of the recursion.
5- Start a for-loop to see which coordinate is ".". If you see ".", apply steps:[6,7,8,9]
6- Now we need to insert a valid number here. A valid number is a number that does not exist in the current row, col, and box.
7- If you find a valid number, insert it into the board and update rows, cols, and boxes.
8- After we inserted the valid point, we call backtrack function for the next ".".
9- When calling the backtrack, decide at which point you are. If you are in the last column, your next backtrack function will start from the next row and column 0. But if you are in the middle of row, you just increase the column parameter for next backtrack function.
10- If in step 5 your value is not ".", that means there is already a valid number here so call next backtracking depending on where position is. If you are in the last column, your next backtrack function will start from the next row and column 0. But if you are in the middle of row, you just increase the column parameter for next backtrack function.
class Solution:
def solveSudoku(self, board: List[List[str]]) -> None:
"""
Do not return anything, modify board in-place instead.
"""
n=len(board)
# create state variables,keep track of rows, cols and boxes
rows=[{} for _ in range(n)]
cols=[{} for _ in range(n)]
boxes=[{} for _ in range(n)]
# get the initial state of the grid
for r in range(n):
for c in range(n):
if board[r][c]!='.':
val=board[r][c]
box_id=self.get_box_id(r,c)
boxes[box_id][val]=True
rows[r][val]=True
cols[c][val]=True
# this backtracking just tells if shoul move to the next cell or not
self.backtrack(board,boxes,rows,cols,0,0)
def backtrack(self,board,boxes,rows,cols,r,c):
# base case. If I hit the last row or col, means all digits were correct so far
if r>=len(board) or c>=len(board[0]):
return True
# situation when cell is empty, fill it with correct value
if board[r][c]==".":
for num in range(1,10):
box_id=self.get_box_id(r,c)
box=boxes[box_id]
row=rows[r]
col=cols[c]
str_num=str(num)
# check rows, cols and boxes make sure str_num is not used before
if self.is_valid(box,col,row,str_num):
board[r][c]=str_num
boxes[box_id][str_num]=True
cols[c][str_num]=True
rows[r][str_num]=True
# if I am the last col and I placed the correct val, move to next row. So first col of the next row
if c==len(board)-1:
if self.backtrack(board,boxes,rows,cols,r+1,0):
return True
# if I am in col between 0-8, move to the col+1, in the same row
else:
if self.backtrack(board,boxes,rows,cols,r,c+1):
return True
# If I got a wrong value, then backtrack. So clear the state that you mutated
del box[str_num]
del row[str_num]
del col[str_num]
board[r][c]="."
# if cell is not empty just call the next backtracking
else:
if c==len(board)-1:
if self.backtrack(board,boxes,rows,cols,r+1,0):
return True
else:
if self.backtrack(board,boxes,rows,cols,r,c+1):
return True
return False
def is_valid(self,box,row,col,num):
if num in box or num in row or num in col:
return False
else:
return True
# a helper to get the id of the 3x3 sub grid, given row and column
def get_box_id(self,r,c):
row=(r//3)*3
col=c//3
return row+col

Google Foobar solution works on Jupyter notebook but not on Google's terminal

I'm on level 3 of Google Foobar, and the code I've written works in a Jupyter notebook, but when I run it in the Foobar command line none of the test cases pass. There's no error of any kind when I run it in Foobar, it just says the answer is incorrect.
Doomsday Fuel
=============
Making fuel for the LAMBCHOP's reactor core is a tricky process because of the exotic matter involved. It starts as raw ore, then during processing, begins randomly changing between forms, eventually reaching a stable form. There may be multiple stable forms that a sample could ultimately reach, not all of which are useful as fuel.
Commander Lambda has tasked you to help the scientists increase fuel creation efficiency by predicting the end state of a given ore sample. You have carefully studied the different structures that the ore can take and which transitions it undergoes. It appears that, while random, the probability of each structure transforming is fixed. That is, each time the ore is in 1 state, it has the same probabilities of entering the next state (which might be the same state). You have recorded the observed transitions in a matrix. The others in the lab have hypothesized more exotic forms that the ore can become, but you haven't seen all of them.
Write a function solution(m) that takes an array of array of nonnegative ints representing how many times that state has gone to the next state and return an array of ints for each terminal state giving the exact probabilities of each terminal state, represented as the numerator for each state, then the denominator for all of them at the end and in simplest form. The matrix is at most 10 by 10. It is guaranteed that no matter which state the ore is in, there is a path from that state to a terminal state. That is, the processing will always eventually end in a stable state. The ore starts in state 0. The denominator will fit within a signed 32-bit integer during the calculation, as long as the fraction is simplified regularly.
For example, consider the matrix m:
[
[0,1,0,0,0,1], # s0, the initial state, goes to s1 and s5 with equal probability
[4,0,0,3,2,0], # s1 can become s0, s3, or s4, but with different probabilities
[0,0,0,0,0,0], # s2 is terminal, and unreachable (never observed in practice)
[0,0,0,0,0,0], # s3 is terminal
[0,0,0,0,0,0], # s4 is terminal
[0,0,0,0,0,0], # s5 is terminal
]
So, we can consider different paths to terminal states, such as:
s0 -> s1 -> s3
s0 -> s1 -> s0 -> s1 -> s0 -> s1 -> s4
s0 -> s1 -> s0 -> s5
Tracing the probabilities of each, we find that
s2 has probability 0
s3 has probability 3/14
s4 has probability 1/7
s5 has probability 9/14
So, putting that together, and making a common denominator, gives an answer in the form of
[s2.numerator, s3.numerator, s4.numerator, s5.numerator, denominator] which is
[0, 3, 2, 9, 14].
Languages
=========
To provide a Java solution, edit Solution.java
To provide a Python solution, edit solution.py
Test cases
==========
Your code should pass the following test cases.
Note that it may also be run against hidden test cases not shown here.
-- Java cases --
Input:
Solution.solution({{0, 2, 1, 0, 0}, {0, 0, 0, 3, 4}, {0, 0, 0, 0, 0}, {0, 0, 0, 0,0}, {0, 0, 0, 0, 0}})
Output:
[7, 6, 8, 21]
Input:
Solution.solution({{0, 1, 0, 0, 0, 1}, {4, 0, 0, 3, 2, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}})
Output:
[0, 3, 2, 9, 14]
-- Python cases --
Input:
solution.solution([[0, 2, 1, 0, 0], [0, 0, 0, 3, 4], [0, 0, 0, 0, 0], [0, 0, 0, 0,0], [0, 0, 0, 0, 0]])
Output:
[7, 6, 8, 21]
Input:
solution.solution([[0, 1, 0, 0, 0, 1], [4, 0, 0, 3, 2, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]])
Output:
[0, 3, 2, 9, 14]
My solution is as follows:
import numpy as np
import pandas as pd
from fractions import Fraction
def solution(m):
if m == [[0]]:
return [1, 1]
else:
return run_matrix_computation(m)
def run_matrix_computation(starting_ore_matrix):
numpy_matrix = convert_to_numpy_matrix(starting_ore_matrix)
ordered_matrix = order_matrix(numpy_matrix)
absorption_matrix, split_index = create_absorption_matrix(ordered_matrix)
R, Q = store_R_and_Q(absorption_matrix, split_index)
FR = compute_FR(R, Q)
first_row = get_first_row_of_FR(FR)
common_denominator, fraction_list = calculate_common_denominator(first_row)
return return_int_array(common_denominator, fraction_list)
def convert_to_numpy_matrix(original_matrix):
return np.asarray(original_matrix)
def order_matrix(numpy_matrix):
labeled_dataframe = pd.DataFrame(data=numpy_matrix)
index_order = labeled_dataframe.sum(axis=1).sort_values(ascending=True).index
converted_matrix = convert_matrix_to_fractions(labeled_dataframe)
return converted_matrix.iloc[index_order, index_order]
def convert_matrix_to_fractions(original_matrix):
for i in range(len(original_matrix.index)):
sum = original_matrix.sum(axis=1).iloc[i]
if sum != 0:
for j in range(len(original_matrix.columns)):
if original_matrix.iloc[i, j]:
original_matrix.iloc[i, j] = original_matrix.iloc[i, j] / sum
return original_matrix
def create_absorption_matrix(sorted_matrix):
count = 0
for i in range(len(sorted_matrix.index)):
if not(sorted_matrix.sum(axis=1).iloc[i]):
sorted_matrix.iloc[i, i] = 1
count = count + 1
return sorted_matrix, count
def store_R_and_Q(absorption_matrix, split_index):
return split_into_new_matrices(absorption_matrix, split_index)
def split_into_new_matrices(absorption_matrix, split_index):
numpy_matrix = absorption_matrix.to_numpy()
R = numpy_matrix[split_index:, :split_index]
Q = numpy_matrix[split_index:, split_index:]
return R, Q
def calculate_F(R, Q):
num_rows, num_cols = Q.shape
I = np.identity(num_rows)
IQ = I - Q
return np.linalg.inv(IQ)
def compute_FR(R, Q):
F = calculate_F(R, Q)
return np.matmul(F, R)
def get_first_row_of_FR(FR):
return FR[0, :]
def calculate_common_denominator(list):
fraction_list = convert_to_fractions(list)
list_denominators = get_denominators(fraction_list)
GCD = calculate_greatest_common_denominator(list_denominators)
return GCD, fraction_list
def convert_to_fractions(list):
fraction_list = []
for i in range(len(list)):
fraction_list.append(Fraction(list[i]).limit_denominator())
return fraction_list
def get_denominators(fractions):
denominators = []
for i in range(len(fractions)):
denominators.append(fractions[i].denominator)
return denominators
def calculate_greatest_common_denominator(denominators):
GCD = 0
if len(denominators) == 1:
return denominators
else:
for i in range(len(denominators) - 1):
cur_GCD = np.lcm(denominators[i], denominators[i + 1])
if cur_GCD > GCD:
GCD = cur_GCD
return GCD
def return_int_array(denominator, fractions):
final_list = []
for i in range(len(fractions)):
if(not fractions[i].numerator):
final_list.append(0)
else:
multiplier = denominator/fractions[i].denominator
final_list.append(int(fractions[i].numerator * multiplier))
final_list.append(int(denominator))
return final_list
Running any of the test cases with this code works, yet every test fails on Foobar. Is there some sort of formatting error? I've examined the type of object that's returned vs the type of object that Foobar is looking for and they're both int lists. Everything in my code is, to my knowledge, supported by Python 2.7.13 which is what Foobar uses. The libraries I used were also allowed.

I think you should try to check the libraries first, if they are installed and the versions.
Then you should use replit or something else with the exact environment.
And finally, is your code on solution class?

I am late to this but I believe the problem lies in how python 2.7 handles division. It was fixed for me by putting this at the start of my file.
from __future__ import division

2D List being returned as 1D

When given an input
Example:
runGenerations2d([[0, 0, 1, 0], [0, 0, 1, 1], [1, 1, 1, 0], [1, 0, 0, 1]])
With a click on any box the program will convert the number from 1 to 0 else 0 to 1. The problem arises after a click the information passed back to runGenerations2d is coming in a 1D list rather than the original 2D list. I believe the function causing the problem is evolve2d
Any advice on where i'm going wrong here?
import time # provides time.sleep(0.5)
from csplot import * # provides the visual board
from random import * # provides choice( [0,1] ), etc.
import sys # larger recursive stack
sys.setrecursionlimit(100000) # 100,000 deep
def runGenerations2d(L , x=0 ,y=0 ):
show(L)
print( L ) # display the list, L
time.sleep(5) # pause a bit
newL = evolve2d( L ) # evolve L into newL
print(newL)
if min(L) == 1:
#I like read outs to be explained so I added an extra print command.
if x<=1: # Takes into account the possibility of a 1 click completition.
print ('BaseCase Reached!... it took %i click to complete' % (x))
print (x)
done()#removes the need to input done() into the shell
else:
print ('BaseCase Reached!... it took %i clicks to complete' % (x))
print (x)
done()#removes the need to input done() into the shell
return
x = x+1 # add 1 to x before every recusion
runGenerations2d( newL , x,y ) # recurse
def evolve2d( L ):
N = len(L)
x,y = sqinput2()
print(x,y)
show(L)
print("Two")
time.sleep(5)
return [setNewElement2d(L, xx, yy, x, y) for xx in range(N) for yy in range(N)]
def setNewElement2d( L, xx, yy, x=0,y=0 ):
show(L)
print(L)
print("Three")
#time.sleep(5)
if (xx,yy) == (x,y): # if it's the user's chosen row and column
if L[xx][yy]==1:
return 0
else:
return 1 # If it's already 1 return 0 else return 1
else: # otherwise
print("Lastly")
return L[xx][yy] # return the original
Error given:
[[0, 0, 1, 0], [0, 0, 1, 1], [1, 1, 1, 0], [1, 0, 0, 1]]
Three
Lastly
[0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1]
[0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1]
The data does not seem 2d.
Try using sqinput instead.

As mentioned by Brian, create a 2D list comprehension:
[[... for ... in ...] for ... in ...]
instead of a 1D list comprehension:
[... for ... in ... for ... in ...]

Detect whether sequence is a multiple of a subsequence in Python

I have a tuple of zeros and ones, for instance:
(1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1)
It turns out:
(1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1) == (1, 0, 1, 1) * 3
I want a function f such that if s is a non-empty tuple of zeros and ones, f(s) is the shortest subtuple r such that s == r * n for some positive integer n.
So for instance,
f( (1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1) ) == (1, 0, 1, 1)
What is a slick way to write the function f in Python?
Edit:
The naive method I am currently using
def f(s):
for i in range(1,len(s)):
if len(s)%i == 0 and s == s[:i] * (len(s)/i):
return s[:i]

I believe I have an O(n) time solution (actually 2n+r, n is length of tuple, r is sub tuplle) which does not use suffix trees, but uses a string matching algorithm (like KMP, which you should find off-the shelf).
We use the following little known theorem:
If x,y are strings over some alphabet,
then xy = yx if and only if x = z^k and y = z^l for some string z and integers k,l.
I now claim that, for the purposes of our problem, this means that all we need to do is determine if the given tuple/list (or string) is a cyclic shift of itself!
To determine if a string is a cyclic shift of itself, we concatenate it with itself (it does not even have to be a real concat, just a virtual one will do) and check for a substring match (with itself).
For a proof of that, suppose the string is a cyclic shift of itself.
The we have that the given string y = uv = vu.
Since uv = vu, we must have that u = z^k and v= z^l and hence y = z^{k+l} from the above theorem. The other direction is easy to prove.
Here is the python code. The method is called powercheck.
def powercheck(lst):
count = 0
position = 0
for pos in KnuthMorrisPratt(double(lst), lst):
count += 1
position = pos
if count == 2:
break
return lst[:position]
def double(lst):
for i in range(1,3):
for elem in lst:
yield elem
def main():
print powercheck([1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1])
if __name__ == "__main__":
main()
And here is the KMP code which I used (due to David Eppstein).
# Knuth-Morris-Pratt string matching
# David Eppstein, UC Irvine, 1 Mar 2002
def KnuthMorrisPratt(text, pattern):
'''Yields all starting positions of copies of the pattern in the text.
Calling conventions are similar to string.find, but its arguments can be
lists or iterators, not just strings, it returns all matches, not just
the first one, and it does not need the whole text in memory at once.
Whenever it yields, it will have read the text exactly up to and including
the match that caused the yield.'''
# allow indexing into pattern and protect against change during yield
pattern = list(pattern)
# build table of shift amounts
shifts = [1] * (len(pattern) + 1)
shift = 1
for pos in range(len(pattern)):
while shift <= pos and pattern[pos] != pattern[pos-shift]:
shift += shifts[pos-shift]
shifts[pos+1] = shift
# do the actual search
startPos = 0
matchLen = 0
for c in text:
while matchLen == len(pattern) or \
matchLen >= 0 and pattern[matchLen] != c:
startPos += shifts[matchLen]
matchLen -= shifts[matchLen]
matchLen += 1
if matchLen == len(pattern):
yield startPos
For your sample this outputs
[1,0,1,1]
as expected.
I compared this against shx2's code(not the numpy one), by generating a random 50 bit string, then replication to make the total length as 1 million. This was the output (the decimal number is the output of time.time())
1362988461.75
(50, [1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1])
1362988465.96
50 [1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1]
1362988487.14
The above method took ~4 seconds, while shx2's method took ~21 seconds!
Here was the timing code. (shx2's method was called powercheck2).
def rand_bitstring(n):
rand = random.SystemRandom()
lst = []
for j in range(1, n+1):
r = rand.randint(1,2)
if r == 2:
lst.append(0)
else:
lst.append(1)
return lst
def main():
lst = rand_bitstring(50)*200000
print time.time()
print powercheck(lst)
print time.time()
powercheck2(lst)
print time.time()

The following solution is O(N^2), but has the advantage of not creating any copies (or slices) of your data, as it is based on iterators.
Depending on the size of your input, the fact you avoid making copies of the data can result in a significant speed-up, but of course, it would not scale as well for huge inputs as algorithms with lower complexity (e.g. O(N*logN)).
[This is the second revision of my solution, the first one is given below. This one is simpler to understand, and is more along the lines of OP's tuple-multiplication, only using iterators.]
from itertools import izip, chain, tee
def iter_eq(seq1, seq2):
""" assumes the sequences have the same len """
return all( v1 == v2 for v1, v2 in izip(seq1, seq2) )
def dup_seq(seq, n):
""" returns an iterator which is seq chained to itself n times """
return chain(*tee(seq, n))
def is_reps(arr, slice_size):
if len(arr) % slice_size != 0:
return False
num_slices = len(arr) / slice_size
return iter_eq(arr, dup_seq(arr[:slice_size], num_slices))
s = (1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1)
for i in range(1,len(s)):
if is_reps(s, i):
print i, s[:i]
break
[My original solution]
from itertools import islice
def is_reps(arr, num_slices):
if len(arr) % num_slices != 0:
return False
slice_size = len(arr) / num_slices
for i in xrange(slice_size):
if len(set( islice(arr, i, None, num_slices) )) > 1:
return False
return True
s = (1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1)
for i in range(1,len(s)):
if is_reps(s, i):
print i, s[:i]
break
You can avoid the call to set() by using something like:
def is_iter_unique(seq):
""" a faster version of testing len(set(seq)) <= 1 """
seen = set()
for x in seq:
seen.add(x)
if len(seen) > 1:
return False
return True
and replacing this line:
if len(set( islice(arr, i, None, num_slices) )) > 1:
with:
if not is_iter_unique(islice(arr, i, None, num_slices)):

Simplifying Knoothe's solution. His algorithm is right, but his implementation is too complex. This implementation is also O(n).
Since your array is only composed of ones and zeros, what I do is use existing str.find implementation (Bayer Moore) to implement Knoothe's idea. It's suprisingly simpler and amazingly faster at runtime.
def f(s):
s2 = ''.join(map(str, s))
return s[:(s2+s2).index(s2, 1)]

Here's another solution (competing with my earlier iterators-based solution), leveraging numpy.
It does make a (single) copy of your data, but taking advantage of the fact your values are 0s and 1s, it is super-fast, thanks to numpy's magics.
import numpy as np
def is_reps(arr, slice_size):
if len(arr) % slice_size != 0:
return False
arr = arr.reshape((-1, slice_size))
return (arr.all(axis=0) | (~arr).all(axis=0)).all()
s = (1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1) * 1000
a = np.array(s, dtype=bool)
for i in range(1,len(s)):
if is_reps(a, i):
print i, s[:i]
break

Just a different approach to the problem
I first determine all the factors of the length and then split the list and check if all the parts are same
>>> def f(s):
def factors(n):
#http://stackoverflow.com/a/6800214/977038
return set(reduce(list.__add__,
([i, n//i] for i in range(2, int(n**0.5) + 1) if n % i == 0)))
_len = len(s)
for fact in reversed(list(factors(_len))):
compare_set = set(izip(*[iter(s)]*fact))
if len(compare_set) == 1:
return compare_set
>>> f(t)
set([(1, 0, 1, 1)])

You can archive it in sublinear time by XOR'ing the rotated binary form for the input array:
get the binary representation of the array, input_binary
loop from i = 1 to len(input_array)/2, and for each loop, rotate the input_binary to the right by i bits, save it as rotated_bin, then compare the XOR of rotated_bin and input_binary.
The first i that yields 0, is the index to which is the desired substring.
Complete code:
def get_substring(arr):
binary = ''.join(map(str, arr)) # join the elements to get the binary form
for i in xrange(1, len(arr) / 2):
# do a i bit rotation shift, get bit string sub_bin
rotated_bin = binary[-i:] + binary[:-i]
if int(rotated_bin) ^ int(binary) == 0:
return arr[0:i]
return None
if __name__ == "__main__":
test = [1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1]
print get_substring(test) # [1,0,1,1]

This one is just a dumb recursive comparison in Haskell. It takes about one second for Knoothe's million long string (f a). Cool problem! I'll think about it some more.
a = concat $ replicate 20000
[1,1,1,0,0,1,0,1,0,0,1,0,0,1,1,1,0,0,
0,0,0,0,1,1,1,1,0,0,0,1,1,0,1,1,1,1,
1,1,1,0,0,1,1,1,0,0,0,0,0,1]
f s =
f' s [] where
f' [] result = []
f' (x:xs) result =
let y = result ++ [x]
in if concat (replicate (div (length s) (length y)) y) == s
then y
else f' xs y