I'm trying to convert a Python algorithm from this Stack Overflow answer to split a string without spaces into words to C#.
Unfortunately I don't know anything about Python so the translation is proving very difficult.
The lines I don't understand are:
wordcost = dict((k, log((i+1)*log(len(words)))) for i,k in enumerate(words)) <= THIS LINE
and
def best_match(i):
candidates = enumerate(reversed(cost[max(0, i-maxword):i]))
return min((c + wordcost.get(s[i-k-1:i], 9e999), k+1) for k,c in candidates) <= THIS LINE
It looks as though best_match(i) it should return a Tuple<>. What is the equivalent in C#?
Here is the full Python script:
from math import log
# Build a cost dictionary, assuming Zipf's law and cost = -math.log(probability).
words = open("words-by-frequency.txt").read().split()
wordcost = dict((k, log((i+1)*log(len(words)))) for i,k in enumerate(words))
maxword = max(len(x) for x in words)
def infer_spaces(s):
"""Uses dynamic programming to infer the location of spaces in a string
without spaces."""
# Find the best match for the i first characters, assuming cost has
# been built for the i-1 first characters.
# Returns a pair (match_cost, match_length).
def best_match(i):
candidates = enumerate(reversed(cost[max(0, i-maxword):i]))
return min((c + wordcost.get(s[i-k-1:i], 9e999), k+1) for k,c in candidates)
# Build the cost array.
cost = [0]
for i in range(1,len(s)+1):
c,k = best_match(i)
cost.append(c)
# Backtrack to recover the minimal-cost string.
out = []
i = len(s)
while i>0:
c,k = best_match(i)
assert c == cost[i]
out.append(s[i-k:i])
i -= k
return " ".join(reversed(out))
I found that algorithm interesting so here is my translation:
class WordSplitter {
private readonly Dictionary<string, double> _wordCosts;
private readonly int _maxWordLength;
public WordSplitter(string freqFilePath) {
// words = open("words-by-frequency.txt").read().split()
var words = File.ReadAllLines(freqFilePath);
// wordcost = dict((k, log((i+1)*log(len(words)))) for i,k in enumerate(words))
_wordCosts = words.Select((k, i) => new { Key = k, Value = Math.Log((i + 1) * Math.Log(words.Length)) }).ToDictionary(c => c.Key, c => c.Value);
// maxword = max(len(x) for x in words)
_maxWordLength = words.Select(c => c.Length).Max();
}
public string InferSpaces(string target) {
// cost = [0]
var costs = new List<double>() { 0 };
foreach (var i in Enumerable.Range(1, target.Length)) {
var (c, k) = BestMatch(i);
costs.Add(c);
}
var output = new List<string>();
int len = target.Length;
while (len > 0) {
var (c, k) = BestMatch(len);
Debug.Assert(k > 0);
Debug.Assert(c == costs[len]);
// use Substring if your compiler version doesn't support slicing
// but pay attention that Substring second argument is length, not end index
output.Add(target[(len - k)..len]);
len -= k;
}
output.Reverse();
return String.Join(" ", output);
(double cost, int length) BestMatch(int i) {
var start = Math.Max(0, i - _maxWordLength);
// GetRange second argument is length
var x = costs.GetRange(start, i - start);
x.Reverse();
// now, this part is easier to comprehend if it's expanded a bit
// you can do it in cryptic way too like in python though if you like
(double cost, int length)? result = null;
for (int k = 0; k < x.Count; k++) {
var c = x[k];
var sub = target[(i - k - 1)..i];
var cost = c + (_wordCosts.ContainsKey(sub) ? _wordCosts[sub] : 9e99); // 9e99 is just some big number. 9e999 is outside of double range in C#, so use smaller one
// save minimal cost
if (result == null || result.Value.cost > cost)
result = (cost, k + 1);
}
// return minimal cost
return result.Value;
}
}
}
Usage:
var splitter = new WordSplitter(#"C:\tmp\words.txt");
var result = splitter.InferSpaces("thumbgreenappleactiveassignmentweeklymetaphor");
I'm playing around with OR-Tools to create puzzles like a < b, a < c, b < c, c = d. The idea is to give people these equations and they need to find the numbers that need to be entered into the variables a-d to create a valid solution. OR-Tools should create solutions and I remove the numbers to create the puzzle. Some numbers will be left to give a start. For example, in the above equations b=2 might be given, the player can then figure out that a must be 1. Players are given the variables, the equations with operators, and some start values and need to figure out the set of variable assignments that make the set of equations valid.
Creating variables and constraints like this works like a charm:
model.NewIntVar(min_number, max_number, 'a')
model.NewIntVar(min_number, max_number, 'b')
model.NewIntVar(min_number, max_number, 'c')
model.NewIntVar(min_number, max_number, 'd')
model.Add(a < b)
model.Add(a < c)
model.Add(b < c)
model.Add(c = d)
This makes or-tools find the operands.
Now I also want the mathematical operators to be "calculated", so that I don't have to provide them.
My current workaround is randomizing them with operator before creating the constraints like this:
def rand_op():
return random.choice(['<', '>', '='])
ops = {
'<': operator.lt,
'>': operator.gt,
'=': operator.eq
}
model.Add(ops.get(rand_op())(a, b))
model.Add(ops.get(rand_op())(b, c))
...
That does solve my problem, but obviously kind of takes some magic from or-tools as with this randomness many many problems that do not have a solution are created and it takes quite a lot of loops to find solutions.
So I'm wondering how I could achieve this in a better way. One naive approach would be to work with allowed assignments. So I would take two variables for the operands and one variable for the operator (with encoded operator signs) and then make allowed assignments:
ops = {
'<': 0,
'>': 1,
'=': 2
}
a = model.NewIntVar(min_number, max_number, 'a')
b = model.NewIntVar(min_number, max_number, 'b')
op_ab = model.NewIntVar(min_op_number, max_op_number, 'op_ab')
model.AddAllowedAssignments([a, b, op_ab], [
(1, 2, 0),
(2, 1, 1),
(1, 1, 2),
...
])
I'm pretty sure this will work, but it would create quite a lot of allowed constraints. Depending on the size of the puzzle it could go into the millions. The more variables and operators available the more constraints would be needed and the larger the domain for the variables the larger the list of allowed assignments per constraint. So it would grow horizontally and vertically in size, probably exponentially.
Is there another approach I could use? Any ideas on a more performant way? Any constraints that I should look into?
Here's my take on this; I hope I haven't missed some important requirement.
The following model uses OR-tools CP-SAT solver in Python. It is a quite direct approach to the problem, connecting the operators and the variables using OnlyEnforceIf via an array of boolean of length 3 (for the 3 operators) and then require that the sum of these is 1 (exactly one operator).
The model generates all possible solutions given n (number of variables) and their domain (min_val and max_val).
from __future__ import print_function
from ortools.sat.python import cp_model as cp
"""
Print solutions
"""
class SolutionPrinter(cp.CpSolverSolutionCallback):
def __init__(self, x,ops,ops_d,num_sols=0):
cp.CpSolverSolutionCallback.__init__(self)
self.__x = x
self.__ops = ops
self.__ops_d = ops_d
self.__num_sols = num_sols
self.__solution_count = 0
def OnSolutionCallback(self):
self.__solution_count += 1
x = [self.Value(v) for v in self.__x]
n = len(x)
# ops = [self.Value(v) for v in self.__ops]
ops_d = self.__ops_d
num_sols = self.__num_sols
print("===================")
print("Problem:")
for i in range(n):
for j in range(i+1,n):
op = ops_d[self.Value(self.__ops[i,j])]
print(f"x{i} {op} x{j}")
print()
print("Solution:")
print("x:",x)
for i in range(n):
for j in range(i+1,n):
op = ops_d[self.Value(self.__ops[i,j])]
print(f"{x[i]} {op} {x[j]}")
print()
if num_sols > 0 and self.__solution_count >= num_sols:
self.StopSearch()
def SolutionCount(self):
return self.__solution_count
def generate_math_puzzle(n=3,min_val=1,max_val=10,num_sols=0):
model = cp.CpModel()
lt = 0 # <
eq = 1 # >
gt = 2 # =
ops_a = [lt,eq,gt]
num_ops = len(ops_a)
# loopup for presentation
ops_d = {0:"<",
1:"=",
2:">"}
num_pairs = (n*(n-1)) // 2
# variables
x = [model.NewIntVar(min_val, max_val, f'x[{i}]') for i in range(n)]
# We have n-1 operators
# ops = [model.NewIntVar(lt, gt, f'ops[{i}]') for i in range(n-1)]
ops = {}
for i in range(n):
for j in range(i+1,n):
ops[i,j] = model.NewIntVar(lt,gt,f"op_{i}_{j}")
bs = [model.NewBoolVar(f"bs_{i}_{j}") for j in range(num_ops)]
for op in ops_a:
model.Add(ops[i,j]==op).OnlyEnforceIf(bs[op])
model.Add(sum(bs) == 1) # exactly one operation
model.Add(x[i] < x[j]).OnlyEnforceIf(bs[lt])
model.Add(x[i] == x[j]).OnlyEnforceIf(bs[eq])
model.Add(x[i] > x[j]).OnlyEnforceIf(bs[gt])
print("ModelStats:", model.ModelStats())
solver = cp.CpSolver()
# solver.parameters.num_search_workers = 8
# solution_printer = SolutionPrinter(x_flat)
solution_printer = SolutionPrinter(x,ops,ops_d,num_sols)
status = solver.SearchForAllSolutions(model, solution_printer)
if not status in [cp.OPTIMAL,cp.FEASIBLE]:
print("No solution!")
print()
print("NumSolutions:", solution_printer.SolutionCount())
print("NumConflicts:", solver.NumConflicts())
print("NumBranches:", solver.NumBranches())
print("WallTime:", solver.WallTime())
n=3
min_val=1
max_val=10
num_sols = 0
generate_math_puzzle(n,min_val,max_val)
Here are some solutions for the simple problem with n=3, min_val=1, and max_val=10.
Problem:
x0 > x1
x0 < x2
x1 < x2
Solution:
x: [2, 1, 3]
2 > 1
2 < 3
1 < 3
===================
Problem:
x0 = x1
x0 < x2
x1 < x2
Solution:
x: [1, 1, 2]
1 = 1
1 < 2
1 < 2
In all, it's 1000 solutions and takes about 0.1s to generate them all.
Here is a result for a larger problem n=30, min_val=1, and max_val=100:
===================
Problem:
x0 = x1
x0 < x2
x0 < x3
x0 < x4
x0 < x5
x0 < x6
x0 < x7
x0 < x8
x0 < x9
x1 < x2
x1 < x3
x1 < x4
x1 < x5
x1 < x6
x1 < x7
x1 < x8
x1 < x9
x2 < x3
x2 < x4
x2 < x5
x2 < x6
x2 < x7
x2 < x8
x2 < x9
x3 < x4
x3 < x5
x3 < x6
x3 < x7
x3 < x8
x3 < x9
x4 < x5
x4 < x6
x4 < x7
x4 < x8
x4 < x9
x5 < x6
x5 < x7
x5 < x8
x5 < x9
x6 < x7
x6 < x8
x6 < x9
x7 < x8
x7 < x9
x8 < x9
Solution:
x: [1, 1, 2, 3, 4, 5, 6, 7, 15, 65]
1 = 1
1 < 2
1 < 3
1 < 4
1 < 5
1 < 6
1 < 7
1 < 15
1 < 65
1 < 2
1 < 3
1 < 4
1 < 5
1 < 6
1 < 7
1 < 15
1 < 65
2 < 3
2 < 4
2 < 5
2 < 6
2 < 7
2 < 15
2 < 65
3 < 4
3 < 5
3 < 6
3 < 7
3 < 15
3 < 65
4 < 5
4 < 6
4 < 7
4 < 15
4 < 65
5 < 6
5 < 7
5 < 15
5 < 65
6 < 7
6 < 15
6 < 65
7 < 15
7 < 65
15 < 65
Note that for some problem instances (i.e. pairs of xi op xj) there might be multiple solutions.
#bechtold, your problem was interesting enough that I attempted a solution. It is based on my last comment to your post.
Caveat: It's not exactly what you want because
I use c#, not python (I simply don't have enough experience in python and don't have a working installation of it) and
I used the old solver (Google.OrTools.ConstraintSolver) instead of the newer SAT solver (Google.OrTools.Sat); I'll explain the reasons for that below.
Class Term
First I created a class Term with some overloaded operators to make it easier to create the expressions for the puzzle.
The overloaded operators + and * allow creation of terms like 2 * a or a + 2 to make the puzzles a little more interesting. These create a new IntVar and constrain it to be equal to the arithmetic operation on the original variable.
The overloaded operators >, == and < are a little more subtle. They create a constraint like a < b, but do not add it directly to the model. Instead, they also create a new enforced BoolVar (the IsInPuzzle variable of my comment), and constrain it to be less than or equal to the Var() of the constraint. This effectively makes the constraint optional, because depending on whether enforced == 1 or is 0 or has an open domain, the constraint does or doesn't have to be fulfilled. This is pretty much the equivalent of the OnlyEnforceIf() method on constraints in the new SAT solver.
The overloaded operators return Expression objects (see below).
The class includes some housekeeping methods and an overridden ToString() method for easier output.
using Google.OrTools.ConstraintSolver;
using System;
namespace SO69225141_create_puzzles
{
#pragma warning disable CS0660 // Type defines operator == or operator != but does not override Object.Equals(object o)
#pragma warning disable CS0661 // Type defines operator == or operator != but does not override Object.GetHashCode()
internal class Term
#pragma warning restore CS0661 // Type defines operator == or operator != but does not override Object.GetHashCode()
#pragma warning restore CS0660 // Type defines operator == or operator != but does not override Object.Equals(object o)
{
protected Google.OrTools.ConstraintSolver.Solver solver;
public string name { get; private set; }
public IntVar intVar { get; private set; }
private string basedOnVariableName;
public Term(Solver solver_, string name_, long minNumber_, long maxNumber_)
{
this.solver = solver_ ?? throw new ArgumentNullException(nameof(solver_));
if (string.IsNullOrEmpty(name_))
{
name = "unknown";
}
else
{
name = name_;
}
intVar = solver.MakeIntVar(minNumber_, maxNumber_, name);
basedOnVariableName = name;
}
public Term(Solver solver_, string name_) : this(solver_, name_, PuzzleCreator.minNumber, PuzzleCreator.maxNumber) { }
// Returns true if this term is based on the same variable as the other term (e.g. true for "(a + 2)" given "a" but not for "(3 * b)".
public bool isBasedOnSameVariableAs(Term other)
{
return this.basedOnVariableName == other.basedOnVariableName;
}
// Creates a new Term whose IntVar is constrained to be term + val
public static Term operator +(Term term, long val)
{
if (object.ReferenceEquals(term, null)) { throw new ArgumentNullException(nameof(term)); }
Term newTerm = new Term(term.solver, $"({term.name} + {val})", term.intVar.Min() + val, term.intVar.Max() + val);
newTerm.basedOnVariableName = term.basedOnVariableName;
term.solver.Add(newTerm.intVar == (term.intVar + val));
return newTerm;
}
// Creates a new Term whose IntVar is constrained to be val * term
public static Term operator *(long val, Term term)
{
if (object.ReferenceEquals(term, null)) { throw new ArgumentNullException(nameof(term)); }
Term newTerm = new Term(term.solver, $"({val} * {term.name})", val * term.intVar.Min(), val * term.intVar.Max());
newTerm.basedOnVariableName = term.basedOnVariableName;
term.solver.Add(newTerm.intVar == (val * term.intVar));
return newTerm;
}
public static Expression operator >(Term left, Term right)
{
if (object.ReferenceEquals(left, null)) { throw new ArgumentNullException(nameof(left)); }
if (object.ReferenceEquals(right, null)) { throw new ArgumentNullException(nameof(right)); }
Solver solver = left.intVar.solver();
Constraint constraint = solver.MakeGreater(left.intVar, right.intVar);
string name = $"{left.name} > {right.name}";
IntVar enforced = solver.MakeBoolVar(name);
solver.Add(constraint.Var() >= enforced);
Expression expr = new Expression(name, enforced, constraint.Var());
return expr;
}
public static Expression operator <(Term left, Term right)
{
if (object.ReferenceEquals(left, null)) { throw new ArgumentNullException(nameof(left)); }
if (object.ReferenceEquals(right, null)) { throw new ArgumentNullException(nameof(right)); }
Solver solver = left.intVar.solver();
Constraint constraint = solver.MakeLess(left.intVar, right.intVar);
string name = $"{left.name} < {right.name}";
IntVar enforced = solver.MakeBoolVar(name);
solver.Add(constraint.Var() >= enforced);
Expression expr = new Expression(name, enforced, constraint.Var());
return expr;
}
public static Expression operator ==(Term left, Term right)
{
if (object.ReferenceEquals(left, null)) { throw new ArgumentNullException(nameof(left)); }
if (object.ReferenceEquals(right, null)) { throw new ArgumentNullException(nameof(right)); }
Solver solver = left.intVar.solver();
Constraint constraint = solver.MakeEquality(left.intVar, right.intVar);
string name = $"{left.name} == {right.name}";
IntVar enforced = solver.MakeBoolVar(name);
solver.Add(constraint.Var() >= enforced);
Expression expr = new Expression(name, enforced, constraint.Var());
return expr;
}
public static Expression operator !=(Term left, Term right)
{
if (object.ReferenceEquals(left, null)) { throw new ArgumentNullException(nameof(left)); }
if (object.ReferenceEquals(right, null)) { throw new ArgumentNullException(nameof(right)); }
Solver solver = left.intVar.solver();
Constraint constraint = solver.MakeNonEquality(left.intVar, right.intVar);
string name = $"{left.name} != {right.name}";
IntVar enforced = solver.MakeBoolVar(name);
solver.Add(constraint.Var() >= enforced);
Expression expr = new Expression(name, enforced, constraint.Var());
return expr;
}
public override string ToString()
{
if (intVar.Bound())
{
return $"{name} = {intVar.Value()}";
}
else
{
return intVar.ToString();
}
}
}
}
Class Expression
The class Expression keeps track of the enforced and constraint variable as well some helper methods to aid decision building and output.
The isEnforced property is true when the expression is being enforced as part of a puzzle.
The Bound() property will be true either after a decision to enforce the expression or after refutation of the decision, but also if the truth value of the expression is already determined via the propagation of the previously enforced expressions.
using System;
using System.Text;
using Google.OrTools.ConstraintSolver;
namespace SO69225141_create_puzzles
{
internal class Expression
{
public Expression(string name_, IntVar enforcedVar_, IntVar constraintVar_)
{
this.name = name_ ?? throw new ArgumentNullException(nameof(name_));
this.enforcedVar = enforcedVar_ ?? throw new ArgumentNullException(nameof(enforcedVar_));
this.constraintVar = constraintVar_ ?? throw new ArgumentNullException(nameof(constraintVar_));
}
public string name { get; private set; }
public IntVar enforcedVar { get; private set; }
public IntVar constraintVar { get; private set; }
public bool Bound()
{
return enforcedVar.Bound() || constraintVar.Bound();
}
public bool isEnforced
{
get
{
return enforcedVar.Bound() && (enforcedVar.Value() == 1);
}
}
public override string ToString()
{
var sb = new StringBuilder();
sb.Append(name);
if (!isEnforced)
{
sb.Append("; not enforced");
if (constraintVar.Bound())
{
switch (constraintVar.Value())
{
case 0:
sb.Append("; not fulfillable");
break;
case 1:
sb.Append("; fulfilled");
break;
}
}
}
return sb.ToString();
}
}
}
Class PuzzleCreator
The main class that assembles the model and controls the search is called PuzzleCreator.
The method initModel(Solver) does the following:
Creates the base variables a, b, c (as many as are specified in numberOfVariables) as Term objects.
Creates combinations of the base variables and some constants as derived Term objects like a + 2 or 2 * a
Creates an optional constraint for each combination of a pair of terms and a comparison operator, skipping pairs of terms based on the same variable so that expressions like a < a + 2 don't clutter up the model.
Adds the enforced variables of the optional constraints to the decision variables list.
There is a dump() method to output everything to the screen for checking.
The method solveModel(Solver) searches for new puzzles. It creates a custom decision builder (described below) to control the search, and continues looping as long as solutions are found or until the limit on number of solutions is reached. Each solution of the model corresponds to a new puzzle.
The method Main of course, puts things together to allow everything to be executed.
using Google.OrTools.ConstraintSolver;
using System;
using System.Collections.Generic;
namespace SO69225141_create_puzzles
{
public class PuzzleCreator
{
// Parameters for model creation
public const int numberOfVariables = 3;
public const long minNumber = 1;
public const long maxNumber = 3;
public const long minConstant = 2;
public const long maxConstant = 2;
public const int numberOfPuzzlesToCreate = 10;
public static void Main(string[] args)
{
PuzzleCreator model = new PuzzleCreator();
using (Google.OrTools.ConstraintSolver.Solver solver = new Google.OrTools.ConstraintSolver.Solver("Test"))
{
model.InitModel(solver);
model.dump();
model.solveModel(solver);
}
Console.WriteLine("Press any key to continue...");
Console.ReadKey();
}
// List of all the terms (a, b, ... 2 * a, a + 2, ...)
List<Term> allTerms;
// List of all the base variables only (a, b, c...)
List<Term> baseVariables;
// The decision variables for the solver, namely whether a certain expression is enforced.
Expression[] allExpressions;
public void InitModel(Google.OrTools.ConstraintSolver.Solver solver)
{
allTerms = new List<Term>();
baseVariables = new List<Term>();
// Make the base variables
for (int i = 0; i < numberOfVariables; i++)
{
string name = ((char)((int)'a' + i)).ToString();
Term variable = new Term(solver, name);
allTerms.Add(variable);
baseVariables.Add(variable);
}
// Just to make it more interesting, also create some terms
// like (a + 2) and (2 * a)
foreach (Term term in baseVariables)
{
foreach (arithmeticOperators op in Enum.GetValues(typeof(arithmeticOperators)))
{
switch (op)
{
case arithmeticOperators.plus:
for (long addend = Math.Max(1L, minConstant); addend <= maxConstant; addend++)
{
Term sum = (term + addend);
allTerms.Add(sum);
}
break;
case arithmeticOperators.times:
for (long factor = Math.Max(2L, minConstant); factor <= maxConstant; factor++)
{
Term product = (factor * term);
allTerms.Add(product);
}
break;
}
}
}
// Now create all combinations of terms and comparison operators
var allExpressionsList = new List<Expression>();
foreach (Term left in allTerms)
{
foreach (Term right in allTerms)
{
if (!left.isBasedOnSameVariableAs(right))
{
foreach (comparisonOperators op in Enum.GetValues(typeof(comparisonOperators)))
{
switch (op)
{
case comparisonOperators.less:
{
string name = $"{left.name} < {right.name}";
Expression comparison = (left < right);
allExpressionsList.Add(comparison);
}
break;
case comparisonOperators.equals:
{
string name = $"{left.name} == {right.name}";
Expression comparison = (left == right);
allExpressionsList.Add(comparison);
}
break;
case comparisonOperators.greater:
{
string name = $"{left.name} > {right.name}";
Expression comparison = (left > right);
allExpressionsList.Add(comparison);
}
break;
}
}
}
}
}
allExpressions = allExpressionsList.ToArray();
}
public void dump()
{
Console.WriteLine("Complete model:");
Console.WriteLine($"{baseVariables.Count} variables:");
foreach (Term term in baseVariables)
{
Console.WriteLine(term.ToString());
}
Console.WriteLine($"{allTerms.Count} arithmetic expressions:");
foreach (Term term in allTerms)
{
Console.WriteLine(term.ToString());
}
Console.WriteLine($"{allExpressions.Length} comparison expressions:");
foreach (Expression expr in allExpressions)
{
Console.WriteLine(expr.name);
}
}
public void solveModel(Solver solver)
{
// Could add limits if things are taking too long
// SearchLimit limits = solver.MakeLimit(maxTime, Int64.MaxValue, Int64.MaxValue, maxSolutions, true, true);
// Start a new solution search
CustomDecisionBuilder decisionBuilder = new CustomDecisionBuilder(allExpressions, baseVariables);
solver.NewSearch(decisionBuilder);
bool solutionFound = true;
int nsols = 0;
string indent = " ";
while (solutionFound && (nsols < numberOfPuzzlesToCreate))
{
solutionFound = solver.NextSolution();
if (solutionFound)
{
// TODO: Verify that the given expressions really result in a unique solution for the base variables.
// TODO: Check if expressions can be removed and still result in a unique solution.
Console.WriteLine($"\r\nPuzzle {nsols}:");
Console.Write(indent);
Console.WriteLine($"Find {numberOfVariables} numbers from {minNumber} to {maxNumber} such that");
foreach (Expression expr in decisionBuilder.decisionsMadeFor)
{
if (expr.isEnforced)
{
Console.Write(indent);
Console.WriteLine(expr.ToString());
}
}
Console.WriteLine("Solution:");
foreach (Term term in baseVariables)
{
Console.Write(indent);
Console.WriteLine(term.ToString());
}
nsols++;
}
}
// Solver-Statistik ausgeben
Console.WriteLine("Solver:" + solver.ToString());
Console.WriteLine(string.Format("State: {0}", solver.State()));
Console.WriteLine(string.Format("Number of solutions: {0}", solver.Solutions()));
Console.WriteLine(string.Format("WallTime: {0} ms", solver.WallTime()));
Console.WriteLine(string.Format("Failures: {0}", solver.Failures()));
Console.WriteLine(string.Format("Branches: {0} ", solver.Branches()));
Console.WriteLine(string.Format("Memory: {0} ", Solver.MemoryUsage()));
}
}
enum comparisonOperators
{
less,
equals,
greater
}
enum arithmeticOperators
{
plus,
times
}
}
Class CustomDecisionBuilder
You may have noted that the base variables a, b, c ... are not part of the decision variables. The decision variables only control which expressions are active in the puzzle; the base variables are a result of those decisions.
This custom decision builder is designed to keep returning decisions on which expressions to enforce until the base variables are all bound (i.e. have specific values). After (almost) each decision, the Google solver propagates the constraints, in other words it makes inferences about the domains of all the variables in the problem to fulfill the constraints. This includes the enforced variables as well! (For example, if a < b was enforced, the solver can infer that b < a can no longer be enforced and sets its enforced variable to 0. When the domains of the base variables have been reduced to a single value (i.e. they are Bound()), the expressions being enforced are sufficient to define a puzzle having a unique solution, so the decision builder stops returning decisions to indicate that a new puzzle has been found.
The new Google.OrTools.Sat solver does not have custom decision builders, which I believe is a great lacking, so this approach could not be used with it.
Their absence completely prevents its use in the real-world application with hundreds of users for which I productively use the old solver.
The propagation is not perfect, for example after having enforced a < b, the propagation did not recognize that (a + 2) < (b + 2) must also then always be true (see puzzle 5 in the output below). Therefore, an additional decision was taken to also enforce (a + 2) < (b + 2). The puzzles generated with this version are therefore overconstrained and contain redundant constraints. Some constraints could be removed and still have a unique solution for the puzzle.
The relevant method in the class is NextWrapper(Solver) which
checks if all the base variables are bound and returns null to indicate "no more decisions"
otherwise looks for the first undecided expression and
returns a decision to enforce the expression.
So it basically continues enforcing expressions until a new puzzle with a unique solution has been found.
When a solution has been found, the Google solver cuts that branch off of the search space by refuting the last decision. At that point, it also restores the domains of all the variables to the situation before the decision was made. Here, that means that the last expression to be enforced is freed up again, the base variables now have enlarged domains again, and the decision builder proceeds to select other expressions to enforce.
Here is the code of the custom decision builder.
using System;
using System.Collections.Generic;
using Google.OrTools.ConstraintSolver;
namespace SO69225141_create_puzzles
{
/// <summary>
/// Custom decision builder for puzzle creation expression enforcing variables.
/// Decides to enforce expressions until the base variables are all bound, then no more decisions
/// </summary>
internal class CustomDecisionBuilder : DecisionBuilder
{
Expression[] expressions;
List<Term> baseVariables;
private HashSet<Expression> decisionsMadeFor_ = new HashSet<Expression>();
public CustomDecisionBuilder(Expression[] expressions_, List<Term> baseVariables_)
{
this.expressions = expressions_ ?? throw new ArgumentNullException(nameof(expressions_));
this.baseVariables = baseVariables_ ?? throw new ArgumentNullException(nameof(baseVariables_));
}
public IEnumerable<Expression> decisionsMadeFor
{
get
{
return decisionsMadeFor_;
}
}
public override Decision NextWrapper(Solver solver)
{
// Are the base variables all bound?
bool allBaseVariablesBound = true;
foreach (Term baseVariable in baseVariables)
{
if (!baseVariable.intVar.Bound())
{
allBaseVariablesBound = false;
break;
}
}
if (allBaseVariablesBound)
{
// Done with decisions, all variables are bound.
return null;
}
// Find first expression that is neither enforced nor already fixed via propagation
for (int i = 0; i < expressions.Length; i++)
{
if (!expressions[i].Bound())
{
decisionsMadeFor_.Add(expressions[i]);
return solver.MakeAssignVariableValue(expressions[i].enforcedVar, 1L);
}
}
// All variables bound !!!!????, no more decisions to make.
// This means all expressions are definitely enforced (or definitely not)
// but the solution is still not unique. Consider this as failure
return solver.MakeFailDecision();
}
}
}
Here is a sample output of the problem:
Puzzle 0:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b < c
Solution:
a = 1
b = 2
c = 3
Puzzle 1:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
b < (2 * a)
Solution:
a = 2
b = 3
c = 3
Puzzle 2:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
b == (2 * a)
Solution:
a = 1
b = 2
c = 2
Puzzle 3:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
c < (2 * a)
Solution:
a = 2
b = 3
c = 3
Puzzle 4:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
c == (2 * a)
Solution:
a = 1
b = 2
c = 2
Puzzle 5:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
(a + 2) < (b + 2)
(a + 2) < (2 * b)
(a + 2) < (c + 2)
(a + 2) < (2 * c)
(2 * a) == b
Solution:
a = 1
b = 2
c = 2
Puzzle 6:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
(a + 2) < (b + 2)
(a + 2) < (2 * b)
(a + 2) < (c + 2)
(a + 2) < (2 * c)
(2 * a) > b
Solution:
a = 2
b = 3
c = 3
Puzzle 7:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
(a + 2) < (b + 2)
(a + 2) < (2 * b)
(a + 2) < (c + 2)
(a + 2) < (2 * c)
(2 * a) == c
Solution:
a = 1
b = 2
c = 2
Puzzle 8:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
(a + 2) < (b + 2)
(a + 2) < (2 * b)
(a + 2) < (c + 2)
(a + 2) < (2 * c)
(2 * a) > c
Solution:
a = 2
b = 3
c = 3
Puzzle 9:
Find 3 numbers from 1 to 3 such that
a < b
a < c
b == c
b < (a + 2)
c < (a + 2)
(a + 2) < (b + 2)
(a + 2) < (2 * b)
(a + 2) < (c + 2)
(a + 2) < (2 * c)
(2 * a) < (b + 2)
(2 * a) < (2 * b)
(2 * a) < (c + 2)
(2 * a) < (2 * c)
(b + 2) == (c + 2)
(b + 2) < (2 * c)
Solution:
a = 2
b = 3
c = 3
The method isn't 100% accurate, because the propagation done by the solver afer a decision is not perfect. Some dependencies can only be discovered by backtracking, and it might also not be recognized that a particular expression has become redundant. For this reason, the puzzles generated may have more expressions in them than would be strictly necessary to find a unique solution.
Although I did a cursory check of the output, the program is not 100% tested by any means. It has a couple of TODO's in it to verify the solutions in place and remove expressions that are redundant...
For example, here is an or expression:
c = f1 == 0 or f1 - f0 > th
Here is the compiled C code:
__pyx_t_24 = (__pyx_v_f1 == 0);
if (!__pyx_t_24) {
} else {
__pyx_t_23 = __pyx_t_24;
goto __pyx_L5_bool_binop_done;
}
__pyx_t_24 = ((__pyx_v_f1 - __pyx_v_f0) > __pyx_v_th);
__pyx_t_23 = __pyx_t_24;
__pyx_L5_bool_binop_done:;
__pyx_v_c = __pyx_t_23;
Why not output this?
__pyx_v_c = (__pyx_v_f1 == 0) || ((__pyx_v_f1 - __pyx_v_f0) > __pyx_v_th)
is the goto version faster than ||?
Using the following two files:
test.c:
int main(int argc, char** argv) {
int c, f0, f1, th;
int hold, hold1;
f0 = (int) argv[1];
f1 = (int) argv[2];
th = (int) argv[3];
hold1 = (f0 == 0);
if (!hold1) {
} else {
hold = hold1;
goto done;
}
hold1 = (f1 - f0 > th);
hold = hold1;
done: c = hold;
return c;
}
test2.c:
int main(int argc, char** argv) {
int c, f0, f1, th;
f0 = (int) argv[1];
f1 = (int) argv[2];
th = (int) argv[3];
c = (f1 == 0) || (f1 - f0 > th);
return c;
}
I had to assign f0, f1 and th to something so that the compiler would not just return 1 as the C specification states that ints are initialized to 0 and f1 == 0 would yield true, and thus the whole boolean statement would yield true and the assembly would be:
main:
.LFB0:
.cfi_startproc
.L2:
movl $1, %eax
ret
.cfi_endproc
Compiling using GCC with -S -O2 flags (optimization enabled), both test.s and test2.s become:
main:
.LFB0:
.cfi_startproc
movl 8(%rsi), %edx
movq 16(%rsi), %rdi
movl $1, %eax
movq 24(%rsi), %rcx
testl %edx, %edx
je .L2
subl %edx, %edi
xorl %eax, %eax
cmpl %ecx, %edi
setg %al
.L2:
rep
ret
.cfi_endproc
So it unless you disable optimizations, in which the one with goto would have about 50% more instructions, the result would be the same.
The reason the output C code is ugly is because of the way the interpreter visits the nodes in the AST. When an or node is visited, the interpreter first evaluates the first parameter and then the second. If the boolean expression was much more complex, it would be much more easier to parse. Imagine calling a lambda function that returns a boolean (I'm not sure if Cython supports this); the interpreter would have to follow the structure:
hold = ... evaluate the lambda expression...
if (hold) {
result = hold;
goto done; // short circuit
}
hold = ... evaluate the second boolean expression...
done:
...
It would be a daunting task to optimize during the interpreting phase and thus Cython won't even bother.
If my understanding of C is correct (and I last used C many years ago , so it may be rusty) the '||' ( OR ) operator in C only return Boolean values ( that is 0 for False or 1 for True ) . If this is correct , then it's not about if goto is faster or slower .
The || would give different results than the goto code . This is because of how 'or' in Python works , let's take an example -
c = a or b
In the above statement first as value is evaluated , If it is a true like value that value is returned from the or expression (not true or 1 , but as value) , if the value is false like ( false like values in Python are 0 , empty string , empty lists, False , etc) then b's value is evaluated and it is returned . Please note 'or' returns the last evaluated value , and not True(1) or False(0).
This is basically useful when you want to set default values like -
s = d or 'default value'
I'm working on making a URL shortener for my site, and my current plan (I'm open to suggestions) is to use a node ID to generate the shortened URL. So, in theory, node 26 might be short.com/z, node 1 might be short.com/a, node 52 might be short.com/Z, and node 104 might be short.com/ZZ. When a user goes to that URL, I need to reverse the process (obviously).
I can think of some kludgy ways to go about this, but I'm guessing there are better ones. Any suggestions?
ASCII to int:
ord('a')
gives 97
And back to a string:
in Python2: str(unichr(97))
in Python3: chr(97)
gives 'a'
>>> ord("a")
97
>>> chr(97)
'a'
If multiple characters are bound inside a single integer/long, as was my issue:
s = '0123456789'
nchars = len(s)
# string to int or long. Type depends on nchars
x = sum(ord(s[byte])<<8*(nchars-byte-1) for byte in range(nchars))
# int or long to string
''.join(chr((x>>8*(nchars-byte-1))&0xFF) for byte in range(nchars))
Yields '0123456789' and x = 227581098929683594426425L
What about BASE58 encoding the URL? Like for example flickr does.
# note the missing lowercase L and the zero etc.
BASE58 = '123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ'
url = ''
while node_id >= 58:
div, mod = divmod(node_id, 58)
url = BASE58[mod] + url
node_id = int(div)
return 'http://short.com/%s' % BASE58[node_id] + url
Turning that back into a number isn't a big deal either.
Use hex(id)[2:] and int(urlpart, 16). There are other options. base32 encoding your id could work as well, but I don't know that there's any library that does base32 encoding built into Python.
Apparently a base32 encoder was introduced in Python 2.4 with the base64 module. You might try using b32encode and b32decode. You should give True for both the casefold and map01 options to b32decode in case people write down your shortened URLs.
Actually, I take that back. I still think base32 encoding is a good idea, but that module is not useful for the case of URL shortening. You could look at the implementation in the module and make your own for this specific case. :-)
apparently I'm late to the party, jus like to share a snippet I use very often.
/**
* 62 = 26 + 26 +10
*
* #param id
* #return
*/
public String base62(long id) {
StringBuilder sb = new StringBuilder();
while (id >= 62) {
int remainer = (int) (id % 62);
id = id / 62;
sb.append(index2char(remainer));
}
sb.append(index2char(id));
return sb.reverse().toString();
}
public long reverseBase62(String s) {
long r = 0;
for (int i = 0; i < s.length(); i++) {
r = r * 62;
int index = char2index(s.charAt(i));
if (index == -1) {
throw new IllegalArgumentException(
String.format("[%s] is in malformation, should only contain 0~9, a~z, A~Z", s));
}
r += index;
}
return r;
}
private char index2char(long index) {
if (index < 10) {
return (char) ('0' + index);
}
if (index < 36) {
return (char) ('a' + index - 10);
}
return (char) ('A' + index - 36);
}
private int char2index(char c) {
if ('0' <= c && c <= '9') {
return c - '0';
}
if ('a' <= c && c <= 'z') {
return c - 'a' + 10;
}
if ('A' <= c && c <= 'Z') {
return c - 'A' + 36;
}
return -1;
}