Exact
Exact answers, even for runs of any length
Fractions, not simulations. A game that can last forever still gets its exact odds, and the expected number of rolls.
-- Craps: the shooter keeps rolling until the point comes up again or a 7.
-- That can take any number of rolls, and Haze still gets the exact answer.
enum Phase = ComeOut | OnPoint | Won | Lost
system Craps {
var phase : Phase := ComeOut
var point : Num := 0
step {
let roll := uniform(1, 6) + uniform(1, 6)
match phase {
ComeOut => {
if roll == 7 or roll == 11 { phase := Won }
else if roll == 2 or roll == 3 or roll == 12 { phase := Lost }
else { point := roll; phase := OnPoint }
}
OnPoint => { if roll == point { phase := Won } else if roll == 7 { phase := Lost } }
_ => skip
}
}
}
claim shooter_wins : Craps |= eventually phase == Won -- 244/495
#expect Craps |= steps until phase == Won or phase == Lost -- rolls on average
Rough numbers
Uncertain rates are welcome
Write "loss is about 2%, maybe 10%" and get a range: how strongly what you know guarantees the property, and how strongly it allows it.
-- You rarely know a failure rate exactly. Write what you do know:
-- "loss is about 2%, could be 5%, possibly 10%".
-- Each value comes with how plausible it is (1 = fully plausible).
param loss : Num ~= {0.01: 0.6, 0.02: 1, 0.05: 0.5, 0.1: 0.2}
-- A request is retried up to 3 times; it fails only if all 3 are lost.
input lost1 : Bool ~ flip(loss)
input lost2 : Bool ~ flip(loss)
input lost3 : Bool ~ flip(loss)
-- The answer is a range: how strongly what you know guarantees it
-- (belief) and how strongly it allows it (plausibility). This bound holds
-- at the likely rates but not at the pessimistic 10%, so it fails.
claim delivered : not (lost1 and lost2 and lost3) with prob >= 0.9999
#prob lost1 and lost2 and lost3
Counterexamples
When a guarantee fails, see the run
Theorems must hold on every run. When one doesn't, Haze shows the state and step where it breaks.
-- A payment client retries when a request or its reply is lost.
-- The server charges the card every time a request reaches it.
-- Is a customer ever charged twice?
enum Phase = Sending | Waiting | Done | GaveUp
def loss : Num := 0.05 -- each message is lost 5% of the time
system Client {
var phase : Phase := Sending
var attempts : Num := 0
var charges : Num := 0
step {
match phase {
Sending => {
attempts := attempts + 1
let request_lost := flip(loss)
if request_lost { phase := Waiting }
else {
charges := charges + 1 -- the server charges the card
let reply_lost := flip(loss)
if reply_lost { phase := Waiting } else { phase := Done }
}
}
Waiting => { if attempts < 3 { phase := Sending } else { phase := GaveUp } }
_ => skip
}
}
}
claim delivered : Client |= eventually phase == Done with prob >= 0.99
theorem never_charged_twice : Client |= always charges <= 1
-- Fix: charge once per idempotency key, `if charges == 0 { charges := 1 }`
Testing
Tests ranked by probability
Generate test cases from the most likely runs, measure coverage by probability, and get a statistical test with a known flake rate.
-- Tests ranked by probability: Haze lists the runs of the model from the
-- most likely to the least, so a handful of tests covers almost everything
-- that actually happens, plus the rare runs where the property fails.
enum Phase = Sending | Waiting | Done | GaveUp
def loss : Num := 0.1
system Client {
var phase : Phase := Sending
var attempts : Num := 0
var acked : Bool := false
step {
match phase {
Sending => {
attempts := attempts + 1
let request_lost := flip(loss)
let reply_lost := flip(loss)
if request_lost or reply_lost { phase := Waiting }
else { acked := true; phase := Done }
}
Waiting => { if attempts < 3 { phase := Sending } else { phase := GaveUp } }
_ => skip
}
}
}
-- The 4 most likely runs, the rare failing runs, and a statistical test
-- with a known flake rate for checking the real code.
#tests[4] Client |= after[6] acked
