Skip to content

Commit 14da03a

Browse files
fix: make the test suite pass (12 failures tracked by #47) (#49)
## Summary This fixes the 12 test failures that #47 tracks. Each fix is in `src/` at the root cause, or in a test that called an API that never existed. No test is skipped, deleted or loosened. | # | Failure | Root cause | Fix | |---|---|---|---| | 1 | Statistics Parsing / Parse numeric statistics, `runtests.jl:876–881` (3 fail + 3 `KeyError`) | `parse_statistics` matched only a single-pair list `(:key value)` that closes at once. Solver statistics come as one list of many pairs, and Z3 prints it over several lines. | It now finds each `(:` keyword list and reads every `:key value` pair in it. Quoted strings are consumed first, so a `(:…)` inside an error message is not read as a list. Against real Z3 4.13.3 `(get-info :all-statistics)` output it returns 13 entries. | | 2 | evaluate / Boolean logic, `runtests.jl:931–932` | Julia parses `p && q` and `p \|\| q` as `Expr(:&&)` / `Expr(:\|\|)`, not as calls. The `:call` branch never ran, so the substituted `Expr` came back. | `evaluate` computes `&&` and `\|\|` once every operand is a `Bool`. If any operand is still unknown, it returns the substituted `Expr`. | | 3 | Unsat core extraction (Z3), `runtests.jl:1153` | Two defects. First, `set_option!(ctx, :produce_unsat_cores, true)` sent the option name with underscores, which is not the SMT-LIB name (cvc5 1.1.2 answers `unsupported`). Second, plain `check_sat(ctx)` never sent `(get-unsat-core)`, even with the option on. | The `Symbol` method of `set_option!` maps `_` to `-`. `check_sat`'s `get_unsat_core` now defaults to the context's `produce-unsat-cores` option. An explicit `get_unsat_core=true/false` behaves as before. Z3 now returns `[:pos, :neg]`. | | 4 | e2e `declare → assert → check_sat`, `e2e_test.jl:23`; e2e `to_smtlib_script`, `e2e_test.jl:41`; property `assert!` count, `property_test.jl:31` (3 `UndefVarError`) | **These tests were wrong.** `declare_const` and `to_smtlib_script` have never existed in `src/`. Commit `ed8eff1` added them only as calls, in the tests and in `benches/benchmarks.jl`. | The tests and benches now use the exported, documented API: `declare(ctx, name, sort)` (README: "`declare(ctx, name, sort)` — Declare a constant or function") and `SMTLib.build_script(ctx, get_model)`. The "Script Building" testset already exercises `build_script`. | **Why fix 4 changes the tests rather than adding `declare_const`.** #47 leaves the choice open. Adding the two functions would give the package a second name for `declare` and a public wrapper around an internal function, and only test code from a single commit would use them. Keeping `declare` as the one public way to declare a constant is the simpler choice. The alternative, adding and exporting `declare_const(ctx, name, ::Symbol)` and `to_smtlib_script`, is easy to do if the owner prefers it. **No assertion is weakened, and two are stronger:** - *"unknown sort raises an error"* passed vacuously before. The `UndefVarError` for `declare_const` satisfied `@test_throws Exception`. The test now asserts that an unmapped type (`Vector{Int}`) raises a `MethodError` and that no declaration is recorded. I checked that `declare(ctx, :z, Vector{Int})` really throws `MethodError: no method matching smt_type(::Type{Vector{Int64}})`. - The pipeline test now also checks that exactly three `(assert` forms are emitted, as its own comment says. `CHANGELOG.adoc` gains a `=== Fixed` section under `[Unreleased]`. **Second commit (`318efef`), from CodeRabbit's review.** A quoted string that is not a pair's value could leak a false pair. On `6604a9c`, `(:a 1 "stray :time 99")` gave `time => 99`; I reproduced this. CodeRabbit's own example, `(:reason "message :time 99")`, was already parsed correctly, because there the string is a value and is consumed whole. The pair search now consumes any stray string whole as well. The commit adds 9 regression tests: - Z3's multi-line layout; - quoted values containing parentheses and `:key value` text; - the stray string; - an error message containing `(:foo bar)`; - a keyword value. It also replaces the tautological `isempty(stats) || !isempty(stats)` in "Empty output" with `isempty(stats)`. **Third commit (`a6c6bc6`): make the CI matrix able to run the tests.** #47's last acceptance criterion is *"`Pkg.test()` passes on the CI matrix (1.10, 1.11, nightly × ubuntu/macos/windows)"*. Two things stopped that, on `main` and on this branch before this commit: - Every `test (*)` job stopped at **Build package** with `expected package AcceleratorGate [59f742c7] to be registered`. `julia-buildpkg` adds only General, and AcceleratorGate is in `hyperpolymath/julia-professional-registry`. `CI.yml` now passes that registry to `julia-buildpkg` through its existing `localregistry` input; the test step reuses the same depot. The registry needs no credentials: #50's Release run passed its "Add registries" step on a hosted runner. - CI installs neither z3 nor cvc5. Four testsets (the three e2e `SMTContext` testsets and the property `assert!` count) only build SMT-LIB text, but they called `SMTContext()`, which looks a solver up on `PATH` and raises `No SMT solver found. Install z3 or cvc5.` These calls were already on `main`. They now pass `SMTContext` a solver record that is never run (`SMTSolver(:z3, "/nonexistent/z3", "never run")`). `declare`, `assert!` and `build_script` never read the context's solver; only `run_solver` does. The first of them is titled "(no solver)" and its comment says it "must not error even without a solver installed", so this makes the test do what it says. Closes #47. All nine `CI / test (*)` jobs on this head (`a6c6bc6`) are green: run 37912491992. Eight passed on the first attempt. `test (nightly, macos-latest)` first failed at **Setup Julia**, before any package code ran, with `getaddrinfo ENOTFOUND julialangnightlies-s3.julialang.org`. That is a runner DNS failure; the host resolved and answered 200 from here. The job was re-run unchanged and passed (job 113764403648, 2026-10-09T09:50Z). 1.11, nightly, macOS and Windows are verified by CI alone; I ran 1.10 and 1.12 on Linux. ## Type of change - [x] 🐛 Bug fix (non-breaking change that fixes an issue) - [ ] ✨ New feature (non-breaking change that adds functionality). No new public function. - [ ] 💥 Breaking change (would change existing behaviour). There are two small behaviour changes, both towards SMT-LIB-correct output. A `Symbol` option key now has `_` mapped to `-`; the old underscore name was not a valid SMT-LIB option name. Plain `check_sat` now requests the unsat core when the context already enables `produce-unsat-cores`. String keys and explicit keyword arguments are unchanged. - [ ] 🕳️ Soundness fix. Not a checker or proof false-negative. - [ ] 📖 Documentation. The only doc changes are docstrings and the CHANGELOG entry. - [ ] 🧹 Refactor / tech debt - [ ] ⚡ Performance - [x] 🔧 Build / CI / tooling. `CI.yml`: `julia-buildpkg` gains the `localregistry` input (third commit). ## 📌 New pins Head SHA: **a6c6bc6effde7ddb5cd5b4f8d173bd671e69a4c4** No pins added or changed. The `julia-buildpkg` pin (`e3eb439f…`, v1.7.0) is unchanged; only its `with:` input is new. There are no `actions.lock` changes (the repo has none) and no `Project.toml` compat changes. `Manifest.toml` is gitignored. `CI.yml` stays in block YAML, because this repo's gates do not yet meet the KYAML preconditions. ## How has this been verified? I ran `Pkg.test()` from a throwaway depot (General plus `hyperpolymath/julia-professional-registry`), with z3 4.13.3 and cvc5 1.1.2 on `PATH`, so the solver-dependent testsets, including the Z3 unsat-core test, actually ran: | Julia | `main` @ 487c7ab | 6604a9c (first commit) | **318efef (second commit)** | |---|---|---|---| | 1.12.6 | 818 pass / 6 fail / 6 error (830) | 864 pass (864) | **873 pass** (873) | | 1.10.10 | 818 pass / 6 fail / 6 error (830) | 864 pass (864) | **873 pass** (873) | | 1.11 | not run, not installed on this machine | not run | not run | From `main` to the first commit, the total rises by 34: - The property test now runs all 30 iterations instead of erroring on the first (+29). - The e2e pipeline testset runs 4 checks instead of 1 error, including the new assert count (+3). - The e2e `build_script` testset runs 2 checks instead of 1 error (+1). - The unknown-sort testset gains the "no declaration recorded" check (+1). The second commit adds 9 tests to "Statistics Parsing" (864 → 873). They are described below. I also checked `parse_statistics` directly against these inputs: - real Z3 multi-line statistics (13 keys; `time` → `0.0`, `max-memory` → `17.82`, `num-allocs` → `35972`); - the test string; - a quoted value containing parentheses (`(:name "Z3 (x)")`); - a keyword value (`(:reason-unknown :incomplete)`); - an error message containing `(:foo bar)`, which correctly yields nothing. The new `check_sat` default has one path that no test covers. When `produce-unsat-cores` is on and the problem is satisfiable, plain `check_sat` now also sends `(get-unsat-core)`. I ran this case directly on both solvers: - **z3:** `status=sat`, model `x => 6`, followed by `(error "… unsat core is not available")`. - **cvc5:** `status=sat`, model `x => 6`, followed by `(error "Cannot get unsat core unless in unsat mode.")`. In both cases the status and the model parse correctly. **The CI condition: no solver on `PATH`.** For the third commit I also ran `Pkg.test()` with a `PATH` that has neither z3 nor cvc5 (`command -v z3 cvc5` finds nothing; `Sys.which` is the package's lookup): | Julia, no solver | 318efef | **a6c6bc6 (head)** | |---|---|---| | 1.10.10 | 718 pass / 4 error (722), rc=1 | **757 pass** (757), rc=0 | | 1.12.6 | not run | **757 pass** (757), rc=0 | The 4 errors were the `SMTContext()` calls described above. 757 = 718 + 39: the four testsets now run their 5 + 2 + 2 + 30 checks. With z3 and cvc5 on `PATH`, the head still gives **873 pass** (873) on 1.10.10, the same as `318efef`. CI note: the solver-dependent testsets ("Solver Integration", the Z3 optimisation and unsat-core tests) log `No SMT solver found, skipping …` and run only where a solver is on `PATH`, so CI does not exercise them. Installing a solver in CI is a separate change. ## Checklist - [x] My commits are **signed**. All three commits are SSH-signed: `git log --format=%G?` gives `G` for each. GitHub reported `verified: true` (`valid`) for the first two, and the GraphQL commit `signature` reports `VALID` (signer `hyperpolymath`) for the third. - [x] I ran the project's own checks/tests locally and they pass. `Pkg.test()` passes on 1.12.6 and 1.10.10, with and without solvers, as above. 1.11 was not run. `actionlint` passes on `CI.yml`. - [ ] New files carry the correct `SPDX-License-Identifier`. No new files; every touched file keeps its existing header. - [x] Docs are updated, and no public claim now overstates what the code does. The docstrings for `parse_statistics`, `evaluate`, `check_sat` and both `set_option!` methods describe the new behaviour. The two `set_option!` methods had no docstring before. - [x] I have not introduced a soundness hole (or I have flagged where I might have). The `check_sat` default only adds a `(get-unsat-core)` request when the user has turned unsat cores on. ## Notes for reviewers - **Decision for the owner:** #47 offers two paths for fix 4. This PR keeps `declare` as the only public way to declare a constant (see above). Say if you want `declare_const` added instead. - **Findings, not fixed here (out of scope):** - CI installs no SMT solver, so the solver-dependent testsets are skipped there (see the CI note). - `parse_unsat_core` reads one line. cvc5 1.1.2 prints the core one label per line (`(` / `pos` / `neg` / `)`), so cvc5 cores come back empty. The test is Z3-only, so this is not one of the 12 failures. - cvc5's statistics format, `(:all-statistics (("name" v) …))`, is a nested list. `parse_statistics` deliberately skips it rather than producing wrong values. - The estate docstring scanner (`standards/.githooks/docstring-scan.sh`) skips `.jl` files, so it reported no verdict (`skipped=4`). I checked the docstrings by hand. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016XJoh8YA2sCWQmq93CF8RX --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 487c7ab commit 14da03a

7 files changed

Lines changed: 128 additions & 30 deletions

File tree

‎.github/workflows/CI.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,12 @@ jobs:
2525
version: ${{ matrix.julia-version }}
2626
- name: Cache Julia artifacts
2727
uses: julia-actions/cache@e32267b41e0fa7df82a3d53e8c7273620d52c586 # v3.4.0
28+
# AcceleratorGate is registered in the hyperpolymath registry, not in
29+
# General. The registry is added to the depot that the test step reuses.
2830
- name: Build package
2931
uses: julia-actions/julia-buildpkg@e3eb439fad4f9aba7da2667e7510e4a46ebc46e1 # v1.7.0
32+
with:
33+
localregistry: https://github.com/hyperpolymath/julia-professional-registry
3034
- name: Run tests
3135
uses: julia-actions/julia-runtest@d0c4f093badade621cd041bba567d1e832480ac2 # v1.10.0
3236
- name: Process coverage

‎CHANGELOG.adoc‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,15 @@ The format is based on Keep a Changelog and this project follows Semantic Versio
1717
- Roadmap rewritten to a concrete Must/Intend/Like delivery model.
1818
- `.machine_readable/STATE.scm` aligned with package version and current roadmap intent.
1919

20+
=== Fixed
21+
- `parse_statistics` reads every `:key value` pair of a multi-pair keyword list, including Z3's multi-line `(get-info :all-statistics)` block; it previously matched only single-pair `(:key value)` lists (#47).
22+
- `evaluate` computes `a && b` and `a || b`, which Julia parses as `Expr(:&&)` / `Expr(:||)` rather than calls (#47).
23+
- `set_option!` with a `Symbol` key translates `_` to `-`, so `:produce_unsat_cores` emits the SMT-LIB option `:produce-unsat-cores` (#47).
24+
- `check_sat` requests the unsat core by default when the context has `produce-unsat-cores` set (#47).
25+
- The e2e and property tests and the benchmarks called `declare_const` and `to_smtlib_script`, which never existed; they now use `declare` and `build_script` (#47).
26+
- The tests that only build SMT-LIB text give `SMTContext` a solver record that is never run, so they pass where no SMT solver is installed, as on CI (#47).
27+
- CI adds `hyperpolymath/julia-professional-registry`, where AcceleratorGate is registered, so the `test` jobs can resolve the package (#47).
28+
2029
== [0.1.0] - 2026-01-25
2130

2231
=== Added

‎benches/benchmarks.jl‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,17 +24,17 @@ SUITE["context"]["create"] = @benchmarkable SMTContext()
2424

2525
SUITE["context"]["declare_assert_script_10"] = @benchmarkable begin
2626
ctx = SMTContext()
27-
declare_const(ctx, :x, :Int)
27+
declare(ctx, :x, Int)
2828
for i in 1:10
2929
assert!(ctx, :(x > 0))
3030
end
31-
to_smtlib_script(ctx)
31+
SMTLib.build_script(ctx, false)
3232
end
3333

3434
SUITE["context"]["declare_10_consts"] = @benchmarkable begin
3535
ctx = SMTContext()
3636
for i in 1:10
37-
declare_const(ctx, Symbol("v$i"), :Int)
37+
declare(ctx, Symbol("v$i"), Int)
3838
end
3939
end
4040

‎src/SMTLib.jl‎

Lines changed: 65 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -969,10 +969,27 @@ function set_option!(ctx::SMTContext, key::String, value::String)
969969
nothing
970970
end
971971

972+
"""
973+
set_option!(ctx::SMTContext, key::Symbol, value)
974+
975+
Set a solver option from a Julia `Symbol` key and any value.
976+
977+
SMT-LIB2 option names are hyphenated (`:produce-unsat-cores`), but a Julia
978+
symbol literal cannot contain `-`, so underscores in `key` are translated to
979+
hyphens: `set_option!(ctx, :produce_unsat_cores, true)` emits
980+
`(set-option :produce-unsat-cores true)`. The value is stringified. Use the
981+
`String`-key method to pass an option name verbatim.
982+
"""
972983
function set_option!(ctx::SMTContext, key::Symbol, value::Any)
973-
set_option!(ctx, string(key), string(value))
984+
set_option!(ctx, replace(string(key), '_' => '-'), string(value))
974985
end
975986

987+
"""
988+
set_option!(ctx::SMTContext, key::String, value)
989+
990+
Set a solver option with a verbatim option name and any value, which is
991+
stringified (e.g. `set_option!(ctx, "random-seed", 42)`).
992+
"""
976993
function set_option!(ctx::SMTContext, key::String, value::Any)
977994
set_option!(ctx, key, string(value))
978995
end
@@ -982,7 +999,7 @@ end
982999
# ============================================================================
9831000

9841001
"""
985-
check_sat(ctx::SMTContext; get_model=true, get_unsat_core=false) -> SMTResult
1002+
check_sat(ctx::SMTContext; get_model=true, get_unsat_core=<produce-unsat-cores option>) -> SMTResult
9861003
9871004
Check satisfiability of the current context assertions.
9881005
@@ -993,7 +1010,9 @@ parses the model. If `get_unsat_core` is true, requests the unsat core.
9931010
# Arguments
9941011
- `ctx::SMTContext`: The solving context.
9951012
- `get_model::Bool`: Request model on sat (default `true`).
996-
- `get_unsat_core::Bool`: Request unsat core on unsat (default `false`).
1013+
- `get_unsat_core::Bool`: Request unsat core on unsat. Defaults to `true` when
1014+
the context already has `produce-unsat-cores` set to `true` (for example via
1015+
`set_option!(ctx, :produce_unsat_cores, true)`), otherwise `false`.
9971016
9981017
# Returns
9991018
An `SMTResult` with status, model, unsat core, statistics, and raw output.
@@ -1004,7 +1023,8 @@ result = check_sat(ctx)
10041023
result = check_sat(ctx; get_model=true, get_unsat_core=true)
10051024
```
10061025
"""
1007-
function check_sat(ctx::SMTContext; get_model::Bool=true, get_unsat_core::Bool=false)
1026+
function check_sat(ctx::SMTContext; get_model::Bool=true,
1027+
get_unsat_core::Bool=get(ctx.solver_options, "produce-unsat-cores", "false") == "true")
10081028
# If unsat core requested, ensure the option is set
10091029
if get_unsat_core
10101030
ctx.solver_options["produce-unsat-cores"] = "true"
@@ -1234,23 +1254,45 @@ function parse_result(output::String)
12341254
SMTResult(status, model, unsat_core, stats, output)
12351255
end
12361256

1257+
# A keyword list `(:key value :key value ...)` as printed by `get-info`: it may
1258+
# span several lines and hold quoted strings (which may contain parentheses),
1259+
# but no nested parenthesised values. The first alternative consumes a quoted
1260+
# string on its own, so text inside a string (e.g. an error message) is never
1261+
# read as a list.
1262+
const STATISTICS_LIST_REGEX = r"\"(?:[^\"]|\"\")*\"|\(:(?:[^()\"]|\"(?:[^\"]|\"\")*\")*\)"
1263+
1264+
# One `:key value` pair inside such a list. The key must start a token (it
1265+
# follows `(` or whitespace); the value is a quoted string or a single atom,
1266+
# optionally itself a keyword (`:key :value`). The first alternative consumes a
1267+
# quoted string that is not a value, so `:key value` text inside it is never
1268+
# read as a pair; such a match has no captures.
1269+
const STATISTICS_PAIR_REGEX = r"\"(?:[^\"]|\"\")*\"|(?<![^\s(]):([^\s()\":]+)\s+(\"(?:[^\"]|\"\")*\"|:?[^\s()\"]+)"
1270+
12371271
"""
12381272
parse_statistics(output::String) -> Dict{String, Any}
12391273
12401274
Parse SMT-LIB2 statistics output (e.g. from `(get-info :all-statistics)`).
1275+
1276+
Every keyword list in `output`, such as Z3's multi-line
1277+
`(:time 0.01 :memory 12.5 :conflicts 0)`, contributes one entry per
1278+
`:key value` pair. Values are converted with `parse_smt_value`, so numbers
1279+
become `Int` or `Float64`; a keyword value (`:key :value`) is stored without
1280+
its leading colon. Lists whose values are nested S-expressions are skipped.
12411281
"""
12421282
function parse_statistics(output::String)
12431283
stats = Dict{String, Any}()
1244-
# Support both (:key value) and (:key :value)
1245-
# Match (:key followed by value until next ) or end of line
1246-
for m in eachmatch(r"\(:([a-zA-Z0-9\._-]+)\s+((?:\d+\.\d+)|\d+|:?[a-zA-Z0-9\._-]+)\)", output)
1247-
key = m.captures[1]
1248-
val_str = m.captures[2]
1249-
# Strip leading : from value if present
1250-
if startswith(val_str, ":")
1251-
val_str = val_str[2:end]
1284+
for list in eachmatch(STATISTICS_LIST_REGEX, output)
1285+
startswith(list.match, '"') && continue # a string literal, not a list
1286+
for m in eachmatch(STATISTICS_PAIR_REGEX, list.match)
1287+
key = m.captures[1]
1288+
key === nothing && continue # a stray string literal, not a pair
1289+
val_str = m.captures[2]
1290+
# Strip leading : from a keyword value
1291+
if startswith(val_str, ":")
1292+
val_str = val_str[2:end]
1293+
end
1294+
stats[key] = parse_smt_value(val_str)
12521295
end
1253-
stats[key] = parse_smt_value(val_str)
12541296
end
12551297
stats
12561298
end
@@ -2292,13 +2334,23 @@ end
22922334
22932335
Evaluate a Julia expression against a model dictionary.
22942336
Replaces variables in `expr` with their values from `model` and evaluates.
2337+
2338+
Function calls on known operators are computed once their arguments are
2339+
values. The short-circuit forms `a && b` and `a || b`, which Julia parses as
2340+
`Expr(:&&)` / `Expr(:||)` rather than calls, are computed once every operand
2341+
is a `Bool`. Anything that cannot be computed is returned as a substituted
2342+
`Expr`.
22952343
"""
22962344
function evaluate(model::Dict{Symbol, Any}, expr)
22972345
if expr isa Symbol
22982346
return get(model, expr, expr)
22992347
elseif expr isa Expr
23002348
# Recursively evaluate args
23012349
new_args = [evaluate(model, a) for a in expr.args]
2350+
# `&&` and `||` are syntax (their own Expr heads), not calls
2351+
if expr.head in (:&&, :||) && all(a -> a isa Bool, new_args)
2352+
return expr.head == :&& ? all(new_args) : any(new_args)
2353+
end
23022354
# Attempt to evaluate the expression if all arguments are literals
23032355
if expr.head == :call
23042356
func = new_args[1]

‎test/e2e_test.jl‎

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,11 @@ using SMTLib
77

88
@testset "E2E Pipeline Tests" begin
99

10+
# These tests only build SMT-LIB text, which never runs the solver, so they
11+
# use a solver record that is never executed. `SMTContext()` would look for
12+
# z3 or cvc5 on PATH and fail on a machine that has neither, such as CI.
13+
script_only = SMTSolver(:z3, "/nonexistent/z3", "never run")
14+
1015
@testset "Full SMT-LIB generation pipeline" begin
1116
# Nested arithmetic and logical expression round-trip through to_smtlib
1217
exprs = [
@@ -22,34 +27,37 @@ using SMTLib
2227

2328
@testset "SMTContext declare → assert → check_sat pipeline (no solver)" begin
2429
# These operations must not error even without a solver installed
25-
ctx = SMTContext()
30+
ctx = SMTContext(solver=script_only)
2631
@test ctx isa SMTContext
2732

28-
declare_const(ctx, :x, :Int)
29-
declare_const(ctx, :y, :Int)
33+
declare(ctx, :x, Int)
34+
declare(ctx, :y, Int)
3035
assert!(ctx, :(x > 0))
3136
assert!(ctx, :(y < 10))
3237
assert!(ctx, :(x + y == 5))
3338

3439
# Script should contain all three assertions
35-
script = to_smtlib_script(ctx)
40+
script = SMTLib.build_script(ctx, false)
3641
@test script isa String
3742
@test occursin("declare-const", script)
3843
@test occursin("assert", script)
44+
@test length(collect(eachmatch(r"\(assert", script))) == 3
3945
end
4046

41-
@testset "to_smtlib_script includes check-sat and get-model" begin
42-
ctx = SMTContext()
43-
declare_const(ctx, :a, :Int)
47+
@testset "build_script includes check-sat and get-model" begin
48+
ctx = SMTContext(solver=script_only)
49+
declare(ctx, :a, Int)
4450
assert!(ctx, :(a > 100))
45-
script = to_smtlib_script(ctx; check_sat=true, get_model=true)
51+
script = SMTLib.build_script(ctx, true)
4652
@test occursin("check-sat", script)
4753
@test occursin("get-model", script)
4854
end
4955

5056
@testset "Error handling: unknown sort raises an error" begin
51-
ctx = SMTContext()
52-
@test_throws Exception declare_const(ctx, :z, :UnknownSort123)
57+
# A Julia type with no SMT-LIB sort mapping has no `smt_type` method
58+
ctx = SMTContext(solver=script_only)
59+
@test_throws MethodError declare(ctx, :z, Vector{Int})
60+
@test isempty(ctx.declarations)
5361
end
5462

5563
end

‎test/property_test.jl‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,11 @@ using SMTLib
77

88
@testset "Property-Based Tests" begin
99

10+
# The SMTContext invariant below only builds SMT-LIB text, which never runs
11+
# the solver, so it uses a solver record that is never executed.
12+
# `SMTContext()` would fail on a machine with no z3 or cvc5, such as CI.
13+
script_only = SMTSolver(:z3, "/nonexistent/z3", "never run")
14+
1015
@testset "Invariant: to_smtlib of integer literals is the decimal string" begin
1116
for _ in 1:50
1217
n = rand(0:999)
@@ -30,13 +35,13 @@ using SMTLib
3035

3136
@testset "Invariant: SMTContext assert! count matches calls" begin
3237
for _ in 1:30
33-
ctx = SMTContext()
34-
declare_const(ctx, :x, :Int)
38+
ctx = SMTContext(solver=script_only)
39+
declare(ctx, :x, Int)
3540
n = rand(1:10)
3641
for i in 1:n
3742
assert!(ctx, :(x > 0))
3843
end
39-
script = to_smtlib_script(ctx)
44+
script = SMTLib.build_script(ctx, false)
4045
# Count occurrences of "assert" in the script
4146
count = length(collect(eachmatch(r"\(assert", script)))
4247
@test count == n

‎test/runtests.jl‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -893,7 +893,27 @@ using SMTLib
893893
result = SMTResult(:sat, Dict{Symbol,Any}(), Symbol[],
894894
Dict{String,Any}(), "sat\n")
895895
stats = get_statistics(result)
896-
@test isempty(stats) || !isempty(stats) # May or may not find stats
896+
@test isempty(stats)
897+
end
898+
899+
@testset "Multi-line keyword list (Z3 layout)" begin
900+
output = "unsat\n(:conflicts 1\n :max-memory 17.82\n :num-allocs 35972\n :time 0.00)\n"
901+
stats = SMTLib.parse_statistics(output)
902+
@test stats["conflicts"] == 1
903+
@test stats["max-memory"] == 17.82
904+
@test stats["num-allocs"] == 35972
905+
@test stats["time"] == 0.0
906+
@test length(stats) == 4
907+
end
908+
909+
@testset "Quoted strings are not read as pairs or lists" begin
910+
stats = SMTLib.parse_statistics("(:name \"Z3 (x) :time 99\" :version \"4.13.3\")")
911+
@test stats == Dict{String,Any}("name" => "Z3 (x) :time 99", "version" => "4.13.3")
912+
stats = SMTLib.parse_statistics("(:a 1 \"stray :time 99\")")
913+
@test stats == Dict{String,Any}("a" => 1)
914+
@test isempty(SMTLib.parse_statistics("(error \"line 3: unknown (:foo bar)\")"))
915+
@test SMTLib.parse_statistics("(:reason-unknown :incomplete)") ==
916+
Dict{String,Any}("reason-unknown" => "incomplete")
897917
end
898918
end
899919

0 commit comments

Comments
 (0)