Skip to content

Bring the tutorials up to v1.2: edit correction, the stored LSI, profile_id, refit (#48) - #53

Merged
sadit merged 1 commit into
mainfrom
docs-tutorial-48
Sep 24, 2026
Merged

sadit merged 1 commit into
mainfrom
docs-tutorial-48

Conversation

@sadit

@sadit sadit commented Sep 24, 2026

Copy link
Copy Markdown
Owner

Closes #48. Don't merge until en is on profiles-1.1: the new test checks that the default release has all seven languages, and today it has six (en is still being fitted for #42). CI will fail on that check until then.

1. The profile walkthrough

  • Fixed along with Republish the v1.1.0 profile assets: they no longer load, and they are ~6x larger than they need to be #42/Default the profile downloads to a release tied to the format, and add the LSI step to the pipeline (#42) #51: the default tag is now profiles-1.1, the release is published, and the tutorial no longer names v1.1.0.
  • Rot the issue didn't mention: quietctx() used SimilaritySearch.LogList, which SimilaritySearch 1.5 removed. So every inverted-file example in the tutorial failed to run: LogList is undefined, and the ones after it fail with wctx undefined. The build is warnonly=true, so nothing flagged it. It is now InvertedFileContext(reporters=[]), and the build has no warnings in tutorial.md.
  • Should it be executable? Yes, but in the suite rather than in the docs build, since the build would have to download tens of MB. testprofile.jl now downloads eu (the smallest, 12.6 MB) from the default release and loads it. That is exactly the rot that happened: a default release in a format the package cannot read. The block in the tutorial stays a plain julia fence.

2. The three undocumented surfaces

All as @example, so they run in the build (outputs verified):

  • :edit correction: derive_edits plus resolve_query_tokens(...; edits). Fortuanto → Fortunato (edit), with the unique-candidate rule and its measured numbers.
  • profile_id: what it covers. Re-saving keeps the id and a refit changes it.
  • The stored LSI: save_lsi/load_lsi, exact truncation, the refusal against the wrong profile (showing the real message), and quantized_wordvectors with ScalarQuant.Cosine(), not SQu8.NormCosine(). The "Pre-trained profiles" section also lists ru and shows how to fetch and bind a <nickname>-lsi.zip.

3. refit in the app README

The Tutorial had no refit step, so there were no old flags to fix; --keep-rate/--base-weight only survive in a test comment. The whole Tutorial was rerun:

  • Two outputs were stale: the lineage line in info now includes min_ndocs=1, and the size in uninstall is 10.4 KiB rather than 30.5 KiB, because of the deflate.
  • New step 6 with refit, with real output, explaining --kappa and --min-ndocs, and why jardin is reported rather than filtered.

Verified

  • julia --project=docs docs/make.jl: no failing @example blocks.
  • App: 245/245 tests pass.
  • testprofile.jl: everything passes except the check for en, as expected.

🤖 Generated with Claude Code

…ile_id, refit (#48)

docs/src/tutorial.md:
- quietctx() used SimilaritySearch.LogList, which 1.5 removed, so every
  inverted-file example after it failed to run (the build is warnonly, so
  nothing noticed). It now passes reporters=[].
- New sections for :edit correction (derive_edits), profile_id, and the
  stored LSI artifact (save_lsi/load_lsi, the profile_id refusal, and
  quantized_wordvectors with ScalarQuant.Cosine rather than NormCosine).
- The pre-trained profiles section lists ru and shows how to fetch and
  bind a <nickname>-lsi.zip.

apps/textsearch/README.md: the Tutorial was rerun; the info lineage and
the uninstall size were stale, and it gains a refit step with real output.

test/testprofile.jl: downloads and loads a profile from the default release,
which is the walkthrough that rotted, rather than making the docs build
download one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@sadit
sadit merged commit 879ac3e into main Sep 24, 2026
3 of 5 checks passed
sadit referenced this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The tutorial is behind: the profile walkthrough no longer runs, and three new surfaces are undocumented

1 participant