Bring the tutorials up to v1.2: edit correction, the stored LSI, profile_id, refit (#48) - #53
Merged
Merged
Conversation
…ile_id, refit (#48) docs/src/tutorial.md: - quietctx() used SimilaritySearch.LogList, which 1.5 removed, so every inverted-file example after it failed to run (the build is warnonly, so nothing noticed). It now passes reporters=[]. - New sections for :edit correction (derive_edits), profile_id, and the stored LSI artifact (save_lsi/load_lsi, the profile_id refusal, and quantized_wordvectors with ScalarQuant.Cosine rather than NormCosine). - The pre-trained profiles section lists ru and shows how to fetch and bind a <nickname>-lsi.zip. apps/textsearch/README.md: the Tutorial was rerun; the info lineage and the uninstall size were stale, and it gains a refit step with real output. test/testprofile.jl: downloads and loads a profile from the default release, which is the walkthrough that rotted, rather than making the docs build download one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #48. Don't merge until
enis onprofiles-1.1: the new test checks that the default release has all seven languages, and today it has six (enis still being fitted for #42). CI will fail on that check until then.1. The profile walkthrough
profiles-1.1, the release is published, and the tutorial no longer namesv1.1.0.quietctx()usedSimilaritySearch.LogList, which SimilaritySearch 1.5 removed. So every inverted-file example in the tutorial failed to run:LogListis undefined, and the ones after it fail withwctxundefined. The build iswarnonly=true, so nothing flagged it. It is nowInvertedFileContext(reporters=[]), and the build has no warnings intutorial.md.testprofile.jlnow downloadseu(the smallest, 12.6 MB) from the default release and loads it. That is exactly the rot that happened: a default release in a format the package cannot read. The block in the tutorial stays a plainjuliafence.2. The three undocumented surfaces
All as
@example, so they run in the build (outputs verified)::editcorrection:derive_editsplusresolve_query_tokens(...; edits).Fortuanto→Fortunato (edit), with the unique-candidate rule and its measured numbers.profile_id: what it covers. Re-saving keeps the id and a refit changes it.save_lsi/load_lsi, exact truncation, the refusal against the wrong profile (showing the real message), andquantized_wordvectorswithScalarQuant.Cosine(), notSQu8.NormCosine(). The "Pre-trained profiles" section also listsruand shows how to fetch and bind a<nickname>-lsi.zip.3.
refitin the app READMEThe Tutorial had no
refitstep, so there were no old flags to fix;--keep-rate/--base-weightonly survive in a test comment. The whole Tutorial was rerun:lineageline ininfonow includesmin_ndocs=1, and the size inuninstallis 10.4 KiB rather than 30.5 KiB, because of the deflate.refit, with real output, explaining--kappaand--min-ndocs, and whyjardinis reported rather than filtered.Verified
julia --project=docs docs/make.jl: no failing@exampleblocks.testprofile.jl: everything passes except the check foren, as expected.🤖 Generated with Claude Code