fix(glyphs): close two positional-correlation gaps (#57, #58) - #86
Merged
Conversation
#57 Mongolian Supplement U+1166C and #58 Myanmar Extended-C U+116DA were both shipped as missing glyphs in every ucode release since the positional correlator landed. Two root causes: 1. **Font name truncation.** mutool trace emits font names truncated to 31 chars (PDF base-font-name limit, PDF 32000-1 §7.9.6). The BaseFont dict carries the full original name. For long-named fonts like HBBJCP+Uni11660Mongoliansupplement (34 chars) the trace emits HBBJCP+Uni11660Mongoliansupplem, and every verbatim == comparison against the BaseFont name silently failed. Fix: TraceGlyph.normalize_name / TraceGlyph.name_match? truncate both sides to the 31-char limit before comparing. PageTraceCache and TraceCorrelator now route every name comparison through these helpers. 2. **Partial ToUnicode coverage.** The Unicode Consortium ships charts whose embedded fonts render MORE glyphs than their /ToUnicode CMaps admit (orphan glyphs whose CIDs were forgotten in the CMap build). For these fonts, ToUnicodeStrategy returns a partial {cp => gid} map and TraceStrategy never fires because needs_positional? only checked for empty/out-of-scope results. Fix: needs_positional? now also returns true when the trace cache observes the font rendering more distinct GIDs than the intrinsic result covers. The positional pass then fills the gaps via the chart's hex labels; merge_with_positional_precedence keeps the intrinsic mappings untouched where they agree. After this fix: - Mongolian_Supplement: 12 -> 13 SVGs (U+1166C recovered) - Myanmar_Extended-C: 19 -> 20 SVGs (U+116DA recovered) Adds four regression specs covering the long-name tolerance in PageTraceCache's lookup methods. The fix would have caught both regressions; the next pre-existing partial-CMap font will now Just Work.
The 0.5.1 release bumped lib/ucode/version.rb but missed syncing config/unicode17_universal_glyph_set.yml#ucode_version. Same gap that happened on 0.5.0; the smoke spec asserting the two stay in sync catches this on the next PR's CI run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #57 (Mongolian Supplement U+1166C) and #58 (Myanmar Extended-C U+116DA).
Root causes
Font name truncation. mutool trace emits font names truncated to 31 chars (PDF base-font-name limit). The BaseFont dict carries the full original name. For long-named fonts like
HBBJCP+Uni11660Mongoliansupplement(34 chars) the trace emitsHBBJCP+Uni11660Mongoliansupplem, and every verbatim==comparison against the BaseFont name silently failed.Partial ToUnicode coverage. The Unicode Consortium ships charts whose embedded fonts render MORE glyphs than their
/ToUnicodeCMaps admit (orphan glyphs whose CIDs were forgotten in the CMap build).needs_positional?only checked for empty/out-of-scope intrinsic results, so partial coverage fell through.Fix
TraceGlyph.normalize_name/TraceGlyph.name_match?truncate both sides to the 31-char PDF limit before comparing.PageTraceCacheandTraceCorrelatorroute every name comparison through these helpers.CodepointMapper#needs_positional?gains a 4th predicate:intrinsic_has_uncovered_gids?queries the trace cache for distinct GIDs and fires positional when the font ships more glyphs than the intrinsic result covers.Test plan
page_trace_cache_spec.rbfor long-name tolerance across#each_page_for,#references_font?,#distinct_gids_for,#find_glyph.Mongolian_Supplement12 → 13 SVGs (U+1166C recovered).Myanmar_Extended-C19 → 20 SVGs (U+116DA recovered).spec/ucode/code_chart/,spec/ucode/glyphs/embedded_fonts/,spec/ucode/cli_spec.rbpass.bundle exec rubocopclean on touched files.