Problem statement
Stringy's entire premise is format-aware string extraction. Plain ASCII/UTF-8 text is a format -- arguably the simplest one -- so "unrecognized container, fall back to unstructured byte scan and score-reorder the lines" is the tool failing its own thesis, not a harmless edge case. If stringy isn't aware of text, it isn't format-aware.
Running stringy on a plain ASCII/UTF-8 text file reorders the file's lines by semantic score instead of preserving document order:
stringy AI_POLICY.md
Info: Unsupported container format; proceeding with unstructured byte scan
Everything else follows from that. <- a concluding sentence from the middle
## New Contributors
We build operator-focused security tools...
... <- headers and intro appear scrambled
All 16 lines are present, but the semantic ranking scrambles a human-readable document. For non-container text input this is worse than strings/cat, which preserve order. The reordering only makes sense once the user has asked for it (ranking, top-N, tag filters).
Proposed solution
When the input is not a recognized container (raw byte-scan fallback) and no ordering/ranking/filter flags are supplied, default to file-order passthrough (cat-like) rather than score-reordered output. Ranking/dedup/tag-filtering kick in only when the user opts into them (--top, --only-tags, --enc, etc.).
Open design questions:
- Does "no params" also imply skipping dedup, or just skipping reordering?
- Should this apply only to the unstructured-fallback path, or also when a container is detected but the user passed no filters?
Touches: pipeline (ordering/normalization gating on input type + flags), main.rs.
Alternatives considered
- Always rank regardless of input type (current behavior): surprising and less useful than
strings for text.
- A dedicated
--passthrough/--raw-like flag: --raw already exists but also strips tags/score and is a different intent; the ask here is a sensible default, not a new flag.
Area
Pipeline / CLI
Problem statement
Stringy's entire premise is format-aware string extraction. Plain ASCII/UTF-8 text is a format -- arguably the simplest one -- so "unrecognized container, fall back to unstructured byte scan and score-reorder the lines" is the tool failing its own thesis, not a harmless edge case. If stringy isn't aware of text, it isn't format-aware.
Running
stringyon a plain ASCII/UTF-8 text file reorders the file's lines by semantic score instead of preserving document order:All 16 lines are present, but the semantic ranking scrambles a human-readable document. For non-container text input this is worse than
strings/cat, which preserve order. The reordering only makes sense once the user has asked for it (ranking, top-N, tag filters).Proposed solution
When the input is not a recognized container (raw byte-scan fallback) and no ordering/ranking/filter flags are supplied, default to file-order passthrough (cat-like) rather than score-reordered output. Ranking/dedup/tag-filtering kick in only when the user opts into them (
--top,--only-tags,--enc, etc.).Open design questions:
Touches:
pipeline(ordering/normalization gating on input type + flags),main.rs.Alternatives considered
stringsfor text.--passthrough/--raw-like flag:--rawalready exists but also strips tags/score and is a different intent; the ask here is a sensible default, not a new flag.Area
Pipeline / CLI