Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/compare-text.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@astryxdesign/cli': patch
---

[fix] The CLI orders names the same way on every machine. Build, search, discovery and the CLI's lists (such as `astryx component --list`) broke ties with the host's locale, so a machine set to another language could pick a different template or print lists in a different order. They now use one fixed text order, and a lint rule keeps locale-dependent comparisons out of the CLI.

@josephfarina
44 changes: 23 additions & 21 deletions docs/specs/AST-048-build-recommendation/spec.md

Large diffs are not rendered by default.

18 changes: 10 additions & 8 deletions docs/specs/AST-083-result-order/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ owns response fields and compatibility.
- **FR5 — the order is local, deterministic, and static.** The order inside a
run comes from plain code and fixed values shipped with the CLI. No model
runtime runs, no network is used, and the same inputs give the same order.
Tied names order the same way on every host: the host's language setting
never changes the order of a result or a list.
Comment on lines +78 to +79

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this line covers "a result or a list", but this record's Intent still names only astryx search, build's blocks and the not-found suggestions, so the rule behind astryx component --list, doctor, upgrade and manifest order is hard to find here. The FR5 verification row also still lists only the offline and repeat checks. Consider adding api/build/build.test.mjs ("gives the same answers under every host locale") and clients/cli/commands/component.host-locale.test.mjs to that row (and the AST-048 FR6, FR7 row), and naming the CLI's other lists in the Intent or in architecture:cli-surface.

- **FR6 — the order changes only when measured.** A change to the order
inside runs, whether new weights, a new searcher, or a new rule, ships only
when measured against the held-out, fresh, synthetic, and cold-start
Expand Down Expand Up @@ -110,14 +112,14 @@ owns response fields and compatibility.

## Verification

| Contract | Verification | Representative states | Mutation or failure expectation |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| FR1, FR2 | `api/search/_order.test.mjs` against the lexical ranker; a replay over every real and synthetic query | multi-word queries, doc sections, domain-priority merges | a non-lexical result, a changed score, reason, or count, a cross-band move, or extra results fail |
| FR3 | `api/search/_main-searcher.test.mjs` whole-name cases | exact names, names typed with spaces for dashes | an answer that differs from the lexical ranker's fails |
| FR4 | `api/build/blocks/blocks.test.mjs`, `api/build/tree.test.mjs` | exactly named blocks, integration blocks, unfitted blocks | a block outside the catalog, a repeated block, a changed entry, an entry without a score and reason, a moved exactly named block, or more blocks fail |
| FR5 | `_main-searcher.test.mjs` and `tree.test.mjs` offline and repeat checks | network stubbed to throw; repeated runs | a network call or a differing repeat fails |
| FR6 | measured comparison on the evaluation sets, reported in the change's description | held-out, fresh, synthetic, cold-start | a row whose paired CI lies below zero blocks the change |
| FR7 | `foundation/text/suggest-order.test.mjs`; the component, docs and template command tests; a replay over every labeled failed lookup | compound names, other libraries' names, routes under a namespace | a longer list, a suggestion that is not an entry, a changed resolution, or a worse measured row fails |
| Contract | Verification | Representative states | Mutation or failure expectation |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| FR1, FR2 | `api/search/_order.test.mjs` against the lexical ranker; a replay over every real and synthetic query | multi-word queries, doc sections, domain-priority merges | a non-lexical result, a changed score, reason, or count, a cross-band move, or extra results fail |
| FR3 | `api/search/_main-searcher.test.mjs` whole-name cases | exact names, names typed with spaces for dashes | an answer that differs from the lexical ranker's fails |
| FR4 | `api/build/blocks/blocks.test.mjs`, `api/build/tree.test.mjs` | exactly named blocks, integration blocks, unfitted blocks | a block outside the catalog, a repeated block, a changed entry, an entry without a score and reason, a moved exactly named block, or more blocks fail |
| FR5 | `_main-searcher.test.mjs` and `tree.test.mjs` offline and repeat checks; `component.host-locale.test.mjs` and `build.test.mjs` host-locale checks | network stubbed to throw; repeated runs; the same commands under English, Czech and Hawaiian host locales | a network call or a differing repeat fails; an order that differs between host locales fails |
Comment thread
astracat-bot[bot] marked this conversation as resolved.
Resolved
| FR6 | measured comparison on the evaluation sets, reported in the change's description | held-out, fresh, synthetic, cold-start | a row whose paired CI lies below zero blocks the change |
| FR7 | `foundation/text/suggest-order.test.mjs`; the component, docs and template command tests; a replay over every labeled failed lookup | compound names, other libraries' names, routes under a namespace | a longer list, a suggestion that is not an entry, a changed resolution, or a worse measured row fails |

## Decision log

Expand Down
27 changes: 27 additions & 0 deletions eslint.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,25 @@ const apiImportRules = (/** @type {string[]} */ allowed = []) => [
},
];

// `localeCompare` and `Intl.Collator` without a fixed locale read the host's
// locale, so the same names sort differently on different machines and the
// CLI answers differently. CLI code sorts machine text with compareText, or
// names a fixed locale.
const HOST_LOCALE_ORDER = [
{
selector:
"CallExpression[callee.property.name='localeCompare']:not([arguments.1.type='Literal'])",
message:
'Sort machine text with compareText (foundation/text/compare-text.mjs), or pass a fixed locale: localeCompare without one follows the host locale, so the order changes between machines.',
},
{
selector:
":matches(NewExpression, CallExpression)[callee.property.name='Collator']:not([arguments.0.type='Literal'])",
message:
'Give Intl.Collator a fixed locale, or sort machine text with compareText (foundation/text/compare-text.mjs): without one it follows the host locale.',
},
];

export default defineConfig(
js.configs.recommended,
tseslintRecommended,
Expand Down Expand Up @@ -826,6 +845,11 @@ export default defineConfig(
],
},
},
{
files: ['packages/cli/**/*.mjs'],
ignores: ['**/*.test.mjs'],
rules: {'no-restricted-syntax': ['error', ...HOST_LOCALE_ORDER]},
},
// A command's --help and its manifest entry are generated from its colocated
// CommandDoc via defineCommand. Registering straight onto Commander bypasses
// the doc, so the docs silently stop describing the real CLI.
Expand All @@ -840,6 +864,7 @@ export default defineConfig(
message:
'Register commands with defineCommand(parent, doc, {fn, action}) so --help and the manifest come from the colocated CommandDoc. See CONTRIBUTING > Working on the astryx CLI.',
},
...HOST_LOCALE_ORDER,
],
},
},
Expand All @@ -859,6 +884,7 @@ export default defineConfig(
message:
'Register commands with defineCommand(parent, doc, {fn, action}) so --help and the manifest come from the colocated CommandDoc. See CONTRIBUTING > Working on the astryx CLI.',
},
...HOST_LOCALE_ORDER,
{
selector: "CallExpression[callee.property.name='padEnd']",
message:
Expand Down Expand Up @@ -900,6 +926,7 @@ export default defineConfig(
message:
'Register commands with defineCommand(parent, doc, {fn, action}) so --help and the manifest come from the colocated CommandDoc. See CONTRIBUTING > Working on the astryx CLI.',
},
...HOST_LOCALE_ORDER,
],
},
},
Expand Down
70 changes: 70 additions & 0 deletions packages/cli/api/build/build.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -604,3 +604,73 @@ describe('build kit — every page starts from a template', () => {
expect(r.data.start?.name).not.toMatch(/^product-/);
});
});

// A host locale moves names only when Node carries that locale's collation
// data (official builds do; an English-only ICU build does not).
const HOST_LOCALES = ['cs_CZ.UTF-8', 'haw_US.UTF-8', 'tr_TR.UTF-8'];
const hasLocaleData =
Intl.Collator.supportedLocalesOf(['cs', 'haw', 'tr']).length === 3;

describe('build is deterministic', () => {
// Made-up phrases. Several are close to exact ties between templates, and
// "data table" and "checkout page" order differently under a Hawaiian
// collation when names sort by the host locale.
const ideas = [
'code editor',
'item detail page',
'settings in a modal dialog',
'ide with a file tree',
'chat',
'inbox table with filters',
'yearly report with charts',
'data table',
'checkout page',
];
/** @param {any} r */
const answer = r => ({
start: r.data?.start?.name ?? null,
basis: r.data?.start?.basis ?? null,
alternatives: (r.data?.start?.alternatives ?? []).map(a => a.name),
blocks: (r.data?.blocks ?? []).map(b => b.name),
});
const answers = async () => {
const out = [];
for (const idea of ideas) out.push(answer(await build(idea, {cwd: REPO})));
return out;
};

it('gives the same answers twice in one process', async () => {
const first = await answers();
expect(await answers()).toEqual(first);
}, 120000);

it.skipIf(!hasLocaleData)(
'gives the same answers under every host locale',
async () => {
const {spawnSync} = await import('node:child_process');
const url = new URL('./build.mjs', import.meta.url).href;
const script = `import {build} from ${JSON.stringify(url)};
const out = [];
for (const idea of ${JSON.stringify(ideas)}) {
const d = (await build(idea, {cwd: ${JSON.stringify(REPO)}})).data ?? {};
out.push({start: d.start?.name ?? null, basis: d.start?.basis ?? null,
alternatives: (d.start?.alternatives ?? []).map(a => a.name),
blocks: (d.blocks ?? []).map(b => b.name)});
}
console.log(JSON.stringify(out));`;
/** @param {string} lang */
const under = lang => {
const res = spawnSync(
process.execPath,
['--input-type=module', '-e', script],
{encoding: 'utf8', env: {...process.env, LANG: lang, LC_ALL: lang}},
);
expect(res.status, res.stderr).toBe(0);
return JSON.parse(res.stdout.trim().split('\n').pop() ?? '[]');
};
const expected = under('en_US.UTF-8');
for (const lang of HOST_LOCALES) expect(under(lang), lang).toEqual(expected);
},
180000,
);
});
3 changes: 2 additions & 1 deletion packages/cli/api/build/kit/rank.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ import {
keywordWordsFor,
framesOf,
} from './words.mjs';
import {compareText} from '../../../foundation/text/compare-text.mjs';

/**
* @typedef {import('./words.mjs').PageTemplate} PageTemplate
Expand Down Expand Up @@ -289,7 +290,7 @@ export function rankPages(query, pages) {
(a, b) =>
b.score - a.score ||
a.name.split('-').length - b.name.split('-').length ||
a.name.localeCompare(b.name),
compareText(a.name, b.name),
);
return baseFirst(ranked);
}
Expand Down
42 changes: 42 additions & 0 deletions packages/cli/api/build/kit/rank.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -442,3 +442,45 @@ describe('pickStart on parts and edits (spec:AST-048/FR3)', () => {
expect(framed('a toggle in a modal on the settings page')).toBe(false);
});
});

describe('rankPages is a total order', () => {
const ideas = [
'code editor',
'item detail page',
'settings in a modal dialog',
'ide with a file tree',
'chat',
'inbox table with filters',
];
/** @param {any[]} ranked */
const names = ranked => ranked.map(r => `${r.name}:${r.score}`);

it('ranks the same however the catalog is ordered', async () => {
const pages = await loadPageTemplates(REPO);
const reversed = [...pages].reverse();
const rotated = [...pages.slice(7), ...pages.slice(0, 7)];
for (const idea of ideas) {
const expected = names(rankPages(idea, pages));
expect(names(rankPages(idea, reversed))).toEqual(expected);
expect(names(rankPages(idea, rotated))).toEqual(expected);
}
});

it('breaks an exact tie by name, without the host locale', () => {
// Same words, same score: only the names differ, in case and in letters
// some locales move ("y" in Lithuanian, dotless "i" in Turkish).
const catalog = ['Yard-Board', 'ide-board', 'IDE-board', 'item-board'].map(
name => page(name, 'Tools', 'a board of cards'),
);
const order = rankPages('board of cards', catalog).map(r => r.name);
expect(order).toEqual([
'ide-board',
'IDE-board',
'item-board',
'Yard-Board',
]);
expect(
rankPages('board of cards', [...catalog].reverse()).map(r => r.name),
).toEqual(order);
});
});
3 changes: 2 additions & 1 deletion packages/cli/api/component/list/list.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import {
loadIntegrationsSafely,
withOwnership,
} from '../_adapter.mjs';
import {compareText} from '../../../foundation/text/compare-text.mjs';

/**
* @typedef {import('../component.type.mjs').ComponentListResponse} ComponentListResponse
Expand Down Expand Up @@ -470,7 +471,7 @@ async function componentListNow(coreDir, {cwd, category, detail, zh, dense, lang
});
}
for (const [key, members] of byGroup) {
members.sort((a, b) => a.name.localeCompare(b.name));
members.sort((a, b) => compareText(a.name, b.name));
listData[key] = members;
}
}
Expand Down
5 changes: 3 additions & 2 deletions packages/cli/api/discover/search/search.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import {levenshteinDistance} from '../../../foundation/text/string-utils.mjs';
import {AstryxError} from '../../error.mjs';
import {ERROR_CODES} from '../../../foundation/response/error-codes.mjs';
import {DISCOVER_KINDS} from '../../../authoring/discover/parse.mjs';
import {compareText} from '../../../foundation/text/compare-text.mjs';

/**
* @typedef {import('../_package-scanner.mjs').ScannedPackage} ScannedPackage
Expand Down Expand Up @@ -131,8 +132,8 @@ export async function search(packages, query, opts) {
a.score - b.score ||
Number(b.item.installed) - Number(a.item.installed) ||
KIND_ORDER.indexOf(a.item.kind) - KIND_ORDER.indexOf(b.item.kind) ||
a.item.package.localeCompare(b.item.package) ||
a.item.name.localeCompare(b.item.name),
compareText(a.item.package, b.item.package) ||
compareText(a.item.name, b.item.name),
);

if (ranked.length > 0) {
Expand Down
3 changes: 2 additions & 1 deletion packages/cli/api/docs/_adapter.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ import {
orderRouteSuggestions,
} from '../../foundation/text/suggest-order.mjs';
import {formatCliCommand} from '../../foundation/env/package-manager.mjs';
import {compareText} from '../../foundation/text/compare-text.mjs';

export {OVERLAY_LANGUAGES, overlayLanguages};

Expand Down Expand Up @@ -2293,6 +2294,6 @@ export async function integrationDocTypes(integration, cwd) {
unchecked: [
...report.unchecked,
...unread.map(entry => ({doc: rel(entry.path), reason: entry.reason})),
].sort((a, b) => a.doc.localeCompare(b.doc)),
].sort((a, b) => compareText(a.doc, b.doc)),
};
}
5 changes: 3 additions & 2 deletions packages/cli/api/doctor/_adapter.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ import {
findProjectCoreDir,
findInstalledPackage,
} from '../../foundation/fs/paths.mjs';
import {compareText} from '../../foundation/text/compare-text.mjs';
export {
isLocalThemeOwner,
planThemeAppWrite,
Expand Down Expand Up @@ -282,7 +283,7 @@ export function findIconPlaceholders(projectDir) {
uses: uses.length,
});
}
return found.sort((a, b) => a.file.localeCompare(b.file));
return found.sort((a, b) => compareText(a.file, b.file));
}

/** @param {string} projectDir */
Expand Down Expand Up @@ -796,7 +797,7 @@ export function readBuiltPath(target, {fileLimit = BUILT_FILE_LIMIT} = {}) {
} catch {
continue;
}
entries.sort((a, b) => a.name.localeCompare(b.name));
entries.sort((a, b) => compareText(a.name, b.name));
for (const entry of entries) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
Expand Down
3 changes: 2 additions & 1 deletion packages/cli/api/doctor/built-checks.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import {
} from './_adapter.mjs';
import {formatCliCommand} from '../../foundation/env/package-manager.mjs';
import {babelParse as parseModule} from '../../foundation/runtime/lazy-dependencies.mjs';
import {compareText} from '../../foundation/text/compare-text.mjs';

/** Source files, read for imports and JSX. */
const SOURCE_EXT = new Set(['.tsx', '.jsx', '.ts', '.js', '.mjs', '.cjs']);
Expand Down Expand Up @@ -716,7 +717,7 @@ export function scanBuilt(target, packages, {fileLimit} = {}) {
}
}
scan.packages = [...imported].sort();
scan.findings.sort((a, b) => a.file.localeCompare(b.file) || a.line - b.line);
scan.findings.sort((a, b) => compareText(a.file, b.file) || a.line - b.line);
return scan;
}

Expand Down
3 changes: 2 additions & 1 deletion packages/cli/api/icon/_records.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import {AstryxError} from '../error.mjs';
import {ERROR_CODES} from '../../foundation/response/error-codes.mjs';
import {levenshteinDistance} from '../../foundation/text/string-utils.mjs';
import {compareText} from '../../foundation/text/compare-text.mjs';
export {
answeringMatches,
iconRecord,
Expand Down Expand Up @@ -115,7 +116,7 @@ export function oneNamedSet(sets, withdrawn, selector) {
export function closest(wanted, candidates, reason, max = 5) {
return [...new Set(candidates)]
.map(name => ({name, distance: levenshteinDistance(wanted, name)}))
.sort((a, b) => a.distance - b.distance || a.name.localeCompare(b.name))
.sort((a, b) => a.distance - b.distance || compareText(a.name, b.name))
.slice(0, max)
.map(({name}) => ({name, reason}));
}
Loading
Loading