fix(skills): surface offending line and quoting hint on SKILL.md YAML…#3335
fix(skills): surface offending line and quoting hint on SKILL.md YAML…#3335Huixin615 wants to merge 3 commits into
Conversation
… errors
When a SKILL.md front-matter fails to parse, the existing log only
echoes PyYAML's raw message, leaving authors to grep the file for the
offending line. This is especially painful for the very common
LLM-authored mistake of an unquoted scalar containing ': '
(e.g. 'description: foo: bar'), which fails with
'mapping values are not allowed here' and silently drops the skill.
Enrich the error log with:
- the source line PyYAML pointed at via problem_mark
- a targeted, copy-pasteable quoting hint when (and only when) the
error is the well-known 'mapping values are not allowed' scanner
error on an unquoted value
The skill is still rejected (no semantics are guessed or rewritten);
only the diagnostic is improved.
Fixes bytedance#3333
|
Read through the diff and tried the repro. To be clear on framing: this isn't really a bug fix, it's making the error message friendlier, which is useful. One thing in the description isn't quite accurate. PyYAML already prints the offending line itself — it comes with the error. Here's what you get on So the source line was already there before this PR. That also means the test's A few small things:
Overall it's an improvement over dumping the raw PyYAML message. Mostly just fix the framing. |
Per review on bytedance#3335: - Log the file line number (mark.line + 2) instead of the front-matter-internal line number, so authors land on the right row in their editor. - Use exc.problem == "mapping values are not allowed here" for a tighter match than substring-scanning str(exc). - Preserve the offending key's leading whitespace in the quoting hint so nested mappings stay nested when authors paste the fix back. - Rewrite the regression test to actually exercise the new behaviour: PyYAML's own message already echoes the offending line (and truncates it with "..."), so the old assertion passed on main. New assertions pin (a) the file-line number, (b) the full untruncated line, and (c) the copy-pasteable hint. - Add a guard test for nested-key indentation so the partition()/strip() shape cannot regress silently. Refs bytedance#3333, bytedance#3335
There was a problem hiding this comment.
Pull request overview
Improves SKILL.md YAML front-matter parse diagnostics so misquoted scalar values (notably unquoted : in description) no longer fail silently; the parser now logs the offending source line (file-line numbering) and conditionally adds a copy/paste quoting hint for the specific, common PyYAML scanner error.
Changes:
- Add
_format_yaml_error()to enrich YAML parse error logs with the offending source line and a targeted quoting hint. - Update
parse_skill_file()to use the formatted error output instead of logging the raw PyYAML exception only. - Add regression tests to validate the new log output (line number, full line not truncated, hint boundaries, indentation preservation).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| backend/packages/harness/deerflow/skills/parser.py | Formats YAML parse errors to include offending line + optional quoting hint; updates error logging callsite. |
| backend/tests/test_skills_parser.py | Adds tests covering the new friendly YAML error reporting behavior and guardrails. |
| if value and value[0] not in {'"', "'", "|", ">", "[", "{"}: | ||
| escaped = value.replace('"', '\\"') | ||
| lines.append(f' hint: values containing ":" must be quoted, e.g. {key}: "{escaped}"') |
The hint emitted by _format_yaml_error previously escaped only double quotes, so values containing backslashes (e.g. Windows paths like C:\Temp or regex escapes like \d) produced a suggested scalar that was either invalid YAML or silently re-interpreted by PyYAML's double-quoted escape rules when pasted back. Escape order matters: backslashes first, then double quotes. Adds two regression tests covering Windows-path and regex-style backslashes. Address Copilot CR feedback on PR bytedance#3335.
|
Hi @WillemJiang Thanks for the review! I've addressed Copilot's review comments, Could you please take another look when you have time, thanks a lot. |
Fixes #3333
Why
Trigger: Issue #3333 reports that a custom SKILL.md whose
descriptioncontains an unquoted:(e.g.description: StarRun collector: progress, errors) is rejected by PyYAML withmapping values are not allowed here, and the skill is then silently dropped at load time. The author of the issue also asks whether the failure could be surfaced to users instead of being invisible.Pain: The existing log only echoes PyYAML's raw message, so authors have to grep the file to find the offending line. This hits especially hard for LLM-authored SKILL.md, where unquoted values containing
:are a recurring mistake — every occurrence makes a skill silently disappear with no actionable hint.What changed
When a SKILL.md front-matter fails YAML parsing, the error log now contains:
problem_mark.line), so the author no longer has to open the file to find the offending row, andmapping values are not allowedscanner error on an unquoted value. Other YAML errors are left untouched to avoid misleading the author.No semantics are guessed or rewritten — the skill is still rejected, the diagnostic is just useful now.
Before
After
Surface area
frontend/backend/applanggraph.json, or prompt changedocker/or sandboxed executionskills/(parser-side diagnostic improvement)backend/pyproject.tomlorfrontend/package.json(say what it buys us)Screenshots / Recording
N/A — backend-only change, no UI surface.
Bug fix verification
backend/tests/test_skills_parser.py::test_parse_unquoted_colon_value_logs_line_and_hintmain/ green on this branch? Yes — the test asserts the log contains both the offending source line and adescription: "..."quoting hint; before this PR neither was present, so the test fails onmainand passes here.test_parse_unrelated_yaml_error_omits_quoting_hint— the hint must NOT attach to unrelated YAML errors (no misleading suggestions).test_parse_valid_skill_emits_no_error_log— a valid SKILL.md must still produce zero error logs.Validation
Formatting (CI gate):
Targeted unit tests (in the Docker gateway container, equivalent to
make test):Full backend test suite: