You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
swift eval broken with evalscope>=1.11: EvalType.SERVICE was removed but ms-swift still uses it #10465
swift eval --eval_backend Native crashes with AttributeError: type object 'EvalType' has no attribute 'SERVICE' when the installed evalscope is >= 1.11.0.
evalscope 1.11.0 removed EvalType.SERVICE (it was kept as a deprecated alias SERVICE = 'openai_api' up to 1.10.0), but ms-swift still references it in swift/pipelines/eval/eval.py:
eval_type=EvalType.SERVICE,
This is still the case in the latest release (4.6.0, verified from the PyPI sdist). Meanwhile requirements/eval.txt declares evalscope>=1.0.0 with no upper bound, so a fresh pip install "ms-swift[eval]" today resolves evalscope 1.11+ and swift eval is broken out of the box.
File ".../swift/pipelines/eval/eval.py", line 96, in get_native_task_cfg
eval_type=EvalType.SERVICE,
AttributeError: type object 'EvalType' has no attribute 'SERVICE'
Expected behavior
Either migrate to EvalType.OPENAI_API (with a fallback for older evalscope), or cap the requirement as evalscope>=1.0.0,<1.11 until the migration lands.
Workaround
Pinning evalscope==1.9.1 (or <1.11) makes swift eval work again.
Activity
ryznren commentedon Oct 29, 2026
I'm working on a PR to fix the EvalScope compatibility issue, including regression tests for the supported versions.