Skip to content

[Fix] Video-MME-v2: improve acc calculation and data preparation#1551

Open
EliYuan30 wants to merge 2 commits into
open-compass:mainfrom
EliYuan30:fix/video-mme-v2-update
Open

[Fix] Video-MME-v2: improve acc calculation and data preparation#1551
EliYuan30 wants to merge 2 commits into
open-compass:mainfrom
EliYuan30:fix/video-mme-v2-update

Conversation

@EliYuan30

Copy link
Copy Markdown

This PR improves Video-MME-v2 support by:

  • making dataset preparation resumable when extracted videos are missing or incomplete
  • validating extracted video file sizes against zip metadata
  • adding progress reporting for Video-MME-v2 video preparation
  • defaulting Video-MME-v2 evaluation to exact matching when no judge is provided
  • adding accuracy results alongside group-level rating results

@EliYuan30 EliYuan30 force-pushed the fix/video-mme-v2-update branch from bfea348 to ed8dc87 Compare May 24, 2026 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant