Skip to content

Document authoritative Developer Info contract for tool-call evaluation #192

Description

@sakov-ms

Context

M365-Copilot-Agent-Evals PR https://github.com/microsoft/M365-Copilot-Agent-Evals/pull/431 depends on an authoritative WorkIQ contract for application/vnd.ms-workiq.developer-info. The current developer-info.md is marked Proposal and not yet implemented.

Related Evals issues: https://github.com/microsoft/M365-Copilot-Agent-Evals/issues/224 and https://github.com/microsoft/M365-Copilot-Agent-Evals/issues/280.

Required contract

  • Accepted media type and AgentCard/accepted_output_modes activation
  • A2A message/send and message/stream accumulation semantics across artifact updates
  • Append/replace/patch, ordering, deduplication, and conflict rules
  • Terminal behavior proving Developer Info is complete for a turn
  • Conversation/request/turn correlation invariants
  • Canonical nested DeveloperInfo.plugins.executions[] structure
  • Canonical tool naming, case sensitivity, uniqueness, and stability invariants
  • tool_call_id, attempt identity, parent/logical-call identity, and retry correlation
  • Which selected, attempted, failed, canceled, interrupted, and auth-required executions are emitted
  • Authoritative successful/failed/indeterminate execution outcome algorithm

Acceptance criteria

  1. Publish the finalized contract in the owned WorkIQ documentation.
  2. Add conformance examples for single call, multiple calls, failed call, retry-success, and multi-frame streaming.
  3. Identify the implementation/rollout status consumers can use to gate support.
  4. Link the finalized document back to PR #431.

This issue is an approval dependency for PR #431, not an informal follow-up.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions