Skip to content

MCP Tools in Drills: Drill Validate for Execution - #3355

Open
dynamicdhx wants to merge 2 commits into
mainfrom
users/dhruvbharuka/drillvalidateforexecution
Open

MCP Tools in Drills: Drill Validate for Execution #3355
dynamicdhx wants to merge 2 commits into
mainfrom
users/dhruvbharuka/drillvalidateforexecution

Conversation

@dynamicdhx

Copy link
Copy Markdown
Contributor

What does this PR do?

[Provide a clear, concise description of the changes]

[Add additional context, screenshots, or information that helps reviewers]

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

@dynamicdhx

Copy link
Copy Markdown
Contributor Author

Tool Description Evaluator Results

Test 1

Expected Tool: resilience_drill_validate-for-execution
Prompt: Validate resilience drill <drill_name> in service group <service_group> for execution from source location <source_location>

Results

Rank Score Tool Status
1 0.680021 resilience_drill_validate-for-execution EXPECTED
2 0.623021 resilience_drill_get
3 0.566597 resilience_drill_resource_get
4 0.453351 resilience_recovery_plan_delete
5 0.405873 resilience_recovery_plan_get

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Resilience Management drill operation to the Azure MCP toolset: validating a drill for execution from specified source locations. This expands the existing resilience drill command group with a new tool/command and wires it through service implementation, registration, tests, and documentation surfaces.

Changes:

  • Introduces DrillValidateForExecutionCommand with options/model types and JSON source-generation registration.
  • Implements ValidateDrillForExecutionAsync in the Resilience Management service layer and exposes it via IResilienceManagementService.
  • Adds unit + command tests and updates Azure MCP documentation, consolidated tool mappings, and a changelog entry.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/ResilienceManagementCommandTests.cs Adds an end-to-end style test invoking the new drill validate-for-execution tool.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/Drills/DrillValidateForExecutionCommandTests.cs Adds unit tests covering validation, success path, and error handling for the new command.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/assets.json Updates recorded test asset tag to include the new behavior.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/ResilienceManagementService.cs Implements drill validate-for-execution service operation using the ARM SDK.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/IResilienceManagementService.cs Adds the new service method contract.
tools/Azure.Mcp.Tools.ResilienceManagement/src/ResilienceManagementSetup.cs Registers the new command in DI and adds it to the drill command group.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/Drills/DrillValidateForExecutionOptions.cs Defines CLI/tool options for validate-for-execution.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillValidateForExecutionResult.cs Adds a result model for returning operation ID and completion state.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/ResilienceManagementJsonContext.cs Registers new result types for AOT-safe source-generated JSON serialization.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/DrillValidateForExecutionCommand.cs Implements the new command, option validation, execution flow, and error mapping.
servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json Maps the new tool under the resilience management “create” consolidated tool and updates its metadata/description.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Adds an e2e prompt for the new tool name.
servers/Azure.Mcp.Server/docs/azmcp-commands.md Adds the command reference entry for azmcp resilience drill validate-for-execution.
servers/Azure.Mcp.Server/changelog-entries/dhruvbharuka-drill-validate-for-execution.yml Adds a changelog entry for the new tool.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +47 to +56
"name": "create_azure_resilience_management_resources",
"description": "Create, update, or delete Azure Resilience Management resources, including usage plans, usage plan enrollments, and recovery plans for Azure service groups.",
"description": "Create, update, delete, or validate Azure Resilience Management resources, including usage plans, usage plan enrollments, recovery plans, and drills for Azure service groups.",
"toolMetadata": {
"destructive": {
"value": true,
"description": "This tool may delete or modify existing resources in its environment."
},
"idempotent": {
"value": true,
"description": "Running this operation multiple times with the same arguments produces the same result without additional effects."
"value": false,
"description": "This tool includes operations that may produce additional effects when run multiple times with the same arguments."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants