ABL Engineering Assistant

This project packages the Kore.ai ABL Arch MCP registry as a Microsoft 365 Copilot declarative agent. It is designed for project discovery, package validation, runtime diagnosis, evaluation analysis, documentation lookup, and approved engineering changes.

Get started with the template

Prerequisites

To run this app template in your local dev machine, you will need:

• Node.js, supported versions: 22
• A Microsoft 365 account for development.
• Microsoft 365 Agents Toolkit Visual Studio Code Extension version 5.0.0 and higher or Microsoft 365 Agents Toolkit CLI
• Microsoft 365 Copilot license

1. Open ATK in VS Code

   Click the Microsoft 365 Agents Toolkit icon in the Activity Bar.

2. Sign in

   Open the Agents Toolkit by click on the toolkit icon in the VS Code sidebar. Under Account, authenticate with your dev M365 account.

3. Scaffold a new DA

   In the Agents Toolkit menu, click 'Create new Agent/app', select 'Declarative Agent', choose a folder and name (e.g. ABL Engineering Assistant).

4. Add your MCP Server

   In the ATK sidebar click Add Action → Start with an MCP server, then enter your MCP discovery URL (e.g. https://mcp.contoso.com/discover).

5. Start your MCP Server

   After the DA project generated, click the "Start" button in the mcp.json file to start your MCP server, when prompt, enter the user ID and password for authentication.

6. Fetch and select tools

   When prompted, click ATK: Fetch Action from MCP in mcp.json. This project uses dynamic MCP discovery, so the remote registry remains the source of tool schemas and run_for_functions binds the declared catalog. Keep the complete catalog available; the agent instructions and concise function descriptions guide tool selection without removing capability.

   The project does not currently maintain a checked-in static mcp-tools.json catalog.

7. Configure Auth

   ATK will retrieve the authentication information for your MCP server and store these values are in env/.env.development and a secure reference is injected into appPackage/ai-plugin.json. If the inputted MCP server authentication information is not configure correctly according to the MCP protocol, ATK will prompt errors.

8. Review generated files

   • appPackage/ai-plugin.json (model-facing function descriptions, runtime spec, and auth)
   • appPackage/declarativeAgent.json (agent identity, capabilities, and conversation starters)
   • appPackage/instruction.txt (operating rules and safety policy)
   • appPackage/manifest.json (Teams and Copilot app metadata)
   • docs/embedded-knowledge-guide.md (knowledge-source design and setup)
   • docs/agent-skills.md (package-level skills and validation)
   • docs/skill-local-to-remote-migration-guide.md (plan for local-to-remote skill references)
   • docs/tool-schema-and-discoverability-strategies.md (tool catalog and discovery strategies)

9. Provision & debug

   Use the Provision button in ATK to create resources. ATK will detect if your MCP server requires OAuth2 or API‐Key. Provide your Client ID/Secret or Key when prompted. Then Start Debugging to Preview agent in Copilot in Edge/Chrome. Your DA will appear under Copilot chats.

10. Test your MCP‑powered agent

    Open the Copilot pane, select your agent and invoke any of the MCP tools with natural‑language prompts.

Project Structure

Folder	Description
.vscode	ATK debug & .vscode/mcp.json for MCP server config
appPackage	- appPackage/ai-plugin.json (function definitions, runtime spec + auth) - This file defines the the action or operations that Copilot can interact with - appPackage/declarativeAgent.json (agent configuration & sample prompts) - This file is the definition of your declarative agent - appPackage/manifest.json (Teams/Outlook integration)
env	Local environment files (.env.development, .env.local)
m365agents.yml	Defines your DA stages & lifecycle for ATK

MCP‑specific tips

• Discovery URL: your MCP server’s /discover endpoint must expose JSON‑Schema for every action.

• Tool selection: run_for_functions in ai-plugin.json binds the declared functions to the remote MCP runtime. This package keeps all declared functions available; descriptions and instructions provide the UX routing layer.

• Auth flows: ATK supports both OAuth2.1 and API‑Key; you don’t need to hand‑edit auth blocks.

• Versioning: when your MCP server schema changes, simply rerun ATK: Fetch Action from MCP to refresh your plugin file.

• Error logging: basic request/response logs appear in the ATK console; errors bubble up in your Copilot chat.

Microsoft capabilities and knowledge

• CodeInterpreter analyzes user-provided traces, JSON, CSV, package files, and evaluation output. It does not authorize or perform MCP writes.
• WebSearch is scoped to the public Kore.ai ABL documentation at https://docs.kore.ai/agent-platform/.
• ABL documentation uses the MCP debug_docs tool or scoped WebSearch. EmbeddedKnowledge is disabled to remove competing bundled-file retrieval. The reference files remain in the repo but are not included in the agent package. See the decision and verification steps.
• The app requests identity only. messageTeamMembers is unnecessary for this declarative-agent/MCP package because it has no proactive Teams messaging surface.

Evaluating Agents

Install the Microsoft 365 Copilot Agent Evaluations CLI (@microsoft/m365-copilot-eval) NPM package to test, measure, and improve the quality of your agent with structured evaluations and rich result reports with AI-based scoring.

Requires Admin consent at tenant level.

1. Run npm install -g @microsoft/m365-copilot-eval

2. Add the following environment variables. See here on how to get them.

   AZURE_AI_OPENAI_ENDPOINT=
   AZURE_AI_API_KEY=
   AZURE_AI_API_VERSION=
   AZURE_AI_MODEL_NAME=
   

3. Provision the project first (select Provision in the Microsoft 365 Agents Toolkit) so the agent is available in your tenant before evaluation. Skip this step if you have already provisioned (or started a local debug session) for this project.

4. Run runevals or runevals --env dev

A sample dataset evals/prompts.json is created in this project to help you get started right away. Read more.

Learn More

• Build Declarative Agents (official docs)

• Model Context Protocol (MCP)

• Agents Toolkit guide on GitHub

Happy building!

With MCP + Declarative Agents, you’ll have a turnkey path from your existing APIs to a fully operational Copilot‑powered experience.