What's included
Understand the parts of your Evals installation.
| Part | What it does | Setup |
|---|---|---|
| Plugin | Adds Evals instructions and tools to Copilot | Install the Plugin |
| Skills | Six skills: Create, Review, Improve, Run, Compare and Setup | Skills & agents |
| Extension | Puts evals and results in VS Code | VS Code extension |
| Canvas workbench | Project files, drafts, run controls and visual comparison | Set up Canvas |
| Command-line runner / CI | Runs supported saved tests automatically | Add evals to CI |
| MCP connection | Lets your app use the shared Evals tools | MCP setup |
| Samples | Skills, agents and tools using made-up project data | Try the samples |
Canvas is a workbench, not a separate runner. The VS Code extension includes the Plugin.
Terms and developer libraries
A skill supplies repeatable instructions; an agent has a role and tools. MCP connects apps to tools. Evals can test the sample agents and servers; they do not add Evals commands. Core and Runtime are libraries for integration developers. Developer reference.
These layers work together: a plugin can distribute a skill and a tool connection, and an agent can use both. Choose the Evals Plugin for supported Copilot workflows; choose standalone MCP for another local tool-capable host. Compare concrete scenarios and MCP limits.
Packages
Download packages from releases.
Read the notes and requirements before installing. Use the file checksums in
SHA256SUMS to verify your download matches the published copy.
Run your first eval after setup. Review files before saving and approve execution separately.