Evals helper skills
The skills that help you evaluate your own skills, agents, and tools.
The Evals Plugin includes helpers for the work below. They are not the skill or agent you are testing. Install the Plugin or the VS Code extension, which includes it.
| Helper | Copilot CLI / GitHub Copilot App command | Purpose |
|---|---|---|
| Setup | /eval setup or /eval init | Optionally configure folders and suites. |
| Create | /eval create | Propose an eval from real source and requirements. |
| Review | /eval review | Explain an eval or saved result. |
| Improve | /eval improve | Propose a separate eval version; keep the original. |
| Run | /eval run | Prepare and separately approve execution. |
| Compare | /eval compare | Inspect saved results or explore one skill’s contribution. |
The built Copilot CLI / GitHub Copilot App extension provides these commands. Its six helper skills are hidden from slash menus but can be requested in natural language. Repository skills alone do not provide native commands or the built save/run tools.
In trusted local desktop VS Code, the extension VSIX bundles the same six skills
(setup, create, review, improve, run, and compare) hidden from slash
menus so they can be requested in natural language or through Evals sidebar,
Command Palette, and Canvas actions. There, GitHub Copilot Evals: Compare
opens Canvas’s Compare view; opening it does not
approve preparation or execution.
The install identity is eval@github-copilot-evals; the product name is
GitHub Copilot Evals.
Ask in your own words
Review this eval. What does it check, and what could it miss?
Do not edit or run it.
Improve this eval so it catches a partial answer.
Keep the original and show the proposed changes.
Improve changes the eval, not the tested asset. Review files before saving
and approve runs separately. In Copilot CLI / GitHub Copilot App, describe a goal
after /eval or use the action picker’s Other option. Both accept requests
spanning multiple evals and actions. Copilot considers the whole request, even
when its first word is a command name; structured commands and explicit targets
retain their direct selection flow.
Sample agents
Prepare the samples before using these source paths:
| Agent | Source |
|---|---|
| Project Maintainer | .github\agents\maintainer.agent.md |
| Release Manager | .github\agents\release-manager.agent.md |
Prepared skills live under .agents\skills. Keep the sample’s versions together.
Markdown source alone does not mean your app can run it.
Run your first eval · Configure a project or CI · Understand comparison limits