GitHub Copilot Evals
InstallSearch /github/evals

Examples

Try release notes, backlog triage, and tool checks with fictional project data.

Download a sample project

Open GitHub Releases, choose a version, and download its samples ZIP. It contains Project Maintainer with version-specific START-HERE.md instructions. Keep hidden files when extracting it. Downloads and setup do not start an eval.

Prepare Project Maintainer, then follow your interface’s first-eval guide.

Project Maintainer

Question: does the release-note skill announce only completed work? The fictional backlog supplies the facts for checking the answer. Skills, agents, and local tools have improved and deliberately flawed versions to explore across Create, Run, Review, Improve, and Compare, starting with a no-AI control (mock-baseline.eval.yaml). Do not mix versions or weaken expected answers.

Local tool checks use no AI; they do not test model behavior. Model runs need separate approval.

Design your own case

Check known correct and incorrect answers against the task’s requirements. File or tool lists do not establish correct behavior.

Assets view with the sample release-note skill selected and its redacted source open.
Select the file in Assets. Evals Canvas · sample data