Try the samples
Explore Evals with made-up projects instead of your own data.
Samples
Before you start
All samples need Node.js 24+. For native VS Code Evals views, add the
extension before opening the sample.
The local eval runner (@microsoft/vally-cli) reads the eval definition,
runs its checks and returns results.
In VS Code, choose GitHub Copilot Evals: Set up test runner to find or
separately approve installing a compatible runner before your first eval run.
The no-AI sample uses mock responses and does not need the full Copilot CLI, Git or model sign-in. To test a real agent, the full Copilot CLI runs that agent for the editor; you also need Git and model access. See the runner requirements. Installing tools does not approve running an eval.
- Download the samples ZIP from releases.
- Extract it and open the
project-maintainerfolder. Keep its hidden files. - Open
project-maintainerin your chosen app: start Copilot CLI from that folder, open it in VS Code, or select it for your GitHub Copilot App session’s Canvas. It contains a fictional backlog, skills, agents and evals to explore.
Opening the folder does not sign in, start a model or run an eval. Once it is ready, you can explore it with Create or Review.
Create your first eval with this sample follows the interface you selected on this site. See the examples overview for what the sample covers.
Check the sample tools without AI
From the project-maintainer folder:
node .\helpers\smoke.mjs
This smoke test checks the tools, not how a model uses them.
Try the deliberately flawed version
Use node .\helpers\smoke.mjs --profile mixed to inspect the deliberate negative-control
formatter without AI. Keep expected answers unchanged.
Review .mcp.json before enabling; tools can change sample data.
Build from source
Use the build instructions. Prebuilt samples do not need a source build.
If it fails
If the destination already exists, choose a new folder rather than overwrite
it. Keep hidden files when extracting the archive, and prepare the sample again
on another machine so its paths remain correct. The archive’s START-HERE.md
has the instructions for that version.