Blog

Meet the Workflows: Testing & Validation

Jan 13, 2026

Don Syme

Peli de Halleux

Mara Kiefer

Right this way! Let’s continue our grand tour of Peli’s Agent Factory! Into the verification chamber where nothing escapes scrutiny!

In our previous post, we explored ChatOps workflows - agents that respond to slash commands and GitHub reactions, providing on-demand assistance with full context.

But making code better is only half the battle. We also need to ensure it keeps working. As we refactor, optimize, and evolve our codebase, how do we know we haven’t broken something? How do we catch regressions before users do? That’s where testing and validation workflows come in - the skeptical guardians that continuously verify our systems still function as expected. We learned that AI infrastructure needs constant health checks, because what worked yesterday might silently fail today. These workflows embody trust but verify.

Testing & Validation Workflows

These agents keep everything running smoothly through continuous testing:

Code Quality & Test Validation

• Daily Testify Uber Super Expert - Analyzes test files daily and suggests testify-based improvements - 19 issues created, 13 led to merged PRs (100% causal chain merge rate)
• Daily Test Improver - Identifies coverage gaps and implements new tests incrementally
• Daily Compiler Quality Check - Analyzes compiler code to ensure it meets quality standards

User Experience & Integration Testing

• Daily Multi-Device Docs Tester - Tests documentation across devices with Playwright - 2 merged PRs out of 2 proposed (100% merge rate)
• CLI Consistency Checker - Inspects the CLI for inconsistencies, typos, and documentation gaps - 80 merged PRs out of 102 proposed (78% merge rate)

CI/CD Optimization

• CI Coach - Analyzes CI pipelines and suggests optimizations - 9 merged PRs out of 9 proposed (100% merge rate)
• Workflow Health Manager - Meta-orchestrator monitoring health of all agentic workflows - 40 issues created, 5 direct PRs + 14 causal chain PRs merged

The Daily Testify Expert has created 19 issues analyzing test quality, and 13 of those issues led to merged PRs by downstream agents - a perfect 100% causal chain merge rate. For example, issue #13701 led to #13722 modernizing console render tests with testify assertions. The Daily Test Improver works alongside it to identify coverage gaps and implement new tests.

The Multi-Device Docs Tester uses Playwright to test our documentation on different screen sizes - it has created 2 PRs (both merged), including adding —network host to Playwright Docker containers. It found mobile rendering issues we never would have caught manually. The CLI Consistency Checker has contributed 80 merged PRs out of 102 proposed (78% merge rate), maintaining consistency in CLI interface and documentation. Recent examples include removing undocumented CLI commands and fixing upgrade command documentation.

CI Optimization Coach has contributed 9 merged PRs out of 9 proposed (100% merge rate), optimizing CI pipelines for speed and efficiency with perfect execution. Examples include removing unnecessary test dependencies and fixing duplicate test execution.

The Workflow Health Manager has created 40 issues monitoring the health of all other workflows, with 25 of those issues leading to 34 PRs (14 merged) by downstream agents - plus 5 direct PRs merged. For example, issue #14105 about a missing runtime file led to #14127 fixing the workflow configuration.

These workflows embody the principle: trust but verify. Just because it worked yesterday doesn’t mean it works today.

Using These Workflows

You can add these workflows to your own repository and remix them. Get going with our Quick Start, then run one of the following:

Daily Testify Uber Super Expert:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-testify-uber-super-expert.md

Daily Test Improver:

gh aw add-wizard githubnext/agentics/daily-test-improver

Daily Compiler Quality Check:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-compiler-quality.md

Daily Multi-Device Docs Tester:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-multi-device-docs-tester.md

CLI Consistency Checker:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/cli-consistency-checker.md

CI Coach:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/ci-coach.md

Workflow Health Manager:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/workflow-health-manager.md

Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.

You can also create your own workflows.

Learn More

• GitHub Agentic Workflows - The technology behind the workflows
• Quick Start - How to write and compile workflows

Next Up: Monitoring the Monitors

But what about the infrastructure itself? Who watches the watchers? Time to go meta.

Continue reading: Tool & Infrastructure Workflows →

---

This is part 14 of a 19-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Interactive & ChatOps

Jan 13, 2026

Don Syme

Peli de Halleux

Mara Kiefer

Onwards, onwards! Let’s keep exploring the wonders of Peli’s Agent Factory! To the command center where instant magic happens!

In our previous post, we explored creative and culture workflows - agents that bring joy, build team culture, and create moments of delight. We discovered that AI agents don’t have to be all business; they can have personality while making work more enjoyable.

But sometimes you need help right now, at the exact moment you’re stuck on a problem. You don’t want to wait for a scheduled run - you want to summon an expert agent with a command. That’s where interactive workflows and ChatOps come in. These agents respond to slash commands and GitHub reactions, providing on-demand assistance with full context of the current situation.

We learned that the right agent at the right moment with the right information is a valuable addition to an agent portfolio.

Interactive & ChatOps Workflows

These agents respond to commands, providing on-demand assistance whenever you need it:

• Q - Workflow optimizer that investigates performance and creates PRs - 69 merged PRs out of 88 proposed (78% merge rate)
• Grumpy Reviewer - Performs critical code reviews with personality - creates issues for downstream agents
• Workflow Generator - Creates new workflows from issue requests - scaffolds workflow files

Interactive workflows changed how we think about agent invocation. Instead of everything running on a schedule, these respond to slash commands and reactions - /q summons the workflow optimizer, a reaction triggers analysis. Q (yes, named after the James Bond quartermaster) became our go-to troubleshooter - it has contributed 69 merged PRs out of 88 proposed (78% merge rate), responding to commands and investigating workflow issues on demand. Recent examples include fixing the daily-fact workflow action-tag and configuring PR triage reports with 1-day expiration.

The Grumpy Reviewer performs opinionated code reviews, creating issues that flag security risks and code quality concerns (e.g., #13990 about risky event triggers) for downstream agents to fix. It gave us surprisingly valuable feedback with a side of sass (“This function is so nested it has its own ZIP code”).

Workflow Generator creates new agentic workflows from issue requests, scaffolding the markdown workflow files that other agents then refine (e.g., #13379 requesting AWF mode changes).

We learned that context is king - these agents work because they’re invoked at the right moment with the right context, not because they run on a schedule.

Using These Workflows

You can add these workflows to your own repository and remix them. Get going with our Quick Start, then run one of the following:

Q:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/q.md

Grumpy Reviewer:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/grumpy-reviewer.md

Workflow Generator:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/workflow-generator.md

Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.

You can also create your own workflows.

Learn More

• GitHub Agentic Workflows - The technology behind the workflows
• Quick Start - How to write and compile workflows

Next Up: Testing & Validation Workflows

While ChatOps agents respond to commands, we also need workflows that continuously verify our systems still function as expected.

Continue reading: Testing & Validation Workflows →

---

This is part 13 of a 19-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Teamwork & Culture

Jan 13, 2026

Don Syme

Peli de Halleux

Mara Kiefer

Oh, my dear friends! Let’s explore the playful workshop - the most fun corner of Peli’s Agent Factory!

In our previous post, we explored security and compliance workflows - the essential guardrails that manage vulnerability campaigns, validate network security, and prevent credential exposure. These workflows let us sleep soundly knowing our agents operate within safe boundaries.

But here’s the thing: work doesn’t have to be all business. While we’ve built serious, production-critical workflows for quality, releases, and security, we also discovered something unexpected - AI agents can bring joy, build team culture, and create moments of delight. Not every workflow needs to solve a critical problem; some can simply make your day better. Let’s explore the playful side of our agent factory, where we learned that personality and fun drive engagement just as powerfully as utility.

Teamwork & Culture Workflows

These agents facilitate team communication and remind us that work can be fun:

• Daily Team Status - Shares team mood and status updates - 22 issues, 17 discussions (plus 2 causal chain PRs!)
• Daily News - Curates relevant news for the team - 45 news digest discussions
• Poem Bot - Responds to /poem-bot commands with creative verses (yes, really)
• Weekly Issue Summary - Creates digestible summaries complete with charts and trends - 5 weekly analysis discussions
• Daily Repo Chronicle - Narrates the day’s activity like a storyteller - 6 chronicle discussions

The Poem Bot started as a whimsy in our Copilot for PRs project in 2022. Someone said “wouldn’t it be funny if we had an agent that writes poems about our code?” and then we built it. Poem Bot responds to /poem-bot commands with creative verses about code, adding a touch of whimsy to the development workflow. We learned that AI agents don’t have to be all business - they can build culture and create moments of joy.

Daily News has created 45 news digest discussions curating relevant developments for the team - for example, #6932 with the daily status roundup. It shares links, adds commentary and connects them to our work.

Daily Team Status has created 22 issues and 17 discussions sharing daily team status updates - for example, #6930 with the daily team status report. Two of its issues even led to merged PRs by downstream agents, showing that even “soft” workflows can drive concrete improvements.

Weekly Issue Summary has created 5 weekly analysis discussions with digestible summaries, charts, and trends - for example, #5844 analyzing the week of December 1-8, 2025.

Daily Repo Chronicle has created 6 chronicle discussions narrating the repository’s activity like a storyteller - for example, #6750 chronicling a development surge with 42 active PRs.

A theme here is the reduction of cognitive load. Having agents summarize and narrate daily activity means we don’t have to mentally parse long lists of issues or PRs. Instead, we get digestible stories that highlight what’s important. This frees up mental bandwidth for actual work.

Another theme is that tone can help make things more enjoyable. The Daily Repo Chronicle started writing summaries in a narrative, almost journalistic style. The outputs from AI agents don’t have to be robotic - they can have personality while still being informative.

These communication workflows help build team cohesion and remind us that work can be delightful.

Using These Workflows

You can add these workflows to your own repository and remix them. Get going with our Quick Start, then run one of the following:

Daily Team Status:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-team-status.md

Daily News:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-news.md

Poem Bot:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/poem-bot.md

Weekly Issue Summary:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/weekly-issue-summary.md

Daily Repo Chronicle:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-repo-chronicle.md

Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.

You can also create your own workflows.

Learn More

• GitHub Agentic Workflows - The technology behind the workflows
• Quick Start - How to write and compile workflows

Next Up: Summon an Agent on Demand

Scheduled workflows are great, but sometimes you need help right now. Enter ChatOps and interactive workflows.

Continue reading: Interactive & ChatOps Workflows →

---

This is part 12 of a 19-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Security-related

Jan 13, 2026

Don Syme

Peli de Halleux

Mara Kiefer

Splendid! How great to have you back at Peli’s Agent Factory! Now, let me show you the guardian chamber - where the watchful protectors stand vigil!

In our previous post, we explored operations and release workflows that handle the critical process of shipping software - building, testing, generating release notes, and publishing. These workflows need to be rock-solid reliable because they represent the moment when our work reaches users.

But reliability alone isn’t enough - we also need security. When AI agents can access APIs, modify code, and interact with external services, security becomes paramount. How do we ensure agents only access authorized resources? How do we track vulnerabilities and enforce compliance deadlines? How do we prevent credential exposure? That’s where security and compliance workflows become our essential guardrails - the watchful guardians that let us sleep soundly at night.

Security-related Workflows

These agents are our security guards, keeping watch and enforcing the rules:

• Security Compliance - Runs vulnerability campaigns with deadline tracking
• Firewall - Tests network security and validates rules - 59 daily firewall report discussions, 5 smoke test issues
• Daily Secrets Analysis - Scans for exposed credentials (yes, it happens)
• Daily Malicious Code Scan - Reviews recent code changes for suspicious patterns
• Static Analysis Report - Daily security scans using zizmor, poutine, and actionlint - 57 analysis discussions plus 12 Zizmor security reports

Security Compliance manages vulnerability remediation campaigns with deadline tracking, ensuring security issues are addressed within defined SLAs - perfect for those “audit in 3 weeks” panic moments.

The Firewall workflow has created 59 daily firewall report discussions and 5 smoke test issues, validating that our agents can’t access unauthorized resources - for example, #6943 with the daily firewall analysis. It’s the bouncer that enforces network rules.

Daily Secrets Analysis scans for exposed credentials in commits and discussions, providing an automated security net against accidental secret exposure - catching those “oops, I committed my API key” moments before they become incidents.

Daily Malicious Code Scan reviews recent code changes for suspicious patterns, adding an automated defense layer against supply chain attacks.

Static Analysis Report has created 57 analysis discussions plus 12 Zizmor security reports, running comprehensive daily security audits using industry-standard tools - for example, #6973 with the latest static analysis findings and #3033 with a Zizmor security analysis. This shows how traditional security tools can be integrated into an AI agent workflow.

Using These Workflows

You can add these workflows to your own repository and remix them. Get going with our Quick Start, then run one of the following:

Security Compliance:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/security-compliance.md

Firewall:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/firewall.md

Daily Secrets Analysis:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-secrets-analysis.md

Daily Malicious Code Scan:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-malicious-code-scan.md

Static Analysis Report:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/static-analysis-report.md

Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.

You can also create your own workflows.

Learn More

• GitHub Agentic Workflows - The technology behind the workflows
• Quick Start - How to write and compile workflows

Next Up: Teamwork & Culture Workflows

After all this serious talk, let’s explore the fun side: agents that bring joy and build team culture.

Continue reading: Teamwork & Culture Workflows →

---

This is part 11 of a 19-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Operations & Release

Jan 13, 2026

Don Syme

Peli de Halleux

Mara Kiefer

Ah! Right this way to our next chamber in Peli’s Agent Factory! The chamber where our AI agents enhance the magical moment of shipping software.

In our previous post, we explored metrics and analytics workflows - the agents that monitor other agents, turning raw activity data into actionable insights.

Operations & Release Workflows

The agents that help us actually ship software:

• Changeset - Manages version bumps and changelog entries for releases - 22 merged PRs out of 28 proposed (78% merge rate)
• Daily Workflow Updater - Keeps GitHub Actions and dependencies current

Shipping software is stressful enough without worrying about whether you formatted your release notes correctly.

Changeset Generator has contributed 22 merged PRs out of 28 proposed (78% merge rate), automating version bumps and changelog generation for every release. It analyzes commits since the last release, determines the appropriate version bump (major, minor, patch), and updates the changelog accordingly.

Daily Workflow Updater keeps GitHub Actions and dependencies current, ensuring workflows don’t fall behind on security patches or new features.

Using These Workflows

You can add these workflows to your own repository and remix them. Get going with our Quick Start, then run one of the following:

Changeset:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/changeset.md

Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.

You can also create your own workflows.

Learn More

• GitHub Agentic Workflows - The technology behind the workflows
• Quick Start - How to write and compile workflows

Next Up: Security-related Workflows

After all this focus on shipping, we need to talk about the guardrails: how do we ensure these powerful agents operate safely?

Continue reading: Security-related Workflows →

---

This is part 10 of a 19-part series exploring the workflows in Peli’s Agent Factory.

Newer posts

Older posts