Ah! Right this way to our next chamber in Peli’s Agent Factory! The chamber where our AI agents enhance the magical moment of shipping software.
In our previous post, we explored metrics and analytics workflows - the agents that monitor other agents, turning raw activity data into actionable insights.
Shipping software is stressful enough without worrying about whether you formatted your release notes correctly.
Changeset Generator has contributed 22 merged PRs out of 28 proposed (78% merge rate), automating version bumps and changelog generation for every release. It analyzes commits since the last release, determines the appropriate version bump (major, minor, patch), and updates the changelog accordingly.
Daily Workflow Updater keeps GitHub Actions and dependencies current, ensuring workflows don’t fall behind on security patches or new features.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Excellent journey! Now it’s time to plunge into the observatory - the nerve center of Peli’s Agent Factory!
In our previous post, we explored quality and hygiene workflows - the vigilant caretakers that investigate failed CI runs, detect schema drift, and catch breaking changes before users do. These workflows maintain codebase health by spotting problems before they escalate.
When you’re running dozens of AI agents, how do you know if they’re actually working well? How do you spot performance issues, cost problems, or quality degradation? That’s where metrics and analytics workflows come in - they’re the agents that monitor other agents. The aim is to turn raw activity data into actionable insights.
Audit Workflows - A meta-agent that audits all the other agents’ runs
The Metrics Collector has created 41 daily metrics discussions tracking performance across the agent ecosystem - for example, #6986 with the daily code metrics report. It became our central nervous system, gathering performance data that feeds into higher-level orchestrators.
Portfolio Analyst has created 7 portfolio analysis discussions identifying cost reduction opportunities and token optimization patterns - for example, #6499 with a weekly portfolio analysis. The workflow has identified workflows that were costing us money unnecessarily (turns out some agents were way too chatty with their LLM calls).
Audit Workflows is our most prolific discussion-creating agent with 93 audit report discussions and 9 issues, acting as a meta-agent that analyzes logs, costs, errors, and success patterns across all other workflow runs. Four of its issues led to PRs by downstream agents.
Observability isn’t optional when you’re running dozens of AI agents - it’s the difference between a well-oiled machine and an expensive black box.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Ah, splendid! Welcome back to Peli’s Agent Factory! Come, let me show you the chamber where vigilant caretakers investigate faults before they escalate!
In our previous post, we explored issue and PR management workflows.
Now let’s shift from collaboration ceremony to fault investigation.
While issue workflows help us handle what comes in, fault investigation workflows act as vigilant caretakers - spotting problems before they escalate and keeping our codebase healthy. These are the agents that investigate failed CI runs, detect schema drift, and catch breaking changes before users do.
These are our diligent caretakers - the agents that spot problems before they become bigger problems:
CI Doctor - Investigates failed workflows and opens diagnostic issues - 9 merged PRs out of 13 proposed (69% merge rate)
Schema Consistency Checker - Detects when schemas, code, and docs drift apart - 55 analysis discussions created
Breaking Change Checker - Watches for changes that might break things for users - creates alert issues
The CI Doctor (also known as “CI Failure Doctor”) was one of our most important workflows. Instead of drowning in CI failure notifications, we now get timely, investigated failures with actual diagnostic insights. The agent doesn’t just tell us something broke - it analyzes logs, identifies patterns, searches for similar past issues, and even suggests fixes - even before the human has read the failure notification. CI Failure Doctor has contributed 9 merged PRs out of 13 proposed (69% merge rate), including fixes like adding Go module download pre-flight checks and adding retry logic to prevent proxy 403 failures. We learned that agents excel at the tedious investigation work that humans find draining.
The Schema Consistency Checker has created 55 analysis discussions examining schema drift between JSON schemas, Go structs, and documentation - for example, #7020 analyzing conditional logic consistency across the codebase. It caught drift that would have taken us days to notice manually.
Breaking Change Checker is a newer workflow that monitors for backward-incompatible changes and creates alert issues (e.g., #14113 flagging CLI version updates) before they reach production.
These “hygiene” workflows became our first line of defense, catching issues before they reached users.
The CI Doctor has inspired a growing range of similar workflows inside GitHub, where agents proactively do depth investigations of site incidents and failures. This is the future of operational excellence: AI agents kicking in immediately to do depth investigation, for faster organizational response.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Next up, we look at workflows which help us understand if the agent collection as a whole is working well That’s where metrics and analytics workflows come in.
Ah! Let’s discuss the art of managing issues and pull requests at Peli’s Agent Factory! A most delicious topic indeed!
In our previous post, we explored documentation and content workflows - agents that maintain glossaries, technical docs, slide decks, and blog content. We learned how we took a heterogeneous approach to documentation agents - some workflows generate content, others maintain it, and still others validate it.
Now let’s talk about the daily rituals of software development: managing issues and pull requests. GitHub provides excellent primitives for collaboration, but there’s ceremony involved - linking related issues, merging main into PR branches, assigning work, closing completed sub-issues, optimizing templates. These are small papercuts individually, but they can add up to significant friction.
The Issue Arborist is an organizational workflow that has created 77 discussion reports (titled “[Issue Arborist] Issue Arborist Report”) and 18 parent issues to group related sub-issues. It keeps the issue tracker organized by automatically linking related issues, building a dependency tree we’d never maintain manually. For example, #12037 grouped engine documentation updates.
The Issue Monster is the task dispatcher - it assigns issues to the GitHub platform’s asynchronous Copilot coding agent one at a time. It doesn’t create PRs itself, but enables every other agent’s work by feeding them tasks. This prevents the chaos of parallel work on the same codebase.
Mergefest is an orchestrator workflow that automatically merges main into PR branches, keeping long-lived PRs up to date without manual intervention. It eliminates the “please merge main” dance.
Sub Issue Closer automatically closes completed sub-issues when their parent issue is resolved, keeping the issue tracker clean.
Issue and PR management workflows don’t replace GitHub’s features; they enhance them, removing ceremony and making collaboration feel smoother.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Step right up, step right up, and enter the documentation chamber of Peli’s Agent Factory! Pure imagination meets technical accuracy in this most delightful corner of our establishment!
In our previous posts, we explored autonomous cleanup agents - workflows that continuously improve code quality by simplifying complexity, refactoring structure, polishing style, and maintaining overall repository health. These agents never take a day off, quietly working to make our codebase better.
Now let’s address one of software development’s eternal challenges: keeping documentation accurate and up-to-date. Code evolves rapidly; docs… not so much. Terminology drifts, API examples become outdated, slide decks grow stale, and blog posts reference deprecated features. The question isn’t “can AI agents write good documentation?” but rather “can they maintain it as code changes?” Documentation and content workflows challenge conventional wisdom about AI-generated technical content. Spoiler: the answer involves human review, but it’s way better than the alternative (no docs at all).
These agents maintain high-quality documentation and content:
Daily Documentation Updater - Reviews and updates documentation to ensure accuracy and completeness - 57 merged PRs out of 59 proposed (96% merge rate)
Glossary Maintainer - Keeps glossary synchronized with codebase - 10 merged PRs out of 10 proposed (100% merge rate)
Documentation Unbloat - Reviews and simplifies documentation by reducing verbosity - 88 merged PRs out of 103 proposed (85% merge rate)
Documentation Noob Tester - Tests documentation as a new user would, identifying confusing steps - 9 merged PRs (43% merge rate) via causal chain
Slide Deck Maintainer - Maintains presentation slide decks - 2 merged PRs out of 5 proposed (40% merge rate)
Multi-device Docs Tester - Tests documentation site across mobile, tablet, and desktop devices - 2 merged PRs out of 2 proposed (100% merge rate)
Blog Auditor - Verifies blog posts are accessible and contain expected content - 6 audits completed (5 passed, 1 flagged issues)
Documentation is where we challenged conventional wisdom. Can AI agents write good documentation?
The Technical Doc Writer generates API docs from code, but more importantly, it maintains them - updating docs when code changes. The Glossary Maintainer caught terminology drift (“we’re using three different terms for the same concept”).
The Slide Deck Maintainer keeps our presentation materials current without manual updates.
The Multi-device Docs Tester uses Playwright to verify our documentation site works across phones, tablets, and desktops - testing responsive layouts, accessibility, and interactive elements. It catches visual regressions and layout issues that only appear on specific screen sizes.
The Blog Auditor ensures our blog posts stay accurate as the codebase evolves - it flags outdated code examples and broken links. Blog Auditor is a validation-only workflow that creates audit reports rather than code changes. It has run 6 audits (5 passed, 1 flagged out-of-date content), confirming blog accuracy.
Documentation Noob Tester deserves special mention for its exploratory nature. It has produced 9 merged PRs out of 21 proposed (43% merge rate) through a causal chain: 62 discussions analyzed → 21 issues created → 21 PRs. The lower merge rate reflects this workflow’s exploratory nature - it identifies many potential improvements, some of which are too ambitious for immediate implementation. For example, Discussion #8477 led to Issue #8486 which spawned PRs #8716 and #8717, both merged.
AI-generated docs need human/agent review, but they’re dramatically better than no docs (which is often the alternative). Validation can be automated to a large extent, freeing writers to focus on content shaping, topic, clarity, tone, and accuracy.
In this collection of agents, we took a heterogeneous approach - some workflows generate content, others maintain it, and still others validate it. Other approaches are possible - all tasks can be rolled into a single agent. We found that it’s easier to explore the space by using multiple agents, to separate concerns, and that encouraged us to use agents for other communication outputs such as blogs and slides.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
