Excellent journey! Now it’s time to plunge into the observatory - the nerve center of Peli’s Agent Factory!
In our previous post, we explored quality and hygiene workflows - the vigilant caretakers that investigate failed CI runs, detect schema drift, and catch breaking changes before users do. These workflows maintain codebase health by spotting problems before they escalate.
When you’re running dozens of AI agents, how do you know if they’re actually working well? How do you spot performance issues, cost problems, or quality degradation? That’s where metrics and analytics workflows come in - they’re the agents that monitor other agents. The aim is to turn raw activity data into actionable insights.
Audit Workflows - A meta-agent that audits all the other agents’ runs
The Metrics Collector has created 41 daily metrics discussions tracking performance across the agent ecosystem - for example, #6986 with the daily code metrics report. It became our central nervous system, gathering performance data that feeds into higher-level orchestrators.
Portfolio Analyst has created 7 portfolio analysis discussions identifying cost reduction opportunities and token optimization patterns - for example, #6499 with a weekly portfolio analysis. The workflow has identified workflows that were costing us money unnecessarily (turns out some agents were way too chatty with their LLM calls).
Audit Workflows is our most prolific discussion-creating agent with 93 audit report discussions and 9 issues, acting as a meta-agent that analyzes logs, costs, errors, and success patterns across all other workflow runs. Four of its issues led to PRs by downstream agents.
Observability isn’t optional when you’re running dozens of AI agents - it’s the difference between a well-oiled machine and an expensive black box.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Ah, splendid! Welcome back to Peli’s Agent Factory! Come, let me show you the chamber where vigilant caretakers investigate faults before they escalate!
In our previous post, we explored issue and PR management workflows.
Now let’s shift from collaboration ceremony to fault investigation.
While issue workflows help us handle what comes in, fault investigation workflows act as vigilant caretakers - spotting problems before they escalate and keeping our codebase healthy. These are the agents that investigate failed CI runs, detect schema drift, and catch breaking changes before users do.
These are our diligent caretakers - the agents that spot problems before they become bigger problems:
CI Doctor - Investigates failed workflows and opens diagnostic issues - 9 merged PRs out of 13 proposed (69% merge rate)
Schema Consistency Checker - Detects when schemas, code, and docs drift apart - 55 analysis discussions created
Breaking Change Checker - Watches for changes that might break things for users - creates alert issues
The CI Doctor (also known as “CI Failure Doctor”) was one of our most important workflows. Instead of drowning in CI failure notifications, we now get timely, investigated failures with actual diagnostic insights. The agent doesn’t just tell us something broke - it analyzes logs, identifies patterns, searches for similar past issues, and even suggests fixes - even before the human has read the failure notification. CI Failure Doctor has contributed 9 merged PRs out of 13 proposed (69% merge rate), including fixes like adding Go module download pre-flight checks and adding retry logic to prevent proxy 403 failures. We learned that agents excel at the tedious investigation work that humans find draining.
The Schema Consistency Checker has created 55 analysis discussions examining schema drift between JSON schemas, Go structs, and documentation - for example, #7020 analyzing conditional logic consistency across the codebase. It caught drift that would have taken us days to notice manually.
Breaking Change Checker is a newer workflow that monitors for backward-incompatible changes and creates alert issues (e.g., #14113 flagging CLI version updates) before they reach production.
These “hygiene” workflows became our first line of defense, catching issues before they reached users.
The CI Doctor has inspired a growing range of similar workflows inside GitHub, where agents proactively do depth investigations of site incidents and failures. This is the future of operational excellence: AI agents kicking in immediately to do depth investigation, for faster organizational response.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Next up, we look at workflows which help us understand if the agent collection as a whole is working well That’s where metrics and analytics workflows come in.
Ah! Let’s discuss the art of managing issues and pull requests at Peli’s Agent Factory! A most delicious topic indeed!
In our previous post, we explored documentation and content workflows - agents that maintain glossaries, technical docs, slide decks, and blog content. We learned how we took a heterogeneous approach to documentation agents - some workflows generate content, others maintain it, and still others validate it.
Now let’s talk about the daily rituals of software development: managing issues and pull requests. GitHub provides excellent primitives for collaboration, but there’s ceremony involved - linking related issues, merging main into PR branches, assigning work, closing completed sub-issues, optimizing templates. These are small papercuts individually, but they can add up to significant friction.
The Issue Arborist is an organizational workflow that has created 77 discussion reports (titled “[Issue Arborist] Issue Arborist Report”) and 18 parent issues to group related sub-issues. It keeps the issue tracker organized by automatically linking related issues, building a dependency tree we’d never maintain manually. For example, #12037 grouped engine documentation updates.
The Issue Monster is the task dispatcher - it assigns issues to the GitHub platform’s asynchronous Copilot coding agent one at a time. It doesn’t create PRs itself, but enables every other agent’s work by feeding them tasks. This prevents the chaos of parallel work on the same codebase.
Mergefest is an orchestrator workflow that automatically merges main into PR branches, keeping long-lived PRs up to date without manual intervention. It eliminates the “please merge main” dance.
Sub Issue Closer automatically closes completed sub-issues when their parent issue is resolved, keeping the issue tracker clean.
Issue and PR management workflows don’t replace GitHub’s features; they enhance them, removing ceremony and making collaboration feel smoother.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
Step right up, step right up, and enter the documentation chamber of Peli’s Agent Factory! Pure imagination meets technical accuracy in this most delightful corner of our establishment!
In our previous posts, we explored autonomous cleanup agents - workflows that continuously improve code quality by simplifying complexity, refactoring structure, polishing style, and maintaining overall repository health. These agents never take a day off, quietly working to make our codebase better.
Now let’s address one of software development’s eternal challenges: keeping documentation accurate and up-to-date. Code evolves rapidly; docs… not so much. Terminology drifts, API examples become outdated, slide decks grow stale, and blog posts reference deprecated features. The question isn’t “can AI agents write good documentation?” but rather “can they maintain it as code changes?” Documentation and content workflows challenge conventional wisdom about AI-generated technical content. Spoiler: the answer involves human review, but it’s way better than the alternative (no docs at all).
These agents maintain high-quality documentation and content:
Daily Documentation Updater - Reviews and updates documentation to ensure accuracy and completeness - 57 merged PRs out of 59 proposed (96% merge rate)
Glossary Maintainer - Keeps glossary synchronized with codebase - 10 merged PRs out of 10 proposed (100% merge rate)
Documentation Unbloat - Reviews and simplifies documentation by reducing verbosity - 88 merged PRs out of 103 proposed (85% merge rate)
Documentation Noob Tester - Tests documentation as a new user would, identifying confusing steps - 9 merged PRs (43% merge rate) via causal chain
Slide Deck Maintainer - Maintains presentation slide decks - 2 merged PRs out of 5 proposed (40% merge rate)
Multi-device Docs Tester - Tests documentation site across mobile, tablet, and desktop devices - 2 merged PRs out of 2 proposed (100% merge rate)
Blog Auditor - Verifies blog posts are accessible and contain expected content - 6 audits completed (5 passed, 1 flagged issues)
Documentation is where we challenged conventional wisdom. Can AI agents write good documentation?
The Technical Doc Writer generates API docs from code, but more importantly, it maintains them - updating docs when code changes. The Glossary Maintainer caught terminology drift (“we’re using three different terms for the same concept”).
The Slide Deck Maintainer keeps our presentation materials current without manual updates.
The Multi-device Docs Tester uses Playwright to verify our documentation site works across phones, tablets, and desktops - testing responsive layouts, accessibility, and interactive elements. It catches visual regressions and layout issues that only appear on specific screen sizes.
The Blog Auditor ensures our blog posts stay accurate as the codebase evolves - it flags outdated code examples and broken links. Blog Auditor is a validation-only workflow that creates audit reports rather than code changes. It has run 6 audits (5 passed, 1 flagged out-of-date content), confirming blog accuracy.
Documentation Noob Tester deserves special mention for its exploratory nature. It has produced 9 merged PRs out of 21 proposed (43% merge rate) through a causal chain: 62 discussions analyzed → 21 issues created → 21 PRs. The lower merge rate reflects this workflow’s exploratory nature - it identifies many potential improvements, some of which are too ambitious for immediate implementation. For example, Discussion #8477 led to Issue #8486 which spawned PRs #8716 and #8717, both merged.
AI-generated docs need human/agent review, but they’re dramatically better than no docs (which is often the alternative). Validation can be automated to a large extent, freeing writers to focus on content shaping, topic, clarity, tone, and accuracy.
In this collection of agents, we took a heterogeneous approach - some workflows generate content, others maintain it, and still others validate it. Other approaches are possible - all tasks can be rolled into a single agent. We found that it’s easier to explore the space by using multiple agents, to separate concerns, and that encouraged us to use agents for other communication outputs such as blogs and slides.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
In our previous posts, we’ve explored autonomous cleanup agents. Now we complete the picture with agents that analyze dependencies, type safety, and overall repository quality.
The Go Module Usage Expert is perhaps the most uniquely characterized workflow in the factory - an “enthusiastic Go module expert” who performs daily deep-dive reviews of the project’s Go dependencies. This isn’t just dependency scanning - it’s thoughtful analysis of how well we’re using the tools we’ve chosen.
Most dependency tools focus on vulnerabilities or outdated versions. Go Module Usage Expert asks deeper and more positive questions: Are we using this module’s best features? Have recent updates introduced better patterns we should adopt? Could we use a more appropriate module for this use case? Are we following the module’s recommended practices?
Go Module Usage Expert uses an intelligent selection algorithm. It extracts direct dependencies from go.mod, fetches GitHub metadata for each dependency including last update time, sorts by recency to prioritize recently updated modules, uses round-robin selection to cycle through modules ensuring comprehensive coverage, and maintains persistent memory through cache-memory to track which modules were recently reviewed.
This ensures recently updated modules get reviewed first since new features might be relevant, all modules eventually get reviewed so nothing is forgotten, and reviews don’t repeat unnecessarily thanks to cache tracking.
For each module, Go Module Usage Expert researches the repository (releases, docs, best practices), analyzes actual usage patterns using Serena, and generates actionable recommendations. It saves summaries under scratchpad/mods/ and opens GitHub Discussions.
The output of Go Module Usage Expert is a discussion, which is then often “task mined” for actionable tasks using the TaskOps design pattern.
Let’s take a look at an example of how this works:
Go Module Usage Expert created the Go Module Review: actionlint discussion after noticing the actionlint module was updated.
Through this multi-agent causal chain pattern, Go Module Usage Expert has generated 58 merged PRs out of 74 proposed (78% merge rate) across 67 module reviews. Notable chains include: spinner improvements (4 PRs from briandowns/spinner review), MCP SDK v1.2.0 upgrade (5 PRs from go-sdk review), and terminal styling overhaul (3 PRs from lipgloss review).
The Typist analyzes Go type usage patterns with a singular focus: improving type safety. It hunts for untyped code that should be strongly typed, and identifies duplicated type definitions that create confusion.
Typist looks for untyped usages: interface{} or any where specific types would be better, untyped constants that should have explicit types, and type assertions that could be eliminated with better design. It also hunts for duplicated type definitions - the same types defined in multiple packages, similar types with different names, and type aliases that could be unified.
Using grep patterns and Serena’s semantic analysis, it discovers type definitions, identifies semantic duplicates, analyzes untyped usage patterns, and generates refactoring recommendations.
Typist also uses the TaskOps pattern. This means the job of Typist is not to fix code, but to analyze code and recommend possible improvements.
Let’s take a look at an example of this in practice:
Through this multi-agent causal chain, Typist has produced 19 merged PRs out of 25 proposed (76% merge rate) from 57 discussions → 22 issues → 25 PRs. The blog example (Discussion #4082 → Issue #4155 → PR #4158) is a verified causal chain.
The static v. dynamic typing debate has raged for decades. Today’s hybrid languages like Go, C#, TypeScript and F# support both strong and dynamic typing. Continuous typing improvement offers a new and refreshing perspective on this old debate: rather than enforcing strict typing upfront, we can develop quickly with flexibility, then let autonomous agents like Typist trail behind, strengthening type safety over time. This allows us to get the best of both worlds: rapid development without getting bogged down in type design, while still achieving strong typing and safety as the codebase matures.
Functional Pragmatist applies moderate functional programming techniques to improve code clarity and safety, balancing pragmatism with functional principles.
The workflow focuses on seven patterns: immutability, functional initialization, transformative operations (map/filter/reduce), functional options pattern, avoiding shared mutable state, pure functions, and reusable logic wrappers.
It searches for opportunities (mutable variables, imperative loops, initialization anti-patterns, global state), scores by safety/clarity/testability improvements, uses Serena for deep analysis, and implements changes like converting to composite literals, using functional options, eliminating globals, extracting pure functions, and creating reusable wrappers (Retry, WithTiming, Memoize).
The workflow is pragmatic: Go’s simple style is respected, for-loops stay when clearer, and abstraction is added only where it genuinely improves code. It runs Tuesday and Thursday mornings, systematically improving patterns over time.
Functional Pragmatist (originally named “Functional Enhancer”) is a recent addition - so far it has created 2 PRs (both merged, 100% merge rate), demonstrating that its pragmatic approach to functional patterns is well-received.
Repository Quality Improver takes the widest view, selecting a different focus area each day to analyze the repository from that perspective.
It uses cache memory to ensure diverse coverage: 60% custom areas (repository-specific concerns), 30% standard categories (code quality, documentation, testing, security, performance), and 10% revisits for consistency.
Standard categories cover fundamentals. Custom areas are repository-specific: error message consistency, CLI flag naming conventions, workflow YAML generation patterns, console output formatting, configuration validation.
The workflow loads recent history, selects the next area, spends 20 minutes on deep analysis, generates discussions with recommendations, and saves state. It looks for cross-cutting concerns that don’t fit neatly into other categories but impact overall quality.
Example reports from our own use of this workflow are:
Through its multi-agent causal chain (59 discussions → 30 issues → 40 PRs), Repository Quality Improver has produced 25 merged PRs out of 40 proposed (62% merge rate), taking a holistic view of quality from multiple angles.
Then edit and remix the workflow specifications to meet your needs, regenerate the lock file using gh aw compile, and push to your repository. See our Quick Start for further installation and setup instructions.
