OpenTelemetry

Observability in GitHub Agentic Workflows often begins with inspecting a single run. That is usually enough at first, but it breaks down once you need retained telemetry, visibility across runs, and a way to investigate without repeatedly spending GitHub API quota or running into rate limits.

OpenTelemetry is the standard way to do that. It lets you export workflow traces through the OpenTelemetry Protocol (OTLP) to backends such as Datadog, Grafana, or Sentry, and read telemetry back through MCP when agents need to investigate. Most workflows should use either write-side OTLP or read-side MCP. Use both only when you need to correlate newly emitted spans with traces already in the backend.

Choose a backend

OpenTelemetry gives you a standard way to export workflow traces. The main choice is which backend should receive and surface that telemetry.

Datadog is a strong fit when workflow telemetry needs to plug into broader operational monitoring and service health.
Grafana is a strong fit for teams that want an OpenTelemetry-first stack for traces, dashboards, and investigation.
Sentry is a strong fit when workflow telemetry should live next to application errors and performance debugging.

Write telemetry through OTLP

This is configured with observability.otlp in workflow frontmatter:

---
network:
  allowed:
    - "*.datadoghq.com"
    - "*.datadoghq.eu"
    - "*.ddog-gov.com"
observability:
  otlp:
    endpoint:
      - url: ${{ secrets.GH_AW_OTEL_DATADOG_ENDPOINT }}
        headers:
          DD-API-KEY: ${{ secrets.DD_API_KEY }}
---

---
network:
  allowed:
    - "*.grafana.net"
observability:
  otlp:
    endpoint:
      - url: ${{ secrets.GH_AW_OTEL_GRAFANA_ENDPOINT }}
        headers:
          Authorization: ${{ secrets.GH_AW_OTEL_GRAFANA_AUTHORIZATION }}
---

---
network:
  allowed:
    - "*.sentry.io"
observability:
  otlp:
    endpoint:
      - url: ${{ secrets.GH_AW_OTEL_SENTRY_ENDPOINT }}
        headers:
          Authorization: ${{ secrets.GH_AW_OTEL_SENTRY_AUTHORIZATION }}
---

We also support sending to multiple OTLP endpoints in the same workflow. Use the array form when the workflow should fan out to more than one collector, for example Datadog and Grafana or Datadog and Sentry.

Once configured, GitHub Agentic Workflows exports built-in workflow spans such as setup and conclusion events to the configured OTLP backend or backends, such as Datadog, Grafana, Sentry, or another OTLP-compatible system. You can also emit custom spans from your workflow code using the OpenTelemetry API and an OTLP-compatible client library in your workflow language. In the backend, those spans are available as traces for querying and drilldown.

Read telemetry through MCP

This is configured with mcp-servers in workflow frontmatter. Use the read path when the agent needs to inspect telemetry that already exists in a backend such as Datadog, Grafana, Sentry, Tempo, or another OpenTelemetry-compatible system.

---
mcp-servers:
  datadog:
    url: "https://mcp.datadoghq.com/api/unstable/mcp-server/mcp"
    headers:
      DD_API_KEY: "${{ secrets.DD_API_KEY }}"
      DD_APPLICATION_KEY: "${{ secrets.DD_APPLICATION_KEY }}"
      DD_SITE: "${{ secrets.DD_SITE || 'datadoghq.com' }}"
    allowed:
      - search_datadog_dashboards
      - search_datadog_slos
      - search_datadog_metrics
      - get_datadog_metric
---

---
mcp-servers:
  grafana:
    container: "grafana/mcp-grafana"
    entrypointArgs:
      - "-t"
      - "stdio"
      - "--disable-write"
    allowed:
      - list_datasources
      - tempo_traceql-search
      - tempo_get-trace
      - tempo_get-attribute-names
      - tempo_get-attribute-values
    env:
      GRAFANA_URL: "${{ secrets.GRAFANA_URL }}"
      GRAFANA_SERVICE_ACCOUNT_TOKEN: "${{ secrets.GRAFANA_SERVICE_ACCOUNT_TOKEN }}"
---

---
mcp-servers:
  sentry:
    command: "npx"
    args: ["@sentry/mcp-server@0.33.0"]
    allowed:
      - whoami
      - find_organizations
      - find_projects
      - get_trace_details
      - search_events
      - search_issues
    env:
      SENTRY_ACCESS_TOKEN: ${{ secrets.SENTRY_ACCESS_TOKEN }}
      SENTRY_HOST: ${{ env.SENTRY_HOST || 'sentry.io' }}
---

In this model, the workflow does not emit new spans by itself. It gives the agent a tool that can query existing traces and spans from an external backend.

Keep both write-side OTLP configuration and read-side MCP configuration in shared workflow files and import them where needed. In this repository, that usually means shared/otlp.md for the combined Sentry and Grafana OTLP pattern, shared/mcp/grafana.md or shared/mcp/datadog.md for read access, and shared/otel-queries.md for the query playbook.

Setup

Choose your backend here and follow the matching setup path. When many workflows need the same observability wiring, shared imports are usually the right default.

To set up Datadog as an OTLP write backend and an MCP read source, follow these steps.

Datadog uses one OTLP endpoint plus DD_API_KEY for write, and DD_API_KEY, DD_APPLICATION_KEY, and DD_SITE for read.

Choose the Datadog site you will use. Keep write-side OTLP export and read-side MCP access pointed at the same Datadog site. Store that site in DD_SITE. Common values include datadoghq.com, datadoghq.eu, ddog-gov.com, us5.datadoghq.com, and ap1.datadoghq.com.
Create a Datadog API key. In Datadog, open Organization Settings, then API Keys, and create a key for workflow telemetry. Store it as DD_API_KEY. This key is used for OTLP write and is also sent to the Datadog MCP server.
Create a Datadog application key. In Datadog, open Organization Settings, then Application Keys, and create a key for read-side investigation workflows. Store it as DD_APPLICATION_KEY. The MCP configuration needs this key in addition to DD_API_KEY.
Store the OTLP endpoint for the same site. Get the Datadog OTLP traces endpoint for the site selected in step 1 and store it as GH_AW_OTEL_DATADOG_ENDPOINT. Keep GH_AW_OTEL_DATADOG_ENDPOINT, DD_API_KEY, DD_APPLICATION_KEY, and DD_SITE aligned to the same Datadog account and site.
Run one workflow to write and one workflow to read. Trigger a workflow that exports spans through OTLP, then run a Datadog-backed investigation workflow that imports shared/mcp/datadog.md. If write-side OTLP and read-side MCP point at different Datadog sites, export may succeed while the investigation workflow reads the wrong account or environment.

To set up Grafana as both an OTLP backend and an MCP source, follow these steps:

Choose a Grafana stack. To start, create or choose a dedicated non-production stack such as a dev, staging, or sandbox stack. If multiple stacks exist, choose one that both exposes OTLP connection details for trace ingestion and is the Grafana instance whose Tempo datasource should be queried by workflows.
Get the OTLP write credentials. Open the selected stack and get to the OpenTelemetry setup page. Depending on where Grafana sends you, this is usually either the Grafana Cloud Portal OpenTelemetry tile with Configure, or the Grafana app UI under Connections, Add new connection, and OpenTelemetry (OTLP). Copy the generated OTLP endpoint and authorization header values. Map those values to gh-aw secrets as follows: GH_AW_OTEL_GRAFANA_ENDPOINT should be the value shown for OTEL_EXPORTER_OTLP_ENDPOINT, and GH_AW_OTEL_GRAFANA_AUTHORIZATION should be the value used in the Authorization header, typically Basic .... Use only the header value for GH_AW_OTEL_GRAFANA_AUTHORIZATION. gh-aw adds the Authorization: header name when compiling the workflow.
Get the Grafana read credentials. In Grafana for the same stack, open Administration, go to Users and access, then Service accounts, create a service account for workflow telemetry queries, and add a token for it. Map those values to gh-aw secrets as follows: GRAFANA_URL should be the base URL of the selected stack, typically https://<stack>.grafana.net, and GRAFANA_SERVICE_ACCOUNT_TOKEN should be the generated service account token. Start with a read-oriented role such as Viewer. If the token can connect to Grafana but cannot query traces, the next thing to check is datasource or Tempo permissions for that service account.
Store all four values for the same stack. For a Grafana-backed workflow, GH_AW_OTEL_GRAFANA_ENDPOINT, GH_AW_OTEL_GRAFANA_AUTHORIZATION, GRAFANA_URL, and GRAFANA_SERVICE_ACCOUNT_TOKEN must all point to the same stack. It is normal for the write-side and read-side credentials to be different values. Grafana OTLP write uses GH_AW_OTEL_GRAFANA_AUTHORIZATION, while Grafana MCP read uses GRAFANA_SERVICE_ACCOUNT_TOKEN.

Import a starter dashboard template

If you want a dashboard people can upload instead of building from scratch, start with gh-aw-observability-starter.json.

Import it in Grafana with Dashboards, then New, then Import dashboard, upload the JSON file, and select your Tempo datasource when Grafana prompts for it.

This starter focuses on the telemetry GitHub Agentic Workflows already emits today:

workflow run volume
P95 agent span duration
input and output tokens over time
cache-read token reuse
engine mix and model mix
recent matching traces for drilldown

It is intentionally a first-pass operations and token-economics dashboard. Cost-versus-value, agentic-versus-deterministic, and portfolio-management views need extra outcome instrumentation in your spans, such as execution mode, accepted outcomes, escalations, or review-required markers.

To set up Sentry as an OTLP write backend, follow these steps.

This section is write-side only. Sentry uses one OTLP endpoint plus one Sentry OTLP auth value, and there is no separate Sentry read token in this guide. If your workflow reads telemetry from another backend, that backend has its own read credentials and they are independent from the Sentry write secrets.

Choose or create a Sentry project. Start with a dedicated non-production project such as gh-aw-dev or gh-aw-sandbox. If your organization already has several Sentry projects, choose the one that should receive workflow traces from GitHub Agentic Workflows. Keep the first setup isolated from production alerting until export is working the way you expect.
Open the project keys page. In Sentry for that project, do not use the project overview or details page. Open Project Settings, then Client Keys (DSN) instead. This is the page Sentry uses for first-time setup and it is also where Sentry exposes the values needed for direct OTLP trace export.
Get the OTLP write credentials. On the same project, copy the direct OTLP traces endpoint and the authentication header Sentry provides for OpenTelemetry export. The header Sentry expects for direct OTLP traces is x-sentry-auth, and the value is commonly in the form sentry sentry_key=.... In other words, the wire format Sentry expects is x-sentry-auth: sentry sentry_key=.... Map those values to gh-aw secrets as follows: GH_AW_OTEL_SENTRY_ENDPOINT should be the direct OTLP traces endpoint for that project, and GH_AW_OTEL_SENTRY_AUTHORIZATION should hold only the Sentry header value, typically sentry sentry_key=.... Do not use the DSN itself as the OTLP endpoint. Use the direct OTLP traces URL for the project. For direct Sentry OTLP, the value is typically not Bearer ... or Basic .... If Sentry shows you a full header expression, store only the header value in GH_AW_OTEL_SENTRY_AUTHORIZATION, not the header name.
Store both values for the same Sentry project. GH_AW_OTEL_SENTRY_ENDPOINT and GH_AW_OTEL_SENTRY_AUTHORIZATION must refer to the same Sentry project. Mixing an endpoint from one project with auth from another is an easy first-time setup mistake and usually results in export failures or data arriving in the wrong place. If you are splitting write and read across different systems, that does not create a second Sentry credential here. It only means Sentry keeps its write-side endpoint and auth, while the separate read backend uses its own read-side credentials.
Run a workflow once and verify traces arrive in Sentry. Trigger a workflow that imports the shared OTLP configuration, then open the same Sentry project and check the trace or performance views for a new run. If nothing arrives, the first things to check are that the secrets were copied from the correct project, the OTLP endpoint is the direct traces endpoint rather than the DSN, and the auth value was stored without the header name.

Imports for bundling shared observability configuration
Architecture for the broader runtime observability model