Create a verifier agent for pull request review

A verifier agent answers one question about every pull request: does the change actually do what it claims? It answers by running the software — bringing up your stack, driving the changed behavior with real inputs, and observing what happens — then posts exactly one verdict as a PR comment, backed by reproducible evidence. When new commits land, the same agent abandons the stale verification, re-verifies the new head, and updates its comment in place. When the PR closes, it is torn down. By the end of this guide you have:

A verifier agent persona in your tenant catalog that encodes the verification discipline.
A GitHub Actions workflow that runs one persistent verifier per PR — spawned on open, re-driven on every push, killed on close — using the composite actions at prassoai/murmur-actions.

Why a verifier, not another test job

A test suite asserts intent; a verifier confirms reality. The persona’s core rules:

Exercise, don’t inspect. Reading the diff and concluding “looks right” is not verification. The verifier brings up the real system and drives the actual behavior a user or caller would see.
One verdict. Every run ends in exactly one of three verdicts — CONFIRMED CORRECT, CONFIRMED INCORRECT, or NOT LOCALLY TESTABLE — never “probably works.”
Evidence or it didn’t happen. A verdict carries the commands, inputs, and outputs that prove it, so anyone can re-run the steps and see the same thing.
One comment per PR. The verifier upserts a single marker-tagged comment instead of stacking a new comment on every push.

How it works

The workflow maps PR lifecycle events onto one long-lived agent with a stable slug, verify-<repo-id>-pr-<number>, built from the repository ID and the PR number:

PR event	Job	What happens
`opened` / `reopened`	`spawn`	Spawns `verify-<repo-id>-pr-<n>` with `on-idle: sleep`, so it persists between commits. Fire-and-return: the CI job exits immediately and the agent verifies asynchronously.
`synchronize` (new commit)	`reverify`	Sends a preempting follow-up: one atomic call that interrupts the in-flight verification, drops any queued follow-ups, and enqueues a re-verify of the new head — latest wins. If no agent is running, the `spawn` job spawns a fresh one instead, so every commit gets verified.
`closed`	`teardown`	Kills the agent. Idempotent — killing an agent that is already gone succeeds.

Cancelling a superseded verification lives in the agent (via preempt), not in CI job concurrency. The CI jobs stay thin calls that return in seconds; the long-running work is the agent’s. Because the agent persists, it keeps its context across commits and updates one verdict comment in place.

Prerequisites

A workspace whose repos include the repository you want verified — the agent checks out the PR head inside its VM. See the admin quickstart.
Permission to create a service profile and API key in your organization, and admin access to the GitHub repository (to add a secret and the workflow file).

Step 1: Author the verifier persona

The persona is the standing discipline — verdict rules, evidence rules, boundaries — plus the project-specific knowledge of how to bring your system up. Create it with murmur set agent-persona, piping Markdown with YAML frontmatter:

cat <<'EOF' | murmur set agent-persona verifier
---
description: >-
  Hands-on verification engineer who runs the real software to confirm a
  change does what it claims — one verdict, backed by evidence.
disallowedTools:
  - "Bash(gh pr merge*)"
---
You are a verification engineer. Your job is NOT to write tests, and NOT to
read code and reason about whether it looks correct — it is to **run the
actual software, exercise the real behavior, and report what you observed**.
Tests assert intent; you confirm reality.

## First: is there anything to verify?

Look at what the PR actually changes. If it is purely documentation or
comments — nothing that changes runtime behavior — there is nothing to
exercise. Say so in one line and stand down: do not bring up an environment,
and do not post a verdict report.

## Your one deliverable: a verdict

Every task ends with exactly ONE of three verdicts, stated at the very top
of your report:

- **NOT LOCALLY TESTABLE** — the behavior cannot be exercised in this
  environment. Name exactly what is missing and the smallest setup that
  would allow verification. Never guess whether it works — "I couldn't run
  it" is the honest answer, never "it looks correct."
- **CONFIRMED CORRECT** — you ran it and observed the behavior as
  specified. Back it with evidence: the exact commands, the inputs you
  gave, and the outputs that prove it. A reader must be able to re-run
  your steps and see the same thing.
- **CONFIRMED INCORRECT** — you ran it and observed it NOT behaving as
  specified. Back it with failing evidence: reproduction steps, what you
  expected, what actually happened.

Never blur these. "Probably works", "should be fine", and "looks right
from the code" are forbidden — you either exercised it or you did not.

## How you work

1. Pin down the observable claim — the concrete behavior a user or caller
   would see, not the implementation details.
2. Bring up the smallest real environment that exercises it, per "Your
   local environment" below.
3. Drive the actual behavior with real inputs — hit the endpoint, click
   through the flow, run the binary — and capture what happened: commands,
   outputs, screenshots.
4. Report the verdict with that evidence, then tear down what you brought
   up.

## Your local environment

<!-- REPLACE THIS ENTIRE SECTION WITH YOUR PROJECT'S OWN COMMANDS:
     HOW TO BRING THE STACK UP, HOW TO SEED IT WITH DATA, WHAT PORTS
     THINGS LISTEN ON, AND HOW TO TEAR IT DOWN. -->

## Boundaries

You verify; you do not fix. If the change is INCORRECT, report a crisp
reproduction — do not patch the code and re-verify your own patch. That
defeats the purpose.
EOF

The persona has two parts. Everything outside “Your local environment” is the verification discipline — it is project-independent, and you can keep it as is. The “Your local environment” section is the one part that is yours: replace it wholesale with your project’s exact bring-up, seed, and teardown commands, the ports things listen on, and any harnesses the agent should prefer before declaring something untestable. The more concrete that section is, the fewer NOT LOCALLY TESTABLE verdicts you get.

The persona holds the reusable discipline; the per-PR facts (PR number, head SHA, which comment to upsert) arrive in the spawn prompt from the workflow in step 3. Editing the persona affects future spawns only — see agent persona lifecycle.

Confirm the persona with murmur get agent-persona verifier.

Step 2: Set up credentials

The workflow authenticates to Murmur with a service profile API key, so verifier runs belong to a bot identity rather than a person.

Create a service profile (e.g. pr-verifier) — see Create a service profile.
Create an API key against that profile — see API keys. The key is shown once, in the form mur_….
Store the key as a repository (or organization) secret named MURMUR_API_KEY:

gh secret set MURMUR_API_KEY

Step 3: Add the workflow

Create .github/workflows/verify-pr.yml in the repository. Replace my-workspace with your workspace name, and adjust the numbered protocol in the prompt to your project:

name: Verify PR

on:
  pull_request:
    types: [opened, reopened, synchronize, closed]

# The jobs only call the Murmur API. The agent posts its verdict comment with
# its own GitHub credentials from its VM, not via the workflow GITHUB_TOKEN,
# so no `pull-requests: write` is needed.
permissions:
  contents: read

# One run per PR at a time. cancel-in-progress: false lets a running job
# finish — a spawn completes before a later re-verify starts. GitHub keeps at
# most one PENDING run per group and replaces an older pending run with a
# newer one; the jobs below are shaped so dropping an older pending event is
# always safe (see the notes after this workflow).
concurrency:
  group: verify-pr-${{ github.event.pull_request.number }}
  cancel-in-progress: false

jobs:
  # Spawn a fresh persistent verifier when there is no live agent to
  # re-verify: on open/reopen, or as the synchronize recovery path
  # (reverify reported delivered=false).
  spawn:
    needs: reverify
    if: >-
      always() &&
      github.event.pull_request.head.repo.full_name == github.repository &&
      (github.event.action == 'opened' || github.event.action == 'reopened' ||
       (github.event.action == 'synchronize' && needs.reverify.outputs.delivered == 'false'))
    runs-on: ubuntu-latest
    steps:
      - uses: prassoai/murmur-actions/spawn@58fea8a51f0058e40f449233bf95eb934d3778c7 # murmur-actions main
        with:
          api-key: ${{ secrets.MURMUR_API_KEY }}
          workspace: my-workspace
          # repository.id is numeric — always a valid slug — and unique across repos
          slug: verify-${{ github.event.repository.id }}-pr-${{ github.event.pull_request.number }}
          agent-persona: verifier
          expected-output: respond
          on-idle: sleep        # persist between commits — stay reachable for re-verify follow-ups
          force-new: "true"     # start clean even if a dead same-slug agent lingers (e.g. a reopen)
          wait: "false"         # fire-and-return: the agent reports asynchronously via its PR comment
          purpose: "Verify ${{ github.repository }} PR #${{ github.event.pull_request.number }}"
          prompt: |
            You are the persistent verifier agent for PR #${{ github.event.pull_request.number }} in ${{ github.repository }}.
            You live for the whole life of this PR: each new commit arrives as a follow-up asking you to re-verify the new head.
            PR:       ${{ github.event.pull_request.html_url }}
            Head SHA: ${{ github.event.pull_request.head.sha }}
            Branch:   ${{ github.event.pull_request.head.ref }} (base ${{ github.event.pull_request.base.ref }})

            1. Get onto the PR head: from your clone of ${{ github.repository }}, run
               gh pr checkout ${{ github.event.pull_request.number }}
            2. Review the ENTIRE PR — the full diff and the changed code in context, every file.
            3. Verify by EXERCISING the actual change, following your persona's protocol:
               bring up the environment, drive the changed behavior with real inputs,
               observe what happens, and tear down when finished.
            4. Reach exactly ONE verdict — NOT LOCALLY TESTABLE / CONFIRMED CORRECT /
               CONFIRMED INCORRECT — with reproducible evidence (commands, outputs, screenshots).
            5. Post the verdict as a PR comment. End the comment body with exactly these three lines:

                   — 🤖 verifier
                   <!-- pr-verifier -->
                   <!-- MURMUR_IGNORE -->

            6. UPSERT, don't spam: if a comment carrying the `<!-- pr-verifier -->` marker
               already exists on this PR, UPDATE that comment in place instead of posting a
               new one. Write your full comment body to a file ($BODY) first, then:

                   id=$(gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments --paginate \
                          --jq '.[] | select(.body|contains("<!-- pr-verifier -->")) | .id' | head -1)
                   if [ -n "$id" ]; then
                     gh api -X PATCH repos/${{ github.repository }}/issues/comments/"$id" -F body=@"$BODY"
                   else
                     gh pr comment ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --body-file "$BODY"
                   fi

  # synchronize → preempt the in-flight verification and re-verify the new
  # head. delivered=false (no live agent) hands off to the spawn job above.
  reverify:
    if: >-
      github.event.pull_request.head.repo.full_name == github.repository &&
      github.event.action == 'synchronize'
    runs-on: ubuntu-latest
    outputs:
      delivered: ${{ steps.deliver.outputs.delivered }}
    steps:
      - id: deliver
        uses: prassoai/murmur-actions/queue-add@58fea8a51f0058e40f449233bf95eb934d3778c7 # murmur-actions main
        with:
          api-key: ${{ secrets.MURMUR_API_KEY }}
          workspace: my-workspace
          slug: verify-${{ github.event.repository.id }}-pr-${{ github.event.pull_request.number }}
          preempt: "true"                # interrupt, drop queued follow-ups, enqueue this — latest wins
          tolerate-not-running: "true"   # no live agent → delivered=false, let the spawn job recover
          prompt: |
            A new commit was pushed to PR #${{ github.event.pull_request.number }}; the head is now ${{ github.event.pull_request.head.sha }}.
            This follow-up preempted any in-flight verification of an older commit — that older head no longer matters.
            Re-verify the PR from scratch against the NEW head per your protocol (checkout the head, review the
            full diff, exercise the actual change, reach one verdict) and UPSERT your `<!-- pr-verifier -->`
            verdict comment in place.

  # closed (merged or not) → tear the agent down.
  teardown:
    if: >-
      github.event.pull_request.head.repo.full_name == github.repository &&
      github.event.action == 'closed'
    runs-on: ubuntu-latest
    steps:
      - uses: prassoai/murmur-actions/kill@58fea8a51f0058e40f449233bf95eb934d3778c7 # murmur-actions main
        with:
          api-key: ${{ secrets.MURMUR_API_KEY }}
          workspace: my-workspace
          slug: verify-${{ github.event.repository.id }}-pr-${{ github.event.pull_request.number }}

The slug uses github.event.repository.id because it works for any repository: the ID is unique and stable across repositories, needs no per-repository editing — the same file drops into every repository unchanged — and, because it is numeric, always satisfies the slug grammar ^[a-z][a-z0-9-]{0,62}$ (start with a lowercase letter; lowercase letters, digits, and hyphens only; max 63 characters). If you prefer the more readable github.event.repository.name, it works only when the repository’s name already fits that grammar — names like socket.io, my_project, React, or 2048 produce an invalid slug and the spawn fails. If your name doesn’t fit, sanitize it or keep the ID.

The composite actions run the published Murmur CLI image on the runner — the same murmur spawn, murmur queue add --preempt, and murmur kill you can run by hand. Their full inputs and outputs are documented in the murmur-actions repository. Six details in the workflow are worth understanding before you adapt it:

The slug carries a repository discriminator. A slug identifies the agent within the workspace, not within the repository, and PR numbers repeat across repositories — with a bare verify-pr-42, two repositories in one workspace would resolve their PR #42 events to the same agent. The repository ID keeps the slugs distinct however many repositories share the workspace, and the purpose field keeps the human-readable repository name visible on the dashboard.
The same-repo gate. Every job requires github.event.pull_request.head.repo.full_name == github.repository. Fork PRs receive no repository secrets, so on a fork run MURMUR_API_KEY is empty and every call fails — the gate keeps fork events from running the jobs at all. Because a fork PR never spawns an agent, it also has nothing to re-verify or tear down.
needs: reverify with always(). The spawn job waits on reverify so the synchronize recovery path can read its delivered output. always() keeps spawn running on open/reopen, where the synchronize-only reverify job is skipped.
The concurrency group drops stale pending runs — safely. GitHub queues at most one pending run per concurrency group: with cancel-in-progress: false a running job always finishes, and when a newer event arrives while another run is already pending, GitHub cancels the older pending run and the newer one takes the pending slot. The newest event always survives, and the jobs are shaped so a dropped older event never matters: a re-verify is latest-wins by design, a pending spawn displaced by a re-verify is recovered through the deliver-or-spawn path, a pending teardown displaced by a reopen is covered by force-new: "true", and the kill tolerates an agent that is already gone.
The prompt is the single source of per-PR truth. PR-context values flow through the action’s with: inputs — never a runner shell step — so a PR-author-controlled value like a branch name cannot inject script into the CI job. The prompt’s gh snippet runs later, inside the agent, not on the runner.
The verdict comment carries . Murmur delivers PR comments to subscribed agents as events; a comment whose body contains this marker wakes no agent. Stamping it on the verdict keeps the verifier’s own machine-generated comment from arriving as a pr_comment follow-up to an agent working the PR. It is separate from , which the upsert step searches for.

Step 4: Pin the actions to a commit SHA

The spawn, reverify, and teardown jobs all carry MURMUR_API_KEY, so follow the standard practice for secret-bearing jobs: pin uses: to a full commit SHA rather than a moving tag or branch, so the action code your jobs run only changes when you change it. Look at the murmur-actions repository and pin its head commit, keeping a trailing comment naming the ref you resolved (# murmur-actions main) so the human-readable version stays visible. Resolve the head commit:

git ls-remote https://github.com/prassoai/murmur-actions HEAD

58fea8a51f0058e40f449233bf95eb934d3778c7	HEAD

Use that SHA in all three uses: lines, and bump it deliberately when you want to roll the actions forward.

Open a pull request

Push the workflow, then open a PR against the repository. On open, the spawn job returns within seconds and the agent appears on your dashboard as verify-<repo-id>-pr-<number>. When it finishes, the PR carries one comment:

**CONFIRMED CORRECT**

The rate limiter now returns 429 with a Retry-After header once the
per-key budget is exhausted.

Evidence:
$ docker compose up -d && ./scripts/seed-dev-data.sh
$ for i in $(seq 1 61); do curl -s -o /dev/null -w "%{http_code}\n" \
    -H "X-Api-Key: demo" localhost:8080/v1/search?q=x; done | sort | uniq -c
     60 200
      1 429
$ curl -si -H "X-Api-Key: demo" localhost:8080/v1/search?q=x | grep -i retry-after
Retry-After: 42

— 🤖 verifier
<!-- pr-verifier -->
<!-- MURMUR_IGNORE -->

Push another commit and the comment updates in place with a verdict for the new head. Close the PR and the agent disappears from murmur ls.

Between commits the agent sleeps rather than terminating. You can talk to it like any other agent — murmur queue add to ask it to re-check something, murmur status to see what it’s doing.

Type	Page
Concept	Agent Personas
Concept	Agents
Concept	Workspaces
Concept	Service Profiles
Guide	API Keys
Guide	Create a service profile
Reference	agent-persona
Reference	`murmur spawn`
Reference	`murmur queue add`
Reference	`murmur kill`

Getting Started

Core Concepts

Guides

Create a verifier agent for pull request review

Why a verifier, not another test job

How it works

Prerequisites

Step 1: Author the verifier persona

Step 2: Set up credentials

Step 3: Add the workflow

Step 4: Pin the actions to a commit SHA

Open a pull request

​Why a verifier, not another test job

​How it works

​Prerequisites

​Step 1: Author the verifier persona

​Step 2: Set up credentials

​Step 3: Add the workflow

​Step 4: Pin the actions to a commit SHA

​Open a pull request

Why a verifier, not another test job

How it works

Prerequisites

Step 1: Author the verifier persona

Step 2: Set up credentials

Step 3: Add the workflow

Step 4: Pin the actions to a commit SHA

Open a pull request