GitHub Integration Design

HerdOS treats GitHub as its database, event bus, and execution platform. All state lives in Issues, PRs, and Actions — no local database, no custom server. This document consolidates the design for how HerdOS uses GitHub’s primitives.

1. Issues as Work Items

Every task a worker executes is defined by a GitHub Issue. Issues carry structured metadata in YAML front matter and human-readable sections that become the agent’s prompt.

Label Taxonomy

All labels use the herd/ prefix to avoid collisions. Created automatically by herd init.

Status labels (mutually exclusive — an issue has exactly one at any time):

Type labels:

Issue Body Format

Issues use YAML front matter followed by structured markdown sections. The front matter carries machine-readable metadata (schema version, batch number, dependency list, file scope, complexity estimate, optional runner label override). Integrator-created issues add fields for fix cycle tracking, batch PR reference, and conflict resolution context.

The body sections serve distinct purposes:

• Task — what to build (concise summary)
• Implementation Details — how to build it, with exact file paths, function signatures, and references to existing code. Makes the issue self-contained so the worker never needs to read other issues.
• Conventions — project-specific patterns discovered by the Planner during codebase exploration
• Context from Dependencies — information from upstream issues inlined by the Planner, so the worker has full context without traversing the dependency graph
• Acceptance Criteria — concrete, verifiable checks
• Files to Modify — explicit list of files to create or edit

Workers parse the front matter for metadata and pass the human-readable sections directly to the agent as the prompt. Manual issues (created by users) are also supported — YAML front matter is optional, and without it the full body becomes the prompt.

Body Size Limit

GitHub caps issue and comment bodies at 65,536 characters. When herd creates an issue whose body would exceed a 65,000-char safety margin (e.g. a /herd fix issue with a long conversation history, a Review fix-issue with many findings, a CI fix-issue with a large log excerpt, or a conflict-resolution issue), the body is truncated at the last clean newline boundary and ends with the marker:

⚠️ Body truncated to fit GitHub’s 65536-char limit. See follow-up comment on this issue for the rest.

The remainder is posted as one or more follow-up comments on the same issue. When the overflow spans multiple comments, each is prefixed with a Part N of M (continued from issue body) header so readers and the worker agent can reassemble them in order. Workers read the full picture from the issue body plus its comments, so no content is lost.

Lifecycle State Machine

                    +-------------------------+
                    |                         |
                    v                         |
+----------+   +----------+   +-----------------+   +--------+
| blocked  |-->|  ready   |-->|  in-progress    |-->|  done  |
+----------+   +----------+   +-----------------+   +--------+
     |              ^                |
     |              |                |
     |              |                v
     |              |          +----------+
     |              +----------| failed   |
     |             (re-dispatch)+----------+
     |              |                |
     |              v                v
     |         +------------+
     +-------->| cancelled  |  (terminal — batch PR closed without merge)
               +------------+

Transition Table

2. Actions Workflows

HerdOS ships three workflow files, installed by herd init into .github/workflows/. All follow a thin-YAML philosophy: the workflow does checkout and calls a single herd CLI command. All logic lives in the Go binary, making it portable across platforms.

Secrets Management

Agent credentials are stored as repository secrets. The workflows pass all supported agent secrets; the CLI uses whichever matches the configured provider. Only one needs to be set. For Claude Code, the OAuth token is recommended as it shares an existing Pro/Max subscription at no extra per-token cost. On self-hosted runners where the agent is already authenticated locally, no agent secrets may be needed.

GITHUB_TOKEN is provided automatically by GitHub Actions with permissions scoped to the repository.

3. Event Architecture

HerdOS uses GitHub’s event system instead of polling. Events chain together to drive the full lifecycle from dispatch through merge.

USER ACTION           GITHUB EVENT              HERDOS RESPONSE
-----------           ------------              ---------------

herd dispatch #42  -> workflow_dispatch       -> Worker starts
                                                      |
Worker completes   -> workflow_run.completed  -> Integrator consolidates
                                                      |
Tier complete      -> (integrator logic)      -> Dispatch next tier
                                                      |
Manual task closed -> issues.closed           -> Integrator advances + reviews
                                                      |
All tiers done     -> (integrator logic)      -> Batch PR opened
                                                      |
CI check completes -> check_run.completed     -> Integrator checks CI status, dispatches fix worker if needed
                                                      |
Agent review       -> (integrator logic)      -> Approve or fix cycle
                                                      |
Human approves PR  -> pull_request_review     -> Integrator merges (if CI passes)
                                                      |
Batch PR closed    -> pull_request.closed     -> Issues closed, cleanup
                                                (merged: issues closed as-is)
                                                (not merged: non-done issues
                                                 labelled cancelled, then closed)
                                                      |
Cron fires         -> schedule                -> Monitor patrols
Worker fails       -> workflow_dispatch       -> Monitor patrols (immediate)
Comment posted     -> issue_comment.created   -> handle-comment parses and executes

Event Types

• workflow_dispatch — primary dispatch mechanism. Only users with write access can trigger it (enforced by GitHub). The ref parameter points to the branch containing the workflow YAML, not the branch the worker checks out.
• workflow_run — triggers the Integrator when a worker completes (success or failure). When integrator.ci_workflows is configured, CI workflow completions still run herd integrator check-ci --ci-run-id for each configured workflow completion; only the follow-up automatic review can skip cheaply if the current PR head SHA already has an approved Herd review-result marker.
• check_run — triggers the Integrator when an individual CI check completes on a batch branch. Unlike check_suite, check_run fires per individual check completion and is more reliable across external CI providers. The job fires multiple times (once per check_run completion) but is idempotent: it calls GetCombinedStatus which returns “pending” if other checks are still running (no-op), “success” if all passed (no-op), or “failure” if all done with failures (dispatches fix workers up to ci_max_fix_cycles). CheckCI also pauses dispatching a new CI fix worker if any fix-type worker — review fix, CI fix, or conflict resolution — is still in progress in the same batch milestone; the next workflow_run trigger when that worker completes re-runs CheckCI. The integrate job no longer runs check-ci inline, as it executed immediately after consolidation when CI hadn’t started yet and always saw “pending”. The Monitor patrol remains as a fallback, checking CI status on open batch PRs every 15 minutes.
• issues — triggers the Integrator when an issue is closed. Used for manual task completion — the Integrator advances the tier and runs agent review if all tiers are done.
• pull_request_review — triggers the Integrator to merge batch PRs after human approval + CI pass.
• pull_request — triggers cleanup when a batch PR is closed (merged or not). On merge: issues are closed, milestone is closed, branch is deleted. On close without merge: non-done issues are labelled herd/status:cancelled before closing, milestone is closed, and branch is deleted. Issues already herd/status:done are closed without relabelling in both cases.
• schedule — triggers Monitor patrol. GitHub may delay or skip scheduled runs under load; the Monitor is stateless and catches up on the next patrol.
• issue_comment — triggers the Integrator’s handle-comment job when a comment starting with /herd is posted. The workflow validates the commenter’s permissions, parses the command, and executes it. This is the entry point for both user-initiated commands and Monitor-posted commands (retry, fix-ci).

All workflows require the HERD_ENABLED repository variable to be set to true. This prevents workflow storms when herd init pushes workflow files before runners are configured. All ${{ }} expressions in run: blocks are passed through environment variables to prevent shell injection.

The checkout action in all workflows uses HERD_GITHUB_TOKEN (falling back to GITHUB_TOKEN) to configure git credentials for pushes. HERD_GITHUB_TOKEN is a fine-grained PAT with the following permissions:

Worker and integrator jobs that need repository history use actions/checkout with fetch-depth: 0. This is required for local merge/rebase operations and lets agent review prefer local git diffs before using GitHub raw diff or changed-file metadata fallback for large PRs.

Issues auto-close via GitHub’s native “Closes #N” references in the batch PR description when the PR is merged. When a batch PR is closed without merging, the Integrator explicitly closes milestone issues after labelling non-done issues as cancelled. Dependency unblocking is handled by the Integrator’s tier advancement logic, with the Monitor as a safety net.

Infinite Loop Prevention

Several automated feedback loops exist. Each has explicit termination:

Additional safeguards: actions performed with GITHUB_TOKEN do not trigger further workflow runs (GitHub’s built-in loop prevention), label filters restrict reactions to herd/-prefixed labels, guard clauses check for bot actors, and all state changes are idempotent.

Concurrency and Race Conditions

Multiple workers can complete near-simultaneously, triggering concurrent Integrator runs.

Concurrent consolidation: A single herd integrator consolidate --run-id <N> invocation is a milestone-wide operation: it enumerates every herd/status:done issue in the run’s milestone whose worker branch is still on the remote and merges each in a deterministic, idempotent loop. If two integrator runs race, GitHub Actions concurrency groups serialize the integrate job per head_branch; if a run is cancelled mid-loop and leaves stranded worker branches, the next successful integrator run re-scans the milestone and picks them up automatically. If two pushes into the batch branch still race (e.g., a manual /herd integrate overlapping with a workflow_run trigger), the loser observes a non-fast-forward push, relabels the affected issue as herd/status:failed for retry, and continues with the next candidate.

Double-dispatch prevention: The advance command uses issue labels as an atomic guard — it sets in-progress before dispatching, and skips any issue already in-progress. Label transitions via the GitHub API are atomic, so only one advance call can transition a given issue. As a belt-and-suspenders check, dispatchReadyIssues also lists in_progress and queued worker runs and skips any tier issue whose number already appears in an active run’s inputs (labels are left untouched and the issue does not count against dispatched). This catches edge cases where labels and run state diverge — for example, if a prior dispatch’s HTTP call failed after GitHub queued the run.

General rule: All operations are idempotent. Labeling an already-labeled issue, dispatching an already-in-progress issue, and merging an already-merged branch are all detected and skipped.

Review has two separate protections: review locks serialize active review attempts, while review-result markers make automatic review idempotent after Herd has approved an exact PR head SHA.

Workflow Concurrency Groups

All integrator workflow jobs use GitHub Actions concurrency groups as defense-in-depth to reduce obvious overlap. Without these, two workers completing close together (or a /herd integrate comment firing at the same time as a workflow_run trigger) can cause duplicate fix issues, duplicate worker dispatches, and duplicate review attempts.

All groups use cancel-in-progress: false (except check-ci-on-completion) so that queued runs wait rather than being cancelled. Every integrator run should complete — we want serialization, not cancellation.

Cross-job limitation: Workflow concurrency cannot express one shared review group across every trigger HerdOS supports. Different event payloads expose different identifiers (workflow_run, check_run, issues, pull_request_review, and issue_comment), and manual commands do not always have the same batch PR fields available at expression time. HerdOS therefore treats workflow concurrency as a secondary guard. The correctness guarantee for agent review comes from an application-level GitHub-backed lock on the batch PR.

Agent Review Locking

Batch PR reviews acquire an application-level GitHub-backed lock before the Integrator fetches context or starts the review agent. The lock branch remains herd/review-lock/pr-N for PR N, but blocking is scoped to the current PR head. An active lock with a valid recorded batch_branch_sha equal to the current head blocks duplicate review for that head; an active lock with a valid recorded batch_branch_sha from an older head is stale for the updated diff and may be reclaimed immediately. Independent batch PRs can still review concurrently. Workflow concurrency is defense-in-depth only; correctness comes from the app-level lock and GitHub’s atomic ref-update behavior.

Lock acquisition and release are compare-and-swap operations against opaque lock state stored in GitHub. Lock history is append-only on herd/review-lock/pr-N: normal acquisition appends a lock commit, release appends a release commit, and old-head reclaim appends a replacement lock commit for the current head. If two runs attempt to acquire, reclaim, or release the lock concurrently, only the run whose view of the current state is still valid succeeds; the other run reloads the latest state and either retries or observes that another review already owns the lock. GitHub atomic ref updates ensure concurrent stale reclaim still has one winner. A release is scoped to the holder that acquired the lock, so stale holders cannot release a newer review’s lock.

Duplicate triggers do not launch another agent review. They log or comment:

Review already in progress for PR #N; skipping duplicate review trigger.

Stale locks expire conservatively after their recorded expiry window so a dead Actions job does not block future reviews forever. The implementation treats expiry as recovery from an abandoned review, not as permission for overlapping same-head active reviews. Manual /herd review still bypasses stable-disagreement suspension, but it never bypasses a same-head active-review lock.

Malformed lock state and legacy active locks without a recorded head SHA fail closed or preserve existing blocking behavior until release or expiry rather than risking duplicate reviews. Pre-append-only lock branches from older HerdOS versions are migrated only when their old marker commit is recognizable as a Herd review lock and is stale according to its acquisition timestamp. Unknown or fresh legacy state fails closed rather than risking duplicate reviews.

Hidden PR comments are not authoritative for locking. If diagnostic lock comments exist, HerdOS filters them out of review context and never uses them to decide whether a lock is active; the GitHub-backed lock state is authoritative.

Review Result Idempotency

Every batch Herd review result comment includes a hidden herd:review-result marker with version, pr_number, batch_number, head_sha, status, cycle, findings_count, and created_at. This marker records the review outcome for the exact PR head SHA that was reviewed; it is separate from the active-review lock and is not used to decide whether another review is currently running.

Automatic review triggers skip the agent when the current PR head SHA already has an approved Herd review-result marker. The skip is logged instead of posted as another PR comment, so repeated automatic triggers do not add PR noise. Manual /herd review intentionally forces a fresh review for the current head SHA, even if an approved marker already exists.

A new commit changes the PR head SHA and invalidates prior approved-head idempotency, so the updated diff can be reviewed. changes_requested and max_cycles_hit markers do not suppress review as approved; existing fix-worker gates, max-cycle rules, stable-disagreement rules, and stale-head checks still govern those flows.

4. Runners

Workers execute on GitHub Actions self-hosted runners. HerdOS provides a Docker image as the primary deployment method.

Runner Types

Self-hosted runners do not consume GitHub Actions minutes.

Docker Runner Concept

The base image is intentionally minimal: GitHub Actions runner binary (configured with --ephemeral), herd CLI, git, and gh CLI. No agent, no programming languages, no build tools. Users extend it with their project’s toolchain.

Ephemeral lifecycle:

1. Boot — creates a registration token via GitHub API, registers with --ephemeral
2. Idle — long-polls GitHub for work
3. Job — executes one job, then exits
4. Restart — Docker restart: always brings a clean container. No leftover files or stale state.
5. Shutdown — traps SIGTERM/SIGINT, deregisters the runner from GitHub before exiting. No ghost runners.

Every job runs in a clean environment at the cost of a few seconds overhead for container restart and re-registration. Multiple replicas run as independent ephemeral runners (3 replicas = 3 concurrent workers).

Runner Labels

Labels route jobs to runners with specific capabilities.

The default label is configured in .herdos.yml. Individual issues can override it via the runner_label front matter field, set by the Planner when a task requires specific hardware.

Resource Considerations

Each worker needs approximately 2 GB RAM for the agent. CPU is less of a constraint (workers are mostly I/O-bound waiting for API responses). Disk usage is per-job since ephemeral runners check out the full repo each time.

5. Permissions and Security

Access Control

GitHub enforces the security boundary: only users with write access can trigger workflow_dispatch.

Write access is the minimum required to use HerdOS.

Branch Protection Recommendations

Batch PRs should go through branch protection. Two modes:

• Review required (default): A human reviews the consolidated batch PR. HerdOS reports review coverage and may split large diffs into bounded review chunks before approval. Recommended starting point.
• No review required: Fully autonomous with auto_merge: true. Best for trusted codebases with strong CI.

A middle ground: require review only for PRs touching specific paths (security-sensitive code, configuration).

Workflow Permissions

Each workflow’s GITHUB_TOKEN is scoped to the minimum required:

• Worker: contents write (push branches), issues write (update labels), actions write (trigger Monitor on failure)
• Integrator: contents write, issues write, pull-requests write (create/manage batch PRs), actions write (dispatch next tier and fix workers)
• Monitor: contents read, issues write, actions write (check run status, dispatch workers)

Runner registration requires admin access to the repository (for the registration token API call). This is a one-time setup operation.

Runner Security

Private repos (recommended): Self-hosted runners are reasonably safe. Only collaborators can trigger workflows, workflow files are reviewed via PR, and workflow_dispatch requires write access.

Public repos (caution): Self-hosted runners are a security risk. Fork PRs can trigger workflows, potentially executing arbitrary code on your runner and exposing secrets. Recommendations: use GitHub-hosted runners instead, disable fork PR triggers, run in containers with no host access, use a dedicated machine, and rotate credentials if compromised.

For maximum isolation, run each runner in a container with ephemeral mode so it is destroyed after each job.

Comment Command Permissions

Comment commands (/herd) are restricted to users with OWNER, MEMBER, or COLLABORATOR association on the repository, plus bot accounts. This is enforced by the CLI, not by GitHub’s native permissions. The author_association field from the webhook payload is used for validation.

Automatic Retry on Transient Errors

The GitHub API client automatically retries requests that receive transient server errors. This handles intermittent GitHub outages without requiring manual intervention or workflow re-runs.

Network errors and client errors (4xx) are not retried — these indicate a problem with the request itself, not a transient server issue. If the request’s context is cancelled during a backoff wait, the retry loop exits immediately.

The retry transport wraps the underlying http.RoundTripper, so it applies to all GitHub API calls made through the client, with one exception: POST requests targeting /actions/workflows/{id}/dispatches (workflow_dispatch) are sent as a single round trip with no retries. workflow_dispatch is not idempotent — GitHub may queue the run internally and then return a 5xx, so a retry would trigger a duplicate worker run. Callers that need to surface a transient dispatch failure must handle the 5xx themselves.

Rate Limits and Audit

GitHub API limits (5,000 req/hour REST, 500 dispatches/10 min) are not a concern for typical usage (5-20 issues per session). All operations are traceable through GitHub’s existing audit mechanisms: issue history, action logs, PR history, and workflow run logs. No additional audit infrastructure is needed.

6. GitHub App

Why a GitHub App

1. Bot identity — commits show the HerdOS logo via the herd-os[bot] account, giving clear visual attribution in the commit history
2. Fine-grained permissions — scoped to exactly what HerdOS needs, with no personal token required for workflow operations
3. No seat cost — GitHub Apps do not consume a seat in your organization
4. Installation-level auth — tokens are scoped to the repos where the App is installed, limiting blast radius

Commit Co-Authorship

Worker commits use the dispatching user as author and HerdOS as co-author via the Co-authored-by git trailer. The user’s git identity (name and email) is captured at dispatch time from git config and passed to the worker. GitHub renders the App’s avatar alongside the user’s in the commit history — the same mechanism used by Dependabot and Renovate. This can be disabled with pull_requests.co_author: false.

Two Authentication Modes

Mode 1: Personal Token (default). Users authenticate with their own token. The App is only used for commit attribution. This is the default for herd init.

Mode 2: App Installation Token (recommended for teams). The GitHub App generates installation tokens for API calls, removing the dependency on personal tokens and giving the organization control over permissions. herd init detects whether the App is installed and configures accordingly; if not installed, it prints an installation link and falls back to personal token auth.

The App also solves a branch protection constraint: since the Integrator (via the App) opens the batch PR, the human user is not the PR author and can approve their own work. The App must be added to the branch protection bypass list to merge PRs.

Future Plans

The official herd-os GitHub App is maintained by the HerdOS project. Users install it rather than creating their own. For enterprise or self-hosted deployments, organizations can create a custom App with the same permissions. The App handles GitHub API auth and bot identity only — it does not replace agent credentials, which remain separate.