@docsxai/backend

Authenticated service that persists doc packs (projects, revisions, flow-files, screenshots, annotations, style artifacts, run history). REST + per-resource endpoints; OAuth 2.1 (authorization-code + PKCE) plus a pre-issued CI bearer token; content-addressed blob storage; immutable finalized revisions.

Status: the endpoint shape is what production will be. Storage is in-memory by default and filesystem-backed with a data dir; a hosted multi-tenant deployment (real consent UI, durable token store, DB) is post-MVP and owner-gated.

Surface

• ROUTES in src/api.ts - the canonical endpoint list. /v1/workspaces/{ws}/projects/{p}/revisions/{rev}/{flows|annotations|screenshots|style|locators}, plus run history, blobs, the auth-cache relay, and the OAuth endpoints. Versioned via the Docsxai-Api-Version header.
• Linear immutable revisions - every calibrate / run / human edit creates a new revision with id / parent_revision_id / kind / author / created_at. No branches; concurrent-edit conflicts surface as failed pushes resolved via re-pull + re-edit.
• createBackendStub({ token?, store?, dataDir? }) - starts the server bound to loopback; used by the engine’s integration tests and as the local dev backend.
• docsxai-backend bin - the same server as a standalone process: docsxai-backend --port=4477 --data-dir=~/.docsxai-data.

Persistence modes

Mode	Selection	Layout
MemoryStore (default)	no dataDir, no DOCSX_DATA_DIR	per-process, resets on restart
FsStore	dataDir option, --data-dir= flag, or DOCSX_DATA_DIR	see below
custom	pass any BackendStore implementation via store	yours

FsStore layout under the data dir:

workspaces.json
projects/<projectId>/meta.json
projects/<projectId>/runs.json
projects/<projectId>/revisions/<revId>/meta.json
projects/<projectId>/revisions/<revId>/artifacts/<slot>.json
projects/<projectId>/webhook-config.json
blobs/<sha256>            ← content-addressed, shared across revisions
auth-cache/<wsId>/<role>.json
webhook-deliveries.json   ← replay guard: last 100 delivery ids

Writes are atomic (tmp file + rename); reads always go to disk, so multiple processes pointed at one data dir stay consistent. Every path join is containment-guarded against the data root - traversal-shaped ids read as not-found and traversal-shaped writes throw.

Blob protocol

Binary artifacts (screenshots) never travel as base64-in-JSON. The flow:

1. POST /v1/blobs with the raw bytes (≤ 25 MB) → { sha256, bytes }. Idempotent - the server computes the hash; re-posting the same bytes is a no-op.
2. HEAD /v1/blobs/:sha256 → 200/404, so clients skip uploads for bytes the backend already has (docsxai push HEAD-probes before every upload).
3. The screenshots artifact slot carries a manifest, not bytes: { schema: "docsxai/screenshots@2", files: { "<flow>/screenshots/<file>.png": { sha256, bytes } } }.
4. GET /v1/blobs/:sha256 returns the raw bytes; pulls fetch the manifest, then the blobs, and verify each against its hash.

Blobs are deduplicated across revisions and projects - an unchanged screenshot costs one HEAD per push.

Revision finalization

POST .../revisions/:rev/finalize marks a revision immutable (idempotent; GET reflects finalized: true). Artifact PUTs on a finalized revision are rejected with 409 { "error": "revision-finalized" }. docsxai push finalizes after uploading all artifacts, so a pushed revision is a sealed snapshot; new revisions are unaffected.

Auth

Two ways through the bearer gate (everything except /v1/health and the OAuth endpoints):

• CI token - start the server with DOCSX_TOKEN set (or the token option); callers present it as Authorization: Bearer <token>. If no token is configured the stub accepts any non-empty bearer.
• OAuth 2.1 access token - issued by the built-in authorization server, below.

Failed auth → 401 with a WWW-Authenticate: Bearer header.

OAuth 2.1 + PKCE endpoints

The backend is its own minimal authorization server:

• GET /v1/oauth/authorize?client_id=docsxai-cli&code_challenge=<S256>&code_challenge_method=S256&redirect_uri=http://127.0.0.1:<port>/callback&state=… → 302 to the redirect URI with code + state. Codes are single-use, 5-minute TTL, bound to the challenge + redirect URI. Only loopback redirect URIs are accepted; only S256.
• POST /v1/oauth/token (form-encoded) - grant_type=authorization_code (+ code, code_verifier) → { access_token, token_type: "Bearer", expires_in: 3600, refresh_token }; grant_type=refresh_token rotates (the presented refresh token is invalidated).

Consent is stub-grade by design: the authorize request is auto-approved when it carries Authorization: Bearer <DOCSX_TOKEN> or when DOCSX_OAUTH_AUTO_APPROVE=1. A real interactive consent UI is hosted-deployment scope (owner-gated). Tokens are random 32-byte values; the server stores only their sha256 hashes (with expiry + workspace scope; null scope = all workspaces, matching today’s stub semantics). Issued tokens live in process memory - they don’t survive a restart; the CI-token path does.

docsxai login --backend-url <url> --oauth <workspace> drives the full handshake from the CLI and stores the tokens at <workspace>/.auth/backend-token.json (mode 0600).

Auth-cache relay (zero-knowledge)

PUT / GET / DELETE /v1/workspaces/:ws/auth-cache/:role relays a client-side-encrypted storage-state envelope so a team can share captured target-site sessions through the backend:

{
  "schema": "docsxai/auth-cache@1",
  "alg": "aes-256-gcm",
  "iv": "…",
  "ciphertext": "…",
  "tag": "…",
  "expires_at": 1750000000000
}

The backend validates the envelope shape and stores it opaquely - it never sees the plaintext session. Encryption happens in the engine (BackendStateCache in @docsxai/engine) with a 32-byte key from DOCSX_CACHE_KEY that never leaves the client. Malformed envelopes → 400; DELETE is idempotent.

Body limits

Body	Limit	Over-limit response
JSON (all JSON endpoints)	10 MB	413 { "error": "payload_too_large" }
Raw blob (POST /v1/blobs)	25 MB	413 { "error": "payload_too_large" }

GitHub App (webhook surface)

The GitHub integration is a webhook surface on this backend - one service, no Probot, no separate worker. Signature verification is raw node:crypto HMAC; GitHub API calls are plain fetch. Install-and-go: user repos carry zero YAML - everything per-project lives in the backend’s webhook config.

GitHub push/PR ──▶ POST /v1/github/webhook          (no bearer auth; HMAC-verified)
                     │  repo → project (webhook configs)
                     │  X-Hub-Signature-256 against env[secret_env] (constant-time)
                     │  event filter · replay guard (last 100 delivery ids)
                     ▼  202 { delivery_id, project_id, dispatched: true }
               QueuedDispatcher                      (serial per project)
                     ▼
               SpawnRunner: materialize revision artifacts → temp workspace
                     ├─ spawn engine CLI (`docsxai run --workspace <dir>`)
                     ├─ append run-history row
                     ▼
               output strategy: pr-comment │ viewer-refresh │ wiki-push

Webhook config

GET/PUT /v1/workspaces/:ws/projects/:project/webhook-config (bearer-auth’d, validated):

Field	Type / values	Default	Meaning
repo	"owner/name"	- (required)	GitHub repository this project documents
events	array of push | pull_request	- (required)	deliveries outside this set are acknowledged (200) and ignored
strategy	pr-comment | viewer-refresh | wiki-push	- (required)	where run output goes
workspace_rev	"head" or a revision id	"head"	revision whose artifacts the run executes against
secret_env	env-var name	DOCSX_WEBHOOK_SECRET	which env var holds the HMAC secret (the secret is never stored)
enabled	boolean	true	disabled configs acknowledge deliveries without dispatching
plugin	"<ns>:<name>"	-	publisher plugin (required for wiki-push)
plugin_config	object	-	handed to the publisher; sources: [<dir>] names plugin paths

Webhook endpoint behavior

POST /v1/github/webhook - unauthenticated route, strictly verified:

• 401 missing/invalid X-Hub-Signature-256, or the configured secret env var is unset (fails closed).
• 404 the payload’s repository.full_name maps to no configured project.
• 400 unparsable payload, missing repository.full_name, or missing X-GitHub-Delivery.
• 200 { dispatched: false, reason } for filtered events / disabled configs; { duplicate: true } for replayed delivery ids (last 100 remembered, store-backed).
• 202 { delivery_id, project_id, dispatched: true } - job queued; execution happens after the response, serialized per project.

Output strategies

• pr-comment - posts the run summary via GitHub REST: an issue-comment on the PR (pull_request events) or a commit comment on the pushed head commit. Token: injected tokenProvider (installation-token wiring) or GITHUB_APP_TOKEN env.
• viewer-refresh - re-renders the viewer (docsxai render) from the materialized workspace and records index.html as a content-addressed blob.
• wiki-push - loads the configured publisher plugin with the engine’s plugin contract (docsxai manifest in the plugin’s package.json + register(api) module) from plugin_config.sources dirs (or path: entries in the workspace’s docsxai.config.json) and reports its PublishResult into the run-history summary.

The engine CLI is resolved like the engine resolves its viewer bin: DOCSX_ENGINE_BIN env override → the installed @docsxai/engine package’s docsxai bin → docsxai on PATH.

Owner-gated checklist (App registration)

Everything below requires owner credentials / a public URL and is deliberately not automated:

• Register the GitHub App (org settings → Developer settings → GitHub Apps); permissions: Contents read, Pull requests write, Metadata read; subscribe to push + pull_request.
• Set the App’s webhook URL to the deployed backend’s https://<host>/v1/github/webhook.
• Generate a webhook secret; export it as DOCSX_WEBHOOK_SECRET (or the name in each project’s secret_env) on the backend host.
• Wire installation tokens: exchange the App’s private key for installation tokens and inject a tokenProvider (or export a GITHUB_APP_TOKEN for single-installation setups).
• Install the App on the documented repositories, then PUT each project’s webhook config.

Environment variables

Variable	Effect
DOCSX_DATA_DIR	persist to this directory via FsStore (the --data-dir= flag / dataDir option take precedence)
DOCSX_TOKEN	the pre-issued CI bearer token; also the credential that auto-approves OAuth authorize requests
DOCSX_OAUTH_AUTO_APPROVE	1 auto-approves OAuth authorize requests without a bearer (local dev / tests)
DOCSX_CACHE_KEY	client-side only - the base64 32-byte AES-256-GCM key for the auth-cache relay (the server never reads this)
DOCSX_WEBHOOK_SECRET	default GitHub webhook HMAC secret (per-project override via the config’s secret_env)
DOCSX_ENGINE_BIN	explicit path to the engine CLI the webhook runner spawns
GITHUB_APP_TOKEN	GitHub token for the pr-comment strategy when no tokenProvider is injected
PORT	bin default port (flag --port= wins)

License

Apache-2.0.