Dev CI Runner

ci run-dev-e2e is the single manual entry for dev desired-state smoke verification. It is deliberately smaller than a DevOps control plane: the CLI starts one Git-controlled runner on D601, D601 creates a temporary CI namespace, Tekton runs the smoke check, and the result is written back as files that the CLI can inspect.

Knowledge Ownership

This document is the authoritative source for the ci run-dev-e2e architecture, manifest contract, short launcher payload, host-vs-Tekton fetch boundary, result directory and no-CD safety rules. AGENTS.md, docs/reference/ci.md, docs/reference/cli.md, docs/reference/deploy.md and docs/reference/codex-deploy.md may mention the command only as an index entry or one-line cross-reference. If a future change would require editing the same runner rule in multiple documents, update this file as the single source and replace other copies with links here.

Goal

The runner exists to prove the dev desired state without interrupting production:

Dev/prod isolation: temporary namespaces and dev manifests must not mutate unidesk, unidesk-dev, production PostgreSQL, production Deployments, production Services or main-server Compose services.
Version determinism: all runner inputs come from the pushed origin/master commit that supplied deploy.json and origin/master:deploy.json#environments.dev.
D601 execution: Git fetch, Tekton PipelineRun creation, Kubernetes polling and e2e log collection happen on D601, not on the main master.
CLI observability: the submit command returns a runId, result directory and next commands; ci logs <runId> can recover status after the local CLI exits.
CI only: the flow may create CI-owned temporary resources, but it must not deploy backend-core, frontend, Code Queue, Decision Center, k3sctl-adapter or any other direct/managed service.
Code Queue reproducibility: the runner must use the code-queue commit from environments.dev.services, build or reuse a labeled image from that Git commit on D601, import it into native k3s containerd, and validate the HTTP API inside a temporary namespace.

Design Boundary

The first-principles requirement is to validate origin/master:deploy.json#environments.dev without interrupting production. Persistent DevOps services, run brokers, webhook listeners, generic remote command protocols and full CD are secondary abstractions and are out of scope for this phase unless this document is deliberately superseded. The only automatic execution path is:

CLI -> backend-core /api/dispatch host.ssh -> D601 short launcher -> Git-controlled runner -> Tekton dev e2e PipelineRun

Host SSH is used only to start the bounded one-shot launcher and to read run files. It must not become a general deployment or repair path, and it must not fall back to arbitrary shell bodies when the runner, k3s, Tekton or egress proxy fails.

Non-Goals

Do not add a long-lived DevOps service, run broker, webhook listener or second desired-state file for this phase. Do not turn Host SSH into a general deployment system. Future full-stack dev rollout or CD can reuse the same desired-state principles, but it must be designed as a separate controlled deployment path after this smoke runner is stable.

Manifest Contract

deploy.json remains the only desired-state file. The dev environment may contain one non-service CI declaration:

{
  "schemaVersion": 2,
  "environments": {
    "dev": {
      "ci": {
        "repo": "https://github.com/pikasTech/unidesk",
        "scriptPath": "scripts/ci/dev-e2e.sh",
        "timeoutMs": 1800000
      },
      "services": [
        {
          "id": "backend-core",
          "repo": "https://github.com/pikasTech/unidesk",
          "commitId": "<pushed-commit>"
        },
        {
          "id": "frontend",
          "repo": "https://github.com/pikasTech/unidesk",
          "commitId": "<pushed-commit>"
        },
        {
          "id": "code-queue",
          "repo": "https://github.com/pikasTech/unidesk",
          "commitId": "<pushed-commit>"
        }
      ]
    }
  }
}

scriptPath must be a repo-relative scripts/ci/*.sh path. Inline shell bodies, arbitrary script paths, local dirty scripts and separate develop.json or CI manifest files are forbidden. The script is fetched from the same full 40-character manifest commit that supplied deploy.json, so the runner logic is auditable and rollbackable with the desired state. Persistent dev rollout service scope is owned by docs/reference/dev-environment.md; this runner only consumes the dev service list for smoke verification and must not deploy it. code-queue is required in the dev service list for this smoke runner, but that does not enable deploy apply --env dev --service code-queue.

Execution Path

The automatic path is intentionally single and narrow:

CLI fetches origin/master and reads origin/master:deploy.json#environments.dev.
CLI records the full manifest commit and generates a DNS-safe runId.
CLI sends a short launcher through backend-core /api/dispatch using the existing host.ssh provider capability for D601.
D601 creates /tmp/unidesk-ci/<runId> and /home/ubuntu/.unidesk/runs/<runId>.
D601 fetches the manifest commit from GitHub through the node-local provider-gateway WS egress proxy at http://127.0.0.1:18789.
D601 extracts the runner with git show <commit>:<scriptPath> > /tmp/unidesk-ci/<runId>/runner.sh and the desired-state blob with git show <commit>:deploy.json > /tmp/unidesk-ci/<runId>/deploy.json.
The runner parses the host-fetched deploy.json, requires a full-SHA code-queue service commit, builds or reuses a D601 Docker image for that commit, imports the image and postgres:16-alpine into native k3s containerd, creates the Tekton PipelineRun in unidesk-ci, passes the required dev service commits and Code Queue image tag as PipelineRun params, waits for completion when requested, and writes result.json, launcher.log, runner.log, PipelineRun JSON and pod logs under /home/ubuntu/.unidesk/runs/<runId>/.

The CLI must not upload the runner script body. Tekton dev e2e must not clone the private UniDesk repo itself; repo access and desired-state extraction happen once in the D601 host launcher under the manifest commit. The submitted launcher may contain only repo, full commit, script path, run id, environment, timeout, keep-namespace and fixed workspace path settings plus the fixed fetch/execute wrapper. If k3s, Tekton or the provider egress proxy is unavailable, the run fails with visible logs; it must not fall back to an alternate deployment path.

Runner Contract

The Git-controlled script must accept:

scripts/ci/dev-e2e.sh \
  --run-id <runId> \
  --repo-url <repo> \
  --desired-ref master \
  --manifest-commit <full-sha> \
  --manifest-file /tmp/unidesk-ci/<runId>/deploy.json \
  --environment dev \
  --result-dir /home/ubuntu/.unidesk/runs/<runId> \
  --timeout-ms <ms> \
  [--keep-namespace]

The current script creates a Tekton PipelineRun for pipeline/unidesk-dev-namespace-e2e, stores the generated PipelineRun name in pipelinerun.txt, and writes a final result.json with ok, status, runId, manifestCommit, pipelineRun, temporaryNamespace and finishedAt.

The Tekton task creates a temporary namespace unidesk-ci-e2e-<runId> and may create only CI-owned smoke resources there: postgres-dev, code-queue-scheduler-dev, code-queue-read-dev, code-queue-write-dev, their ClusterIP Services and a per-run Secret/ConfigMap. It must not mutate unidesk or persistent unidesk-dev. Code Queue API validation must use ClusterIP Services and the Kubernetes API services/.../proxy subresource; NodePort, D601 host ports and direct public service exposure are forbidden. The smoke currently proves /health, /live and /api/workdirs GET/POST/DELETE on read/write/scheduler roles, giving follow-up Code Queue API fixes a reproducible CI target before production rollout.

Commands

Start a run and return after dispatch:

bun scripts/cli.ts ci run-dev-e2e

Start a run and wait up to ten minutes for completion:

bun scripts/cli.ts ci run-dev-e2e --wait-ms 600000

Inspect run files on D601:

bun scripts/cli.ts ci logs <runId>

Regular Tekton CI remains documented in docs/reference/ci.md; deployment desired-state and target-side build rules remain documented in docs/reference/deploy.md.

8.3 KiB Raw Blame History