Merge pull request #160 from pikasTech/fix/hwlab-cd-wrapper-340
Harden HWLAB CD wrapper
This commit is contained in:
@@ -65,7 +65,7 @@ UniDesk 是一个以主 server 为统一入口的分布式工作平台;本文
|
||||
- `bun scripts/cli.ts auth-broker contract|health --dry-run|credential-request --dry-run|pr-preflight --dry-run`:查看 Auth Broker P0 Rust skeleton 与 CLI adapter contract,runner 无 `GH_TOKEN`/`GITHUB_TOKEN` 时返回结构化 `auth-missing`/`broker-needed`,不读取或打印 token 值,规则见 `docs/reference/auth-broker.md`。
|
||||
- `bun scripts/cli.ts gh preflight|auth status|issue ...|pr list|files|diff --stat|read|view|preflight|closeout|create|edit|update|comment` / `bun scripts/code-queue-pr-preflight-example.ts`:通过 REST 执行安全 GitHub issue 读写、脱敏 auth/status 诊断、body-file Markdown 写入、当日滚动简报时间线 ClaudeQQ 通知、escape 扫描、只读 cleanup-plan 和 #20 board-audit、PR changed-file/stat summary、PR 创建/评论 dry-run、REST-only 低噪声 PR title/body 编辑、PR 收口元数据观察(含 merged/closed 区分与 merge commit)、低噪声 PR 收口 preflight 与 runner PR preflight;`gh issue/pr read|view` 支持 `owner/repo#number` shorthand,`--raw|--full` 是显式完整披露别名,`gh pr diff` 仅支持 `--stat` 紧凑 JSON,`gh pr merge` 当前仍结构化拒绝但普通 PR 可按任务边界用 repo-owned GitHub 路径收口,规则见 `docs/reference/cli.md` 和 `docs/reference/code-queue-supervision.md`。
|
||||
- `bun scripts/cli.ts commander contract|plan --dry-run|smoke --dry-run|approval request --dry-run|prompt-lint --kind gpt55-pr`:查看 host Codex 指挥官直管微服务 skeleton 的 source/contract、无 daemon smoke 验证计划、.state/commander/ 状态模型、trace summary 聚合、ClaudeQQ 高风险请示草案和 GPT-5.5 PR prompt 边界辅助 lint;当前只返回 dry-run 计划和 backend-core `microservice proxy claudeqq` 授权后候选命令,不接 live bridge、不接管人工指挥官,不发送消息,`prompt-lint` 不作为业务 PR 门禁也不改变 `codex submit` 默认行为,规则见 `docs/reference/host-codex-commander.md`。
|
||||
- `bun scripts/cli.ts hwlab cd status --env dev` / `bun scripts/cli.ts hwlab cd apply --env dev --dry-run`:HWLAB DEV CD 指挥侧 wrapper,调用 HWLAB repo-owned 发布脚本并强制 D601 原生 k3s kubeconfig;真实 apply 只暴露 host-commander-only 命令形状,规则见 `docs/reference/hwlab.md`。
|
||||
- `bun scripts/cli.ts hwlab cd status --env dev` / `bun scripts/cli.ts hwlab cd apply --env dev --dry-run`:HWLAB DEV CD 指挥侧 wrapper,通过 D601 provider SSH 调用 HWLAB repo-owned `scripts/dev-cd-apply.mjs` 并强制原生 k3s kubeconfig;真实 apply 只暴露 host-commander-only 命令形状,规则见 `docs/reference/hwlab.md`。
|
||||
- `bun scripts/cli.ts ci install/status/run/publish-backend-core/publish-user-service/run-dev-e2e/logs`:在 D601 原生 k3s 上安装和运行 Tekton CI,支持每 commit 检查、Code Queue 只读性能门禁、`CI.json` catalog 驱动的 backend-core 与 user-service commit-pinned 镜像发布和手动触发的 `origin/master:deploy.json#environments.dev` 临时 namespace e2e;catalog/producer/consumer 分工见 `docs/reference/cicd-standardization.md`,`run-dev-e2e` 的 Git 控制 runner、短 launcher 和 no-CD 边界见 `docs/reference/dev-ci-runner.md`,Tekton 规则见 `docs/reference/ci.md`。
|
||||
- `bun scripts/cli.ts codex deploy <commitId>`:旧 Code Queue 兼容部署入口已禁用,原因是它会绕过受控部署边界直连 D601 部署 Code Queue;规则见 `docs/reference/codex-deploy.md`。
|
||||
- `bun scripts/cli.ts codex prompt-lint [prompt|--prompt-file path|--prompt-stdin]` / `codex submit [prompt] [--prompt-file path|--prompt-stdin] [--queue <id>]` / `codex pr-preflight [--remote]`:`prompt-lint` 在派发/steer 前 dry-run 检查 runner prompt 的 DEV 测试授权分级(`read-only`/`live-read`/`live-mutating`)且不回显 prompt;`submit --dry-run` 同时给出 MiniMax/GPT/人工路由建议、该 lint 结果和 requested/effective execution mode;真实提交成功只返回写入确认、task id、服务级 runnerPermissions 和后续查看命令,不回显 prompt;`pr-preflight` 只读检查 D601 scheduler/runner 的 GitHub token、egress 和 PR 能力,PR 型派单前必须使用,规则见 `docs/reference/cli.md` 和 `docs/reference/code-queue-supervision.md`。
|
||||
|
||||
@@ -31,7 +31,7 @@ CLI 可以从 `master` 快速演进,但必须兼容 `deploy.json` 固定的 CI
|
||||
- `dev-env prewarm-images [--image image] [--provider-id D601] [--no-pull] [--proxy-url URL] [--pull-timeout-ms N] [--dry-run]` 创建异步 job,通过 UniDesk SSH 维护桥在 D601 上把开发底座依赖镜像从 Docker 缓存导入原生 k3s containerd。默认镜像是 `postgres:16-alpine` 和 `rancher/mirrored-library-busybox:1.36.1`,用于避免 `postgres-dev` 与 local-path helper pod 卡在外部 registry 拉取。该命令固定验证 `/etc/rancher/k3s/k3s.yaml` 指向的 native k3s 上下文,并输出 `dev_env_containerd_image_ready=...` 作为成功判据;它不 apply manifest、不修改生产 `unidesk` namespace。
|
||||
- `artifact-registry plan|render|status|health|install|deploy-backend-core|deploy-service` 管理 D601 host-managed CNCF Distribution registry 的声明、安装、只读检查和 pull-only artifact CD。该 registry 固定为 D601 loopback `127.0.0.1:5000`,由 systemd + Docker Compose 管理,位于 native k3s 故障域外;`deploy-service` 只拉取 CI 已发布的 commit-pinned 镜像、retag/recreate 或导入 native k3s,并做 live commit 验证,不构建 runtime source。`deploy-backend-core` 是 deprecated 兼容名,标准 backend-core prod CD 入口是 `deploy apply --env prod --service backend-core`。长期规则见 `docs/reference/artifact-registry.md`。
|
||||
- `commander contract|plan --dry-run|smoke --dry-run|approval request --dry-run|prompt-lint --kind gpt55-pr` 是 host Codex 指挥官直管微服务 skeleton 入口。当前命令返回 `phase=source-contract`、service/API/state/bridge/prompt/trace/#20/#46/ClaudeQQ 审批边界、.state/commander/ 状态模型、dev 无 daemon smoke contract、dry-run 计划和 GPT-5.5 PR prompt 边界辅助 lint,不接 live bridge、不注入 prompt、不发送 ClaudeQQ。`approval request --dry-run` 会生成 200 字以内中文纯文本 ClaudeQQ 审批草案、`notification-path-unavailable` blocker 和授权后唯一可用的 `bun scripts/cli.ts microservice proxy claudeqq /api/push/text --method POST --body-json '<payload>' --raw` 命令;不得提示使用本机 ClaudeQQ skill、powershell 或本地 server。`prompt-lint` 支持 `--prompt-file` 与 `--stdin`,输出 `ok`、`missingClauses`、`riskLevel`、`suggestedPatchSnippet` 且不回显完整 prompt;它是 commander 辅助检查,不是业务 PR 门禁,也不改变 `codex submit` 默认行为。`plan`、`smoke` 与 `approval request` 必须带 `--dry-run`;缺少时返回 `error=dry-run-required`。长期规则见 `docs/reference/host-codex-commander.md`。
|
||||
- `hwlab cd status|preflight|apply --env dev [--dry-run]` 是 HWLAB DEV CD 指挥侧 wrapper。它只调用 HWLAB repo-owned 受控入口,不内嵌发布 kubectl 逻辑:`status` 汇总固定 CD mirror、Git clean/main/origin-main、`deploy/deploy.json`/artifact catalog/workloads 同源收敛、D601 native k3s guard、CD Lease lock、16666/16667 live revision 和当前 live workload image;`preflight` 进一步检查必需 SecretRef 对象/键存在性并运行 HWLAB `scripts/dev-cd-apply.mjs --dry-run` 受控事务摘要;完整 stdout/stderr 写入 `.state/hwlab-cd/<run-id>/`,stdout 只返回有界摘要。默认 HWLAB CD repo 是 `/home/ubuntu/hwlab_cd`,`/home/ubuntu/hwlab` runner 历史目录不得作为发布真相。wrapper 强制 `KUBECONFIG=/etc/rancher/k3s/k3s.yaml` 并只以这个显式目标作为 gate;显式目标出现 `docker-desktop`、`desktop-control-plane` 或 `127.0.0.1:11700` 信号会结构化拒绝,写操作计划还必须观察到 node `d601`。`apply --dry-run` 只调用 HWLAB `scripts/dev-cd-apply.mjs --dry-run --kubeconfig /etc/rancher/k3s/k3s.yaml`;真实 apply 只暴露 `scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report` 命令形状并标注 host-commander-only,本 runner 不执行 live apply。长期规则见 `docs/reference/hwlab.md`。
|
||||
- `hwlab cd status|preflight|apply --env dev [--dry-run]` 是 HWLAB DEV CD 指挥侧 wrapper。默认通过 UniDesk provider `host.ssh` 进入 D601,再调用 HWLAB repo-owned `scripts/dev-cd-apply.mjs`,不内嵌发布 kubectl 逻辑:`status` 汇总固定 CD mirror、Git clean/main/origin-main、`deploy/deploy.json`/artifact catalog/report、D601 native k3s guard 和 CD Lease lock,并用 `scripts/dev-cd-apply.mjs --status --skip-live-verify` 取得 target/promotion 摘要;`preflight` 进一步检查必需 SecretRef 对象/键存在性并运行 HWLAB `scripts/dev-cd-apply.mjs --dry-run --skip-live-verify` 受控事务摘要。完整远端 stdout/stderr 写入 D601 `~/.state/unidesk-hwlab-cd/<run-id>/` 和本地 `.state/hwlab-cd/<run-id>/` task dump,stdout 只返回有界摘要。默认 HWLAB CD repo 是 `/home/ubuntu/hwlab_cd`,`/home/ubuntu/hwlab` runner 历史目录不得作为发布真相。wrapper 强制 `KUBECONFIG=/etc/rancher/k3s/k3s.yaml` 并只以这个显式目标作为 gate;显式目标出现 `docker-desktop`、`desktop-control-plane` 或 `127.0.0.1:11700` 信号会结构化拒绝,写操作计划还必须观察到 node `d601`。真实 apply 只暴露 `scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report` 命令形状并标注 host-commander-only,本 runner 不执行 live apply、rollout、Lease mutation 或 16666/16667 live verification。长期规则见 `docs/reference/hwlab.md`。
|
||||
- `gh auth status [--repo owner/name]` 探测 GitHub 操作前置条件并输出脱敏 JSON:是否存在 `gh` binary、是否存在 `GH_TOKEN`/`GITHUB_TOKEN` 或可用 `gh auth token` fallback、REST API 是否可达、目标 repo 是否可见、issue 是否可读。degraded reason 必须归类为 `missing-binary`、`missing-token`、`auth-failed`、`github-transient`、`network-proxy-failed`、`permission-denied`、`repo-not-found`、`repo-forbidden`、`issue-not-found`、`pr-not-found`、`scope-insufficient`、`validation-failed`、`invalid-response` 或 `unsupported-command`,不得打印 token;失败对象必须包含 `runnerDisposition=infra-blocked|business-failed`,runner 应优先用该字段分流。`github-transient` 表示 GitHub DNS/API 连接在收到 HTTP 状态前失败,输出应带 `retryable=true` 或等价 commander action;这不是缺 token、认证失败、权限不足或 PR 语义失败。
|
||||
- `codex prompt-lint [prompt|--prompt-file path|--prompt-stdin]` 是派发/steer 前的本地 dry-run prompt lint。它只读取 prompt 文本,返回 `dryRun=true`、`mutation=false`、`declaredClass`、`effectiveClass`、`requiredClass`、`dispatchDisposition`、缺失或矛盾项和有界 evidence,不访问 live service、不提交任务、不打印完整 prompt。分级固定为 `read-only`、`live-read`、`live-mutating`;未声明时按 `read-only` 处理。`codex submit --dry-run` 与 `codex steer --dry-run` 会嵌入同一 `promptLint` 结果,帮助指挥官在 dispatch/steer 前发现缺失或矛盾的 live mutation 授权。长期规则见 `docs/reference/code-queue-supervision.md` 的 DEV 测试授权分级。
|
||||
- `gh issue list [--state open|closed|all] [--limit N] [--repo owner/name] [--json number,title,state,url,updatedAt,createdAt,author,labels]` 通过 GitHub REST 列出 issue,默认 `state=open`、`limit=30`,输出稳定 JSON 且不依赖系统 `gh` binary。`--limit` 会映射到 GitHub `per_page` 并限制返回数量,避免一次拉爆上下文;未知 state 或未知 `--json` 字段必须结构化失败并带 `runnerDisposition=business-failed`。GitHub issues API 可能混入 PR,CLI 会从 `.data.issues` 中过滤 pull request。
|
||||
|
||||
@@ -50,14 +50,14 @@ bun scripts/cli.ts hwlab cd preflight --env dev
|
||||
bun scripts/cli.ts hwlab cd apply --env dev --dry-run
|
||||
```
|
||||
|
||||
wrapper 的职责是把 host commander 常用的 HWLAB DEV rollout 查看/准备动作收敛到单一入口。它只调用 HWLAB repo-owned 受控脚本,不在 UniDesk 内重写发布流程或拼接 ad hoc `kubectl apply`:
|
||||
wrapper 的职责是把 host commander 常用的 HWLAB DEV rollout 查看/准备动作收敛到单一入口。默认路径通过 UniDesk main-server frontend/backend-core 的 provider `host.ssh` 能力进入 D601,再在 D601 上执行有界检查脚本;它只调用 HWLAB repo-owned 受控脚本,不在 UniDesk 内重写发布流程或拼接 ad hoc `kubectl apply`:
|
||||
|
||||
- 默认 HWLAB CD repo 是 D601 固定干净 mirror `/home/ubuntu/hwlab_cd`,也可用 `--hwlab-repo` 显式指定同等干净 clone。wrapper 必须检查 `git status --short --branch`、origin remote、当前 branch `main`、本地 `origin/main`、`FETCH_HEAD` 和 worktree 权限;任何 dirty worktree、错误 remote、非 main、HEAD 未跟上本地 `origin/main` 或权限异常都返回结构化 blocker。`/home/ubuntu/hwlab` 是 runner 历史目录,不得作为发布真相。
|
||||
- `deploy/deploy.json` 是唯一 desired-state。wrapper 只把 `deploy/artifact-catalog.dev.json`、`deploy/k8s/base/workloads.yaml` 和 `reports/dev-gate/dev-artifacts.json` 当作派生/证据读数;`status`/`preflight` 必须显示 target commit/ref、deploy.json、artifact catalog、workloads 和 live workload image 是否同源/收敛,不引入第二套 desired state。
|
||||
- `status` 只读汇总 HWLAB repo path、Git clean/main/origin-main、desired-state 收敛、D601 native k3s guard、`Lease/hwlab-dev/hwlab-dev-cd-lock`、公网 `16666/16667` live revision 和当前 live workload image。
|
||||
- `status` 只读汇总 HWLAB repo path、Git clean/main/origin-main、desired-state 收敛、D601 native k3s guard 和 `Lease/hwlab-dev/hwlab-dev-cd-lock`;同时调用 HWLAB `scripts/dev-cd-apply.mjs --status --skip-live-verify` 取得 repo-owned target/promotion/deploy.json/artifact 摘要。16666/16667 live verification 不由本 runner 执行。
|
||||
- `preflight` 在 `status` 的基础上检查 apply 前 SecretRef:`hwlab-cloud-api-dev-db/database-url`、`hwlab-cloud-api-dev-db-admin/admin-url`、`hwlab-code-agent-provider/openai-api-key`。只验证 Secret 对象和 key 元数据存在性,缺失时返回 blocker、影响范围和修复 runbook;禁止读取或打印 Secret value。
|
||||
- `apply --dry-run` 调用 HWLAB `scripts/dev-cd-apply.mjs --dry-run --kubeconfig /etc/rancher/k3s/k3s.yaml`,只生成受控事务准备/阻塞摘要,不做真实 apply、rollout 或 live verification。历史 `scripts/dev-deploy-apply.mjs` 可作为 HWLAB 内部支持脚本出现,但 UniDesk wrapper 不能把它当成平行 CD 入口。
|
||||
- 完整下游 stdout/stderr、HTTP body 和 kubectl 读命令输出写入 UniDesk `.state/hwlab-cd/<run-id>/` dump 目录;CLI stdout 只显示有界摘要和 dump path。
|
||||
- `apply --dry-run` 调用 HWLAB `scripts/dev-cd-apply.mjs --dry-run --kubeconfig /etc/rancher/k3s/k3s.yaml --skip-live-verify`,只生成受控事务准备/阻塞摘要,不做真实 apply、rollout、Lease mutation 或 live verification。历史 `scripts/dev-deploy-apply.mjs` 可作为 HWLAB 内部支持脚本出现,但 UniDesk wrapper 不能把它当成平行 CD 入口。
|
||||
- 完整下游 stdout/stderr 和 kubectl 读命令输出写入 D601 `~/.state/unidesk-hwlab-cd/<run-id>/`,UniDesk 本地仅保存 provider task stdout/stderr dump;CLI stdout 只显示有界摘要和 dump path。
|
||||
- wrapper 显式注入 `KUBECONFIG=/etc/rancher/k3s/k3s.yaml` 并以这个显式目标作为唯一 gate:目标 context/server/nodes 若出现 `docker-desktop`、`desktop-control-plane` 或 `127.0.0.1:11700` 必须拒绝继续,写操作计划或 `apply --dry-run` 前目标 nodes 必须包含 `d601`。裸 `kubectl` 默认 context 只作为诊断输出;即使默认 kubeconfig 仍残留 Docker Desktop,只要显式 D601 kubeconfig 通过,也不能把默认 context 当成 CD blocker。
|
||||
|
||||
真实 DEV apply 只允许 host commander 在明确授权后执行。UniDesk wrapper 可以展示受控命令形状:
|
||||
|
||||
@@ -1,30 +1,43 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdirSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import assert from "node:assert/strict";
|
||||
import { buildHwlabCdRemoteCommandForTest } from "./src/hwlab-cd";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function dataOf(payload: JsonRecord): JsonRecord {
|
||||
return payload.data as JsonRecord;
|
||||
}
|
||||
|
||||
function runCli(args: string[], env: NodeJS.ProcessEnv = {}): JsonRecord {
|
||||
const result = spawnSync("bun", ["scripts/cli.ts", ...args], {
|
||||
cwd: process.cwd(),
|
||||
env: { ...process.env, ...env },
|
||||
env: {
|
||||
...process.env,
|
||||
UNIDESK_HWLAB_CD_TRANSPORT: "local",
|
||||
...env,
|
||||
},
|
||||
encoding: "utf8",
|
||||
timeout: 20_000,
|
||||
timeout: 25_000,
|
||||
});
|
||||
assert.equal(result.stderr, "", `stderr should be empty: ${result.stderr}`);
|
||||
assert.notEqual(result.stdout.trim(), "", "CLI must not produce empty output");
|
||||
const parsed = JSON.parse(result.stdout) as JsonRecord;
|
||||
if (result.status !== 0) {
|
||||
assert.equal(parsed.ok, false, `nonzero CLI should return ok=false: ${result.stdout}`);
|
||||
}
|
||||
if (result.status !== 0) assert.equal(parsed.ok, false, `nonzero CLI should return ok=false: ${result.stdout}`);
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function writeJson(path: string, value: unknown): void {
|
||||
writeFileSync(path, `${JSON.stringify(value, null, 2)}\n`);
|
||||
}
|
||||
|
||||
function makeFakeHwlabRepo(): string {
|
||||
const root = join(tmpdir(), `unidesk-hwlab-cd-wrapper-${process.pid}-${Date.now()}`);
|
||||
const root = join(tmpdir(), `unidesk-hwlab-cd-wrapper-${process.pid}-${Date.now()}-${Math.random().toString(16).slice(2)}`);
|
||||
mkdirSync(join(root, "scripts"), { recursive: true });
|
||||
mkdirSync(join(root, "deploy/k8s/base"), { recursive: true });
|
||||
mkdirSync(join(root, "reports/dev-gate"), { recursive: true });
|
||||
writeFileSync(join(root, "scripts/dev-cd-apply.mjs"), [
|
||||
"const kubeconfigIndex = process.argv.indexOf('--kubeconfig');",
|
||||
"process.stdout.write(JSON.stringify({",
|
||||
@@ -38,7 +51,7 @@ function makeFakeHwlabRepo(): string {
|
||||
" ref: 'origin/main',",
|
||||
" promotionCommit: 'abc1234567890abcdef',",
|
||||
" shortCommitId: 'abc1234',",
|
||||
" promotionSource: 'deploy-json',",
|
||||
" promotionSource: 'deploy/deploy.json',",
|
||||
" publishRequired: false,",
|
||||
" headCommitId: 'abc1234567890abcdef',",
|
||||
" headMatchesTarget: true,",
|
||||
@@ -50,40 +63,14 @@ function makeFakeHwlabRepo(): string {
|
||||
" artifactCatalog: { path: 'deploy/artifact-catalog.dev.json', commitId: 'abc1234', artifactState: 'published', ciPublished: true, registryVerified: true },",
|
||||
" artifactReport: { path: 'reports/dev-gate/dev-artifacts.json', commitId: 'abc1234' },",
|
||||
" lock: { status: 'absent' },",
|
||||
" liveDelta: { status: 'unknown' },",
|
||||
" liveDelta: { status: 'not_run', reason: '--skip-live-verify' },",
|
||||
" kubeconfig: kubeconfigIndex >= 0 ? process.argv[kubeconfigIndex + 1] : null",
|
||||
"}, null, 2));",
|
||||
].join("\n"));
|
||||
writeFileSync(join(root, "scripts/dev-deploy-apply.mjs"), [
|
||||
"const dryRun = process.argv.includes('--dry-run');",
|
||||
"const kubeconfigIndex = process.argv.indexOf('--kubeconfig');",
|
||||
"process.stdout.write(JSON.stringify({",
|
||||
" reportVersion: 'v1',",
|
||||
" status: dryRun ? 'pass' : 'blocked',",
|
||||
" commitId: 'abc1234',",
|
||||
" namespace: 'hwlab-dev',",
|
||||
" endpoint: 'http://74.48.78.17:16667',",
|
||||
" blockers: [],",
|
||||
" devDeployApply: {",
|
||||
" conclusion: { status: 'ready', blockerCount: 0 },",
|
||||
" artifactPlan: { expectedArtifactCommit: 'abc1234', deployCommitId: 'abc1234', catalogCommitId: 'abc1234', published: true, registryVerified: true, imageCount: 13, requiredServiceCount: 13, unpublishedServices: [] },",
|
||||
" applyBoundary: { currentMode: 'dry-run', defaultNoWrite: true, mutationAttempted: false, mutationAllowed: false, kubeconfigSource: 'flag:--kubeconfig', writeScope: 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply -k deploy/k8s/dev', noWriteScope: 'server-side dry-run only', forbiddenActions: ['prod-deploy'] },",
|
||||
" applyStep: { status: 'pass', command: 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply --dry-run=server -k deploy/k8s/dev', mutationAttempted: false },",
|
||||
" manualCommands: { status: 'ready' }",
|
||||
" },",
|
||||
" kubeconfig: kubeconfigIndex >= 0 ? process.argv[kubeconfigIndex + 1] : null",
|
||||
"}, null, 2));",
|
||||
].join("\n"));
|
||||
writeFileSync(join(root, "scripts/deploy-desired-state-plan.mjs"), [
|
||||
"process.stdout.write(JSON.stringify({",
|
||||
" kind: 'hwlab-deploy-desired-state-plan',",
|
||||
" mode: 'read-only-plan',",
|
||||
" status: 'pass',",
|
||||
" source: { deploy: 'deploy/deploy.json', artifactCatalog: 'deploy/artifact-catalog.dev.json', workloads: 'deploy/k8s/base/workloads.yaml', optionalReport: 'reports/dev-gate/dev-artifacts.json' },",
|
||||
" promotionBoundary: { authoritativeDesiredState: ['deploy/deploy.json', 'deploy/artifact-catalog.dev.json', 'deploy/k8s/base/workloads.yaml'], nonAuthoritativeEvidence: ['reports/dev-gate/dev-artifacts.json'] },",
|
||||
" summary: { desiredCommitId: 'abc1234', desiredImageTag: 'abc1234', artifactState: 'published', ciPublished: true, registryVerified: true, services: 13, workloadContainers: 13, diagnostics: 0, blockers: 0, targetConvergence: 'not_requested' }",
|
||||
"}, null, 2));",
|
||||
].join("\n"));
|
||||
writeJson(join(root, "deploy/deploy.json"), { commitId: "abc1234", environment: "dev", namespace: "hwlab-dev", endpoint: "http://74.48.78.17:16667" });
|
||||
writeJson(join(root, "deploy/artifact-catalog.dev.json"), { commitId: "abc1234", artifactState: "published", publish: { ciPublished: true, registryVerified: true }, services: [{ id: "hwlab-cloud-api", digest: "sha256:" + "a".repeat(64) }] });
|
||||
writeJson(join(root, "reports/dev-gate/dev-artifacts.json"), { commitId: "abc1234", artifactPublish: { sourceCommitId: "abc1234", serviceCount: 1, ciPublished: true, registryVerified: true } });
|
||||
writeFileSync(join(root, "deploy/k8s/base/workloads.yaml"), "apiVersion: v1\nkind: List\nitems: []\n");
|
||||
spawnSync("git", ["init", "-b", "main"], { cwd: root, encoding: "utf8" });
|
||||
spawnSync("git", ["config", "user.email", "test@example.invalid"], { cwd: root, encoding: "utf8" });
|
||||
spawnSync("git", ["config", "user.name", "HWLAB CD Test"], { cwd: root, encoding: "utf8" });
|
||||
@@ -95,19 +82,19 @@ function makeFakeHwlabRepo(): string {
|
||||
return root;
|
||||
}
|
||||
|
||||
function makeFakeBin(mode: "native" | "desktop" | "stale-default" | "wrong-node" | "missing-secret"): string {
|
||||
const bin = join(tmpdir(), `unidesk-hwlab-cd-bin-${process.pid}-${Date.now()}-${mode}`);
|
||||
function makeFakeBin(mode: "native" | "desktop" | "stale-default" | "wrong-node" | "missing-secret" | "second-plane"): string {
|
||||
const bin = join(tmpdir(), `unidesk-hwlab-cd-bin-${process.pid}-${Date.now()}-${mode}-${Math.random().toString(16).slice(2)}`);
|
||||
mkdirSync(bin, { recursive: true });
|
||||
const explicitContext = mode === "desktop" ? "docker-desktop" : "default";
|
||||
const explicitServer = mode === "desktop" ? "https://127.0.0.1:11700" : "https://127.0.0.1:6443";
|
||||
const explicitNodes = mode === "desktop" ? "desktop-control-plane" : mode === "wrong-node" ? "d602" : "d601";
|
||||
const defaultContext = mode === "stale-default" ? "docker-desktop" : explicitContext;
|
||||
const defaultServer = mode === "stale-default" ? "https://127.0.0.1:11700" : explicitServer;
|
||||
const defaultContext = mode === "stale-default" ? "docker-desktop" : mode === "second-plane" ? "other-k3s" : explicitContext;
|
||||
const defaultServer = mode === "stale-default" ? "https://127.0.0.1:11700" : mode === "second-plane" ? "https://10.0.0.2:6443" : explicitServer;
|
||||
const defaultNodes = mode === "stale-default" ? "desktop-control-plane" : explicitNodes;
|
||||
const defaultNamespaceOk = mode === "second-plane";
|
||||
writeFileSync(join(bin, "kubectl"), [
|
||||
"#!/usr/bin/env bash",
|
||||
"set -euo pipefail",
|
||||
"printf 'KUBECONFIG=%s\\n' \"${KUBECONFIG:-}\" >&2",
|
||||
"context=" + JSON.stringify(explicitContext),
|
||||
"server=" + JSON.stringify(explicitServer),
|
||||
"nodes=" + JSON.stringify(explicitNodes),
|
||||
@@ -119,33 +106,56 @@ function makeFakeBin(mode: "native" | "desktop" | "stale-default" | "wrong-node"
|
||||
"if [[ \"$*\" == 'config current-context' ]]; then printf '%s\\n' \"$context\"; exit 0; fi",
|
||||
"if [[ \"$*\" == 'config view --minify -o jsonpath={.clusters[0].cluster.server}' ]]; then printf '%s' \"$server\"; exit 0; fi",
|
||||
"if [[ \"$*\" == 'get nodes -o jsonpath={range .items[*]}{.metadata.name}{\"\\n\"}{end}' ]]; then printf '%s\\n' \"$nodes\"; exit 0; fi",
|
||||
"if [[ \"$*\" == '-n hwlab-dev get lease hwlab-dev-cd-lock -o json' ]]; then printf 'Error from server (NotFound): leases.coordination.k8s.io \"hwlab-dev-cd-lock\" not found\\n' >&2; exit 1; fi",
|
||||
"if [[ \"$*\" == '-n hwlab-dev get deploy -o jsonpath={range .items[*]}{.metadata.name}{\"\\t\"}{range .spec.template.spec.containers[*]}{.name}{\"=\"}{.image}{\",\"}{end}{\"\\n\"}{end}' ]]; then printf 'hwlab-cloud-api\\thwlab-cloud-api=127.0.0.1:5000/hwlab/hwlab-cloud-api:abc1234,\\n'; exit 0; fi",
|
||||
"if [[ \"$*\" == 'get namespace hwlab-dev -o name' ]]; then",
|
||||
" if [[ \"${KUBECONFIG:-}\" == '' && " + JSON.stringify(defaultNamespaceOk ? "yes" : "no") + " == 'yes' ]]; then printf 'namespace/hwlab-dev\\n'; exit 0; fi",
|
||||
" if [[ \"${KUBECONFIG:-}\" != '' ]]; then printf 'namespace/hwlab-dev\\n'; exit 0; fi",
|
||||
" printf 'Error from server (NotFound): namespaces \"hwlab-dev\" not found\\n' >&2; exit 1",
|
||||
"fi",
|
||||
"if [[ \"$*\" =~ ^-n[[:space:]]+hwlab-dev[[:space:]]+get[[:space:]]+lease[[:space:]]+hwlab-dev-cd-lock[[:space:]]+-o[[:space:]]+json$ ]]; then printf 'Error from server (NotFound): leases.coordination.k8s.io \"hwlab-dev-cd-lock\" not found\\n' >&2; exit 1; fi",
|
||||
"if [[ \"$*\" == '-n hwlab-dev get secret hwlab-code-agent-provider -o name' && " + JSON.stringify(mode) + " == 'missing-secret' ]]; then printf 'Error from server (NotFound): secrets \"hwlab-code-agent-provider\" not found\\n' >&2; exit 1; fi",
|
||||
"if [[ \"$*\" =~ ^-n\\ hwlab-dev\\ get\\ secret\\ ([^[:space:]]+)\\ -o\\ name$ ]]; then printf 'secret/%s\\n' \"${BASH_REMATCH[1]}\"; exit 0; fi",
|
||||
"if [[ \"$*\" == '-n hwlab-dev describe secret hwlab-cloud-api-dev-db' ]]; then printf 'Name: hwlab-cloud-api-dev-db\\nData\\n====\\ndatabase-url: 48 bytes\\n'; exit 0; fi",
|
||||
"if [[ \"$*\" == '-n hwlab-dev describe secret hwlab-cloud-api-dev-db-admin' ]]; then printf 'Name: hwlab-cloud-api-dev-db-admin\\nData\\n====\\nadmin-url: 48 bytes\\n'; exit 0; fi",
|
||||
"if [[ \"$*\" == '-n hwlab-dev describe secret hwlab-code-agent-provider' ]]; then printf 'Name: hwlab-code-agent-provider\\nData\\n====\\nopenai-api-key: 48 bytes\\n'; exit 0; fi",
|
||||
"printf '{}\\n'",
|
||||
"printf 'unexpected kubectl args: %s\\n' \"$*\" >&2; exit 99",
|
||||
].join("\n"));
|
||||
spawnSync("chmod", ["+x", join(bin, "kubectl")]);
|
||||
return bin;
|
||||
}
|
||||
|
||||
function scopes(data: JsonRecord): string[] {
|
||||
return ((data.blockers ?? []) as JsonRecord[]).map((blocker) => String(blocker.scope ?? ""));
|
||||
}
|
||||
|
||||
function withLocalTransport(args: string[]): string[] {
|
||||
return [...args, "--transport", "local"];
|
||||
}
|
||||
|
||||
const fakeRepo = makeFakeHwlabRepo();
|
||||
const nativeBin = makeFakeBin("native");
|
||||
const desktopBin = makeFakeBin("desktop");
|
||||
const staleDefaultBin = makeFakeBin("stale-default");
|
||||
const wrongNodeBin = makeFakeBin("wrong-node");
|
||||
const missingSecretBin = makeFakeBin("missing-secret");
|
||||
const liveBody = "data:application/json,%7B%22serviceId%22%3A%22hwlab-cloud-web%22%2C%22environment%22%3A%22dev%22%2C%22status%22%3A%22ok%22%2C%22revision%22%3A%22abc1234%22%7D";
|
||||
const apiBody = "data:application/json,%7B%22serviceId%22%3A%22hwlab-cloud-api%22%2C%22environment%22%3A%22dev%22%2C%22status%22%3A%22ok%22%2C%22revision%22%3A%22abc1234%22%7D";
|
||||
const secondPlaneBin = makeFakeBin("second-plane");
|
||||
|
||||
const help = runCli(["hwlab", "help"]);
|
||||
assert.equal(help.ok, true);
|
||||
assert.equal((help.data as JsonRecord).command, "hwlab cd");
|
||||
|
||||
const runnerHistoryRepo = runCli([
|
||||
const remoteCommand = buildHwlabCdRemoteCommandForTest(withLocalTransport(["cd", "apply", "--env", "dev", "--dry-run"]));
|
||||
assert.equal(remoteCommand.includes("scripts/dev-cd-apply.mjs"), true);
|
||||
assert.equal(remoteCommand.includes("/etc/rancher/k3s/k3s.yaml"), true);
|
||||
assert.equal(remoteCommand.includes("kubectl rollout"), false);
|
||||
|
||||
const realApply = runCli(["hwlab", "cd", "apply", "--env", "dev", "--hwlab-repo", fakeRepo], {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(realApply.ok, false);
|
||||
assert.equal(dataOf(realApply).error, "host-commander-only-real-apply");
|
||||
assert.equal((dataOf(realApply).safety as JsonRecord).rolloutExecuted, false);
|
||||
|
||||
const runnerHistoryRepo = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"status",
|
||||
@@ -153,15 +163,13 @@ const runnerHistoryRepo = runCli([
|
||||
"dev",
|
||||
"--hwlab-repo",
|
||||
"/home/ubuntu/hwlab",
|
||||
], {
|
||||
]), {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(runnerHistoryRepo.ok, false);
|
||||
const runnerHistoryCandidates = ((runnerHistoryRepo.data as JsonRecord).repo as JsonRecord).candidates as JsonRecord[];
|
||||
assert.equal(runnerHistoryCandidates[0]?.rejected, true);
|
||||
assert.equal(runnerHistoryCandidates[0]?.rejectionReason, "runner-history-directory-is-not-hwlab-cd-release-truth");
|
||||
assert.equal((dataOf(runnerHistoryRepo).repo as JsonRecord).rejected, true);
|
||||
|
||||
const applyDryRun = runCli([
|
||||
const applyDryRun = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
@@ -170,55 +178,30 @@ const applyDryRun = runCli([
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
]), {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(applyDryRun.ok, true);
|
||||
const dryRunData = applyDryRun.data as JsonRecord;
|
||||
const dryRunData = dataOf(applyDryRun);
|
||||
assert.equal(dryRunData.dryRun, true);
|
||||
assert.equal(dryRunData.mutation, false);
|
||||
assert.equal(((dryRunData.d601NativeK3sGuard as JsonRecord).injectedEnv as JsonRecord).KUBECONFIG, "/etc/rancher/k3s/k3s.yaml");
|
||||
assert.equal((dryRunData.d601NativeK3sGuard as JsonRecord).requiredNodePresent, true);
|
||||
assert.equal((dryRunData.controlledDryRun as JsonRecord).commandOk, true);
|
||||
assert.equal((dryRunData.secretRefPreflight as JsonRecord).status, "pass");
|
||||
assert.equal(((dryRunData.controlledDryRun as JsonRecord).controlledEntrypoint), "scripts/dev-cd-apply.mjs");
|
||||
assert.equal(((dryRunData.hostCommanderOnlyLiveApply as JsonRecord).commandShape as unknown[]).includes("scripts/dev-cd-apply.mjs"), true);
|
||||
assert.equal((dryRunData.safety as JsonRecord).kubectlApplyExecuted, false);
|
||||
assert.equal((dryRunData.safety as JsonRecord).rolloutExecuted, false);
|
||||
assert.equal((dryRunData.safety as JsonRecord).liveVerifyExecuted, false);
|
||||
assert.equal((dryRunData.safety as JsonRecord).cdLockMutated, false);
|
||||
assert.equal((dryRunData.safety as JsonRecord).secretValuesPrinted, false);
|
||||
assert.equal((dryRunData.remote as JsonRecord).providerId, "D601");
|
||||
assert.equal(((dryRunData.remote as JsonRecord).commandCalls as unknown[]).includes("scripts/dev-cd-apply.mjs"), true);
|
||||
assert.equal((dryRunData.kubeconfig as JsonRecord).path, "/etc/rancher/k3s/k3s.yaml");
|
||||
assert.equal((dryRunData.nodeGuard as JsonRecord).requiredNodePresent, true);
|
||||
assert.equal((dryRunData.worktreeGuard as JsonRecord).clean, true);
|
||||
assert.equal((dryRunData.secretPreflight as JsonRecord).status, "pass");
|
||||
assert.equal((dryRunData.controlledDevCd as JsonRecord).controlledEntrypoint, "scripts/dev-cd-apply.mjs");
|
||||
assert.equal(((dryRunData.target as JsonRecord).promotionCommit), "abc1234567890abcdef");
|
||||
assert.equal(((dryRunData.promotion as JsonRecord).source), "deploy/deploy.json");
|
||||
assert.equal(JSON.stringify(dryRunData).includes("sk-secret"), false);
|
||||
|
||||
const preflight = runCli([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"preflight",
|
||||
"--env",
|
||||
"dev",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
UNIDESK_HWLAB_CD_TEST_FRONTEND_LIVE_URL: liveBody,
|
||||
UNIDESK_HWLAB_CD_TEST_API_LIVE_URL: apiBody,
|
||||
});
|
||||
assert.equal(preflight.ok, true);
|
||||
const preflightData = preflight.data as JsonRecord;
|
||||
assert.equal(preflightData.mutation, false);
|
||||
assert.equal((preflightData.secretRefPreflight as JsonRecord).status, "pass");
|
||||
assert.equal((preflightData.liveWorkloads as JsonRecord).status, "observed");
|
||||
|
||||
const realApply = runCli([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
"--env",
|
||||
"dev",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(realApply.ok, false);
|
||||
assert.equal((realApply.data as JsonRecord).error, "host-commander-only-real-apply");
|
||||
|
||||
const status = runCli([
|
||||
const status = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"status",
|
||||
@@ -226,18 +209,16 @@ const status = runCli([
|
||||
"dev",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
]), {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
UNIDESK_HWLAB_CD_TEST_FRONTEND_LIVE_URL: liveBody,
|
||||
UNIDESK_HWLAB_CD_TEST_API_LIVE_URL: apiBody,
|
||||
});
|
||||
assert.equal(status.ok, true);
|
||||
const statusData = status.data as JsonRecord;
|
||||
assert.equal(((statusData.d601NativeK3sGuard as JsonRecord).injectedEnv as JsonRecord).KUBECONFIG, "/etc/rancher/k3s/k3s.yaml");
|
||||
assert.equal((statusData.liveRevisions as JsonRecord).status, "observed");
|
||||
assert.ok(typeof statusData.dumpDir === "string" && String(statusData.dumpDir).includes(".state/hwlab-cd"));
|
||||
const statusData = dataOf(status);
|
||||
assert.equal((statusData.secretPreflight as JsonRecord).status, "skipped");
|
||||
assert.equal((statusData.lockState as JsonRecord).status, "absent");
|
||||
assert.ok(typeof statusData.reportDumpPath === "string");
|
||||
|
||||
const staleDefaultOk = runCli([
|
||||
const staleDefaultOk = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
@@ -246,18 +227,17 @@ const staleDefaultOk = runCli([
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
]), {
|
||||
PATH: `${staleDefaultBin}:${process.env.PATH ?? ""}`,
|
||||
KUBECONFIG: "",
|
||||
});
|
||||
assert.equal(staleDefaultOk.ok, true);
|
||||
const staleDefaultGuard = (staleDefaultOk.data as JsonRecord).d601NativeK3sGuard as JsonRecord;
|
||||
const staleDefaultGuard = dataOf(staleDefaultOk).nodeGuard as JsonRecord;
|
||||
assert.equal(staleDefaultGuard.status, "pass");
|
||||
assert.equal(staleDefaultGuard.refusal, false);
|
||||
assert.equal((staleDefaultGuard.defaultKubectlDiagnostic as JsonRecord).status, "stale-forbidden-default");
|
||||
assert.deepEqual((staleDefaultGuard.defaultKubectlDiagnostic as JsonRecord).refusalSignals, ["docker-desktop", "desktop-control-plane", "127.0.0.1:11700"]);
|
||||
|
||||
const desktopRefusal = runCli([
|
||||
const desktopRefusal = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
@@ -266,14 +246,15 @@ const desktopRefusal = runCli([
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
]), {
|
||||
PATH: `${desktopBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(desktopRefusal.ok, false);
|
||||
assert.equal((desktopRefusal.data as JsonRecord).error, "native-k3s-guard-refused");
|
||||
assert.deepEqual((desktopRefusal.data as JsonRecord).d601NativeK3sGuard && ((desktopRefusal.data as JsonRecord).d601NativeK3sGuard as JsonRecord).refusalSignals, ["docker-desktop", "desktop-control-plane", "127.0.0.1:11700"]);
|
||||
const desktopData = dataOf(desktopRefusal);
|
||||
assert.equal(desktopData.status, "refused");
|
||||
assert.deepEqual((desktopData.nodeGuard as JsonRecord).refusalSignals, ["docker-desktop", "desktop-control-plane", "127.0.0.1:11700"]);
|
||||
|
||||
const wrongNodeBlocked = runCli([
|
||||
const wrongNodeBlocked = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
@@ -282,16 +263,15 @@ const wrongNodeBlocked = runCli([
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
]), {
|
||||
PATH: `${wrongNodeBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(wrongNodeBlocked.ok, false);
|
||||
const wrongNodeGuard = (wrongNodeBlocked.data as JsonRecord).d601NativeK3sGuard as JsonRecord;
|
||||
assert.equal(wrongNodeGuard.status, "blocked");
|
||||
assert.equal(wrongNodeGuard.requiredNodePresent, false);
|
||||
assert.equal(((wrongNodeBlocked.data as JsonRecord).blockers as JsonRecord[]).some((blocker) => blocker.scope === "d601-native-k3s-guard"), true);
|
||||
const wrongNodeData = dataOf(wrongNodeBlocked);
|
||||
assert.equal((wrongNodeData.nodeGuard as JsonRecord).requiredNodePresent, false);
|
||||
assert.equal(scopes(wrongNodeData).includes("d601-native-k3s-guard"), true);
|
||||
|
||||
const missingSecretBlocked = runCli([
|
||||
const missingSecretBlocked = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
@@ -300,14 +280,46 @@ const missingSecretBlocked = runCli([
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
], {
|
||||
]), {
|
||||
PATH: `${missingSecretBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(missingSecretBlocked.ok, false);
|
||||
const missingSecretData = missingSecretBlocked.data as JsonRecord;
|
||||
assert.equal((missingSecretData.secretRefPreflight as JsonRecord).status, "blocked");
|
||||
assert.equal((missingSecretData.controlledDryRun as JsonRecord).status, "skipped");
|
||||
assert.equal((missingSecretData.blockers as JsonRecord[]).some((blocker) => blocker.scope === "secretref:hwlab-code-agent-provider/openai-api-key"), true);
|
||||
const missingSecretData = dataOf(missingSecretBlocked);
|
||||
assert.equal((missingSecretData.secretPreflight as JsonRecord).status, "blocked");
|
||||
assert.equal((missingSecretData.controlledDevCd as JsonRecord).status, "skipped");
|
||||
assert.equal(scopes(missingSecretData).includes("secretref:hwlab-code-agent-provider/openai-api-key"), true);
|
||||
assert.equal(JSON.stringify(missingSecretData).includes("sk-secret"), false);
|
||||
|
||||
const secondPlaneBlocked = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
"--env",
|
||||
"dev",
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
]), {
|
||||
PATH: `${secondPlaneBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(secondPlaneBlocked.ok, false);
|
||||
assert.equal(((dataOf(secondPlaneBlocked).nodeGuard as JsonRecord).defaultKubectlDiagnostic as JsonRecord).secondControlPlaneRisk, true);
|
||||
assert.equal(scopes(dataOf(secondPlaneBlocked)).includes("second-hwlab-dev-control-plane"), true);
|
||||
|
||||
writeFileSync(join(fakeRepo, "dirty.txt"), "dirty\n");
|
||||
const dirtyBlocked = runCli(withLocalTransport([
|
||||
"hwlab",
|
||||
"cd",
|
||||
"apply",
|
||||
"--env",
|
||||
"dev",
|
||||
"--dry-run",
|
||||
"--hwlab-repo",
|
||||
fakeRepo,
|
||||
]), {
|
||||
PATH: `${nativeBin}:${process.env.PATH ?? ""}`,
|
||||
});
|
||||
assert.equal(dirtyBlocked.ok, false);
|
||||
assert.equal(scopes(dataOf(dirtyBlocked)).includes("hwlab-git-clean"), true);
|
||||
|
||||
console.log(JSON.stringify({ ok: true, checked: "hwlab-cd-wrapper-contract" }));
|
||||
|
||||
@@ -0,0 +1,676 @@
|
||||
const { spawn } = require("node:child_process");
|
||||
const crypto = require("node:crypto");
|
||||
const fs = require("node:fs");
|
||||
const fsp = require("node:fs/promises");
|
||||
const os = require("node:os");
|
||||
const path = require("node:path");
|
||||
|
||||
const namespace = "hwlab-dev";
|
||||
const lockName = "hwlab-dev-cd-lock";
|
||||
const nativeKubeconfig = "/etc/rancher/k3s/k3s.yaml";
|
||||
const defaultRepo = "/home/ubuntu/hwlab_cd";
|
||||
const rejectedRepo = "/home/ubuntu/hwlab";
|
||||
const requiredNodeName = "d601";
|
||||
const tailChars = 1400;
|
||||
const parseCaptureLimitBytes = 1024 * 1024;
|
||||
const requiredSecretRefs = [
|
||||
{ secretName: "hwlab-cloud-api-dev-db", secretKey: "database-url", consumers: ["hwlab-cloud-api"] },
|
||||
{ secretName: "hwlab-cloud-api-dev-db-admin", secretKey: "admin-url", consumers: ["runtime provisioning", "runtime migration"] },
|
||||
{ secretName: "hwlab-code-agent-provider", secretKey: "openai-api-key", consumers: ["hwlab-cloud-api", "code agent provider"] },
|
||||
];
|
||||
|
||||
function decodeOptions() {
|
||||
const raw = process.env.UNIDESK_HWLAB_CD_OPTIONS_B64 || "";
|
||||
if (!raw) throw new Error("UNIDESK_HWLAB_CD_OPTIONS_B64 is required");
|
||||
return JSON.parse(Buffer.from(raw, "base64").toString("utf8"));
|
||||
}
|
||||
|
||||
function redact(value) {
|
||||
return String(value || "")
|
||||
.replace(/\b(Bearer\s+)[A-Za-z0-9._~+/=-]+/giu, "$1<redacted>")
|
||||
.replace(/\b(token|password|api[_-]?key|secret|client-key-data|client-certificate-data|certificate-authority-data)\b(\s*[:=]\s*)(["']?)([^"'\s,}]+)/giu, "$1$2$3<redacted>")
|
||||
.replace(/\b(postgres(?:ql)?:\/\/[^:\s/@]+:)([^@\s]+)(@)/giu, "$1<redacted>$3");
|
||||
}
|
||||
|
||||
function boundedTail(value) {
|
||||
const text = redact(value);
|
||||
return text.slice(Math.max(0, text.length - tailChars));
|
||||
}
|
||||
|
||||
function readFileTail(filePath) {
|
||||
try {
|
||||
return boundedTail(fs.readFileSync(filePath, "utf8"));
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
function commandView(result) {
|
||||
return {
|
||||
id: result.id,
|
||||
command: result.command,
|
||||
cwd: result.cwd,
|
||||
ok: result.ok,
|
||||
exitCode: result.exitCode,
|
||||
signal: result.signal,
|
||||
timedOut: result.timedOut,
|
||||
durationMs: result.durationMs,
|
||||
dump: {
|
||||
stdout: result.dump.stdout,
|
||||
stderr: result.dump.stderr,
|
||||
stdoutBytes: result.dump.stdoutBytes,
|
||||
stderrBytes: result.dump.stderrBytes,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function runCaptured(command, cwd, dumpDir, id, options = {}) {
|
||||
await fsp.mkdir(dumpDir, { recursive: true, mode: 0o700 });
|
||||
const stdoutPath = path.join(dumpDir, `${id}.stdout.txt`);
|
||||
const stderrPath = path.join(dumpDir, `${id}.stderr.txt`);
|
||||
fs.writeFileSync(stdoutPath, "", { mode: 0o600 });
|
||||
fs.writeFileSync(stderrPath, "", { mode: 0o600 });
|
||||
|
||||
const startedAt = Date.now();
|
||||
let timedOut = false;
|
||||
let spawnErrorMessage = "";
|
||||
let stdoutBytes = 0;
|
||||
let stderrBytes = 0;
|
||||
const stdoutChunks = [];
|
||||
const stderrChunks = [];
|
||||
const stdout = fs.createWriteStream(stdoutPath, { flags: "a" });
|
||||
const stderr = fs.createWriteStream(stderrPath, { flags: "a" });
|
||||
const child = spawn(command[0], command.slice(1), {
|
||||
cwd,
|
||||
env: options.env || process.env,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
const timeout = options.timeoutMs
|
||||
? setTimeout(() => {
|
||||
timedOut = true;
|
||||
child.kill("SIGTERM");
|
||||
setTimeout(() => child.kill("SIGKILL"), 1000).unref();
|
||||
}, options.timeoutMs)
|
||||
: null;
|
||||
|
||||
child.stdout.on("data", (chunk) => {
|
||||
stdoutBytes += chunk.byteLength;
|
||||
stdout.write(chunk);
|
||||
if (Buffer.concat(stdoutChunks).byteLength < parseCaptureLimitBytes) stdoutChunks.push(Buffer.from(chunk));
|
||||
});
|
||||
child.stderr.on("data", (chunk) => {
|
||||
stderrBytes += chunk.byteLength;
|
||||
stderr.write(chunk);
|
||||
if (Buffer.concat(stderrChunks).byteLength < parseCaptureLimitBytes) stderrChunks.push(Buffer.from(chunk));
|
||||
});
|
||||
|
||||
const closed = await new Promise((resolve) => {
|
||||
child.on("error", (error) => {
|
||||
spawnErrorMessage = error.message;
|
||||
});
|
||||
child.on("close", (exitCode, signal) => resolve({ exitCode, signal }));
|
||||
});
|
||||
if (timeout !== null) clearTimeout(timeout);
|
||||
await new Promise((resolve) => stdout.end(resolve));
|
||||
await new Promise((resolve) => stderr.end(resolve));
|
||||
|
||||
const stdoutText = stdoutBytes <= parseCaptureLimitBytes ? Buffer.concat(stdoutChunks).toString("utf8") : "";
|
||||
const stderrText = stderrBytes <= parseCaptureLimitBytes ? Buffer.concat(stderrChunks).toString("utf8") : spawnErrorMessage;
|
||||
const exitCode = spawnErrorMessage && closed.exitCode === null ? 127 : closed.exitCode;
|
||||
return {
|
||||
id,
|
||||
command,
|
||||
cwd,
|
||||
ok: exitCode === 0 && !timedOut,
|
||||
exitCode,
|
||||
signal: closed.signal,
|
||||
timedOut,
|
||||
durationMs: Date.now() - startedAt,
|
||||
stdoutText,
|
||||
stderrText,
|
||||
dump: {
|
||||
stdout: stdoutPath,
|
||||
stderr: stderrPath,
|
||||
stdoutBytes,
|
||||
stderrBytes,
|
||||
stdoutTail: boundedTail(stdoutText || readFileTail(stdoutPath)),
|
||||
stderrTail: boundedTail(stderrText || readFileTail(stderrPath)),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function parseJson(text) {
|
||||
if (!String(text || "").trim()) return null;
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function asRecord(value) {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? value : null;
|
||||
}
|
||||
|
||||
function stringValue(value) {
|
||||
return typeof value === "string" && value.length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function sha256(text) {
|
||||
return `sha256:${crypto.createHash("sha256").update(text).digest("hex")}`;
|
||||
}
|
||||
|
||||
function accessCheck(targetPath, mode) {
|
||||
try {
|
||||
fs.accessSync(targetPath, mode);
|
||||
return { ok: true, path: targetPath };
|
||||
} catch (error) {
|
||||
return { ok: false, path: targetPath, error: error instanceof Error ? error.message : String(error) };
|
||||
}
|
||||
}
|
||||
|
||||
function resolveRepo(provided) {
|
||||
const rawPath = provided || process.env.UNIDESK_HWLAB_REPO || defaultRepo;
|
||||
const absolutePath = path.resolve(rawPath);
|
||||
const rejected = absolutePath === rejectedRepo;
|
||||
const devCdApply = path.join(absolutePath, "scripts/dev-cd-apply.mjs");
|
||||
const deployJson = path.join(absolutePath, "deploy/deploy.json");
|
||||
return {
|
||||
status: !rejected && fs.existsSync(absolutePath) && fs.existsSync(devCdApply) && fs.existsSync(deployJson) ? "selected" : "blocked",
|
||||
source: provided ? "option" : process.env.UNIDESK_HWLAB_REPO ? "env:UNIDESK_HWLAB_REPO" : "default:d601-clean-mirror",
|
||||
path: absolutePath,
|
||||
defaultPath: defaultRepo,
|
||||
rejected,
|
||||
rejectionReason: rejected ? "runner-history-directory-is-not-hwlab-cd-release-truth" : null,
|
||||
exists: fs.existsSync(absolutePath),
|
||||
hasDevCdApply: fs.existsSync(devCdApply),
|
||||
hasDeployJson: fs.existsSync(deployJson),
|
||||
};
|
||||
}
|
||||
|
||||
async function gitSummary(repoPath, dumpDir, timeoutMs) {
|
||||
const [branch, head, originMain, remote, gitDir, gitCommonDir, statusShort, statusPorcelain] = await Promise.all([
|
||||
runCaptured(["git", "rev-parse", "--abbrev-ref", "HEAD"], repoPath, dumpDir, "git-branch", { timeoutMs }),
|
||||
runCaptured(["git", "rev-parse", "HEAD"], repoPath, dumpDir, "git-head", { timeoutMs }),
|
||||
runCaptured(["git", "rev-parse", "--verify", "origin/main^{commit}"], repoPath, dumpDir, "git-origin-main", { timeoutMs }),
|
||||
runCaptured(["git", "remote", "get-url", "origin"], repoPath, dumpDir, "git-remote-origin", { timeoutMs }),
|
||||
runCaptured(["git", "rev-parse", "--path-format=absolute", "--git-dir"], repoPath, dumpDir, "git-dir", { timeoutMs }),
|
||||
runCaptured(["git", "rev-parse", "--path-format=absolute", "--git-common-dir"], repoPath, dumpDir, "git-common-dir", { timeoutMs }),
|
||||
runCaptured(["git", "status", "--short", "--branch"], repoPath, dumpDir, "git-status-short", { timeoutMs }),
|
||||
runCaptured(["git", "status", "--porcelain=v1"], repoPath, dumpDir, "git-status-porcelain", { timeoutMs }),
|
||||
]);
|
||||
const branchName = branch.stdoutText.trim();
|
||||
const headCommit = head.stdoutText.trim();
|
||||
const originMainCommit = originMain.stdoutText.trim();
|
||||
const remoteUrl = remote.stdoutText.trim();
|
||||
const gitDirPath = gitDir.stdoutText.trim();
|
||||
const gitCommonDirPath = gitCommonDir.stdoutText.trim() || gitDirPath;
|
||||
const fetchHeadPath = gitCommonDirPath ? path.join(gitCommonDirPath, "FETCH_HEAD") : path.join(repoPath, ".git", "FETCH_HEAD");
|
||||
const worktreesPath = gitCommonDirPath ? path.join(gitCommonDirPath, "worktrees") : path.join(repoPath, ".git", "worktrees");
|
||||
const porcelain = statusPorcelain.stdoutText.trim();
|
||||
const dirtyLines = porcelain ? porcelain.split("\n").filter(Boolean) : [];
|
||||
const worktreeAccess = accessCheck(repoPath, fs.constants.R_OK | fs.constants.W_OK | fs.constants.X_OK);
|
||||
const fetchHeadAccess = accessCheck(fetchHeadPath, fs.constants.R_OK | fs.constants.W_OK);
|
||||
const worktreesAccess = fs.existsSync(worktreesPath)
|
||||
? accessCheck(worktreesPath, fs.constants.R_OK | fs.constants.W_OK | fs.constants.X_OK)
|
||||
: { ok: true, path: worktreesPath, exists: false };
|
||||
const remoteMatches = /github\.com[:/]pikasTech\/HWLAB(?:\.git)?$/u.test(remoteUrl);
|
||||
const blockers = [];
|
||||
if (!branch.ok || !head.ok || !remote.ok || !gitDir.ok || !gitCommonDir.ok || !statusShort.ok || !statusPorcelain.ok) blockers.push({ scope: "hwlab-git-command", summary: "One or more HWLAB git guard commands failed." });
|
||||
if (dirtyLines.length > 0) blockers.push({ scope: "hwlab-git-clean", summary: "HWLAB CD worktree has local modifications; deploy/deploy.json cannot be treated as release truth." });
|
||||
if (branchName !== "main") blockers.push({ scope: "hwlab-git-main", summary: `HWLAB CD worktree branch is ${branchName || "<unknown>"}, expected main.` });
|
||||
if (!remoteMatches) blockers.push({ scope: "hwlab-git-remote", summary: "HWLAB CD worktree origin is not pikasTech/HWLAB." });
|
||||
if (originMain.ok && headCommit && originMainCommit && headCommit !== originMainCommit) blockers.push({ scope: "hwlab-git-origin-main", summary: "HWLAB CD worktree HEAD does not match local origin/main." });
|
||||
if (!worktreeAccess.ok) blockers.push({ scope: "hwlab-worktree-permission", summary: "HWLAB CD worktree is not readable/writable/executable by the wrapper." });
|
||||
if (!fetchHeadAccess.ok) blockers.push({ scope: "hwlab-fetch-head-permission", summary: "HWLAB CD FETCH_HEAD is missing or not writable by the wrapper." });
|
||||
if (!worktreesAccess.ok) blockers.push({ scope: "hwlab-worktrees-permission", summary: "HWLAB .git/worktrees permission guard failed." });
|
||||
return {
|
||||
status: blockers.length === 0 ? "pass" : "blocked",
|
||||
clean: dirtyLines.length === 0,
|
||||
branch: branchName || null,
|
||||
onMain: branchName === "main",
|
||||
remote: remoteUrl || null,
|
||||
remoteMatches,
|
||||
expectedRemote: "git@github.com:pikasTech/HWLAB.git or https://github.com/pikasTech/HWLAB.git",
|
||||
headCommit: headCommit || null,
|
||||
originMainCommit: originMain.ok ? originMainCommit || null : null,
|
||||
headMatchesOriginMain: originMain.ok && headCommit.length > 0 && headCommit === originMainCommit,
|
||||
worktreeAccess,
|
||||
fetchHead: { path: fetchHeadPath, exists: fs.existsSync(fetchHeadPath), writable: fetchHeadAccess.ok, error: fetchHeadAccess.error || null },
|
||||
worktrees: worktreesAccess,
|
||||
statusShort: statusShort.stdoutText.trim().split("\n").filter(Boolean).slice(0, 30),
|
||||
dirtyCount: dirtyLines.length,
|
||||
dirtyPreview: dirtyLines.slice(0, 20),
|
||||
blockers,
|
||||
commands: [branch, head, originMain, remote, gitDir, gitCommonDir, statusShort, statusPorcelain].map(commandView),
|
||||
};
|
||||
}
|
||||
|
||||
function readJsonSummary(repoPath, relativePath, fallback = {}) {
|
||||
const absolutePath = path.join(repoPath, relativePath);
|
||||
try {
|
||||
const raw = fs.readFileSync(absolutePath, "utf8");
|
||||
const parsed = JSON.parse(raw);
|
||||
return {
|
||||
path: relativePath,
|
||||
exists: true,
|
||||
hash: sha256(raw),
|
||||
commitId: parsed.commitId || parsed.sourceCommitId || null,
|
||||
environment: parsed.environment || null,
|
||||
namespace: parsed.namespace || namespace,
|
||||
endpoint: parsed.endpoint || null,
|
||||
artifactState: parsed.artifactState || parsed.publish?.artifactState || null,
|
||||
ciPublished: parsed.publish?.ciPublished ?? parsed.artifactPublish?.ciPublished ?? null,
|
||||
registryVerified: parsed.publish?.registryVerified ?? parsed.artifactPublish?.registryVerified ?? null,
|
||||
serviceCount: Array.isArray(parsed.services) ? parsed.services.length : parsed.artifactPublish?.serviceCount ?? null,
|
||||
...fallback,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
path: relativePath,
|
||||
exists: false,
|
||||
hash: null,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
...fallback,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function forbiddenKubeSignals(text) {
|
||||
const signals = [];
|
||||
if (/docker-desktop/iu.test(text)) signals.push("docker-desktop");
|
||||
if (/desktop-control-plane/iu.test(text)) signals.push("desktop-control-plane");
|
||||
if (/127\.0\.0\.1:11700/u.test(text)) signals.push("127.0.0.1:11700");
|
||||
return [...new Set(signals)];
|
||||
}
|
||||
|
||||
async function observeKube(kubeconfig, dumpDir, timeoutMs, withExplicitEnv) {
|
||||
const env = withExplicitEnv ? { ...process.env, KUBECONFIG: kubeconfig } : { ...process.env };
|
||||
if (!withExplicitEnv) {
|
||||
delete env.KUBECONFIG;
|
||||
delete env.HWLAB_DEV_KUBECONFIG;
|
||||
}
|
||||
const [context, server, nodes, namespaceCheck] = await Promise.all([
|
||||
runCaptured(["kubectl", "config", "current-context"], process.cwd(), dumpDir, withExplicitEnv ? "k3s-current-context" : "default-current-context", { env, timeoutMs }),
|
||||
runCaptured(["kubectl", "config", "view", "--minify", "-o", "jsonpath={.clusters[0].cluster.server}"], process.cwd(), dumpDir, withExplicitEnv ? "k3s-server" : "default-server", { env, timeoutMs }),
|
||||
runCaptured(["kubectl", "get", "nodes", "-o", "jsonpath={range .items[*]}{.metadata.name}{\"\\n\"}{end}"], process.cwd(), dumpDir, withExplicitEnv ? "k3s-nodes" : "default-nodes", { env, timeoutMs }),
|
||||
runCaptured(["kubectl", "get", "namespace", namespace, "-o", "name"], process.cwd(), dumpDir, withExplicitEnv ? "k3s-namespace" : "default-namespace", { env, timeoutMs }),
|
||||
]);
|
||||
const combinedText = [context.stdoutText, context.stderrText, server.stdoutText, server.stderrText, nodes.stdoutText, nodes.stderrText, namespaceCheck.stdoutText, namespaceCheck.stderrText].join("\n");
|
||||
return {
|
||||
checked: true,
|
||||
explicit: withExplicitEnv,
|
||||
currentContext: context.stdoutText.trim() || null,
|
||||
apiServer: server.stdoutText.trim() || null,
|
||||
nodeNames: nodes.stdoutText.split(/\r?\n/u).map((line) => line.trim()).filter(Boolean),
|
||||
namespaceObserved: namespaceCheck.ok,
|
||||
commandsOk: context.ok && server.ok && nodes.ok,
|
||||
refusalSignals: forbiddenKubeSignals(combinedText),
|
||||
commands: [context, server, nodes, namespaceCheck].map(commandView),
|
||||
};
|
||||
}
|
||||
|
||||
async function nodeGuard(kubeconfig, dumpDir, timeoutMs) {
|
||||
const [explicit, bare] = await Promise.all([
|
||||
observeKube(kubeconfig, dumpDir, timeoutMs, true),
|
||||
observeKube(kubeconfig, dumpDir, timeoutMs, false),
|
||||
]);
|
||||
const wrongKubeconfig = kubeconfig !== nativeKubeconfig;
|
||||
const requiredNodePresent = explicit.nodeNames.includes(requiredNodeName);
|
||||
const explicitRefusal = explicit.refusalSignals.length > 0;
|
||||
const secondControlPlaneRisk = Boolean(
|
||||
bare.namespaceObserved &&
|
||||
explicit.apiServer &&
|
||||
bare.apiServer &&
|
||||
bare.apiServer !== explicit.apiServer
|
||||
);
|
||||
const status = explicitRefusal
|
||||
? "refused"
|
||||
: wrongKubeconfig || !explicit.commandsOk || !requiredNodePresent
|
||||
? "blocked"
|
||||
: "pass";
|
||||
return {
|
||||
status,
|
||||
refusal: explicitRefusal,
|
||||
refusalSignals: explicit.refusalSignals,
|
||||
kubeconfig,
|
||||
expectedKubeconfig: nativeKubeconfig,
|
||||
currentContext: explicit.currentContext,
|
||||
apiServer: explicit.apiServer,
|
||||
nodeNames: explicit.nodeNames,
|
||||
nodeCount: explicit.nodeNames.length,
|
||||
requiredNodeName,
|
||||
requiredNodePresent,
|
||||
commandsOk: explicit.commandsOk,
|
||||
secondControlPlaneRisk,
|
||||
defaultKubectlDiagnostic: {
|
||||
checked: true,
|
||||
currentContext: bare.currentContext,
|
||||
apiServer: bare.apiServer,
|
||||
nodeNames: bare.nodeNames,
|
||||
namespaceObserved: bare.namespaceObserved,
|
||||
refusalSignals: bare.refusalSignals,
|
||||
status: bare.refusalSignals.length > 0 ? "stale-forbidden-default" : bare.commandsOk ? "clean" : "unavailable",
|
||||
secondControlPlaneRisk,
|
||||
summary: secondControlPlaneRisk
|
||||
? "Bare kubectl can observe hwlab-dev through a different control plane; write paths must stop."
|
||||
: bare.refusalSignals.length > 0
|
||||
? "Bare kubectl shows a forbidden local control-plane signal; explicit D601 KUBECONFIG remains the target."
|
||||
: "Bare kubectl did not prove a second HWLAB DEV control plane.",
|
||||
},
|
||||
summary: explicitRefusal
|
||||
? "Refusing because explicit D601 kubeconfig resolved to a forbidden Docker Desktop control-plane signal."
|
||||
: wrongKubeconfig
|
||||
? `Expected KUBECONFIG=${nativeKubeconfig}.`
|
||||
: !explicit.commandsOk
|
||||
? "Explicit D601 kubeconfig could not read context, server, and nodes."
|
||||
: !requiredNodePresent
|
||||
? "Explicit D601 kubeconfig did not report node d601."
|
||||
: "D601 native k3s guard passed with explicit KUBECONFIG.",
|
||||
commands: [...explicit.commands, ...bare.commands],
|
||||
};
|
||||
}
|
||||
|
||||
async function lockStatus(kubeconfig, guard, dumpDir, timeoutMs) {
|
||||
if (guard.status !== "pass") {
|
||||
return { status: "skipped", reason: "d601-native-k3s-guard-not-pass", namespace, lockName, present: null };
|
||||
}
|
||||
const env = { ...process.env, KUBECONFIG: kubeconfig };
|
||||
const result = await runCaptured(["kubectl", "-n", namespace, "get", "lease", lockName, "-o", "json"], process.cwd(), dumpDir, "cd-lock", { env, timeoutMs });
|
||||
const text = `${result.stdoutText}\n${result.stderrText}`;
|
||||
if (!result.ok && /not\s*found|notfound/iu.test(text)) {
|
||||
return { status: "absent", namespace, lockName, present: false, held: false, stale: false, command: commandView(result) };
|
||||
}
|
||||
const parsed = asRecord(parseJson(result.stdoutText));
|
||||
const metadata = asRecord(parsed?.metadata) || {};
|
||||
const spec = asRecord(parsed?.spec) || {};
|
||||
const annotations = asRecord(metadata.annotations) || {};
|
||||
const annotation = (name) => stringValue(annotations[`hwlab.pikastech.local/${name}`]);
|
||||
const lock = parsed === null ? null : {
|
||||
lockBackend: "Lease",
|
||||
lockName: stringValue(metadata.name) || lockName,
|
||||
namespace: stringValue(metadata.namespace) || namespace,
|
||||
holderIdentity: stringValue(spec.holderIdentity),
|
||||
resourceVersion: stringValue(metadata.resourceVersion),
|
||||
ownerTaskId: annotation("ownerTaskId"),
|
||||
transactionId: annotation("transactionId"),
|
||||
phase: annotation("phase") || "unknown",
|
||||
promotionCommit: annotation("promotionCommit"),
|
||||
deployJsonHash: annotation("deployJsonHash"),
|
||||
targetRef: annotation("targetRef"),
|
||||
targetNamespace: annotation("targetNamespace"),
|
||||
updatedAt: annotation("updatedAt") || stringValue(spec.renewTime),
|
||||
ttlSeconds: Number(annotation("ttlSeconds") || spec.leaseDurationSeconds || 3600),
|
||||
releaseStatus: annotation("releaseStatus"),
|
||||
releasedAt: annotation("releasedAt"),
|
||||
};
|
||||
const updatedAt = stringValue(lock?.updatedAt) || "";
|
||||
const ttlSeconds = Number(lock?.ttlSeconds || 3600);
|
||||
const expiresAtMs = Number.isFinite(Date.parse(updatedAt)) ? Date.parse(updatedAt) + ttlSeconds * 1000 : 0;
|
||||
const retryAfterSeconds = Math.max(0, Math.ceil((expiresAtMs - Date.now()) / 1000));
|
||||
return {
|
||||
status: result.ok && lock !== null ? "observed" : "blocked",
|
||||
present: result.ok && lock !== null,
|
||||
...(lock || {}),
|
||||
held: lock !== null && lock.phase !== "released" && retryAfterSeconds > 0,
|
||||
stale: lock !== null && lock.phase !== "released" && retryAfterSeconds <= 0,
|
||||
retryAfterSeconds,
|
||||
expiresAt: expiresAtMs > 0 ? new Date(expiresAtMs).toISOString() : null,
|
||||
command: commandView(result),
|
||||
};
|
||||
}
|
||||
|
||||
function parseSecretDescribeKeys(stdout) {
|
||||
const keys = new Set();
|
||||
let inData = false;
|
||||
for (const rawLine of String(stdout || "").split(/\r?\n/u)) {
|
||||
const line = rawLine.trim();
|
||||
if (/^Data\b/iu.test(line)) {
|
||||
inData = true;
|
||||
continue;
|
||||
}
|
||||
if (!inData || line === "" || /^=+$/u.test(line)) continue;
|
||||
const match = line.match(/^([A-Za-z0-9_.-]+):\s+\d+\s+bytes\b/iu);
|
||||
if (match) keys.add(match[1]);
|
||||
}
|
||||
return [...keys].sort();
|
||||
}
|
||||
|
||||
async function secretPreflight(kubeconfig, guard, dumpDir, timeoutMs) {
|
||||
if (guard.status !== "pass") {
|
||||
return {
|
||||
status: "skipped",
|
||||
reason: "d601-native-k3s-guard-not-pass",
|
||||
namespace,
|
||||
mutation: false,
|
||||
safety: { secretValuesRead: false, secretValuesPrinted: false, secretKeyNamesOnly: true },
|
||||
secretRefs: requiredSecretRefs.map((ref) => ({ ...ref, status: "not_checked" })),
|
||||
blockers: [],
|
||||
};
|
||||
}
|
||||
const env = { ...process.env, KUBECONFIG: kubeconfig };
|
||||
const observations = [];
|
||||
for (const ref of requiredSecretRefs) {
|
||||
const idBase = `secretref-${ref.secretName}-${ref.secretKey}`.replace(/[^A-Za-z0-9_.-]/gu, "-");
|
||||
const exists = await runCaptured(["kubectl", "-n", namespace, "get", "secret", ref.secretName, "-o", "name"], process.cwd(), dumpDir, `${idBase}-exists`, { env, timeoutMs });
|
||||
if (!exists.ok) {
|
||||
observations.push({ ...ref, status: "missing-secret", exists: false, keyPresent: false, command: commandView(exists) });
|
||||
continue;
|
||||
}
|
||||
const describe = await runCaptured(["kubectl", "-n", namespace, "describe", "secret", ref.secretName], process.cwd(), dumpDir, `${idBase}-describe`, { env, timeoutMs });
|
||||
const keysObserved = parseSecretDescribeKeys(describe.stdoutText);
|
||||
const keyPresent = keysObserved.includes(ref.secretKey);
|
||||
observations.push({
|
||||
...ref,
|
||||
status: keyPresent ? "present" : describe.ok ? "missing-key" : "key-observation-blocked",
|
||||
exists: true,
|
||||
keyPresent,
|
||||
keysObserved,
|
||||
command: commandView(describe),
|
||||
});
|
||||
}
|
||||
const blockers = observations
|
||||
.filter((observation) => observation.status !== "present")
|
||||
.map((observation) => ({
|
||||
scope: `secretref:${observation.secretName}/${observation.secretKey}`,
|
||||
summary: observation.status === "missing-secret"
|
||||
? `Required Secret ${observation.secretName} is missing in ${namespace}.`
|
||||
: `Required SecretRef ${observation.secretName}/${observation.secretKey} is not present as key metadata in ${namespace}.`,
|
||||
impact: `${observation.consumers.join(", ")} would fail after a DEV CD apply or runtime preflight Job.`,
|
||||
runbook: `Create or repair Secret ${observation.secretName} with key ${observation.secretKey} in namespace ${namespace}; verify key presence only and do not print the Secret value.`,
|
||||
}));
|
||||
return {
|
||||
status: blockers.length === 0 ? "pass" : "blocked",
|
||||
namespace,
|
||||
mutation: false,
|
||||
safety: { secretValuesRead: false, secretValuesPrinted: false, secretKeyNamesOnly: true },
|
||||
requiredSecretRefs: requiredSecretRefs.map((ref) => `${ref.secretName}/${ref.secretKey}`),
|
||||
secretRefs: observations,
|
||||
blockers,
|
||||
};
|
||||
}
|
||||
|
||||
function summarizeControlled(parsed, command) {
|
||||
const record = asRecord(parsed);
|
||||
const target = asRecord(record?.target);
|
||||
const desiredStateCheck = asRecord(target?.desiredStateCheck);
|
||||
const artifactBoundary = asRecord(target?.artifactBoundary);
|
||||
return {
|
||||
status: stringValue(record?.status) || (command.ok ? "unknown" : "blocked"),
|
||||
commandOk: command.ok,
|
||||
mode: stringValue(record?.mode),
|
||||
mutationAttempted: record?.mutationAttempted ?? false,
|
||||
prodTouched: record?.prodTouched ?? false,
|
||||
controlledEntrypoint: "scripts/dev-cd-apply.mjs",
|
||||
target: target === null ? null : {
|
||||
ref: stringValue(target.ref),
|
||||
promotionCommit: stringValue(target.promotionCommit),
|
||||
shortCommitId: stringValue(target.shortCommitId),
|
||||
promotionSource: stringValue(target.promotionSource),
|
||||
publishRequired: target.publishRequired ?? null,
|
||||
headCommitId: stringValue(target.headCommitId),
|
||||
headMatchesTarget: target.headMatchesTarget ?? null,
|
||||
desiredStateCheck: desiredStateCheck === null ? null : {
|
||||
status: stringValue(desiredStateCheck.status),
|
||||
summary: desiredStateCheck.summary ?? null,
|
||||
diagnostics: Array.isArray(desiredStateCheck.diagnostics) ? desiredStateCheck.diagnostics.slice(0, 5) : [],
|
||||
},
|
||||
artifactBoundary: artifactBoundary === null ? null : {
|
||||
status: stringValue(artifactBoundary.status),
|
||||
desiredState: artifactBoundary.desiredState ?? null,
|
||||
},
|
||||
namespace: stringValue(target.namespace) || namespace,
|
||||
},
|
||||
deployJson: asRecord(record?.deployJson),
|
||||
artifactCatalog: asRecord(record?.artifactCatalog),
|
||||
artifactReport: asRecord(record?.artifactReport),
|
||||
lock: asRecord(record?.lock),
|
||||
liveDelta: asRecord(record?.liveDelta),
|
||||
blockers: Array.isArray(record?.blockers) ? record.blockers.slice(0, 12) : [],
|
||||
nextActions: Array.isArray(record?.nextActions) ? record.nextActions.slice(0, 12) : [],
|
||||
parsed: record !== null,
|
||||
command: commandView(command),
|
||||
};
|
||||
}
|
||||
|
||||
async function runDevCdApply(repoPath, kubeconfig, action, dumpDir, timeoutMs) {
|
||||
const modeArg = action === "status" ? "--status" : "--dry-run";
|
||||
const command = ["node", "scripts/dev-cd-apply.mjs", modeArg, "--kubeconfig", kubeconfig, "--skip-live-verify"];
|
||||
const result = await runCaptured(command, repoPath, dumpDir, `controlled-dev-cd-apply-${action === "status" ? "status" : "dry-run"}`, {
|
||||
env: { ...process.env, KUBECONFIG: kubeconfig },
|
||||
timeoutMs,
|
||||
});
|
||||
return summarizeControlled(parseJson(result.stdoutText), result);
|
||||
}
|
||||
|
||||
function collectBlockers({ repo, git, guard, lock, secretRefs, controlled, action }) {
|
||||
const blockers = [];
|
||||
if (repo.status !== "selected") {
|
||||
blockers.push({ scope: "hwlab-repo", summary: repo.rejected ? "Rejected runner history HWLAB directory." : "No eligible HWLAB CD repo with scripts/dev-cd-apply.mjs and deploy/deploy.json was found." });
|
||||
}
|
||||
if (git?.blockers) blockers.push(...git.blockers);
|
||||
if (guard) {
|
||||
if (guard.refusal) blockers.push({ scope: "d601-native-k3s-guard", summary: guard.summary, refusal: true });
|
||||
else if (guard.status !== "pass") blockers.push({ scope: "d601-native-k3s-guard", summary: guard.summary });
|
||||
if ((action === "preflight" || action === "apply") && guard.secondControlPlaneRisk) {
|
||||
blockers.push({ scope: "second-hwlab-dev-control-plane", summary: "Bare kubectl can observe hwlab-dev through a different control plane; refusing write-path planning." });
|
||||
}
|
||||
}
|
||||
if ((action === "preflight" || action === "apply") && lock?.held === true) {
|
||||
blockers.push({ scope: "cd-lock", summary: `HWLAB DEV CD lock is held by ${lock.ownerTaskId || lock.holderIdentity || "unknown"}.` });
|
||||
}
|
||||
if (secretRefs?.blockers) blockers.push(...secretRefs.blockers);
|
||||
if (controlled) {
|
||||
if (controlled.commandOk === false) blockers.push({ scope: "controlled-dev-cd", summary: "HWLAB scripts/dev-cd-apply.mjs did not complete successfully." });
|
||||
if (controlled.status === "blocked") blockers.push({ scope: "controlled-dev-cd-status", summary: "HWLAB scripts/dev-cd-apply.mjs reported blocked status." });
|
||||
}
|
||||
return blockers;
|
||||
}
|
||||
|
||||
function nextSafeCommand(action, blockers) {
|
||||
if (blockers.length > 0) return "resolve the structured blockers, then rerun bun scripts/cli.ts hwlab cd status --env dev";
|
||||
if (action === "status") return "bun scripts/cli.ts hwlab cd apply --env dev --dry-run";
|
||||
return "host commander or the unique CD runner may decide whether to run node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report on D601; this wrapper did not apply or rollout";
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const options = decodeOptions();
|
||||
const timeoutMs = Math.min(Number(options.timeoutMs || 45000), 60000);
|
||||
const runId = options.runId || `${new Date().toISOString().replace(/[:.]/g, "-")}-${process.pid}`;
|
||||
const dumpDir = path.join(os.homedir(), ".state", "unidesk-hwlab-cd", runId);
|
||||
await fsp.mkdir(dumpDir, { recursive: true, mode: 0o700 });
|
||||
|
||||
const repo = resolveRepo(options.repoPath || null);
|
||||
const action = options.action;
|
||||
const base = {
|
||||
ok: false,
|
||||
env: "dev",
|
||||
environment: "dev",
|
||||
action,
|
||||
dryRun: action === "apply" ? Boolean(options.dryRun) : action !== "status",
|
||||
mutation: false,
|
||||
remoteHost: "D601",
|
||||
workspace: repo.status === "selected" ? { path: repo.path, source: repo.source } : null,
|
||||
kubeconfig: { source: "explicit", path: options.kubeconfig || nativeKubeconfig, required: nativeKubeconfig },
|
||||
dumpPath: dumpDir,
|
||||
safety: {
|
||||
wrapperCallsOnlyRepoOwnedDevCdApply: true,
|
||||
deployJsonAuthoritative: "deploy/deploy.json",
|
||||
kubectlApplyExecuted: false,
|
||||
rolloutExecuted: false,
|
||||
liveVerifyExecuted: false,
|
||||
secretValuesRead: false,
|
||||
secretValuesPrinted: false,
|
||||
cdLockMutated: false,
|
||||
prodTouched: false,
|
||||
},
|
||||
};
|
||||
|
||||
if (repo.status !== "selected") {
|
||||
const blockers = collectBlockers({ repo, action });
|
||||
console.log(JSON.stringify({ ...base, status: "blocked", error: "hwlab-repo-not-found", repo, blockers, nextSafeCommand: nextSafeCommand(action, blockers) }, null, 2));
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
const repoPath = repo.path;
|
||||
process.chdir(repoPath);
|
||||
const [git, guard] = await Promise.all([
|
||||
gitSummary(repoPath, dumpDir, Math.min(timeoutMs, 15000)),
|
||||
nodeGuard(options.kubeconfig || nativeKubeconfig, dumpDir, Math.min(timeoutMs, 15000)),
|
||||
]);
|
||||
const desiredState = {
|
||||
deployJson: readJsonSummary(repoPath, "deploy/deploy.json"),
|
||||
artifactCatalog: readJsonSummary(repoPath, "deploy/artifact-catalog.dev.json"),
|
||||
artifactReport: readJsonSummary(repoPath, "reports/dev-gate/dev-artifacts.json"),
|
||||
};
|
||||
const lock = await lockStatus(options.kubeconfig || nativeKubeconfig, guard, dumpDir, Math.min(timeoutMs, 15000));
|
||||
const needsSecretPreflight = action === "preflight" || action === "apply";
|
||||
const secretRefs = needsSecretPreflight
|
||||
? await secretPreflight(options.kubeconfig || nativeKubeconfig, guard, dumpDir, Math.min(timeoutMs, 15000))
|
||||
: { status: "skipped", reason: "status-does-not-run-secret-preflight", namespace, mutation: false, blockers: [] };
|
||||
const preCommandBlockers = collectBlockers({ repo, git, guard, lock, secretRefs, action });
|
||||
const canRunControlled = preCommandBlockers.length === 0 || action === "status";
|
||||
const controlled = canRunControlled && git.status === "pass" && guard.status === "pass"
|
||||
? await runDevCdApply(repoPath, options.kubeconfig || nativeKubeconfig, action === "status" ? "status" : "apply", dumpDir, Math.max(1000, timeoutMs - 5000))
|
||||
: { status: "skipped", commandOk: true, reason: "preflight-blockers-before-controlled-dev-cd-apply" };
|
||||
const blockers = collectBlockers({ repo, git, guard, lock, secretRefs, controlled, action });
|
||||
const ok = blockers.length === 0;
|
||||
const target = controlled.target || {
|
||||
ref: "deploy/deploy.json",
|
||||
promotionCommit: desiredState.deployJson.commitId || git.headCommit,
|
||||
shortCommitId: desiredState.deployJson.commitId || null,
|
||||
promotionSource: "deploy/deploy.json",
|
||||
namespace,
|
||||
};
|
||||
console.log(JSON.stringify({
|
||||
...base,
|
||||
ok,
|
||||
status: ok ? action === "status" ? "ready" : "prepared" : guard.refusal ? "refused" : "blocked",
|
||||
repo,
|
||||
workspace: { path: repoPath, source: repo.source },
|
||||
worktreeGuard: git,
|
||||
nodeGuard: guard,
|
||||
secretPreflight: secretRefs,
|
||||
lockState: lock,
|
||||
desiredState,
|
||||
target,
|
||||
promotion: {
|
||||
source: "deploy/deploy.json",
|
||||
deployJson: desiredState.deployJson,
|
||||
artifactCatalog: desiredState.artifactCatalog,
|
||||
artifactReport: desiredState.artifactReport,
|
||||
},
|
||||
controlledDevCd: controlled,
|
||||
reportDumpPath: controlled.command?.dump?.stdout || dumpDir,
|
||||
blockers,
|
||||
nextSafeCommand: nextSafeCommand(action, blockers),
|
||||
}, null, 2));
|
||||
process.exitCode = ok ? 0 : 1;
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
console.log(JSON.stringify({ ok: false, env: "dev", status: "blocked", error: "remote-wrapper-exception", message: redact(message) }, null, 2));
|
||||
process.exitCode = 1;
|
||||
});
|
||||
+332
-948
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user