diff --git a/.agents/skills/unidesk-cicd/SKILL.md b/.agents/skills/unidesk-cicd/SKILL.md index 54cec7ba..27503802 100644 --- a/.agents/skills/unidesk-cicd/SKILL.md +++ b/.agents/skills/unidesk-cicd/SKILL.md @@ -27,6 +27,9 @@ bun scripts/cli.ts platform-infra pipelines-as-code history --target JD01 --limi bun scripts/cli.ts platform-infra pipelines-as-code history --target JD01 --id bun scripts/cli.ts platform-infra pipelines-as-code debug-step --target JD01 --consumer bun scripts/cli.ts platform-infra pipelines-as-code debug-step --target JD01 --consumer --id +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact check --target NC01 --consumer --source-worktree +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact write --target NC01 --consumer --source-worktree --confirm +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact verify-runtime --target NC01 --consumer --source-worktree --source-commit ``` 节点级只读状态必须优先用 `cicd status --node `。它从 `config/platform-infra/pipelines-as-code.yaml` 找到该 node 的所有当前 PaC consumer，一次性汇总 PipelineRun、Argo/GitOps、runtime readiness 和诊断；不要再靠阅读源码或手动拼三条 consumer 命令来回答 “NC01 的 CI/CD 流水线情况”。`platform-infra pipelines-as-code status --target --consumer ` 只作为单 consumer drill-down。 @@ -48,6 +51,15 @@ bun scripts/cli.ts platform-infra pipelines-as-code debug-step --target JD01 --c - CI/CD source authority 只能来自 YAML 声明的 Kubernetes 托管 source authority：legacy lane 使用 k8s git-mirror snapshot，Gitea/PaC migrated lane 使用 GitHub PR merge -> GitHub webhook bridge -> Gitea controlled mirror + immutable snapshot ref -> Pipelines-as-Code。受控命令先在 k8s 内同步/创建不可变 `refs/unidesk/snapshots/.../` stage ref；build/status/publish 只消费该 snapshot，host worktree、本地 `git fetch/pull`、可变 branch ref 或 Pipeline 内直连 GitHub 都不能作为 authoritative source。 - JD01/NC01 `agentrun--v02`、`sentinel--v03`、`hwlab--v03` 的正式 CI/CD closeout 入口是 `cicd status --node ` 和 `platform-infra pipelines-as-code closeout|status|history --target --consumer `。`closeout` 是通用 consumer 引导入口，只读取/等待 PaC、Tekton、GitOps、Argo 和 runtime 对齐，不触发旧手动发布。`cicd branch-follower` 和 `cicd gitea-actions-poc` 对这些 consumer 只保留历史/迁移只读用途，不得作为当前交付判断入口。 - PaC `.tekton` 文件必须用 Repository CR 的 target/node 参数隔离 JD01/NC01，避免同一个 Gitea push 在一个 target cluster 内额外创建另一个 target 的 PipelineRun；history/detail 必须按实际 PipelineRun prefix/pipeline 归属 consumer。 +- PaC source artifact 必须遵循以下边界： + - 由 `config/platform-infra/pipelines-as-code.yaml#consumers[].sourceArtifact` 显式声明； + - 通过 `source-artifact plan|check|write|status|verify-runtime` 管理； + - `taskRunTemplate` 等运行事实归属 owning YAML，代码只负责校验和渲染； + - desired 只来自 owning YAML 与共享 renderer，禁止从 live Pipeline/PipelineRun 反向导出； + - `verify-runtime` 使用干净且 `HEAD` 等于完整 source commit 的独立 worktree； + - 未声明 owner、worktree 证据不成立或运行面证据冲突时 fail-closed； + - 错误和 drift 在默认、`--json`、`--full` 中都只能披露路径、类型、长度、SHA-256 或 URL fingerprint； + - 禁止回显脚本、URL 凭据、查询参数、Authorization、Secret 和远端 stderr。 - PaC source-only closeout 必须使用目标侧结构化证据： - `g14-ci-plan` 的 source commit、affected/build/rollout 计数必须完整； - `collect-artifacts` 与 `gitops-promote` 必须同时明确 `no-build-no-rollout-plan`； diff --git a/.agents/skills/unidesk-cicd/references/gitea-pac.md b/.agents/skills/unidesk-cicd/references/gitea-pac.md index a2acc7ad..ef091b84 100644 --- a/.agents/skills/unidesk-cicd/references/gitea-pac.md +++ b/.agents/skills/unidesk-cicd/references/gitea-pac.md @@ -18,6 +18,42 @@ GitHub remains the upstream write authority. A delivery should be triggered by m - PaC Repository CR `spec.url` matches the URL in Gitea webhook payloads. Keep `config/platform-infra/pipelines-as-code.yaml#repositories[].url` on the public Gitea repository URL, and keep k8s-internal service URLs only in `cloneUrl` or `params.git_read_url`; otherwise PaC logs `cannot find a repository match` and no PipelineRun is created. - Repositories shared by JD01 and NC01 must pass a target-specific `node` Repository param, and each `.tekton` PipelineRun must filter on that param with `pipelinesascode.tekton.dev/on-cel-expression`. A push for one target must not create the other target's PipelineRun in the same target cluster. +## PaC 源码侧制品同步 + +当 owning Pipeline 与消费仓 `.tekton` 内联 `pipelineSpec` 或远程 Pipeline 文件存在漂移时，统一使用以下入口： + +```bash +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact plan --target --consumer --source-worktree +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact check --target --consumer --source-worktree +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact write --target --consumer --source-worktree --confirm +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact status --target --consumer --source-worktree [--source-commit ] +bun scripts/cli.ts platform-infra pipelines-as-code source-artifact verify-runtime --target --consumer --source-worktree --source-commit +``` + +- `config/platform-infra/pipelines-as-code.yaml#consumers[].sourceArtifact` 是生成职责的声明入口；未声明的 consumer 必须稳定拒绝，模板不得替它发明 owner。 +- desired 只来自 owning YAML 与共享 domain renderer；live Pipeline 和 PipelineRun 只能作为只读诊断证据，禁止从运行面反向导出 desired 或源码制品。 +- `plan`、`check`、`write` 只比较或更新显式消费仓 worktree，不访问运行面；worktree 必须是绝对路径、Git 根目录，并与声明仓库的精确 remote identity 一致。 +- `write` 先完成全部路径、内容和 renderer 合同校验，再通过同目录临时文件与 rename 更新；连续执行必须无差异。 +- `status` 是非门禁诊断，允许不传 commit。 +- `verify-runtime` 必须满足以下条件： + - 传入完整四十位 `--source-commit`； + - `--source-worktree` 干净； + - worktree `HEAD` 与该 commit 完全相同； + - 合并提交的验收新建精确提交 worktree，不使用带未提交修改或停在父提交的开发 worktree； + - PipelineRun 按 PaC/source-commit label 或 annotation 精确筛选，不按 prefix 直接取全局 latest。 +- 同一 source commit 因 webhook 重投或 PaC retry 产生多个 PipelineRun 时，只在完整内联 spec 与 provenance 全部一致时确定性选择最新一条并披露候选数；存在冲突必须 fail-closed。 +- runtime observer 必须满足以下约束： + - 区分对象 `NotFound`、transport/RBAC/命令不可用和 spec drift； + - 完整大 spec 通过受控临时文件传入目标侧原生 observer，不放入 shell argv 或环境变量； + - 默认、`--json` 和 `--full` 的错误与 drift 只输出结构路径、类型、长度、SHA-256 或 URL fingerprint； + - 不回显脚本、URL 凭据、查询参数、Authorization、Secret、stdout 或 stderr 原文。 +- 完整 spec 比较只移除六类已验证的 Tekton admission default，并限定在裸 Pipeline spec、完整 Pipeline 或 PipelineRun `pipelineSpec` 三个明确根；其他同名字段不得过滤。 +- `sourceArtifact.taskRunTemplate` 是 `hostNetwork`、`dnsPolicy` 和 `fsGroup` 的唯一所有者；生成器不得另设硬编码默认值。 +- HWLAB source artifact 必须经过完整 domain renderer、正式 postprocess/verify 和完整 spec 比较。 + - Kafka refresh 等业务合同由 owning renderer 与消费者结构化负例测试保障； + - 不得在通用生成器中用字符串 marker 扫描建立第二真相。 +- AgentRun source artifact 通过完整 owning Pipeline 与完整 spec 比较保留 RBAC、参数和 manager 环境合同。 + ## Coverage Matrix | Consumer | Source | PaC namespace | Pipeline | Argo application | Status | diff --git a/config/agentrun.yaml b/config/agentrun.yaml index 7ead6dec..ae60ce2a 100644 --- a/config/agentrun.yaml +++ b/config/agentrun.yaml @@ -454,6 +454,7 @@ controlPlane: source: statusMode: k3s-git-mirror repository: pikasTech/agentrun + worktreeRemote: git@github.com:pikasTech/agentrun.git branch: v0.2 sourceAuthority: mode: gitMirrorSnapshot diff --git a/config/platform-infra/pipelines-as-code.yaml b/config/platform-infra/pipelines-as-code.yaml index 74206148..3ad13012 100644 --- a/config/platform-infra/pipelines-as-code.yaml +++ b/config/platform-infra/pipelines-as-code.yaml @@ -104,6 +104,16 @@ consumers: variables: NODE: NC01 LANE: v02 + sourceArtifact: + mode: embedded-pipeline-spec + renderer: agentrun-control-plane + configRef: config/agentrun.yaml#controlPlane.lanes.nc01-v02 + pipelineRunPath: .tekton/agentrun-nc01-v02.yaml + maxKeepRuns: 8 + taskRunTemplate: + hostNetwork: true + dnsPolicy: ClusterFirstWithHostNet + fsGroup: 1000 - extends: templates.consumers.sentinelV03 variables: NODE: JD01 @@ -138,6 +148,17 @@ consumers: variables: NODE: NC01 LANE: v03 + sourceArtifact: + mode: remote-pipeline-annotation + renderer: hwlab-runtime-lane + configRef: config/hwlab-node-lanes.yaml#lanes.v03.targets.NC01 + pipelinePath: ci/pipelines/hwlab-nc01-v03-ci-image-publish.yaml + pipelineRunPath: .tekton/hwlab-nc01-v03-pac.yaml + maxKeepRuns: 8 + taskRunTemplate: + hostNetwork: true + dnsPolicy: ClusterFirstWithHostNet + fsGroup: 1000 templates: repositories: agentrunV02: @@ -212,6 +233,9 @@ templates: pipeline_name: "hwlab-${nodeLower}-v03-ci-image-publish" pipeline_run_prefix: "hwlab-${nodeLower}-v03-ci-poll" service_account: "hwlab-${nodeLower}-v03-tekton-runner" + workspace_pvc_size: 8Gi + git_ssh_secret: hwlab-git-ssh + pipeline_timeout: 1h0m0s node: "${NODE}" lane: v03 runtime_namespace: hwlab-v03 diff --git a/docs/MDTODO/details/yaml-owned-pac-source-artifacts/R1.1_Task_Report.md b/docs/MDTODO/details/yaml-owned-pac-source-artifacts/R1.1_Task_Report.md new file mode 100644 index 00000000..06c3d4a0 --- /dev/null +++ b/docs/MDTODO/details/yaml-owned-pac-source-artifacts/R1.1_Task_Report.md @@ -0,0 +1,22 @@ +# R1.1 任务报告 + +关联问题：[UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745)。 + +## 完成内容 + +- 在 `config/platform-infra/pipelines-as-code.yaml` 为 NC01 AgentRun 与 HWLAB consumer 声明独立 `sourceArtifact` owner；未声明的 JD01 consumer 保持 fail-closed。 +- 实现 `plan`、`check`、`write`、`status`、`verify-runtime` 五个受控动作，要求显式 target、consumer 和绝对 worktree。 +- AgentRun 复用正式 control-plane Pipeline renderer 生成内联 `pipelineSpec`；HWLAB 复用正式 runtime lane renderer、deploy overlay、postprocess 和 verify 生成远程 Pipeline。 +- owning Pipeline、源码侧制品和运行面共用 config ref、effective config hash、renderer 与 mode provenance。 +- runtime verify 强制绑定完整 source commit，并精确筛选同 commit PipelineRun；重复交付仅在 spec 与 provenance 一致时确定性选取，冲突 fail-closed。 +- 远端 observer 已抽到原生 `.mjs`；完整大 spec 通过目标侧临时文件传递，避免 shell argv/env 的 `ARG_MAX`。 + +## 验证证据 + +- AgentRun `plan` 成功识别旧内联 spec 漂移。 +- HWLAB `plan` 与 `check` 稳定得到 desired `3fb6e00ba833`、source `7d51de639afa`，且不写消费仓。 +- HWLAB e76 source commit 的真实 `status` 精确返回 live `3fb6e00ba833`、embedded `7d51de639afa` 和单一 PipelineRun；`verify-runtime` 对旧 embedded drift 以失败退出。 + +## 未包含范围 + +- 本项不更新消费仓 artifact，不触发 rollout，也不从 live 反向导出 desired。 diff --git a/docs/MDTODO/details/yaml-owned-pac-source-artifacts/R1.2_Task_Report.md b/docs/MDTODO/details/yaml-owned-pac-source-artifacts/R1.2_Task_Report.md new file mode 100644 index 00000000..f3881f0f --- /dev/null +++ b/docs/MDTODO/details/yaml-owned-pac-source-artifacts/R1.2_Task_Report.md @@ -0,0 +1,23 @@ +# R1.2 任务报告 + +关联问题：[UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745)。 + +## 完成内容 + +- 完整比较 Pipeline spec，并输出 canonical hash、provenance 与首个漂移路径，不输出完整 spec 或 Secret。 +- canonicalizer 只移除六类已验证 Tekton admission default，并严格限定三个合法 spec 根。 +- source worktree 校验绝对路径、Git 根、精确 remote identity 和 symlink 边界；写入采用同目录临时文件与 rename，重复写入无变化。 +- 默认 validation error 为有界三行文本；显式 JSON/full 才保留结构化诊断。 +- 外层 CLI 的 `source-artifact --help` 与 ` --help` 均返回专项帮助。 +- `$unidesk-cicd` 的 PaC reference 已记录 desired authority、五动作、exact-commit 门禁、大 spec transport 和 fail-closed 规则。 + +## 测试证据 + +- 新增定向测试共十三项、七十二个断言，全部通过。 +- 覆盖六类 admission default 正向与同名字段负向、本地/远端 canonicalizer 一致、同 commit retry/conflict、NotFound/transport 区分、未声明 JD01 owner、错误脱敏、共享 HWLAB overlay 等价性。 +- 使用大于当前约 203 KiB HWLAB Pipeline 的 payload 实跑原生 observer 临时文件 transport，live 与 embedded 均完成完整 hash 比较。 +- HWLAB postprocess、verify 与六个 Kafka refresh marker 逐项删除均能触发 fail-closed。 + +## 基线说明 + +- 额外运行历史 `scripts/src/agentrun.test.ts` 时出现五个既有失败，集中在旧测试 fixture 的 `version` 与已迁移路由断言；本分支未修改这些测试或 AgentRun client parser。 diff --git a/docs/MDTODO/yaml-owned-pac-source-artifacts.md b/docs/MDTODO/yaml-owned-pac-source-artifacts.md new file mode 100644 index 00000000..ea27e0a7 --- /dev/null +++ b/docs/MDTODO/yaml-owned-pac-source-artifacts.md @@ -0,0 +1,22 @@ +# YAML owning Pipeline 到 PaC source artifact 正规生成 + +- 来源：[UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745) +- 关联故障：[UniDesk #1726](https://github.com/pikasTech/unidesk/issues/1726) +- 范围：统一连接 owning YAML/domain renderer 与 consumer 声明的 source artifact；同时支持 PipelineRun inline `pipelineSpec` 和 `.tekton` annotation 引用 remote Pipeline 两种载体，禁止一次性手工 patch。 + + +## R1 [in_progress] + +落实 [UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745)：建立 owning YAML/domain renderer 到 consumer-declared PaC source artifact 的生成、校验、部署和验收闭环，完成任务后将详细报告写入[任务报告](./details/yaml-owned-pac-source-artifacts/R1_Task_Report.md)。 +### R1.1 [completed] + +基于 [UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745) 实现 target/consumer 隔离的 source artifact schema、inline pipelineSpec/remote Pipeline renderer 与 plan/check/write/status/verify-runtime CLI，完成任务后将详细报告写入[任务报告](./details/yaml-owned-pac-source-artifacts/R1.1_Task_Report.md)。 +### R1.2 [completed] + +为 [UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745) 补齐完整 spec、provenance、canonical hash、first-drift、身份边界、脱敏输出测试和长期文档，完成任务后将详细报告写入[任务报告](./details/yaml-owned-pac-source-artifacts/R1.2_Task_Report.md)。 +### R1.3 [in_progress] + +落实 [UniDesk #1745](https://github.com/pikasTech/unidesk/issues/1745)：使用生成器更新 AgentRun source artifact，并向 HWLAB 消费侧交付 remote Pipeline 接口与 fixture；经受控 PR/preflight 交付，不做手工 env/base64 patch，完成任务后将详细报告写入[任务报告](./details/yaml-owned-pac-source-artifacts/R1.3_Task_Report.md)。 +### R1.4 + +协同 rollout owner 在 NC01 验证 runtime embedded spec、manager Healthy、stale-pending 周期回收及 dsflash Secret 未越权，并回写 [UniDesk #1726](https://github.com/pikasTech/unidesk/issues/1726) 与 #1745，完成任务后将详细报告写入[任务报告](./details/yaml-owned-pac-source-artifacts/R1.4_Task_Report.md)。 diff --git a/scripts/native/cicd/pac-source-artifact-runtime.mjs b/scripts/native/cicd/pac-source-artifact-runtime.mjs new file mode 100644 index 00000000..50e003a0 --- /dev/null +++ b/scripts/native/cicd/pac-source-artifact-runtime.mjs @@ -0,0 +1,348 @@ +import { createHash } from "node:crypto"; +import { execFileSync } from "node:child_process"; +import { readFileSync } from "node:fs"; + +const provenanceKeys = { + configRef: "unidesk.ai/owning-config-ref", + effectiveConfigSha256: "unidesk.ai/effective-config-sha256", + renderer: "unidesk.ai/source-artifact-renderer", + mode: "unidesk.ai/source-artifact-mode", +}; + +const sourceCommitKeys = [ + "pipelinesascode.tekton.dev/sha", + "pipelinesascode.tekton.dev/commit", + "agentrun.pikastech.local/source-commit", + "unidesk.ai/source-commit", + "hwlab.pikastech.local/source-commit", +]; + +const knownDriftPathKeys = new Set([ + "accessModes", "annotations", "apiVersion", "args", "command", "computeResources", "default", "description", "dnsPolicy", "env", "envFrom", + "executionMode", "finally", "fsGroup", "generateName", "hostNetwork", "image", "kind", "labels", "length", "metadata", "mode", "mountPath", "name", "namespace", "onError", "operator", + "optional", "params", "pipeline", "pipelineRef", "pipelineSpec", "podTemplate", "readinessProbe", "readOnly", "requests", "resources", + "results", "retries", "runAfter", "script", "secret", "secretName", "securityContext", "serviceAccountName", "sidecars", "spec", "steps", + "storage", "subPath", "taskRunTemplate", "taskSpec", "tasks", "timeout", "timeouts", "type", "value", "values", "volumeClaimTemplate", + "volumeMounts", "volumes", "when", "workspaces", "workingDir", "wrapper", +]); + +function record(value) { + return value !== null && typeof value === "object" && !Array.isArray(value) ? value : {}; +} + +function stableValue(value) { + if (Array.isArray(value)) return value.map(stableValue); + if (value !== null && typeof value === "object") { + return Object.fromEntries(Object.keys(value).sort().map((key) => [key, stableValue(value[key])])); + } + if (value === undefined) return { __type: "undefined" }; + if (typeof value === "bigint") return { __type: "bigint", value: String(value) }; + return value; +} + +function valueEvidence(value) { + const type = value === null ? "null" : Array.isArray(value) ? "array" : typeof value; + const serialized = JSON.stringify(stableValue(value)); + const text = serialized === undefined ? String(value) : serialized; + const length = typeof value === "string" + ? Buffer.byteLength(value, "utf8") + : Array.isArray(value) + ? value.length + : value !== null && typeof value === "object" + ? Object.keys(value).length + : Buffer.byteLength(text, "utf8"); + const sha256 = createHash("sha256").update(text).digest("hex"); + return `type=${type},length=${length},sha256=${sha256}`; +} + +function safeRuntimeReason(reason) { + const text = typeof reason === "string" ? reason : String(reason); + const fixed = new Set([ + "source-commit-not-specified", + "source-commit-run-not-found", + "source-commit-identity-conflict", + "pipeline-get-not-found", + "pipelinerun-list-not-found", + "pipelinerun-list-invalid-shape", + ]); + if (fixed.has(text)) return text; + if (/^source-commit-run-conflict:\d+$/u.test(text)) return text; + if (/^(?:pipeline-get|pipelinerun-list)-command-failed:(?:unknown|\d+):type=string,length=\d+,sha256=[0-9a-f]{64}$/u.test(text)) return text; + if (/^runtime-observer-input-error:type=string,length=\d+,sha256=[0-9a-f]{64}$/u.test(text)) return text; + return `runtime-observer-reason:${valueEvidence(text)}`; +} + +function unavailable(reason, sourceCommit = null, candidateCount = 0) { + return { + status: "unavailable", + name: null, + canonicalSha256: null, + firstDrift: null, + provenanceAligned: false, + configRef: null, + effectiveConfigSha256: null, + sourceCommit, + reason: safeRuntimeReason(reason), + candidateCount, + }; +} + +function missing(reason, sourceCommit = null) { + return { ...unavailable(reason, sourceCommit, 0), status: "missing" }; +} + +function isAdmissionDefault(path, value) { + const normalized = path.map((segment) => typeof segment === "number" ? "*" : segment).join("."); + const emptyRecord = value !== null && typeof value === "object" && !Array.isArray(value) && Object.keys(value).length === 0; + const atTaskSpec = (suffix) => [ + `tasks.*.taskSpec.${suffix}`, + `spec.tasks.*.taskSpec.${suffix}`, + `spec.pipelineSpec.tasks.*.taskSpec.${suffix}`, + ].includes(normalized); + if (atTaskSpec("metadata")) return emptyRecord; + if (atTaskSpec("spec")) return value === null; + if (atTaskSpec("params.*.type")) return value === "string"; + if (atTaskSpec("results.*.type")) return value === "string"; + if (atTaskSpec("steps.*.computeResources")) return emptyRecord; + if (atTaskSpec("sidecars.*.computeResources")) return emptyRecord; + return false; +} + +export function canonicalizePacPipelineSpec(value, path = []) { + if (Array.isArray(value)) return value.map((item, index) => canonicalizePacPipelineSpec(item, [...path, index])); + if (value === null || typeof value !== "object") return value; + const output = {}; + for (const key of Object.keys(value).sort()) { + const child = value[key]; + const childPath = [...path, key]; + if (isAdmissionDefault(childPath, child)) continue; + output[key] = canonicalizePacPipelineSpec(child, childPath); + } + return output; +} + +export function canonicalPacPipelineSha256(value) { + return `sha256:${createHash("sha256").update(JSON.stringify(canonicalizePacPipelineSpec(value))).digest("hex")}`; +} + +function firstCanonicalDrift(expected, actual, path = "$") { + if (Object.is(expected, actual)) return null; + if (Array.isArray(expected) || Array.isArray(actual)) { + if (!Array.isArray(expected) || !Array.isArray(actual)) return { path, expected: valueEvidence(expected), actual: valueEvidence(actual) }; + if (expected.length !== actual.length) return { path: `${path}.length`, expected: valueEvidence(expected.length), actual: valueEvidence(actual.length) }; + for (let index = 0; index < expected.length; index += 1) { + const drift = firstCanonicalDrift(expected[index], actual[index], `${path}[${index}]`); + if (drift !== null) return drift; + } + return null; + } + const expectedRecord = expected !== null && typeof expected === "object"; + const actualRecord = actual !== null && typeof actual === "object"; + if (expectedRecord || actualRecord) { + if (!expectedRecord || !actualRecord) return { path, expected: valueEvidence(expected), actual: valueEvidence(actual) }; + const keys = [...new Set([...Object.keys(expected), ...Object.keys(actual)])].sort(); + for (const key of keys) { + const expectedOwnsKey = Object.prototype.hasOwnProperty.call(expected, key); + const actualOwnsKey = Object.prototype.hasOwnProperty.call(actual, key); + const childPath = knownDriftPathKeys.has(key) ? `${path}.${key}` : `${path}.[key:${valueEvidence(key)}]`; + if (!expectedOwnsKey || !actualOwnsKey) { + return { path: childPath, expected: valueEvidence(expected[key]), actual: valueEvidence(actual[key]) }; + } + const drift = firstCanonicalDrift(expected[key], actual[key], childPath); + if (drift !== null) return drift; + } + return null; + } + return { path, expected: valueEvidence(expected), actual: valueEvidence(actual) }; +} + +function firstDrift(expected, actual) { + return firstCanonicalDrift(canonicalizePacPipelineSpec(expected), canonicalizePacPipelineSpec(actual)); +} + +function kubectlJson(args, label) { + try { + return { + kind: "ok", + value: JSON.parse(execFileSync("kubectl", args, { encoding: "utf8", timeout: 30_000, maxBuffer: 64 * 1024 * 1024 })), + }; + } catch (error) { + const stderr = typeof error?.stderr === "string" ? error.stderr : Buffer.isBuffer(error?.stderr) ? error.stderr.toString("utf8") : ""; + const stdout = typeof error?.stdout === "string" ? error.stdout : Buffer.isBuffer(error?.stdout) ? error.stdout.toString("utf8") : ""; + const message = stderr.trim().split(/\r?\n/u)[0] || stdout.trim().split(/\r?\n/u)[0] || String(error?.message ?? "command failed"); + if (/\bNotFound\b|\bnot found\b/u.test(message)) return { kind: "not-found", reason: `${label}-not-found` }; + const status = Number.isInteger(error?.status) ? error.status : "unknown"; + return { kind: "unavailable", reason: `${label}-command-failed:${status}:${valueEvidence(message)}` }; + } +} + +function manifestProvenance(manifest) { + const annotations = record(record(manifest?.metadata).annotations); + return { + configRef: typeof annotations[provenanceKeys.configRef] === "string" ? annotations[provenanceKeys.configRef] : null, + effectiveConfigSha256: typeof annotations[provenanceKeys.effectiveConfigSha256] === "string" ? annotations[provenanceKeys.effectiveConfigSha256] : null, + renderer: typeof annotations[provenanceKeys.renderer] === "string" ? annotations[provenanceKeys.renderer] : null, + mode: typeof annotations[provenanceKeys.mode] === "string" ? annotations[provenanceKeys.mode] : null, + }; +} + +function manifestSourceCommits(manifest) { + const metadata = record(manifest?.metadata); + const labels = record(metadata.labels); + const annotations = record(metadata.annotations); + return [...new Set(sourceCommitKeys.flatMap((key) => [labels[key], annotations[key]]) + .filter((value) => typeof value === "string" && /^[0-9a-f]{40}$/iu.test(value)) + .map((value) => value.toLowerCase()))]; +} + +function safeManifestName(value) { + return typeof value === "string" + && value.length <= 253 + && /^[a-z0-9](?:[-a-z0-9.]*[a-z0-9])?$/u.test(value) + ? value + : null; +} + +function safeAlignedConfigRef(observed, expected) { + return observed === expected + && typeof expected === "string" + && expected.length <= 512 + && /^[A-Za-z0-9_./-]+#[A-Za-z0-9_.-]+$/u.test(expected) + ? expected + : null; +} + +function safeAlignedSha256(observed, expected) { + return observed === expected && typeof expected === "string" && /^sha256:[0-9a-f]{64}$/u.test(expected) ? expected : null; +} + +function pipelineRunWrapper(manifest) { + const spec = record(manifest?.spec); + const hasPipelineSpec = Object.prototype.hasOwnProperty.call(spec, "pipelineSpec"); + const hasPipelineRef = Object.prototype.hasOwnProperty.call(spec, "pipelineRef"); + const observedMode = manifestProvenance(manifest).mode; + const mode = observedMode === "embedded-pipeline-spec" || observedMode === "remote-pipeline-annotation" ? observedMode : "invalid"; + const executionMode = hasPipelineSpec && !hasPipelineRef ? "pipeline-spec" : "invalid"; + const wrapperSpec = { ...spec }; + delete wrapperSpec.pipelineSpec; + delete wrapperSpec.pipelineRef; + return { mode, executionMode, spec: wrapperSpec }; +} + +function observedItem(input, manifest, spec, sourceCommit = null, candidateCount = 0, wrapper = null) { + const observedProvenance = manifestProvenance(manifest); + const provenanceAligned = observedProvenance.configRef === input.provenance.configRef + && observedProvenance.effectiveConfigSha256 === input.provenance.effectiveConfigSha256 + && observedProvenance.renderer === input.provenance.renderer + && observedProvenance.mode === input.provenance.mode; + const comparePipelineSpec = wrapper === null || wrapper.executionMode === "pipeline-spec"; + const specDrift = comparePipelineSpec ? firstDrift(input.desiredSpec, spec) : null; + const wrapperDrift = wrapper === null || input.desiredWrapper === null + ? null + : firstDrift(input.desiredWrapper, wrapper); + const drift = specDrift ?? wrapperDrift; + return { + status: drift === null ? "aligned" : "drifted", + name: safeManifestName(record(manifest.metadata).name), + canonicalSha256: canonicalPacPipelineSha256( + wrapper === null + ? spec + : wrapper.executionMode === "pipeline-spec" + ? { pipelineSpec: spec, wrapper } + : { wrapper }, + ), + firstDrift: drift, + provenanceAligned, + configRef: safeAlignedConfigRef(observedProvenance.configRef, input.provenance.configRef), + effectiveConfigSha256: safeAlignedSha256(observedProvenance.effectiveConfigSha256, input.provenance.effectiveConfigSha256), + renderer: observedProvenance.renderer === input.provenance.renderer ? input.provenance.renderer : null, + mode: observedProvenance.mode === input.provenance.mode ? input.provenance.mode : null, + sourceCommit, + reason: null, + candidateCount, + }; +} + +export function selectPipelineRunBySourceCommit(items, pipelineRunPrefix, sourceCommit) { + if (sourceCommit === null) return { kind: "unavailable", reason: "source-commit-not-specified", run: null, candidates: [] }; + const candidates = items.filter((run) => { + const name = String(record(run?.metadata).name ?? ""); + return name.startsWith(pipelineRunPrefix) && manifestSourceCommits(run).includes(sourceCommit); + }); + if (candidates.length === 0) return { kind: "missing", reason: "source-commit-run-not-found", run: null, candidates: [] }; + if (candidates.some((run) => { + const commits = manifestSourceCommits(run); + return commits.length !== 1 || commits[0] !== sourceCommit; + })) return { kind: "unavailable", reason: "source-commit-identity-conflict", run: null, candidates }; + const ordered = [...candidates].sort((left, right) => { + const time = String(record(right?.metadata).creationTimestamp ?? "").localeCompare(String(record(left?.metadata).creationTimestamp ?? "")); + return time !== 0 ? time : String(record(right?.metadata).name ?? "").localeCompare(String(record(left?.metadata).name ?? "")); + }); + return { kind: "ok", reason: null, run: ordered[0], candidates: ordered }; +} + +export function observePacSourceArtifactRuntime(input, readKubectlJson = kubectlJson) { + const liveResult = readKubectlJson(["get", "pipeline", input.pipeline, "-n", input.namespace, "-o", "json"], "pipeline-get"); + const live = liveResult.kind === "ok" + ? observedItem(input, liveResult.value, record(liveResult.value.spec)) + : liveResult.kind === "not-found" + ? missing(liveResult.reason) + : unavailable(liveResult.reason); + + let embedded; + if (input.sourceCommit === null) { + embedded = unavailable("source-commit-not-specified"); + } else { + const runsResult = readKubectlJson(["get", "pipelinerun", "-n", input.namespace, "-o", "json"], "pipelinerun-list"); + if (runsResult.kind !== "ok") { + embedded = runsResult.kind === "not-found" ? missing(runsResult.reason, input.sourceCommit) : unavailable(runsResult.reason, input.sourceCommit); + } else { + const items = Array.isArray(runsResult.value?.items) ? runsResult.value.items : null; + if (items === null) { + embedded = unavailable("pipelinerun-list-invalid-shape", input.sourceCommit); + } else { + const selected = selectPipelineRunBySourceCommit(items, input.pipelineRunPrefix, input.sourceCommit); + if (selected.kind === "ok") { + const signatures = selected.candidates.map((run) => JSON.stringify({ + canonicalSha256: (() => { + const wrapper = pipelineRunWrapper(run); + return canonicalPacPipelineSha256(wrapper.executionMode === "pipeline-spec" + ? { pipelineSpec: record(record(run.spec).pipelineSpec), wrapper } + : { wrapper }); + })(), + provenance: manifestProvenance(run), + })); + embedded = new Set(signatures).size === 1 + ? observedItem( + input, + selected.run, + record(record(selected.run.spec).pipelineSpec), + input.sourceCommit, + selected.candidates.length, + pipelineRunWrapper(selected.run), + ) + : unavailable(`source-commit-run-conflict:${selected.candidates.length}`, input.sourceCommit, selected.candidates.length); + } else { + embedded = selected.kind === "missing" + ? missing(selected.reason, input.sourceCommit) + : unavailable(selected.reason, input.sourceCommit, selected.candidates.length); + } + } + } + } + return { live, embedded }; +} + +if (typeof process.env.PAC_SOURCE_ARTIFACT_INPUT_FILE === "string") { + try { + const inputText = readFileSync(process.env.PAC_SOURCE_ARTIFACT_INPUT_FILE, "utf8"); + const input = JSON.parse(inputText); + process.stdout.write(JSON.stringify(observePacSourceArtifactRuntime(input))); + } catch (error) { + const evidence = valueEvidence(String(error?.message ?? error)); + process.stdout.write(JSON.stringify({ + live: unavailable(`runtime-observer-input-error:${evidence}`), + embedded: unavailable(`runtime-observer-input-error:${evidence}`), + })); + } +} diff --git a/scripts/src/agentrun-lanes.ts b/scripts/src/agentrun-lanes.ts index 682ee63c..05615f30 100644 --- a/scripts/src/agentrun-lanes.ts +++ b/scripts/src/agentrun-lanes.ts @@ -108,6 +108,7 @@ export interface AgentRunLaneSpec { readonly source: { readonly statusMode: "host-worktree" | "k3s-git-mirror"; readonly repository: string; + readonly worktreeRemote: string; readonly branch: string; readonly sourceAuthority: AgentRunSourceAuthoritySpec | null; readonly sourceSnapshot: AgentRunSourceSnapshotSpec | null; @@ -595,6 +596,7 @@ function parseLane(lane: string, node: AgentRunNodeSpec, input: Record[ return `sha256:${createHash("sha256").update(yamlAll(objects)).digest("hex")}`; } -function agentRunPipelineManifest(spec: AgentRunLaneSpec): Record { +export function renderAgentRunPipelineManifest(spec: AgentRunLaneSpec): Record { const build = spec.deployment.manager.imageBuild; if (spec.ci.buildkitImage === null) throw new Error(`config/agentrun.yaml controlPlane.lanes.${spec.lane}.ci.buildkitImage is required for AgentRun Tekton image build`); return { @@ -182,6 +183,12 @@ function agentRunPipelineManifest(spec: AgentRunLaneSpec): Record (await import("./cicd")).cicdHelp(), cicdHelpSummary()); if (top === "agentrun") return loadHelp(async () => (await import("./agentrun")).agentRunHelp(), agentRunHelpSummary()); + if (top === "platform-infra" && (sub === "pipelines-as-code" || sub === "pac") && args[2] === "source-artifact") return null; if (top === "platform-infra") return loadHelp(async () => (await import("./platform-infra")).platformInfraHelp(), platformInfraHelpSummary()); if (top === "platform-db") return platformDbHelp(); if (top === "secrets") return secretsHelp(); diff --git a/scripts/src/hwlab-node/deploy-overlay.ts b/scripts/src/hwlab-node/deploy-overlay.ts new file mode 100644 index 00000000..877e5758 --- /dev/null +++ b/scripts/src/hwlab-node/deploy-overlay.ts @@ -0,0 +1,66 @@ +export function applyNodeRuntimeDeployYamlOverlay(document: Record, overlay: Record): Record { + const doc = structuredClone(document); + const nodeId = requiredString(overlay.nodeId, "overlay.nodeId"); + const laneId = requiredString(overlay.lane, "overlay.lane"); + const nodes = record(doc.nodes); + const node = record(nodes[nodeId]); + nodes[nodeId] = { ...node, gitopsRoot: overlay.gitopsRoot, sourceRepo: overlay.gitUrl }; + doc.nodes = nodes; + const lanes = record(doc.lanes); + const lane = record(lanes[laneId]); + const envRecipe = record(lane.envRecipe); + const downloadStack = { + ...record(envRecipe.downloadStack), + httpProxy: overlay.dockerProxyHttp, + httpsProxy: overlay.dockerProxyHttps, + noProxy: overlay.dockerNoProxyList, + }; + const nextLane: Record = { + ...lane, + node: nodeId, + sourceBranch: overlay.sourceBranch, + gitopsBranch: overlay.gitopsBranch, + namespace: overlay.runtimeNamespace, + endpoint: overlay.publicApiUrl, + publicEndpoints: { frontend: overlay.publicWebUrl, api: overlay.publicApiUrl }, + artifactCatalog: overlay.catalogPath, + runtimePath: overlay.runtimePath, + imageTagMode: "full", + sourceRepo: overlay.gitUrl, + observability: overlay.observability, + envRecipe: { ...envRecipe, downloadStack }, + }; + if (overlay.externalPostgres === undefined || overlay.externalPostgres === null) delete nextLane.externalPostgres; + else nextLane.externalPostgres = overlay.externalPostgres; + if (overlay.runtimeStore !== undefined) nextLane.runtimeStore = overlay.runtimeStore; + if (overlay.codeAgentRuntime !== undefined) nextLane.codeAgentRuntime = overlay.codeAgentRuntime; + if (overlay.deployYamlGitMirror !== undefined) nextLane.gitMirror = overlay.deployYamlGitMirror; + lanes[laneId] = nextLane; + doc.lanes = lanes; + return doc; +} + +export function nodeRuntimeDeployYamlOverlayShellScript(): string[] { + return [ + "node - \"$overlay_b64\" <<'NODE_UNIDESK_DEPLOY_OVERLAY'", + "const fs = require('fs');", + "const YAML = require('yaml');", + `const applyNodeRuntimeDeployYamlOverlay = ${applyNodeRuntimeDeployYamlOverlay.toString()};`, + `const record = ${record.toString()};`, + `const requiredString = ${requiredString.toString()};`, + "const overlay = JSON.parse(Buffer.from(process.argv[2], 'base64').toString('utf8'));", + "const path = 'deploy/deploy.yaml';", + "const doc = YAML.parse(fs.readFileSync(path, 'utf8'));", + "fs.writeFileSync(path, YAML.stringify(applyNodeRuntimeDeployYamlOverlay(doc, overlay)));", + "NODE_UNIDESK_DEPLOY_OVERLAY", + ]; +} + +function record(value: unknown): Record { + return typeof value === "object" && value !== null && !Array.isArray(value) ? value as Record : {}; +} + +function requiredString(value: unknown, label: string): string { + if (typeof value !== "string" || value.length === 0) throw new Error(`${label} must be a non-empty string`); + return value; +} diff --git a/scripts/src/hwlab-node/render.ts b/scripts/src/hwlab-node/render.ts index 2a49b5f8..8de169af 100644 --- a/scripts/src/hwlab-node/render.ts +++ b/scripts/src/hwlab-node/render.ts @@ -40,6 +40,7 @@ import { externalPostgresBridgeStatus, externalPostgresSecretStatus, getNodeRunt import { webObserveShort, webObserveText } from "./web-probe-observe"; import { readNodeRuntimeStatusPolicy, runNodeRuntimeStatusProbe, skippedNodeRuntimeStatusCommand, type NodeRuntimeStatusPolicy, type NodeRuntimeStatusWorkerEvent } from "./control-plane-status-observed"; import { hwlabRuntimeActiveExternalPostgres } from "../hwlab-node-lanes"; +import { nodeRuntimeDeployYamlOverlayShellScript } from "./deploy-overlay"; const runtimeGitopsObservabilityNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-observability.mjs"), "utf8").trimEnd(); const runtimeGitopsPipelineGuardNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-pipeline-guard.mjs"), "utf8").trimEnd(); @@ -1468,44 +1469,7 @@ export function renderNodeRuntimeControlPlaneOnNode(spec: HwlabRuntimeLaneSpec, "git -C \"$worktree_dir\" checkout --detach \"$source_commit\"", "cd \"$worktree_dir\"", ...yamlDependencyInstallScript(spec.downloadProfile.npm.registry, spec.downloadProfile.npm.fetchTimeoutSeconds, spec.downloadProfile.npm.retries, "control-plane-render"), - "node - \"$overlay_b64\" <<'NODE'", - "const fs = require('fs');", - "const YAML = require('yaml');", - "const overlay = JSON.parse(Buffer.from(process.argv[2], 'base64').toString('utf8'));", - "const path = 'deploy/deploy.yaml';", - "const doc = YAML.parse(fs.readFileSync(path, 'utf8'));", - "doc.nodes = doc.nodes || {};", - "doc.nodes[overlay.nodeId] = { ...(doc.nodes[overlay.nodeId] || {}), gitopsRoot: overlay.gitopsRoot, sourceRepo: overlay.gitUrl };", - "doc.lanes = doc.lanes || {};", - "const lane = doc.lanes[overlay.lane] || {};", - "const downloadStack = {", - " ...(lane.envRecipe?.downloadStack || {}),", - " httpProxy: overlay.dockerProxyHttp,", - " httpsProxy: overlay.dockerProxyHttps,", - " noProxy: overlay.dockerNoProxyList,", - "};", - "doc.lanes[overlay.lane] = {", - " ...lane,", - " node: overlay.nodeId,", - " sourceBranch: overlay.sourceBranch,", - " gitopsBranch: overlay.gitopsBranch,", - " namespace: overlay.runtimeNamespace,", - " endpoint: overlay.publicApiUrl,", - " publicEndpoints: { frontend: overlay.publicWebUrl, api: overlay.publicApiUrl },", - " artifactCatalog: overlay.catalogPath,", - " runtimePath: overlay.runtimePath,", - " imageTagMode: 'full',", - " sourceRepo: overlay.gitUrl,", - " observability: overlay.observability,", - " envRecipe: { ...(lane.envRecipe || {}), downloadStack },", - "};", - "if (overlay.externalPostgres === undefined || overlay.externalPostgres === null) delete doc.lanes[overlay.lane].externalPostgres;", - "else doc.lanes[overlay.lane].externalPostgres = overlay.externalPostgres;", - "if (overlay.runtimeStore !== undefined) doc.lanes[overlay.lane].runtimeStore = overlay.runtimeStore;", - "if (overlay.codeAgentRuntime !== undefined) doc.lanes[overlay.lane].codeAgentRuntime = overlay.codeAgentRuntime;", - "if (overlay.deployYamlGitMirror !== undefined) doc.lanes[overlay.lane].gitMirror = overlay.deployYamlGitMirror;", - "fs.writeFileSync(path, YAML.stringify(doc));", - "NODE", + ...nodeRuntimeDeployYamlOverlayShellScript(), "if [ -f scripts/gitops-render.mjs ]; then render_script=scripts/gitops-render.mjs; else echo 'render script missing: scripts/gitops-render.mjs' >&2; exit 43; fi", [ "node scripts/run-bun.mjs \"$render_script\"", @@ -2557,6 +2521,11 @@ export function nodeRuntimePipelinePostprocessScript(): string[] { " doc.metadata = doc.metadata || {};", " if (typeof overlay.pipelineName === 'string' && overlay.pipelineName.length > 0) doc.metadata.name = overlay.pipelineName;", " doc.metadata.annotations = doc.metadata.annotations || {};", + " const provenance = overlay.sourceArtifactProvenance || {};", + " if (provenance.configRef) doc.metadata.annotations['unidesk.ai/owning-config-ref'] = provenance.configRef;", + " if (provenance.effectiveConfigSha256) doc.metadata.annotations['unidesk.ai/effective-config-sha256'] = provenance.effectiveConfigSha256;", + " if (provenance.renderer) doc.metadata.annotations['unidesk.ai/source-artifact-renderer'] = provenance.renderer;", + " if (provenance.mode) doc.metadata.annotations['unidesk.ai/source-artifact-mode'] = provenance.mode;", " doc.metadata.annotations['hwlab.pikastech.local/download-profile'] = overlay.downloadProfileId;", " doc.metadata.annotations['hwlab.pikastech.local/network-profile'] = overlay.networkProfileId;", " for (const task of doc.spec?.tasks || []) {", diff --git a/scripts/src/hwlab-node/web-probe.ts b/scripts/src/hwlab-node/web-probe.ts index bd12830d..ff37f7bc 100644 --- a/scripts/src/hwlab-node/web-probe.ts +++ b/scripts/src/hwlab-node/web-probe.ts @@ -40,6 +40,7 @@ import { assertKnownOptions, nodeWebProbeAutoCommandTimeoutSeconds, normalizeNod import { resolveNodeWebProbeCliOrigin } from "./web-probe-origin"; import { hwlabRuntimeActiveExternalPostgres } from "../hwlab-node-lanes"; import { resolveEgressProxySourceRef } from "../egress-proxy-sources"; +import { stableJsonSha256 } from "../stable-json"; export function nodeRuntimeRenderOverlay(spec: HwlabRuntimeLaneSpec): Record { const gitSshProxy = httpProxyEndpoint(spec.networkProfile.proxy.http); @@ -74,6 +75,12 @@ export function nodeRuntimeRenderOverlay(spec: HwlabRuntimeLaneSpec): Record { + return { + "unidesk.ai/owning-config-ref": provenance.configRef, + "unidesk.ai/effective-config-sha256": provenance.effectiveConfigSha256, + "unidesk.ai/source-artifact-renderer": provenance.renderer, + "unidesk.ai/source-artifact-mode": provenance.mode, + }; +} + +function pipeline(): Record { + return { + metadata: { name: "agentrun-nc01-v02-ci-image-publish", annotations: manifestAnnotations() }, + spec: desiredSpec, + }; +} + +function pipelineRun(name: string, creationTimestamp: string, spec: Record = desiredSpec): Record { + return { + metadata: { + name, + creationTimestamp, + labels: { "pipelinesascode.tekton.dev/sha": sourceCommit }, + annotations: manifestAnnotations(), + }, + spec: { pipelineSpec: spec, ...structuredClone(runtimeWrapperSpec) }, + }; +} + +function runtimeInput(commit: string | null = sourceCommit): Record { + return { + namespace: "agentrun-ci", + pipeline: "agentrun-nc01-v02-ci-image-publish", + pipelineRunPrefix: "agentrun-nc01-v02-ci", + desiredSpec, + desiredWrapper: { mode: "embedded-pipeline-spec", executionMode: "pipeline-spec", spec: runtimeWrapperSpec }, + provenance, + sourceCommit: commit, + }; +} + +describe("PaC source artifact CLI contract", () => { + test("verify-runtime requires and normalizes a full source commit", () => { + expect(() => parsePacSourceArtifactOptions([ + "verify-runtime", "--target", "NC01", "--consumer", "agentrun-nc01-v02", "--source-worktree", "/tmp/repo", + ])).toThrow("source-commit-required"); + const parsed = parsePacSourceArtifactOptions([ + "verify-runtime", "--target", "NC01", "--consumer", "agentrun-nc01-v02", "--source-worktree", "/tmp/repo", "--source-commit", sourceCommit.toUpperCase(), + ]); + expect(parsed.sourceCommit).toBe(sourceCommit); + expect(() => parsePacSourceArtifactOptions([ + "plan", "--target", "NC01", "--consumer", "agentrun-nc01-v02", "--source-worktree", "/tmp/repo", "--source-commit", sourceCommit, + ])).toThrow("accepted only"); + }); + + test("real outer CLI returns bounded scoped help for both help positions", () => { + for (const tail of [["--help"], ["check", "--help"]]) { + const result = spawnSync("bun", ["scripts/cli.ts", "platform-infra", "pipelines-as-code", "source-artifact", ...tail], { + cwd: rootPath(), encoding: "utf8", timeout: 30_000, + }); + expect(result.status).toBe(0); + expect(result.stdout).toContain("PAC SOURCE ARTIFACT"); + expect(result.stdout).toContain("verify-runtime"); + expect(result.stdout).toContain("--source-commit "); + expect(result.stdout.split("\n").length).toBeLessThan(25); + expect(result.stdout).not.toContain("platform-infra sub2api plan"); + } + }); + + test("predictable validation failures omit stack and rendered specs", () => { + const result = spawnSync("bun", [ + "scripts/cli.ts", "platform-infra", "pipelines-as-code", "source-artifact", "plan", + "--target", "NC01", "--consumer", "agentrun-nc01-v02", "--source-worktree", rootPath(), + ], { cwd: rootPath(), encoding: "utf8", timeout: 30_000 }); + expect(result.status).toBe(1); + expect(result.stdout).toContain("code=source-worktree-origin-mismatch"); + expect(result.stdout).toContain("observedUrlFingerprint=sha256:"); + expect(result.stdout).not.toContain("observedOwnerRepo"); + expect(result.stdout).not.toContain("stack"); + expect(result.stdout).not.toContain("taskSpec"); + expect(result.stdout.length).toBeLessThan(1500); + }); + + test("default, JSON, and full failures never disclose a malicious origin", () => { + const temporary = mkdtempSync(resolve(tmpdir(), "unidesk-pac-origin-test-")); + const secrets = [ + "review-user", + "REVIEW_SUPER_SECRET", + "QUERY_SUPER_SECRET", + "Authorization", + "https://review-user:REVIEW_SUPER_SECRET@example.invalid/pikasTech/agentrun.git?token=QUERY_SUPER_SECRET", + ]; + try { + for (const args of [ + ["init"], + ["config", "user.email", "pac-test@example.invalid"], + ["config", "user.name", "PaC Test"], + ]) expect(spawnSync("git", args, { cwd: temporary, encoding: "utf8" }).status).toBe(0); + writeFileSync(resolve(temporary, "README.md"), "fixture\n"); + expect(spawnSync("git", ["add", "README.md"], { cwd: temporary, encoding: "utf8" }).status).toBe(0); + expect(spawnSync("git", ["commit", "-m", "fixture"], { cwd: temporary, encoding: "utf8" }).status).toBe(0); + expect(spawnSync("git", ["remote", "add", "origin", secrets[4] as string], { cwd: temporary, encoding: "utf8" }).status).toBe(0); + const origins = [ + secrets[4] as string, + "audit@example.invalid:pikasTech/agentrun.git?token=SCP_QUERY_SUPER_SECRET", + "https://review-user:REVIEW_SUPER_SECRET@github.com/pikasTech/agentrun.git", + "https://github.com/pikasTech/agentrun.git?token=QUERY_SUPER_SECRET", + "https://github.com/pikasTech/agentrun.git#QUERY_SUPER_SECRET", + ]; + for (const origin of origins) { + expect(spawnSync("git", ["remote", "set-url", "origin", origin], { cwd: temporary, encoding: "utf8" }).status).toBe(0); + for (const outputFlag of [null, "--json", "--full"] as const) { + const args = [ + "scripts/cli.ts", "platform-infra", "pipelines-as-code", "source-artifact", "plan", + "--target", "NC01", "--consumer", "agentrun-nc01-v02", "--source-worktree", temporary, + ]; + if (outputFlag !== null) args.push(outputFlag); + const result = spawnSync("bun", args, { cwd: rootPath(), encoding: "utf8", timeout: 30_000 }); + const output = `${result.stdout}\n${result.stderr}`; + expect(result.status).toBe(1); + expect(output).toContain("source-worktree-origin-mismatch"); + expect(output).toContain("observedUrlFingerprint"); + expect(output).not.toContain("observedOwnerRepo"); + for (const secret of [...secrets, "SCP_QUERY_SUPER_SECRET"]) expect(output).not.toContain(secret); + } + } + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }, 15_000); + + test("verification requires a clean worktree at the exact requested commit", () => { + const exact = { head: sourceCommit, clean: true, matchesSourceCommit: true }; + expect(() => assertPacSourceArtifactVerifyWorktree("verify-runtime", sourceCommit, exact)).not.toThrow(); + expect(() => assertPacSourceArtifactVerifyWorktree("status", sourceCommit, { ...exact, clean: false })).not.toThrow(); + expect(() => assertPacSourceArtifactVerifyWorktree("verify-runtime", sourceCommit, { ...exact, clean: false })).toThrow("source-worktree-not-clean"); + expect(() => assertPacSourceArtifactVerifyWorktree("verify-runtime", sourceCommit, { ...exact, head: "b".repeat(40), matchesSourceCommit: false })).toThrow("source-worktree-commit-mismatch"); + }); + + test("taskRunTemplate operational facts are declared in owning YAML", () => { + const document = Bun.YAML.parse(readFileSync(rootPath("config", "platform-infra", "pipelines-as-code.yaml"), "utf8")) as Record; + const consumers = document.consumers as Array>; + for (const template of ["templates.consumers.agentrunV02", "templates.consumers.hwlabV03"]) { + const sourceArtifact = consumers.find((item) => item.extends === template && item.variables?.NODE === "NC01")?.sourceArtifact; + expect(sourceArtifact?.taskRunTemplate).toEqual({ hostNetwork: true, dnsPolicy: "ClusterFirstWithHostNet", fsGroup: 1000 }); + } + }); + + test("undeclared JD01 source artifact owner fails closed", () => { + const result = spawnSync("bun", [ + "scripts/cli.ts", "platform-infra", "pipelines-as-code", "source-artifact", "plan", + "--target", "JD01", "--consumer", "agentrun-jd01-v02", "--source-worktree", rootPath(), + ], { cwd: rootPath(), encoding: "utf8", timeout: 30_000 }); + expect(result.status).toBe(1); + expect(result.stdout).toContain("code=source-artifact-operation-failed"); + expect(result.stdout).toContain("evidence=type=string"); + }); +}); + +describe("Tekton admission canonicalization", () => { + const admitted = { + tasks: [{ + name: "build", + taskSpec: { + metadata: {}, + spec: null, + params: [{ name: "input", type: "string" }], + results: [{ name: "output", type: "string" }], + steps: [{ name: "step", computeResources: {} }], + sidecars: [{ name: "sidecar", computeResources: {} }], + }, + }], + }; + const desired = { + tasks: [{ + name: "build", + taskSpec: { + params: [{ name: "input" }], + results: [{ name: "output" }], + steps: [{ name: "step" }], + sidecars: [{ name: "sidecar" }], + }, + }], + }; + + test("removes exactly the six proven defaults and keeps local/native hashes equal", () => { + expect(canonicalizePacPipelineSpec(admitted)).toEqual(desired); + expect(nativeCanonicalize(admitted)).toEqual(desired); + expect(canonicalSha256(admitted)).toBe(canonicalSha256(desired)); + expect(nativeCanonicalSha256(admitted)).toBe(canonicalSha256(admitted)); + }); + + test("does not strip same-name fields outside the three Pipeline spec roots", () => { + const negative = { + unrelated: admitted, + tasks: [{ taskSpec: { + metadata: { labels: {} }, + spec: {}, + params: [{ type: "array" }], + results: [{ type: "object" }], + steps: [{ computeResources: { requests: { cpu: "1" } } }], + sidecars: [{ computeResources: { limits: { memory: "1Gi" } } }], + } }], + }; + expect(canonicalizePacPipelineSpec(negative)).toEqual(negative); + expect(nativeCanonicalize(negative)).toEqual(negative); + }); +}); + +describe("source artifact serialization", () => { + test("remote Pipeline artifacts are deterministic reviewable multi-line YAML", () => { + const manifest = { + apiVersion: "tekton.dev/v1", + kind: "Pipeline", + metadata: { name: "fixture" }, + spec: { params: [{ name: "revision", type: "string" }], tasks: [{ name: "build", taskSpec: { steps: [{ name: "build", script: "set -eu\necho ok\n" }] } }] }, + }; + const first = pacSourceArtifactYaml(manifest); + const second = pacSourceArtifactYaml(manifest); + expect(first).toBe(second); + expect(first.split("\n").length).toBeGreaterThan(10); + expect(first).toContain("apiVersion: tekton.dev/v1\nkind: Pipeline\n"); + expect(Bun.YAML.parse(first)).toEqual(manifest); + }); +}); + +describe("runtime source-commit selection", () => { + test("same-commit retries select the newest only when spec and provenance agree", () => { + const oldRun = pipelineRun("agentrun-nc01-v02-ci-old", "2026-07-10T00:00:00Z"); + const newRun = pipelineRun("agentrun-nc01-v02-ci-new", "2026-07-11T00:00:00Z"); + const selected = selectPipelineRunBySourceCommit([oldRun, newRun], "agentrun-nc01-v02-ci", sourceCommit); + expect(selected.kind).toBe("ok"); + expect(selected.run.metadata.name).toBe("agentrun-nc01-v02-ci-new"); + const observed = observePacSourceArtifactRuntime(runtimeInput(), (args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: pipeline() } + : { kind: "ok", value: { items: [oldRun, newRun] } }); + expect(observed.embedded.status).toBe("aligned"); + expect(observed.embedded.name).toBe("agentrun-nc01-v02-ci-new"); + expect(observed.embedded.candidateCount).toBe(2); + expect(observed.embedded.sourceCommit).toBe(sourceCommit); + }); + + test("same-commit conflicting embedded specs fail closed", () => { + const observed = observePacSourceArtifactRuntime(runtimeInput(), (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: pipeline() } + : { kind: "ok", value: { items: [ + pipelineRun("agentrun-nc01-v02-ci-old", "2026-07-10T00:00:00Z"), + pipelineRun("agentrun-nc01-v02-ci-new", "2026-07-11T00:00:00Z", { params: [], tasks: [{ name: "changed" }] }), + ] } }); + expect(observed.embedded.status).toBe("unavailable"); + expect(observed.embedded.reason).toBe("source-commit-run-conflict:2"); + expect(observed.embedded.candidateCount).toBe(2); + }); + + test("embedded mode compares the complete PipelineRun wrapper", () => { + const run = pipelineRun("agentrun-nc01-v02-ci-wrapper", "2026-07-11T00:00:00Z"); + (run.spec as Record).taskRunTemplate.serviceAccountName = "wrong-service-account"; + const observed = observePacSourceArtifactRuntime(runtimeInput(), (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: pipeline() } + : { kind: "ok", value: { items: [run] } }); + expect(observed.embedded.status).toBe("drifted"); + expect(observed.embedded.firstDrift?.path).toBe("$.spec.taskRunTemplate.serviceAccountName"); + }); + + test("remote mode validates Pipeline spec live and wrapper on the exact-commit Run", () => { + const remoteProvenance = { ...provenance, mode: "remote-pipeline-annotation" }; + const remoteAnnotations = { + ...manifestAnnotations(), + "unidesk.ai/source-artifact-mode": "remote-pipeline-annotation", + }; + const wrapper = { + mode: "remote-pipeline-annotation", + executionMode: "pipeline-spec", + spec: { ...runtimeWrapperSpec, timeouts: { pipeline: "1h0m0s" } }, + }; + const run = { + metadata: { + name: "hwlab-nc01-v03-ci-poll-remote", + creationTimestamp: "2026-07-11T00:00:00Z", + labels: { "pipelinesascode.tekton.dev/sha": sourceCommit }, + annotations: remoteAnnotations, + }, + spec: { pipelineSpec: desiredSpec, ...wrapper.spec }, + }; + const input = { + ...runtimeInput(), + pipeline: "hwlab-nc01-v03-ci-image-publish", + pipelineRunPrefix: "hwlab-nc01-v03-ci-poll", + provenance: remoteProvenance, + desiredWrapper: wrapper, + }; + const observed = observePacSourceArtifactRuntime(input, (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: { metadata: { name: input.pipeline, annotations: remoteAnnotations }, spec: desiredSpec } } + : { kind: "ok", value: { items: [run] } }); + expect(observed.live.status).toBe("aligned"); + expect(observed.embedded.status).toBe("aligned"); + expect(observed.embedded.firstDrift).toBeNull(); + }); + + test("NotFound, missing commit, and transport failures remain distinct", () => { + const noCommit = observePacSourceArtifactRuntime(runtimeInput(null), () => ({ kind: "not-found", reason: "pipeline-get-not-found" })); + expect(noCommit.live.status).toBe("missing"); + expect(noCommit.live.reason).toBe("pipeline-get-not-found"); + expect(noCommit.embedded.status).toBe("unavailable"); + expect(noCommit.embedded.reason).toBe("source-commit-not-specified"); + + const secret = "Authorization: Bearer REVIEW_RUNTIME_SECRET https://review:password@example.invalid/path?token=QUERY_SECRET"; + const transport = observePacSourceArtifactRuntime(runtimeInput(), () => ({ kind: "unavailable", reason: secret })); + expect(transport.live.status).toBe("unavailable"); + expect(transport.embedded.status).toBe("unavailable"); + expect(transport.live.reason).toMatch(/^runtime-observer-reason:type=string,length=\d+,sha256=[0-9a-f]{64}$/u); + expect(transport.embedded.reason).toMatch(/^runtime-observer-reason:type=string,length=\d+,sha256=[0-9a-f]{64}$/u); + expect(JSON.stringify(transport)).not.toContain(secret); + }); + + test("first drift reports structural evidence without script or token values", () => { + const expectedSecret = "Authorization: Bearer EXPECTED_SUPER_SECRET"; + const actualSecret = "https://review-user:ACTUAL_SUPER_SECRET@example.invalid/path?token=QUERY_SUPER_SECRET"; + const input = { + ...runtimeInput(), + desiredSpec: { tasks: [{ taskSpec: { steps: [{ script: expectedSecret }] } }] }, + }; + const observed = observePacSourceArtifactRuntime(input, (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: { metadata: { name: "pipeline", annotations: manifestAnnotations() }, spec: { tasks: [{ taskSpec: { steps: [{ script: actualSecret }] } }] } } } + : { kind: "ok", value: { items: [pipelineRun("agentrun-nc01-v02-ci-secret", "2026-07-11T00:00:00Z", { tasks: [{ taskSpec: { steps: [{ script: actualSecret }] } }] })] } }); + const serialized = JSON.stringify(observed); + expect(observed.live.firstDrift?.path).toBe("$.tasks[0].taskSpec.steps[0].script"); + expect(observed.live.firstDrift?.expected).toBe(pacValueEvidence(expectedSecret)); + expect(observed.live.firstDrift?.actual).toBe(pacValueEvidence(actualSecret)); + expect(serialized).not.toContain(expectedSecret); + expect(serialized).not.toContain(actualSecret); + expect(serialized).not.toContain("Authorization"); + expect(serialized).not.toContain("QUERY_SUPER_SECRET"); + }); + + test("unknown runtime keys and forged paths are fingerprinted", () => { + const maliciousKey = "HEADER_QUERY_SUPER_SECRET"; + const local = firstPacSourceArtifactDrift({ tasks: [], [maliciousKey]: "expected" }, { tasks: [], [maliciousKey]: "actual" }); + expect(local?.path).not.toContain(maliciousKey); + expect(local?.path).toContain("type=string,length="); + + const maliciousInput = { ...runtimeInput(), desiredSpec: { ...desiredSpec, [maliciousKey]: "expected" } }; + const observed = observePacSourceArtifactRuntime(maliciousInput, (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: { ...pipeline(), spec: { ...desiredSpec, [maliciousKey]: "actual" } } } + : { kind: "ok", value: { items: [pipelineRun("agentrun-nc01-v02-ci-key", "2026-07-11T00:00:00Z", { ...desiredSpec, [maliciousKey]: "actual" })] } }); + expect(JSON.stringify(observed)).not.toContain(maliciousKey); + expect(pacRuntimeSafePath(`$.${maliciousKey}`)).toBe(`path(${pacValueEvidence(`$.${maliciousKey}`)})`); + }); + + test("untrusted runtime annotations and identities are never echoed", () => { + const secret = "Authorization: Bearer OBSERVER_SUPER_SECRET https://u:p@example.invalid/x?token=QUERY_SECRET"; + const annotations = { + ...manifestAnnotations(), + "unidesk.ai/owning-config-ref": secret, + "unidesk.ai/effective-config-sha256": secret, + "unidesk.ai/source-artifact-renderer": secret, + }; + const observed = observePacSourceArtifactRuntime(runtimeInput(), (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: { metadata: { name: secret, annotations }, spec: desiredSpec } } + : { kind: "ok", value: { items: [{ ...pipelineRun("agentrun-nc01-v02-ci-annotation", "2026-07-11T00:00:00Z"), metadata: { name: secret, creationTimestamp: "2026-07-11T00:00:00Z", labels: { "pipelinesascode.tekton.dev/sha": sourceCommit }, annotations } }] } }); + const serialized = JSON.stringify(observed); + expect(observed.live.provenanceAligned).toBe(false); + expect(observed.live.configRef).toBeNull(); + expect(observed.live.effectiveConfigSha256).toBeNull(); + expect(observed.live.name).toBeNull(); + expect(serialized).not.toContain(secret); + expect(serialized).not.toContain("OBSERVER_SUPER_SECRET"); + expect(serialized).not.toContain("QUERY_SECRET"); + + const forged = pacRuntimeSourceArtifactItem({ + status: "drifted", + name: "observer-super-secret", + canonicalSha256: `sha256:${"c".repeat(64)}`, + firstDrift: { path: "$.HEADER_QUERY_SUPER_SECRET", expected: secret, actual: secret }, + provenanceAligned: false, + configRef: "SECRET/TOKEN#QUERY", + effectiveConfigSha256: `sha256:${"d".repeat(64)}`, + renderer: provenance.renderer, + mode: provenance.mode, + sourceCommit: "e".repeat(40), + reason: secret, + candidateCount: 1, + }, { + configRef: provenance.configRef, + effectiveConfigSha256: `sha256:${"f".repeat(64)}`, + renderer: provenance.renderer, + mode: provenance.mode, + sourceCommit, + exactName: null, + namePrefix: "agentrun-nc01-v02-ci", + }); + const forgedSerialized = JSON.stringify(forged); + expect(forged.name).toBeNull(); + expect(forged.configRef).toBeNull(); + expect(forged.effectiveConfigSha256).toBeNull(); + expect(forged.provenanceAligned).toBe(false); + expect(forged.sourceCommit).toBeNull(); + expect(forged.firstDrift?.path).not.toContain("HEADER_QUERY_SUPER_SECRET"); + expect(forgedSerialized).not.toContain(secret); + expect(forgedSerialized).not.toContain("SECRET/TOKEN#QUERY"); + + const forgedAligned = pacRuntimeSourceArtifactItem({ + status: "aligned", + provenanceAligned: true, + configRef: provenance.configRef, + effectiveConfigSha256: provenance.effectiveConfigSha256, + renderer: "forged-renderer", + mode: provenance.mode, + sourceCommit, + candidateCount: 1, + }, { + configRef: provenance.configRef, + effectiveConfigSha256: provenance.effectiveConfigSha256, + renderer: provenance.renderer, + mode: provenance.mode, + sourceCommit, + exactName: null, + namePrefix: null, + }); + expect(forgedAligned.provenanceAligned).toBe(false); + }); + + test("typed runtime alignment never reports a synthetic pending state", () => { + const aligned = observePacSourceArtifactRuntime(runtimeInput(), (_args: string[], label: string) => label === "pipeline-get" + ? { kind: "ok", value: pipeline() } + : { kind: "ok", value: { items: [pipelineRun("agentrun-nc01-v02-ci-aligned", "2026-07-11T00:00:00Z")] } }); + const drifted = structuredClone(aligned); + drifted.embedded.status = "drifted"; + const exact = { head: sourceCommit, clean: true, matchesSourceCommit: true }; + expect(sourceArtifactRuntimeAlignment(null, exact, sourceCommit)).toBe("not-requested"); + expect(sourceArtifactRuntimeAlignment(aligned, exact, sourceCommit)).toBe("aligned"); + expect(sourceArtifactRuntimeAlignment(drifted, exact, sourceCommit)).toBe("drifted"); + expect(sourceArtifactRuntimeAlignment(aligned, { ...exact, clean: false }, sourceCommit)).toBe("unavailable"); + expect(JSON.stringify([aligned, drifted])).not.toContain("pending"); + }); + + test("local transport and remote observer failures retain only bounded evidence", () => { + const secret = "Authorization: Bearer TRANSPORT_SUPER_SECRET https://review:password@example.invalid/path?token=QUERY_SECRET"; + const localReason = pacRuntimeTransportFailureReason({ exitCode: 126, stdout: secret, stderr: secret }); + expect(localReason).toContain(pacValueEvidence(secret)); + expect(localReason).not.toContain(secret); + expect(pacRuntimeSafeReason(secret)).toBe(`runtime-observer-reason:${pacValueEvidence(secret)}`); + + const temporary = mkdtempSync(resolve(tmpdir(), "unidesk-pac-runtime-redaction-test-")); + try { + const bin = resolve(temporary, "bin"); + mkdirSync(bin); + const kubectl = resolve(bin, "kubectl"); + writeFileSync(kubectl, `#!/bin/sh\nprintf '%s\\n' '${secret}' >&2\nexit 126\n`); + chmodSync(kubectl, 0o755); + const shell = renderPacSourceArtifactRuntimeObserverShell(runtimeInput()); + const result = spawnSync("sh", [], { + input: shell, + encoding: "utf8", + timeout: 30_000, + env: { ...process.env, PATH: `${bin}:${process.env.PATH ?? ""}` }, + }); + expect(result.status).toBe(0); + expect(result.stdout).toContain("command-failed:126:type=string"); + expect(result.stdout).not.toContain(secret); + expect(result.stdout).not.toContain("Authorization"); + expect(result.stdout).not.toContain("QUERY_SECRET"); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }); + + test("temp-file transport handles a payload larger than the current 203 KiB HWLAB Pipeline", () => { + const observerSource = readFileSync(rootPath("scripts", "native", "cicd", "pac-source-artifact-runtime.mjs"), "utf8"); + expect(observerSource).not.toMatch(/process\.env\.PAC_SOURCE_ARTIFACT_INPUT(?!_FILE)/u); + const temporary = mkdtempSync(resolve(tmpdir(), "unidesk-pac-runtime-test-")); + try { + const bin = resolve(temporary, "bin"); + mkdirSync(bin); + const hugeSpec = { params: [], tasks: [{ name: "render", taskSpec: { steps: [{ name: "render", script: "x".repeat(220 * 1024) }] } }] }; + const input = { + namespace: "hwlab-ci", + pipeline: "hwlab-nc01-v03-ci-image-publish", + pipelineRunPrefix: "hwlab-nc01-v03-ci-poll", + desiredSpec: hugeSpec, + desiredWrapper: { mode: "embedded-pipeline-spec", executionMode: "pipeline-spec", spec: runtimeWrapperSpec }, + provenance, + sourceCommit, + }; + expect(Buffer.byteLength(JSON.stringify(input), "utf8")).toBeGreaterThan(203 * 1024); + const pipelineFile = resolve(temporary, "pipeline.json"); + const runsFile = resolve(temporary, "runs.json"); + writeFileSync(pipelineFile, JSON.stringify({ metadata: { name: input.pipeline, annotations: manifestAnnotations() }, spec: hugeSpec })); + writeFileSync(runsFile, JSON.stringify({ items: [pipelineRun("hwlab-nc01-v03-ci-poll-large", "2026-07-11T00:00:00Z", hugeSpec)] })); + const kubectl = resolve(bin, "kubectl"); + writeFileSync(kubectl, "#!/bin/sh\nset -eu\ncase \"$2\" in pipeline) cat \"$PAC_TEST_PIPELINE\" ;; pipelinerun) cat \"$PAC_TEST_RUNS\" ;; *) exit 2 ;; esac\n"); + chmodSync(kubectl, 0o755); + const shell = renderPacSourceArtifactRuntimeObserverShell(input); + expect(shell).toContain("observer_input=$(mktemp)"); + expect(shell).toContain("PAC_SOURCE_ARTIFACT_INPUT_FILE=\"$observer_input\""); + expect(shell).not.toContain("PAC_SOURCE_ARTIFACT_INPUT="); + const result = spawnSync("sh", [], { + input: shell, + encoding: "utf8", + timeout: 30_000, + maxBuffer: 8 * 1024 * 1024, + env: { + ...process.env, + PATH: `${bin}:${process.env.PATH ?? ""}`, + PAC_TEST_PIPELINE: pipelineFile, + PAC_TEST_RUNS: runsFile, + }, + }); + expect(result.status).toBe(0); + const observed = JSON.parse(result.stdout) as Record; + expect(observed.live.status).toBe("aligned"); + expect(observed.embedded.status).toBe("aligned"); + expect(observed.embedded.sourceCommit).toBe(sourceCommit); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }); +}); + +describe("shared HWLAB deploy overlay", () => { + test("preserves exact undefined/null semantics without mutating its input", () => { + const document = { + nodes: { NC01: { keep: "node" } }, + lanes: { v03: { + keep: "lane", + externalPostgres: { old: true }, + runtimeStore: { old: true }, + codeAgentRuntime: { old: true }, + gitMirror: { old: true }, + envRecipe: { keep: "recipe", downloadStack: { keep: "download" } }, + } }, + }; + const overlay = { + nodeId: "NC01", + lane: "v03", + gitopsRoot: "deploy/gitops/node/nc01", + gitUrl: "git@example/HWLAB.git", + sourceBranch: "v0.3", + gitopsBranch: "v0.3-gitops", + runtimeNamespace: "hwlab-v03", + publicApiUrl: "https://api.example", + publicWebUrl: "https://web.example", + catalogPath: "deploy/artifacts/nc01.yaml", + runtimePath: "deploy/gitops/node/nc01/hwlab-v03", + observability: { enabled: true }, + dockerProxyHttp: null, + dockerProxyHttps: null, + dockerNoProxyList: ["localhost"], + externalPostgres: null, + runtimeStore: null, + codeAgentRuntime: null, + deployYamlGitMirror: null, + }; + const rendered = applyNodeRuntimeDeployYamlOverlay(document, overlay); + expect(document.lanes.v03.externalPostgres).toEqual({ old: true }); + expect(rendered.lanes.v03).not.toHaveProperty("externalPostgres"); + expect(rendered.lanes.v03.runtimeStore).toBeNull(); + expect(rendered.lanes.v03.codeAgentRuntime).toBeNull(); + expect(rendered.lanes.v03.gitMirror).toBeNull(); + expect(rendered.lanes.v03.keep).toBe("lane"); + expect(nodeRuntimeDeployYamlOverlayShellScript().join("\n")).not.toContain(": Record { + const temporary = mkdtempSync(resolve(tmpdir(), "unidesk-pac-overlay-test-")); + try { + mkdirSync(resolve(temporary, "deploy")); + const document = { nodes: { NC01: {} }, lanes: { v03: { envRecipe: { downloadStack: {} } } } }; + const overlay = { + nodeId: "NC01", lane: "v03", gitopsRoot: "deploy/gitops/node/nc01", gitUrl: "git@example/HWLAB.git", + sourceBranch: "v0.3", gitopsBranch: "v0.3-gitops", runtimeNamespace: "hwlab-v03", + publicApiUrl: "https://api.example", publicWebUrl: "https://web.example", catalogPath: "deploy/artifacts/nc01.yaml", + runtimePath: "deploy/gitops/node/nc01/hwlab-v03", observability: { enabled: true }, + dockerProxyHttp: "http://proxy", dockerProxyHttps: "http://proxy", dockerNoProxyList: ["localhost"], + externalPostgres: { host: "postgres" }, runtimeStore: { mode: "postgres" }, codeAgentRuntime: { enabled: true }, + deployYamlGitMirror: { readUrl: "http://mirror" }, + }; + writeFileSync(resolve(temporary, "deploy", "deploy.yaml"), Bun.YAML.stringify(document)); + const overlayBase64 = Buffer.from(JSON.stringify(overlay), "utf8").toString("base64"); + const shell = [`overlay_b64='${overlayBase64}'`, ...nodeRuntimeDeployYamlOverlayShellScript()].join("\n"); + const result = spawnSync("sh", [], { + cwd: temporary, + input: shell, + encoding: "utf8", + timeout: 30_000, + env: { ...process.env, NODE_PATH: resolve(rootPath(), "node_modules") }, + }); + expect(result.status).toBe(0); + const shellRendered = Bun.YAML.parse(readFileSync(resolve(temporary, "deploy", "deploy.yaml"), "utf8")) as Record; + expect(shellRendered).toEqual(applyNodeRuntimeDeployYamlOverlay(document, overlay)); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }); +}); diff --git a/scripts/src/platform-infra-pipelines-as-code-source-artifact.ts b/scripts/src/platform-infra-pipelines-as-code-source-artifact.ts new file mode 100644 index 00000000..ab099f76 --- /dev/null +++ b/scripts/src/platform-infra-pipelines-as-code-source-artifact.ts @@ -0,0 +1,1091 @@ +import { createHash } from "node:crypto"; +import { cpSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, isAbsolute, relative, resolve, sep } from "node:path"; +import { spawnSync } from "node:child_process"; + +import { rootPath } from "./config"; +import { AGENTRUN_CONFIG_PATH, resolveAgentRunLaneTarget } from "./agentrun-lanes"; +import { renderAgentRunPipelineManifest } from "./agentrun-manifests"; +import { hwlabRuntimeLaneSpecForNode, isHwlabRuntimeLane, type HwlabRuntimeLaneSpec } from "./hwlab-node-lanes"; +import { nodeRuntimeGitopsRoot } from "./hwlab-node/cleanup"; +import { nodeRuntimePipelinePostprocessScript } from "./hwlab-node/render"; +import { nodeRuntimeRenderOverlay } from "./hwlab-node/web-probe"; +import { applyNodeRuntimeDeployYamlOverlay } from "./hwlab-node/deploy-overlay"; +import type { RenderedCliResult } from "./output"; +import { renderedCliResult } from "./agentrun/render"; +import { stableJsonSha256, stableJsonValue } from "./stable-json"; + +export type PacSourceArtifactMode = "embedded-pipeline-spec" | "remote-pipeline-annotation"; +export type PacSourceArtifactRenderer = "agentrun-control-plane" | "hwlab-runtime-lane"; +export type PacSourceArtifactAction = "plan" | "check" | "write" | "status" | "verify-runtime"; +export type PacSourceArtifactRuntimeAlignment = "not-requested" | "aligned" | "drifted" | "missing" | "unavailable"; + +export interface PacSourceArtifactTaskRunTemplateSpec { + readonly hostNetwork: boolean; + readonly dnsPolicy: "ClusterFirst" | "ClusterFirstWithHostNet" | "Default" | "None"; + readonly fsGroup: number; +} + +export interface PacSourceArtifactSpec { + readonly mode: PacSourceArtifactMode; + readonly renderer: PacSourceArtifactRenderer; + readonly configRef: string; + readonly pipelineRunPath: string; + readonly pipelinePath: string | null; + readonly maxKeepRuns: number; + readonly taskRunTemplate: PacSourceArtifactTaskRunTemplateSpec; +} + +export interface PacSourceArtifactBinding { + readonly target: { readonly id: string }; + readonly consumer: { + readonly id: string; + readonly node: string; + readonly lane: string; + readonly namespace: string; + readonly pipeline: string; + readonly pipelineRunPrefix: string; + readonly sourceArtifact: PacSourceArtifactSpec; + }; + readonly repository: { + readonly id: string; + readonly url: string; + readonly cloneUrl: string; + readonly owner: string; + readonly repo: string; + readonly params: Readonly>; + }; +} + +export interface PacSourceArtifactOptions { + readonly action: PacSourceArtifactAction; + readonly targetId: string; + readonly consumerId: string; + readonly sourceWorktree: string; + readonly sourceCommit: string | null; + readonly confirm: boolean; + readonly json: boolean; + readonly full: boolean; +} + +export interface PacSourceArtifactRuntimeObservation { + readonly live: PacSourceArtifactRuntimeItem; + readonly embedded: PacSourceArtifactRuntimeItem; +} + +export interface PacSourceArtifactRuntimeItem { + readonly status: "aligned" | "drifted" | "missing" | "unavailable"; + readonly name: string | null; + readonly canonicalSha256: string | null; + readonly firstDrift: PacSourceArtifactDrift | null; + readonly provenanceAligned: boolean; + readonly configRef: string | null; + readonly effectiveConfigSha256: string | null; + readonly sourceCommit: string | null; + readonly reason: string | null; + readonly candidateCount: number; +} + +export interface PacSourceArtifactDrift { + readonly path: string; + readonly expected: string; + readonly actual: string; +} + +export type PacSourceArtifactRuntimeObserver = (input: { + readonly binding: PacSourceArtifactBinding; + readonly desiredSpec: Record; + readonly desiredWrapper: Record | null; + readonly provenance: PacSourceArtifactProvenance; + readonly sourceCommit: string | null; +}) => Promise; + +interface PacSourceArtifactProvenance { + readonly configRef: string; + readonly effectiveConfigSha256: string; + readonly renderer: PacSourceArtifactRenderer; + readonly mode: PacSourceArtifactMode; +} + +interface DesiredArtifact { + readonly pipeline: Record; + readonly pipelineRun: Record; + readonly provenance: PacSourceArtifactProvenance; + readonly desiredSpec: Record; + readonly files: readonly { readonly path: string; readonly content: string }[]; +} + +interface SourceInspection { + readonly status: "aligned" | "drifted" | "missing"; + readonly aligned: boolean; + readonly pipelineCanonicalSha256: string | null; + readonly pipelineRunCanonicalSha256: string | null; + readonly firstDrift: PacSourceArtifactDrift | null; + readonly provenanceAligned: boolean; +} + +export interface SourceWorktreeState { + readonly head: string; + readonly clean: boolean; + readonly matchesSourceCommit: boolean | null; +} + +export class PacSourceArtifactError extends Error { + readonly code: string; + readonly safeDetails: Readonly>; + + constructor(code: string, safeDetails: Readonly> = {}) { + super(code); + this.name = "PacSourceArtifactError"; + this.code = code; + this.safeDetails = safeDetails; + } +} + +export interface PacSourceArtifactSafeError { + readonly code: string; + readonly evidence: string; + readonly details: Readonly>; +} + +export function pacSourceArtifactSafeError(error: unknown): PacSourceArtifactSafeError { + const message = error instanceof Error ? error.message : String(error); + const allowedDetailKeys = new Set([ + "expectedUrlFingerprint", + "observedUrlFingerprint", + "head", + "requestedSourceCommit", + ]); + const details = error instanceof PacSourceArtifactError + ? Object.fromEntries(Object.entries(error.safeDetails).map(([key, value]) => [key, allowedDetailKeys.has(key) ? value : pacValueEvidence(value)])) + : {}; + return { + code: error instanceof PacSourceArtifactError ? error.code : "source-artifact-operation-failed", + evidence: pacValueEvidence(message), + details, + }; +} + +const provenanceKeys = { + configRef: "unidesk.ai/owning-config-ref", + effectiveConfigSha256: "unidesk.ai/effective-config-sha256", + renderer: "unidesk.ai/source-artifact-renderer", + mode: "unidesk.ai/source-artifact-mode", +} as const; + +const knownDriftPathKeys = new Set([ + "accessModes", "annotations", "apiVersion", "args", "command", "computeResources", "default", "description", "dnsPolicy", "env", "envFrom", + "executionMode", "finally", "fsGroup", "generateName", "hostNetwork", "image", "kind", "labels", "length", "metadata", "mode", "mountPath", "name", "namespace", "onError", "operator", + "optional", "params", "pipeline", "pipelineRef", "pipelineSpec", "podTemplate", "readinessProbe", "readOnly", "requests", "resources", + "results", "retries", "runAfter", "script", "secret", "secretName", "securityContext", "serviceAccountName", "sidecars", "spec", "steps", + "storage", "subPath", "taskRunTemplate", "taskSpec", "tasks", "timeout", "timeouts", "type", "value", "values", "volumeClaimTemplate", + "volumeMounts", "volumes", "when", "workspaces", "workingDir", "wrapper", +]); + +export function parsePacSourceArtifactOptions(args: readonly string[]): PacSourceArtifactOptions { + const action = args[0]; + if (!isSourceArtifactAction(action)) { + throw new Error("source-artifact requires one of plan, check, write, status, verify-runtime"); + } + let targetId: string | null = null; + let consumerId: string | null = null; + let sourceWorktree: string | null = null; + let sourceCommit: string | null = null; + let confirm = false; + let json = false; + let full = false; + for (let index = 1; index < args.length; index += 1) { + const arg = args[index]; + if (arg === "--confirm") { + confirm = true; + continue; + } + if (arg === "--json") { + json = true; + continue; + } + if (arg === "--full") { + full = true; + continue; + } + if (arg === "--target" || arg === "--consumer" || arg === "--source-worktree" || arg === "--source-commit") { + const value = args[index + 1]; + if (value === undefined || value.startsWith("--")) throw new Error(`${arg} requires a value`); + if (arg === "--target") targetId = value; + else if (arg === "--consumer") consumerId = value; + else if (arg === "--source-worktree") sourceWorktree = value; + else sourceCommit = value.toLowerCase(); + index += 1; + continue; + } + throw new Error(`unsupported source-artifact option: ${arg}`); + } + if (targetId === null) throw new Error("source-artifact requires explicit --target"); + if (consumerId === null) throw new Error("source-artifact requires explicit --consumer"); + if (sourceWorktree === null) throw new Error("source-artifact requires explicit --source-worktree"); + if (!isAbsolute(sourceWorktree)) throw new Error("--source-worktree must be an absolute path"); + if (sourceCommit !== null && !/^[0-9a-f]{40}$/u.test(sourceCommit)) throw new Error("--source-commit must be a full 40-character git SHA"); + if (sourceCommit !== null && action !== "status" && action !== "verify-runtime") throw new Error("--source-commit is accepted only by source-artifact status or verify-runtime"); + if (action === "verify-runtime" && sourceCommit === null) throw new PacSourceArtifactError("source-commit-required"); + if (action === "write" && !confirm) throw new Error("source-artifact write requires --confirm"); + if (action !== "write" && confirm) throw new Error("--confirm is accepted only by source-artifact write"); + return { action, targetId, consumerId, sourceWorktree, sourceCommit, confirm, json, full }; +} + +export async function runPacSourceArtifact( + binding: PacSourceArtifactBinding, + options: PacSourceArtifactOptions, + observeRuntime?: PacSourceArtifactRuntimeObserver, +): Promise> { + if (binding.target.id.toLowerCase() !== options.targetId.toLowerCase()) { + throw new Error(`source-artifact target ${options.targetId} does not match resolved target ${binding.target.id}`); + } + if (binding.consumer.id.toLowerCase() !== options.consumerId.toLowerCase()) { + throw new Error(`source-artifact consumer ${options.consumerId} does not match resolved consumer ${binding.consumer.id}`); + } + const sourceWorktree = validateSourceWorktree(options.sourceWorktree, binding); + const sourceHead = git(sourceWorktree, ["rev-parse", "HEAD"]).toLowerCase(); + const sourceWorktreeState: SourceWorktreeState = { + head: sourceHead, + clean: git(sourceWorktree, ["status", "--porcelain=v1", "--untracked-files=all"]).length === 0, + matchesSourceCommit: options.sourceCommit === null ? null : sourceHead === options.sourceCommit, + }; + assertPacSourceArtifactVerifyWorktree(options.action, options.sourceCommit, sourceWorktreeState); + const desired = renderDesiredArtifact(binding, sourceWorktree); + let source = inspectSourceArtifact(sourceWorktree, binding, desired); + const written: string[] = []; + if (options.action === "write") { + written.push(...writeArtifactFilesAtomically(sourceWorktree, desired.files)); + source = inspectSourceArtifact(sourceWorktree, binding, desired); + } + const needsRuntime = options.action === "status" || options.action === "verify-runtime"; + const runtime = needsRuntime + ? observeRuntime === undefined + ? unavailableRuntimeObservation() + : await observeRuntime({ + binding, + desiredSpec: desired.desiredSpec, + desiredWrapper: options.sourceCommit === null ? null : desiredRuntimePipelineRunWrapper(binding, desired.pipelineRun, options.sourceCommit), + provenance: desired.provenance, + sourceCommit: options.sourceCommit, + }) + : null; + const sourceOk = source.aligned && source.provenanceAligned; + const runtimeAlignment = sourceArtifactRuntimeAlignment(runtime, sourceWorktreeState, options.sourceCommit); + const runtimeOk = runtimeAlignment === "aligned"; + const ok = options.action === "check" + ? sourceOk + : options.action === "verify-runtime" + ? sourceOk && runtimeOk + : options.action === "write" + ? sourceOk + : true; + const desiredSpecSha256 = canonicalSha256(desired.desiredSpec); + return { + ok, + action: `platform-infra-pipelines-as-code-source-artifact-${options.action}`, + mutation: options.action === "write" && written.length > 0, + target: binding.target.id, + consumer: binding.consumer.id, + node: binding.consumer.node, + lane: binding.consumer.lane, + mode: binding.consumer.sourceArtifact.mode, + renderer: binding.consumer.sourceArtifact.renderer, + sourceWorktree, + files: { + pipelineRun: binding.consumer.sourceArtifact.pipelineRunPath, + pipeline: binding.consumer.sourceArtifact.pipelinePath, + written, + }, + provenance: desired.provenance, + desired: { + canonicalSha256: desiredSpecSha256, + pipeline: binding.consumer.pipeline, + namespace: binding.consumer.namespace, + }, + source, + sourceWorktreeState, + runtime, + runtimeTarget: { sourceCommit: options.sourceCommit }, + runtimeAlignment, + boundary: { + desiredAuthority: "owning YAML plus domain renderer", + liveExportAllowed: false, + sourceWorktreeExplicit: true, + canonicalAdmissionDefaultsRemoved: [ + "tasks[].taskSpec.metadata={}", + "tasks[].taskSpec.spec=null", + "tasks[].taskSpec.params[].type=string", + "tasks[].taskSpec.results[].type=string", + "tasks[].taskSpec.steps[].computeResources={}", + "tasks[].taskSpec.sidecars[].computeResources={}", + ], + valuesPrinted: false, + }, + next: sourceArtifactNext(binding, options, sourceOk, sourceWorktreeState, runtime, runtimeAlignment), + }; +} + +export function assertPacSourceArtifactVerifyWorktree( + action: PacSourceArtifactAction, + sourceCommit: string | null, + worktree: SourceWorktreeState, +): void { + if (action !== "verify-runtime") return; + if (sourceCommit === null) throw new PacSourceArtifactError("source-commit-required"); + if (!worktree.clean) throw new PacSourceArtifactError("source-worktree-not-clean", { head: worktree.head }); + if (worktree.matchesSourceCommit !== true) { + throw new PacSourceArtifactError("source-worktree-commit-mismatch", { + head: worktree.head, + requestedSourceCommit: sourceCommit, + }); + } +} + +export function sourceArtifactRuntimeAlignment( + runtime: PacSourceArtifactRuntimeObservation | null, + worktree: SourceWorktreeState, + sourceCommit: string | null, +): PacSourceArtifactRuntimeAlignment { + if (runtime === null) return "not-requested"; + if (sourceCommit !== null && (!worktree.clean || worktree.matchesSourceCommit !== true)) return "unavailable"; + if (runtime.live.status === "unavailable" || runtime.embedded.status === "unavailable") return "unavailable"; + if (runtime.live.status === "missing" || runtime.embedded.status === "missing") return "missing"; + if (runtime.live.status === "drifted" || runtime.embedded.status === "drifted") return "drifted"; + if (!runtime.live.provenanceAligned || !runtime.embedded.provenanceAligned) return "drifted"; + return "aligned"; +} + +function sourceArtifactNext( + binding: PacSourceArtifactBinding, + options: PacSourceArtifactOptions, + sourceOk: boolean, + worktree: SourceWorktreeState, + runtime: PacSourceArtifactRuntimeObservation | null, + runtimeAlignment: PacSourceArtifactRuntimeAlignment, +): string { + if (!sourceOk) { + return options.action === "write" + ? "Generated files remain inconsistent; inspect the safe firstDrift evidence and do not publish." + : `Run source-artifact write --confirm for ${binding.consumer.id}, then rerun check.`; + } + if (runtime === null) return `After the consumer PR rolls out, use a clean exact-commit worktree and run verify-runtime for ${binding.consumer.id} with --source-commit .`; + if (!worktree.clean || (options.sourceCommit !== null && worktree.matchesSourceCommit !== true)) { + return "Create a clean worktree at the exact GitHub merge commit, then rerun status or verify-runtime with that full source commit."; + } + if (runtime.live.status === "unavailable") return "Repair the owning control-plane runtime observer or RBAC, then rerun status before verification."; + if (runtime.live.status === "missing") return "Apply the owning YAML control plane, confirm the live Pipeline exists, then rerun verify-runtime."; + if (runtime.live.status === "drifted" || !runtime.live.provenanceAligned) return "Apply the owning YAML control plane so the live Pipeline spec and provenance align, then rerun verify-runtime."; + if (runtime.embedded.status === "unavailable") return "Inspect the exact-commit PaC PipelineRun identity and observer access, then rerun verify-runtime."; + if (runtime.embedded.status === "missing") return "Wait for or repair the exact-commit PaC PipelineRun; do not validate a prefix-selected run."; + if (runtime.embedded.status === "drifted" || !runtime.embedded.provenanceAligned) return "Regenerate and merge the consumer source artifact, then verify the new exact-commit PipelineRun."; + return runtimeAlignment === "aligned" + ? "Exact source, live Pipeline, embedded PipelineRun, and provenance are aligned; continue consumer closeout." + : "Resolve the reported runtime layer and rerun verify-runtime."; +} + +export function renderPacSourceArtifactResult(result: Record): RenderedCliResult { + const files = record(result.files); + const desired = record(result.desired); + const source = record(result.source); + const sourceWorktreeState = record(result.sourceWorktreeState); + const runtime = result.runtime === null ? null : record(result.runtime); + const live = runtime === null ? null : record(runtime.live); + const embedded = runtime === null ? null : record(runtime.embedded); + const firstDrift = source.firstDrift === null ? null : record(source.firstDrift); + const liveDrift = live?.firstDrift === null ? null : record(live?.firstDrift); + const embeddedDrift = embedded?.firstDrift === null ? null : record(embedded?.firstDrift); + const lines = [ + "PAC SOURCE ARTIFACT", + `target=${text(result.target)} consumer=${text(result.consumer)} node=${text(result.node)} lane=${text(result.lane)}`, + `mode=${text(result.mode)} renderer=${text(result.renderer)} mutation=${text(result.mutation)}`, + `pipelineRun=${text(files.pipelineRun)} pipeline=${text(files.pipeline ?? "-")}`, + `desired=${shortSha(desired.canonicalSha256)} source=${shortSha(source.pipelineCanonicalSha256)} aligned=${text(source.aligned)} provenance=${text(source.provenanceAligned)}`, + `sourceWorktree=head:${shortSha(sourceWorktreeState.head)} clean:${text(sourceWorktreeState.clean)} commitMatch:${text(sourceWorktreeState.matchesSourceCommit)}`, + `sourceDrift=${formatDrift(firstDrift)}`, + `runtime=${text(result.runtimeAlignment)} commit=${text(embedded?.sourceCommit ?? record(result.runtimeTarget).sourceCommit)} candidates=${text(embedded?.candidateCount)}`, + `live=status:${text(live?.status)} name:${text(live?.name)} hash:${shortSha(live?.canonicalSha256)} provenance:${text(live?.provenanceAligned)} drift:${formatDrift(liveDrift)}`, + `embedded=status:${text(embedded?.status)} name:${text(embedded?.name)} hash:${shortSha(embedded?.canonicalSha256)} provenance:${text(embedded?.provenanceAligned)} drift:${formatDrift(embeddedDrift)}`, + `runtimeReason=live:${text(live?.reason)} embedded:${text(embedded?.reason)}`, + `written=${Array.isArray(files.written) && files.written.length > 0 ? files.written.join(",") : "-"}`, + ]; + if (typeof result.next === "string") lines.push(`next=${result.next}`); + return renderedCliResult(result.ok !== false, String(result.action), `${lines.join("\n")}\n`); +} + +export function canonicalizePacPipelineSpec(value: unknown, path: readonly (string | number)[] = []): unknown { + if (Array.isArray(value)) return value.map((item, index) => canonicalizePacPipelineSpec(item, [...path, index])); + if (!isRecord(value)) return value; + const output: Record = {}; + for (const key of Object.keys(value).sort()) { + const child = value[key]; + const childPath = [...path, key]; + if (isProvenTektonAdmissionDefault(childPath, child)) continue; + output[key] = canonicalizePacPipelineSpec(child, childPath); + } + return output; +} + +export function canonicalSha256(value: unknown): string { + return `sha256:${createHash("sha256").update(JSON.stringify(canonicalizePacPipelineSpec(value))).digest("hex")}`; +} + +export function firstPacSourceArtifactDrift(expected: unknown, actual: unknown, path = "$"): PacSourceArtifactDrift | null { + const left = canonicalizePacPipelineSpec(expected); + const right = canonicalizePacPipelineSpec(actual); + return firstCanonicalDrift(left, right, path); +} + +function renderDesiredArtifact(binding: PacSourceArtifactBinding, sourceWorktree: string): DesiredArtifact { + const sourceArtifact = binding.consumer.sourceArtifact; + const rendered = sourceArtifact.renderer === "agentrun-control-plane" + ? renderAgentRunDesiredPipeline(binding) + : renderHwlabDesiredPipeline(binding, sourceWorktree); + const pipeline = withPipelineProvenance(rendered.pipeline, rendered.provenance); + const desiredSpec = requiredRecord(pipeline.spec, "rendered Pipeline.spec"); + assertPipelineIdentity(pipeline, binding); + const pipelineRun = sourceArtifact.mode === "embedded-pipeline-spec" + ? embeddedPipelineRun(binding, desiredSpec, rendered.provenance) + : remotePipelineRun(binding, pipeline, rendered.provenance); + const files = sourceArtifact.mode === "embedded-pipeline-spec" + ? [{ path: sourceArtifact.pipelineRunPath, content: pacSourceArtifactYaml(pipelineRun) }] + : [ + { path: requiredString(sourceArtifact.pipelinePath, "sourceArtifact.pipelinePath"), content: pacSourceArtifactYaml(pipeline) }, + { path: sourceArtifact.pipelineRunPath, content: pacSourceArtifactYaml(pipelineRun) }, + ]; + return { pipeline, pipelineRun, provenance: rendered.provenance, desiredSpec, files }; +} + +function renderAgentRunDesiredPipeline(binding: PacSourceArtifactBinding): { pipeline: Record; provenance: PacSourceArtifactProvenance } { + const { spec } = resolveAgentRunLaneTarget({ node: binding.consumer.node, lane: binding.consumer.lane }); + const expectedConfigRef = `${AGENTRUN_CONFIG_PATH}#controlPlane.lanes.${spec.lane}`; + assertConfigRef(binding.consumer.sourceArtifact.configRef, expectedConfigRef); + assertBindingIdentity(binding, { + node: spec.nodeId, + lane: spec.lane, + namespace: spec.ci.namespace, + pipeline: spec.ci.pipeline, + pipelineRunPrefix: spec.ci.pipelineRunPrefix, + serviceAccount: spec.ci.serviceAccountName, + sourceBranch: spec.source.branch, + gitopsBranch: spec.gitops.branch, + }); + return { + pipeline: renderAgentRunPipelineManifest(spec), + provenance: { + configRef: expectedConfigRef, + effectiveConfigSha256: stableJsonSha256(spec), + renderer: "agentrun-control-plane", + mode: binding.consumer.sourceArtifact.mode, + }, + }; +} + +function renderHwlabDesiredPipeline(binding: PacSourceArtifactBinding, sourceWorktree: string): { pipeline: Record; provenance: PacSourceArtifactProvenance } { + if (!isHwlabRuntimeLane(binding.consumer.lane)) throw new Error(`HWLAB source artifact lane ${binding.consumer.lane} is not declared`); + const spec = hwlabRuntimeLaneSpecForNode(binding.consumer.lane, binding.consumer.node); + const expectedConfigRef = `config/hwlab-node-lanes.yaml#lanes.${spec.lane}.targets.${spec.nodeId}`; + assertConfigRef(binding.consumer.sourceArtifact.configRef, expectedConfigRef); + assertBindingIdentity(binding, { + node: spec.nodeId, + lane: spec.lane, + namespace: "hwlab-ci", + pipeline: spec.pipeline, + pipelineRunPrefix: spec.pipelineRunPrefix, + serviceAccount: spec.serviceAccountName, + sourceBranch: spec.sourceBranch, + gitopsBranch: spec.gitopsBranch, + }); + const pipeline = renderHwlabPipelineFromOwningYaml(spec, sourceWorktree); + return { + pipeline, + provenance: { + configRef: expectedConfigRef, + effectiveConfigSha256: stableJsonSha256(spec), + renderer: "hwlab-runtime-lane", + mode: binding.consumer.sourceArtifact.mode, + }, + }; +} + +function renderHwlabPipelineFromOwningYaml(spec: HwlabRuntimeLaneSpec, sourceWorktree: string): Record { + const temporaryRoot = mkdtempSync(resolve(tmpdir(), "unidesk-pac-source-artifact-")); + const temporarySource = resolve(temporaryRoot, "source"); + const output = resolve(temporaryRoot, "rendered"); + const sourceCommit = git(sourceWorktree, ["rev-parse", "HEAD"]); + try { + copySourceWorktreeForRender(sourceWorktree, temporarySource); + const overlayRecord = nodeRuntimeRenderOverlay(spec); + const deployPath = resolve(temporarySource, "deploy", "deploy.yaml"); + const deploy = parseYamlRecord(readFileSync(deployPath, "utf8"), deployPath); + writeFileSync(deployPath, `${Bun.YAML.stringify(applyNodeRuntimeDeployYamlOverlay(deploy, overlayRecord)).trim()}\n`); + const renderArgs = [ + "scripts/run-bun.mjs", + "scripts/gitops-render.mjs", + "--lane", spec.lane, + "--node", spec.nodeId, + "--gitops-root", nodeRuntimeGitopsRoot(spec), + "--catalog-path", spec.catalogPath, + "--image-tag-mode", "full", + "--source-revision", sourceCommit, + "--source-repo", spec.gitUrl, + "--source-branch", spec.sourceBranch, + "--gitops-branch", spec.gitopsBranch, + "--git-read-url", spec.gitReadUrl, + "--git-write-url", spec.gitWriteUrl, + "--registry-prefix", spec.registryPrefix, + "--runtime-endpoint", spec.publicApiUrl, + "--web-endpoint", spec.publicWebUrl, + "--out", output, + ]; + runChecked("node", renderArgs, temporarySource, 120_000, "HWLAB owning YAML renderer"); + const overlay = Buffer.from(JSON.stringify(overlayRecord), "utf8").toString("base64"); + const postprocess = [ + "set -eu", + `render_dir=${shellQuote(output)}`, + `overlay_b64=${shellQuote(overlay)}`, + ...nodeRuntimePipelinePostprocessScript(), + ].join("\n"); + runChecked("sh", ["-c", postprocess], temporarySource, 120_000, "HWLAB UniDesk domain postprocess/verify renderer"); + const path = resolve(output, spec.tektonDir, "pipeline.yaml"); + if (!existsSync(path)) throw new Error(`HWLAB domain renderer did not produce ${path}`); + return parseYamlRecord(readFileSync(path, "utf8"), path); + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }); + } +} + +function copySourceWorktreeForRender(sourceWorktree: string, destination: string): void { + const excluded = new Set([".git", ".worktree", ".state", "node_modules", "coverage", "tmp", ".tmp"]); + cpSync(sourceWorktree, destination, { + recursive: true, + filter: (source) => { + const rel = relative(sourceWorktree, source); + if (rel === "") return true; + return !excluded.has(rel.split(sep)[0] ?? ""); + }, + }); + const nodeModules = resolve(sourceWorktree, "node_modules"); + if (existsSync(nodeModules)) symlinkSync(nodeModules, resolve(destination, "node_modules"), "dir"); +} + +function embeddedPipelineRun(binding: PacSourceArtifactBinding, desiredSpec: Record, provenance: PacSourceArtifactProvenance): Record { + return { + apiVersion: "tekton.dev/v1", + kind: "PipelineRun", + metadata: { + name: `${binding.consumer.pipelineRunPrefix}-{{ revision }}`, + namespace: binding.consumer.namespace, + annotations: pipelineRunAnnotations(binding, provenance), + labels: pipelineRunLabels(binding, "agentrun"), + }, + spec: { + pipelineSpec: desiredSpec, + taskRunTemplate: taskRunTemplate(binding), + params: pipelineRunParams(binding, desiredSpec), + workspaces: pipelineRunWorkspaces(binding, desiredSpec), + }, + }; +} + +function remotePipelineRun(binding: PacSourceArtifactBinding, pipeline: Record, provenance: PacSourceArtifactProvenance): Record { + const desiredSpec = requiredRecord(pipeline.spec, "rendered Pipeline.spec"); + const pipelinePath = requiredString(binding.consumer.sourceArtifact.pipelinePath, "sourceArtifact.pipelinePath"); + const annotations = { + ...pipelineRunAnnotations(binding, provenance), + "pipelinesascode.tekton.dev/pipeline": pipelinePath, + }; + if (provenance.renderer === "hwlab-runtime-lane") { + if (!isHwlabRuntimeLane(binding.consumer.lane)) throw new Error(`HWLAB source artifact lane ${binding.consumer.lane} is not declared`); + const spec = hwlabRuntimeLaneSpecForNode(binding.consumer.lane, binding.consumer.node); + Object.assign(annotations, { + "hwlab.pikastech.local/ci-contract": "tekton-native-primitive-tasks", + "hwlab.pikastech.local/download-profile": spec.downloadProfileId, + "hwlab.pikastech.local/gitops-branch": spec.gitopsBranch, + "hwlab.pikastech.local/network-profile": spec.networkProfileId, + "hwlab.pikastech.local/node": spec.nodeId, + "hwlab.pikastech.local/policy": "native-per-service-taskrun-image-publish", + "hwlab.pikastech.local/runtime-path": spec.runtimePath, + "hwlab.pikastech.local/source-branch": spec.sourceBranch, + "hwlab.pikastech.local/source-config": binding.consumer.sourceArtifact.pipelineRunPath, + }); + } + return { + apiVersion: "tekton.dev/v1", + kind: "PipelineRun", + metadata: { + generateName: `${binding.consumer.pipelineRunPrefix}-`, + namespace: binding.consumer.namespace, + annotations, + labels: pipelineRunLabels(binding, "hwlab"), + }, + spec: { + timeouts: { pipeline: binding.repository.params.pipeline_timeout }, + taskRunTemplate: taskRunTemplate(binding), + pipelineRef: { name: binding.consumer.pipeline }, + workspaces: pipelineRunWorkspaces(binding, desiredSpec), + params: pipelineRunParams(binding, desiredSpec), + }, + }; +} + +function pipelineRunAnnotations(binding: PacSourceArtifactBinding, provenance: PacSourceArtifactProvenance): Record { + const branch = requiredParam(binding, "source_branch"); + return { + "pipelinesascode.tekton.dev/on-event": "[push]", + "pipelinesascode.tekton.dev/on-target-branch": `[${branch}]`, + "pipelinesascode.tekton.dev/on-cel-expression": `event == 'push' && target_branch == '${branch}' && node == '${binding.consumer.node}'`, + "pipelinesascode.tekton.dev/max-keep-runs": String(binding.consumer.sourceArtifact.maxKeepRuns), + [provenanceKeys.configRef]: provenance.configRef, + [provenanceKeys.effectiveConfigSha256]: provenance.effectiveConfigSha256, + [provenanceKeys.renderer]: provenance.renderer, + [provenanceKeys.mode]: binding.consumer.sourceArtifact.mode, + }; +} + +function pipelineRunLabels(binding: PacSourceArtifactBinding, partOf: "agentrun" | "hwlab"): Record { + return partOf === "agentrun" + ? { + "app.kubernetes.io/part-of": "agentrun", + "agentrun.pikastech.local/lane": "v0.2", + "agentrun.pikastech.local/node": binding.consumer.node, + "agentrun.pikastech.local/source-commit": "{{ revision }}", + "agentrun.pikastech.local/trigger": "pipelines-as-code", + } + : { + "app.kubernetes.io/name": `${binding.consumer.id}-pac`, + "app.kubernetes.io/part-of": "hwlab", + "unidesk.ai/source-commit": "{{ revision }}", + "hwlab.pikastech.local/gitops-target": binding.consumer.lane, + "hwlab.pikastech.local/source-commit": "{{ revision }}", + "hwlab.pikastech.local/trigger": "pipelines-as-code", + }; +} + +function taskRunTemplate(binding: PacSourceArtifactBinding): Record { + const declared = binding.consumer.sourceArtifact.taskRunTemplate; + return { + serviceAccountName: `{{ service_account }}`, + podTemplate: { + hostNetwork: declared.hostNetwork, + dnsPolicy: declared.dnsPolicy, + securityContext: { fsGroup: declared.fsGroup }, + }, + }; +} + +function desiredRuntimePipelineRunWrapper( + binding: PacSourceArtifactBinding, + pipelineRun: Record, + sourceCommit: string, +): Record { + const sourceSpec = structuredClone(requiredRecord(pipelineRun.spec, "generated PipelineRun.spec")); + delete sourceSpec.pipelineSpec; + delete sourceSpec.pipelineRef; + return { + mode: binding.consumer.sourceArtifact.mode, + executionMode: "pipeline-spec", + spec: resolvePacRuntimeTemplates(sourceSpec, binding, sourceCommit), + }; +} + +function resolvePacRuntimeTemplates(value: unknown, binding: PacSourceArtifactBinding, sourceCommit: string): unknown { + if (Array.isArray(value)) return value.map((item) => resolvePacRuntimeTemplates(item, binding, sourceCommit)); + if (isRecord(value)) { + return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, resolvePacRuntimeTemplates(item, binding, sourceCommit)])); + } + if (typeof value !== "string") return value; + const templateValues: Readonly> = { + ...binding.repository.params, + revision: sourceCommit, + repo_url: binding.repository.url, + }; + const resolved = value.replace(/\{\{\s*([A-Za-z0-9_]+)\s*\}\}/gu, (_match, key: string) => { + const replacement = templateValues[key]; + if (replacement === undefined) throw new PacSourceArtifactError("source-artifact-runtime-template-unresolved", { template: pacValueEvidence(key) }); + return replacement; + }); + if (/\{\{|\}\}/u.test(resolved)) throw new PacSourceArtifactError("source-artifact-runtime-template-invalid"); + return resolved; +} + +function pipelineRunParams(binding: PacSourceArtifactBinding, desiredSpec: Record): Record[] { + const params = arrayRecords(desiredSpec.params, "Pipeline.spec.params"); + return params.map((param) => { + const name = requiredString(param.name, "Pipeline.spec.params[].name"); + if (name === "revision" || name === "source-commit") return { name, value: "{{ revision }}" }; + if (name === "source-stage-ref") return { name, value: "{{ source_snapshot_prefix }}/{{ revision }}" }; + if (name === "git-url") return { name, value: "{{ repo_url }}" }; + const repositoryParam = name.replaceAll("-", "_"); + if (Object.prototype.hasOwnProperty.call(binding.repository.params, repositoryParam)) { + return { name, value: `{{ ${repositoryParam} }}` }; + } + if (Object.prototype.hasOwnProperty.call(param, "default")) return { name, value: param.default }; + throw new Error(`Pipeline param ${name} has no default or PaC repository parameter`); + }); +} + +function pipelineRunWorkspaces(binding: PacSourceArtifactBinding, desiredSpec: Record): Record[] { + return arrayRecords(desiredSpec.workspaces, "Pipeline.spec.workspaces").map((workspace) => { + const name = requiredString(workspace.name, "Pipeline.spec.workspaces[].name"); + if (name === "source") { + return { + name, + volumeClaimTemplate: { + spec: { + accessModes: ["ReadWriteOnce"], + resources: { requests: { storage: requiredParam(binding, "workspace_pvc_size") } }, + }, + }, + }; + } + if (name === "git-ssh") return { name, secret: { secretName: requiredParam(binding, "git_ssh_secret") } }; + throw new Error(`unsupported Pipeline workspace ${name}; declare a renderer mapping before generating the source artifact`); + }); +} + +function withPipelineProvenance(pipeline: Record, provenance: PacSourceArtifactProvenance): Record { + const next = structuredClone(pipeline); + const metadata = requiredRecord(next.metadata, "rendered Pipeline.metadata"); + const annotations = isRecord(metadata.annotations) ? metadata.annotations : {}; + metadata.annotations = { + ...annotations, + [provenanceKeys.configRef]: provenance.configRef, + [provenanceKeys.effectiveConfigSha256]: provenance.effectiveConfigSha256, + [provenanceKeys.renderer]: provenance.renderer, + [provenanceKeys.mode]: provenance.mode, + }; + return next; +} + +function inspectSourceArtifact(sourceWorktree: string, binding: PacSourceArtifactBinding, desired: DesiredArtifact): SourceInspection { + const sourceArtifact = binding.consumer.sourceArtifact; + const pipelineRunPath = safeArtifactPath(sourceWorktree, sourceArtifact.pipelineRunPath); + const pipelinePath = sourceArtifact.pipelinePath === null ? null : safeArtifactPath(sourceWorktree, sourceArtifact.pipelinePath); + if (!existsSync(pipelineRunPath) || (pipelinePath !== null && !existsSync(pipelinePath))) { + return { status: "missing", aligned: false, pipelineCanonicalSha256: null, pipelineRunCanonicalSha256: null, firstDrift: drift("$", { artifact: "generated" }, undefined), provenanceAligned: false }; + } + const actualPipelineRun = parseYamlRecord(readFileSync(pipelineRunPath, "utf8"), sourceArtifact.pipelineRunPath); + const actualPipeline = sourceArtifact.mode === "remote-pipeline-annotation" + ? parseYamlRecord(readFileSync(requiredString(pipelinePath, "pipelinePath"), "utf8"), requiredString(sourceArtifact.pipelinePath, "pipelinePath")) + : { + apiVersion: "tekton.dev/v1", + kind: "Pipeline", + metadata: desired.pipeline.metadata, + spec: requiredRecord(requiredRecord(actualPipelineRun.spec, "PipelineRun.spec").pipelineSpec, "PipelineRun.spec.pipelineSpec"), + }; + const pipelineDrift = firstPacSourceArtifactDrift(desired.pipeline, actualPipeline); + const pipelineRunDrift = firstPacSourceArtifactDrift(desired.pipelineRun, actualPipelineRun); + const actualProvenance = sourceArtifact.mode === "remote-pipeline-annotation" + ? provenanceFromManifest(actualPipeline) + : provenanceFromManifest(actualPipelineRun); + const wrapperProvenance = provenanceFromManifest(actualPipelineRun); + const provenanceAligned = provenanceEquals(actualProvenance, desired.provenance) && provenanceEquals(wrapperProvenance, desired.provenance); + const aligned = pipelineDrift === null && pipelineRunDrift === null && provenanceAligned; + return { + status: aligned ? "aligned" : "drifted", + aligned, + pipelineCanonicalSha256: canonicalSha256(requiredRecord(actualPipeline.spec, "source Pipeline.spec")), + pipelineRunCanonicalSha256: canonicalSha256(actualPipelineRun), + firstDrift: pipelineDrift ?? pipelineRunDrift ?? (provenanceAligned ? null : drift("$.metadata.annotations.provenance", desired.provenance, actualProvenance)), + provenanceAligned, + }; +} + +function validateSourceWorktree(input: string, binding: PacSourceArtifactBinding): string { + if (!existsSync(input) || !statSync(input).isDirectory()) throw new Error(`source worktree does not exist or is not a directory: ${input}`); + const worktree = realpathSync(input); + const top = realpathSync(git(worktree, ["rev-parse", "--show-toplevel"])); + if (top !== worktree) throw new Error(`--source-worktree must be the git worktree root; resolved ${top}`); + const remote = git(worktree, ["remote", "get-url", "origin"]); + const expectedRemote = owningSourceRemote(binding); + const expectedIdentity = remoteRepositoryIdentity(expectedRemote); + const observedIdentity = remoteRepositoryIdentity(remote); + if (expectedIdentity === null || observedIdentity === null + || expectedIdentity.host !== observedIdentity.host + || expectedIdentity.ownerRepo !== observedIdentity.ownerRepo) { + throw new PacSourceArtifactError("source-worktree-origin-mismatch", { + expectedUrlFingerprint: fingerprint(expectedRemote), + observedUrlFingerprint: fingerprint(remote), + }); + } + safeArtifactPath(worktree, binding.consumer.sourceArtifact.pipelineRunPath); + if (binding.consumer.sourceArtifact.pipelinePath !== null) safeArtifactPath(worktree, binding.consumer.sourceArtifact.pipelinePath); + return worktree; +} + +function owningSourceRemote(binding: PacSourceArtifactBinding): string { + if (binding.consumer.sourceArtifact.renderer === "agentrun-control-plane") { + return resolveAgentRunLaneTarget({ node: binding.consumer.node, lane: binding.consumer.lane }).spec.source.worktreeRemote; + } + if (!isHwlabRuntimeLane(binding.consumer.lane)) throw new PacSourceArtifactError("owning-source-lane-unresolved"); + return hwlabRuntimeLaneSpecForNode(binding.consumer.lane, binding.consumer.node).gitUrl; +} + +function remoteRepositoryIdentity(remote: string): { readonly host: string; readonly ownerRepo: string } | null { + const trimmed = remote.trim(); + const scp = /^[^/@\s]+@([^:\s]+):(.+)$/u.exec(trimmed); + let host = scp?.[1] ?? ""; + let path = scp?.[2] ?? ""; + if (host.length === 0 || path.length === 0) { + try { + const parsed = new URL(trimmed); + if (parsed.username.length > 0 || parsed.password.length > 0 || parsed.search.length > 0 || parsed.hash.length > 0) return null; + host = parsed.hostname; + path = parsed.pathname; + } catch { + return null; + } + } + const segments = path.replace(/^\/+|\/+$/gu, "").split("/").filter(Boolean); + if (segments.length !== 2) return null; + const owner = segments[0]?.toLowerCase(); + const repo = segments[1]?.replace(/\.git$/iu, "").toLowerCase(); + if (!host || !owner || !repo || !/^[a-z0-9_.-]+$/u.test(owner) || !/^[a-z0-9_.-]+$/u.test(repo)) return null; + return { host: host.toLowerCase(), ownerRepo: `${owner}/${repo}` }; +} + +function writeArtifactFilesAtomically(sourceWorktree: string, files: readonly { readonly path: string; readonly content: string }[]): string[] { + const pending = files.map((file) => ({ ...file, output: safeArtifactPath(sourceWorktree, file.path) })) + .filter((file) => !existsSync(file.output) || readFileSync(file.output, "utf8") !== file.content); + if (pending.length === 0) return []; + for (const file of pending) { + mkdirSync(dirname(file.output), { recursive: true }); + safeArtifactPath(sourceWorktree, file.path); + } + const token = `${process.pid}-${Date.now()}`; + const staged = pending.map((file, index) => ({ ...file, temporary: `${file.output}.unidesk-${token}-${index}.tmp` })); + try { + for (const file of staged) writeFileSync(file.temporary, file.content, { flag: "wx" }); + for (const file of staged) renameSync(file.temporary, file.output); + } finally { + for (const file of staged) rmSync(file.temporary, { force: true }); + } + return pending.map((file) => file.path); +} + +function safeArtifactPath(worktree: string, relativePath: string): string { + if (relativePath.length === 0 || isAbsolute(relativePath) || relativePath.split(/[\\/]/u).includes("..")) { + throw new Error(`source artifact path must be a non-empty worktree-relative path without ..: ${relativePath}`); + } + const root = realpathSync(worktree); + const candidate = resolve(root, relativePath); + if (candidate !== root && !candidate.startsWith(`${root}${sep}`)) throw new Error(`source artifact path escapes worktree: ${relativePath}`); + let current = root; + for (const segment of relativePath.split(/[\\/]/u).filter(Boolean)) { + current = resolve(current, segment); + if (!existsSync(current)) continue; + if (lstatSync(current).isSymbolicLink()) throw new Error(`source artifact path crosses symlink: ${relativePath}`); + const resolved = realpathSync(current); + if (resolved !== root && !resolved.startsWith(`${root}${sep}`)) throw new Error(`source artifact path resolves outside worktree: ${relativePath}`); + } + return candidate; +} + +function assertConfigRef(actual: string, expected: string): void { + if (actual !== expected) throw new Error(`sourceArtifact.configRef must equal resolved owning selector ${expected}; observed ${actual}`); + const [file, selector] = actual.split("#", 2); + if (file === undefined || selector === undefined || selector.length === 0) throw new Error(`invalid sourceArtifact.configRef: ${actual}`); + const parsed = Bun.YAML.parse(readFileSync(rootPath(...file.split("/")), "utf8")) as unknown; + let value = parsed; + for (const segment of selector.split(".")) { + if (!isRecord(value) || !Object.prototype.hasOwnProperty.call(value, segment)) throw new Error(`sourceArtifact.configRef selector does not exist: ${actual}`); + value = value[segment]; + } + if (!isRecord(value)) throw new Error(`sourceArtifact.configRef must resolve to an owning YAML object: ${actual}`); +} + +function assertBindingIdentity(binding: PacSourceArtifactBinding, expected: { + node: string; + lane: string; + namespace: string; + pipeline: string; + pipelineRunPrefix: string; + serviceAccount: string; + sourceBranch: string; + gitopsBranch: string; +}): void { + const observed = { + node: binding.consumer.node, + lane: binding.consumer.lane, + namespace: binding.consumer.namespace, + pipeline: binding.consumer.pipeline, + pipelineRunPrefix: binding.consumer.pipelineRunPrefix, + serviceAccount: requiredParam(binding, "service_account"), + sourceBranch: requiredParam(binding, "source_branch"), + gitopsBranch: requiredParam(binding, "gitops_branch"), + }; + for (const key of Object.keys(expected) as Array) { + if (observed[key] !== expected[key]) throw new Error(`PaC ${binding.consumer.id} ${key}=${observed[key]} does not match owning renderer ${expected[key]}`); + } + if (requiredParam(binding, "pipeline_name") !== expected.pipeline) throw new Error(`PaC ${binding.consumer.id} repository pipeline_name does not match ${expected.pipeline}`); + if (requiredParam(binding, "pipeline_run_prefix") !== expected.pipelineRunPrefix) throw new Error(`PaC ${binding.consumer.id} repository pipeline_run_prefix does not match ${expected.pipelineRunPrefix}`); + if (requiredParam(binding, "node") !== expected.node) throw new Error(`PaC ${binding.consumer.id} repository node does not match ${expected.node}`); +} + +function assertPipelineIdentity(pipeline: Record, binding: PacSourceArtifactBinding): void { + if (pipeline.apiVersion !== "tekton.dev/v1" || pipeline.kind !== "Pipeline") throw new Error("domain renderer must return tekton.dev/v1 Pipeline"); + const metadata = requiredRecord(pipeline.metadata, "rendered Pipeline.metadata"); + if (metadata.name !== binding.consumer.pipeline) throw new Error(`rendered Pipeline name ${String(metadata.name)} does not match consumer ${binding.consumer.pipeline}`); + if (metadata.namespace !== binding.consumer.namespace) throw new Error(`rendered Pipeline namespace ${String(metadata.namespace)} does not match consumer ${binding.consumer.namespace}`); +} + +function provenanceFromManifest(manifest: Record): PacSourceArtifactProvenance | null { + const metadata = isRecord(manifest.metadata) ? manifest.metadata : {}; + const annotations = isRecord(metadata.annotations) ? metadata.annotations : {}; + const configRef = annotations[provenanceKeys.configRef]; + const effectiveConfigSha256 = annotations[provenanceKeys.effectiveConfigSha256]; + const renderer = annotations[provenanceKeys.renderer]; + const mode = annotations[provenanceKeys.mode]; + if (typeof configRef !== "string" || typeof effectiveConfigSha256 !== "string" || (renderer !== "agentrun-control-plane" && renderer !== "hwlab-runtime-lane") || (mode !== "embedded-pipeline-spec" && mode !== "remote-pipeline-annotation")) return null; + return { configRef, effectiveConfigSha256, renderer, mode }; +} + +function provenanceEquals(actual: PacSourceArtifactProvenance | null, expected: PacSourceArtifactProvenance): boolean { + return actual !== null + && actual.configRef === expected.configRef + && actual.effectiveConfigSha256 === expected.effectiveConfigSha256 + && actual.renderer === expected.renderer + && actual.mode === expected.mode; +} + +function firstCanonicalDrift(expected: unknown, actual: unknown, path: string): PacSourceArtifactDrift | null { + if (Object.is(expected, actual)) return null; + if (Array.isArray(expected) || Array.isArray(actual)) { + if (!Array.isArray(expected) || !Array.isArray(actual)) return drift(path, expected, actual); + if (expected.length !== actual.length) return drift(`${path}.length`, expected.length, actual.length); + for (let index = 0; index < expected.length; index += 1) { + const child = firstCanonicalDrift(expected[index], actual[index], `${path}[${index}]`); + if (child !== null) return child; + } + return null; + } + if (isRecord(expected) || isRecord(actual)) { + if (!isRecord(expected) || !isRecord(actual)) return drift(path, expected, actual); + const keys = [...new Set([...Object.keys(expected), ...Object.keys(actual)])].sort(); + for (const key of keys) { + const expectedOwnsKey = Object.prototype.hasOwnProperty.call(expected, key); + const actualOwnsKey = Object.prototype.hasOwnProperty.call(actual, key); + const childPath = knownDriftPathKeys.has(key) ? `${path}.${key}` : `${path}.[key:${pacValueEvidence(key)}]`; + if (!expectedOwnsKey || !actualOwnsKey) return drift(childPath, expected[key], actual[key]); + const child = firstCanonicalDrift(expected[key], actual[key], childPath); + if (child !== null) return child; + } + return null; + } + return drift(path, expected, actual); +} + +function isProvenTektonAdmissionDefault(path: readonly (string | number)[], value: unknown): boolean { + const normalized = path.map((segment) => typeof segment === "number" ? "*" : segment).join("."); + const emptyRecord = isRecord(value) && Object.keys(value).length === 0; + const atTaskSpec = (suffix: string): boolean => [ + `tasks.*.taskSpec.${suffix}`, + `spec.tasks.*.taskSpec.${suffix}`, + `spec.pipelineSpec.tasks.*.taskSpec.${suffix}`, + ].includes(normalized); + if (atTaskSpec("metadata")) return emptyRecord; + if (atTaskSpec("spec")) return value === null; + if (atTaskSpec("params.*.type")) return value === "string"; + if (atTaskSpec("results.*.type")) return value === "string"; + if (atTaskSpec("steps.*.computeResources")) return emptyRecord; + if (atTaskSpec("sidecars.*.computeResources")) return emptyRecord; + return false; +} + +function drift(path: string, expected: unknown, actual: unknown): PacSourceArtifactDrift { + return { path, expected: pacValueEvidence(expected), actual: pacValueEvidence(actual) }; +} + +export function pacValueEvidence(value: unknown): string { + const type = value === null ? "null" : Array.isArray(value) ? "array" : typeof value; + const stable = stableJsonValue(value); + const serialized = JSON.stringify(stable) ?? String(stable); + const length = typeof value === "string" + ? Buffer.byteLength(value, "utf8") + : Array.isArray(value) + ? value.length + : isRecord(value) + ? Object.keys(value).length + : Buffer.byteLength(serialized, "utf8"); + return `type=${type},length=${length},sha256=${fingerprint(serialized).replace(/^sha256:/u, "")}`; +} + +function fingerprint(value: string): string { + return `sha256:${createHash("sha256").update(value).digest("hex")}`; +} + +function formatDrift(value: Record | null): string { + return value === null ? "-" : `${text(value.path)} expected:${text(value.expected)} actual:${text(value.actual)}`; +} + +function unavailableRuntimeObservation(): PacSourceArtifactRuntimeObservation { + const item: PacSourceArtifactRuntimeItem = { status: "unavailable", name: null, canonicalSha256: null, firstDrift: null, provenanceAligned: false, configRef: null, effectiveConfigSha256: null, sourceCommit: null, reason: "runtime-observer-not-configured", candidateCount: 0 }; + return { live: item, embedded: item }; +} + +function parseYamlRecord(text: string, label: string): Record { + const parsed = Bun.YAML.parse(text) as unknown; + if (!isRecord(parsed)) throw new Error(`${label} must contain one YAML object`); + return parsed; +} + +export function pacSourceArtifactYaml(value: Record): string { + return `${Bun.YAML.stringify(value, null, 2).trim()}\n`; +} + +function requiredParam(binding: PacSourceArtifactBinding, key: string): string { + return requiredString(binding.repository.params[key], `PaC repository ${binding.repository.id} params.${key}`); +} + +function requiredString(value: unknown, label: string): string { + if (typeof value !== "string" || value.length === 0) throw new Error(`${label} must be a non-empty string`); + return value; +} + +function requiredRecord(value: unknown, label: string): Record { + if (!isRecord(value)) throw new Error(`${label} must be an object`); + return value; +} + +function arrayRecords(value: unknown, label: string): Record[] { + if (!Array.isArray(value) || value.some((item) => !isRecord(item))) throw new Error(`${label} must be an array of objects`); + return value as Record[]; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function record(value: unknown): Record { + return isRecord(value) ? value : {}; +} + +function isSourceArtifactAction(value: string | undefined): value is PacSourceArtifactAction { + return value === "plan" || value === "check" || value === "write" || value === "status" || value === "verify-runtime"; +} + +function git(worktree: string, args: readonly string[]): string { + const result = spawnSync("git", ["-C", worktree, ...args], { encoding: "utf8", timeout: 15_000, maxBuffer: 1024 * 1024 }); + if (result.status !== 0) throw new Error(`git ${args.join(" ")} failed for ${worktree}: ${(result.stderr || result.stdout).trim().slice(0, 1000)}`); + return result.stdout.trim(); +} + +function runChecked(command: string, args: readonly string[], cwd: string, timeout: number, label: string): void { + const result = spawnSync(command, [...args], { cwd, encoding: "utf8", timeout, maxBuffer: 32 * 1024 * 1024 }); + if (result.status !== 0) throw new Error(`${label} failed: ${(result.stderr || result.stdout).trim().slice(-4000)}`); +} + +function shellQuote(value: string): string { + return `'${value.replaceAll("'", `'"'"'`)}'`; +} + +function text(value: unknown): string { + if (value === null || value === undefined || value === "") return "-"; + return String(value); +} + +function shortSha(value: unknown): string { + return typeof value === "string" && value.length > 0 ? value.replace(/^sha256:/u, "").slice(0, 12) : "-"; +} diff --git a/scripts/src/platform-infra-pipelines-as-code.ts b/scripts/src/platform-infra-pipelines-as-code.ts index 841363c1..5293cd28 100644 --- a/scripts/src/platform-infra-pipelines-as-code.ts +++ b/scripts/src/platform-infra-pipelines-as-code.ts @@ -1,4 +1,4 @@ -import { createHash, randomBytes } from "node:crypto"; +import { randomBytes } from "node:crypto"; import { chmodSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import type { UniDeskConfig } from "./config"; @@ -17,11 +17,23 @@ import { sha256Fingerprint, } from "./platform-infra-ops-library"; import { materializeYamlComposition } from "./yaml-composition"; +import { + canonicalizePacPipelineSpec, + pacSourceArtifactSafeError, + pacValueEvidence, + parsePacSourceArtifactOptions, + renderPacSourceArtifactResult, + runPacSourceArtifact, + type PacSourceArtifactBinding, + type PacSourceArtifactRuntimeObservation, + type PacSourceArtifactSpec, +} from "./platform-infra-pipelines-as-code-source-artifact"; const configFile = rootPath("config", "platform-infra", "pipelines-as-code.yaml"); const configLabel = "config/platform-infra/pipelines-as-code.yaml"; const remoteScriptFile = rootPath("scripts", "src", "platform-infra-pipelines-as-code-remote.sh"); const evaluatorFile = rootPath("scripts", "native", "cicd", "pac-status-evaluator.cjs"); +const sourceArtifactRuntimeObserverFile = rootPath("scripts", "native", "cicd", "pac-source-artifact-runtime.mjs"); const fieldManager = "unidesk-platform-infra-pipelines-as-code"; const y = createYamlFieldReader(configLabel); @@ -132,6 +144,7 @@ interface PacConsumer { repositoryRef: string; closeoutGitOpsMirrorFlush: boolean; closeoutGitOpsMirrorLane: "v02" | "v03" | null; + sourceArtifact: PacSourceArtifactSpec | null; } interface CommonOptions { @@ -216,12 +229,87 @@ export async function runPlatformInfraPipelinesAsCodeCommand(config: UniDeskConf const result = await webhookTest(config, options); return options.json || options.full || options.raw ? result : renderWebhookTest(result); } + if (action === "source-artifact") { + if (args.length === 1 || args.slice(1).includes("--help") || args.slice(1).includes("-h")) return sourceArtifactHelp(); + const structuredError = args.includes("--json") || args.includes("--full"); + try { + const options = parsePacSourceArtifactOptions(args.slice(1)); + const pac = readPacConfig(); + const target = resolveTarget(pac, options.targetId); + const consumer = resolveConsumer(pac, options.consumerId); + if (consumer.node.toLowerCase() !== target.id.toLowerCase()) throw new Error(`Pipelines-as-Code consumer ${consumer.id} belongs to ${consumer.node}, not target ${target.id}`); + if (consumer.sourceArtifact === null) throw new Error(`Pipelines-as-Code consumer ${consumer.id} has no sourceArtifact owner in ${configLabel}`); + const repository = resolveRepository(pac, consumer.repositoryRef); + const binding: PacSourceArtifactBinding = { + target: { id: target.id }, + consumer: { + id: consumer.id, + node: consumer.node, + lane: consumer.lane, + namespace: consumer.namespace, + pipeline: consumer.pipeline, + pipelineRunPrefix: consumer.pipelineRunPrefix, + sourceArtifact: consumer.sourceArtifact, + }, + repository: { + id: repository.id, + url: repository.url, + cloneUrl: repository.cloneUrl, + owner: repository.owner, + repo: repository.repo, + params: repository.params, + }, + }; + const result = await runPacSourceArtifact(binding, options, async ({ desiredSpec, desiredWrapper, provenance, sourceCommit }) => await observeSourceArtifactRuntime(config, target, consumer, desiredSpec, desiredWrapper, provenance, sourceCommit)); + return options.json || options.full ? result : renderPacSourceArtifactResult(result); + } catch (error) { + const safeError = pacSourceArtifactSafeError(error); + if (structuredError) { + return { + ok: false, + action: "platform-infra-pipelines-as-code-source-artifact-validation", + error: safeError, + next: "Fix the declared owner, exact source identity, worktree proof, or CLI arguments; rerun plan before write.", + }; + } + return sourceArtifactValidationError(safeError); + } + } return { ok: false, error: "unsupported-platform-infra-pipelines-as-code-command", args, help: help() }; } +function sourceArtifactHelp(): RenderedCliResult { + const lines = [ + "PAC SOURCE ARTIFACT", + "Generate and verify consumer-declared PaC source artifacts from owning YAML plus the shared domain renderer.", + "", + "Usage:", + " ... source-artifact plan --target --consumer --source-worktree ", + " ... source-artifact check --target --consumer --source-worktree ", + " ... source-artifact write --target --consumer --source-worktree --confirm", + " ... source-artifact status --target --consumer --source-worktree [--source-commit ]", + " ... source-artifact verify-runtime --target --consumer --source-worktree --source-commit ", + "", + "plan/check/write compare desired YAML-rendered state with source files and never access runtime.", + "status is non-gating runtime diagnostics; verify-runtime is fail-closed and binds the PipelineRun to the exact source commit.", + "Use --json or --full for explicit structured output; source specs and Secret values are never printed.", + ]; + return { ok: true, command: "platform-infra-pipelines-as-code-source-artifact-help", renderedText: `${lines.join("\n")}\n`, contentType: "text/plain" }; +} + +function sourceArtifactValidationError(error: ReturnType): RenderedCliResult { + const details = Object.entries(error.details).map(([key, value]) => `${key}=${value}`).join(" "); + return { + ok: false, + command: "platform-infra-pipelines-as-code-source-artifact-validation", + renderedText: `PAC SOURCE ARTIFACT ERROR\ncode=${error.code} evidence=${error.evidence}${details ? ` details=${details}` : ""}\nnext=Fix the declared owner, exact source identity, worktree proof, or CLI arguments; rerun plan before write.\n`, + contentType: "text/plain", + }; +} + function help(): Record { return { - command: "platform-infra pipelines-as-code plan|apply|status|closeout|history|debug-step", + command: "platform-infra pipelines-as-code plan|apply|status|closeout|history|debug-step|source-artifact", configTruth: configLabel, usage: [ "bun scripts/cli.ts platform-infra pipelines-as-code plan --target JD01", @@ -232,6 +320,9 @@ function help(): Record { "bun scripts/cli.ts platform-infra pipelines-as-code history --target JD01 [--consumer hwlab-jd01-v03] [--limit 10]", "bun scripts/cli.ts platform-infra pipelines-as-code history --target JD01 --id ", "bun scripts/cli.ts platform-infra pipelines-as-code debug-step --target JD01 [--consumer ] [--id ] [--json]", + "bun scripts/cli.ts platform-infra pipelines-as-code source-artifact check --target NC01 --consumer agentrun-nc01-v02 --source-worktree /abs/worktree", + "bun scripts/cli.ts platform-infra pipelines-as-code source-artifact write --target NC01 --consumer agentrun-nc01-v02 --source-worktree /abs/worktree --confirm", + "bun scripts/cli.ts platform-infra pipelines-as-code source-artifact verify-runtime --target NC01 --consumer agentrun-nc01-v02 --source-worktree /abs/worktree --source-commit ", ], diagnostics: "webhook-test exists only for bounded connectivity diagnosis and must not be used as delivery evidence.", boundary: "Sole CI trigger path for GH-1552/GH-1607: GitHub PR merge -> GitHub webhook bridge -> Gitea mirror/snapshot -> Pipelines-as-Code -> Tekton -> GitOps/Argo/k8s runtime.", @@ -353,9 +444,203 @@ function parseConsumer(consumer: Record, path: string, defaultR repositoryRef: typeof consumer.repositoryRef === "string" && consumer.repositoryRef.length > 0 ? consumer.repositoryRef : defaultRepositoryRef, closeoutGitOpsMirrorFlush: y.booleanField(consumer, "closeoutGitOpsMirrorFlush", path), closeoutGitOpsMirrorLane: consumer.closeoutGitOpsMirrorLane === undefined ? null : y.enumField(consumer, "closeoutGitOpsMirrorLane", path, ["v02", "v03"] as const), + sourceArtifact: consumer.sourceArtifact === undefined ? null : parseSourceArtifact(y.objectField(consumer, "sourceArtifact", path), `${path}.sourceArtifact`), }; } +function parseSourceArtifact(value: Record, path: string): PacSourceArtifactSpec { + const mode = y.enumField(value, "mode", path, ["embedded-pipeline-spec", "remote-pipeline-annotation"] as const); + const renderer = y.enumField(value, "renderer", path, ["agentrun-control-plane", "hwlab-runtime-lane"] as const); + const pipelineRunPath = sourceArtifactPath(y.stringField(value, "pipelineRunPath", path), `${path}.pipelineRunPath`); + const pipelinePath = value.pipelinePath === undefined ? null : sourceArtifactPath(y.stringField(value, "pipelinePath", path), `${path}.pipelinePath`); + const taskRunTemplate = y.objectField(value, "taskRunTemplate", path); + if (mode === "embedded-pipeline-spec" && pipelinePath !== null) throw new Error(`${path}.pipelinePath is forbidden for embedded-pipeline-spec`); + if (mode === "remote-pipeline-annotation" && pipelinePath === null) throw new Error(`${path}.pipelinePath is required for remote-pipeline-annotation`); + if (renderer === "agentrun-control-plane" && mode !== "embedded-pipeline-spec") throw new Error(`${path}.renderer agentrun-control-plane requires embedded-pipeline-spec`); + if (renderer === "hwlab-runtime-lane" && mode !== "remote-pipeline-annotation") throw new Error(`${path}.renderer hwlab-runtime-lane requires remote-pipeline-annotation`); + return { + mode, + renderer, + configRef: y.stringField(value, "configRef", path), + pipelineRunPath, + pipelinePath, + maxKeepRuns: positiveInteger(value, "maxKeepRuns", path), + taskRunTemplate: { + hostNetwork: y.booleanField(taskRunTemplate, "hostNetwork", `${path}.taskRunTemplate`), + dnsPolicy: y.enumField(taskRunTemplate, "dnsPolicy", `${path}.taskRunTemplate`, ["ClusterFirst", "ClusterFirstWithHostNet", "Default", "None"] as const), + fsGroup: positiveInteger(taskRunTemplate, "fsGroup", `${path}.taskRunTemplate`), + }, + }; +} + +function sourceArtifactPath(value: string, path: string): string { + if (value.startsWith("/") || value.split(/[\\/]/u).includes("..") || !value.endsWith(".yaml")) throw new Error(`${path} must be a worktree-relative .yaml path without ..`); + return value; +} + +async function observeSourceArtifactRuntime( + config: UniDeskConfig, + target: PacTarget, + consumer: PacConsumer, + desiredSpec: Record, + desiredWrapper: Record | null, + provenance: { readonly configRef: string; readonly effectiveConfigSha256: string; readonly renderer: string; readonly mode: string }, + sourceCommit: string | null, +): Promise { + const script = renderPacSourceArtifactRuntimeObserverShell({ + namespace: consumer.namespace, + pipeline: consumer.pipeline, + pipelineRunPrefix: consumer.pipelineRunPrefix, + desiredSpec: canonicalizePacPipelineSpec(desiredSpec), + desiredWrapper: desiredWrapper === null ? null : canonicalizePacPipelineSpec(desiredWrapper), + provenance, + sourceCommit, + }); + + const captured = await capture(config, target.route, ["sh"], script); + if (captured.exitCode !== 0) { + return unavailableSourceArtifactRuntime(pacRuntimeTransportFailureReason(captured), sourceCommit); + } + const parsed = parseJsonOutput(captured.stdout); + if (parsed === null) return unavailableSourceArtifactRuntime("runtime-observer-invalid-json", sourceCommit); + return { + live: pacRuntimeSourceArtifactItem(parsed.live, { + configRef: provenance.configRef, + effectiveConfigSha256: provenance.effectiveConfigSha256, + renderer: provenance.renderer, + mode: provenance.mode, + sourceCommit: null, + exactName: consumer.pipeline, + namePrefix: null, + }), + embedded: pacRuntimeSourceArtifactItem(parsed.embedded, { + configRef: provenance.configRef, + effectiveConfigSha256: provenance.effectiveConfigSha256, + renderer: provenance.renderer, + mode: provenance.mode, + sourceCommit, + exactName: null, + namePrefix: consumer.pipelineRunPrefix, + }), + }; +} + +export function pacRuntimeTransportFailureReason(captured: { readonly exitCode: number; readonly stdout: string; readonly stderr: string }): string { + return `runtime-transport-exit-${captured.exitCode}:stderr(${pacValueEvidence(captured.stderr)}):stdout(${pacValueEvidence(captured.stdout)})`; +} + +export function renderPacSourceArtifactRuntimeObserverShell(inputValue: Record): string { + const input = JSON.stringify(inputValue); + const observer = readFileSync(sourceArtifactRuntimeObserverFile, "utf8").trimEnd(); + if (observer.includes("NODE_PAC_SOURCE_ARTIFACT_STATUS") || input.includes("JSON_PAC_SOURCE_ARTIFACT_INPUT")) throw new Error("PaC source artifact observer input contains a reserved heredoc delimiter"); + return `set -eu +observer_input=$(mktemp) +trap 'rm -f "$observer_input"' EXIT HUP INT TERM +cat >"$observer_input" <<'JSON_PAC_SOURCE_ARTIFACT_INPUT' +${input} +JSON_PAC_SOURCE_ARTIFACT_INPUT +PAC_SOURCE_ARTIFACT_INPUT_FILE="$observer_input" node --input-type=module <<'NODE_PAC_SOURCE_ARTIFACT_STATUS' +${observer} +NODE_PAC_SOURCE_ARTIFACT_STATUS +`; +} + +export function pacRuntimeSourceArtifactItem(value: unknown, expected: { + readonly configRef: string; + readonly effectiveConfigSha256: string; + readonly renderer: string; + readonly mode: string; + readonly sourceCommit: string | null; + readonly exactName: string | null; + readonly namePrefix: string | null; +}): PacSourceArtifactRuntimeObservation["live"] { + if (typeof value !== "object" || value === null || Array.isArray(value)) return unavailableSourceArtifactRuntime("runtime-observer-invalid-item", null).live; + const item = value as Record; + const status = item.status; + if (status !== "aligned" && status !== "drifted" && status !== "missing" && status !== "unavailable") return unavailableSourceArtifactRuntime("runtime-observer-invalid-status", null).live; + const drift = item.firstDrift; + const observedName = typeof item.name === "string" && item.name.length <= 253 && /^[a-z0-9](?:[-a-z0-9.]*[a-z0-9])?$/u.test(item.name) ? item.name : null; + const name = observedName !== null + && (expected.exactName === null || observedName === expected.exactName) + && (expected.namePrefix === null || observedName.startsWith(expected.namePrefix)) + ? observedName + : null; + const provenanceAligned = item.configRef === expected.configRef + && item.effectiveConfigSha256 === expected.effectiveConfigSha256 + && item.renderer === expected.renderer + && item.mode === expected.mode; + return { + status, + name, + canonicalSha256: typeof item.canonicalSha256 === "string" && /^sha256:[0-9a-f]{64}$/u.test(item.canonicalSha256) ? item.canonicalSha256 : null, + firstDrift: typeof drift === "object" && drift !== null && !Array.isArray(drift) + ? { + path: pacRuntimeSafePath((drift as Record).path), + expected: pacRuntimeSafeEvidence((drift as Record).expected), + actual: pacRuntimeSafeEvidence((drift as Record).actual), + } + : null, + provenanceAligned, + configRef: item.configRef === expected.configRef && expected.configRef.length <= 512 && /^[A-Za-z0-9_./-]+#[A-Za-z0-9_.-]+$/u.test(expected.configRef) ? expected.configRef : null, + effectiveConfigSha256: item.effectiveConfigSha256 === expected.effectiveConfigSha256 && /^sha256:[0-9a-f]{64}$/u.test(expected.effectiveConfigSha256) ? expected.effectiveConfigSha256 : null, + sourceCommit: item.sourceCommit === expected.sourceCommit && (expected.sourceCommit === null || /^[0-9a-f]{40}$/u.test(expected.sourceCommit)) ? expected.sourceCommit : null, + reason: typeof item.reason === "string" ? pacRuntimeSafeReason(item.reason) : null, + candidateCount: typeof item.candidateCount === "number" && Number.isInteger(item.candidateCount) && item.candidateCount >= 0 ? item.candidateCount : 0, + }; +} + +export function pacRuntimeSafePath(value: unknown): string { + const path = typeof value === "string" ? value : "$"; + const allowedSegments = new Set([ + "accessModes", "annotations", "apiVersion", "args", "command", "computeResources", "default", "description", "dnsPolicy", "env", "envFrom", + "executionMode", "finally", "fsGroup", "generateName", "hostNetwork", "image", "kind", "labels", "length", "metadata", "mode", "mountPath", "name", "namespace", "onError", "operator", + "optional", "params", "pipeline", "pipelineRef", "pipelineSpec", "podTemplate", "readinessProbe", "readOnly", "requests", "resources", + "results", "retries", "runAfter", "script", "secret", "secretName", "securityContext", "serviceAccountName", "sidecars", "spec", "steps", + "storage", "subPath", "taskRunTemplate", "taskSpec", "tasks", "timeout", "timeouts", "type", "value", "values", "volumeClaimTemplate", + "volumeMounts", "volumes", "when", "workspaces", "workingDir", "wrapper", + ]); + if (!path.startsWith("$")) return `path(${pacValueEvidence(path)})`; + const token = /(?:\.([A-Za-z0-9_-]+))|(?:\[(\d+)\])/gu; + let offset = 1; + for (const match of path.slice(1).matchAll(token)) { + if (match.index !== offset - 1) return `path(${pacValueEvidence(path)})`; + if (match[1] !== undefined && !allowedSegments.has(match[1])) return `path(${pacValueEvidence(path)})`; + offset = 1 + (match.index ?? 0) + match[0].length; + } + return offset === path.length ? path : `path(${pacValueEvidence(path)})`; +} + +function pacRuntimeSafeEvidence(value: unknown): string { + return typeof value === "string" && /^type=[a-z]+,length=\d+,sha256=[0-9a-f]{64}$/u.test(value) + ? value + : pacValueEvidence(value); +} + +export function pacRuntimeSafeReason(value: string): string { + const fixed = new Set([ + "source-commit-not-specified", + "source-commit-run-not-found", + "source-commit-identity-conflict", + "pipeline-get-not-found", + "pipelinerun-list-not-found", + "pipelinerun-list-invalid-shape", + "runtime-observer-invalid-json", + "runtime-observer-invalid-item", + "runtime-observer-invalid-status", + "runtime-observer-not-configured", + ]); + if (fixed.has(value)) return value; + if (/^source-commit-run-conflict:\d+$/u.test(value)) return value; + if (/^(?:pipeline-get|pipelinerun-list)-command-failed:(?:unknown|\d+):type=string,length=\d+,sha256=[0-9a-f]{64}$/u.test(value)) return value; + if (/^(?:runtime-observer-input-error|runtime-observer-reason):type=string,length=\d+,sha256=[0-9a-f]{64}$/u.test(value)) return value; + return `runtime-observer-reason:${pacValueEvidence(value)}`; +} + +function unavailableSourceArtifactRuntime(reason: string, sourceCommit: string | null): PacSourceArtifactRuntimeObservation { + const item = { status: "unavailable" as const, name: null, canonicalSha256: null, firstDrift: null, provenanceAligned: false, configRef: null, effectiveConfigSha256: null, sourceCommit, reason, candidateCount: 0 }; + return { live: item, embedded: item }; +} + function parseTarget(record: Record, index: number): PacTarget { const path = `targets[${index}]`; return { diff --git a/scripts/src/stable-json.ts b/scripts/src/stable-json.ts new file mode 100644 index 00000000..3217f1b4 --- /dev/null +++ b/scripts/src/stable-json.ts @@ -0,0 +1,12 @@ +import { createHash } from "node:crypto"; + +export function stableJsonValue(value: unknown): unknown { + if (Array.isArray(value)) return value.map(stableJsonValue); + if (typeof value !== "object" || value === null) return value; + const input = value as Record; + return Object.fromEntries(Object.keys(input).sort().map((key) => [key, stableJsonValue(input[key])])); +} + +export function stableJsonSha256(value: unknown): string { + return `sha256:${createHash("sha256").update(JSON.stringify(stableJsonValue(value))).digest("hex")}`; +}