mirrors/pikasTech-unidesk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into feat/pikaoa-platform

Browse Source 
# Conflicts:
#	config/secrets-distribution.yaml
#	docs/MDTODO/agentrun-runtime-reliability.md
#	scripts/src/platform-infra-pipelines-as-code.ts
This commit is contained in:
Codex
2026-07-14 11:09:30 +02:00
parent c4668ce3c8 1f57332190
commit e36a50a114
79 changed files with 3469 additions and 360 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/unidesk-cicd/SKILL.md
+12
View File
@@ -119,6 +119,18 @@ bun scripts/cli.ts hwlab nodes control-plane legacy-cicd --help
- 未迁移的 legacy CI/CD 若仍保留人工入口，必须由单一受控命令完成 source sync、构建、发布、GitOps/Argo 收敛、runtime provenance 校验和 `/health` 端点验证。PaC migrated consumer 不得使用该人工入口，其唯一交付触发是 GitHub PR merge。
- CI/CD 端到端 wall-clock 目标及预算只由 owning YAML 声明；PaC migrated consumer 从 GitHub PR merge 事件计时，legacy lane 才从操作者触发受控命令计时，均到 runtime ready 且 `/health` 端点验证完成为止。
- CI/CD validation 阶段只能验证部署对象的 `/health` 端点和必要 provenance；禁止在 CI/CD gate 中运行 web-probe、Playwright、远程浏览器截图、用户路径 E2E 或等价重型业务探针。业务/用户入口验证只能作为发布后的独立 post-deploy validation 证据，不得阻塞 CI/CD 一键交付。
- 产品功能配置 schema 的 CI/CD 合同：
- repo 内唯一约定路径固定为 `config/feature-config.schema.json`；
- schema 固定使用 JSON Schema Draft 2020-12，并由 lockfile 锁定的完整结构化 validator 校验；禁止用正则、浅层字段遍历或关闭整体 strict 代替；
- schema 只描述产品功能配置，不得承载数据 authority、事件路径、投影实现或交付路径选择；
- 每个顶层 `properties` 成员必须声明唯一非空 `x-unidesk-feature`；一个产品功能只允许一个规范配置 property，禁止别名、反向变量或重复开关；
- candidate snapshot 必须由正常 source-artifact 阶段从 owning YAML 或已渲染 workload 显式生成；禁止读取 runner `process.env` 作为产品配置 authority；
- HWLAB、AgentRun 与 Sentinel consumer 必须复用同一 shared helper，并在正常 source-artifact/GitOps publish 路径执行，不得只挂到某个 consumer 的专属后处理；
- CI/CD 必须检查 schema 语法、嵌套配置值匹配和功能到配置 property 的一对一关系，并输出有界 typed warning；输入快照缺失使用独立 typed warning，不得与 schema 非法混淆；
- schema 缺失、非法、candidate 缺失、配置不匹配、重复 property、validator/依赖异常或 OTel 导出失败时，只允许记录 warning；embedded 命令必须以 `0` 退出，PipelineRun、artifact、GitOps promote、Argo reconcile、runtime rollout 和 `/health` closeout 必须继续；
- schema 校验器只读，禁止写回配置、注入默认值、删除变量、关闭功能、切换架构路径或执行任何补救操作；运行时必须原样接收本次待发布配置；
- schema warning 必须实际导出 CI/CD OTel span/event；endpoint、serviceName、sampling 和 exporter timeout 只读 `config/platform-infra/pipelines-as-code.yaml#observability` 与精确 repository params，零/多 repository 匹配在 renderer 生成期 fail-closed；
- OTel exporter 自身失败继续输出 `blocking=false` warning；warning 同时进入 `status|history|debug-step` 的同构有界投影，但不得改变成功/失败终态。
- 任一 CI/CD 阶段或总耗时超过 owning YAML 预算时，不要继续死等或把超长等待视为正常；先输出阶段耗时分解，并优先从 env reuse、git mirror、BuildKit/cache、GitOps/Argo watch 和 runtime readiness 探测方向优化后再继续交付。
- 仅 legacy lane 的 node-scoped `trigger-current --wait` 可以把 source sync、pre/post flush、PipelineRun、GitOps/Argo、runtime readiness 和 `/health` closeout 放进同一端到端 YAML 预算；超预算时由 CLI 输出阶段分解、Argo target revision、runtime/public 状态和 TaskRun/Pod drill-down。PaC migrated consumer 禁止执行 `trigger-current`，也禁止手动串联状态、sync 或 flush 命令补齐交付。
- node-scoped `control-plane status` 可见性规则：
.agents/skills/unidesk-code-queue/SKILL.md
+1 -2
View File
@@ -28,8 +28,7 @@ bun scripts/cli.ts agentrun create task \
--repo <owner/repo> \
--ref <branch> \
--mdtodo-id <R6.3> \
--prompt-stdin \
--dry-run
--prompt-stdin
```
资源命令的 Target 规则：
.agents/skills/unidesk-code-queue/references/resources.md
+4 -5
View File
@@ -23,12 +23,11 @@ bun scripts/cli.ts agentrun create task \
--repo <owner/repo> \
--ref <branch> \
--mdtodo-id <R6.3> \
--prompt-stdin \
--dry-run
--prompt-stdin
```
- 先用同一命令保留 `--dry-run` 查看完整计划；
- dry-run 与 live 都必须先经 owning YAML 选中的 Target `trans` 重入；
- 默认直接执行 live 单命令；不要把 `--dry-run` 作为派单前置步骤；
- live 必须先经 owning YAML 选中的 Target `trans` 重入并完成只读 preflight；
- preflight 必须验证：
- task worktree 是 Git 根目录且 clean；
- origin 与 `--repo` 一致；
@@ -41,7 +40,7 @@ bun scripts/cli.ts agentrun create task \
- session identity；
- provider/tool SecretRef 的对象、key 与 presence；
- preflight 失败固定 `mutation=false`，不得创建 task；
- live 在 submit 成功后自动 dispatch，不要求用户再运行第二条命令；
- preflight 通过后在同一命令内 submit 并自动 dispatch，不要求用户再运行第二条命令；
- 未显式传 `--idempotency-key` 时，由 Target/repo/ref/MDTODO/verified commit 派生稳定键；
- 幂等重试遇到已有 running 或 terminal task 时，返回既有 attempt/run/session，不重复 dispatch；
- 最终输出必须包含 task、attempt、run、command、runner 和 session identity。
.agents/skills/unidesk-gh/SKILL.md
+4 -1
View File
@@ -27,7 +27,10 @@ GitHub issue/PR 正式读写必须走 `bun scripts/cli.ts gh ...` 或 `trans gh:
- 只有需要复用或正式保留的文件才使用 `--body-file`。
- 不要把 Markdown 塞进 shell 参数。
- `gh pr create` 默认 Next 只给有界 `pr view`、`pr review-plan`、`pr preflight` 和 closeout status 下钻；不得默认提示 `preflight --full|--raw`、手工 CI/CD、mirror sync、PipelineRun 或 Argo refresh。
- PR merge 只走 guarded `gh pr merge`；主代理按 `$unidesk-subagent` 完成 review/preflight/merge 授权判断后再执行，不把“禁止默认 Next”理解成“PR 永远不能由主代理受控合并”；只有确认 ancestry 可丢弃时才显式 `--squash`。
- PR review 先用 `review-plan` 建立文件索引，只对需要审查的文件执行 `pr diff --file`，不得机械下钻每个文件。
- PR merge 只走 guarded `gh pr merge`；它内建 readiness/preflight，普通已审 PR 不先机械执行独立 `pr preflight`。主代理按 `$unidesk-subagent` 完成 review 和 merge 授权判断后执行；只有确认 ancestry 可丢弃时才显式 `--squash`。
- 功能、skill、MDTODO 报告和完成状态在 merge 前已齐备时，必须进入同一个 PR。禁止仅为补写 merge SHA、`mergedAt`、branch deletion 或重复 closeout 再开文档 PR；这些事实由 GitHub PR 和 `gh pr merge` 默认摘要保存。
- 成功 merge 后默认摘要已披露 `mergeCommit` 和 `mergedAt`，不再机械执行 `pr view`。只有输出缺字段、merge 后出现新运行证据，或需要定点排障时才继续查询；只有用户明确要求把 merge 后新证据写回 Git 时，才创建第二个 closeout PR。
## 常用入口
.agents/skills/unidesk-gh/references/pull-requests.md
+6 -3
View File
@@ -2,7 +2,7 @@
PR 工作必须使用受控 UniDesk GitHub 命令：
- review 前先用 `pr review-plan`、`pr diff --file` 和有界文件下钻。
- review 前先用 `pr review-plan` 建立 changed-file 索引，只对需要审查的文件使用 `pr diff --file` 和有界 hunk 下钻；不得逐文件机械展开。
- 人工完整 PR 正文使用 `trans gh:/owner/repo/pr/<number> cat`；定点查找使用同一路由的 `rg <pattern>`。
- `pr view <number> --json body,...`、`--full` 和 `--raw` 只用于显式机器结构化披露；正文只在 `.data.pullRequest.body` 输出一次。
- `pr create` 默认 Next 只给有界 observe/review/preflight/status：
@@ -13,11 +13,14 @@ PR 工作必须使用受控 UniDesk GitHub 命令：
默认不得提示 `preflight --full|--raw`、手工 CI/CD、mirror sync、创建 PipelineRun 或 Argo refresh。
- `pr preflight` 是可选只读诊断；`pr merge` 内部会自行执行 preflight。
- 主代理受控合并仍然允许：
- 主代理按 `$unidesk-subagent` 完成 review/preflight/merge 判断后使用 guarded `pr merge`；
- 主代理按 `$unidesk-subagent` 完成 review 和 merge 授权判断后使用 guarded `pr merge`；
- 不得把 `pr create` Next 限制误解成“PR 永远不能人工受控合并”。
- `pr merge --merge` 默认删除已合并的同 repo head branch，清理匹配且干净的本地 `.worktree`，并快进位于 PR base branch 的本地主工作区。
- `pr merge --merge --sync-node JD01` 在支持时额外执行映射节点 source-workspace sync；当前用于 HWLAB `v0.3`。
- 只有明确需要保留合并后状态时才使用 `--keep-branch` 或 `--skip-local-closeout`。
- 只有 ancestry 和语义吸收都明确安全时才使用 squash。
- 功能实现、skill、MDTODO 报告和完成状态在 merge 前已齐备时，必须用一个 PR 收口。不得仅为补 merge SHA、`mergedAt`、branch deletion、已合并状态或重复 closeout 再开 PR。
- merge commit 与合并时间由 GitHub PR 事实和 `gh pr merge` 默认摘要保存，不要求回写 MDTODO 报告。成功合并后不机械执行 `pr view`；只有默认输出缺少必要字段或需要定点排障时才查询。
- 只有 merge 后才产生新的运行面证据，并且用户明确要求将该证据写回 Git 时，第二个 closeout PR 才合理。
Closeout 应写明 source branch、验证证据和残余风险。
同一 PR 的 closeout 应在 merge 前写明 source branch、验证证据和残余风险。
.agents/skills/unidesk-selfmedia/SKILL.md
+42
View File
@@ -0,0 +1,42 @@
---
name: unidesk-selfmedia
description: UniDesk SelfMedia 工厂的 YAML-first 运维技能，覆盖 NC01 生产 Secret 分发与消费者生效、GitHub 到 Gitea/PaC/Tekton/GitOps/Argo 自动交付、WebTerm/Codex 权限边界、公网 IP 健康与登录鉴权验收。用户提到 SelfMedia、自媒体工厂、selfmedia-nc01、管理员账号密码更新、WebTerm、微信公众号文章、4317 公网入口、SelfMedia 部署或流水线运维时使用。
---
# UniDesk SelfMedia 运维
遵循 `Skill(cli-spec)`，只通过 UniDesk owning YAML 和受控 CLI 操作 NC01 运行面。
## 固定边界
- 把 `config/selfmedia.yaml` 作为部署、存储、运行时、权限和公网暴露真相。
- 把 `config/secrets-distribution.yaml` 作为管理员凭据、API key、Codex 凭据和消费者 rollout 真相。
- 把 `config/platform-infra/pipelines-as-code.yaml` 与 `config/platform-infra/gitea.yaml` 作为自动交付真相。
- 通过 `trans NC01:<workspace>` 在 NC01 选中的最新干净 UniDesk worktree 执行命令。
- 只披露 sourceRef、键名、presence、fingerprint、字节数和步骤摘要；不得读取、打印或从运行面反解 Secret 值。
- SelfMedia 源码交付只由 `pikainc/selfmedia` PR merge 触发；不得人工创建 PipelineRun、推送 Gitea、刷新 Argo 或 patch 运行时。
## 高频流程
1. 管理员凭据更新时，按[运维入口](references/operations.md#管理员凭据分发)依次执行 `plan`、`sync`、job 查询和 `status`。
2. 确认 `selfmedia-nc01` 的 `consumerRollout` 已声明 `selfmedia` Deployment；应用启动时读取 Secret，缺少 rollout 不能判定新凭据已生效。
3. 代码发布只观察 PR merge 后的自动 PaC/GitOps 收敛；只有自动链失败时才下钻 status/history/debug-step。
4. 最后从 YAML 解析公网地址，验证 `/` 与 `/healthz` 可达、匿名业务 API 返回 `401`，再用用户提供的新账号完成登录。
## 验收判定
- Secret 同步结果必须为 `succeeded`、`valuesPrinted=false`，并且消费者 restart/status 均为 `exitCode=0`。
- `secrets status` 必须确认 `selfmedia-access`、`selfmedia-runtime`、`selfmedia-codex` 的声明键完整。
- WebTerm 必须保留 PVC session、restart/resume、统一 SelfMedia CLI 与非 root、无 ServiceAccount token/Kubernetes/Docker/宿主权限边界。
- 公网验收只记录 HTTP 状态、session/job/trace、fingerprint 和 artifact SHA，不记录 cookie、账号、密码或 API key。
## 相关技能
- Secret/YAML 归属或 renderer 修改同时使用 `$unidesk-ymalops`。
- PaC、Tekton、GitOps、Argo 或自动交付排障同时使用 `$unidesk-cicd`。
- 跨节点现场闭环同时使用 `$dad-dev` 与 `$unidesk-trans`。
- 公网页面、布局或浏览器原入口验收同时使用 `$unidesk-webdev`。
## 详细入口
- 凭据分发、自动交付和公网验收命令见[运维入口](references/operations.md)。
.agents/skills/unidesk-selfmedia/agents/openai.yaml
+4
View File
@@ -0,0 +1,4 @@
interface:
display_name: "UniDesk SelfMedia 运维"
short_description: "SelfMedia YAML-first 部署、凭据与公网验收"
default_prompt: "使用 $unidesk-selfmedia 完成 SelfMedia 的 YAML-first 运维和原入口验收。"
.agents/skills/unidesk-selfmedia/references/operations.md
+70
View File
@@ -0,0 +1,70 @@
# SelfMedia 运维入口
## 管理员凭据分发
- 在 owner-only `/root/.unidesk/.env/selfmedia-access.env` 中维护以下键：
- `SELFMEDIA_ADMIN_USERNAME`；
- `SELFMEDIA_ADMIN_PASSWORD`；
- `SELFMEDIA_API_KEY`；
- `SELFMEDIA_SESSION_SECRET`。
- 不用 `cat`、`source`、`env`、Pod env 或 Secret decode 检查值。
- 先预检 sourceRef、键和 fingerprint：
```bash
bun scripts/cli.ts secrets plan --config config/secrets-distribution.yaml --scope selfmedia --target selfmedia-nc01
```
- 提交异步同步作业：
```bash
bun scripts/cli.ts secrets sync --config config/secrets-distribution.yaml --scope selfmedia --target selfmedia-nc01 --confirm
```
- 使用返回的 job ID 查询，直到终态：
```bash
bun scripts/cli.ts job status <job-id> --tail-bytes 12000
```
- confirmed sync 默认只返回 source/Secret fingerprint、键和 rollout 摘要；只有一次性下钻时才使用 `--full`，不得让默认输出因超限隐藏 ready 证据。
- 再核对目标 Secret 的键存在性：
```bash
bun scripts/cli.ts secrets status --config config/secrets-distribution.yaml --scope selfmedia --target selfmedia-nc01
```
- `consumerRollout` 是凭据生效合同：
- `selfmedia` Deployment 通过 Secret `subPath` 挂载凭据；
- 后端在启动时加载管理员凭据和 session secret；
- `sync` 必须按 YAML 重启消费者，并通过多次短连接轮询等待 ready；
- 总等待时限和轮询间隔分别由 `timeoutSeconds`、`pollIntervalSeconds` 声明；
- 只更新 Secret 而没有成功 rollout 时，不得声称新账号已生效。
## 自动交付观察
- SelfMedia PaC consumer 为 `selfmedia-nc01`：
```bash
bun scripts/cli.ts platform-infra pipelines-as-code status --target NC01 --consumer selfmedia-nc01
bun scripts/cli.ts platform-infra pipelines-as-code history --target NC01 --consumer selfmedia-nc01 --limit 10
```
- 只有失败归因时才按具体 PipelineRun 下钻：
```bash
bun scripts/cli.ts platform-infra pipelines-as-code debug-step --target NC01 --consumer selfmedia-nc01 --id <pipeline-run-id>
```
- 正常发布只由 `pikainc/selfmedia` 的 GitHub PR merge 触发，不执行 bootstrap、apply、trigger-current、mirror sync、人工 PipelineRun 或 Argo refresh/sync。
## 公网原入口验收
- 从 `config/selfmedia.yaml#delivery.targets.NC01.deployment` 读取公网 host、端口和健康路径，不在脚本中复制配置值。
- 无凭据检查：
- `/` 返回 `200`；
- `/healthz` 返回 `200`；
- `/api/v1/system` 返回 `401`；
- `/api/v1/terminal/status` 返回 `401`。
- 使用 owner 提供的新账号完成浏览器登录；不得把账号密码写入命令历史、截图、报告、issue 或日志。
- 登录后核对 WebTerm 打开、Codex session resume、统一 CLI 请求和流水线 job 状态；不使用终端访问 Kubernetes、Docker socket、ServiceAccount token 或宿主路径。
.agents/skills/unidesk-subagent/SKILL.md
+33 -5
View File
@@ -14,6 +14,10 @@ description: UniDesk 主代理调度子代理的必读技能。用户提到子
- 既有权限、Secret 脱敏、不可逆操作和损害预防边界继续生效，本规则不授权绕过既有安全底线；
- 发现衍生问题时，只能创建独立 issue 或记录待办，不得扩大当前 prompt、PR、合并范围或验收前置；
- 现有机制阻碍原始目标时，先寻找既有架构内的最小实现路径；确需架构扩展时，必须向用户说明与原始目标的关系并取得明确授权。
- bug fix 不得擅自改变 event authority、transport、persistence、replay 或 source of truth；发现现有架构可能是根因时，也必须先完成证据和规格裁决。
- 涉及事件、投影、实时、回放或持久化的派单，主代理必须在子 issue 中先写出 current data flow 和 desired data flow；两者存在架构变化时，先更新长期 SPEC 并取得用户明确授权，再允许实施。
- 新增测试只能验证已经授权的合同，不能用测试通过把未经授权的架构变化、迁移依赖或新 authority 合法化。
- 主代理审核顺序固定为：先比较用户最新目标、适用 SPEC 和 current/desired data flow，再审核代码内部一致性、测试和实现质量；后者通过不能覆盖前者偏离。
- 执行型和调研型委派默认优先使用 AgentRun `Artificer`：
- Artificer 的 `create task`、`apply` 和 `dispatch` 必须以下文“子 issue + MDTODO”登记为 fail-closed 前置；禁止先创建 task 或派单再补登记；
- 原生子代理和 Artificer 每次派单前都必须已有对应 MDTODO ITEM 或 SUBITEM；派单 prompt 与最终报告必须写明同一个 MDTODO ID；
@@ -38,18 +42,38 @@ description: UniDesk 主代理调度子代理的必读技能。用户提到子
- Artificer 禁止递归修复自身：
- 影响派单、AipodSpec、provider/model、credential 投影、session/workspace、Runner、AgentRun runtime 或其 CI/CD 可用性的故障，调查与实现必须交给原生子代理；
- Artificer 只能在修复自动上线后作为被测对象，执行最短 canary、原 session 续跑或业务验收；
- Artificer 可用时，新任务和可安全接续的剩余任务必须优先交给 Artificer，原生子代理只用于管理性主代理职责以外、且 Artificer 本身修复或恢复所必需的工作；
- Artificer 已被证明确实不可用时，先创建独立 issue 和泛化问题域 MDTODO，再派一个原生子代理修复 Artificer；紧急且边界独立的业务任务可以暂时派原生子代理，但不得把临时 fallback 固化为长期调度入口；
- Artificer 是所有执行型子代理任务的强制默认执行面：用户只说“子代理”“并行代理”或“派代理”时，必须使用 Artificer；只有用户明确指定“原生子代理”，或 Artificer 经有界 readiness、最短真实派单或 typed failure 证明确实不可用时，才允许使用原生子代理；
- 主代理不得因为方便、速度、空闲并发槽、已有原生 worktree、历史习惯或预计任务较短而自行降级到原生子代理；管理性、决策性工作由主代理直接完成，不构成原生子代理例外；
- Artificer 已被证明确实不可用时，先创建独立 issue 和泛化问题域 MDTODO，再派一个原生子代理修复 Artificer；只有与故障修复解耦且用户明确要求继续的紧急业务任务，才允许临时使用原生子代理，并必须在 Artificer 恢复后交接；不得把临时 fallback 固化为长期调度入口；
- Artificer 恢复后，原生子代理必须在 clean commit、PR 或只读报告 checkpoint 停止继续扩展；主代理通过原 issue、MDTODO 和 session 传递接续上下文，再由 Artificer 接替尚未完成的工作；禁止两个执行面共享可写 worktree、重复提交或并发修改同一任务；
- 管理性、决策性文档仍由主代理负责，不为了满足 Artificer 优先规则而外包治理决策。
- 子代理并发必须从用户原始任务的依赖图出发主动扩展：
- 主代理先识别串行定锚项、已就绪任务和后续依赖，再计算当前可安全派发集合；
- 串行定锚完成后，应立即并发派出所有低耦合、可独立验收且具备独立 worktree/branch/issue/PR 的已就绪任务，直到达到可用执行槽、运行面资源预算或真实依赖边界；
- 存在两个及以上已就绪任务时，长期只运行一个子代理属于调度失败：
- 主代理必须补派；
- 无法补派时，在当前 anchor 中写明具名依赖、共享文件冲突、运行面容量或权限阻塞；
- 每个任务终态和依赖变化后重新计算可安全派发集合；
- 主代理自己的审核、规划和等待不计入子代理并发量；不得用一个长期任务加主代理活动宣称已经并行；
- 不得为了凑并发拆出无独立价值的日志采集、重复调查、重复实现或审核代理；并发量服从原始任务边界和成功率，不服从固定数字。
- 运行面故障、用户原入口不可用，或凭据/配置变更导致服务退化时，必须先恢复运行面，再完成工程化：
- 主代理先冻结恢复判定标准，并保留恢复关键路径的控制权；
- 优先用既有受控入口实施最小、可逆、可验证的恢复，只有受控入口本身缺失并直接阻塞恢复时，才在关键路径修改工具；
- 通用抽象、输出优化、skill、报告和长期治理不得成为恢复门禁；
- 恢复达到用户原入口可用的标准后，主代理不得把临时恢复当作任务完成，必须继续完成用户明确要求的根因修复、工程化、PR、验证和治理；
- 与恢复根因解耦且具备独立 issue、MDTODO、worktree 和验收入口的工程化任务，必须在恢复期间立即并行派发，不得全部排到恢复之后串行执行。
- 子代理并行只用于成功率高、耦合度低、能放进独立 worktree/branch/issue/PR 的任务；共享架构方向、公共契约和同文件高冲突修改先串行定锚。
- 子代理产出的审核默认由主代理直接完成：
- 用户允许子代理实施、要求并行或启用多轮审查，不等于允许调度审核子代理；
- 只有用户明确要求子代理审核、独立审核代理或多代理交叉审核时，才允许派发审核型子代理；
- 主代理发现问题后，直接形成纠偏要求并由原执行 session 续跑或在原任务边界内修复，不新增审核代理。
- 管理性、决策性文档由主代理直接完成，不把决策权或治理写入外包给子代理：
- 主代理负责 `AGENTS.md`、skill 核心规则、长期 reference 规范、架构决策、主 issue anchor/closeout，以及 MDTODO FILE 的领域划分、ITEM 结构和状态治理；
- 子代理可以在已明确的 issue、MDTODO ITEM 和验收边界内独立调研，并编写对应任务报告、验证记录、实现说明和证据附件；
- 子代理报告只承载其任务事实与结论，不能自行改写上层目标、架构取舍、优先级、主线状态或跨任务治理规则；
- 主代理审阅子代理报告后，亲自把被采纳的结论写入管理性、决策性文档。
- 纯文档交付可以按 `$git-spec` 的稳定分支快路径由主代理直接 commit/push，不要求临时 branch、`.worktree` 或 PR；发现本地分叉、并行改动、分支保护或运行面影响时必须保留状态并回到隔离交付。
- 正式 GitHub issue/PR/comment/merge 仍走 `$unidesk-gh`；子代理可以提交 PR 和写调查结论，主代理负责 review/preflight/merge，除非用户明确授权某个子代理自上线自验证。
- 正式 GitHub issue/PR/comment/merge 仍走 `$unidesk-gh`；子代理可以提交 PR 和写调查结论，主代理负责 bounded review 和 guarded merge，独立 preflight 只用于定点排障，除非用户明确授权某个子代理自上线自验证。
- 主代理和子代理必须通过 issue/PR/comment 链接传递可复用上下文、调查结论、证据链接和下一步边界；派发前先读既有评论，prompt 中只引用链接和增量任务，不复述长结论，避免不同子代理重复调查同一事实。
- 主代理派发任何执行型子代理前必须完成以下登记；对 Artificer，全部步骤必须早于 AgentRun `create task`、`apply` 和 `dispatch`：
- 创建子 issue，把主要任务、接续上下文、目标分支/worktree、范围、禁止项和验收入口写入正文；
@@ -76,7 +100,11 @@ description: UniDesk 主代理调度子代理的必读技能。用户提到子
- 当前用户任务只要派出过子代理：
- 主代理就必须保持任务进行中；
- 等待本轮全部子代理进入真实终态；
- 不得把 task 创建成功、dispatch 成功、`running`、已提交 commit 或已创建 PR 当作主代理可以结束的交付点。
- 不得把 task 创建成功、dispatch 成功、`running`、已提交 commit 或已创建 PR 当作主代理可以结束的交付点；
- 等待期间仍要维护并发窗口：
- 任一子代理终态、依赖解除或新任务就绪后，立即重新计算可安全派发集合；
- 还有独立已就绪任务时及时补派，避免并发窗口退化为长期单任务等待；
- 只有具名依赖、共享写入边界、运行面容量或权限阻塞时才允许保持较低并发，并把理由写入主线 anchor。
- 子代理返回后，主代理必须继续完成职责范围内的收尾：
- 完成必要的纠偏和 post-task；
- 审核 PR 并执行获授权的合并；
@@ -110,7 +138,7 @@ description: UniDesk 主代理调度子代理的必读技能。用户提到子
- CI/CD、rollout、PipelineRun、Argo closeout：`$unidesk-cicd`。
- WebProbe、Workbench、浏览器复测：`$unidesk-webdev`。
- 子代理完成后的反馈收口和反馈 issue 判定：`$post-task`。
- Artificer 和其他执行型子代理派单前的任务登记必须使用 `$mdtodo-edit`；只有用户明确要求多轮审查或循环修复时才使用 `$mdtodo-loop`。
- Artificer 和其他执行型子代理派单前的任务登记必须使用 `$mdtodo-edit`；只有用户明确要求多轮审查或循环修复时才使用 `$mdtodo-loop`，启用后仍由主代理审核，除非用户明确要求审核子代理。
## 何时读取 reference
.agents/skills/unidesk-subagent/references/gh-workflow.md
+29 -5
View File
@@ -4,13 +4,33 @@
## 调度原则
- 执行型任务默认且必须交给 Artificer。用户泛称“子代理”“并行代理”或“派代理”不等于授权原生子代理；只有用户明确指定原生子代理，或 Artificer 经有界 readiness、最短真实派单或 typed failure 证明不可用时，才允许原生执行面。
- 主代理不能以方便、速度、空闲槽、已有 worktree、任务较短或历史习惯作为原生降级理由。Artificer 故障修复可由原生子代理承担；管理性与决策性工作仍由主代理直接完成。
- 主代理先明确总目标、目标 repo/branch/lane、架构方向和不可倒退边界，再拆任务。不能把“决定架构是否倒退”“是否关闭用户问题”“是否合并 PR”整体外包给子代理。
- 主代理直接审核子代理产出的 diff、验证证据和 PR。允许子代理实施、要求并行或多轮审查都不授权审核子代理；只有用户明确要求子代理审核时，才建立独立审核任务。
- 只有低耦合任务才并行：不同 worktree、不同模块或不同 issue，产物可以独立 review，失败不会阻塞其他任务继续产出证据。
- 高耦合任务先串行定锚：公共类型/契约、source-of-truth、请求治理 authority、共享 runtime policy、同一大文件或同一状态机的修改，应先由主代理或一个子代理形成基线，其他子代理基于基线继续。
- 任务要按成功率分层：调查、工具增强、业务修复、验证、PR review、上线 closeout 可以并行，但“修同一个根因的两套实现”通常不应并行。
- 主代理需要持续调度，而不是把多个子任务排队串行执行。用户明确要求并行且任务在不同 worktree 时，应同时派发能并行的子任务，并通过 issue/PR 状态汇总。
- 主代理必须维护可安全并发窗口：
- 从原始任务建立依赖图，区分串行定锚、已就绪任务和等待依赖任务；
- 每次定锚完成、子代理终态或依赖变化后重新计算已就绪集合，并立即补派低耦合任务；
- 并发目标是 `min(已就绪独立任务数, 可用执行槽, 运行面安全容量)`，不硬编码固定数量；
- 当两个及以上任务已就绪却长期只有一个活跃子代理时，必须视为调度缺口并补派，不能把主代理活动计入子代理并发；
- 只有具名依赖、共享文件冲突、权限或运行面容量阻塞时才降低并发，并把原因和下一次重算触发点写入主线 anchor。
- 并发不能靠重复调查、同根因的多套实现、无独立验收价值的日志采集或默认审核代理虚增；每条并发线都必须有独立 issue、worktree、交付物和验收入口。
- 派发 CI/CD 或运行面调优子任务时，主代理不要在 prompt 或评论里反复解释历史；用 issue/comment 链接引用既有结论，把下一步限定为可由子代理自主真实触发的单步 gate，并要求子代理在该 gate 内小闭环调优到通过后再提交 PR。
## 恢复优先与工程化并发
- 依赖图必须先标出运行面恢复关键路径和用户原入口恢复标准。主代理控制恢复关键路径，避免多个代理同时修改同一生产对象、恢复状态机或共享配置。
- 恢复优先指先让用户原入口通过既有受控路径恢复到可验证状态，不表示工程化任务整体串行：
- 恢复关键路径上的最小诊断、配置修正和复测按真实依赖顺序执行；
- 与根因解耦的 CLI、reference、报告、独立仓库修复和长期治理，在具备独立 issue、MDTODO、worktree、PR 与验收入口时立即并行；
- 只有缺少受控恢复入口直接阻塞恢复时，才允许把工具修改放到恢复关键路径。
- 运行面恢复后，主代理继续完成根因修复、持久化配置、自动交付、原入口复测和治理收口；不得以临时恢复或单次手工成功代替用户要求的终态。
- 恢复期间降低并发必须有具名原因，例如共享生产对象、同一状态机、权限边界或运行面容量；原因解除后立即重新计算可安全并发窗口。
## 模型与思考等级
- Artificer 默认不显式覆盖模型：
@@ -60,8 +80,11 @@
## PR 工作流
- 子代理 PR 应小而可审：一个根因、一个模块边界、一个目标 issue；跨模块架构 PR 必须在 body 中写明为什么不是局部修补。
- 主代理 review 时先看架构约束和倒退风险，再看实现细节。重点检查是否重新引入旧 authority、并行请求源、裸 API 绕过、隐藏默认、阈值硬编码或降低探针能力。
- 合并前使用 `$unidesk-gh` 的 review/preflight/merge 入口。独立 PR 可以并行 review；有依赖的 PR 按契约基线、实现、验证工具、closeout 的顺序合并。
- 主代理本人 review：
- 先看架构约束和倒退风险，再看实现细节；
- 重点检查是否重新引入旧 authority、并行请求源、裸 API 绕过、隐藏默认、阈值硬编码或降低探针能力；
- 默认不再派审核子代理。
- 合并前使用 `$unidesk-gh` 的 bounded review 和 guarded merge 入口；`pr merge` 已内建 preflight，普通已审 PR 不机械重复执行独立 `pr preflight`。独立 PR 可以并行 review；有依赖的 PR 按契约基线、实现、验证工具、closeout 的顺序合并。
- 子代理 PR 合并后，主代理负责同步目标 worktree、触发受控 CI/CD、执行原入口复测，并把 PipelineRun、GitOps revision、observer、trace、report SHA 等证据写回 issue。
- 若用户明确让子代理“自己上线自己验证”，子代理可以执行受控 CI/CD 和原入口验证，但必须在 issue/PR 留下完整证据；主代理仍要抽查并最终汇总。
@@ -83,9 +106,10 @@ Prompt 至少包含以下字段，按任务裁剪：
1. 建立计划：列出可并行任务、串行依赖和每个子代理交付物。
2. 为每个执行型子代理先创建子 issue，把任务正文写进子 issue。
3. 使用 `$mdtodo-edit` 把子 issue 登记到泛化问题域 ITEM/SUBITEM 并标记进行中；登记失败时停止派单。
4. 同时派发低耦合子任务；Artificer 的 `create task`、`apply` 和 `dispatch` 均只能发生在 MDTODO 登记之后；共享契约先派一个基线任务。
5. 轮询子代理结果：子 issue comment、PR、验证摘要、阻塞。
6. 对每个 PR 做架构 review、bounded diff、preflight 和必要本地验证。
4. 计算当前可安全并发窗口并同时派发全部已就绪的低耦合子任务；共享契约先派一个基线任务。
- Artificer 的 `create task`、`apply` 和 `dispatch` 均只能发生在 MDTODO 登记之后。
5. 轮询子代理结果：子 issue comment、PR、验证摘要、阻塞；任一任务终态或依赖变化后立即重新计算窗口并补派，不能退化为长期单任务等待。
6. 对每个 PR 做架构 review、bounded diff 和必要本地验证；只有定点排障时单独执行 preflight，正常收口直接使用内建 readiness 的 guarded merge。
7. 按依赖顺序合并；合并后同步目标 worktree。
8. 触发受控 CI/CD 或让明确授权的子代理上线；主代理核对 closeout。
9. 用原入口复测；把剩余问题拆到新 issue 或追加既有 issue。
.tekton/web-probe-sentinel-jd01-pac.yaml
+1
View File
@@ -68,6 +68,7 @@ spec:
bun scripts/native/cicd/render-sentinel-publish-task.ts \
--node JD01 \
--lane v03 \
--repository-id sentinel-jd01-v03 \
--sentinel jd01-web-probe-sentinel \
--pipeline-run "$(context.pipelineRun.name)" \
--source-commit "{{revision}}" \
.tekton/web-probe-sentinel-nc01-pac.yaml
+1
View File
@@ -68,6 +68,7 @@ spec:
bun scripts/native/cicd/render-sentinel-publish-task.ts \
--node NC01 \
--lane v03 \
--repository-id sentinel-nc01-v03 \
--sentinel nc01-web-probe-sentinel \
--pipeline-run "$(context.pipelineRun.name)" \
--source-commit "{{revision}}" \
bun.lock
+8
View File
@@ -4,6 +4,10 @@
"workspaces": {
"": {
"name": "unidesk-root",
"dependencies": {
"ajv-dist": "8.17.1",
"yaml": "2.3.4",
},
"devDependencies": {
"@types/bun": "latest",
"@types/node": "latest",
@@ -98,6 +102,8 @@
"adler-32": ["adler-32@1.3.1", "", {}, "sha512-ynZ4w/nUUv5rrsR8UUGoe1VC9hZj6V5hU9Qw1HlMDJGEJw5S7TfTErWTjMys6M7vr0YWcPqs3qAr4ss0nDfP+A=="],
"ajv-dist": ["ajv-dist@8.17.1", "", {}, "sha512-KzJwANMzTTR/RERGnkx+bHzmxIfMTPMMv7+cH1d6Lx9UQ7BZyhiieq4hnO5lRuBWOtYTUL8hyWs7RJYI/45Rtg=="],
"bun-types": ["bun-types@1.3.13", "", { "dependencies": { "@types/node": "*" } }, "sha512-QXKeHLlOLqQX9LgYaHJfzdBaV21T63HhFJnvuRCcjZiaUDpbs5ED1MgxbMra71CsryN/1dAoXuJJJwIv/2drVA=="],
"cfb": ["cfb@1.2.2", "", { "dependencies": { "adler-32": "~1.3.0", "crc-32": "~1.2.0" } }, "sha512-KfdUZsSOw19/ObEWasvBP/Ac4reZvAGauZhs6S/gqNhXhI7cKwvlH7ulj+dOEYnca4bm4SGo8C1bTAQvnTjgQA=="],
@@ -184,6 +190,8 @@
"xlsx": ["xlsx@0.18.5", "", { "dependencies": { "adler-32": "~1.3.0", "cfb": "~1.2.1", "codepage": "~1.15.0", "crc-32": "~1.2.1", "ssf": "~0.11.2", "wmf": "~1.0.1", "word": "~0.3.0" }, "bin": { "xlsx": "bin/xlsx.njs" } }, "sha512-dmg3LCjBPHZnQp5/F/+nnTa+miPJxUXB6vtk42YjBBKayDNagxGEeIdWApkYPOf3Z3pm3k62Knjzp7lMeTEtFQ=="],
"yaml": ["yaml@2.3.4", "", {}, "sha512-8aAvwVUSHpfEqTQ4w/KMlf3HcRdt50E5ODIQJBw1fQ5RL34xabzxtUlzTXVqc4rkZsPbvrXKWnABCD7kWSmocA=="],
"vite/fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="],
}
}
config/feature-config.schema.json
+36
View File
@@ -0,0 +1,36 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://unidesk.pikapython.com/schema/feature-config.schema.json",
"title": "UniDesk Product Feature Configuration",
"type": "object",
"properties": {
"webProbeSentinel": {
"x-unidesk-feature": "web-probe-sentinel",
"type": "object",
"required": [
"enabled",
"scheduler"
],
"properties": {
"enabled": {
"type": "boolean"
},
"scheduler": {
"type": "object",
"required": [
"intervalSeconds"
],
"properties": {
"intervalSeconds": {
"type": "integer",
"minimum": 30
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}
config/hwlab-node-lanes.yaml
+19 -14
View File
@@ -123,6 +123,7 @@ lanes:
configRef: config/hwlab-node-lanes.yaml#lanes.v03.targets.NC01
renderer: hwlab-runtime-lane
mode: remote-pipeline-annotation
maxInlineScriptBytes: 32768
git:
url: git@github.com:pikasTech/HWLAB.git
readUrl: http://gitea-http.devops-infra.svc.cluster.local:3000/mirrors/pikasTech-HWLAB.git
@@ -209,26 +210,30 @@ lanes:
codexStdioSupervisor: repo-owned
kafkaEventBridge:
enabled: true
features:
directPublish: true
liveKafkaSse: true
kafkaRefreshReplay: true
transactionalProjector: false
projectionOutboxRelay: false
projectionRealtime: false
refreshReplay:
groupIdPrefix: hwlab-v03-cloud-api-sse
timeoutMs: 30000
scanLimit: 1000000
matchedEventLimit: 2000
liveBufferLimit: 2000
transactionalProjector:
heartbeatIntervalMs: 2000
projectionOutboxRelay:
intervalMs: 250
batchSize: 100
leaseMs: 30000
sendTimeoutMs: 10000
retryBackoffMs: 1000
projectionRealtime:
outboxTailBatchSize: 100
sseHeartbeatMs: 15000
sseDrainTimeoutMs: 2500
healthThresholds:
consumerLagWarning: 0
outboxBacklogWarning: 0
retryCountWarning: 0
failedInboxWarning: 0
dlqWarning: 0
configRef: config/platform-infra/kafka.yaml#clients.hwlab-v03-cloud-api
bootstrapServers: platform-infra-kafka-kafka-bootstrap.platform-infra.svc.cluster.local:9092
stdioTopic: codex-stdio.raw.v1
agentRunEventTopic: agentrun.event.v1
hwlabEventTopic: hwlab.event.v1
clientId: hwlab-v03-cloud-api
directPublishConsumerGroupId: hwlab-v03-agentrun-event-direct-publish
transactionalProjectorConsumerGroupId: hwlab-v03-agentrun-event-projector
hwlabEventConsumerGroupId: hwlab-v03-workbench-live-sse
publicExposure:
config/platform-infra/sub2api.yaml
+1 -1
View File
@@ -82,7 +82,7 @@ targets:
redisReplicas: 1
image:
repository: docker.1panel.live/weishaw/sub2api
tag: 0.1.153
tag: 0.1.155
pullPolicy: IfNotPresent
dependencyImages:
redis: docker.m.daocloud.io/library/redis:8-alpine
config/secrets-distribution.yaml
+44
View File
@@ -9,6 +9,7 @@ metadata:
- 300
- 313
- 2256
- 1964
sources:
root: /root/.unidesk/.state/secrets
@@ -147,6 +148,27 @@ sources:
randomBase64Url:
bytes: 24
prefix: poa_
- sourceRef: /root/.unidesk/.env/selfmedia-access.env
type: env
requiredKeys:
- SELFMEDIA_ADMIN_USERNAME
- SELFMEDIA_ADMIN_PASSWORD
- SELFMEDIA_API_KEY
- SELFMEDIA_SESSION_SECRET
createIfMissing:
enabled: true
values:
SELFMEDIA_ADMIN_USERNAME: admin
randomBase64Url:
SELFMEDIA_ADMIN_PASSWORD:
bytes: 32
prefix: smp_
SELFMEDIA_API_KEY:
bytes: 32
prefix: smk_
SELFMEDIA_SESSION_SECRET:
bytes: 32
prefix: sms_
- sourceRef: ~/.env/TOKEN
type: raw-file
required: true
@@ -192,6 +214,11 @@ targets:
namespace: selfmedia
scope: selfmedia
enabled: true
consumerRollout:
deployments:
- selfmedia
timeoutSeconds: 180
pollIntervalSeconds: 5
- id: pikaoa-nc01
route: NC01:k3s
namespace: pikaoa
@@ -219,6 +246,23 @@ kubernetesSecrets:
- sourceRef: platform-infra/pk01-frp.env
sourceKey: FRP_TOKEN
targetKey: FRP_TOKEN
- name: selfmedia-access
targetId: selfmedia-nc01
secretName: selfmedia-access
type: Opaque
data:
- sourceRef: /root/.unidesk/.env/selfmedia-access.env
sourceKey: SELFMEDIA_ADMIN_USERNAME
targetKey: admin-username
- sourceRef: /root/.unidesk/.env/selfmedia-access.env
sourceKey: SELFMEDIA_ADMIN_PASSWORD
targetKey: admin-password
- sourceRef: /root/.unidesk/.env/selfmedia-access.env
sourceKey: SELFMEDIA_API_KEY
targetKey: api-key
- sourceRef: /root/.unidesk/.env/selfmedia-access.env
sourceKey: SELFMEDIA_SESSION_SECRET
targetKey: session-secret
- name: selfmedia-runtime
targetId: selfmedia-nc01
secretName: selfmedia-runtime
config/selfmedia.yaml
+14
View File
@@ -172,6 +172,20 @@ delivery:
mountPath: /home/codex/.codex/state
subPath: state
secrets:
access:
sourceRef: /root/.unidesk/.env/selfmedia-access.env
target:
secretName: selfmedia-access
readOnly: true
files:
- targetKey: admin-username
mountPath: /run/secrets/selfmedia-access/admin-username
- targetKey: admin-password
mountPath: /run/secrets/selfmedia-access/admin-password
- targetKey: api-key
mountPath: /run/secrets/selfmedia-access/api-key
- targetKey: session-secret
mountPath: /run/secrets/selfmedia-access/session-secret
runtime:
sourceRef: ~/.env/TOKEN
target:
config/unidesk-cli.yaml
+6
View File
@@ -27,6 +27,12 @@ github:
sourceRef: pikainc-selfmedia-gh-token.txt
sourceKey: GH_TOKEN
format: raw-token
- repository: pikainc/pikaoa
priority: before-env
root: /root/.unidesk/.env
sourceRef: pikainc-selfmedia-gh-token.txt
sourceKey: GH_TOKEN
format: raw-token
prMerge:
unknownRetry:
maxAttempts: 5
config/unidesk-host-k8s.yaml
+32
View File
@@ -23,12 +23,38 @@ delivery:
changeDetection:
runtimePaths:
- .tekton/unidesk-host-pac.yaml
- config/feature-config.schema.json
- config/hwlab-node-lanes.yaml
- config/platform-infra/pipelines-as-code.yaml
- config/unidesk-host-k8s.yaml
- bun.lock
- package.json
- scripts/native/cicd/build-unidesk-host-image.sh
- scripts/native/cicd/feature-config-schema-warning.mjs
- scripts/native/cicd/otel-runtime-config.ts
- scripts/native/cicd/pac-status-evaluator.cjs
- scripts/native/cicd/prepare-unidesk-host-release.mjs
- scripts/native/cicd/publish-unidesk-host-gitops.mjs
- scripts/native/cicd/render-sentinel-publish-task.ts
- scripts/native/deploy/render-unidesk-host-service.mjs
- scripts/native/hwlab/runtime-gitops-observability.mjs
- scripts/native/hwlab/runtime-gitops-postprocess.mjs
- scripts/native/hwlab/runtime-gitops-scripts-configmap.mjs
- scripts/native/hwlab/runtime-gitops-verify.mjs
- scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js
- scripts/vendor/ajv-dist/8.17.1/manifest.json
- scripts/src/agentrun-manifests.ts
- scripts/src/hwlab-node/render.ts
- scripts/src/hwlab-node/web-probe.ts
- scripts/src/hwlab-node-web-observe-runner-realtime-source.ts
- scripts/src/hwlab-node-web-sentinel-cicd-jobs.ts
- scripts/src/pac-feature-config-schema-stage.ts
- scripts/src/platform-infra-pac-feature-config-projection.ts
- scripts/src/platform-infra-pipelines-as-code-source-artifact.ts
- scripts/src/platform-infra-pipelines-as-code.ts
- scripts/src/config.ts
- scripts/src/hwlab-node-lanes.ts
- scripts/src/yaml-composition.ts
- src/components/microservices/todo-note/Dockerfile
- src/components/microservices/todo-note/bun.lock
- src/components/microservices/todo-note/package.json
@@ -58,6 +84,12 @@ delivery:
branch: unidesk-host-gitops
manifestPath: deploy/gitops/unidesk-host/todo-note.yaml
releaseStatePath: deploy/gitops-state/unidesk-host/todo-note.json
resources:
- id: hwlab-nc01-v03-runtime-gitops-scripts
renderer: hwlab-runtime-gitops-scripts
configRef: config/hwlab-node-lanes.yaml#lanes.v03.targets.NC01
manifestPath: deploy/gitops/unidesk-host/hwlab-nc01-v03-runtime-gitops-scripts.yaml
namespace: hwlab-ci
author:
name: UniDesk Host CI
email: unidesk-host-ci@unidesk.local
docs/MDTODO/agentrun-runtime-reliability.md
+13 -2
View File
@@ -114,6 +114,17 @@
解决 [UniDesk #1883](https://github.com/pikasTech/unidesk/issues/1883) 与 [AgentRun #342](https://github.com/pikasTech/agentrun/issues/342)：让 Artificer runner 的受控 trans 复用 UniDesk Windows route 解析与 frontend transport，支持 D518:win 及 win/<drive>/<path> 原入口；区分 primary source workspace 与用户授权的远端 tool target，删除 unsupported-operation 和 host path 误判，不新增节点特例、旁路 SSH 或隐藏 fallback。两个 PR 自动上线后用最短 Artificer canary 验收 pwd、ls、cat 和 apply-patch，完成任务后将详细报告写入[任务报告](./details/agentrun-runtime-reliability/R5.6_Task_Report.md)。
### R5.7 [in_progress]
### R5.7
解决 [UniDesk #2014](https://github.com/pikasTech/unidesk/issues/2014)：把 `config/unidesk-cli.yaml#github.auth.repositoryOverrides` 的既有私有仓凭据通过 YAML-first tool credential/resource bundle 投影给 Artificer，使受控 `unidesk-gh` 可读取、创建和评论任务私有仓 PR；Secret 只披露 SecretRef presence/fingerprint/valuesPrinted=false，不新增第二 GitHub 客户端、权限契约、租约或围栏，并与 PikaOA MVP 主线并行且不作为业务门禁，完成任务后将详细报告写入[任务报告](./details/agentrun-runtime-reliability/R5.7_Task_Report.md)。
解决 [UniDesk #1957](https://github.com/pikasTech/unidesk/issues/1957)：修复 Artificer resource bundle 缺失 `$post-task` skill 与 `$unidesk-subagent` 权威收口引用不一致；只沿 owning skill source、AipodSpec、resource bundle 和 runner 投影定位并物化，禁止 prompt 复制规则、手工 Pod/宿主补文件、第二 bundle authority 或人工 CI/CD，完成任务后以真实 Artificer follow-up 加载 `$post-task` 验收，完成任务后将详细报告写入[任务报告](./details/agentrun-runtime-reliability/R5.7_Task_Report.md)。
### R5.8 [in_progress]
解决 [UniDesk #2014](https://github.com/pikasTech/unidesk/issues/2014)：把 `config/unidesk-cli.yaml#github.auth.repositoryOverrides` 的既有私有仓凭据通过 YAML-first tool credential/resource bundle 投影给 Artificer，使受控 `unidesk-gh` 可读取、创建和评论任务私有仓 PR；Secret 只披露 SecretRef presence/fingerprint/valuesPrinted=false，不新增第二 GitHub 客户端、权限契约、租约或围栏，并与 PikaOA MVP 主线并行且不作为业务门禁，完成任务后将详细报告写入[任务报告](./details/agentrun-runtime-reliability/R5.8_Task_Report.md)。
## R6 [completed]
将“先恢复运行面再工程化”固化为 unidesk-subagent 的泛化主线规则：运行面恢复关键路径由主代理控制，低耦合工程化任务及时并行，恢复后继续完成工程化交付；将单 PR 收口、复用 guarded merge 内建 preflight、避免仅补 merge SHA 的重复 PR 和减少碎片化 GitHub 查询固化到 unidesk-gh 及相关 reference，并让 merge 默认摘要直接披露 merge commit 与 mergedAt，完成任务后将详细报告写入[任务报告](./details/agentrun-runtime-reliability/R6_Task_Report.md)。
## R7 [completed]
将 Artificer 固化为所有执行型子代理任务的强制默认执行面：用户只说“子代理”时必须使用 Artificer；只有用户明确指定原生子代理，或 Artificer 经有界 readiness/真实派单证据确认故障不可用时，才允许原生子代理执行；禁止主代理因方便、速度、空闲槽或历史习惯自行降级，并要求 Artificer 恢复后立即停止原生扩展并交接，完成任务后将详细报告写入[任务报告](./details/agentrun-runtime-reliability/R7_Task_Report.md)。
docs/MDTODO/details/agentrun-runtime-reliability/R6_Task_Report.md
+22
View File
@@ -0,0 +1,22 @@
# R6 任务报告
## 结论
已将“先恢复运行面，再工程化”固化为子代理调度的泛化原则：主代理冻结恢复标准并控制恢复关键路径，优先通过既有受控入口做最小、可逆、可验证的恢复；恢复不再阻塞低耦合工程化任务并行，运行面恢复后仍继续完成根因修复、PR、验证与治理收口。
已将减少重复交付固化到 GitHub skill 与 PR reference：review-plan 只建立索引并按需下钻，guarded merge 内建 preflight，功能、skill、MDTODO 报告和完成状态应在同一 PR 收口，禁止仅为补 merge SHA、mergedAt 或重复 closeout 创建第二个 PR。只有 merge 后产生新运行证据且用户明确要求写回 Git 时，才允许第二个 closeout PR。
已更新 GitHub merge 默认文本摘要，成功合并直接披露 mergeCommit 与 mergedAt，并取消机械 pr view 的 Next 提示。
## 验证
- unidesk-subagent skill quick_validate：通过。
- unidesk-gh skill quick_validate：通过。
- bun --check scripts/src/gh/pr-merge.ts：通过。
- bun scripts/cli.ts check --syntax-only：11 项通过，0 项失败。
- renderer 定向验证：成功输出包含 mergeCommit=abc123 与 mergedAt=2026-07-14T10:00:00Z，且不包含 gh pr view。
- git diff --check：通过。
## 交付边界
本任务不修改运行面、不新增状态库或门禁。规则、CLI、MDTODO 报告与完成状态在同一功能 PR 中交付；merge 事实由 GitHub 和受控 merge 摘要保存，不回写本报告。
docs/MDTODO/details/agentrun-runtime-reliability/R7_Task_Report.md
+7
View File
@@ -0,0 +1,7 @@
# R7 任务报告
已将 Artificer 固化为执行型子代理任务的强制默认执行面。用户泛称子代理、并行代理或派代理时必须使用 Artificer；只有用户明确指定原生子代理，或 Artificer 经有界 readiness、最短真实派单或 typed failure 证明确实不可用时，才允许原生执行面。
规则明确禁止主代理因方便、速度、空闲并发槽、已有 worktree、任务较短或历史习惯自行降级。Artificer 故障修复仍可交给原生子代理；管理性和决策性工作由主代理直接完成。临时原生 fallback 必须在 Artificer 恢复后停止扩展并交接。
验证：unidesk-subagent skill quick_validate 通过；git diff --check 通过。
docs/MDTODO/details/selfmedia-production-delivery/R1.3_Task_Report.md
+68
View File
@@ -0,0 +1,68 @@
# R1.3 任务报告
## 结论
- 已从 owner-only sourceRef `/root/.unidesk/.env/selfmedia-access.env` 重新分发用户更新后的 SelfMedia 管理员凭据。
- sourceRef 权限保持 `0600`，新旧 fingerprint 已变化；全过程未读取、解码或输出账号、密码、cookie 与 Secret 值。
- 已将 SelfMedia 的 owning YAML、Secret sync/status、消费者生效、自动交付观察、WebTerm 权限边界和公网鉴权验收沉淀为 `unidesk-selfmedia` skill。
- 已通过 UniDesk PR [#1995](https://github.com/pikasTech/unidesk/pull/1995) 合并到 `master`，merge commit 为 `393711d9773cce8faa738a01e1297a5bf513e663`。
## SelfMedia 凭据兼容修复与交付
- 新管理员密码为 13 字节，旧配置错误地与机器 API key、session secret 共用 32 字节门槛。
- SelfMedia PR [#25](https://github.com/pikainc/selfmedia/pull/25) 拆分强度配置：
- 管理员密码最小长度由 YAML 配置为 12 字节；
- API key 与 session secret 分别保持 32 字节；
- merge commit 为 `a50869afcbccde3c0a0e06d595f1938f5f671f78`。
- 自动 PipelineRun `selfmedia-nc01-a50869afcbccde3c0a0e06d595f1938f5f671f78-8fgcs`：
- 状态为 Succeeded；
- 耗时 194 秒；
- 镜像摘要前缀为 `sha256:acf4191432f`；
- GitOps revision 为 `78c0bfa0745c`；
- Argo 为 `Synced/Healthy`；
- runtime 为 `1/1`。
- SelfMedia PR [#26](https://github.com/pikainc/selfmedia/pull/26) 收口对应 MDTODO 与验收记录，merge commit 为 `11bfcf2657cd4f62e5e7221803872e53143e5f0c`。
## YAML-first 运维能力
- `config/selfmedia.yaml` 继续负责部署、存储、运行时、权限与公网暴露。
- `config/secrets-distribution.yaml` 为 `selfmedia-nc01` 声明 `selfmedia` Deployment 的 `consumerRollout`、等待时限与轮询间隔。
- `scripts/src/secrets.ts` 在 Secret apply 成功后：
- 按 YAML 声明重启消费者；
- 用多次短 `trans` 连接轮询 Deployment ready；
- 默认 confirmed sync 只输出有界摘要；
- `--full` 保留显式完整下钻；
- 所有输出保持 `valuesPrinted=false`。
- 新 skill 文件：
- `.agents/skills/unidesk-selfmedia/SKILL.md`；
- `.agents/skills/unidesk-selfmedia/references/operations.md`；
- `.agents/skills/unidesk-selfmedia/agents/openai.yaml`。
## 真实同步与公网验收
- Secret 同步 job `secrets_sync_20260714052911863_5fd8da`：
- 状态为 Succeeded；
- 总耗时 9 秒；
- Secret apply exitCode 为 0；
- consumer rollout restart exitCode 为 0；
- ready 轮询 2 次，耗时 8 秒；
- Deployment generation 为 18；
- ready/available 为 `1/1`；
- 默认 stdout 为 8032 bytes，未截断；
- `valuesPrinted=false`。
- 公网原入口：
- `/` 返回 200；
- `/healthz` 返回 200；
- 匿名 `/api/v1/system` 返回 401；
- 匿名 `/api/v1/terminal/status` 返回 401；
- 用户更新后的管理员账号登录返回 200；
- 登录 session 访问 `/api/v1/system` 返回 200。
- 登录验证只记录 HTTP 状态和 `valuesPrinted=false`，未记录账号、密码或 cookie。
## 验证
- `quick_validate.py .agents/skills/unidesk-selfmedia`：skill valid。
- `bun scripts/cli.ts check --syntax-only`：11/11。
- `bun --check scripts/src/secrets.ts`：通过。
- `git diff --check`：通过。
- UniDesk PR #1995 preflight：`MERGEABLE/CLEAN`，已用 merge commit 合并。
docs/MDTODO/hwlab-web-product-experience.md
+4
View File
@@ -73,3 +73,7 @@
### R1.14 [completed]
依据用户最新反馈全量重写 HWLAB v0.3 管理域导航、授权、密钥、用户、账务与 MDTODO 页面代码（子 issue [pikasTech/HWLAB#2522](https://github.com/pikasTech/HWLAB/issues/2522)）：禁止以换色、圆角、字母方块、薄包装组件或局部 CSS 冒充重写；先抽取工业运营台通用工作区、状态轨、实体索引、权限矩阵、密钥流向、审计/账本时间线、真实线性图标与 Markdown/报告 Viewer，再重建五页组件树、状态组合和样式并删除旧私有实现。MDTODO 只保留成熟的有界三栏、任务树→正文→报告、Source/File、inline edit、报告调宽/关闭/全屏和深链思想；保留 typed API、稳定 ID、revision/fingerprint、ACL、Secret 脱敏与公共 Workbench Launch 合同，完成 1920/960/390 原入口验收，完成任务后将详细报告写入[任务报告](./details/hwlab-web-product-experience/R1.14_Task_Report.md)。
## R2 [in_progress]
执行 [HWLAB #2537](https://github.com/pikasTech/HWLAB/issues/2537)：纠正此前名义完成但真实效果不达标的问题，依据 OA SPEC 和 R1_Context 全量重写 HWLAB v0.3 非 Workbench 页面代码；先抽取工业风高信息密度通用表格、卡片、弹窗、日志/Markdown/JSON/Trace Viewer、Inspector 和图标/列表/密度切换，再重写 AC/SK/UR/AB、MDTODO、HWPOD 与 HWPOD Node，补 Python 单文件节点下载及分阶段接入引导；保留 MDTODO 三栏布局思想、typed API、权限/Secret/硬件/账务 authority 和 Workbench 主路径，完成 1920/960/390、ARIA、reduced-motion 与完整异步状态验收，完成任务后将详细报告写入[任务报告](./details/hwlab-web-product-experience/R2_Task_Report.md)。
docs/MDTODO/observability-trace-reliability.md
+50 -2
View File
@@ -57,8 +57,56 @@
### R4.3 [in_progress]
执行 [UniDesk #1949](https://github.com/pikasTech/unidesk/issues/1949)：由 config/hwlab-node-lanes.yaml 选择 transactionalProjector、projectionOutboxRelay、projectionRealtime 单一产品 authority，关闭 live-only direct authority，保持 renderer/status/preflight YAML-first，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.3_Task_Report.md)。
执行 [UniDesk #1949](https://github.com/pikasTech/unidesk/issues/1949) 与 [UniDesk #1981](https://github.com/pikasTech/unidesk/issues/1981)：删除 config/hwlab-node-lanes.yaml、renderer、runtime GitOps 和 CI/CD 中对 Workbench 数据 authority、direct/live/refresh、projector/outbox/realtime 的架构选择与六个布尔 env 硬门禁；HWLAB v0.3 固定纯 Kafka transactional projector→outbox→实时/回放 SSE，YAML 只保留 broker/topic/group/tuning 与产品功能配置，schema 校验仅 warning 且不阻塞滚动上线，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.3_Task_Report.md)。
### R4.4
### R4.4 [in_progress]
依赖 R4.1-R4.3，主代理完成 PR 架构审核、顺序合并、自动 CI/CD 和 NC01/v03 原 Workbench 新 hi turn 验收；核对 command/run 实际耗时、Kafka backlog/lag、运行中增量、terminal/final、刷新 replay、公开 result 与 OTel 一致后收口，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4_Task_Report.md)。
#### R4.4.1 [in_progress]
执行 [HWLAB #2527](https://github.com/pikasTech/HWLAB/issues/2527)：修复 HWLAB PaC source artifact 未吸收 UniDesk owning YAML transactional projection authority 的漂移，以新的正常 PR merge 自动生成 direct/live/replay=false、projector/outbox/realtime=true 的 GitOps manifest；禁止手工 PipelineRun、mirror sync、Argo refresh 和运行面 patch，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.1_Task_Report.md)。
#### R4.4.2 [in_progress]
执行 [UniDesk #1959](https://github.com/pikasTech/unidesk/issues/1959) 及其后续解阻任务：修复 HWLAB PaC source artifact 的 runtime GitOps helper、ARG_MAX、marker 与 overlay 投影问题，并按最新 R4.4.2.10/R6 合同删除六个架构 env 及其 CI 硬门禁；大段逻辑继续使用 repo-native 文件或等价受管制品，产品 schema 失败只 warning 并继续滚动上线，禁止手改生成物、HTTP 补链、页面轮询、第二 authority 和人工 PipelineRun/同步/刷新/patch，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2_Task_Report.md)。
##### R4.4.2.1 [in_progress]
执行 [UniDesk #1963](https://github.com/pikasTech/unidesk/issues/1963)：修复 HWLAB runtime GitOps scripts ConfigMap 的 Git-backed 自动 ownership/交付漂移，只由正常 PR merge 自动事件验收；保持 `UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE`、纯 Kafka transactional projector 与实时/回放 SSE，禁止 B64 回退、HTTP 补链、轮询、第二 authority、人工 PipelineRun/ConfigMap apply/运行面 patch及额外锁租约状态库，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.1_Task_Report.md)。
##### R4.4.2.2 [in_progress]
执行 [HWLAB #2530](https://github.com/pikasTech/HWLAB/issues/2530)：增加 repo-owned PaC 回归验证 `UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE`、禁止 B64 内联并校对 ConfigMap mount/helper 合同，以正常 v0.3 PR merge 触发真实自动交付；保持纯 Kafka transactional projector 与实时/回放 SSE，禁止空提交、人工 PipelineRun/同步/刷新/运行面 patch及额外锁租约状态库，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.2_Task_Report.md)。
##### R4.4.2.3 [in_progress]
执行 [UniDesk #1967](https://github.com/pikasTech/unidesk/issues/1967)：修复 runtime GitOps postprocess 仅 JSON.parse 而跳过真实 YAML workload，使用结构化 YAML API 覆盖普通、多文档与 List 并显式写入六项 `false,false,false,true,true,true` 能力 env；只由正常 PR merge 自动交付和新 HWLAB merge 验收，禁止放宽 required boolean、手工 patch/补跑/同步/刷新及额外锁租约状态库，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.3_Task_Report.md)。
##### R4.4.2.4 [in_progress]
执行 [UniDesk #1968](https://github.com/pikasTech/unidesk/issues/1968)：对齐 runtime GitOps guard 的 typed `continuing/no-build-no-rollout-plan-gitops-verify` 与 PaC terminal evaluator，保持普通 skip 与 verify 模式显式分离且缺证据 fail closed，使 status/history/debug-step 同构；禁止按 TaskRun 成功启发式放宽、人工补跑/同步/刷新/patch及额外状态库锁租约，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.4_Task_Report.md)。
##### R4.4.2.5 [in_progress]
执行 [HWLAB #2532](https://github.com/pikasTech/HWLAB/issues/2532)：修复 prepare-source 仍期待 `require.resolve("yaml")` 的过期测试，结构化验证 CI node dependency bootstrap、renderer dependency-install-disabled 与 runtime GitOps helper 的 YAML 模块可解析合同，跑通全文件测试并以正常 v0.3 PR merge触发最终自动验收；禁止运行时 npm install、B64、手改生成物、空提交及人工补跑/patch，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.5_Task_Report.md)。
##### R4.4.2.6 [in_progress]
执行 [UniDesk #1971](https://github.com/pikasTech/unidesk/issues/1971)：修复 runtime GitOps verify guard 保留旧 `!willRunGitopsPromote` 条件导致 typed continuing marker 永不发出的合同缺口，使无 build/rollout plan 与继续 verify 正交并让 status/history/debug-step 精确返回 runtime-gitops-verify/valid=true；禁止放宽 evaluator、人工补跑/同步/刷新/patch、第二 authority及额外锁租约状态库，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.6_Task_Report.md)。
##### R4.4.2.7 [in_progress]
执行 [HWLAB #2534](https://github.com/pikasTech/HWLAB/issues/2534)：在 UniDesk #1972 自动交付后用受控 source-artifact renderer 重新生成 HWLAB NC01/v03 PaC artifact，使无 build/rollout plan 的 typed continuing marker 与 `willRunGitopsPromote` 解耦，并以正常 v0.3 PR merge 验证 runtime-gitops-verify/valid=true；禁止手改生成物、空提交、人工补跑/同步/刷新/patch、放宽 evaluator及第二 authority，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.7_Task_Report.md)。
##### R4.4.2.8 [in_progress]
执行 [UniDesk #1973](https://github.com/pikasTech/unidesk/issues/1973)：修复 PaC evaluator 对 runtime GitOps verify 真实 marker 组合的误判，允许零运行时变更 plan 下 collect-artifacts skip 与 gitops-promote continuing 共存，同时继续拒绝 promote skipped/continuing 冲突、非零 plan 与终态证据缺失；由正常 PR merge 自动交付后只读复核 `runtime-gitops-verify/valid=true`，禁止人工补跑、同步、刷新、patch及第二 authority，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.8_Task_Report.md)。
##### R4.4.2.9 [in_progress]
执行 [UniDesk #1975](https://github.com/pikasTech/unidesk/issues/1975)：修复 runtime GitOps scripts ConfigMap 丢失 owning YAML `codeAgentRuntime` overlay 导致 postprocess 静默未写纯 Kafka capability env、cloud-api CrashLoop 的问题，并让 verify 在工作负载未命中或 transactional projection capability/env 不完整时 fail closed；以正常 PR merge 自动链和新的 HWLAB source merge 验证 Pod ready、Argo `Synced/Healthy`、projector/outbox/realtime health 归零，禁止人工补跑、同步、刷新、patch及第二 authority，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.9_Task_Report.md)。
##### R4.4.2.10 [in_progress]
执行 [HWLAB #2538](https://github.com/pikasTech/HWLAB/issues/2538) P0 架构纠偏：确认 PR #2540 把已工作的 agentrun.event.v1→hwlab.event.v1→实时 SSE/Kafka retention 回放错误替换为强制 PostgreSQL transactional projector，构成重大架构偏移；从最新 v0.3 分析偏移提交并创建新的精确纠偏提交，恢复 direct publish、live Kafka SSE、Kafka refresh replay 及其单一 Kafka authority，删除强制 v8 schema 启动门禁和数据库迁移依赖，保留 canonical duration 修复、feature-config schema、HTTP fallback 禁令及其他无关改动；通过新 PR 和自动 PaC/GitOps/Argo 上线后校对实时、回放、terminal/final 与 canonical duration，完成任务后将详细报告写入[任务报告](./details/observability-trace-reliability/R4.4.2.10_Task_Report.md)。
docs/MDTODO/pr-merge-driven-automatic-delivery.md
+16
View File
@@ -82,3 +82,19 @@
### R5.3
合并修复 PR 后只观察新的 GitHub webhook→Gitea mirror→PaC→Tekton→GitOps/Argo 自动事件，验证 watcher Ready、TaskRun 创建及 AgentRun/Monitor runtime 收敛，不人工补齐既有 queued run，完成任务后将详细报告写入[任务报告](./details/pr-merge-driven-automatic-delivery/R5.3_Task_Report.md)。
## R6 [in_progress]
执行 [UniDesk #1981](https://github.com/pikasTech/unidesk/issues/1981)：固定仓库产品功能 schema 路径为 config/feature-config.schema.json，CI/CD 只读校验 schema 语法、值匹配和功能到唯一配置变量的一对一关系；缺失、非法、不匹配或重复仅输出 typed warning 并投影到 OTel/status/history/debug-step，必须继续 artifact、GitOps/Argo 和滚动上线，禁止写回、默认值、变量删除、功能关闭、架构切换及其他补救操作，完成任务后将详细报告写入[任务报告](./details/pr-merge-driven-automatic-delivery/R6_Task_Report.md)。
## R7 [in_progress]
执行 [UniDesk #2008](https://github.com/pikasTech/unidesk/issues/2008)：按 PJ2026-01060106 建立 HWLAB 开发/生产双环境发布；v0.3 固定为 NC01 development，release 固定为 production，两个环境在 source、Pipeline、GitOps、Argo、namespace、public origin、PostgreSQL database/role/Secret/migration ledger 和 Kafka consumer group 上隔离，所有值由 owning YAML 控制，完成任务后将详细报告写入[任务报告](./details/pr-merge-driven-automatic-delivery/R7_Task_Report.md)。
### R7.1 [in_progress]
迁移 [UniDesk #2009](https://github.com/pikasTech/unidesk/issues/2009)：保持 v0.3 开发 branch/namespace/DB consumer，删除其对 hwlab.pikapython.com 的所有权，选择并 YAML 固化 NC01 公网 IP 的空闲 HTTP 端口和同源 API/health/probe，通过新 PR 和自动链完成原入口验证，完成任务后将详细报告写入[任务报告](./details/pr-merge-driven-automatic-delivery/R7.1_Task_Report.md)。
### R7.2 [in_progress]
建立 [UniDesk #2010](https://github.com/pikasTech/unidesk/issues/2010)：跟随 release branch 的独立 production PaC/Tekton/GitOps/Argo、namespace、host PostgreSQL database/role/Secret/migration ledger、Kafka consumer group 与 https://hwlab.pikapython.com YAML publicExposure；release 只从完成 P0 Kafka 纠偏并验证的 v0.3 commit 初始化，完成任务后将详细报告写入[任务报告](./details/pr-merge-driven-automatic-delivery/R7.2_Task_Report.md)。
docs/MDTODO/selfmedia-production-delivery.md
+12
View File
@@ -11,3 +11,15 @@
### R1.2 [completed]
有界恢复 NC01 AgentRun manager 受控代理可见性：定位连续 agentrun-proxy-exec-failed，只允许检查并恢复既有 manager.baseUrl、lane service proxy 与授权 presence，不打印凭据，不修改 Artificer 主线源码，不新增安全围栏；恢复后用原始 agentrun events/logs 入口验证并写任务报告，完成任务后将详细报告写入[任务报告](./details/selfmedia-production-delivery/R1.2_Task_Report.md)。
### R1.3 [completed]
重新分发用户更新后的 SelfMedia 管理员凭据，并将 owning YAML、Secret sync/status、自动交付与公网鉴权验收最短路径沉淀到 `unidesk-selfmedia` skill；不得读取或输出凭据值，不得影响其他仓库 token 选择，完成任务后将详细报告写入[任务报告](./details/selfmedia-production-delivery/R1.3_Task_Report.md)。
## R2 [in_progress]
依据 [UniDesk #2006](https://github.com/pikasTech/unidesk/issues/2006) 优化 selfmedia-nc01 PaC 流水线的 env reuse 与依赖缓存：由 owning YAML、lockfile 和 source artifact 生成环境身份，依赖不变时复用并披露 hit/miss 与阶段耗时，依赖变化时正确失效；通过正常 SELFMEDIA PR merge 自动交付验收，保持 commit-pinned artifact、GitOps/Argo、runtime health 和单一 source authority，完成任务后将详细报告写入[任务报告](./details/selfmedia-production-delivery/R2_Task_Report.md)。
## R3 [in_progress]
依据 [UniDesk #2007](https://github.com/pikasTech/unidesk/issues/2007) 为 pikainc/selfmedia release 分支建立独立生产 PaC/GitOps 流水线：master 保持开发 consumer，release 精确驱动独立生产 namespace、Pipeline、GitOps branch/Application、runtime/PVC/Secret 与 YAML-first 公网端口；禁止 branch follower、手工同步和共享可写状态，以正常 release PR merge 验收生产公网入口、登录、digest 与 health，完成任务后将详细报告写入[任务报告](./details/selfmedia-production-delivery/R3_Task_Report.md)。
docs/reference/agentrun.md
+4 -4
View File
@@ -244,12 +244,12 @@ bun scripts/cli.ts agentrun create task \
--repo <owner/repo> \
--ref <branch> \
--mdtodo-id <R6.3> \
--prompt-stdin \
--dry-run
--prompt-stdin
```
- dry-run 与 live 都先经 Target `trans` 重入并执行 source/worktree/Aipod/SecretRef 预检；
- live 的 submit 成功后自动 dispatch；
- 默认直接执行 live 单命令，不把 `--dry-run` 作为派单前置步骤；
- live 先经 Target `trans` 重入并执行 source/worktree/Aipod/SecretRef 只读预检；
- 预检通过后在同一命令内 submit 并自动 dispatch；
- 幂等重试不得重复创建 attempt 或重复 dispatch；
- UniDesk 不实现 AgentRun queue 协议，也不把 task double-write 回旧 Code Queue。
docs/reference/code-queue-supervision.md
+6
View File
@@ -200,6 +200,12 @@ PR 是审查型交付入口，不是所有 Code Queue 任务的默认出口。Un
PR handoff 的职责默认分开：runner 实现、测试、提交、push head branch 并创建 PR；指挥官监督并发、steer、审阅、确认 checks 和合并裁决。短期内 GPT-5.5 runner 如果收到明确 PR 收口授权，并且 PR 是普通 UniDesk source 变更、checks 满足任务要求、无冲突且不涉及 prod/runtime/release/security/database/破坏性回滚，可以自行用 repo-owned GitHub merge/close 路径完成收口并报告 SHA。高风险、边界不清、checks 失败或用户/指挥官保留 final action 的 PR 仍必须交回 commander 审查。host commander 也不把直接编辑业务代码当成常规 PR 替代路径。
- PR 审核默认由指挥官主代理直接完成：
- runner 或 Artificer 的实施授权不包含审核授权；
- 并行实施、多轮审查或 reviewer feedback 不自动创建审核 runner；
- 只有用户明确要求子代理审核、独立审核代理或多代理交叉审核时，才允许派发审核型 AgentRun；
- 主代理发现问题后，优先通过原 session follow-up 纠偏，再由主代理复核。
PR 支持本身是 Code Queue 能力的一部分。当前 UniDesk CLI 支持 `gh pr list|view|create|update|comment create|comment update|comment edit|comment delete|close|reopen`，其中 create 需要显式 `--title`、`--base`、`--head` 和正文来源，update 需要显式 PR number、正文来源和 `--mode replace|append`，comment create/update/edit 需要显式 commentId 或目标 shorthand 和正文来源，且推荐使用 `--body-stdin` 或 `--body-file`。PR 收口观察应使用 `gh pr view <number> --json state,stateDetail,closed,closedAt,merged,mergedAt,mergeCommit,headRefName,baseRefName,mergeable,mergeStateStatus,statusCheckRollup` 获取普通关闭/已合并区分、merge commit、目标分支、源分支、mergeability 和检查汇总；该路径仍是只读元数据，不执行 merge，且 `closeoutMetadata.ok=false`、`missingOrUnknownFields` 或 GraphQL failure 都只能作为“需要人工复核/重试”的信号。`pr create --dry-run`、`pr update --dry-run` 与 `pr comment create/update/edit --dry-run` 只返回 planned operation，不创建 PR、不更新正文、不写评论；非 dry-run 创建前会校验 repo、base、head 和 compare ahead 状态，append 更新会先读取当前 PR body。`gh pr list` 不开放 mergeability/statusCheckRollup 列表字段，避免默认拉取 noisy/raw 状态汇总。`gh pr delete` 在 UniDesk REST CLI 中返回 `unsupported-command`，不能伪造硬删除；需要移除活跃 PR 时使用 `gh pr close`。`gh pr merge` 是 guarded REST write，没有 `--confirm` 可以绕过 preflight；获得收口授权的 GPT-5.5 runner 可使用 UniDesk `gh pr merge`、GitHub UI 或等价 repo-owned GitHub merge path 处理普通 PR，并报告结果 SHA。需要 PR 交付时，prompt 必须明确允许的人工、runner 或后续工具路径，并报告未覆盖范围。
### PR 驱动派单模板
package.json
+4
View File
@@ -19,5 +19,9 @@
"vite": "^8.1.0",
"vue": "3",
"xlsx": "^0.18.5"
},
"dependencies": {
"ajv-dist": "8.17.1",
"yaml": "2.3.4"
}
}
project-management/PJ2026-01/deviations/PJ2026-010401080313-20260714-pure-kafka-architecture-deviation.md
+84
View File
@@ -0,0 +1,84 @@
# PJ2026-010401080313 纯Kafka权威架构偏离复盘
## 1. 事件摘要
| 字段 | 内容 |
| --- | --- |
| 严重度 | P0 开发事故 |
| 日期 | 2026-07-14 |
| 影响规格 | [PJ2026-010401080313 Workbench实时权威](../specs/PJ2026-010401080313-workbench-realtime-authority.md) |
| 关联执行 | pikasTech/HWLAB#2538、PR #2540 |
| 事故类型 | 未经授权改变 event authority、persistence 和 replay 架构，并引入数据库 schema 启动硬前置 |
| 当前状态 | 已停止继续扩展，正在通过新 PR 精确纠偏；禁止粗暴 reset 或覆盖无关修改 |
事故边界如下：
- 原始目标是修复 Workbench timing/duration 与一次 17 分钟投影异常。
- 原始架构要求坚持 `agentrun.event.v1 -> hwlab.event.v1 -> 实时 SSE / Kafka retention 回放 SSE`。
- 实际实现把链路替换为 PostgreSQL transactional projector、facts/outbox 和数据库 replay。
- 实际实现删除 direct publish、live Kafka SSE 与 Kafka refresh replay，构成对用户目标和实时权威的根本偏离。
## 2. 时间线
| 阶段 | 事实 |
| --- | --- |
| 事故前 | 运行配置启用 direct publish、live Kafka SSE、Kafka refresh replay；transactional projector 与 projection outbox relay 关闭。 |
| 任务定义 | #2538 和 MDTODO 被错误写成“删除 direct/live/refresh，固定 transactional projector”，没有先证明 duration 故障需要改变 authority。 |
| 实现 | `3e03e94e` 固定 transactional 投影并修改 timing；`fc87a7e5` 删除 direct/live/refresh；`bef23280` 删除 HTTP 自动补链；`2603b00c` 继续清理旧投影残余。 |
| 合并 | PR #2540 通过 merge commit `6ae0d1b5a48118b6a65a521919de0bc5801e969d` 进入 `v0.3`。 |
| 部署 | PipelineRun/GitOps 成功，但新 Cloud API Pod 因 `workbench_transactional_realtime_schema_blocked` CrashLoop；Argo 为 `Synced/Degraded`，旧 Pod 继续服务。 |
| 判定 | 数据库迁移被误认为新架构的必要条件，随后确认它不是原纯 Kafka实时/回放能力的依赖。 |
| 止损 | 主代理纠正 #2538 与 MDTODO，要求从最新 `v0.3` 建新分支和新 PR，逐文件恢复纯 Kafka路径并保留无关修复。 |
## 3. 影响
- 新 Cloud API Pod 无法启动，滚动发布退化；旧 Pod 暂时维持服务，因此未发生完整业务中断。
- CI/CD 显示成功而运行面 `Degraded`，扩大了“流水线成功等于发布成功”的可见性误差。
- 原本无需数据库迁移的实时/回放链被增加 v8 schema 前置，导致修复路径被错误引向数据库迁移。
- direct publish、live/replay Kafka 代码被删除，回退成本从配置切换扩大为代码纠偏。
- 子代理和主代理的多轮 review 围绕新架构内部一致性优化，消耗时间并偏离原始 Web 增强主线。
## 4. 直接原因
1. 任务合同直接把未经用户授权的 PostgreSQL transactional projector 写成目标架构。
2. PR 删除原 authority 路径，并用启动断言强制新 schema 就绪。
3. review 把“Kafka 仍在链路中”误判为“纯 Kafka”，没有检查 Kafka 是否仍是实时和回放 authority。
4. canonical duration 修复与 authority 替换被耦合在同一 PR，未保持问题与架构变更的因果边界。
## 5. 系统性原因
- 派单前没有写出 current data flow、desired data flow 和两者差异，也没有要求用户授权 authority/persistence/replay 变化。
- 主代理以测试和内部一致性替代对用户目标、最新 SPEC、线上运行配置和数据流的优先审查。
- 较旧“Workbench唯一投影”规格包含数据库 outbox 设计，较新的纯 Kafka要求尚未形成明确优先级裁决；冲突未先解决就进入实现。
- 新测试验证了 transactional 架构，却没有证明该架构是被授权目标；测试把偏移固化成了代码门禁。
- rollout 验证只看到 PipelineRun/GitOps 成功，没有把新 Pod readiness、Argo health 和启动 blocker 作为同一发布判定。
## 6. 主代理责任
主代理对事故负最终责任：主代理创建和接受了错误任务合同，未在派单前完成架构裁决；review 时继续要求删除旧路径；在看到 schema blocker 后仍一度沿数据库迁移方向调查。子代理按错误合同实现不能替代主代理的架构审查责任。
## 7. 检测与恢复
本次事故由新 Pod CrashLoop、Argo `Synced/Degraded` 和 `workbench_transactional_realtime_schema_blocked` 暴露。恢复不执行数据库迁移来承认新架构，而是从最新 `v0.3` 创建精确纠偏提交：
- 恢复 direct publish、live Kafka SSE 和 Kafka retention replay。
- 删除 v8 schema 作为 Cloud API 启动前置。
- 保留 canonical duration、feature config schema、HTTP fallback 禁令和无关修复。
- 通过新的 PR、自动 PaC/GitOps/Argo 上线，并在原入口校对 live/replay SSE、terminal/final 和 canonical duration。
禁止 reset、force push、整提交粗暴反转或人工 runtime patch，以免覆盖 PR #2540 中与架构偏移无关的正确修改。
## 8. 纠正与防复发措施
| 措施 | Owner | 完成判定 |
| --- | --- | --- |
| 更新实时权威 SPEC | 主代理 | 明确纯 Kafka data flow、数据库非阻塞边界和新旧规格优先级。 |
| 精确纠偏 PR #2540 | HWLAB 实施代理 | 新 PR 恢复 live/replay 路径，Cloud API 不再依赖 v8 schema 启动。 |
| 原入口验证 | 主代理审核 | 自动上线后以同一 session/trace 校对实时、刷新回放、terminal/final 与 duration。 |
| 固化派单规则 | 主代理 | `unidesk-subagent` 要求 current/desired data flow 和架构变化授权。 |
| 固化 review 顺序 | 主代理 | 先比较用户目标、SPEC、运行 data flow，再看代码内部一致性和测试。 |
| CI/CD 可见性 | 独立任务 | PipelineRun 成功但 Argo/Pod degraded 时明确报告发布未完成，不以 schema warning 阻塞滚动。 |
## 9. 长期判定
修复 timing、duration、丢事件或 UI 收敛问题时，不得默认改变 event authority、transport、persistence、replay 或 source of truth。若证据表明确需架构变化，必须先更新长期 SPEC，写清 current/desired data flow、迁移与回退方案，并取得用户明确授权；新增测试只能验证已授权架构，不能反向把未经授权的实现合法化。
project-management/PJ2026-01/specs/PJ2026-0104010803-workbench-unique-projection.md
+2
View File
@@ -7,6 +7,8 @@
当前正文仍在规格治理草稿中；未定稿前不新增版本号，不为单次编辑追加 `待提交` 版本。
> 裁决优先级：本规格中以 PostgreSQL aggregate event stream、transactional projector、durable outbox 或 `/v1/workbench/sync` 作为实时/回放 authority 的条款，已被 2026-07-14 的 [PJ2026-010401080313 Workbench实时权威](PJ2026-010401080313-workbench-realtime-authority.md) `draft-2026-07-14-p0-pure-kafka-authority` 取代。当前权威链固定为 `agentrun.event.v1 -> mapper -> hwlab.event.v1 -> 实时 SSE / Kafka retention 回放 SSE`；PostgreSQL 只可作为非阻塞派生读模型，不能成为 Cloud API 启动、Kafka 实时或回放的前置。未完成正文重整前，冲突条款一律以最新专项裁决为准。
## 正文
## PJ2026-0104010803 Workbench唯一投影需求规格
project-management/PJ2026-01/specs/PJ2026-010401080313-workbench-realtime-authority.md
+16 -7
View File
@@ -19,7 +19,7 @@
| 短名 | Workbench实时权威 |
| 层级 | L4 专项规格切片 |
| 状态 | 草稿 |
| 实现引用版本 | draft-2026-07-08-p0-workbench-realtime-authority-v2; draft-2026-07-09-p1-single-step-debug |
| 实现引用版本 | draft-2026-07-08-p0-workbench-realtime-authority-v2; draft-2026-07-09-p1-single-step-debug; draft-2026-07-14-p0-pure-kafka-authority |
| 需求规格模板 | [ISO/IEC/IEEE 29148 需求规格模板](../../templates/iso-iec-ieee-29148-requirements-spec-template.md) |
| 上级规格 | [PJ2026-010401 Web工作台](PJ2026-010401-web-workbench.md) |
| 关联规格 | [PJ2026-0104010803 Workbench唯一投影](PJ2026-0104010803-workbench-unique-projection.md)、[PJ2026-0106050514 Workbench实时运行面](PJ2026-0106050514-workbench-realtime-runtime.md)、[PJ2026-010403 API契约](PJ2026-010403-api-contract.md)、[PJ2026-01060508 Web哨兵](PJ2026-01060508-web-probe-sentinel.md) |
@@ -31,14 +31,21 @@
### 2.1 目的
Workbench实时权威负责把 Workbench 主 timeline、session rail、turn card、Final Response、Trace detail 和 transport diagnostic 的实时输入收敛到同一 durable projection 语义。当前目标不是“永远不用 REST”，而是禁止前端多端点补洞成为 correctness path：SSE typed event 是 live 主路径，`/v1/workbench/sync` 只能作为与 SSE 同语义的统一 replay/delta，initial snapshot 只负责页面初始水位，detail/history 只服务用户显式查看。
Workbench实时权威负责把 Workbench 主 timeline、session rail、turn card、Final Response、Trace detail 和 transport diagnostic 收敛到纯 Kafka 事件权威：
- 固定数据流为 `agentrun.event.v1 -> mapper -> hwlab.event.v1 -> 实时 SSE / Kafka retention 回放 SSE`。
- 浏览器只消费同一 SSE 合同。
- 断线、刷新和 cursor 缺口由服务端按 Kafka retention 回放后继续 live tail。
- `/v1/workbench/sync` 或业务 REST 补链不得成为 correctness path。
PostgreSQL 可以保存查询优化所需的派生读模型，但不是 `agentrun.event.v1` 与 `hwlab.event.v1` 之间的 inline authority，也不是实时或回放 SSE 的启动前置。读模型 schema 缺失、迁移未执行或投影器退化必须形成 warning/diagnostic，不能关闭 direct publish、live SSE、Kafka replay，不能阻塞 Cloud API 启动或滚动上线。
本规格同时定义单步调试工作台。调试工作台必须能在不接触真实运行面、不触发 automatic recovery 和不污染生产 Workbench store 的前提下，用 fake SSE 数据逐条驱动同一 reducer，暴露每一步的 authority decision、entity version、state diff、DOM triad 和禁止请求。
### 2.2 范围内
- `/v1/workbench/events` typed event 的 authority metadata、cursor、entity family/id/version、projectionRevision 和 terminal seal 语义。
- `/v1/workbench/sync` replay/delta 的语义边界：它是 SSE replay 等价物，不是第二套 REST repair 事实源。
- Kafka retention 回放 SSE 的 cursor、去重、顺序和回放到 live tail 的切换语义。
- initial snapshot、explicit detail/history、trace-detail-only payload、session list/detail/messages read model 和主状态投影之间的合并边界。
- 前端 reducer/adapter 的 authority gate、sealed guard、detail-only rejection、late stale rejection 和 diagnostic 输出。
- 自动恢复、SSE error、cursor gap、cross-tab projection signal 和 refresh/reconnect 的允许动作与禁止动作。
@@ -56,9 +63,9 @@ Workbench实时权威负责把 Workbench 主 timeline、session rail、turn card
| 术语 | 定义 |
| --- | --- |
| Realtime Authority v2 | Workbench 主状态只能由 initial snapshot、SSE typed event、统一 `/v1/workbench/sync` replay/delta 和显式 detail/history 中各自被授权的 projection 输入更新的规则集合。 |
| Realtime Authority v2 | Workbench 主状态只能由 initial snapshot、`hwlab.event.v1` 的 live/replay SSE typed event 和显式 detail/history 中各自被授权的输入更新的规则集合。 |
| 高纯度 SSE | Workbench live correctness 优先由 SSE typed event 驱动；断线、重复、乱序和缺口通过同一 durable cursor/replay 语义处理，不回退到多个业务 REST endpoint 自行补事实。 |
| sync replay | `/v1/workbench/sync` 或等价入口按 session/trace scope 与 cursor 返回的 typed delta。它必须能被转换为与 SSE 相同的 reducer event，并携带同一 authority metadata。 |
| Kafka replay SSE | SSE 服务按客户端 cursor 从 `hwlab.event.v1` retention 回放 typed event，追上水位后在同一连接继续 live tail；replay 与 live 使用同一事件 schema 和 reducer。 |
| 多源补洞 | 前端在 automatic recovery 中同时或顺序调用 `/turns`、`/sessions/:id/messages`、`/traces/:id/events`、旧 `/v1/agent/*` 等端点来推断 terminal/final/message/session 主状态。该模式禁止。 |
| detail-only | 只服务 Trace detail、历史页、诊断和审计的 payload。它可以展示过程，不能覆盖主 message/finalResponse/turn/session authority。 |
| terminal seal | 同一 durable projection revision 内写入的 message/part terminal status、turn terminal、Final Response、timing、session running=false 和 cursor checkpoint。 |
@@ -71,7 +78,7 @@ Workbench 主状态只接受以下输入：
- `initial snapshot`：页面初始加载或 explicit user refresh 的水位输入，只能写入其声明的 session/message/turn/trace bucket。
- `SSE typed event`：live delivery 主路径，必须携带 `contractVersion`、`realtimeAuthority`、`entity.family/id/version`、cursor 和 `projectionRevision`。
- `sync replay/delta`：与 SSE 同语义的 replay 输入，必须先转换为 typed event 或同等 reducer action，再进入主 reducer。
- `Kafka replay SSE`：服务端按 cursor 从 `hwlab.event.v1` retention 回放，与 live event 使用同一 schema、authority metadata 和 reducer action。
- `explicit detail/history`：用户显式打开的 trace detail、历史分页或详情读取，只能写 detail bucket 或 diagnostic bucket；声明 `detailProjection=true` 或 `authority=trace-detail-only` 时必须拒绝写主状态。
- `optimistic local echo`：submit admission 前后的本地临时投影，必须有 stable id，对账后由 durable event/sync 覆盖；不得成为跨页面或刷新后的事实源。
@@ -81,6 +88,7 @@ Workbench 主状态只接受以下输入：
- 用 trace tail、trace detail 末行、message text、session list preview、elapsed timeout、localStorage、DOM active card 或 analyzer fallback 推断 `running`、terminal、Final Response 或 session active。
- 让 late session list/detail/messages、detail-only trace payload、transport diagnostic、requestfailed 或 stale running snapshot 覆盖 sealed entity version。
- 用 reload、切换 session、自动 repair helper、测试后门或 probe analyzer suppress 把已经分裂的页面补成通过。
- 用 `/v1/workbench/sync`、数据库 outbox replay、HTTP polling 或读模型查询替代 Kafka retention replay，或在 Kafka replay 可用前要求 PostgreSQL schema 就绪。
## 5. 单步调试工作台
@@ -123,7 +131,7 @@ fake 数据优先来自真实受控样本脱敏后的 fixture。合成 fixture
静态验收：
- 自动恢复路径只允许 `SSE typed event`、`/v1/workbench/sync` replay 或 diagnostic；不得出现旧 `/turns`、`/sessions/:id/messages`、`/traces/:id/events` automatic fan-out 写主状态。
- 自动恢复路径只允许 live/replay `SSE typed event` 或 diagnostic；不得出现 `/v1/workbench/sync`、旧 `/turns`、`/sessions/:id/messages`、`/traces/:id/events` automatic fan-out 写主状态。
- `traceHydration*` 命名和运行时口径必须收敛为 `traceDetail*` 或显式 detail read；旧配置字段如保留，只能作为 deprecated alias。
- `detailProjection=true`、`authority=trace-detail-only` 和缺 authority metadata 的输入必须被 reducer gate 拒绝写主状态，并产生可见 diagnostic。
@@ -133,6 +141,7 @@ fake 数据优先来自真实受控样本脱敏后的 fixture。合成 fixture
- refresh/reconnect/cross-tab signal 后，control/observer/fresh page 不出现 persistent `cross-page-projection-divergence`。
- `workbench-automatic-recovery-fanout-authority`、旧 agent read-through 和旧 Workbench detail fan-out 不回归。
- fake SSE 单步调试可以在无真实 API、无 `/sync`、无补洞的模式下完成 terminal seal、late stale rejection、detail-only rejection 和两页收敛测试。
- Cloud API 在 Workbench 数据库 schema 缺失或读模型迁移未完成时仍能启动，并继续 direct publish、live SSE 与 Kafka retention replay；退化项只形成可见 warning/diagnostic。
## 7. 过程控制
project-management/PJ2026-01/specs/PJ2026-010601-controlled-release.md
+1
View File
@@ -84,6 +84,7 @@
| PJ2026-01060103 | 发布判定 | 本规格 6.3 | source/GitOps/runtime 一致性、health/readiness 和 migration/SecretRef presence | CI/CD、GitOps、YAML运维 | 平台管理员、业务模块 |
| PJ2026-01060104 | 验证分层 | 本规格 6.4 | 自测试、综合联调、CLI/API 交互和真实运行面通过口径 | 目标 runtime、业务模块测试需求 | 发布决策 |
| PJ2026-01060105 | AgentRun发布 | [PJ2026-01060105 AgentRun发布Lane](PJ2026-01060105-agentrun-v01-release-lane.md) | AgentRun `v0.1` 的发布 lane、Pipeline、runtime namespace 和真实联调细则 | 源码同步、YAML运维、Agent编排 | AgentRun runtime |
| PJ2026-01060106 | HWLAB双环境 | [PJ2026-01060106 HWLAB双环境](PJ2026-01060106-hwlab-dev-production-lanes.md) | HWLAB `v0.3` development 与 `release` production 的分支、流水线、公开入口和数据隔离 | 源码同步、YAML运维、Workbench实时权威 | HWLAB runtime |
## 6. 原子需求
project-management/PJ2026-01/specs/PJ2026-01060106-hwlab-dev-production-lanes.md
+182
View File
@@ -0,0 +1,182 @@
# PJ2026-01060106 HWLAB双环境
## 修改历史
| 版本 | 对应 commit id | 更新日期 | 变更说明 |
| --- | --- | --- | --- |
当前正文仍在规格治理草稿中；未定稿前不新增版本号，不为单次编辑追加 `待提交` 版本。
## 正文
## PJ2026-01060106 HWLAB双环境需求规格
## 1. 文档控制
| 字段 | 内容 |
| --- | --- |
| 编号 | PJ2026-01060106 |
| 短名 | HWLAB双环境 |
| 层级 | L3 子课题 |
| 状态 | 草稿 |
| 实现引用版本 | draft-2026-07-14-p0-hwlab-dev-production-lanes |
| 需求规格模板 | [ISO/IEC/IEEE 29148 需求规格模板](../../templates/iso-iec-ieee-29148-requirements-spec-template.md) |
| 上级规格 | [PJ2026-010601 发布流水](PJ2026-010601-controlled-release.md) |
| 关联规格 | [PJ2026-010602 源码同步](PJ2026-010602-source-sync.md)、[PJ2026-010603 YAML运维](PJ2026-010603-yaml-first-ops.md)、[PJ2026-010401080313 Workbench实时权威](PJ2026-010401080313-workbench-realtime-authority.md) |
| 规格治理索引 | [规格治理](spec-governance.md) |
本文定义 HWLAB 在 NC01 上的 development/production 双环境。所有可调事实由 owning YAML 控制，自动发布只由目标 source branch 的 PR merge/branch update 驱动，禁止人工补跑 PipelineRun、手工 Argo sync、运行时 patch 或从集群反解配置。
## 2. 目的和范围
### 2.1 目的
把当前 `v0.3` 固定为开发环境，并建立跟随 `release` 分支的生产环境。两套环境在发布身份、运行资源、公开入口、数据库数据域和 Kafka consumer identity 上隔离，使开发变更可以持续滚动，同时只有经过开发原入口验证的 commit 才能晋升为生产基线。
### 2.2 范围内
- `v0.3` development 与 `release` production 的 source、PaC consumer、Tekton Pipeline、GitOps branch/path、Argo Application、namespace 和 artifact provenance。
- development 的 NC01 公网 IP HTTP 入口与 production 的 `https://hwlab.pikapython.com` 入口。
- 共用 NC01 host PostgreSQL 服务时，两套独立 database、role、Secret consumer 和 migration ledger。
- 两套独立 Kafka consumer group identity，以及纯 Kafka实时/回放权威不被数据库迁移阻塞的边界。
- `v0.3` 到 `release` 的受控晋升、自动部署和原入口验证。
### 2.3 范围外
- Web 页面能力和组件重写归客户端规格。
- AgentRun 自身的发布 lane 和数据库归 AgentRun 专项规格。
- DNS、Caddy、FRP 和证书服务的通用实现归 YAML运维与公开入口平台；本规格只定义 HWLAB 两个 consumer 的所有权。
- 从 development 向 production 复制业务数据、Secret value 或用户会话不在本任务范围内。
## 3. 术语表
| 术语 | 定义 |
| --- | --- |
| development lane | source 为 `v0.3`、保留当前开发 namespace、通过 NC01 公网 IP 空闲 HTTP 端口访问的 HWLAB 环境。 |
| production lane | source 为 `release`、使用独立 production namespace、通过 `https://hwlab.pikapython.com` 访问的 HWLAB 环境。 |
| 晋升 | 把已在 development 完成原入口验证的精确 commit 纳入 `release`，随后由 production 自动链发布。 |
| 数据域隔离 | 两个环境不共享 PostgreSQL database、role、Secret consumer 或 migration ledger，任何 schema/data mutation 只作用于选中 lane。 |
| 入口所有权 | owning YAML 中唯一声明某 public origin 的 consumer；同一 host/origin 不得同时归 development 和 production。 |
## 4. 架构与数据流
### 4.1 系统架构
```mermaid
flowchart LR
V03[HWLAB v0.3] --> DEVPA[development PaC/Pipeline]
DEVPA --> DEVGIT[development GitOps]
DEVGIT --> DEVNS[development namespace]
DEVNS --> DEVHTTP[152.53.229.148:YAML端口]
DEVNS --> DEVDB[(host PostgreSQL: development database)]
REL[HWLAB release] --> PRODPA[production PaC/Pipeline]
PRODPA --> PRODGIT[production GitOps]
PRODGIT --> PRODNS[production namespace]
PRODNS --> PRODHTTPS[https://hwlab.pikapython.com]
PRODNS --> PRODDB[(host PostgreSQL: production database)]
KAFKA[(Kafka)] -->|development consumer group| DEVNS
KAFKA -->|production consumer group| PRODNS
```
### 4.2 运行数据流
```mermaid
flowchart LR
AR[agentrun.event.v1] --> MAP[HWLAB mapper]
MAP --> HE[hwlab.event.v1]
HE -->|development group| DEVSSE[development live/replay SSE]
HE -->|production group| PRODSSE[production live/replay SSE]
DEVSSE --> DEVWEB[development Web]
PRODSSE --> PRODWEB[production Web]
DEVWEB -.显式查询.-> DEVDB[(development read model)]
PRODWEB -.显式查询.-> PRODDB[(production read model)]
```
Kafka event 是实时与回放 authority。数据库只保存各 lane 的业务数据和派生读模型；任一数据库 schema 缺失不得改变 Kafka direct publish/live/replay 路径，也不得成为 Cloud API 启动或自动滚动前置。
### 4.3 晋升与自动发布时序
```mermaid
sequenceDiagram
participant D as v0.3
participant DC as development CI/CD
participant DV as development 原入口验证
participant R as release
participant PC as production CI/CD
participant PV as production 原入口验证
D->>DC: merge/update commit C
DC->>DV: 自动构建、GitOps、rollout
DV-->>D: C 验证通过
D->>R: 受控晋升精确 commit C
R->>PC: branch update 自动触发
PC->>PV: 自动构建、GitOps、rollout
PV-->>R: digest/source/runtime/入口证据
```
初始 `release` 分支只能从完成 P0 纯 Kafka纠偏并在 development 原入口验证通过的 `v0.3` commit 创建。不得从已知强制 PostgreSQL transactional projector 的事故 commit 创建，也不得把 production 自动链的缺失用人工 PipelineRun 或 runtime patch 填补。
## 5. 配置与隔离
### 5.1 YAML-first 配置
owning YAML 至少声明：
- lane id、source repository/branch、PaC consumer、Pipeline、GitOps branch/path、Argo Application 和 namespace。
- public exposure 的 scheme、host、port、target service、health path、Caddy/FRP managed block 引用。
- PostgreSQL host service reference、database、role、Secret sourceRef/targetKey、migration ledger identity。
- Kafka bootstrap SecretRef、topic contract、consumer group identity 和 replay retention/cursor 配置。
- artifact repository、image name、digest provenance 和 source commit label。
代码只校验和渲染 YAML，不内置 development/production 特例，不从现有 Deployment、Secret、Ingress、Caddy 或数据库反解事实。Secret 输出只允许 presence、object/key、fingerprint 和脱敏摘要。
### 5.2 公开入口
- development 正式入口是 `http://152.53.229.148:<development.publicExposure.port>`；实施任务应探测空闲端口并把最终值固化到 owning YAML。
- production 正式入口固定为 `https://hwlab.pikapython.com`，由 production publicExposure 独占。
- development 必须删除 `hwlab.pikapython.com` 的 route、probe、CORS/origin 和 managed block 所有权；Web/API/health 使用同源 development 入口。
- production 证书、Caddy/FRP route 和 health probe 由 YAML 渲染，禁止手改共享配置文件。
### 5.3 PostgreSQL 与 Kafka 隔离
- 两个 lane 可以复用同一 NC01 host PostgreSQL 服务进程，但必须使用不同 database、role、Secret consumer 和 migration ledger。
- migration 命令必须显式选中 lane，只能迁移其 database；禁止用共享 DSN、共享 schema 或默认 database 回退。
- production 初始化不复制 development 会话、用户业务数据或 migration history；需要数据导入时必须另立受控任务。
- development 与 production 使用不同 Kafka consumer group identity。运行状态和 lag 必须带 lane 标签，禁止同名 group 导致 offset 互相推进。
- 数据库、Kafka 或 schema warning 不得触发功能关闭、authority 切换、HTTP fallback 或阻塞其他 lane 滚动上线。
## 6. 原子需求
### 6.1 HWLAB-LANE-REQ-001 Development迁移
`v0.3` 自动链必须继续部署到 development namespace，并迁移到 YAML 声明的 NC01 公网 IP HTTP 入口。迁移完成后 `https://hwlab.pikapython.com` 只指向 production，不再路由到 development。
### 6.2 HWLAB-LANE-REQ-002 Production自动链
`release` 必须拥有独立 PaC consumer、Tekton Pipeline/ServiceAccount、GitOps branch/path、Argo Application、namespace、image provenance 和 release status。`release` update 自动触发完整链，不依赖主代理或值班人员人工补跑。
### 6.3 HWLAB-LANE-REQ-003 Source与制品对齐
每个 lane 的 PipelineRun、GitOps desired state、runtime image digest 和 workload label 必须能关联同一 source commit。缺失或不一致必须可见，但微服务契约、版本或 schema 检查只输出 warning，不得阻塞业务和滚动上线。
### 6.4 HWLAB-LANE-REQ-004 数据边界
任一 lane 的应用、migration、Secret 和管理命令不得连接另一 lane 的 database/role。验收必须证明两个环境写入的 canary 数据、migration ledger 和 consumer group offset 互不出现于对方数据域。
### 6.5 HWLAB-LANE-REQ-005 原入口验收
原入口验收要求如下：
- development 从新的公网 IP HTTP 入口完成 Web、API、health 和至少一次实时/回放 SSE 验收。
- production 从 `https://hwlab.pikapython.com` 完成同样验收。
- production 同时核对 source commit、image digest、namespace、database identity 和 Kafka consumer group。
- 源码测试、PipelineRun 成功或 Argo Synced 不能单独替代原入口通过。
## 7. 过程控制
- 父任务由 pikasTech/unidesk#2008 跟踪。
- development 入口迁移由 pikasTech/unidesk#2009 和 MDTODO `R7.1` 跟踪。
- production release lane 由 pikasTech/unidesk#2010 和 MDTODO `R7.2` 跟踪。
- 两个实施任务可以在独立 worktree 并行；共享 YAML/publicExposure 基线的合并和上线顺序由主代理审核。
- production 实现可以先准备，但 `release` 分支创建、生产部署和公网切换必须等待 P0 纯 Kafka纠偏在 development 验证完成。
scripts/native/cicd/ajv2020-vendor.test.ts
+19
View File
@@ -0,0 +1,19 @@
import { expect, test } from "bun:test";
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
const repositoryRoot = resolve(import.meta.dir, "../../..");
const manifestPath = resolve(repositoryRoot, "scripts/vendor/ajv-dist/8.17.1/manifest.json");
test("repo-owned Ajv 2020 bundle matches the locked dependency and source artifact", () => {
const manifest = JSON.parse(readFileSync(manifestPath, "utf8"));
const packageJson = JSON.parse(readFileSync(resolve(repositoryRoot, "package.json"), "utf8"));
const vendor = readFileSync(resolve(repositoryRoot, manifest.vendorPath));
const source = readFileSync(resolve(repositoryRoot, manifest.sourcePath));
expect(packageJson.dependencies[manifest.package]).toBe(manifest.version);
expect(vendor.byteLength).toBe(manifest.bytes);
expect(createHash("sha256").update(vendor).digest("hex")).toBe(manifest.sha256);
expect(vendor.equals(source)).toBe(true);
});
scripts/native/cicd/feature-config-schema-warning.mjs
+409
View File
@@ -0,0 +1,409 @@
#!/usr/bin/env node
import { randomBytes } from "node:crypto";
import { existsSync, lstatSync, readFileSync, readdirSync } from "node:fs";
import path from "node:path";
import { createRequire } from "node:module";
import { pathToFileURL } from "node:url";
const require = createRequire(import.meta.url);
const cwdRequire = createRequire(path.join(process.cwd(), "package.json"));
const YAML = globalThis.unideskYaml ?? optionalRequire("yaml") ?? optionalRequire("yaml", cwdRequire);
const VALIDATOR_UNAVAILABLE = "FEATURE_CONFIG_VALIDATOR_UNAVAILABLE";
export const FEATURE_CONFIG_SCHEMA_PATH = "config/feature-config.schema.json";
export function validateFeatureConfigSchema(options = {}) {
const repoDir = path.resolve(options.repoDir ?? process.cwd());
const schemaPath = FEATURE_CONFIG_SCHEMA_PATH;
const absoluteSchemaPath = path.join(repoDir, schemaPath);
if (!existsSync(absoluteSchemaPath)) return result("feature-config-schema-missing", "schema file is missing", 1, null);
let schema;
try {
schema = JSON.parse(readFileSync(absoluteSchemaPath, "utf8"));
} catch (error) {
return result("feature-config-schema-invalid", `schema JSON parse failed: ${errorMessage(error)}`, 1, null);
}
const featureErrors = featureDeclarationErrors(schema);
if (featureErrors.invalid.length > 0) return result("feature-config-schema-invalid", featureErrors.invalid[0], featureErrors.invalid.length, null);
if (featureErrors.duplicates.length > 0) return result("feature-config-variable-duplicate", featureErrors.duplicates[0], featureErrors.duplicates.length, null);
let validate;
try {
const Ajv2020 = ajv2020();
const ajv = new Ajv2020({ allErrors: true, strict: true, validateFormats: false });
ajv.addKeyword({ keyword: "x-unidesk-feature", schemaType: "string", valid: true });
validate = ajv.compile(schema);
} catch (error) {
return result(
error?.code === VALIDATOR_UNAVAILABLE ? "feature-config-validator-unavailable" : "feature-config-schema-invalid",
errorMessage(error),
1,
null,
);
}
let candidate;
try {
candidate = options.candidate ?? renderedWorkloadCandidate({
renderedRoot: options.renderedRoot,
propertyNames: Object.keys(schema.properties),
});
} catch (error) {
return result("feature-config-validator-failure", errorMessage(error), 1, null);
}
if (candidate === null || !isRecord(candidate.values) || Object.keys(candidate.values).length === 0) {
return result("feature-config-input-missing", "rendered workload candidate snapshot is missing", 1, candidate?.summary ?? null);
}
if (!validate(candidate.values)) {
const errors = Array.isArray(validate.errors) ? validate.errors : [];
return result("feature-config-value-mismatch", boundedAjvError(errors[0]), errors.length || 1, candidate.summary);
}
return result("feature-config-schema-valid", null, 0, candidate.summary);
}
export function emitFeatureConfigSchemaValidation(options = {}) {
const validation = validateFeatureConfigSchema(options);
process.stderr.write(`${JSON.stringify(validation)}\n`);
return validation;
}
export async function runFeatureConfigSchemaValidation(options = {}) {
let validation;
try {
validation = emitFeatureConfigSchemaValidation(options);
} catch (error) {
validation = result("feature-config-validator-failure", errorMessage(error), 1, null);
process.stderr.write(`${JSON.stringify(validation)}\n`);
}
try {
const endpoint = options.otelEndpoint ?? process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT ?? "";
const timeoutMs = positiveInteger(options.otelTimeoutMs ?? process.env.OTEL_EXPORTER_TIMEOUT_MS);
const samplingRatio = samplingRatioValue(options.otelSamplingRatio ?? process.env.OTEL_TRACES_SAMPLER_ARG);
if (endpoint.length === 0) {
const exportWarning = otelWarning("ENDPOINT_MISSING", null, false, null, timeoutMs);
process.stderr.write(`${JSON.stringify(exportWarning)}\n`);
return { validation, otelPayload: null, otelWarning: exportWarning };
}
if (timeoutMs === null) {
const exportWarning = otelWarning("TIMEOUT_MISSING", null, false, null, null);
process.stderr.write(`${JSON.stringify(exportWarning)}\n`);
return { validation, otelPayload: null, otelWarning: exportWarning };
}
const otelPayload = featureConfigOtelPayload(validation, {
serviceName: options.otelServiceName ?? process.env.OTEL_SERVICE_NAME ?? "unidesk-cicd",
consumer: options.consumer ?? process.env.UNIDESK_PAC_CONSUMER ?? null,
samplingRatio,
traceparent: options.traceparent ?? process.env.TRACEPARENT ?? process.env.traceparent ?? "",
nowMs: options.nowMs,
traceId: options.traceId,
spanId: options.spanId,
});
const exportWarning = await exportFeatureConfigOtel(endpoint, otelPayload, timeoutMs, options.fetchImpl ?? fetch);
if (exportWarning !== null) process.stderr.write(`${JSON.stringify(exportWarning)}\n`);
return { validation, otelPayload, otelWarning: exportWarning };
} catch (error) {
const exportWarning = otelWarning("EXPORT_INTERNAL_FAILURE", null, false, error, positiveInteger(options.otelTimeoutMs ?? process.env.OTEL_EXPORTER_TIMEOUT_MS));
process.stderr.write(`${JSON.stringify(exportWarning)}\n`);
return { validation, otelPayload: null, otelWarning: exportWarning };
}
}
export function featureConfigOtelPayload(validation, options = {}) {
const parent = parseTraceparent(options.traceparent);
const traceId = options.traceId ?? parent.traceId ?? randomBytes(16).toString("hex");
const spanId = options.spanId ?? randomBytes(8).toString("hex");
const nowMs = options.nowMs ?? Date.now();
const startTimeUnixNano = String(BigInt(nowMs) * 1_000_000n);
return {
resourceSpans: [{
resource: { attributes: otelAttributes({
"service.name": options.serviceName ?? "unidesk-cicd",
"unidesk.pac.consumer": options.consumer ?? null,
"unidesk.otel.sampling_ratio": options.samplingRatio ?? null,
"unidesk.values_redacted": true,
}) },
scopeSpans: [{
scope: { name: "unidesk.cicd.feature-config", version: "v1" },
spans: [{
traceId,
spanId,
...(parent.spanId === null ? {} : { parentSpanId: parent.spanId }),
name: "cicd.feature_config.schema",
kind: 1,
startTimeUnixNano,
endTimeUnixNano: startTimeUnixNano,
attributes: otelAttributes({
"feature_config.warning": validation.warning,
"feature_config.blocking": false,
"feature_config.code": validation.code,
"feature_config.schema_path": validation.schemaPath,
"feature_config.error_count": validation.errorCount,
}),
events: [{
timeUnixNano: startTimeUnixNano,
name: "feature-config-schema-validation",
attributes: otelAttributes({
"feature_config.warning": validation.warning,
"feature_config.code": validation.code,
"feature_config.first_error": validation.firstError,
}),
}],
status: { code: 1 },
}],
}],
}],
};
}
async function exportFeatureConfigOtel(endpoint, payload, timeoutMs, fetchImpl) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const response = await fetchImpl(endpoint, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(payload),
signal: controller.signal,
});
if (response.ok) return null;
return otelWarning(`HTTP_${response.status}`, response.status, false, null, timeoutMs);
} catch (error) {
return otelWarning(error instanceof Error && error.name === "AbortError" ? "TIMEOUT" : "EXPORT_FAILED", null, error instanceof Error && error.name === "AbortError", error, timeoutMs);
} finally {
clearTimeout(timeout);
}
}
function featureDeclarationErrors(schema) {
if (!isRecord(schema) || schema.type !== "object" || !isRecord(schema.properties)) {
return { invalid: ["schema root must be an object schema with properties"], duplicates: [] };
}
const invalid = [];
const variablesByFeature = new Map();
for (const [propertyName, propertySchema] of Object.entries(schema.properties)) {
if (!isRecord(propertySchema)) {
invalid.push(`property ${propertyName} schema must be an object`);
continue;
}
const feature = propertySchema["x-unidesk-feature"];
if (typeof feature !== "string" || feature.trim().length === 0) {
invalid.push(`property ${propertyName} must declare x-unidesk-feature`);
continue;
}
const variables = variablesByFeature.get(feature.trim()) ?? [];
variables.push(propertyName);
variablesByFeature.set(feature.trim(), variables);
}
const duplicates = [...variablesByFeature.entries()]
.filter(([, variables]) => variables.length > 1)
.map(([feature, variables]) => `feature ${feature} maps to multiple properties: ${variables.join(",")}`);
return { invalid, duplicates };
}
function renderedWorkloadCandidate({ renderedRoot, propertyNames }) {
if (typeof renderedRoot !== "string" || renderedRoot.length === 0 || !existsSync(renderedRoot) || YAML === null) return null;
const wanted = new Set(propertyNames);
const values = {};
let fileCount = 0;
let workloadCount = 0;
let matchedValueCount = 0;
for (const file of listStructuredFiles(renderedRoot)) {
fileCount += 1;
for (const document of parseDocuments(file)) {
for (const item of kubernetesItems(document)) {
const podSpec = item?.spec?.template?.spec;
if (!isRecord(podSpec)) continue;
workloadCount += 1;
for (const group of ["containers", "initContainers"]) {
for (const container of Array.isArray(podSpec[group]) ? podSpec[group] : []) {
for (const env of Array.isArray(container?.env) ? container.env : []) {
if (!wanted.has(env?.name) || typeof env?.value !== "string") continue;
values[env.name] = parseRenderedValue(env.value);
matchedValueCount += 1;
}
}
}
}
}
}
return {
values,
summary: {
source: "rendered-workload-env",
renderedRoot: path.relative(process.cwd(), path.resolve(renderedRoot)) || ".",
fileCount,
workloadCount,
matchedValueCount,
propertyCount: propertyNames.length,
valuesPrinted: false,
},
};
}
function listStructuredFiles(root) {
const maxFiles = 2_000;
const maxDirectories = 2_000;
const files = [];
const pending = [path.resolve(root)];
let directoryCount = 0;
while (pending.length > 0) {
const current = pending.pop();
directoryCount += 1;
if (directoryCount > maxDirectories) throw new Error(`rendered workload directory limit exceeded: ${maxDirectories}`);
for (const name of readdirSync(current)) {
const file = path.join(current, name);
const stat = lstatSync(file);
if (stat.isSymbolicLink()) continue;
if (stat.isDirectory()) pending.push(file);
else if (/\.(?:ya?ml|json)$/iu.test(name)) {
files.push(file);
if (files.length > maxFiles) throw new Error(`rendered workload file limit exceeded: ${maxFiles}`);
}
}
}
return files.sort();
}
function parseDocuments(file) {
const text = readFileSync(file, "utf8");
if (/\.json$/iu.test(file)) {
try { return [JSON.parse(text)]; } catch { return []; }
}
try { return YAML.parseAllDocuments(text).map((document) => document.toJS()).filter((document) => document !== null); } catch { return []; }
}
function kubernetesItems(document) {
return document?.kind === "List" && Array.isArray(document.items) ? document.items : [document];
}
function parseRenderedValue(value) {
try { return JSON.parse(value); } catch { return value; }
}
function boundedAjvError(error) {
if (!isRecord(error)) return "candidate does not match schema";
const instancePath = typeof error.instancePath === "string" ? error.instancePath : "";
const keyword = typeof error.keyword === "string" ? error.keyword : "validation";
const message = typeof error.message === "string" ? error.message : "failed";
return `${instancePath || "/"} ${keyword} ${message}`;
}
function result(code, firstError, errorCount, candidate) {
const warning = errorCount > 0;
const boundedError = typeof firstError === "string" ? firstError.replace(/\s+/gu, " ").trim().slice(0, 240) : null;
return {
event: "feature-config-schema-validation",
warning,
blocking: false,
code,
schemaPath: FEATURE_CONFIG_SCHEMA_PATH,
errorCount,
firstError: boundedError,
candidate,
mutation: false,
valuesPrinted: false,
otel: {
spanEvent: "cicd.feature_config.schema",
attributes: {
"feature_config.warning": warning,
"feature_config.code": code,
"feature_config.schema_path": FEATURE_CONFIG_SCHEMA_PATH,
"feature_config.error_count": errorCount,
"feature_config.blocking": false,
},
},
};
}
function optionalRequire(name, loader = require) {
try { return loader(name); } catch { return null; }
}
function ajv2020() {
if (globalThis.unideskAjv2020) return globalThis.unideskAjv2020;
if (typeof process.env.UNIDESK_AJV2020_BUNDLE === "string" && process.env.UNIDESK_AJV2020_BUNDLE.length > 0) {
return bundledAjv(process.env.UNIDESK_AJV2020_BUNDLE);
}
try {
return require("ajv-dist/dist/ajv2020.bundle.js");
} catch {
throw validatorUnavailable("Ajv 2020 validator dependency is unavailable");
}
}
function bundledAjv(file) {
if (!existsSync(file)) throw validatorUnavailable("Ajv 2020 validator bundle is missing");
try {
const module = { exports: {} };
Function("module", "exports", readFileSync(file, "utf8"))(module, module.exports);
const exported = module.exports.default ?? module.exports;
if (typeof exported !== "function") throw new TypeError("bundle export is not a constructor");
return exported;
} catch (error) {
if (error?.code === VALIDATOR_UNAVAILABLE) throw error;
throw validatorUnavailable(`Ajv 2020 validator bundle is corrupt (${error instanceof Error ? error.name : "Error"})`);
}
}
function validatorUnavailable(message) {
const error = new Error(message);
error.code = VALIDATOR_UNAVAILABLE;
return error;
}
function isRecord(value) {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function errorMessage(error) {
return error instanceof Error ? error.message : String(error);
}
function positiveInteger(value) {
const parsed = Number(value);
return Number.isInteger(parsed) && parsed > 0 ? parsed : null;
}
function samplingRatioValue(value) {
const parsed = Number(value);
return Number.isFinite(parsed) && parsed >= 0 && parsed <= 1 ? parsed : null;
}
function parseTraceparent(value) {
const match = /^00-([0-9a-f]{32})-([0-9a-f]{16})-[0-9a-f]{2}$/iu.exec(String(value));
return { traceId: match?.[1]?.toLowerCase() ?? null, spanId: match?.[2]?.toLowerCase() ?? null };
}
function otelAttributes(values) {
return Object.entries(values).filter(([, value]) => value !== null && value !== undefined).map(([key, value]) => ({
key,
value: typeof value === "boolean"
? { boolValue: value }
: typeof value === "number"
? { intValue: String(value) }
: { stringValue: String(value) },
}));
}
function otelWarning(causeCode, httpStatus, timedOut, error, timeoutMs) {
return {
event: "cicd.otel.export",
warning: true,
blocking: false,
causeCode,
...(httpStatus === null ? {} : { httpStatus }),
timedOut,
errorType: error instanceof Error ? error.name : null,
timeoutMs,
valuesRedacted: true,
};
}
if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
const renderedRootIndex = process.argv.indexOf("--rendered-root");
await runFeatureConfigSchemaValidation({ renderedRoot: renderedRootIndex === -1 ? null : process.argv[renderedRootIndex + 1] });
process.exitCode = 0;
}
scripts/native/cicd/feature-config-schema-warning.test.ts
+248
View File
@@ -0,0 +1,248 @@
import { afterEach, expect, test } from "bun:test";
import { mkdtempSync, mkdirSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { spawnSync } from "node:child_process";
import {
featureConfigOtelPayload,
runFeatureConfigSchemaValidation,
validateFeatureConfigSchema,
} from "./feature-config-schema-warning.mjs";
const roots: string[] = [];
const validatorScript = resolve(import.meta.dir, "feature-config-schema-warning.mjs");
const vendorBundle = resolve(import.meta.dir, "../../vendor/ajv-dist/8.17.1/ajv2020.min.js");
afterEach(() => {
while (roots.length > 0) rmSync(roots.pop()!, { recursive: true, force: true });
});
test("valid nested Draft 2020-12 candidate remains warning-free", () => {
const root = fixture(schema({
tracePanel: feature("trace-panel", {
type: "object",
required: ["enabled", "views"],
properties: {
enabled: { type: "boolean" },
views: { type: "array", minItems: 1, items: { type: "string", pattern: "^[a-z-]+$" } },
},
additionalProperties: false,
}),
}), { tracePanel: '{"enabled":true,"views":["timeline"]}' });
expect(validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") })).toMatchObject({
warning: false,
blocking: false,
code: "feature-config-schema-valid",
errorCount: 0,
mutation: false,
candidate: { source: "rendered-workload-env", matchedValueCount: 1 },
});
});
test("repo-owned Ajv 2020 bundle performs real validation without workspace dependencies", () => {
const root = fixture(schema({ tracePanel: feature("trace-panel", { type: "boolean" }) }), { tracePanel: "true" });
const run = runValidator(root, vendorBundle);
expect(run.status).toBe(0);
expect(run.stderr).toContain('"code":"feature-config-schema-valid"');
expect(run.stderr).not.toContain("feature-config-validator-unavailable");
});
test("missing or corrupt Ajv bundle is a typed warning and exits zero", () => {
const root = fixture(schema({ tracePanel: feature("trace-panel", { type: "boolean" }) }), { tracePanel: "true" });
const missing = runValidator(root, join(root, "missing-ajv2020.min.js"));
expect(missing.status).toBe(0);
expect(missing.stderr).toContain('"code":"feature-config-validator-unavailable"');
const corruptBundle = join(root, "corrupt-ajv2020.min.js");
writeFileSync(corruptBundle, "this is not JavaScript");
const corrupt = runValidator(root, corruptBundle);
expect(corrupt.status).toBe(0);
expect(corrupt.stderr).toContain('"code":"feature-config-validator-unavailable"');
});
test("missing schema is a typed non-blocking warning", () => {
const root = fixture();
expect(validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") })).toMatchObject({
warning: true,
blocking: false,
code: "feature-config-schema-missing",
});
});
test("invalid JSON and unsupported Draft 2020-12 schema are typed warnings", () => {
const invalidJsonRoot = fixture("{");
expect(validateFeatureConfigSchema({ repoDir: invalidJsonRoot, renderedRoot: join(invalidJsonRoot, "rendered") }).code).toBe("feature-config-schema-invalid");
const invalidSchemaRoot = fixture(JSON.stringify({ $schema: "https://json-schema.org/draft/2020-12/schema", type: "object", properties: { tracePanel: { type: "not-a-json-schema-type", "x-unidesk-feature": "trace-panel" } } }));
expect(validateFeatureConfigSchema({ repoDir: invalidSchemaRoot, renderedRoot: join(invalidSchemaRoot, "rendered") }).code).toBe("feature-config-schema-invalid");
});
test("missing rendered candidate is distinct from schema failure", () => {
const root = fixture(schema({ tracePanel: feature("trace-panel", { type: "boolean" }) }));
expect(validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "missing") })).toMatchObject({
warning: true,
blocking: false,
code: "feature-config-input-missing",
});
});
test("nested value mismatch is bounded and does not disclose the value", () => {
const root = fixture(schema({ tracePanel: feature("trace-panel", { type: "object", required: ["enabled"], properties: { enabled: { const: true } } }) }), {
tracePanel: '{"enabled":false,"secret":"do-not-print"}',
});
const result = validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") });
expect(result).toMatchObject({ warning: true, blocking: false, code: "feature-config-value-mismatch", valuesPrinted: false });
expect(result.firstError).not.toContain("do-not-print");
});
test("duplicate feature properties are reported without blocking", () => {
const root = fixture(schema({
tracePanel: feature("trace-panel", { type: "boolean" }),
tracePanelAlias: feature("trace-panel", { type: "boolean" }),
}));
expect(validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") })).toMatchObject({
warning: true,
blocking: false,
code: "feature-config-variable-duplicate",
errorCount: 1,
});
});
test("actual OTLP span and event carry redacted validation attributes", () => {
const root = fixture();
const validation = validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") });
const payload = featureConfigOtelPayload(validation, {
serviceName: "unidesk-cicd",
consumer: "hwlab-nc01-v03",
samplingRatio: 1,
nowMs: 1,
traceId: "1".repeat(32),
spanId: "2".repeat(16),
});
const span = payload.resourceSpans[0].scopeSpans[0].spans[0];
expect(span.name).toBe("cicd.feature_config.schema");
expect(span.events[0].name).toBe("feature-config-schema-validation");
expect(JSON.stringify(payload)).not.toContain("do-not-print");
expect(JSON.stringify(payload)).toContain("unidesk.otel.sampling_ratio");
});
test("missing endpoint remains a non-blocking OTel warning", async () => {
const root = fixture();
const result = await runFeatureConfigSchemaValidation({
repoDir: root,
renderedRoot: join(root, "rendered"),
otelEndpoint: "",
otelTimeoutMs: 300,
});
expect(result.otelWarning).toMatchObject({
event: "cicd.otel.export",
warning: true,
blocking: false,
causeCode: "ENDPOINT_MISSING",
timeoutMs: 300,
});
});
test("export failure uses configured timeout and remains non-blocking", async () => {
const root = fixture();
let requestSignal: AbortSignal | null = null;
const result = await runFeatureConfigSchemaValidation({
repoDir: root,
renderedRoot: join(root, "rendered"),
otelEndpoint: "http://otel.example.test/v1/traces",
otelServiceName: "unidesk-cicd",
otelSamplingRatio: 1,
otelTimeoutMs: 300,
fetchImpl: async (_input: URL | RequestInfo, init?: RequestInit) => {
requestSignal = init?.signal ?? null;
throw new Error("collector unavailable");
},
});
expect(requestSignal).not.toBeNull();
expect(result.otelWarning).toMatchObject({
event: "cicd.otel.export",
warning: true,
blocking: false,
causeCode: "EXPORT_FAILED",
timeoutMs: 300,
});
});
test("unexpected validator failure is typed and never rejects", async () => {
const options: Record<string, unknown> = {
repoDir: fixture(schema({ webProbeSentinel: feature("web-probe-sentinel", { type: "object" }) })),
renderedRoot: "unused",
otelEndpoint: "",
otelTimeoutMs: 300,
};
Object.defineProperty(options, "candidate", { get: () => { throw new Error("unexpected candidate failure"); } });
const result = await runFeatureConfigSchemaValidation(options);
expect(result.validation).toMatchObject({
warning: true,
blocking: false,
code: "feature-config-validator-failure",
mutation: false,
});
expect(result.otelWarning).toMatchObject({ warning: true, blocking: false, causeCode: "ENDPOINT_MISSING" });
});
test("candidate scan ignores directory symlinks and bounds directory traversal", () => {
const root = fixture(schema({ webProbeSentinel: feature("web-probe-sentinel", { type: "boolean" }) }), {
webProbeSentinel: "true",
});
symlinkSync(join(root, "rendered"), join(root, "rendered", "loop"), "dir");
expect(validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") })).toMatchObject({
warning: false,
code: "feature-config-schema-valid",
});
for (let index = 0; index <= 2_000; index += 1) mkdirSync(join(root, "rendered", `directory-${index}`));
expect(validateFeatureConfigSchema({ repoDir: root, renderedRoot: join(root, "rendered") })).toMatchObject({
warning: true,
blocking: false,
code: "feature-config-validator-failure",
firstError: "rendered workload directory limit exceeded: 2000",
});
});
function fixture(content?: string, env: Record<string, string> = {}) {
const root = mkdtempSync(join(tmpdir(), "feature-config-schema-"));
roots.push(root);
mkdirSync(join(root, "rendered"));
if (content !== undefined) {
mkdirSync(join(root, "config"));
writeFileSync(join(root, "config", "feature-config.schema.json"), content);
}
writeFileSync(join(root, "rendered", "deployment.yaml"), deployment(env));
return root;
}
function runValidator(root: string, bundlePath: string) {
return spawnSync(process.execPath, [validatorScript, "--rendered-root", join(root, "rendered")], {
cwd: root,
encoding: "utf8",
env: {
...process.env,
UNIDESK_AJV2020_BUNDLE: bundlePath,
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "",
OTEL_EXPORTER_TIMEOUT_MS: "300",
},
});
}
function schema(properties: Record<string, unknown>) {
return JSON.stringify({
$schema: "https://json-schema.org/draft/2020-12/schema",
type: "object",
properties,
additionalProperties: false,
});
}
function feature(name: string, value: Record<string, unknown>) {
return { ...value, "x-unidesk-feature": name };
}
function deployment(env: Record<string, string>) {
const rows = Object.entries(env).map(([name, value]) => ` - name: ${name}\n value: ${JSON.stringify(value)}`).join("\n");
return `apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: fixture\nspec:\n template:\n spec:\n containers:\n - name: fixture\n env:\n${rows || " []"}\n`;
}
scripts/native/cicd/hwlab-node-control-plane-refresh.mjs
+20 -2
View File
@@ -336,10 +336,16 @@ function injectRuntimeGitopsGuard(pipeline) {
const step = steps.find((item) => typeof recordOrNull(item)?.script === "string" && item.script.includes("scripts/gitops-render.mjs"));
if (!step) throw new Error("runtime GitOps guard injection failed: gitops-promote render step missing");
const originalScript = String(step.script);
let script = patchNoBuildNoRolloutSkip(originalScript);
const noBuildSkipPatched = script !== originalScript;
let script = patchNoBuildNoRolloutCondition(originalScript);
const noBuildConditionPatched = script !== originalScript;
const beforeSkipPatch = script;
script = patchNoBuildNoRolloutSkip(script);
const noBuildSkipPatched = script !== beforeSkipPatch;
const beforeInjection = script;
script = injectRuntimeGitopsCommands(script);
if (hasWillRunGitopsPromoteCoupling(script)) {
throw new Error("runtime GitOps guard injection failed: no-build-no-rollout marker remains coupled to willRunGitopsPromote");
}
if (hasNoBuildNoRolloutEarlyExit(script)) {
throw new Error("runtime GitOps guard injection failed: no-build-no-rollout-plan early exit remains after patch");
}
@@ -354,12 +360,20 @@ function injectRuntimeGitopsGuard(pipeline) {
present: true,
configMapName: runtimeGitopsConfigMapName,
stepName: stringOrNull(step.name),
noBuildConditionPatched,
noBuildSkipPatched,
postprocessInjected,
verifyInjected,
};
}
function patchNoBuildNoRolloutCondition(script) {
return String(script).replace(
/const willRunGitopsPromote = plan\.ciCdPlan && plan\.ciCdPlan\.willRunGitopsPromote === true;\nprocess\.exit\(!willRunGitopsPromote && buildServices\.length === 0 && affectedServices\.length === 0 \? 0 : 1\);/u,
"process.exit(buildServices.length === 0 && affectedServices.length === 0 ? 0 : 1);",
);
}
function patchNoBuildNoRolloutSkip(script) {
return String(script).replace(
/(^|\n)([ \t]*)echo '\{"event":"gitops-promote","status":"skipped","reason":"no-build-no-rollout-plan"\}'(?:\s*>\s*&2)?\n[ \t]*exit 0(?=\n|$)/u,
@@ -367,6 +381,10 @@ function patchNoBuildNoRolloutSkip(script) {
);
}
function hasWillRunGitopsPromoteCoupling(script) {
return /process\.exit\(!willRunGitopsPromote && buildServices\.length === 0 && affectedServices\.length === 0 \? 0 : 1\);/u.test(String(script));
}
function hasNoBuildNoRolloutEarlyExit(script) {
return /echo '\{"event":"gitops-promote","status":"skipped","reason":"no-build-no-rollout-plan"\}'(?:\s*>\s*&2)?\n[ \t]*exit 0(?=\n|$)/u.test(String(script));
}
scripts/native/cicd/otel-runtime-config.ts
+7 -8
View File
@@ -28,8 +28,7 @@ export interface CicdOtelExportWarning {
}
export function loadCicdOtelRuntimeConfig(
node: string,
lane: string,
repositoryId: string,
env: Record<string, string | undefined> = process.env,
): CicdOtelRuntimeConfig {
const root = materializeYamlComposition(
@@ -37,7 +36,7 @@ export function loadCicdOtelRuntimeConfig(
{ label: "platform-infra-pipelines-as-code", stripTemplateKeys: true },
).value;
const observability = record(root.observability, `${pacConfigPath}#observability`);
const repository = repositoryFor(root.repositories, node, lane);
const repository = selectCicdOtelRepository(root.repositories, repositoryId);
const params = record(repository.params, `${pacConfigPath}#repositories.${repository.id}.params`);
const fallbackCodes: string[] = [];
const endpoint = authoritativeValue(
@@ -117,12 +116,12 @@ export async function exportCicdOtelPayload(
}
}
function repositoryFor(value: unknown, node: string, lane: string): Record<string, unknown> {
export function selectCicdOtelRepository(value: unknown, repositoryId: string): Record<string, unknown> {
if (!Array.isArray(value)) throw new Error(`${pacConfigPath}#repositories must be an array`);
const expectedId = `sentinel-${node.toLowerCase()}-${lane.toLowerCase()}`;
const repository = value.find((item) => recordOrNull(item)?.id === expectedId);
if (repository === undefined) throw new Error(`${pacConfigPath} has no repository ${expectedId}`);
return record(repository, `${pacConfigPath}#repositories.${expectedId}`);
if (!/^[A-Za-z0-9][A-Za-z0-9._-]*$/u.test(repositoryId)) throw new Error(`invalid PaC repository id ${repositoryId}`);
const repositories = value.filter((item) => recordOrNull(item)?.id === repositoryId);
if (repositories.length !== 1) throw new Error(`${pacConfigPath} requires exactly one repository ${repositoryId}; matched ${repositories.length}`);
return record(repositories[0], `${pacConfigPath}#repositories.${repositoryId}`);
}
function authoritativeValue(
scripts/native/cicd/pac-status-evaluator.cjs
+143 -6
View File
@@ -25,6 +25,41 @@ function exactSkipMarker(records, event) {
&& item.reason === "no-build-no-rollout-plan") || null;
}
function exactRuntimeGitopsVerifyMarker(records) {
return [...records].reverse().find((item) => item.event === "gitops-promote"
&& item.status === "continuing"
&& item.reason === "no-build-no-rollout-plan-gitops-verify") || null;
}
function featureConfigSchemaObservation(records) {
const item = [...records].reverse().find((recordValue) => recordValue.event === "feature-config-schema-validation") || null;
if (item === null) return null;
const candidate = record(item.candidate);
const code = stringOrNull(item.code);
const schemaPath = stringOrNull(item.schemaPath);
const firstError = stringOrNull(item.firstError);
return {
warning: item.warning === true,
blocking: false,
code: code === null ? "feature-config-schema-invalid" : code.slice(0, 100),
schemaPath: schemaPath === null ? "config/feature-config.schema.json" : schemaPath.slice(0, 200),
errorCount: Number.isInteger(item.errorCount) ? Math.max(0, item.errorCount) : null,
firstError: firstError === null ? null : firstError.replace(/\s+/gu, " ").trim().slice(0, 240),
candidate: Object.keys(candidate).length === 0 ? null : {
source: stringOrNull(candidate.source),
renderedRoot: stringOrNull(candidate.renderedRoot),
fileCount: Number.isInteger(candidate.fileCount) ? candidate.fileCount : null,
workloadCount: Number.isInteger(candidate.workloadCount) ? candidate.workloadCount : null,
matchedValueCount: Number.isInteger(candidate.matchedValueCount) ? candidate.matchedValueCount : null,
propertyCount: Number.isInteger(candidate.propertyCount) ? candidate.propertyCount : null,
valuesPrinted: false,
},
spanEvent: "cicd.feature_config.schema",
mutation: false,
valuesPrinted: false,
};
}
const pacContractTasks = new Set(["plan-artifacts", "collect-artifacts", "gitops-promote"]);
function firstString(...values) {
@@ -362,13 +397,15 @@ function terminalSource(task, marker, markerStatus = "skipped", markerSource = "
function extractPacSourceObservation(recordsInput) {
const records = Array.isArray(recordsInput) ? recordsInput.map(record) : [];
const featureConfigSchema = featureConfigSchemaObservation(records);
const plan = [...records].reverse().find((item) => item.event === "pac-delivery-plan" || item.event === "g14-ci-plan") || null;
const collectMarker = exactSkipMarker(records, "collect-artifacts");
const promoteMarker = exactSkipMarker(records, "gitops-promote");
const runtimeGitopsVerifyMarker = exactRuntimeGitopsVerifyMarker(records);
const planTask = exactTaskTerminal(records, "plan-artifacts");
const collectTask = exactTaskTerminal(records, "collect-artifacts");
const promoteTask = exactTaskTerminal(records, "gitops-promote");
if (plan === null && collectMarker === null && promoteMarker === null && planTask === null && collectTask === null && promoteTask === null) return null;
if (plan === null && collectMarker === null && promoteMarker === null && runtimeGitopsVerifyMarker === null && planTask === null && collectTask === null && promoteTask === null) return null;
const affected = stringArray(plan?.affectedServices);
const rollout = stringArray(plan?.rolloutServices);
@@ -384,6 +421,8 @@ function extractPacSourceObservation(recordsInput) {
const noRuntimeChangeMarkers = collectMarker !== null && promoteMarker !== null;
const skipMarkerSeen = collectMarker !== null || promoteMarker !== null;
const skipMarkerConflict = skipMarkerSeen && !noRuntimeChangePlan;
const runtimeGitopsVerifyMarkerConflict = runtimeGitopsVerifyMarker !== null
&& (!noRuntimeChangePlan || promoteMarker !== null);
const deliveryTerminalReady = planTask?.status === "succeeded"
&& collectTask?.status === "succeeded"
&& promoteTask?.status === "succeeded";
@@ -394,11 +433,15 @@ function extractPacSourceObservation(recordsInput) {
let mode = "inconsistent";
let valid = false;
let reason = "structured-plan-or-terminal-marker-is-incomplete";
if (planShapeValid && sourceCommitValid && noRuntimeChangePlan && noRuntimeChangeMarkers) {
if (planShapeValid && sourceCommitValid && noRuntimeChangePlan && noRuntimeChangeMarkers && runtimeGitopsVerifyMarker === null) {
mode = "no-runtime-change";
valid = true;
reason = "no-build-no-rollout-plan";
} else if (planShapeValid && sourceCommitValid && !noRuntimeChangePlan && !skipMarkerConflict && deliveryTerminalReady) {
} else if (planShapeValid && sourceCommitValid && noRuntimeChangePlan && runtimeGitopsVerifyMarker !== null && !runtimeGitopsVerifyMarkerConflict && deliveryTerminalReady) {
mode = "runtime-gitops-verify";
valid = true;
reason = "no-build-no-rollout-plan-gitops-verify";
} else if (planShapeValid && sourceCommitValid && !noRuntimeChangePlan && runtimeGitopsVerifyMarker === null && !skipMarkerConflict && deliveryTerminalReady) {
mode = "delivery";
valid = true;
reason = "artifact-or-runtime-delivery-required";
@@ -406,10 +449,14 @@ function extractPacSourceObservation(recordsInput) {
reason = "source-commit-invalid-or-missing";
} else if (!planShapeValid) {
reason = "delivery-plan-shape-invalid-or-missing";
} else if (runtimeGitopsVerifyMarkerConflict) {
reason = "runtime-gitops-verify-marker-conflicts-with-plan-or-skip-marker";
} else if (skipMarkerConflict) {
reason = "no-runtime-change-marker-conflicts-with-delivery-plan";
} else if (deliveryTerminalFailed) {
reason = "delivery-terminal-failed";
} else if (noRuntimeChangePlan && runtimeGitopsVerifyMarker !== null) {
reason = "runtime-gitops-verify-terminal-evidence-incomplete";
} else if (!noRuntimeChangePlan) {
reason = "delivery-terminal-evidence-incomplete";
}
@@ -440,8 +487,11 @@ function extractPacSourceObservation(recordsInput) {
terminalSources: {
planArtifacts: terminalSource(planTask, plan, "observed", planMarkerSource),
collectArtifacts: terminalSource(collectTask, collectMarker),
gitopsPromote: terminalSource(promoteTask, promoteMarker),
gitopsPromote: runtimeGitopsVerifyMarker === null
? terminalSource(promoteTask, promoteMarker)
: terminalSource(promoteTask, runtimeGitopsVerifyMarker, "continuing", "runtime-gitops-verify-structured-log"),
},
featureConfigSchema,
valuesPrinted: false,
};
}
@@ -602,14 +652,14 @@ function evaluatePacStatus(inputValue) {
const deliveryDisabled = artifact.imageStatus === "disabled";
const deliverySkipped = artifact.imageStatus === "skipped";
const sourceObservationMode = stringOrNull(sourceObservation.mode);
const hwlabCatalogDelivery = sourceObservationMode === "delivery"
const hwlabCatalogDelivery = (sourceObservationMode === "delivery" || sourceObservationMode === "runtime-gitops-verify")
&& sourceObservation.contract === "hwlab-g14-ci-plan";
const noRuntimeChange = sourceObservationMode === "no-runtime-change";
const catalog = record(artifact.catalog);
const catalogPresent = catalog.present === true;
const catalogSourceCommit = stringOrNull(catalog.sourceCommit);
const catalogSourceMatches = catalogSourceCommit !== null && catalogSourceCommit === sourceCommit;
const catalogDeliveryEvidence = sourceObservationMode === "delivery"
const catalogDeliveryEvidence = (sourceObservationMode === "delivery" || sourceObservationMode === "runtime-gitops-verify")
&& sourceObservation.valid === true
&& catalogPresent
&& catalog.status === "published"
@@ -892,6 +942,25 @@ function deliveryObservation(sourceCommit = "c".repeat(40)) {
]);
}
function runtimeGitopsVerifyObservation(sourceCommit = "c".repeat(40)) {
return extractPacSourceObservation([
{
event: "g14-ci-plan",
sourceCommitId: sourceCommit,
affectedServices: [],
rolloutServices: [],
buildServices: [],
reusedServices: Array.from({ length: 8 }, (_item, index) => `service-${index}`),
noImageBuildReason: "test-only-change",
},
{ event: "collect-artifacts", status: "skipped", reason: "no-build-no-rollout-plan" },
{ event: "gitops-promote", status: "continuing", reason: "no-build-no-rollout-plan-gitops-verify" },
{ event: "pac-task-terminal", pipelineTask: "plan-artifacts", taskRun: "plan", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "collect-artifacts", taskRun: "collect", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "gitops-promote", taskRun: "promote", status: "succeeded", reason: "Succeeded", results: [] },
]);
}
function publishedCatalog(sourceCommit = "c".repeat(40)) {
return { present: true, status: "published", sourceCommit, registryVerified: true };
}
@@ -902,6 +971,7 @@ function runPacStatusFixtureChecks() {
const revision = "b".repeat(40);
const retained = noRuntimeChangeObservation();
const delivery = deliveryObservation();
const runtimeGitopsVerify = runtimeGitopsVerifyObservation();
const exactArgo = { sync: "Synced", health: "Healthy", revision, revisionRelation: { relation: "exact", expectedRevision: revision, observedRevision: revision } };
const cases = [
{
@@ -910,6 +980,24 @@ function runPacStatusFixtureChecks() {
expectedCode: "pac-ready-no-runtime-change",
input: fixtureInput({ artifact: { imageStatus: "retained", sourceObservation: retained } }),
},
{
id: "runtime-gitops-verify-ready",
expectedOk: true,
expectedCode: "pac-ready-gitops-exact",
input: fixtureInput({ artifact: { imageStatus: "reused", digest: digestA, digests: [digestA], gitopsCommit: revision, sourceObservation: runtimeGitopsVerify, catalog: publishedCatalog() }, argo: exactArgo }),
},
{
id: "runtime-gitops-verify-catalog-missing",
expectedOk: false,
expectedCode: "pac-artifact-catalog-missing",
input: fixtureInput({ artifact: { imageStatus: "reused", digest: digestA, digests: [digestA], gitopsCommit: revision, sourceObservation: runtimeGitopsVerify }, argo: exactArgo }),
},
{
id: "runtime-gitops-verify-gitops-missing",
expectedOk: false,
expectedCode: "pac-gitops-missing",
input: fixtureInput({ artifact: { imageStatus: "reused", digest: digestA, digests: [digestA], sourceObservation: runtimeGitopsVerify, catalog: publishedCatalog() }, argo: exactArgo }),
},
{
id: "retained-gitops-missing",
expectedOk: false,
@@ -1000,6 +1088,55 @@ function runPacStatusFixtureChecks() {
expectedCode: "pac-source-observation-inconsistent",
input: fixtureInput({ artifact: { sourceObservation: extractPacSourceObservation([{ event: "g14-ci-plan", sourceCommitId: "c".repeat(40), affectedServices: [], rolloutServices: [], buildServices: [], reusedServices: [] }]) } }),
},
{
id: "runtime-gitops-verify-marker-missing",
expectedOk: false,
expectedCode: "pac-source-observation-inconsistent",
input: fixtureInput({ artifact: { sourceObservation: extractPacSourceObservation([
{ event: "g14-ci-plan", sourceCommitId: "c".repeat(40), affectedServices: [], rolloutServices: [], buildServices: [], reusedServices: ["service"] },
{ event: "pac-task-terminal", pipelineTask: "plan-artifacts", taskRun: "plan", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "collect-artifacts", taskRun: "collect", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "gitops-promote", taskRun: "promote", status: "succeeded", reason: "Succeeded", results: [] },
]) } }),
},
{
id: "runtime-gitops-verify-promote-marker-conflict",
expectedOk: false,
expectedCode: "pac-source-observation-inconsistent",
input: fixtureInput({ artifact: { sourceObservation: extractPacSourceObservation([
{ event: "g14-ci-plan", sourceCommitId: "c".repeat(40), affectedServices: [], rolloutServices: [], buildServices: [], reusedServices: ["service"] },
{ event: "collect-artifacts", status: "skipped", reason: "no-build-no-rollout-plan" },
{ event: "gitops-promote", status: "skipped", reason: "no-build-no-rollout-plan" },
{ event: "gitops-promote", status: "continuing", reason: "no-build-no-rollout-plan-gitops-verify" },
{ event: "pac-task-terminal", pipelineTask: "plan-artifacts", taskRun: "plan", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "collect-artifacts", taskRun: "collect", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "gitops-promote", taskRun: "promote", status: "succeeded", reason: "Succeeded", results: [] },
]) } }),
},
{
id: "runtime-gitops-verify-nonzero-plan-conflict",
expectedOk: false,
expectedCode: "pac-source-observation-inconsistent",
input: fixtureInput({ artifact: { sourceObservation: extractPacSourceObservation([
{ event: "g14-ci-plan", sourceCommitId: "c".repeat(40), affectedServices: ["service"], rolloutServices: ["service"], buildServices: [], reusedServices: ["service"] },
{ event: "gitops-promote", status: "continuing", reason: "no-build-no-rollout-plan-gitops-verify" },
{ event: "pac-task-terminal", pipelineTask: "plan-artifacts", taskRun: "plan", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "collect-artifacts", taskRun: "collect", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "gitops-promote", taskRun: "promote", status: "succeeded", reason: "Succeeded", results: [] },
]) } }),
},
{
id: "runtime-gitops-verify-terminal-evidence-missing",
expectedOk: false,
expectedCode: "pac-source-observation-inconsistent",
input: fixtureInput({ artifact: { sourceObservation: extractPacSourceObservation([
{ event: "g14-ci-plan", sourceCommitId: "c".repeat(40), affectedServices: [], rolloutServices: [], buildServices: [], reusedServices: ["service"] },
{ event: "collect-artifacts", status: "skipped", reason: "no-build-no-rollout-plan" },
{ event: "gitops-promote", status: "continuing", reason: "no-build-no-rollout-plan-gitops-verify" },
{ event: "pac-task-terminal", pipelineTask: "plan-artifacts", taskRun: "plan", status: "succeeded", reason: "Succeeded", results: [] },
{ event: "pac-task-terminal", pipelineTask: "collect-artifacts", taskRun: "collect", status: "succeeded", reason: "Succeeded", results: [] },
]) } }),
},
{
id: "retained-source-mismatch",
expectedOk: false,
scripts/native/cicd/publish-unidesk-host-gitops.mjs
+80 -2
View File
@@ -2,6 +2,8 @@
import { spawnSync } from "node:child_process";
import { mkdirSync, readFileSync, rmSync, unlinkSync, writeFileSync } from "node:fs";
import { dirname, resolve } from "node:path";
import { runtimeGitopsScriptsConfigMap } from "../hwlab/runtime-gitops-scripts-configmap.mjs";
import { hwlabRuntimeLaneSpecForNode } from "../../src/hwlab-node-lanes.ts";
function option(name) {
const index = process.argv.indexOf(name);
@@ -43,6 +45,49 @@ function safeReleaseStatePath(value) {
return path;
}
function safeResourceManifestPath(value, pathLabel) {
const path = required(value, pathLabel);
if (path.startsWith("/") || path.split("/").includes("..") || !path.endsWith(".yaml")) throw new Error(`${pathLabel} must be a safe relative .yaml path`);
return path;
}
export function renderGitOpsResources(gitops, sourceRoot) {
if (gitops.resources === undefined) return [];
if (!Array.isArray(gitops.resources)) throw new Error("delivery.gitops.resources must be an array");
return gitops.resources.map((value, index) => {
const pathLabel = `delivery.gitops.resources[${index}]`;
const resource = record(value, pathLabel);
const renderer = required(resource.renderer, `${pathLabel}.renderer`);
if (renderer !== "hwlab-runtime-gitops-scripts") throw new Error(`${pathLabel}.renderer is unsupported: ${renderer}`);
const configRef = required(resource.configRef, `${pathLabel}.configRef`);
const match = /^config\/hwlab-node-lanes\.yaml#lanes\.([^.]+)\.targets\.([^.]+)$/u.exec(configRef);
if (match === null) throw new Error(`${pathLabel}.configRef must select config/hwlab-node-lanes.yaml#lanes.<lane>.targets.<node>`);
const lane = match[1];
const node = match[2];
const spec = hwlabRuntimeLaneSpecForNode(lane, node);
const overlay = {
nodeId: spec.nodeId,
lane: spec.lane,
runtimePath: spec.runtimePath,
observability: spec.observability,
};
const configMapName = `${spec.pipeline}-runtime-gitops-scripts`;
const manifest = Bun.YAML.stringify(runtimeGitopsScriptsConfigMap({
overlay,
scriptsDir: resolve(sourceRoot, "scripts/native/hwlab"),
featureConfigValidatorPath: resolve(sourceRoot, "scripts/native/cicd/feature-config-schema-warning.mjs"),
ajvBundlePath: resolve(sourceRoot, "scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js"),
namespace: required(resource.namespace, `${pathLabel}.namespace`),
configMapName,
}));
return {
id: required(resource.id, `${pathLabel}.id`),
path: safeResourceManifestPath(resource.manifestPath, `${pathLabel}.manifestPath`),
content: `${manifest.trimEnd()}\n`,
};
});
}
function releaseValue(releaseDir, name) {
return required(readFileSync(resolve(releaseDir, name), "utf8").trim(), `${releaseDir}/${name}`);
}
@@ -100,6 +145,7 @@ const writeUrl = required(gitops.writeUrl, "delivery.gitops.writeUrl");
const branch = required(gitops.branch, "delivery.gitops.branch");
const manifestPath = safeManifestPath(gitops.manifestPath);
const releaseStatePath = safeReleaseStatePath(gitops.releaseStatePath);
const gitopsResources = action === "skip" ? [] : renderGitOpsResources(gitops, sourceRoot);
let digest = null;
let digestRef = null;
let manifest = null;
@@ -146,6 +192,26 @@ if (action === "skip") {
digest = existingState.digest;
digestRef = existingState.digestRef;
runtimeSourceCommit = existingState.sourceCommit;
const gitopsCommit = run("git", ["rev-parse", "HEAD"], worktree).stdout.trim();
process.stdout.write(`${JSON.stringify({
ok: true,
phase: "gitops-publish",
action,
reason,
status: "skipped",
imageStatus: "skipped",
sourceCommit,
baselineSourceCommit,
runtimeSourceCommit,
digest,
digestRef,
gitopsCommit,
resources: [],
changed: false,
pushAttempts: 0,
valuesPrinted: false,
})}\n`);
return;
} else if (action === "build" && manifest !== null) {
mkdirSync(dirname(targetPath), { recursive: true });
writeFileSync(targetPath, manifest, "utf8");
@@ -163,12 +229,23 @@ if (action === "skip") {
removeFile(statePath);
}
for (const resource of gitopsResources) {
const resourcePath = resolve(worktree, resource.path);
if (!resourcePath.startsWith(`${worktree}/`)) throw new Error(`resolved resource path escaped the GitOps worktree: ${resource.id}`);
if (enabled) {
mkdirSync(dirname(resourcePath), { recursive: true });
writeFileSync(resourcePath, resource.content, "utf8");
} else {
removeFile(resourcePath);
}
}
run("git", ["config", "user.name", required(author.name, "delivery.gitops.author.name")], worktree);
run("git", ["config", "user.email", required(author.email, "delivery.gitops.author.email")], worktree);
run("git", ["add", "-A"], worktree);
const changed = run("git", ["diff", "--cached", "--quiet"], worktree, true).status !== 0;
if (changed) {
const commitAction = action === "build" ? "deploy" : "remove";
const commitAction = action === "disabled" ? "remove" : "deploy";
run("git", ["commit", "-m", `unidesk-host: ${commitAction} ${serviceRef} ${sourceCommit.slice(0, 12)}`], worktree);
}
@@ -208,10 +285,11 @@ process.stdout.write(`${JSON.stringify({
digest,
digestRef,
gitopsCommit,
resources: gitopsResources.map((resource) => ({ id: resource.id, path: resource.path })),
changed,
pushAttempts,
valuesPrinted: false,
})}\n`);
}
if (!process.execArgv.includes("--check")) main();
if (import.meta.main && !process.execArgv.includes("--check")) main();
scripts/native/cicd/publish-unidesk-host-gitops.test.ts
+115
View File
@@ -0,0 +1,115 @@
import { afterEach, describe, expect, test } from "bun:test";
import { spawnSync } from "node:child_process";
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { rootPath } from "../../src/config";
const temporaryRoots: string[] = [];
afterEach(() => {
for (const root of temporaryRoots.splice(0)) rmSync(root, { recursive: true, force: true });
});
describe("UniDesk Host GitOps publisher", () => {
test("skip preserves the GitOps branch and every managed resource", () => {
const root = mkdtempSync(join(tmpdir(), "unidesk-host-gitops-skip-"));
temporaryRoots.push(root);
const seed = join(root, "seed");
const bare = join(root, "gitops.git");
const releaseDir = join(root, "release");
const publisherWorktree = join(root, "publisher");
const configPath = join(root, "config.yaml");
const statePath = "deploy/gitops-state/unidesk-host/todo-note.json";
const resourcePath = "deploy/gitops/unidesk-host/hwlab-runtime-gitops-scripts.yaml";
const runtimeSourceCommit = "a".repeat(40);
const sourceCommit = "b".repeat(40);
const digest = `sha256:${"c".repeat(64)}`;
mkdirSync(seed);
git(["init", "-b", "unidesk-host-gitops"], seed);
identity(seed);
write(seed, "deploy/gitops/unidesk-host/todo-note.yaml", "existing service manifest\n");
write(seed, resourcePath, "stale configmap must remain unchanged\n");
write(seed, statePath, `${JSON.stringify({
version: 1,
kind: "UniDeskHostReleaseState",
serviceRef: "services.todoNote",
sourceCommit: runtimeSourceCommit,
digest,
digestRef: `registry.example/todo-note@${digest}`,
}, null, 2)}\n`);
git(["add", "."], seed);
git(["commit", "-m", "seed gitops"], seed);
const originalHead = git(["rev-parse", "HEAD"], seed).stdout.trim();
git(["init", "--bare", bare], root);
git(["remote", "add", "origin", `file://${bare}`], seed);
git(["push", "origin", "unidesk-host-gitops"], seed);
mkdirSync(releaseDir);
writeFileSync(join(releaseDir, "action"), "skip\n");
writeFileSync(join(releaseDir, "reason"), "runtime-inputs-unchanged\n");
writeFileSync(join(releaseDir, "baseline-source-commit"), `${runtimeSourceCommit}\n`);
writeFileSync(configPath, Bun.YAML.stringify({
delivery: {
enabled: true,
serviceRef: "services.todoNote",
image: { repository: "registry.example/todo-note" },
gitops: {
readUrl: `file://${bare}`,
writeUrl: `file://${bare}`,
branch: "unidesk-host-gitops",
manifestPath: "deploy/gitops/unidesk-host/todo-note.yaml",
releaseStatePath: statePath,
resources: [{
id: "hwlab-runtime-gitops-scripts",
renderer: "hwlab-runtime-gitops-scripts",
configRef: "config/hwlab-node-lanes.yaml#lanes.v03.targets.NC01",
manifestPath: resourcePath,
namespace: "hwlab-ci",
}],
author: { name: "UniDesk Test", email: "unidesk-test@example.invalid" },
},
},
}));
const result = spawnSync("bun", [
"scripts/native/cicd/publish-unidesk-host-gitops.mjs",
"--config", configPath,
"--source-root", rootPath(),
"--metadata", join(root, "unused-metadata.json"),
"--release-dir", releaseDir,
"--source-commit", sourceCommit,
"--worktree", publisherWorktree,
], { cwd: rootPath(), encoding: "utf8", timeout: 30_000 });
expect(result.status, result.stderr).toBe(0);
const output = JSON.parse(result.stdout) as Record<string, any>;
expect(output).toMatchObject({ action: "skip", status: "skipped", changed: false, pushAttempts: 0, resources: [] });
expect(output.gitopsCommit).toBe(originalHead);
expect(git(["rev-parse", "refs/heads/unidesk-host-gitops"], bare).stdout.trim()).toBe(originalHead);
expect(show(bare, resourcePath)).toBe("stale configmap must remain unchanged\n");
expect(JSON.parse(show(bare, statePath)).sourceCommit).toBe(runtimeSourceCommit);
});
});
function write(root: string, relativePath: string, content: string) {
const path = join(root, relativePath);
mkdirSync(dirname(path), { recursive: true });
writeFileSync(path, content);
}
function identity(root: string) {
git(["config", "user.name", "UniDesk Test"], root);
git(["config", "user.email", "unidesk-test@example.invalid"], root);
}
function show(bare: string, relativePath: string): string {
return git(["show", `refs/heads/unidesk-host-gitops:${relativePath}`], bare).stdout;
}
function git(args: string[], cwd: string) {
const result = spawnSync("git", args, { cwd, encoding: "utf8" });
if (result.status !== 0) throw new Error(`git ${args.join(" ")} failed: ${result.stderr || result.stdout}`);
return result;
}
scripts/native/cicd/render-sentinel-publish-task.test.ts
+19 -4
View File
@@ -3,7 +3,7 @@ import { mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { rootPath } from "../../src/config";
import { exportCicdOtelPayload, loadCicdOtelRuntimeConfig } from "./otel-runtime-config";
import { exportCicdOtelPayload, loadCicdOtelRuntimeConfig, selectCicdOtelRepository } from "./otel-runtime-config";
const roots: string[] = [];
@@ -12,7 +12,7 @@ afterEach(() => {
});
test("unexpanded Repository inputs fall back to materialized owning YAML", () => {
const config = loadCicdOtelRuntimeConfig("NC01", "v03", {
const config = loadCicdOtelRuntimeConfig("sentinel-nc01-v03", {
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "{{otel_traces_endpoint}}",
OTEL_SERVICE_NAME: "{{otel_service_name}}",
OTEL_TRACES_SAMPLER_ARG: "{{otel_sampling_ratio}}",
@@ -34,7 +34,7 @@ test("unexpanded Repository inputs fall back to materialized owning YAML", () =>
});
test("valid but mismatched runtime inputs cannot override owning YAML", () => {
const config = loadCicdOtelRuntimeConfig("NC01", "v03", {
const config = loadCicdOtelRuntimeConfig("sentinel-nc01-v03", {
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "http://collector.example.invalid:4318/v1/traces",
OTEL_SERVICE_NAME: "different-service",
OTEL_TRACES_SAMPLER_ARG: "0.5",
@@ -53,7 +53,7 @@ test("valid but mismatched runtime inputs cannot override owning YAML", () => {
});
test("legal owning endpoint and non-blocking exporter failure share the bounded helper", async () => {
const config = loadCicdOtelRuntimeConfig("NC01", "v03", {
const config = loadCicdOtelRuntimeConfig("sentinel-nc01-v03", {
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "http://otel-collector.platform-infra.svc.cluster.local:4318/v1/traces",
OTEL_SERVICE_NAME: "unidesk-cicd-sentinel-nc01",
OTEL_TRACES_SAMPLER_ARG: "1",
@@ -81,6 +81,14 @@ test("legal owning endpoint and non-blocking exporter failure share the bounded
});
});
test("repository identity selection fails closed on zero or multiple exact matches", () => {
expect(() => selectCicdOtelRepository([], "sentinel-nc01-v03")).toThrow("matched 0");
expect(() => selectCicdOtelRepository([
{ id: "sentinel-nc01-v03", params: {} },
{ id: "sentinel-nc01-v03", params: {} },
], "sentinel-nc01-v03")).toThrow("matched 2");
});
test("actual renderer replaces unexpanded inputs without changing business exit", async () => {
const result = await render({ OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "{{otel_traces_endpoint}}" }, false);
expect(result.exitCode).toBe(0);
@@ -89,6 +97,12 @@ test("actual renderer replaces unexpanded inputs without changing business exit"
expect(result.stderr).toContain('"endpointHostname":"otel-collector.platform-infra.svc.cluster.local"');
expect(result.stderr).not.toContain("{{otel_traces_endpoint}}");
expect(readFileSync(join(result.outputDir, "source.sh"), "utf8")).toContain("http://otel-collector.platform-infra.svc.cluster.local:4318/v1/traces");
const publish = readFileSync(join(result.outputDir, "publish.sh"), "utf8");
expect(publish).toContain("UNIDESK_PAC_CONSUMER='sentinel-nc01-v03'");
expect(publish).toContain("feature-config-schema-validation");
expect(publish).toContain("cicd.feature_config.schema");
expect(publish).toContain("OTEL_EXPORTER_TIMEOUT_MS='300'");
expect(publish).toContain("feature-config-validator-process-failure");
});
async function render(env: Record<string, string>, traced = true) {
@@ -100,6 +114,7 @@ async function render(env: Record<string, string>, traced = true) {
"scripts/native/cicd/render-sentinel-publish-task.ts",
"--node", "NC01",
"--lane", "v03",
"--repository-id", "sentinel-nc01-v03",
"--sentinel", "nc01-web-probe-sentinel",
"--pipeline-run", "hwlab-web-probe-sentinel-nc01-test",
"--source-commit", "a".repeat(40),
scripts/native/cicd/render-sentinel-publish-task.ts
+12 -4
View File
@@ -9,6 +9,7 @@ import {
sentinelPublishSourceShell,
} from "../../src/hwlab-node-web-sentinel-cicd-jobs";
import { exportCicdOtelPayload, loadCicdOtelRuntimeConfig } from "./otel-runtime-config";
import { renderPacFeatureConfigSchemaStage } from "../../src/pac-feature-config-schema-stage";
const options = parseOptions(process.argv.slice(2));
const sourceAuthority = options.sourceAuthority === "gitea-snapshot" ? "gitea-snapshot" : fail("--source-authority must be gitea-snapshot");
@@ -26,7 +27,7 @@ const state = loadSentinelCicdState(
);
const outputDir = resolve(options.outputDir);
mkdirSync(outputDir, { recursive: true });
const otelConfig = loadCicdOtelRuntimeConfig(options.node, options.lane);
const otelConfig = loadCicdOtelRuntimeConfig(options.repositoryId);
const otelSetup = [
`export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=${shellQuote(otelConfig.endpoint)}`,
`export OTEL_EXPORTER_TIMEOUT_MS=${shellQuote(String(otelConfig.timeoutMs))}`,
@@ -38,6 +39,12 @@ const traceSetup = [
"if [ -f /workspace/meta/tracestate ]; then TRACESTATE=$(cat /workspace/meta/tracestate); export TRACESTATE; fi",
].join("\n");
const proxySetup = sentinelImageBuildProxyEnv(state).map(({ name, value }) => `export ${name}=${shellQuote(value)}`).join("\n");
const featureConfigSchemaStage = renderPacFeatureConfigSchemaStage({
repositoryId: options.repositoryId,
consumer: options.repositoryId,
repoDirExpression: "'/workspace/source'",
renderedRootExpression: '"$gitops_worktree"',
});
writeExecutable("source.sh", [
"#!/bin/sh",
"set -eu",
@@ -49,7 +56,7 @@ writeExecutable("source.sh", [
"if [ -n \"$incoming_tracestate\" ]; then printf '%s' \"$incoming_tracestate\" > /workspace/meta/tracestate; fi",
].join("\n"));
writeExecutable("image-build.sh", ["#!/bin/sh", "set -eu", otelSetup, traceSetup, proxySetup, sentinelPublishImageBuildShell(state, options.pipelineRun)].join("\n"));
writeExecutable("publish.sh", ["#!/bin/sh", "set -eu", otelSetup, traceSetup, proxySetup, sentinelPublishShell(state, options.pipelineRun, true)].join("\n"));
writeExecutable("publish.sh", ["#!/bin/sh", "set -eu", otelSetup, traceSetup, proxySetup, sentinelPublishShell(state, options.pipelineRun, true, featureConfigSchemaStage)].join("\n"));
emitConfigFallbackWarning();
await emitOuterSpan();
process.stdout.write(`${JSON.stringify({ ok: true, action: "render-sentinel-publish-task", node: options.node, lane: options.lane, sentinel: options.sentinel, pipelineRun: options.pipelineRun, sourceCommit: options.sourceCommit, outputDir, traceId: traceId(process.env.TRACEPARENT), valuesRedacted: true })}\n`);
@@ -60,7 +67,7 @@ function writeExecutable(name: string, content: string): void {
chmodSync(path, 0o755);
}
function parseOptions(args: string[]): Record<"node" | "lane" | "sentinel" | "pipelineRun" | "sourceCommit" | "sourceStageRef" | "sourceAuthority" | "outputDir", string> {
function parseOptions(args: string[]): Record<"node" | "lane" | "repositoryId" | "sentinel" | "pipelineRun" | "sourceCommit" | "sourceStageRef" | "sourceAuthority" | "outputDir", string> {
const values = new Map<string, string>();
for (let index = 0; index < args.length; index += 2) {
const key = args[index];
@@ -72,6 +79,7 @@ function parseOptions(args: string[]): Record<"node" | "lane" | "sentinel" | "pi
const result = {
node: required("node"),
lane: required("lane"),
repositoryId: required("repository-id"),
sentinel: required("sentinel"),
pipelineRun: required("pipeline-run"),
sourceCommit: required("source-commit"),
@@ -80,7 +88,7 @@ function parseOptions(args: string[]): Record<"node" | "lane" | "sentinel" | "pi
outputDir: required("output-dir"),
};
if (!/^[0-9a-f]{40}$/u.test(result.sourceCommit)) fail("--source-commit must be a 40-character lowercase Git commit");
for (const key of ["node", "lane", "sentinel", "pipelineRun"] as const) if (!/^[A-Za-z0-9._-]+$/u.test(result[key])) fail(`--${key} must be a simple id`);
for (const key of ["node", "lane", "repositoryId", "sentinel", "pipelineRun"] as const) if (!/^[A-Za-z0-9._-]+$/u.test(result[key])) fail(`--${key.replace(/[A-Z]/gu, (letter) => `-${letter.toLowerCase()}`)} must be a simple id`);
if (!result.sourceStageRef.startsWith("refs/unidesk/snapshots/")) fail("--source-stage-ref must be an immutable UniDesk snapshot ref");
return result;
}
scripts/native/hwlab/runtime-gitops-pipeline-guard.mjs
+28 -32
View File
@@ -4,6 +4,7 @@
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import path from "node:path";
import { createRequire } from "node:module";
import { runtimeGitopsScriptsConfigMap } from "./runtime-gitops-scripts-configmap.mjs";
const requireFromScript = createRequire(import.meta.url);
const requireFromCwd = createRequire(path.join(process.cwd(), "package.json"));
@@ -53,10 +54,16 @@ function injectRuntimeGitopsGuard(pipeline) {
if (!step) throw new Error("runtime GitOps guard injection failed: gitops-promote render step missing");
const originalScript = String(step.script);
let script = patchNoBuildNoRolloutSkip(originalScript);
const noBuildSkipPatched = script !== originalScript;
let script = patchNoBuildNoRolloutCondition(originalScript);
const noBuildConditionPatched = script !== originalScript;
const beforeSkipPatch = script;
script = patchNoBuildNoRolloutSkip(script);
const noBuildSkipPatched = script !== beforeSkipPatch;
const beforeInjection = script;
script = injectRuntimeGitopsCommands(script);
if (hasWillRunGitopsPromoteCoupling(script)) {
throw new Error("runtime GitOps guard injection failed: no-build-no-rollout marker remains coupled to willRunGitopsPromote");
}
if (hasNoBuildNoRolloutEarlyExit(script)) {
throw new Error("runtime GitOps guard injection failed: no-build-no-rollout-plan early exit remains after patch");
}
@@ -69,12 +76,20 @@ function injectRuntimeGitopsGuard(pipeline) {
present: true,
configMapName,
stepName: typeof step.name === "string" ? step.name : null,
noBuildConditionPatched,
noBuildSkipPatched,
postprocessInjected: !hasPostprocess(beforeInjection) && postprocessPresent,
verifyInjected: !hasVerify(beforeInjection) && verifyPresent,
};
}
function patchNoBuildNoRolloutCondition(script) {
return String(script).replace(
/const willRunGitopsPromote = plan\.ciCdPlan && plan\.ciCdPlan\.willRunGitopsPromote === true;\nprocess\.exit\(!willRunGitopsPromote && buildServices\.length === 0 && affectedServices\.length === 0 \? 0 : 1\);/u,
"process.exit(buildServices.length === 0 && affectedServices.length === 0 ? 0 : 1);",
);
}
function patchNoBuildNoRolloutSkip(script) {
return String(script).replace(
/(^|\n)([ \t]*)echo '\{"event":"gitops-promote","status":"skipped","reason":"no-build-no-rollout-plan"\}'(?:\s*>\s*&2)?\n[ \t]*exit 0(?=\n|$)/u,
@@ -82,19 +97,21 @@ function patchNoBuildNoRolloutSkip(script) {
);
}
function hasWillRunGitopsPromoteCoupling(script) {
return /process\.exit\(!willRunGitopsPromote && buildServices\.length === 0 && affectedServices\.length === 0 \? 0 : 1\);/u.test(String(script));
}
function hasNoBuildNoRolloutEarlyExit(script) {
return /echo '\{"event":"gitops-promote","status":"skipped","reason":"no-build-no-rollout-plan"\}'(?:\s*>\s*&2)?\n[ \t]*exit 0(?=\n|$)/u.test(String(script));
}
function injectRuntimeGitopsCommands(script) {
if (hasPostprocess(script) && hasVerify(script)) return script;
const overlayEnv = `UNIDESK_RUNTIME_GITOPS_OVERLAY_B64=${shellSingle(Buffer.from(JSON.stringify({
runtimePath: overlay.runtimePath,
observability: overlay.observability,
}), "utf8").toString("base64"))}`;
const overlayEnv = "UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE=/etc/unidesk-cicd-runtime-gitops/runtime-gitops-overlay.json";
const migratedScript = String(script).replace(/UNIDESK_RUNTIME_GITOPS_OVERLAY_B64='[^'\n]*'/gu, overlayEnv);
if (hasPostprocess(migratedScript) && hasVerify(migratedScript)) return migratedScript;
const postprocess = `${overlayEnv} node /etc/unidesk-cicd-runtime-gitops/runtime-gitops-postprocess.mjs`;
const verify = `${overlayEnv} node /etc/unidesk-cicd-runtime-gitops/runtime-gitops-verify.mjs`;
return String(script).replace(
return migratedScript.replace(
/(node scripts\/run-bun\.mjs scripts\/gitops-render\.mjs[^\n]*--use-deploy-images[^\n]*)/g,
(match) => {
if (match.includes("--check")) return hasVerify(script) ? match : verify;
@@ -127,26 +144,9 @@ function ensureRuntimeGitopsScriptsMount(taskSpec, step) {
function writeScriptsConfigMap(targetPath, namespace) {
if (scriptsDir === null) throw new Error("--scripts-dir is required with --scripts-configmap");
const data = {};
for (const name of ["runtime-gitops-observability.mjs", "runtime-gitops-postprocess.mjs", "runtime-gitops-verify.mjs"]) {
data[name] = readFileSync(path.join(scriptsDir, name), "utf8");
}
writeFileSync(targetPath, `${YAML.stringify({
apiVersion: "v1",
kind: "ConfigMap",
metadata: {
name: configMapName,
namespace,
labels: {
"app.kubernetes.io/name": "hwlab-runtime-gitops-scripts",
"app.kubernetes.io/part-of": "unidesk-hwlab-control-plane",
"hwlab.pikastech.local/node": overlay.nodeId ?? null,
"hwlab.pikastech.local/lane": overlay.lane ?? null,
},
},
data,
}).trimEnd()}\n`, "utf8");
return { name: configMapName, namespace, keyCount: Object.keys(data).length, path: targetPath };
const configMap = runtimeGitopsScriptsConfigMap({ overlay, scriptsDir, namespace, configMapName });
writeFileSync(targetPath, `${YAML.stringify(configMap).trimEnd()}\n`, "utf8");
return { name: configMapName, namespace, keyCount: Object.keys(configMap.data).length, path: targetPath };
}
function pipelineNamespace(docs) {
@@ -193,10 +193,6 @@ function objectOr(value) {
return value && typeof value === "object" && !Array.isArray(value) ? value : {};
}
function shellSingle(value) {
return `'${String(value).replaceAll("'", "'\\''")}'`;
}
function requireYaml() {
try {
return requireFromScript("yaml");
scripts/native/hwlab/runtime-gitops-postprocess.mjs
+115 -7
View File
@@ -2,8 +2,12 @@
// SPEC: PJ2026-01060703 CI/CD branch follower runtime GitOps postprocess.
// Responsibility: mutate rendered runtime GitOps files after HWLAB source render, before publish.
import { existsSync, readdirSync, readFileSync, statSync, unlinkSync, writeFileSync } from "node:fs";
import { createRequire } from "node:module";
import path from "node:path";
const requireFromScript = createRequire(import.meta.url);
const requireFromCwd = createRequire(path.join(process.cwd(), "package.json"));
const YAML = requireYaml();
const repoDir = process.cwd();
const overlay = readOverlay();
const runtimePath = requiredOverlayString("runtimePath");
@@ -20,26 +24,106 @@ emit({ ok: true, runtimePath, ...result });
function postprocessRuntimeGitops() {
const prometheusOperatorDisabled = overlay?.observability?.prometheusOperator === false;
if (!prometheusOperatorDisabled) {
return { observabilityPrometheusOperator: overlay?.observability?.prometheusOperator ?? null, observabilityWorkloadsChanged: false, kustomizationChanged: false };
}
const changedFiles = [];
const deletedFiles = [];
for (const file of listYamlFiles(runtimeDir)) {
const changed = stripPrometheusOperatorResourcesFromFile(file);
const changed = prometheusOperatorDisabled ? stripPrometheusOperatorResourcesFromFile(file) : "unchanged";
if (changed === "deleted") deletedFiles.push(path.relative(repoDir, file));
if (changed === "changed") changedFiles.push(path.relative(repoDir, file));
}
const kustomizationChanged = pruneMissingKustomizationResources();
const kustomizationChanged = prometheusOperatorDisabled ? pruneMissingKustomizationResources() : false;
const codeAgentRuntimeChanged = patchCodeAgentRuntimeWorkloads();
return {
observabilityPrometheusOperator: false,
observabilityPrometheusOperator: overlay?.observability?.prometheusOperator ?? null,
observabilityWorkloadsChanged: changedFiles.length > 0 || deletedFiles.length > 0,
kustomizationChanged,
codeAgentRuntimeChanged,
changedFiles,
deletedFiles,
};
}
function patchCodeAgentRuntimeWorkloads() {
const runtime = overlay?.codeAgentRuntime;
if (!runtime?.enabled || !runtime.kafkaEventBridge) return false;
const kafka = runtime.kafkaEventBridge;
let changed = false;
for (const file of listYamlFiles(runtimeDir)) {
const documents = parseStructuredDocuments(readFileSync(file, "utf8"), file);
let fileChanged = false;
for (const document of documents.values) {
const items = isKubernetesList(document) ? document.items : [document];
for (const item of items) {
const workloadName = String(item?.metadata?.labels?.["app.kubernetes.io/name"] ?? item?.metadata?.name ?? "");
const containers = item?.spec?.template?.spec?.containers;
if (!Array.isArray(containers)) continue;
for (const container of containers) {
if (!Array.isArray(container?.env)) continue;
if (workloadName === "hwlab-cloud-api" && container.name === "hwlab-cloud-api") {
fileChanged = patchCloudApiKafkaEnv(container, kafka) || fileChanged;
}
}
}
}
if (fileChanged) {
writeFileSync(file, serializeStructuredDocuments(documents), "utf8");
changed = true;
}
}
return changed;
}
function patchCloudApiKafkaEnv(container, kafka) {
let changed = false;
changed = setEnvValue(container, "HWLAB_KAFKA_ENABLED", String(kafka.enabled)) || changed;
changed = setEnvValue(container, "HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED", String(kafka.enabled)) || changed;
changed = setEnvValue(container, "HWLAB_KAFKA_PROJECTOR_GROUP_ID", String(kafka.transactionalProjectorConsumerGroupId)) || changed;
changed = setEnvValue(container, "HWLAB_KAFKA_HWLAB_EVENT_GROUP_ID", String(kafka.hwlabEventConsumerGroupId)) || changed;
changed = patchOptionalEnvGroup(container, kafka.enabled, {
HWLAB_KAFKA_PROJECTOR_HEARTBEAT_INTERVAL_MS: kafka.transactionalProjector?.heartbeatIntervalMs,
}) || changed;
changed = patchOptionalEnvGroup(container, kafka.enabled, {
HWLAB_KAFKA_OUTBOX_RELAY_INTERVAL_MS: kafka.projectionOutboxRelay?.intervalMs,
HWLAB_KAFKA_OUTBOX_RELAY_BATCH_SIZE: kafka.projectionOutboxRelay?.batchSize,
HWLAB_KAFKA_OUTBOX_RELAY_LEASE_MS: kafka.projectionOutboxRelay?.leaseMs,
HWLAB_KAFKA_OUTBOX_RELAY_SEND_TIMEOUT_MS: kafka.projectionOutboxRelay?.sendTimeoutMs,
HWLAB_KAFKA_OUTBOX_RELAY_RETRY_BACKOFF_MS: kafka.projectionOutboxRelay?.retryBackoffMs,
}) || changed;
changed = patchOptionalEnvGroup(container, kafka.enabled, {
HWLAB_WORKBENCH_OUTBOX_TAIL_BATCH_SIZE: kafka.projectionRealtime?.outboxTailBatchSize,
HWLAB_WORKBENCH_SSE_HEARTBEAT_MS: kafka.projectionRealtime?.sseHeartbeatMs,
HWLAB_WORKBENCH_SSE_DRAIN_TIMEOUT_MS: kafka.projectionRealtime?.sseDrainTimeoutMs,
}) || changed;
return changed;
}
function patchOptionalEnvGroup(container, enabled, values) {
let changed = false;
for (const [name, value] of Object.entries(values)) {
changed = enabled ? setEnvValue(container, name, String(value)) || changed : removeEnvValue(container, name) || changed;
}
return changed;
}
function setEnvValue(container, name, value) {
const existing = container.env.find((item) => item?.name === name);
if (existing?.value === value && existing.valueFrom === undefined) return false;
if (existing) {
existing.value = value;
delete existing.valueFrom;
} else {
container.env.push({ name, value });
}
return true;
}
function removeEnvValue(container, name) {
const next = container.env.filter((item) => item?.name !== name);
if (next.length === container.env.length) return false;
container.env = next;
return true;
}
function stripPrometheusOperatorResourcesFromFile(file) {
const original = readFileSync(file, "utf8");
const jsonResult = stripPrometheusOperatorResourcesFromJsonFile(file, original);
@@ -146,6 +230,20 @@ function parseJsonDocument(text) {
}
}
function parseStructuredDocuments(text, file) {
const json = parseJsonDocument(text);
if (json !== null) return { format: "json", values: [json] };
const documents = YAML.parseAllDocuments(text);
const errors = documents.flatMap((document) => document.errors);
if (errors.length > 0) throw new Error(`invalid YAML in ${file}: ${errors.map((error) => error.message).join("; ")}`);
return { format: "yaml", values: documents.map((document) => document.toJS()).filter((document) => document !== null) };
}
function serializeStructuredDocuments(documents) {
if (documents.format === "json") return `${JSON.stringify(documents.values[0], null, 2)}\n`;
return `${documents.values.map((document) => YAML.stringify(document).trimEnd()).join("\n---\n")}\n`;
}
function leadingSpaces(value) {
const match = value.match(/^ */u);
return match ? match[0].length : 0;
@@ -166,11 +264,21 @@ function listYamlFiles(root) {
}
function readOverlay() {
const file = process.env.UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE;
if (file) return JSON.parse(readFileSync(file, "utf8"));
const encoded = process.env.UNIDESK_RUNTIME_GITOPS_OVERLAY_B64;
if (!encoded) throw new Error("UNIDESK_RUNTIME_GITOPS_OVERLAY_B64 is required");
if (!encoded) throw new Error("UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE or UNIDESK_RUNTIME_GITOPS_OVERLAY_B64 is required");
return JSON.parse(Buffer.from(encoded, "base64").toString("utf8"));
}
function requireYaml() {
try {
return requireFromScript("yaml");
} catch {
return requireFromCwd("yaml");
}
}
function requiredOverlayString(name) {
const value = overlay[name];
if (typeof value !== "string" || value.length === 0) throw new Error(`overlay.${name} is required`);
scripts/native/hwlab/runtime-gitops-postprocess.test.ts
+110
View File
@@ -0,0 +1,110 @@
import { afterEach, expect, test } from "bun:test";
import { spawnSync } from "node:child_process";
import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
const roots: string[] = [];
const script = resolve(import.meta.dir, "runtime-gitops-postprocess.mjs");
afterEach(() => {
while (roots.length > 0) rmSync(roots.pop()!, { recursive: true, force: true });
});
test("patches fixed Kafka transport settings without architecture capability env", () => {
const root = mkdtempSync(join(tmpdir(), "runtime-gitops-postprocess-"));
roots.push(root);
const runtimeDir = join(root, "runtime");
mkdirSync(runtimeDir);
writeFileSync(join(root, "overlay.json"), JSON.stringify(overlay("runtime")));
writeFileSync(join(runtimeDir, "cloud-api.yaml"), deploymentYaml("hwlab-cloud-api"));
writeFileSync(join(runtimeDir, "cloud-web.yaml"), `apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: unchanged\n---\n${deploymentYaml("hwlab-cloud-web")}`);
writeFileSync(join(runtimeDir, "list.yaml"), `apiVersion: v1\nkind: List\nitems:\n${listItem(deploymentYaml("hwlab-cloud-api"))}\n`);
writeFileSync(join(runtimeDir, "json-workload.yaml"), `${JSON.stringify(Bun.YAML.parse(deploymentYaml("hwlab-cloud-api")), null, 2)}\n`);
const result = spawnSync(process.execPath, [script], {
cwd: root,
env: { ...process.env, UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE: join(root, "overlay.json") },
encoding: "utf8",
});
if (result.status !== 0) throw new Error(result.stderr);
expect(result.stderr).toContain('"codeAgentRuntimeChanged":true');
const api = Bun.YAML.parse(readFileSync(join(runtimeDir, "cloud-api.yaml"), "utf8")) as any;
expect(envValues(api)).toMatchObject({
HWLAB_KAFKA_ENABLED: "true",
HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED: "true",
HWLAB_KAFKA_PROJECTOR_GROUP_ID: "hwlab-projector",
HWLAB_KAFKA_HWLAB_EVENT_GROUP_ID: "hwlab-events",
});
expect(architectureEnvNames(api)).toEqual([]);
const multiDocs = readFileSync(join(runtimeDir, "cloud-web.yaml"), "utf8").split(/^---$/mu).map((document) => Bun.YAML.parse(document) as any);
expect(multiDocs[0].kind).toBe("ConfigMap");
expect(envValues(multiDocs[1])).toEqual({ EXISTING: "preserved" });
const list = Bun.YAML.parse(readFileSync(join(runtimeDir, "list.yaml"), "utf8")) as any;
expect(list.kind).toBe("List");
expect(architectureEnvNames(list.items[0])).toEqual([]);
const jsonText = readFileSync(join(runtimeDir, "json-workload.yaml"), "utf8");
expect(jsonText.startsWith("{\n")).toBe(true);
expect(architectureEnvNames(JSON.parse(jsonText))).toEqual([]);
const afterFirstRun = snapshot(runtimeDir);
const second = spawnSync(process.execPath, [script], {
cwd: root,
env: { ...process.env, UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE: join(root, "overlay.json") },
encoding: "utf8",
});
if (second.status !== 0) throw new Error(second.stderr);
expect(second.stderr).toContain('"codeAgentRuntimeChanged":false');
expect(snapshot(runtimeDir)).toEqual(afterFirstRun);
});
function overlay(runtimePath: string) {
return {
runtimePath,
codeAgentRuntime: {
enabled: true,
kafkaEventBridge: {
enabled: true,
transactionalProjectorConsumerGroupId: "hwlab-projector",
hwlabEventConsumerGroupId: "hwlab-events",
transactionalProjector: { heartbeatIntervalMs: 2000 },
projectionOutboxRelay: { intervalMs: 250, batchSize: 100, leaseMs: 30000, sendTimeoutMs: 10000, retryBackoffMs: 1000 },
projectionRealtime: { outboxTailBatchSize: 100, sseHeartbeatMs: 1000, sseDrainTimeoutMs: 5000 },
},
},
};
}
function deploymentYaml(name: string) {
return `apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: ${name}\n labels:\n app.kubernetes.io/name: ${name}\nspec:\n template:\n spec:\n containers:\n - name: ${name}\n env:\n - name: EXISTING\n value: preserved\n`;
}
function listItem(value: string) {
const [first, ...rest] = value.trimEnd().split("\n");
return [` - ${first}`, ...rest.map((line) => ` ${line}`)].join("\n");
}
function envValues(workload: any) {
return Object.fromEntries(workload.spec.template.spec.containers[0].env.map((item: any) => [item.name, item.value]));
}
function architectureEnvNames(workload: any) {
const forbidden = new Set([
"HWLAB_KAFKA_DIRECT_PUBLISH_ENABLED",
"HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED",
"HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED",
"HWLAB_KAFKA_TRANSACTIONAL_PROJECTOR_ENABLED",
"HWLAB_KAFKA_PROJECTION_OUTBOX_RELAY_ENABLED",
"HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED",
]);
return workload.spec.template.spec.containers[0].env.map((item: any) => item.name).filter((name: string) => forbidden.has(name));
}
function snapshot(runtimeDir: string) {
return ["cloud-api.yaml", "cloud-web.yaml", "list.yaml", "json-workload.yaml"].map((name) => readFileSync(join(runtimeDir, name), "utf8"));
}
scripts/native/hwlab/runtime-gitops-scripts-configmap.mjs
+56
View File
@@ -0,0 +1,56 @@
import { readFileSync } from "node:fs";
import path from "node:path";
export function runtimeGitopsScriptsConfigMap({ overlay, scriptsDir, featureConfigValidatorPath = null, ajvBundlePath = null, namespace, configMapName }) {
const data = {
"runtime-gitops-overlay.json": `${JSON.stringify({
runtimePath: requiredString(overlay.runtimePath, "overlay.runtimePath"),
observability: objectOr(overlay.observability),
codeAgentRuntime: objectOr(overlay.codeAgentRuntime),
})}\n`,
};
for (const name of [
"runtime-gitops-observability.mjs",
"runtime-gitops-postprocess.mjs",
"runtime-gitops-verify.mjs",
]) {
data[name] = readFileSync(path.join(scriptsDir, name), "utf8");
}
const featureConfigValidator = optionalFile(featureConfigValidatorPath ?? path.join(scriptsDir, "feature-config-schema-warning.mjs"));
if (featureConfigValidator !== null) data["feature-config-schema-warning.mjs"] = featureConfigValidator;
const ajvBundle = optionalFile(ajvBundlePath ?? path.join(scriptsDir, "ajv2020.min.js"));
if (ajvBundle !== null) data["ajv2020.min.js"] = ajvBundle;
return {
apiVersion: "v1",
kind: "ConfigMap",
metadata: {
name: configMapName,
namespace,
labels: {
"app.kubernetes.io/name": "hwlab-runtime-gitops-scripts",
"app.kubernetes.io/part-of": "unidesk-hwlab-control-plane",
"app.kubernetes.io/managed-by": "unidesk-host-gitops",
"hwlab.pikastech.local/node": overlay.nodeId ?? null,
"hwlab.pikastech.local/lane": overlay.lane ?? null,
},
},
data,
};
}
function requiredString(value, pathLabel) {
if (typeof value !== "string" || value.length === 0) throw new Error(`${pathLabel} must be a non-empty string`);
return value;
}
function objectOr(value) {
return value && typeof value === "object" && !Array.isArray(value) ? value : {};
}
function optionalFile(file) {
try {
return readFileSync(file, "utf8");
} catch {
return null;
}
}
scripts/native/hwlab/runtime-gitops-scripts-configmap.test.ts
+72
View File
@@ -0,0 +1,72 @@
import { afterEach, expect, test } from "bun:test";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { runtimeGitopsScriptsConfigMap } from "./runtime-gitops-scripts-configmap.mjs";
const roots: string[] = [];
afterEach(() => {
while (roots.length > 0) rmSync(roots.pop()!, { recursive: true, force: true });
});
test("preserves codeAgentRuntime in the owning runtime GitOps overlay", () => {
const scriptsDir = mkdtempSync(join(tmpdir(), "runtime-gitops-configmap-"));
roots.push(scriptsDir);
for (const name of [
"runtime-gitops-observability.mjs",
"runtime-gitops-postprocess.mjs",
"runtime-gitops-verify.mjs",
"feature-config-schema-warning.mjs",
"ajv2020.min.js",
]) {
writeFileSync(join(scriptsDir, name), `${name}\n`);
}
const codeAgentRuntime = {
enabled: true,
kafkaEventBridge: {
enabled: true,
transactionalProjector: { heartbeatIntervalMs: 2000 },
projectionOutboxRelay: { intervalMs: 250 },
projectionRealtime: { sseHeartbeatMs: 1000 },
},
};
const configMap = runtimeGitopsScriptsConfigMap({
overlay: { runtimePath: "runtime", observability: { prometheusOperator: false }, codeAgentRuntime },
scriptsDir,
namespace: "hwlab",
configMapName: "runtime-gitops-scripts",
});
expect(JSON.parse(configMap.data["runtime-gitops-overlay.json"])).toEqual({
runtimePath: "runtime",
observability: { prometheusOperator: false },
codeAgentRuntime,
});
expect(configMap.data["feature-config-schema-warning.mjs"]).toBe("feature-config-schema-warning.mjs\n");
expect(configMap.data["ajv2020.min.js"]).toBe("ajv2020.min.js\n");
});
test("missing optional validator artifacts do not block ConfigMap rendering", () => {
const scriptsDir = mkdtempSync(join(tmpdir(), "runtime-gitops-configmap-"));
roots.push(scriptsDir);
for (const name of [
"runtime-gitops-observability.mjs",
"runtime-gitops-postprocess.mjs",
"runtime-gitops-verify.mjs",
]) {
writeFileSync(join(scriptsDir, name), `${name}\n`);
}
const configMap = runtimeGitopsScriptsConfigMap({
overlay: { runtimePath: "runtime" },
scriptsDir,
namespace: "hwlab",
configMapName: "runtime-gitops-scripts",
});
expect(configMap.data["runtime-gitops-verify.mjs"]).toBe("runtime-gitops-verify.mjs\n");
expect(configMap.data["feature-config-schema-warning.mjs"]).toBeUndefined();
expect(configMap.data["ajv2020.min.js"]).toBeUndefined();
});
scripts/native/hwlab/runtime-gitops-verify.mjs
+27 -1
View File
@@ -2,8 +2,12 @@
// SPEC: PJ2026-01060703 CI/CD branch follower runtime GitOps verify.
// Responsibility: fail publish before Argo when rendered runtime GitOps violates UniDesk overlay gates.
import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";
import { createRequire } from "node:module";
import path from "node:path";
const requireFromScript = createRequire(import.meta.url);
const requireFromCwd = createRequire(path.join(process.cwd(), "package.json"));
const YAML = requireYaml();
const repoDir = process.cwd();
const overlay = readOverlay();
const runtimePath = requiredOverlayString("runtimePath");
@@ -20,6 +24,9 @@ if (overlay?.observability?.prometheusOperator === false) {
const refs = findPrometheusOperatorResources();
if (refs.length > 0) fail("prometheus-operator-resource-present", { runtimePath, refs: refs.slice(0, 12), refCount: refs.length });
}
if (overlay?.codeAgentRuntime?.enabled && overlay.codeAgentRuntime.kafkaEventBridge?.enabled) {
checks.push("code-agent-runtime-kafka-event-bridge-enabled");
}
console.error(JSON.stringify({ event: "unidesk-runtime-gitops-verify", ok: true, runtimePath, checks }));
@@ -87,6 +94,15 @@ function parseJsonDocument(text) {
}
}
function parseStructuredDocuments(text, file) {
const json = parseJsonDocument(text);
if (json !== null) return [json];
const documents = YAML.parseAllDocuments(text);
const errors = documents.flatMap((document) => document.errors);
if (errors.length > 0) fail("runtime-yaml-invalid", { file: path.relative(repoDir, file), errors: errors.map((error) => error.message) });
return documents.map((document) => document.toJS()).filter((document) => document !== null);
}
function listYamlFiles(root) {
const out = [];
for (const name of readdirSync(root)) {
@@ -102,11 +118,21 @@ function listYamlFiles(root) {
}
function readOverlay() {
const file = process.env.UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE;
if (file) return JSON.parse(readFileSync(file, "utf8"));
const encoded = process.env.UNIDESK_RUNTIME_GITOPS_OVERLAY_B64;
if (!encoded) throw new Error("UNIDESK_RUNTIME_GITOPS_OVERLAY_B64 is required");
if (!encoded) throw new Error("UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE or UNIDESK_RUNTIME_GITOPS_OVERLAY_B64 is required");
return JSON.parse(Buffer.from(encoded, "base64").toString("utf8"));
}
function requireYaml() {
try {
return requireFromScript("yaml");
} catch {
return requireFromCwd("yaml");
}
}
function requiredOverlayString(name) {
const value = overlay[name];
if (typeof value !== "string" || value.length === 0) throw new Error(`overlay.${name} is required`);
scripts/native/hwlab/runtime-gitops-verify.test.ts
+38
View File
@@ -0,0 +1,38 @@
import { afterEach, expect, test } from "bun:test";
import { spawnSync } from "node:child_process";
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
const roots: string[] = [];
const script = resolve(import.meta.dir, "runtime-gitops-verify.mjs");
afterEach(() => {
while (roots.length > 0) rmSync(roots.pop()!, { recursive: true, force: true });
});
test("architecture capability env and workload matches are not runtime GitOps gates", () => {
const root = mkdtempSync(join(tmpdir(), "runtime-gitops-verify-"));
roots.push(root);
mkdirSync(join(root, "runtime"));
writeFileSync(join(root, "runtime", "unrelated.yaml"), "apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: unrelated\n");
writeFileSync(join(root, "overlay.json"), JSON.stringify({
runtimePath: "runtime",
codeAgentRuntime: {
enabled: true,
kafkaEventBridge: { enabled: true },
},
}));
const result = spawnSync(process.execPath, [script], {
cwd: root,
env: { ...process.env, UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE: join(root, "overlay.json") },
encoding: "utf8",
});
expect(result.status).toBe(0);
expect(result.stderr).toContain('"code-agent-runtime-kafka-event-bridge-enabled"');
expect(result.stderr).not.toContain("capability-env-invalid");
expect(result.stderr).not.toContain("workload-missing");
expect(result.stderr).not.toContain("authority-invalid");
});
scripts/src/agentrun-manifests.ts
+16
View File
@@ -2,6 +2,8 @@
// Renders AgentRun YAML lane policy into runtime manager environment.
import { createHash } from "node:crypto";
import type { AgentRunLaneSpec } from "./agentrun-lanes";
import { gitRepositoryIdentity, resolveCicdDeliveryAuthority } from "./cicd-delivery-authority";
import { renderPacFeatureConfigSchemaStage } from "./pac-feature-config-schema-stage";
import { stableJsonSha256 } from "./stable-json";
export interface AgentRunArtifactService {
@@ -408,6 +410,19 @@ function agentRunTektonGitopsPublishScript(spec: AgentRunLaneSpec): string {
const templateImage = agentRunImageArtifact(spec, { sourceCommit: sourcePlaceholder, envIdentity: envPlaceholder, digest: digestPlaceholder, status: statusPlaceholder });
const templateFiles = renderAgentRunGitopsFiles(spec, { sourceCommit: sourcePlaceholder, image: templateImage });
const templateB64 = Buffer.from(JSON.stringify(templateFiles), "utf8").toString("base64");
const authority = resolveCicdDeliveryAuthority({
node: spec.nodeId,
lane: spec.lane,
sourceRepository: gitRepositoryIdentity(spec.source.repository),
sourceBranch: spec.source.branch,
});
if (authority.kind !== "pac-pr-merge") throw new Error(`AgentRun ${spec.nodeId}/${spec.lane} requires exact PaC consumer authority for feature config validation`);
const featureConfigSchemaStage = renderPacFeatureConfigSchemaStage({
repositoryId: authority.consumer.repositoryRef,
consumer: authority.consumer.consumerId,
repoDirExpression: '"$root"',
renderedRootExpression: '"$PWD/$(params.gitops-root)"',
});
return [
"#!/bin/sh",
"set -eu",
@@ -446,6 +461,7 @@ function agentRunTektonGitopsPublishScript(spec: AgentRunLaneSpec): string {
" writeFileSync(file.path, content);",
"}",
"NODE",
featureConfigSchemaStage,
"git add source.json \"$(params.artifact-catalog)\" \"$(params.gitops-root)\"",
"if git diff --quiet --cached; then changed=false; else changed=true; git -c user.email=agentrun@unidesk.local -c user.name='UniDesk AgentRun PaC' commit -m \"deploy: render AgentRun $(params.gitops-branch) from PaC\"; fi",
"git remote set-url origin \"$git_auth_url\"",
scripts/src/agentrun/rest-bridge.ts
-1
View File
@@ -667,7 +667,6 @@ function targetTaskPreflightFromArgs(args: string[], aipod: string): AgentRunTar
mutation: false,
recoveryActions: [
"修复错误后用同一条 create task 命令重新执行；失败预检不会创建 task。",
"先保留 --dry-run 查看 Target、source、workspaceRef、resourceBundleRef、session 与 SecretRef 摘要。",
],
valuesPrinted: false,
},
scripts/src/gh-token-repository-override.test.ts
+16
View File
@@ -84,6 +84,22 @@ describe("GitHub repository token overrides", () => {
expect(selected.probe).toMatchObject({ source: "yaml-token-source", scope: "repository-override", yamlSourcePriority: "before-env" });
});
test("matches repository overrides without case sensitivity", () => {
delete process.env.GH_TOKEN;
delete process.env.GITHUB_TOKEN;
const authConfig = testAuthConfig();
for (const repo of ["PikaInc/selfmedia", "pikainc/SELFMEDIA", "PIKAINC/SELFMEDIA"]) {
const selected = resolveToken(repo, false, authConfig);
expect(selected.token).toBe("selfmedia-token");
expect(selected.probe).toMatchObject({ scope: "repository-override", sourceRef: "selfmedia-token.txt", valuesPrinted: false });
}
const global = resolveToken("PikasTech/UniDesk", false, authConfig);
expect(global.token).toBe("global-token");
expect(global.probe).toMatchObject({ scope: "global", sourceRef: "global-token.txt", valuesPrinted: false });
});
test("uses the final positional repository for issue and PR token selection", () => {
delete process.env.GH_TOKEN;
delete process.env.GITHUB_TOKEN;
scripts/src/gh/auth-and-safety.ts
+11 -3
View File
@@ -31,6 +31,10 @@ export interface GitHubYamlAuthConfig {
repositoryOverrides: GitHubYamlTokenSourceConfig[];
}
function repositoryLookupKey(repository: string): string {
return repository.toLowerCase();
}
export function readIssueLifecycleCommentBody(options: GitHubOptions, command: string): { body: string; bodySource: Record<string, unknown> } | null {
if (options.comment === undefined && options.commentFile === undefined) return null;
if (options.comment !== undefined) {
@@ -59,7 +63,10 @@ export function tokenFromEnvironment(): GitHubTokenProbe {
}
export function tokenFromYamlSource(repo: string, authConfig: GitHubYamlAuthConfig = readGitHubYamlAuthConfig()): { token: string | null; probe: GitHubTokenProbe } {
const override = authConfig.repositoryOverrides.find((candidate) => candidate.repository === repo) ?? null;
const repoLookupKey = repositoryLookupKey(repo);
const override = authConfig.repositoryOverrides.find((candidate) => (
candidate.repository !== undefined && repositoryLookupKey(candidate.repository) === repoLookupKey
)) ?? null;
const config = override ?? authConfig.tokenSource;
if (config === null || config.enabled === false) {
return {
@@ -162,8 +169,9 @@ function readGitHubYamlAuthConfig(): GitHubYamlAuthConfig {
const configRef = `${GITHUB_REPOSITORY_OVERRIDES_CONFIG_REF}[${index}]`;
const override = asRecord(value, configRef);
const repository = repositoryField(override, "repository", configRef);
if (seenRepositories.has(repository)) throw new Error(`${GITHUB_REPOSITORY_OVERRIDES_CONFIG_REF} contains duplicate repository ${repository}`);
seenRepositories.add(repository);
const repositoryKey = repositoryLookupKey(repository);
if (seenRepositories.has(repositoryKey)) throw new Error(`${GITHUB_REPOSITORY_OVERRIDES_CONFIG_REF} contains duplicate repository ${repository}`);
seenRepositories.add(repositoryKey);
return {
repository,
priority: tokenSourcePriorityField(override, "priority", configRef),
scripts/src/gh/help.ts
+2 -2
View File
@@ -100,14 +100,14 @@ export function ghHelp(): unknown {
"Commander brief ClaudeQQ defaults to private target 645275593 through backend-core /api/microservices/claudeqq/proxy; UNIDESK_COMMANDER_BRIEF_CLAUDEQQ_* env vars can override target, base URL, timeout, and enabled state.",
"comment update/edit PATCHes /repos/{owner}/{repo}/issues/comments/{comment_id} and preserves the comment id/timeline; comment delete is supported because GitHub supports deleting issue comments, but routine wording fixes should use update/edit. issue/pr hard delete is unsupported and close is the lifecycle alternative.",
"PR files is the canonical compact changed-file/stat summary. It uses GitHub REST, returns bounded file rows, additions/deletions/changes when available, truncation metadata, and a next command for full details. Raw diff patches are not emitted by default; gh pr diff <number> --stat is a compatibility alias for the same JSON summary.",
"PR review-plan is the review-first bounded patch index. It uses GitHub REST PR files, prints changed files with additions/deletions/hunk counts/patch-line counts/default truncation flags, and emits one per-file gh pr diff --file drill-down command without creating a local worktree.",
"PR review-plan is the review-first bounded patch index. It uses GitHub REST PR files, prints changed files with additions/deletions/hunk counts/patch-line counts/default truncation flags, and emits per-file gh pr diff --file drill-down commands without creating a local worktree. Use only the drill-downs needed for review; do not mechanically expand every file.",
"PR diff --file reads one changed file patch from GitHub REST and prints only a bounded excerpt by default. Add --hunk N to inspect one hunk, --limit N to change displayed patch lines, or --full/--raw for explicit structured full-patch disclosure. --stat remains the no-patch file/stat compatibility path.",
"PR edit/update PATCHes /repos/{owner}/{repo}/pulls/{number} through REST only, never GitHub Projects Classic GraphQL/projectCards, and returns low-noise JSON with repo, PR number, changedFields, url, and body size/SHA metadata instead of echoing the full body.",
"PR view is the canonical bounded metadata summary; read remains a UniDesk compatibility alias. Human full-body reading uses `trans gh:/owner/repo/pr/<number> cat`, and targeted lookup uses the same route with `rg <pattern>`. `--json body`, `--full`, and `--raw` are explicit structured machine disclosure only. View/read retain positional numbers, GitHub URLs, owner/repo#number shorthand, compatible --number, REST closeout fields, and on-demand GraphQL closeout metadata.",
"PR preflight/closeout accept the same owner/repo#number shorthand as PR view/read so merge readiness checks do not require repeating --repo after a PR URL has already been normalized.",
"PR list does not fetch mergeability or statusCheckRollup; request those closeout fields with gh pr view <number> --json headRefName,baseRefName,mergeable,mergeStateStatus,statusCheckRollup.",
"PR preflight is a low-noise read-only closeout helper for explicit diagnosis only. It combines redacted auth capability, PR branch/state metadata, mergeability, mergeStateStatus, compact status check counts, and an explicit read-only policy. It is not a required step before gh pr merge. Use --full or --raw to include all fetched status contexts.",
"PR merge is the one-command guarded write path: it reads closeout metadata itself, retries GitHub UNKNOWN/null mergeability with YAML-configured exponential backoff, refuses non-open/draft/conflicting/non-clean/failed/pending PRs, then uses GitHub REST merge. It defaults to deleting the merged same-repo head branch, cleaning the matching local .worktree when clean, and fast-forwarding the local main worktree on the PR base branch; --sync-node NODE additionally runs mapped node source-workspace sync. Use --dry-run to see the exact merge and closeout plan without writing.",
"PR merge is the one-command guarded write path: it reads closeout metadata itself, retries GitHub UNKNOWN/null mergeability with YAML-configured exponential backoff, refuses non-open/draft/conflicting/non-clean/failed/pending PRs, then uses GitHub REST merge. Its success summary includes mergeCommit and mergedAt, so a routine follow-up pr view is unnecessary. It defaults to deleting the merged same-repo head branch, cleaning the matching local .worktree when clean, and fast-forwarding the local main worktree on the PR base branch; --sync-node NODE additionally runs mapped node source-workspace sync. Use --dry-run to see the exact merge and closeout plan without writing.",
],
};
}
scripts/src/gh/pr-merge.ts
+9 -2
View File
@@ -169,6 +169,12 @@ export function renderPrMergeTable(result: GitHubCommandResult, fallbackDetails?
: {};
const mergeMethod = result.method ?? result.mergeMethod ?? details.method;
const deleteBranch = result.deleteBranch ?? details.deleteBranch;
const mergeResult = isRecord(result.mergeResult) ? result.mergeResult : {};
const mergeCommitValue = pullRequest.mergeCommit;
const mergeCommit = isRecord(mergeCommitValue)
? mergeCommitValue.oid
: mergeCommitValue ?? mergeResult.sha;
const mergedAt = pullRequest.mergedAt;
const status = result.ok === true
? result.alreadyMerged === true
? "already-merged"
@@ -198,13 +204,14 @@ export function renderPrMergeTable(result: GitHubCommandResult, fallbackDetails?
` repo=${ghText(result.repo)} title=${ghShort(ghText(pullRequest.title), 96)}`,
` blockers=${blockers.length === 0 ? "-" : blockers.join(",")} pending=${pending.length === 0 ? "-" : pending.join(",")}`,
` retry=${retry.attempts !== undefined && retry.maxAttempts !== undefined ? `${ghText(retry.attempts)}/${ghText(retry.maxAttempts)} exhausted=${ghText(retry.exhausted)}` : "-"}`,
` mergeCommit=${ghText(mergeCommit)} mergedAt=${ghText(mergedAt)}`,
` branchDeletion=${ghText(branchDeletion.ok ?? branchDeletion.skippedReason ?? branchDeletion.attempted)}`,
` closeout localWorktree=${closeoutCell(localWorktree)} mainWorktree=${closeoutCell(mainWorktree)} nodeSyncs=${nodeSyncs.length === 0 ? "-" : nodeSyncs.map(closeoutCell).join(",")}`,
"",
"Next:",
];
if (result.ok === true && result.alreadyMerged !== true && result.dryRun !== true) {
lines.push(` bun scripts/cli.ts gh pr view ${ghText(result.number ?? details.number)} --repo ${ghText(result.repo)}`);
if (result.ok === true && result.dryRun !== true) {
lines.push(" no follow-up required; merge facts are included in Summary.");
} else {
lines.push(` ${prMergeRetryCommand(ghText(result.repo), result.number ?? details.number, mergeMethod, deleteBranch)}`);
lines.push(` bun scripts/cli.ts gh pr preflight ${ghText(result.number ?? details.number)} --repo ${ghText(result.repo)} --full`);
scripts/src/hwlab-node-lanes.ts
+73 -45
View File
@@ -545,20 +545,27 @@ export interface HwlabRuntimeKafkaShadowProducerSpec {
export interface HwlabRuntimeKafkaEventBridgeSpec {
readonly enabled: boolean;
readonly features: {
readonly directPublish: boolean;
readonly liveKafkaSse: boolean;
readonly kafkaRefreshReplay: boolean;
readonly transactionalProjector: boolean;
readonly projectionOutboxRelay: boolean;
readonly projectionRealtime: boolean;
readonly transactionalProjector?: {
readonly heartbeatIntervalMs: number;
};
readonly refreshReplay?: {
readonly groupIdPrefix: string;
readonly timeoutMs: number;
readonly scanLimit: number;
readonly matchedEventLimit: number;
readonly liveBufferLimit: number;
readonly projectionOutboxRelay?: {
readonly intervalMs: number;
readonly batchSize: number;
readonly leaseMs: number;
readonly sendTimeoutMs: number;
readonly retryBackoffMs: number;
};
readonly projectionRealtime?: {
readonly outboxTailBatchSize: number;
readonly sseHeartbeatMs: number;
readonly sseDrainTimeoutMs: number;
};
readonly healthThresholds: {
readonly consumerLagWarning: number;
readonly outboxBacklogWarning: number;
readonly retryCountWarning: number;
readonly failedInboxWarning: number;
readonly dlqWarning: number;
};
readonly configRef: string;
readonly bootstrapServers: string;
@@ -566,7 +573,6 @@ export interface HwlabRuntimeKafkaEventBridgeSpec {
readonly agentRunEventTopic: string;
readonly hwlabEventTopic: string;
readonly clientId: string;
readonly directPublishConsumerGroupId: string;
readonly transactionalProjectorConsumerGroupId: string;
readonly hwlabEventConsumerGroupId: string;
}
@@ -628,6 +634,7 @@ export interface HwlabRuntimePipelineProvenanceSpec {
readonly configRef: string;
readonly renderer: "hwlab-runtime-lane";
readonly mode: "remote-pipeline-annotation";
readonly maxInlineScriptBytes: number;
}
export interface HwlabRuntimeLaneSpec {
@@ -1060,6 +1067,7 @@ export function parseHwlabRuntimePipelineProvenance(
configRef,
renderer: enumStringField(raw, "renderer", path, ["hwlab-runtime-lane"]),
mode: enumStringField(raw, "mode", path, ["remote-pipeline-annotation"]),
maxInlineScriptBytes: boundedIntegerField(raw, "maxInlineScriptBytes", path, 1024, 131072),
};
}
@@ -1220,55 +1228,75 @@ function codeAgentKafkaShadowProducerConfig(value: unknown, path: string): Hwlab
function codeAgentKafkaEventBridgeConfig(value: unknown, path: string): HwlabRuntimeKafkaEventBridgeSpec {
const raw = asRecord(value, path);
const features = asRecord(raw.features, `${path}.features`);
const liveKafkaSse = booleanField(features, "liveKafkaSse", `${path}.features`);
const kafkaRefreshReplay = booleanField(features, "kafkaRefreshReplay", `${path}.features`);
const refreshReplay = raw.refreshReplay === undefined
if (raw.refreshReplay !== undefined) throw new Error(`${path}.refreshReplay was removed with the fixed projection realtime architecture`);
const enabled = booleanField(raw, "enabled", path);
const transactionalProjectorConfig = raw.transactionalProjector === undefined
? undefined
: codeAgentKafkaRefreshReplayConfig(raw.refreshReplay, `${path}.refreshReplay`);
if (kafkaRefreshReplay && refreshReplay === undefined) {
throw new Error(`${path}.refreshReplay is required when ${path}.features.kafkaRefreshReplay is true`);
}
if (kafkaRefreshReplay && !liveKafkaSse) {
throw new Error(`${path}.features.liveKafkaSse must be true when ${path}.features.kafkaRefreshReplay is true`);
}
const directPublishConsumerGroupId = stringField(raw, "directPublishConsumerGroupId", path);
: codeAgentKafkaTransactionalProjectorConfig(raw.transactionalProjector, `${path}.transactionalProjector`);
const projectionOutboxRelayConfig = raw.projectionOutboxRelay === undefined
? undefined
: codeAgentKafkaProjectionOutboxRelayConfig(raw.projectionOutboxRelay, `${path}.projectionOutboxRelay`);
const projectionRealtimeConfig = raw.projectionRealtime === undefined
? undefined
: codeAgentKafkaProjectionRealtimeConfig(raw.projectionRealtime, `${path}.projectionRealtime`);
if (transactionalProjectorConfig === undefined) throw new Error(`${path}.transactionalProjector is required by the fixed transactional projection architecture`);
if (projectionOutboxRelayConfig === undefined) throw new Error(`${path}.projectionOutboxRelay is required by the fixed transactional projection architecture`);
if (projectionRealtimeConfig === undefined) throw new Error(`${path}.projectionRealtime is required by the fixed transactional projection architecture`);
const transactionalProjectorConsumerGroupId = stringField(raw, "transactionalProjectorConsumerGroupId", path);
const hwlabEventConsumerGroupId = stringField(raw, "hwlabEventConsumerGroupId", path);
if (new Set([directPublishConsumerGroupId, transactionalProjectorConsumerGroupId, hwlabEventConsumerGroupId]).size !== 3) {
throw new Error(`${path} consumer group ids must be distinct so enabled capabilities receive independent event streams`);
if (transactionalProjectorConsumerGroupId === hwlabEventConsumerGroupId) {
throw new Error(`${path} projector and realtime consumer group ids must be distinct`);
}
return {
enabled: booleanField(raw, "enabled", path),
features: {
directPublish: booleanField(features, "directPublish", `${path}.features`),
liveKafkaSse,
kafkaRefreshReplay,
transactionalProjector: booleanField(features, "transactionalProjector", `${path}.features`),
projectionOutboxRelay: booleanField(features, "projectionOutboxRelay", `${path}.features`),
projectionRealtime: booleanField(features, "projectionRealtime", `${path}.features`),
},
...(refreshReplay === undefined ? {} : { refreshReplay }),
enabled,
transactionalProjector: transactionalProjectorConfig,
projectionOutboxRelay: projectionOutboxRelayConfig,
projectionRealtime: projectionRealtimeConfig,
healthThresholds: codeAgentKafkaHealthThresholdsConfig(raw.healthThresholds, `${path}.healthThresholds`),
configRef: stringField(raw, "configRef", path),
bootstrapServers: stringField(raw, "bootstrapServers", path),
stdioTopic: stringField(raw, "stdioTopic", path),
agentRunEventTopic: stringField(raw, "agentRunEventTopic", path),
hwlabEventTopic: stringField(raw, "hwlabEventTopic", path),
clientId: stringField(raw, "clientId", path),
directPublishConsumerGroupId,
transactionalProjectorConsumerGroupId,
hwlabEventConsumerGroupId,
};
}
function codeAgentKafkaRefreshReplayConfig(value: unknown, path: string): NonNullable<HwlabRuntimeKafkaEventBridgeSpec["refreshReplay"]> {
function codeAgentKafkaTransactionalProjectorConfig(value: unknown, path: string): NonNullable<HwlabRuntimeKafkaEventBridgeSpec["transactionalProjector"]> {
const raw = asRecord(value, path);
return { heartbeatIntervalMs: numberField(raw, "heartbeatIntervalMs", path) };
}
function codeAgentKafkaProjectionOutboxRelayConfig(value: unknown, path: string): NonNullable<HwlabRuntimeKafkaEventBridgeSpec["projectionOutboxRelay"]> {
const raw = asRecord(value, path);
return {
groupIdPrefix: stringField(raw, "groupIdPrefix", path),
timeoutMs: numberField(raw, "timeoutMs", path),
scanLimit: numberField(raw, "scanLimit", path),
matchedEventLimit: numberField(raw, "matchedEventLimit", path),
liveBufferLimit: numberField(raw, "liveBufferLimit", path),
intervalMs: numberField(raw, "intervalMs", path),
batchSize: numberField(raw, "batchSize", path),
leaseMs: numberField(raw, "leaseMs", path),
sendTimeoutMs: numberField(raw, "sendTimeoutMs", path),
retryBackoffMs: numberField(raw, "retryBackoffMs", path),
};
}
function codeAgentKafkaProjectionRealtimeConfig(value: unknown, path: string): NonNullable<HwlabRuntimeKafkaEventBridgeSpec["projectionRealtime"]> {
const raw = asRecord(value, path);
return {
outboxTailBatchSize: numberField(raw, "outboxTailBatchSize", path),
sseHeartbeatMs: numberField(raw, "sseHeartbeatMs", path),
sseDrainTimeoutMs: numberField(raw, "sseDrainTimeoutMs", path),
};
}
function codeAgentKafkaHealthThresholdsConfig(value: unknown, path: string): HwlabRuntimeKafkaEventBridgeSpec["healthThresholds"] {
const raw = asRecord(value, path);
return {
consumerLagWarning: nonNegativeIntegerField(raw, "consumerLagWarning", path),
outboxBacklogWarning: nonNegativeIntegerField(raw, "outboxBacklogWarning", path),
retryCountWarning: nonNegativeIntegerField(raw, "retryCountWarning", path),
failedInboxWarning: nonNegativeIntegerField(raw, "failedInboxWarning", path),
dlqWarning: nonNegativeIntegerField(raw, "dlqWarning", path),
};
}
scripts/src/hwlab-node-web-observe-runner-realtime-source.test.ts
+5 -27
View File
@@ -51,8 +51,8 @@ test("realtime fanout keeps the YAML terminal timeout when durationMs is absent"
conditionalEventTypePairs: { "tool-call": { agentrun: "tool_call", hwlab: "tool" } },
expectedKafka: {
topics: { stdio: "codex-stdio.raw.v1", agentrun: "agentrun.event.v1", hwlab: "hwlab.event.v1" },
groups: { directPublish: "direct", transactionalProjector: "projector", liveSse: "live" },
capabilities: { directPublish: true, liveKafkaSse: true, kafkaRefreshReplay: true },
groups: { transactionalProjector: "projector", liveSse: "live" },
capabilities: { transactionalProjector: true, projectionOutboxRelay: true, projectionRealtime: true },
},
};
@@ -194,7 +194,7 @@ test("realtime fanout validates stdio against the scoped AgentRun session lineag
assert.equal(expectedStreamSessionId("hwlab", hwlabSessionId), hwlabSessionId);
});
test("realtime fanout validates YAML-owned refresh and legacy live-only connected contracts", () => {
test("realtime fanout validates the fixed transactional replay and live connected contract", () => {
const source = nodeWebObserveRunnerRealtimeSource();
const build = new Function(`${source}\nreturn realtimeAssertConnectedContract;`) as () => (
connected: Record<string, unknown>,
@@ -207,7 +207,7 @@ test("realtime fanout validates YAML-owned refresh and legacy live-only connecte
const sessionId = "ses_refresh_contract";
const expectedKafka = {
topics: { hwlab: "hwlab.event.v1" },
capabilities: { directPublish: true, liveKafkaSse: true, kafkaRefreshReplay: true },
capabilities: { transactionalProjector: true, projectionOutboxRelay: true, projectionRealtime: true },
};
const refreshProfile = {
expectedKafka,
@@ -249,28 +249,6 @@ test("realtime fanout validates YAML-owned refresh and legacy live-only connecte
/replayed count exceeds matched count/u,
);
const liveProfile = {
expectedKafka: { topics: { hwlab: "hwlab.event.v1" }, capabilities: { directPublish: true, liveKafkaSse: true, kafkaRefreshReplay: false } },
expectedProductSse: {
deliverySemantics: "live-only",
liveOnly: true,
replay: false,
replaySupported: false,
lossPossible: true,
refreshHandoff: false,
replayPriorTraceOnReconnect: false,
},
};
assert.doesNotThrow(() => assertConnected({
realtimeSource: "hwlab.event.v1",
deliverySemantics: "live-only",
liveOnly: true,
replay: false,
replaySupported: false,
lossPossible: true,
filters: { sessionId, traceId: null },
capabilities: liveProfile.expectedKafka.capabilities,
}, sessionId, "live", liveProfile));
});
test("existing session refresh rejects duplicate or reordered Workbench identities", () => {
@@ -326,7 +304,7 @@ test("existing session refresh preserves rejected connected evidence and rejects
counts: { matched: 8, replayed: 8, buffered: 1, bufferedDelivered: 1, liveDelivered: 0, deduplicated: 0 },
};
const profile = {
expectedKafka: { topics: { hwlab: "hwlab.event.v1" }, capabilities: { directPublish: true, liveKafkaSse: true, kafkaRefreshReplay: true } },
expectedKafka: { topics: { hwlab: "hwlab.event.v1" }, capabilities: { transactionalProjector: true, projectionOutboxRelay: true, projectionRealtime: true } },
expectedProductSse: { deliverySemantics: "kafka-retention-then-live", liveOnly: false, replay: true, replaySupported: true, lossPossible: false, refreshHandoff: true },
};
const connected = {
scripts/src/hwlab-node-web-observe-runner-realtime-source.ts
+11 -4
View File
@@ -803,11 +803,18 @@ function realtimeFanoutEffectiveProfile(profile, durationMs) {
if (!expectedProductSse || typeof expectedProductSse.deliverySemantics !== "string") throw new Error("realtime fanout profile lacks YAML-owned product SSE expectations");
const expectedKafka = profile.expectedKafka && typeof profile.expectedKafka === "object" ? profile.expectedKafka : null;
if (!expectedKafka?.topics || !expectedKafka?.groups || !expectedKafka?.capabilities) throw new Error("realtime fanout profile lacks YAML-derived Kafka expectations");
if (expectedKafka.capabilities.directPublish !== true || expectedKafka.capabilities.liveKafkaSse !== true) {
throw new Error("realtime fanout profile requires YAML-enabled directPublish and liveKafkaSse capabilities");
if (expectedKafka.capabilities.transactionalProjector !== true
|| expectedKafka.capabilities.projectionOutboxRelay !== true
|| expectedKafka.capabilities.projectionRealtime !== true) {
throw new Error("realtime fanout profile requires the fixed transactional projection capabilities");
}
if (expectedProductSse.refreshHandoff !== (expectedKafka.capabilities.kafkaRefreshReplay === true)) {
throw new Error("realtime fanout product SSE refreshHandoff differs from the YAML Kafka capability");
if (expectedProductSse.deliverySemantics !== "kafka-retention-then-live"
|| expectedProductSse.liveOnly !== false
|| expectedProductSse.replay !== true
|| expectedProductSse.replaySupported !== true
|| expectedProductSse.lossPossible !== false
|| expectedProductSse.refreshHandoff !== true) {
throw new Error("realtime fanout product SSE must use fixed retention replay and live handoff semantics");
}
if (expectedProductSse.replayPriorTraceOnReconnect === true && expectedProductSse.replay !== true) {
throw new Error("realtime fanout replayPriorTraceOnReconnect requires replay=true");
scripts/src/hwlab-node-web-sentinel-cicd-jobs.ts
+2 -1
View File
@@ -791,7 +791,7 @@ export function sentinelPublishImageBuildShell(state: SentinelCicdState, jobName
].join("\n");
}
export function sentinelPublishShell(state: SentinelCicdState, jobName: string, publishGitops: boolean): string {
export function sentinelPublishShell(state: SentinelCicdState, jobName: string, publishGitops: boolean, featureConfigSchemaStage = ""): string {
const gitopsFiles = publishGitops ? sentinelGitopsFiles(state) : [];
const filesB64 = Buffer.from(JSON.stringify(gitopsFiles.map((file) => ({
path: file.path,
@@ -871,6 +871,7 @@ export function sentinelPublishShell(state: SentinelCicdState, jobName: string,
"}",
"console.error(JSON.stringify({event:'web-probe-sentinel-gitops-files', fileCount: files.length, valuesRedacted:true}));",
"NODE",
...(featureConfigSchemaStage.length === 0 ? [] : [featureConfigSchemaStage]),
" git add .",
" file_count=$(git diff --cached --name-only | wc -l | tr -d ' ')",
" if git diff --quiet --cached; then changed=false; else changed=true; git -c user.email=web-probe-sentinel@unidesk.local -c user.name='UniDesk Web Probe Sentinel' commit -m \"deploy: render web-probe sentinel ${source_commit}\"; fi",
scripts/src/hwlab-node/render.ts
+69 -53
View File
@@ -44,11 +44,15 @@ import { nodeRuntimeDeployYamlOverlayShellScript } from "./deploy-overlay";
import { pipelineProvenanceAnnotationKeys } from "../pipeline-provenance";
import { hwlabNodeDeliveryAuthority, hwlabNodeDeliveryNext } from "./delivery-authority";
import type { CicdDeliveryAuthority } from "../cicd-delivery-authority";
import { renderPacFeatureConfigSchemaStage } from "../pac-feature-config-schema-stage";
const runtimeGitopsObservabilityNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-observability.mjs"), "utf8").trimEnd();
const runtimeGitopsPipelineGuardNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-pipeline-guard.mjs"), "utf8").trimEnd();
const runtimeGitopsScriptsConfigMapNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-scripts-configmap.mjs"), "utf8").trimEnd();
const runtimeGitopsPostprocessNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-postprocess.mjs"), "utf8").trimEnd();
const runtimeGitopsVerifyNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-gitops-verify.mjs"), "utf8").trimEnd();
const featureConfigSchemaWarningNativeScript = readFileSync(rootPath("scripts/native/cicd/feature-config-schema-warning.mjs"), "utf8").trimEnd();
const ajv2020BundleScript = readFileSync(rootPath("scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js"), "utf8").trimEnd();
const runtimePipelineProvenanceNativeScript = readFileSync(rootPath("scripts/native/hwlab/runtime-pipeline-provenance.mjs"), "utf8").trimEnd();
export function nodeRuntimeGitMirrorJobName(mirror: NodeRuntimeGitMirrorTargetSpec, action: "sync" | "flush"): string {
@@ -753,6 +757,8 @@ export function summarizeNodeRuntimeControlPlaneStatus(
const pipelineRunDiagnostics = record(status.pipelineRunDiagnostics);
const argo = record(status.argo);
const runtime = record(status.runtime);
const codeAgentRuntime = record(runtime.codeAgentRuntime);
const kafkaAuthority = record(codeAgentRuntime.kafkaAuthority);
const publicProbes = record(status.publicProbes);
const gitMirror = record(status.gitMirror);
const gitMirrorCompact = record(gitMirror.compact);
@@ -822,8 +828,25 @@ export function summarizeNodeRuntimeControlPlaneStatus(
externalPostgresReady: runtime.externalPostgresBridge === undefined && runtime.externalPostgresSecrets === undefined
? null
: record(runtime.externalPostgresBridge).ready === true && record(runtime.externalPostgresSecrets).ready === true,
codeAgentRuntimeReady: record(runtime.codeAgentRuntime).required === true ? record(runtime.codeAgentRuntime).ready === true : null,
codeAgentRuntimeReason: typeof record(runtime.codeAgentRuntime).degradedReason === "string" ? record(runtime.codeAgentRuntime).degradedReason : null,
codeAgentRuntimeReady: codeAgentRuntime.required === true ? codeAgentRuntime.ready === true : null,
codeAgentRuntimeReason: typeof codeAgentRuntime.degradedReason === "string" ? codeAgentRuntime.degradedReason : null,
kafkaAuthority: Object.keys(kafkaAuthority).length === 0 ? null : {
authority: kafkaAuthority.authority ?? null,
fixedGroups: kafkaAuthority.fixedGroups ?? null,
capabilities: kafkaAuthority.capabilities ?? null,
liveReplay: kafkaAuthority.liveReplay ?? null,
healthObserved: kafkaAuthority.healthObserved === true,
healthStatus: kafkaAuthority.healthStatus ?? null,
consumerLag: kafkaAuthority.consumerLag ?? null,
backlogCount: kafkaAuthority.backlogCount ?? null,
retryCount: kafkaAuthority.retryCount ?? null,
failedInboxCount: kafkaAuthority.failedInboxCount ?? null,
dlqCount: kafkaAuthority.dlqCount ?? null,
thresholds: kafkaAuthority.thresholds ?? null,
warning: kafkaAuthority.warning === true,
warningReasons: kafkaAuthority.warningReasons ?? [],
blocking: false,
},
},
publicProbe: {
ready: publicProbes.ready === true,
@@ -1520,12 +1543,24 @@ export function renderNodeRuntimeControlPlaneOnNode(spec: HwlabRuntimeLaneSpec,
`--web-endpoint ${shellQuote(spec.publicWebUrl)}`,
`--out ${shellQuote(renderDir)}`,
].join(" "),
...nodeRuntimePipelinePostprocessScript(),
...nodeRuntimePipelinePostprocessScript(nodeRuntimeFeatureConfigSchemaStage(spec)),
].join("\n");
return { result: runNodeHostScriptAsync(spec, script, timeoutSeconds, `${spec.nodeId.toLowerCase()}-${spec.lane}-render`), renderDir, worktreeDir, location: "node-host" };
}
export function nodeRuntimePipelinePostprocessScript(): string[] {
export function nodeRuntimeFeatureConfigSchemaStage(spec: HwlabRuntimeLaneSpec): string {
const authority = hwlabNodeDeliveryAuthority({ node: spec.nodeId, lane: spec.lane, spec });
if (authority.kind !== "pac-pr-merge") throw new Error(`HWLAB ${spec.nodeId}/${spec.lane} requires exact PaC consumer authority for feature config validation`);
return renderPacFeatureConfigSchemaStage({
repositoryId: authority.consumer.repositoryRef,
consumer: authority.consumer.consumerId,
repoDirExpression: '"$PWD"',
renderedRootExpression: JSON.stringify(spec.runtimePath),
ajvBundleExpression: "'/etc/unidesk-cicd-runtime-gitops/ajv2020.min.js'",
});
}
export function nodeRuntimePipelinePostprocessScript(featureConfigSchemaStage: string): string[] {
return [
"node - \"$render_dir\" \"$overlay_b64\" <<'NODE'",
"const fs = require('fs');",
@@ -1534,6 +1569,7 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
"const renderDir = process.argv[2];",
"const overlay = JSON.parse(Buffer.from(process.argv[3], 'base64').toString('utf8'));",
`const runtimeGitopsObservabilityNativeScript = ${JSON.stringify(runtimeGitopsObservabilityNativeScript)};`,
`const featureConfigSchemaStage = ${JSON.stringify(featureConfigSchemaStage)};`,
"const pipelinePath = path.join(renderDir, overlay.tektonDir, 'pipeline.yaml');",
"let text = fs.readFileSync(pipelinePath, 'utf8');",
"let YAML = null;",
@@ -1562,6 +1598,9 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
"echo '{\"event\":\"prepare-source-dependencies\",\"status\":\"not-required\",\"dependency\":\"yaml\",\"manager\":\"inline-service-id-parser\"}' >&2",
"ci_timing_emit prepare-source-dependencies succeeded \"$prepare_source_dependencies_started_ms\"`;",
"}",
"function featureConfigSchemaValidationScript() {",
" return featureConfigSchemaStage;",
"}",
"function validatePrepareSourceDependencyScript(script) {",
" const marker = 'NODE_UNIDESK_YAML_DEPENDENCY';",
" let offset = 0;",
@@ -1818,12 +1857,6 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
"function cloudWebRuntimeEnvEntries() {",
" const exposure = overlay.publicExposure;",
" const entries = !exposure || !Array.isArray(exposure.extraProxies) ? [] : exposure.extraProxies.filter((proxy) => proxy && proxy.cloudWebEnvName && proxy.publicBaseUrl).map((proxy) => ({ name: String(proxy.cloudWebEnvName), value: String(proxy.publicBaseUrl) }));",
" const kafka = overlay.codeAgentRuntime && overlay.codeAgentRuntime.kafkaEventBridge;",
" if (kafka) {",
" entries.push({ name: 'HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED', value: String(kafka.enabled && kafka.features.liveKafkaSse) });",
" entries.push({ name: 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED', value: String(kafka.enabled && kafka.features.kafkaRefreshReplay) });",
" entries.push({ name: 'HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED', value: String(kafka.enabled && kafka.features.projectionRealtime) });",
" }",
" return entries;",
"}",
"function startupProbeFrom(probe) {",
@@ -1921,31 +1954,35 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
" if (codeAgentRuntime.kafkaEventBridge) {",
" const kafka = codeAgentRuntime.kafkaEventBridge;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_ENABLED', String(kafka.enabled)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED', String(kafka.enabled && (kafka.features.directPublish || kafka.features.transactionalProjector))) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED', String(kafka.enabled)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_BOOTSTRAP_SERVERS', String(kafka.bootstrapServers)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_STDIO_TOPIC', String(kafka.stdioTopic)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_AGENTRUN_EVENT_TOPIC', String(kafka.agentRunEventTopic)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_EVENT_TOPIC', String(kafka.hwlabEventTopic)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_CLIENT_ID', String(kafka.clientId)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_AGENTRUN_EVENT_GROUP_ID', String(kafka.directPublishConsumerGroupId)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_PROJECTOR_GROUP_ID', String(kafka.transactionalProjectorConsumerGroupId)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_HWLAB_EVENT_GROUP_ID', String(kafka.hwlabEventConsumerGroupId)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_DIRECT_PUBLISH_ENABLED', String(kafka.enabled && kafka.features.directPublish)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED', String(kafka.enabled && kafka.features.liveKafkaSse)) || codeAgentRuntimeChanged;",
" const refreshReplayEnabled = kafka.enabled && kafka.features.kafkaRefreshReplay;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED', String(refreshReplayEnabled)) || codeAgentRuntimeChanged;",
" if (refreshReplayEnabled) {",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_GROUP_PREFIX', String(kafka.refreshReplay.groupIdPrefix)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_TIMEOUT_MS', String(kafka.refreshReplay.timeoutMs)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_SCAN_LIMIT', String(kafka.refreshReplay.scanLimit)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_MATCHED_EVENT_LIMIT', String(kafka.refreshReplay.matchedEventLimit)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_LIVE_BUFFER_LIMIT', String(kafka.refreshReplay.liveBufferLimit)) || codeAgentRuntimeChanged;",
" if (kafka.enabled) {",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_PROJECTOR_HEARTBEAT_INTERVAL_MS', String(kafka.transactionalProjector.heartbeatIntervalMs)) || codeAgentRuntimeChanged;",
" } else {",
" codeAgentRuntimeChanged = removeEnvValues(container, ['HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_GROUP_PREFIX', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_TIMEOUT_MS', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_SCAN_LIMIT', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_MATCHED_EVENT_LIMIT', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_LIVE_BUFFER_LIMIT']) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = removeEnvValues(container, ['HWLAB_KAFKA_PROJECTOR_HEARTBEAT_INTERVAL_MS']) || codeAgentRuntimeChanged;",
" }",
" if (kafka.enabled) {",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_OUTBOX_RELAY_INTERVAL_MS', String(kafka.projectionOutboxRelay.intervalMs)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_OUTBOX_RELAY_BATCH_SIZE', String(kafka.projectionOutboxRelay.batchSize)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_OUTBOX_RELAY_LEASE_MS', String(kafka.projectionOutboxRelay.leaseMs)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_OUTBOX_RELAY_SEND_TIMEOUT_MS', String(kafka.projectionOutboxRelay.sendTimeoutMs)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_OUTBOX_RELAY_RETRY_BACKOFF_MS', String(kafka.projectionOutboxRelay.retryBackoffMs)) || codeAgentRuntimeChanged;",
" } else {",
" codeAgentRuntimeChanged = removeEnvValues(container, ['HWLAB_KAFKA_OUTBOX_RELAY_INTERVAL_MS', 'HWLAB_KAFKA_OUTBOX_RELAY_BATCH_SIZE', 'HWLAB_KAFKA_OUTBOX_RELAY_LEASE_MS', 'HWLAB_KAFKA_OUTBOX_RELAY_SEND_TIMEOUT_MS', 'HWLAB_KAFKA_OUTBOX_RELAY_RETRY_BACKOFF_MS']) || codeAgentRuntimeChanged;",
" }",
" if (kafka.enabled) {",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_OUTBOX_TAIL_BATCH_SIZE', String(kafka.projectionRealtime.outboxTailBatchSize)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_SSE_HEARTBEAT_MS', String(kafka.projectionRealtime.sseHeartbeatMs)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_SSE_DRAIN_TIMEOUT_MS', String(kafka.projectionRealtime.sseDrainTimeoutMs)) || codeAgentRuntimeChanged;",
" } else {",
" codeAgentRuntimeChanged = removeEnvValues(container, ['HWLAB_WORKBENCH_OUTBOX_TAIL_BATCH_SIZE', 'HWLAB_WORKBENCH_SSE_HEARTBEAT_MS', 'HWLAB_WORKBENCH_SSE_DRAIN_TIMEOUT_MS']) || codeAgentRuntimeChanged;",
" }",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_TRANSACTIONAL_PROJECTOR_ENABLED', String(kafka.enabled && kafka.features.transactionalProjector)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_KAFKA_PROJECTION_OUTBOX_RELAY_ENABLED', String(kafka.enabled && kafka.features.projectionOutboxRelay)) || codeAgentRuntimeChanged;",
" codeAgentRuntimeChanged = setEnvValue(container, 'HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED', String(kafka.enabled && kafka.features.projectionRealtime)) || codeAgentRuntimeChanged;",
" }",
" }",
" }",
@@ -2270,12 +2307,6 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
"function cloudWebRuntimeEnvEntries() {",
" const exposure = overlay.publicExposure;",
" const entries = !exposure || !Array.isArray(exposure.extraProxies) ? [] : exposure.extraProxies.filter((proxy) => proxy && proxy.cloudWebEnvName && proxy.publicBaseUrl).map((proxy) => ({ name: String(proxy.cloudWebEnvName), value: String(proxy.publicBaseUrl) }));",
" const kafka = overlay.codeAgentRuntime && overlay.codeAgentRuntime.kafkaEventBridge;",
" if (kafka) {",
" entries.push({ name: 'HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED', value: String(kafka.enabled && kafka.features.liveKafkaSse) });",
" entries.push({ name: 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED', value: String(kafka.enabled && kafka.features.kafkaRefreshReplay) });",
" entries.push({ name: 'HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED', value: String(kafka.enabled && kafka.features.projectionRealtime) });",
" }",
" return entries;",
"}",
"function workloadRef(item, file, container) { return { file, kind: item && item.kind, name: item && item.metadata && item.metadata.name, container: container && container.name }; }",
@@ -2349,31 +2380,14 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
" if (codeAgentRuntime.kafkaEventBridge) {",
" const kafka = codeAgentRuntime.kafkaEventBridge;",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_ENABLED', String(kafka.enabled));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED', String(kafka.enabled && (kafka.features.directPublish || kafka.features.transactionalProjector)));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED', String(kafka.enabled));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_BOOTSTRAP_SERVERS', String(kafka.bootstrapServers));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_STDIO_TOPIC', String(kafka.stdioTopic));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_AGENTRUN_EVENT_TOPIC', String(kafka.agentRunEventTopic));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_EVENT_TOPIC', String(kafka.hwlabEventTopic));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_CLIENT_ID', String(kafka.clientId));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_AGENTRUN_EVENT_GROUP_ID', String(kafka.directPublishConsumerGroupId));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_PROJECTOR_GROUP_ID', String(kafka.transactionalProjectorConsumerGroupId));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_HWLAB_EVENT_GROUP_ID', String(kafka.hwlabEventConsumerGroupId));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_DIRECT_PUBLISH_ENABLED', String(kafka.enabled && kafka.features.directPublish));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED', String(kafka.enabled && kafka.features.liveKafkaSse));",
" const refreshReplayEnabled = kafka.enabled && kafka.features.kafkaRefreshReplay;",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED', String(refreshReplayEnabled));",
" if (refreshReplayEnabled) {",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_GROUP_PREFIX', String(kafka.refreshReplay.groupIdPrefix));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_TIMEOUT_MS', String(kafka.refreshReplay.timeoutMs));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_SCAN_LIMIT', String(kafka.refreshReplay.scanLimit));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_MATCHED_EVENT_LIMIT', String(kafka.refreshReplay.matchedEventLimit));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_LIVE_BUFFER_LIMIT', String(kafka.refreshReplay.liveBufferLimit));",
" } else {",
" for (const envName of ['HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_GROUP_PREFIX', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_TIMEOUT_MS', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_SCAN_LIMIT', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_MATCHED_EVENT_LIMIT', 'HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_LIVE_BUFFER_LIMIT']) checkCodeAgentRuntimeAbsent(item, file, container, envName);",
" }",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_TRANSACTIONAL_PROJECTOR_ENABLED', String(kafka.enabled && kafka.features.transactionalProjector));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_KAFKA_PROJECTION_OUTBOX_RELAY_ENABLED', String(kafka.enabled && kafka.features.projectionOutboxRelay));",
" checkCodeAgentRuntimeValue(item, file, container, 'HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED', String(kafka.enabled && kafka.features.projectionRealtime));",
" }",
" }",
" }",
@@ -2407,7 +2421,6 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
"if (workloadCheck.wrongCodeAgentRuntimeEnvs.length > 0) fail('code-agent-runtime-env-mismatch', { refs: workloadCheck.wrongCodeAgentRuntimeEnvs.slice(0, 12), count: workloadCheck.wrongCodeAgentRuntimeEnvs.length });",
"if (overlay.codeAgentRuntime && overlay.codeAgentRuntime.enabled) checks.push('code-agent-runtime-env');",
"if (workloadCheck.wrongCloudWebRuntimeEnvs.length > 0) fail('cloud-web-runtime-env-mismatch', { refs: workloadCheck.wrongCloudWebRuntimeEnvs.slice(0, 12), count: workloadCheck.wrongCloudWebRuntimeEnvs.length });",
"if (cloudWebRuntimeEnvEntries().length > 0) checks.push('cloud-web-runtime-env');",
"const pg = overlay.externalPostgres;",
"if (pg && pg.serviceName) {",
" const access = pg.runtimeAccess || { endpointAddress: pg.endpointAddress, port: pg.port };",
@@ -2528,8 +2541,8 @@ export function nodeRuntimePipelinePostprocessScript(): string[] {
" return next;",
" });",
" result = result.replace(/(node scripts\\/run-bun\\.mjs scripts\\/gitops-render\\.mjs[^\\n]*--use-deploy-images[^\\n]*)/g, (match) => {",
" if (match.includes('--check')) return runtimeGitopsVerifyScript();",
" return `${match}\\n${[runtimePathOverlayScript(), runtimeGitopsPostprocessScript()].filter(Boolean).join('\\n')}`;",
" if (match.includes('--check')) return match;",
" return `${match}\\n${featureConfigSchemaValidationScript()}\\n${runtimePathOverlayScript()}`;",
" });",
" result = result.replace(/node scripts\\/run-bun\\.mjs scripts\\/gitops-render\\.mjs([^\\n]*?)--out \"\\$render_check_dir\"/g, (match) => match.includes('--gitops-root ') ? match : `${match} --gitops-root ${JSON.stringify(overlay.gitopsRoot)} --node ${shellSingle(overlay.nodeId)} --git-read-url ${shellSingle(overlay.gitReadUrl)} --git-write-url ${shellSingle(overlay.gitWriteUrl)} --runtime-endpoint ${shellSingle(overlay.publicApiUrl)} --web-endpoint ${shellSingle(overlay.publicWebUrl)}`);",
" validatePrepareSourceDependencyScript(result);",
@@ -2887,9 +2900,12 @@ function runtimeGitopsPipelineGuardScript(): string[] {
"runtime_gitops_guard_dir=\"$render_dir/.unidesk-runtime-gitops\"",
"mkdir -p \"$runtime_gitops_guard_dir\"",
...writeRuntimeGitopsNativeScript("runtime-gitops-pipeline-guard.mjs", runtimeGitopsPipelineGuardNativeScript, "UNIDESK_RUNTIME_GITOPS_PIPELINE_GUARD_MJS"),
...writeRuntimeGitopsNativeScript("runtime-gitops-scripts-configmap.mjs", runtimeGitopsScriptsConfigMapNativeScript, "UNIDESK_RUNTIME_GITOPS_SCRIPTS_CONFIGMAP_MJS"),
...writeRuntimeGitopsNativeScript("runtime-gitops-observability.mjs", runtimeGitopsObservabilityNativeScript, "UNIDESK_RUNTIME_GITOPS_OBSERVABILITY_MJS"),
...writeRuntimeGitopsNativeScript("runtime-gitops-postprocess.mjs", runtimeGitopsPostprocessNativeScript, "UNIDESK_RUNTIME_GITOPS_POSTPROCESS_MJS"),
...writeRuntimeGitopsNativeScript("runtime-gitops-verify.mjs", runtimeGitopsVerifyNativeScript, "UNIDESK_RUNTIME_GITOPS_VERIFY_MJS"),
...writeRuntimeGitopsNativeScript("feature-config-schema-warning.mjs", featureConfigSchemaWarningNativeScript, "UNIDESK_FEATURE_CONFIG_SCHEMA_WARNING_MJS"),
...writeRuntimeGitopsNativeScript("ajv2020.min.js", ajv2020BundleScript, "UNIDESK_AJV2020_MIN_JS"),
[
"UNIDESK_RUNTIME_GITOPS_OVERLAY_B64=\"$overlay_b64\"",
"node \"$runtime_gitops_guard_dir/runtime-gitops-pipeline-guard.mjs\"",
scripts/src/hwlab-node/web-probe-observe-actions.test.ts
+37 -8
View File
@@ -3,11 +3,13 @@ import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { test } from "bun:test";
import { readFileSync } from "node:fs";
import { hwlabRuntimeLaneSpecForNode } from "../hwlab-node-lanes";
import { rootPath } from "../config";
import { buildWebObserveCommandVisibility, nodeWebProbeRealtimeFanoutProfiles, nodeWebProbeWorkbenchKafkaDebugReplay, nodeWebProbeWorkbenchTraceReadability, resolveWebObserveActionJson } from "./web-probe-observe-actions";
test("realtime fanout runner contract derives topics, groups, and independent capabilities from owning YAML", () => {
test("realtime authority runner contract derives topics, groups, and transactional capabilities from owning YAML", () => {
const profiles = nodeWebProbeRealtimeFanoutProfiles(hwlabRuntimeLaneSpecForNode("v03", "NC01"));
const expectedKafka = profiles["pure-kafka-live"]?.expectedKafka as Record<string, any>;
assert.deepEqual(expectedKafka.topics, {
@@ -16,17 +18,34 @@ test("realtime fanout runner contract derives topics, groups, and independent ca
hwlab: "hwlab.event.v1",
});
assert.deepEqual(expectedKafka.groups, {
directPublish: "hwlab-v03-agentrun-event-direct-publish",
transactionalProjector: "hwlab-v03-agentrun-event-projector",
liveSse: "hwlab-v03-workbench-live-sse",
});
assert.deepEqual(expectedKafka.capabilities, {
directPublish: true,
liveKafkaSse: true,
kafkaRefreshReplay: true,
transactionalProjector: false,
projectionOutboxRelay: false,
projectionRealtime: false,
transactionalProjector: true,
projectionOutboxRelay: true,
projectionRealtime: true,
});
const kafka = hwlabRuntimeLaneSpecForNode("v03", "NC01").codeAgentRuntime?.kafkaEventBridge;
assert.deepEqual(kafka?.transactionalProjector, { heartbeatIntervalMs: 2_000 });
assert.deepEqual(kafka?.projectionOutboxRelay, {
intervalMs: 250,
batchSize: 100,
leaseMs: 30_000,
sendTimeoutMs: 10_000,
retryBackoffMs: 1_000,
});
assert.deepEqual(kafka?.projectionRealtime, {
outboxTailBatchSize: 100,
sseHeartbeatMs: 15_000,
sseDrainTimeoutMs: 2_500,
});
assert.deepEqual(kafka?.healthThresholds, {
consumerLagWarning: 0,
outboxBacklogWarning: 0,
retryCountWarning: 0,
failedInboxWarning: 0,
dlqWarning: 0,
});
assert.deepEqual(profiles["pure-kafka-live"]?.requiredEventTypes, {
agentrun: ["user_message", "assistant_message", "terminal_status"],
@@ -38,6 +57,16 @@ test("realtime fanout runner contract derives topics, groups, and independent ca
});
});
test("realtime authority parser and inline render expose no direct authority or duplicate native postprocess execution", () => {
const laneSource = readFileSync(rootPath("scripts", "src", "hwlab-node-lanes.ts"), "utf8");
const renderSource = readFileSync(rootPath("scripts", "src", "hwlab-node", "render.ts"), "utf8");
assert.doesNotMatch(laneSource, /direct-live/u);
assert.equal(renderSource.includes("UNIDESK_RUNTIME_GITOPS_RUNTIME_PATH"), false);
assert.equal(renderSource.includes('UNIDESK_RUNTIME_GITOPS_OVERLAY_B64="$overlay_b64" UNIDESK_RUNTIME_GITOPS_RUNTIME_PATH'), false);
assert.match(renderSource, /runtimeGitopsPostprocessScript\(\)/u);
assert.match(renderSource, /writeRuntimeGitopsNativeScript\("runtime-gitops-postprocess\.mjs"/u);
});
test("Workbench Kafka debug replay runner contract is derived from owning YAML", () => {
assert.deepEqual(nodeWebProbeWorkbenchKafkaDebugReplay(hwlabRuntimeLaneSpecForNode("v03", "NC01")), {
enabled: true,
scripts/src/hwlab-node/web-probe-observe-actions.ts
+5 -2
View File
@@ -23,7 +23,11 @@ export function nodeWebProbeRealtimeFanoutProfiles(spec: HwlabRuntimeLaneSpec):
if (Object.keys(profiles).length === 0) return {};
const kafka = spec.codeAgentRuntime?.kafkaEventBridge;
if (kafka === undefined) throw new Error("realtimeFanoutProfiles requires codeAgentRuntime.kafkaEventBridge in the owning YAML");
const capabilities = Object.fromEntries(Object.entries(kafka.features).map(([name, enabled]) => [name, kafka.enabled && enabled]));
const capabilities = {
transactionalProjector: kafka.enabled,
projectionOutboxRelay: kafka.enabled,
projectionRealtime: kafka.enabled,
};
return Object.fromEntries(Object.entries(profiles).map(([id, profile]) => {
if (profile.businessEvent !== kafka.hwlabEventTopic) {
throw new Error(`realtimeFanoutProfiles.${id}.businessEvent must match codeAgentRuntime.kafkaEventBridge.hwlabEventTopic`);
@@ -38,7 +42,6 @@ export function nodeWebProbeRealtimeFanoutProfiles(spec: HwlabRuntimeLaneSpec):
hwlab: kafka.hwlabEventTopic,
},
groups: {
directPublish: kafka.directPublishConsumerGroupId,
transactionalProjector: kafka.transactionalProjectorConsumerGroupId,
liveSse: kafka.hwlabEventConsumerGroupId,
},
scripts/src/hwlab-node/web-probe.ts
+99 -73
View File
@@ -1482,6 +1482,7 @@ export function nodeRuntimeCodeAgentRuntimeStatus(spec: HwlabRuntimeLaneSpec, na
const apiKey = secretKeyStatus(spec, runtime.apiKeySecretName, runtime.apiKeySecretKey);
const cloudApiEnv = nodeRuntimeCodeAgentCloudApiEnvStatus(spec, runtime);
const cloudWebEnv = nodeRuntimeCodeAgentCloudWebEnvStatus(spec, runtime);
const kafkaAuthority = nodeRuntimeKafkaAuthorityStatus(spec, runtime);
const ready = runnerNamespace.exitCode === 0
&& secretNamespace.exitCode === 0
&& managerNamespace.exitCode === 0
@@ -1538,12 +1539,73 @@ export function nodeRuntimeCodeAgentRuntimeStatus(spec: HwlabRuntimeLaneSpec, na
apiKey,
cloudApiEnv,
cloudWebEnv,
kafkaAuthority,
repoUrlFrom: runtime.repoUrlFrom,
providerIdFrom: runtime.providerIdFrom,
valuesPrinted: false,
};
}
function nodeRuntimeKafkaAuthorityStatus(spec: HwlabRuntimeLaneSpec, runtime: NonNullable<HwlabRuntimeLaneSpec["codeAgentRuntime"]>): Record<string, unknown> {
const kafka = runtime.kafkaEventBridge;
if (kafka === undefined || kafka.enabled === false) return { required: false, authority: kafka === undefined ? null : "transactional-projection", valuesPrinted: false };
const healthUrl = `${spec.publicApiUrl.replace(/\/+$/u, "")}/health/ready`;
const result = runCommand(["curl", "-k", "-sS", "--connect-timeout", "5", "--max-time", "12", healthUrl], repoRoot, { timeoutMs: 15_000 });
let health: Record<string, unknown> = {};
try {
health = result.exitCode === 0 ? record(JSON.parse(result.stdout)) : {};
} catch {
health = {};
}
const projector = record(health.kafkaProjector);
const components = Array.isArray(projector.components) ? projector.components.map(record) : [];
const observations = [projector, ...components];
const consumerLag = observations.map((item) => record(item.consumerLag).total).find((value) => typeof value === "number") ?? null;
const backlogCount = observations.map((item) => item.backlogCount).find((value) => typeof value === "number") ?? null;
const retryCount = observations.map((item) => item.retryCount).find((value) => typeof value === "number") ?? null;
const failedInboxCount = observations.map((item) => item.failedInboxCount).find((value) => typeof value === "number") ?? null;
const dlqCount = observations.map((item) => item.dlqCount).find((value) => typeof value === "number") ?? null;
const thresholds = kafka.healthThresholds;
const warningReasons = [
typeof consumerLag === "number" && consumerLag > thresholds.consumerLagWarning ? "consumer-lag" : null,
typeof backlogCount === "number" && backlogCount > thresholds.outboxBacklogWarning ? "outbox-backlog" : null,
typeof retryCount === "number" && retryCount > thresholds.retryCountWarning ? "outbox-retry" : null,
typeof failedInboxCount === "number" && failedInboxCount > thresholds.failedInboxWarning ? "failed-inbox" : null,
typeof dlqCount === "number" && dlqCount > thresholds.dlqWarning ? "dlq" : null,
].filter((value): value is string => value !== null);
return {
required: true,
authority: "transactional-projection",
fixedGroups: {
transactionalProjector: kafka.transactionalProjectorConsumerGroupId,
liveSse: kafka.hwlabEventConsumerGroupId,
},
capabilities: {
transactionalProjector: true,
projectionOutboxRelay: true,
projectionRealtime: true,
},
liveReplay: {
live: true,
replay: true,
cursor: "projection-outbox-seq",
},
healthObserved: Object.keys(projector).length > 0,
healthStatus: projector.status ?? null,
consumerLag,
backlogCount,
retryCount,
failedInboxCount,
dlqCount,
thresholds,
warning: warningReasons.length > 0,
warningReasons,
blocking: false,
result: compactRuntimeCommand(result),
valuesPrinted: false,
};
}
function nodeRuntimeCodeAgentManagerTarget(managerUrl: string, fallbackNamespace: string): { namespace: string; serviceName: string } {
let parsed: URL | null = null;
try {
@@ -1632,47 +1694,55 @@ function nodeRuntimeCodeAgentCloudApiEnvStatus(spec: HwlabRuntimeLaneSpec, runti
const kafkaEventBridge = runtime.kafkaEventBridge;
if (kafkaEventBridge !== undefined) {
expectValue("HWLAB_KAFKA_ENABLED", String(kafkaEventBridge.enabled));
expectValue("HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED", String(kafkaEventBridge.enabled && (kafkaEventBridge.features.directPublish || kafkaEventBridge.features.transactionalProjector)));
expectValue("HWLAB_KAFKA_AGENTRUN_EVENT_CONSUME_ENABLED", String(kafkaEventBridge.enabled));
expectValue("HWLAB_KAFKA_BOOTSTRAP_SERVERS", kafkaEventBridge.bootstrapServers);
expectValue("HWLAB_KAFKA_STDIO_TOPIC", kafkaEventBridge.stdioTopic);
expectValue("HWLAB_KAFKA_AGENTRUN_EVENT_TOPIC", kafkaEventBridge.agentRunEventTopic);
expectValue("HWLAB_KAFKA_EVENT_TOPIC", kafkaEventBridge.hwlabEventTopic);
expectValue("HWLAB_KAFKA_CLIENT_ID", kafkaEventBridge.clientId);
expectValue("HWLAB_KAFKA_AGENTRUN_EVENT_GROUP_ID", kafkaEventBridge.directPublishConsumerGroupId);
expectValue("HWLAB_KAFKA_PROJECTOR_GROUP_ID", kafkaEventBridge.transactionalProjectorConsumerGroupId);
expectValue("HWLAB_KAFKA_HWLAB_EVENT_GROUP_ID", kafkaEventBridge.hwlabEventConsumerGroupId);
expectValue("HWLAB_KAFKA_DIRECT_PUBLISH_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.directPublish));
expectValue("HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.liveKafkaSse));
const refreshReplayEnabled = kafkaEventBridge.enabled && kafkaEventBridge.features.kafkaRefreshReplay;
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED", String(refreshReplayEnabled));
const refreshReplayEnvNames = [
"HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_GROUP_PREFIX",
"HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_TIMEOUT_MS",
"HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_SCAN_LIMIT",
"HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_MATCHED_EVENT_LIMIT",
"HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_LIVE_BUFFER_LIMIT",
];
if (refreshReplayEnabled) {
const refreshReplay = kafkaEventBridge.refreshReplay;
if (refreshReplay === undefined) throw new Error("codeAgentRuntime.kafkaEventBridge.refreshReplay is required when Kafka refresh replay is enabled");
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_GROUP_PREFIX", refreshReplay.groupIdPrefix);
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_TIMEOUT_MS", String(refreshReplay.timeoutMs));
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_SCAN_LIMIT", String(refreshReplay.scanLimit));
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_MATCHED_EVENT_LIMIT", String(refreshReplay.matchedEventLimit));
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_LIVE_BUFFER_LIMIT", String(refreshReplay.liveBufferLimit));
if (kafkaEventBridge.enabled) {
if (kafkaEventBridge.transactionalProjector === undefined) throw new Error("codeAgentRuntime.kafkaEventBridge.transactionalProjector is required when transactional projector is enabled");
expectValue("HWLAB_KAFKA_PROJECTOR_HEARTBEAT_INTERVAL_MS", String(kafkaEventBridge.transactionalProjector.heartbeatIntervalMs));
} else {
for (const name of refreshReplayEnvNames) expectAbsent(name);
expectAbsent("HWLAB_KAFKA_PROJECTOR_HEARTBEAT_INTERVAL_MS");
}
const relayEnvNames = [
"HWLAB_KAFKA_OUTBOX_RELAY_INTERVAL_MS",
"HWLAB_KAFKA_OUTBOX_RELAY_BATCH_SIZE",
"HWLAB_KAFKA_OUTBOX_RELAY_LEASE_MS",
"HWLAB_KAFKA_OUTBOX_RELAY_SEND_TIMEOUT_MS",
"HWLAB_KAFKA_OUTBOX_RELAY_RETRY_BACKOFF_MS",
];
if (kafkaEventBridge.enabled) {
const relay = kafkaEventBridge.projectionOutboxRelay;
if (relay === undefined) throw new Error("codeAgentRuntime.kafkaEventBridge.projectionOutboxRelay is required when projection outbox relay is enabled");
expectValue("HWLAB_KAFKA_OUTBOX_RELAY_INTERVAL_MS", String(relay.intervalMs));
expectValue("HWLAB_KAFKA_OUTBOX_RELAY_BATCH_SIZE", String(relay.batchSize));
expectValue("HWLAB_KAFKA_OUTBOX_RELAY_LEASE_MS", String(relay.leaseMs));
expectValue("HWLAB_KAFKA_OUTBOX_RELAY_SEND_TIMEOUT_MS", String(relay.sendTimeoutMs));
expectValue("HWLAB_KAFKA_OUTBOX_RELAY_RETRY_BACKOFF_MS", String(relay.retryBackoffMs));
} else {
for (const name of relayEnvNames) expectAbsent(name);
}
const projectionRealtimeEnvNames = ["HWLAB_WORKBENCH_OUTBOX_TAIL_BATCH_SIZE", "HWLAB_WORKBENCH_SSE_HEARTBEAT_MS", "HWLAB_WORKBENCH_SSE_DRAIN_TIMEOUT_MS"];
if (kafkaEventBridge.enabled) {
const realtime = kafkaEventBridge.projectionRealtime;
if (realtime === undefined) throw new Error("codeAgentRuntime.kafkaEventBridge.projectionRealtime is required when projection realtime is enabled");
expectValue("HWLAB_WORKBENCH_OUTBOX_TAIL_BATCH_SIZE", String(realtime.outboxTailBatchSize));
expectValue("HWLAB_WORKBENCH_SSE_HEARTBEAT_MS", String(realtime.sseHeartbeatMs));
expectValue("HWLAB_WORKBENCH_SSE_DRAIN_TIMEOUT_MS", String(realtime.sseDrainTimeoutMs));
} else {
for (const name of projectionRealtimeEnvNames) expectAbsent(name);
}
expectValue("HWLAB_KAFKA_TRANSACTIONAL_PROJECTOR_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.transactionalProjector));
expectValue("HWLAB_KAFKA_PROJECTION_OUTBOX_RELAY_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.projectionOutboxRelay));
expectValue("HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.projectionRealtime));
}
return {
ready: mismatches.length === 0,
deploymentExists: true,
deploymentName: "hwlab-cloud-api",
containerName: "hwlab-cloud-api",
checkedEnvCount: 9 + (kafkaShadowProducer === undefined ? 0 : 5) + (kafkaEventBridge === undefined ? 0 : 21),
checkedEnvCount: 9 + (kafkaShadowProducer === undefined ? 0 : 5) + (kafkaEventBridge === undefined ? 0 : 22),
envCount: envByName.size,
mismatches,
result: compactCodeAgentDeploymentProbeResult(envProbe),
@@ -1681,53 +1751,9 @@ function nodeRuntimeCodeAgentCloudApiEnvStatus(spec: HwlabRuntimeLaneSpec, runti
}
function nodeRuntimeCodeAgentCloudWebEnvStatus(spec: HwlabRuntimeLaneSpec, runtime: NonNullable<HwlabRuntimeLaneSpec["codeAgentRuntime"]>): Record<string, unknown> {
const kafkaEventBridge = runtime.kafkaEventBridge;
if (kafkaEventBridge === undefined) return { required: false, ready: true, checkedEnvCount: 0, valuesPrinted: false };
const envProbe = runNodeK3sArgs(spec, [
"kubectl",
"-n",
spec.runtimeNamespace,
"get",
"deployment",
"hwlab-cloud-web",
"-o",
'jsonpath={range .spec.template.spec.containers[?(@.name=="hwlab-cloud-web")].env[*]}{.name}{"\\t"}{.value}{"\\t"}{.valueFrom.secretKeyRef.name}{"\\t"}{.valueFrom.secretKeyRef.key}{"\\n"}{end}',
], 60);
if (envProbe.exitCode !== 0) {
return {
required: true,
ready: false,
deploymentExists: false,
mismatches: [{ name: "hwlab-cloud-web", reason: "deployment-missing" }],
result: compactCodeAgentDeploymentProbeResult(envProbe),
valuesPrinted: false,
};
}
const envByName = parseCodeAgentEnvProbe(envProbe.stdout);
const mismatches: Array<Record<string, unknown>> = [];
const expectValue = (name: string, expected: string): void => {
const item = envByName.get(name);
if (item === undefined) {
mismatches.push({ name, expected, present: false, kind: "value" });
return;
}
if (item.value !== expected) mismatches.push({ name, expected, value: item.value, present: true, kind: "value" });
};
expectValue("HWLAB_WORKBENCH_LIVE_KAFKA_SSE_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.liveKafkaSse));
expectValue("HWLAB_WORKBENCH_KAFKA_REFRESH_REPLAY_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.kafkaRefreshReplay));
expectValue("HWLAB_WORKBENCH_PROJECTION_REALTIME_ENABLED", String(kafkaEventBridge.enabled && kafkaEventBridge.features.projectionRealtime));
return {
required: true,
ready: mismatches.length === 0,
deploymentExists: true,
deploymentName: "hwlab-cloud-web",
containerName: "hwlab-cloud-web",
checkedEnvCount: 3,
envCount: envByName.size,
mismatches,
result: compactCodeAgentDeploymentProbeResult(envProbe),
valuesPrinted: false,
};
void spec;
void runtime;
return { required: false, ready: true, checkedEnvCount: 0, valuesPrinted: false };
}
type CodeAgentEnvProbeRow = {
scripts/src/pac-feature-config-schema-stage.test.ts
+123
View File
@@ -0,0 +1,123 @@
import { expect, test } from "bun:test";
import { mkdtempSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { spawnSync } from "node:child_process";
import { resolveAgentRunLaneTarget } from "./agentrun-lanes";
import { renderAgentRunPipelineManifest } from "./agentrun-manifests";
import { renderPacFeatureConfigSchemaStage } from "./pac-feature-config-schema-stage";
test("shared stage injects owning YAML OTel values for all three PaC repositories", () => {
for (const [repositoryId, consumer] of [
["hwlab-nc01-v03", "hwlab-nc01-v03"],
["agentrun-nc01-v02", "agentrun-nc01-v02"],
["sentinel-nc01-v03", "sentinel-nc01-v03"],
] as const) {
const stage = renderPacFeatureConfigSchemaStage({
repositoryId,
consumer,
repoDirExpression: "'/workspace/source'",
renderedRootExpression: "'/workspace/rendered'",
});
expect(stage).toContain("feature-config-schema-validation");
expect(stage).toContain("cicd.feature_config.schema");
expect(stage).toContain("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT='http://otel-collector.platform-infra.svc.cluster.local:4318/v1/traces'");
expect(stage).toContain("OTEL_TRACES_SAMPLER_ARG='1'");
expect(stage).toContain("OTEL_EXPORTER_TIMEOUT_MS='300'");
expect(stage).toContain("NODE_UNIDESK_AJV2020_BUNDLE");
expect(stage).not.toContain("process.env.webProbeSentinel");
}
});
test("HWLAB, AgentRun, and Sentinel normal stages validate without source workspace node_modules", () => {
const root = mkdtempSync(join(tmpdir(), "feature-config-consumers-"));
try {
const source = join(root, "source");
const rendered = join(root, "rendered");
const temp = join(root, "tmp");
mkdirSync(join(source, "config"), { recursive: true });
mkdirSync(rendered, { recursive: true });
mkdirSync(temp);
writeFileSync(join(source, "config", "feature-config.schema.json"), JSON.stringify({
$schema: "https://json-schema.org/draft/2020-12/schema",
type: "object",
properties: { webProbeSentinel: { type: "boolean", "x-unidesk-feature": "web-probe-sentinel" } },
additionalProperties: false,
}));
writeFileSync(join(rendered, "deployment.json"), JSON.stringify({
apiVersion: "apps/v1",
kind: "Deployment",
spec: { template: { spec: { containers: [{ name: "fixture", env: [{ name: "webProbeSentinel", value: "true" }] }] } } },
}));
const vendorBundle = resolve(import.meta.dir, "../vendor/ajv-dist/8.17.1/ajv2020.min.js");
for (const consumer of [
{ repositoryId: "hwlab-nc01-v03", consumer: "hwlab-nc01-v03", ajvBundleExpression: JSON.stringify(vendorBundle) },
{ repositoryId: "agentrun-nc01-v02", consumer: "agentrun-nc01-v02" },
{ repositoryId: "sentinel-nc01-v03", consumer: "sentinel-nc01-v03" },
]) {
const stage = renderPacFeatureConfigSchemaStage({
...consumer,
repoDirExpression: JSON.stringify(source),
renderedRootExpression: JSON.stringify(rendered),
});
const result = spawnSync("sh", [], {
input: `set -eu\n${stage}\n`,
encoding: "utf8",
env: { ...process.env, TMPDIR: temp },
});
expect(result.status, consumer.consumer).toBe(0);
expect(result.stderr, consumer.consumer).toContain('"code":"feature-config-schema-valid"');
expect(result.stderr, consumer.consumer).not.toContain("feature-config-validator-unavailable");
expect(result.stderr, consumer.consumer).not.toContain("feature-config-validator-process-failure");
expect(readdirSync(temp), consumer.consumer).toEqual([]);
}
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("embedded node process failure remains a typed warning with zero shell exit", () => {
const root = mkdtempSync(join(tmpdir(), "feature-config-stage-"));
try {
const bin = join(root, "bin");
const temp = join(root, "tmp");
mkdirSync(bin);
mkdirSync(temp);
writeFileSync(join(bin, "node"), "#!/bin/sh\nexit 71\n", { mode: 0o755 });
const stage = renderPacFeatureConfigSchemaStage({
repositoryId: "sentinel-nc01-v03",
consumer: "sentinel-nc01-v03",
repoDirExpression: "'/workspace/source'",
renderedRootExpression: "'/workspace/rendered'",
});
const result = spawnSync("sh", [], {
input: `set -eu\n${stage}\n`,
env: { ...process.env, PATH: `${bin}:${process.env.PATH ?? ""}`, TMPDIR: temp },
encoding: "utf8",
});
expect(result.status).toBe(0);
expect(result.stderr).toContain("feature-config-validator-process-failure");
expect(readdirSync(temp)).toEqual([]);
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("AgentRun normal GitOps publish task contains the shared warning-only stage", () => {
const { spec } = resolveAgentRunLaneTarget({ node: "NC01", lane: "nc01-v02" });
const pipeline = renderAgentRunPipelineManifest(spec) as Record<string, any>;
const scripts = pipeline.spec.tasks.flatMap((task: Record<string, any>) => task.taskSpec.steps.map((step: Record<string, any>) => String(step.script ?? ""))).join("\n");
expect(scripts).toContain("UNIDESK_PAC_CONSUMER='agentrun-nc01-v02'");
expect(scripts).toContain("feature-config-schema-validation");
expect(scripts).toContain("feature-config-validator-process-failure");
expect(scripts).toContain("OTEL_EXPORTER_TIMEOUT_MS='300'");
expect(scripts).toContain("NODE_UNIDESK_AJV2020_BUNDLE");
});
test("repo-owned schema is Draft 2020-12 with one canonical feature property", () => {
const schema = JSON.parse(readFileSync("config/feature-config.schema.json", "utf8"));
expect(schema.$schema).toBe("https://json-schema.org/draft/2020-12/schema");
expect(schema.properties.webProbeSentinel["x-unidesk-feature"]).toBe("web-probe-sentinel");
expect(Object.keys(schema.properties)).toEqual(["webProbeSentinel"]);
});
scripts/src/pac-feature-config-schema-stage.ts
+66
View File
@@ -0,0 +1,66 @@
import { readFileSync } from "node:fs";
import { rootPath } from "./config";
import { loadCicdOtelRuntimeConfig } from "../native/cicd/otel-runtime-config";
const validatorSource = readFileSync(rootPath("scripts/native/cicd/feature-config-schema-warning.mjs"), "utf8").trimEnd();
const ajv2020BundleSource = readFileSync(rootPath("scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js"), "utf8").trimEnd();
export interface PacFeatureConfigSchemaStageOptions {
readonly repositoryId: string;
readonly consumer: string;
readonly repoDirExpression: string;
readonly renderedRootExpression: string;
readonly ajvBundleExpression?: string | null;
}
export function renderPacFeatureConfigSchemaStage(options: PacFeatureConfigSchemaStageOptions): string {
const otel = loadCicdOtelRuntimeConfig(options.repositoryId);
const materializeBundle = options.ajvBundleExpression === undefined || options.ajvBundleExpression === null;
return [
"feature_config_schema_started_ms=\"$(date +%s%3N 2>/dev/null || date +%s000)\"",
`export UNIDESK_FEATURE_CONFIG_REPO_DIR=${options.repoDirExpression}`,
`export UNIDESK_FEATURE_CONFIG_RENDERED_ROOT=${options.renderedRootExpression}`,
`export UNIDESK_PAC_CONSUMER=${shellSingle(options.consumer)}`,
`export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=${shellSingle(otel.endpoint)}`,
`export OTEL_SERVICE_NAME=${shellSingle(otel.serviceName)}`,
`export OTEL_TRACES_SAMPLER_ARG=${shellSingle(String(otel.samplingRatio))}`,
`export OTEL_EXPORTER_TIMEOUT_MS=${shellSingle(String(otel.timeoutMs))}`,
...(materializeBundle
? [
"feature_config_ajv_bundle=''",
"if feature_config_ajv_bundle=\"$(mktemp \"${TMPDIR:-/tmp}/unidesk-ajv2020.XXXXXX\" 2>/dev/null)\"; then",
" if ! cat >\"$feature_config_ajv_bundle\" <<'NODE_UNIDESK_AJV2020_BUNDLE'",
ajv2020BundleSource,
"NODE_UNIDESK_AJV2020_BUNDLE",
" then",
" rm -f \"$feature_config_ajv_bundle\" || true",
" feature_config_ajv_bundle=''",
" fi",
"fi",
"export UNIDESK_AJV2020_BUNDLE=\"${feature_config_ajv_bundle:-${TMPDIR:-/tmp}/unidesk-ajv2020-unavailable}\"",
]
: [`export UNIDESK_AJV2020_BUNDLE=${options.ajvBundleExpression}`]),
"if ! node --input-type=module <<'NODE_UNIDESK_FEATURE_CONFIG_SCHEMA'",
validatorSource,
"await runFeatureConfigSchemaValidation({",
" repoDir: process.env.UNIDESK_FEATURE_CONFIG_REPO_DIR,",
" renderedRoot: process.env.UNIDESK_FEATURE_CONFIG_RENDERED_ROOT,",
" consumer: process.env.UNIDESK_PAC_CONSUMER,",
"});",
"NODE_UNIDESK_FEATURE_CONFIG_SCHEMA",
"then",
" printf '{\"event\":\"feature-config-schema-validation\",\"warning\":true,\"blocking\":false,\"code\":\"feature-config-validator-process-failure\",\"mutation\":false,\"valuesPrinted\":false}\\n' >&2",
"fi",
...(materializeBundle ? ["if [ -n \"$feature_config_ajv_bundle\" ]; then rm -f \"$feature_config_ajv_bundle\" || true; fi"] : []),
"if command -v ci_timing_emit >/dev/null 2>&1; then",
" ci_timing_emit feature-config-schema-validation succeeded \"$feature_config_schema_started_ms\"",
"else",
" printf '{\"event\":\"ci.stage.timing\",\"stage\":\"feature-config-schema-validation\",\"status\":\"succeeded\",\"blocking\":false}\\n' >&2",
"fi",
].join("\n");
}
function shellSingle(value: string): string {
return `'${value.replaceAll("'", `'"'"'`)}'`;
}
scripts/src/platform-infra-pac-failure-evidence.test.ts
+36
View File
@@ -11,6 +11,9 @@ const require = createRequire(import.meta.url);
const { parsePacLogRecords } = require("../native/cicd/pac-status-evaluator.cjs") as {
parsePacLogRecords(value: string): Array<Record<string, unknown>>;
};
const { extractPacSourceObservation } = require("../native/cicd/pac-status-evaluator.cjs") as {
extractPacSourceObservation(records: Array<Record<string, unknown>>, sourceCommit: string): Record<string, unknown>;
};
describe("PaC 失败证据合同", () => {
test("真实 sentinel TaskRun 日志保留 typed 首断点", () => {
@@ -52,6 +55,39 @@ describe("PaC 失败证据合同", () => {
expect(compactStatus).toMatchObject({ artifact: { traceId: row.traceId, firstBreak: row.firstBreak, error: row.error } });
});
test("feature-config warning 在 status、history、debug-step 同构有界投影", () => {
const observation = extractPacSourceObservation([
{
event: "pac-delivery-plan",
sourceCommitId: "a".repeat(40),
affectedServices: [],
rolloutServices: [],
buildServices: [],
reusedServices: [],
},
{
event: "feature-config-schema-validation",
warning: true,
blocking: false,
code: "feature-config-input-missing",
schemaPath: "config/feature-config.schema.json",
errorCount: 1,
firstError: "rendered workload candidate snapshot is missing",
mutation: false,
valuesPrinted: false,
},
]);
const expected = "feature-config-schema: warning=true blocking=false code=feature-config-input-missing path=config/feature-config.schema.json errors=1 first=rendered workload candidate snapshot is missing";
const history = renderHistory({ rows: [{ id: "run", sourceObservation: observation }], detailId: "run", config: {}, historyErrors: [], next: {} }).renderedText;
const debug = renderDebugStep({ realRun: { found: true, pipelineRun: {}, artifact: { sourceObservation: observation } }, checks: [], target: {}, next: {} }).renderedText;
const status = renderStatus({ summary: { latestPipelineRun: {}, artifact: {}, diagnostics: {}, sourceObservation: observation }, consumer: {}, deliveryAuthority: {}, config: {}, coverage: [], target: {}, next: {} }).renderedText;
expect(status).toContain(expected);
expect(debug).toContain(expected);
expect(history).toContain("featureConfigCode");
expect(history).toContain("feature-config-input-missing");
expect(JSON.stringify(observation)).not.toContain("process.env");
});
test("collector 遍历全部 matching TaskRun，terminal record 仍限定合同任务", () => {
const remote = readFileSync(resolve(import.meta.dir, "platform-infra-pipelines-as-code-remote.sh"), "utf8");
expect(remote).toContain("for (const item of matching)");
scripts/src/platform-infra-pac-feature-config-projection.test.ts
+35
View File
@@ -0,0 +1,35 @@
import { expect, test } from "bun:test";
import { featureConfigSchemaHistoryFields, renderFeatureConfigSchemaLine } from "./platform-infra-pac-feature-config-projection";
test("status and debug share one bounded feature config line", () => {
const observation = {
warning: true,
blocking: false,
code: "feature-config-value-mismatch",
schemaPath: "config/feature-config.schema.json",
errorCount: 2,
firstError: " /webProbeSentinel required must have required property ",
};
const line = renderFeatureConfigSchemaLine(observation);
expect(line).toBe(" feature-config-schema: warning=true blocking=false code=feature-config-value-mismatch path=config/feature-config.schema.json errors=2 first=/webProbeSentinel required must have required property");
expect(line.length).toBeLessThan(300);
});
test("history uses the same bounded observation fields", () => {
expect(featureConfigSchemaHistoryFields({
warning: true,
blocking: false,
code: "feature-config-schema-missing",
schemaPath: "config/feature-config.schema.json",
errorCount: 1,
firstError: "schema file is missing",
})).toEqual([
["featureConfigWarning", "true"],
["featureConfigBlocking", "false"],
["featureConfigCode", "feature-config-schema-missing"],
["featureConfigSchemaPath", "config/feature-config.schema.json"],
["featureConfigErrorCount", "1"],
["featureConfigFirstError", "schema file is missing"],
]);
});
scripts/src/platform-infra-pac-feature-config-projection.ts
+35
View File
@@ -0,0 +1,35 @@
export function renderFeatureConfigSchemaLine(value: unknown): string {
const featureConfigSchema = record(value);
return ` feature-config-schema: warning=${boolText(featureConfigSchema.warning)} blocking=${boolText(featureConfigSchema.blocking)} code=${stringValue(featureConfigSchema.code)} path=${stringValue(featureConfigSchema.schemaPath)} errors=${stringValue(featureConfigSchema.errorCount)} first=${compactLine(stringValue(featureConfigSchema.firstError))}`;
}
export function featureConfigSchemaHistoryFields(value: unknown): string[][] {
const featureConfigSchema = record(value);
return [
["featureConfigWarning", stringValue(featureConfigSchema.warning)],
["featureConfigBlocking", stringValue(featureConfigSchema.blocking)],
["featureConfigCode", stringValue(featureConfigSchema.code)],
["featureConfigSchemaPath", stringValue(featureConfigSchema.schemaPath)],
["featureConfigErrorCount", stringValue(featureConfigSchema.errorCount)],
["featureConfigFirstError", compactLine(stringValue(featureConfigSchema.firstError))],
];
}
function record(value: unknown): Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value) ? value as Record<string, unknown> : {};
}
function stringValue(value: unknown, fallback = "-"): string {
if (typeof value === "string" && value.length > 0) return value;
if (typeof value === "number" || typeof value === "boolean") return String(value);
return fallback;
}
function boolText(value: unknown): string {
return value === true ? "true" : "false";
}
function compactLine(value: string): string {
const trimmed = value.replace(/\s+/gu, " ").trim();
return trimmed.length > 0 ? trimmed.slice(0, 120) : "-";
}
scripts/src/platform-infra-pipelines-as-code-source-artifact.test.ts
+193 -1
View File
@@ -7,7 +7,7 @@ import { resolve } from "node:path";
import { rootPath } from "./config";
import { applyNodeRuntimeDeployYamlOverlay, nodeRuntimeDeployYamlOverlayShellScript } from "./hwlab-node/deploy-overlay";
import { hwlabRuntimePipelineProvenance } from "./hwlab-node/pipeline-provenance";
import { nodeRuntimePipelineProvenancePostprocessScript } from "./hwlab-node/render";
import { nodeRuntimeFeatureConfigSchemaStage, nodeRuntimePipelinePostprocessScript, nodeRuntimePipelineProvenancePostprocessScript } from "./hwlab-node/render";
import { nodeRuntimeRenderOverlay } from "./hwlab-node/web-probe";
import { hwlabRuntimeLaneSpecForNode, parseHwlabRuntimePipelineProvenance } from "./hwlab-node-lanes";
import { pipelineProvenanceAnnotationKeys, pipelineProvenanceAnnotations } from "./pipeline-provenance";
@@ -35,6 +35,7 @@ import {
observePacSourceArtifactRuntime,
selectPipelineRunBySourceCommit,
} from "../native/cicd/pac-source-artifact-runtime.mjs";
import { renderGitOpsResources } from "../native/cicd/publish-unidesk-host-gitops.mjs";
const sourceCommit = "a".repeat(40);
const provenance = {
@@ -238,6 +239,189 @@ describe("HWLAB YAML-owned Pipeline provenance", () => {
}, ownerPath, configRef)).toThrow("mode must be one of remote-pipeline-annotation");
});
test("runtime GitOps guard keeps Tekton scripts below the owning YAML budget", () => {
const temporary = mkdtempSync(resolve(tmpdir(), "unidesk-hwlab-runtime-gitops-guard-"));
try {
const spec = hwlabRuntimeLaneSpecForNode("v03", "NC01");
const budget = spec.pipelineProvenance?.maxInlineScriptBytes;
expect(budget).toBeNumber();
const overlay = nodeRuntimeRenderOverlay(spec);
const pipelineDir = resolve(temporary, spec.tektonDir);
const pipelinePath = resolve(pipelineDir, "pipeline.yaml");
const configMapPath = resolve(pipelineDir, "runtime-gitops-scripts.yaml");
const gitMirrorDir = resolve(temporary, "devops-infra");
mkdirSync(pipelineDir, { recursive: true });
mkdirSync(gitMirrorDir, { recursive: true });
writeFileSync(resolve(gitMirrorDir, "git-mirror.yaml"), Bun.YAML.stringify({
apiVersion: "v1",
kind: "List",
items: [
{
apiVersion: "v1",
kind: "ConfigMap",
metadata: { name: overlay.gitMirror.syncConfigMapName },
data: {
"sync.sh": "#!/bin/sh\nset -eu\nrepo_url='ssh://git@github.com/pikasTech/HWLAB.git'\n",
"flush.sh": "#!/bin/sh\nset -eu\nremote='ssh://git@github.com/pikasTech/HWLAB.git'\n",
},
},
{
apiVersion: "apps/v1",
kind: "Deployment",
metadata: { name: overlay.gitMirror.serviceReadName },
spec: { template: { spec: { containers: [{ name: "git-mirror", env: [] }] } } },
},
],
}));
writeFileSync(pipelinePath, Bun.YAML.stringify({
apiVersion: "tekton.dev/v1",
kind: "Pipeline",
metadata: { name: spec.pipeline, namespace: "hwlab-ci" },
spec: { tasks: [{
name: "gitops-promote",
taskSpec: { steps: [{
name: "render",
script: [
"if [ -s /workspace/source/affected-services.json ] && node - /workspace/source/affected-services.json <<'NODE'",
"const fs = require(\"node:fs\");",
"const plan = JSON.parse(fs.readFileSync(process.argv[2], \"utf8\"));",
"const buildServices = Array.isArray(plan.buildServices) ? plan.buildServices : [];",
"const affectedServices = Array.isArray(plan.affectedServices) ? plan.affectedServices : [];",
"const willRunGitopsPromote = plan.ciCdPlan && plan.ciCdPlan.willRunGitopsPromote === true;",
"process.exit(!willRunGitopsPromote && buildServices.length === 0 && affectedServices.length === 0 ? 0 : 1);",
"NODE",
"then",
" printf 'false' > /tekton/results/runtime-ready-required || true",
" echo '{\"event\":\"gitops-promote\",\"status\":\"skipped\",\"reason\":\"no-build-no-rollout-plan\"}'",
" exit 0",
"fi",
"node scripts/run-bun.mjs scripts/gitops-render.mjs --use-deploy-images",
"node scripts/run-bun.mjs scripts/gitops-render.mjs --use-deploy-images --check",
"",
].join("\n"),
}] },
}] },
}));
const overlayBase64 = Buffer.from(JSON.stringify(overlay), "utf8").toString("base64");
const postprocessInput = [
"set -eu",
`render_dir=${JSON.stringify(temporary)}`,
`overlay_b64=${JSON.stringify(overlayBase64)}`,
...nodeRuntimePipelinePostprocessScript(nodeRuntimeFeatureConfigSchemaStage(spec)),
].join("\n");
const result = spawnSync("sh", [], {
cwd: rootPath(),
input: postprocessInput,
encoding: "utf8",
timeout: 30_000,
});
expect(result.status, result.stderr).toBe(0);
const guardedPipelineText = readFileSync(pipelinePath, "utf8");
const repeated = spawnSync("node", [
resolve(temporary, ".unidesk-runtime-gitops", "runtime-gitops-pipeline-guard.mjs"),
"--pipeline", pipelinePath,
], {
cwd: rootPath(),
env: { ...process.env, UNIDESK_RUNTIME_GITOPS_OVERLAY_B64: overlayBase64 },
encoding: "utf8",
timeout: 30_000,
});
expect(repeated.status, repeated.stderr).toBe(0);
expect(readFileSync(pipelinePath, "utf8")).toBe(guardedPipelineText);
const pipeline = Bun.YAML.parse(guardedPipelineText) as Record<string, any>;
const scripts = pipeline.spec.tasks.flatMap((task: Record<string, any>) => task.taskSpec.steps.map((step: Record<string, any>) => String(step.script ?? "")));
expect(Math.max(...scripts.map((script: string) => Buffer.byteLength(script)))).toBeLessThanOrEqual(budget as number);
expect(scripts.join("\n")).toContain("process.exit(buildServices.length === 0 && affectedServices.length === 0 ? 0 : 1);");
expect(scripts.join("\n")).not.toContain("!willRunGitopsPromote");
expect(scripts.join("\n")).toContain("no-build-no-rollout-plan-gitops-verify");
expect(scripts.join("\n")).not.toContain('"status":"skipped","reason":"no-build-no-rollout-plan"');
expect(scripts.join("\n")).toContain("UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE=/etc/unidesk-cicd-runtime-gitops/runtime-gitops-overlay.json");
expect(scripts.join("\n")).toContain("export UNIDESK_AJV2020_BUNDLE='/etc/unidesk-cicd-runtime-gitops/ajv2020.min.js'");
expect(scripts.join("\n")).toContain("feature-config-schema-validation");
expect(scripts.join("\n")).toContain("cicd.feature_config.schema");
expect(scripts.join("\n")).toContain("export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT='http://otel-collector.platform-infra.svc.cluster.local:4318/v1/traces'");
expect(scripts.join("\n")).toContain("export OTEL_SERVICE_NAME='unidesk-cicd'");
expect(scripts.join("\n")).toContain("export OTEL_TRACES_SAMPLER_ARG='1'");
expect(scripts.join("\n")).toContain("export OTEL_EXPORTER_TIMEOUT_MS='300'");
expect(scripts.join("\n")).toContain("feature-config-validator-process-failure");
expect(scripts.join("\n").indexOf("scripts/gitops-render.mjs --use-deploy-images"))
.toBeLessThan(scripts.join("\n").indexOf("feature-config-schema-validation"));
expect(scripts.join("\n")).not.toContain("NODE_UNIDESK_RUNTIME_GITOPS_POSTPROCESS");
expect(scripts.join("\n")).not.toContain("NODE_UNIDESK_RUNTIME_GITOPS_VERIFY");
expect(scripts.join("\n")).not.toContain("UNIDESK_RUNTIME_GITOPS_OVERLAY_B64=");
expect(scripts.join("\n")).not.toContain(Buffer.from(JSON.stringify(overlay.observability), "utf8").toString("base64"));
const configMap = Bun.YAML.parse(readFileSync(configMapPath, "utf8")) as Record<string, any>;
expect(configMap.metadata.labels["app.kubernetes.io/managed-by"]).toBe("unidesk-host-gitops");
expect(JSON.parse(configMap.data["runtime-gitops-overlay.json"])).toEqual({ runtimePath: overlay.runtimePath, observability: overlay.observability, codeAgentRuntime: overlay.codeAgentRuntime });
expect(Object.keys(configMap.data).sort()).toEqual([
"ajv2020.min.js",
"feature-config-schema-warning.mjs",
"runtime-gitops-observability.mjs",
"runtime-gitops-overlay.json",
"runtime-gitops-postprocess.mjs",
"runtime-gitops-verify.mjs",
]);
const hostConfig = Bun.YAML.parse(readFileSync(rootPath("config", "unidesk-host-k8s.yaml"), "utf8")) as Record<string, any>;
const runtimePaths = new Set(hostConfig.delivery.changeDetection.runtimePaths as string[]);
for (const inputPath of [
"bun.lock",
"config/hwlab-node-lanes.yaml",
"package.json",
"scripts/native/cicd/feature-config-schema-warning.mjs",
"scripts/native/hwlab/runtime-gitops-observability.mjs",
"scripts/native/hwlab/runtime-gitops-postprocess.mjs",
"scripts/native/hwlab/runtime-gitops-scripts-configmap.mjs",
"scripts/native/hwlab/runtime-gitops-verify.mjs",
"scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js",
"scripts/vendor/ajv-dist/8.17.1/manifest.json",
"scripts/src/config.ts",
"scripts/src/hwlab-node/render.ts",
"scripts/src/hwlab-node-lanes.ts",
"scripts/src/platform-infra-pipelines-as-code.ts",
"scripts/src/yaml-composition.ts",
]) expect(runtimePaths.has(inputPath), inputPath).toBe(true);
expect(hostConfig.delivery.gitops.resources).toContainEqual({
id: "hwlab-nc01-v03-runtime-gitops-scripts",
renderer: "hwlab-runtime-gitops-scripts",
configRef: "config/hwlab-node-lanes.yaml#lanes.v03.targets.NC01",
manifestPath: "deploy/gitops/unidesk-host/hwlab-nc01-v03-runtime-gitops-scripts.yaml",
namespace: "hwlab-ci",
});
const gitopsResources = renderGitOpsResources(hostConfig.delivery.gitops, rootPath());
expect(gitopsResources).toHaveLength(1);
expect(gitopsResources[0]?.path).toBe("deploy/gitops/unidesk-host/hwlab-nc01-v03-runtime-gitops-scripts.yaml");
const gitopsConfigMap = Bun.YAML.parse(gitopsResources[0]?.content ?? "") as Record<string, any>;
expect(gitopsConfigMap.metadata.name).toBe("hwlab-nc01-v03-ci-image-publish-runtime-gitops-scripts");
expect(gitopsConfigMap.metadata.namespace).toBe("hwlab-ci");
expect(gitopsConfigMap.data["runtime-gitops-postprocess.mjs"]).toContain("UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE");
expect(gitopsConfigMap.data["runtime-gitops-postprocess.mjs"].indexOf("UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE"))
.toBeLessThan(gitopsConfigMap.data["runtime-gitops-postprocess.mjs"].indexOf("UNIDESK_RUNTIME_GITOPS_OVERLAY_B64"));
const legacyPayload = Buffer.from(JSON.stringify({ runtimePath: overlay.runtimePath, observability: overlay.observability }), "utf8").toString("base64");
const legacyPipeline = Bun.YAML.parse(readFileSync(pipelinePath, "utf8")) as Record<string, any>;
const legacyStep = legacyPipeline.spec.tasks[0].taskSpec.steps[0];
legacyStep.script = String(legacyStep.script).replaceAll(
"UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE=/etc/unidesk-cicd-runtime-gitops/runtime-gitops-overlay.json",
`UNIDESK_RUNTIME_GITOPS_OVERLAY_B64='${legacyPayload}'`,
);
writeFileSync(pipelinePath, Bun.YAML.stringify(legacyPipeline));
const migrated = spawnSync("node", [
rootPath("scripts", "native", "hwlab", "runtime-gitops-pipeline-guard.mjs"),
"--pipeline", pipelinePath,
], {
cwd: rootPath(),
env: { ...process.env, UNIDESK_RUNTIME_GITOPS_OVERLAY_B64: Buffer.from(JSON.stringify(overlay), "utf8").toString("base64") },
encoding: "utf8",
timeout: 30_000,
});
expect(migrated.status).toBe(0);
const migratedText = readFileSync(pipelinePath, "utf8");
expect(migratedText).toContain("UNIDESK_RUNTIME_GITOPS_OVERLAY_FILE=/etc/unidesk-cicd-runtime-gitops/runtime-gitops-overlay.json");
expect(migratedText).not.toContain(legacyPayload);
} finally {
rmSync(temporary, { recursive: true, force: true });
}
});
test("normal control-plane postprocess writes exactly the shared four provenance fields", () => {
const temporary = mkdtempSync(resolve(tmpdir(), "unidesk-hwlab-pipeline-provenance-"));
try {
@@ -491,7 +675,15 @@ describe("runtime source-commit selection", () => {
configRef: "config/hwlab-node-lanes.yaml#lanes.v03.targets.NC01",
renderer: "hwlab-runtime-lane",
mode: "remote-pipeline-annotation",
maxInlineScriptBytes: 32768,
});
expect(spec.codeAgentRuntime?.kafkaEventBridge).toMatchObject({
enabled: true,
transactionalProjectorConsumerGroupId: "hwlab-v03-agentrun-event-projector",
hwlabEventConsumerGroupId: "hwlab-v03-workbench-live-sse",
});
expect(spec.codeAgentRuntime?.kafkaEventBridge).not.toHaveProperty("features");
expect(spec.codeAgentRuntime?.kafkaEventBridge).not.toHaveProperty("refreshReplay");
expect(remoteAnnotations).toEqual(pipelineProvenanceAnnotations(remoteProvenance));
expect(Object.keys(remoteAnnotations).sort()).toEqual(Object.values(pipelineProvenanceAnnotationKeys).sort());
expect(readFileSync(rootPath("scripts", "src", "hwlab-node", "render.ts"), "utf8")).not.toContain("overlay.sourceArtifactProvenance");
scripts/src/platform-infra-pipelines-as-code-source-artifact.ts
+2 -2
View File
@@ -9,7 +9,7 @@ import { AGENTRUN_CONFIG_PATH, resolveAgentRunLaneTarget } from "./agentrun-lane
import { renderAgentRunPipelineManifest } from "./agentrun-manifests";
import { hwlabRuntimeLaneSpecForNode, isHwlabRuntimeLane, type HwlabRuntimeLaneSpec } from "./hwlab-node-lanes";
import { nodeRuntimeGitopsRoot } from "./hwlab-node/cleanup";
import { nodeRuntimePipelinePostprocessScript } from "./hwlab-node/render";
import { nodeRuntimeFeatureConfigSchemaStage, nodeRuntimePipelinePostprocessScript } from "./hwlab-node/render";
import { nodeRuntimeRenderOverlay } from "./hwlab-node/web-probe";
import { hwlabRuntimePipelineProvenance } from "./hwlab-node/pipeline-provenance";
import { applyNodeRuntimeDeployYamlOverlay } from "./hwlab-node/deploy-overlay";
@@ -617,7 +617,7 @@ function renderHwlabPipelineFromOwningYaml(spec: HwlabRuntimeLaneSpec, sourceWor
"set -eu",
`render_dir=${shellQuote(output)}`,
`overlay_b64=${shellQuote(overlay)}`,
...nodeRuntimePipelinePostprocessScript(),
...nodeRuntimePipelinePostprocessScript(nodeRuntimeFeatureConfigSchemaStage(spec)),
].join("\n");
runChecked("sh", [], temporarySource, 120_000, "HWLAB UniDesk domain postprocess/verify renderer", postprocess);
const path = resolve(output, spec.tektonDir, "pipeline.yaml");
scripts/src/platform-infra-pipelines-as-code.ts
+4
View File
@@ -36,6 +36,7 @@ import {
renderPacBootstrap,
resolvePacBootstrapMirrorRepository,
} from "./platform-infra-pipelines-as-code-bootstrap";
import { featureConfigSchemaHistoryFields, renderFeatureConfigSchemaLine } from "./platform-infra-pac-feature-config-projection";
const configFile = rootPath("config", "platform-infra", "pipelines-as-code.yaml");
const configLabel = "config/platform-infra/pipelines-as-code.yaml";
@@ -2286,6 +2287,7 @@ export function renderStatus(result: Record<string, unknown>): RenderedCliResult
stringValue(deliveryPlan.reusedServiceCount),
stringValue(sourceObservation.reason),
]]),
renderFeatureConfigSchemaLine(sourceObservation.featureConfigSchema),
"",
"TASKRUN DURATIONS",
...(taskRuns.length === 0 ? ["-"] : table(["TASKRUN", "STATUS", "REASON", "FAILED_STEP", "EXIT", "DURATION_S"], taskRuns.map((item) => {
@@ -2503,6 +2505,7 @@ export function renderDebugStep(result: Record<string, unknown>): RenderedCliRes
stringValue(catalog.digestCount),
short(stringValue(artifact.gitopsCommit), 16),
]]),
renderFeatureConfigSchemaLine(record(artifact.sourceObservation).featureConfigSchema),
]),
"",
"STATUS EVALUATOR FIXTURES",
@@ -2925,6 +2928,7 @@ function renderHistoryDetail(row: Record<string, unknown>): string[] {
["rolloutServices", stringValue(deliveryPlan.rolloutServiceCount)],
["buildServices", stringValue(deliveryPlan.buildServiceCount)],
["reusedServices", stringValue(deliveryPlan.reusedServiceCount)],
...featureConfigSchemaHistoryFields(sourceObservation.featureConfigSchema),
]),
"",
"TERMINAL EVIDENCE",
scripts/src/secrets.ts
+203 -19
View File
@@ -40,7 +40,7 @@ interface SecretDistributionConfig {
version: number;
kind: "unidesk-secret-distribution";
metadata: { id: string; owner: string; relatedIssues: number[] };
sources: { root: string; files: EnvSourceFileConfig[]; externalFiles: RawSourceFileConfig[] };
sources: { root: string; files: EnvSourceFileConfig[]; externalFiles: SourceFileConfig[] };
targets: DistributionTarget[];
kubernetesSecrets: KubernetesSecretConfig[];
}
@@ -76,6 +76,11 @@ interface DistributionTarget {
namespace: string;
scope: string;
enabled: boolean;
consumerRollout: {
deployments: string[];
timeoutSeconds: number;
pollIntervalSeconds: number;
} | null;
}
interface KubernetesSecretConfig {
@@ -263,8 +268,9 @@ async function sync(config: UniDeskConfig, options: SecretsOptions): Promise<Rec
mode: "confirmed",
mutation: true,
config: configSummary(distribution, options),
localSources: sourceSummary(sources),
desiredSecrets: desired.map(desiredSecretSummary),
...(options.full
? { localSources: sourceSummary(sources), desiredSecrets: desired.map(desiredSecretSummary) }
: { localSources: compactSourceSummary(sources), desiredSecrets: desired.map(compactDesiredSecretSummary) }),
targets: perTarget,
valuesPrinted: false,
};
@@ -311,7 +317,7 @@ function readSecretDistributionConfig(pathArg: string): SecretDistributionConfig
files: arrayOfRecords(sources.files, `${label}.sources.files`).map((item, index) => parseSourceFile(item, `${label}.sources.files[${index}]`)),
externalFiles: sources.externalFiles === undefined
? []
: arrayOfRecords(sources.externalFiles, `${label}.sources.externalFiles`).map((item, index) => parseRawSourceFile(item, `${label}.sources.externalFiles[${index}]`)),
: arrayOfRecords(sources.externalFiles, `${label}.sources.externalFiles`).map((item, index) => parseExternalSourceFile(item, `${label}.sources.externalFiles[${index}]`)),
},
targets: arrayOfRecords(root.targets, `${label}.targets`).map((item, index) => parseTarget(item, `${label}.targets[${index}]`)),
kubernetesSecrets: arrayOfRecords(root.kubernetesSecrets, `${label}.kubernetesSecrets`).map((item, index) => parseKubernetesSecret(item, `${label}.kubernetesSecrets[${index}]`)),
@@ -320,13 +326,13 @@ function readSecretDistributionConfig(pathArg: string): SecretDistributionConfig
return config;
}
function parseSourceFile(record: Record<string, unknown>, path: string): EnvSourceFileConfig {
function parseSourceFile(record: Record<string, unknown>, path: string, external = false): EnvSourceFileConfig {
const type = stringField(record, "type", path);
if (type !== "env") throw new Error(`${path}.type must be env`);
const createRaw = record.createIfMissing === undefined ? {} : objectField(record, "createIfMissing", path);
const randomBase64UrlRaw = createRaw.randomBase64Url === undefined ? {} : objectField(createRaw, "randomBase64Url", `${path}.createIfMissing`);
return {
sourceRef: sourceRefField(record, "sourceRef", path),
sourceRef: external ? externalSourceRefField(record, "sourceRef", path) : sourceRefField(record, "sourceRef", path),
type,
requiredKeys: stringArrayField(record, "requiredKeys", path).map((key, index) => envKeyValue(key, `${path}.requiredKeys[${index}]`)),
required: true,
@@ -339,15 +345,16 @@ function parseSourceFile(record: Record<string, unknown>, path: string): EnvSour
};
}
function parseRawSourceFile(record: Record<string, unknown>, path: string): RawSourceFileConfig {
function parseExternalSourceFile(record: Record<string, unknown>, path: string): SourceFileConfig {
const type = stringField(record, "type", path);
if (type === "env") return parseSourceFile(record, path, true);
if (type !== "raw-file") throw new Error(`${path}.type must be raw-file`);
const createRaw = objectField(record, "createIfMissing", path);
const enabled = booleanField(createRaw, "enabled", `${path}.createIfMissing`);
const randomHex = createRaw.randomHex === undefined
const randomHex: Record<string, number> = createRaw.randomHex === undefined
? {}
: { contents: randomByteCount(createRaw.randomHex, `${path}.createIfMissing.randomHex`) };
const randomBase64Url = createRaw.randomBase64Url === undefined
const randomBase64Url: Record<string, { bytes: number; prefix: string }> = createRaw.randomBase64Url === undefined
? {}
: { contents: randomBase64UrlSpec(createRaw.randomBase64Url, `${path}.createIfMissing.randomBase64Url`) };
if (createRaw.values !== undefined) throw new Error(`${path}.createIfMissing.values is not allowed for raw-file sources`);
@@ -369,12 +376,22 @@ function parseRawSourceFile(record: Record<string, unknown>, path: string): RawS
}
function parseTarget(record: Record<string, unknown>, path: string): DistributionTarget {
const rollout = record.consumerRollout === undefined ? null : objectField(record, "consumerRollout", path);
const deployments = rollout === null
? []
: stringArrayField(rollout, "deployments", `${path}.consumerRollout`).map((value, index) => kubernetesNameValue(value, `${path}.consumerRollout.deployments[${index}]`));
const timeoutSeconds = rollout === null ? 0 : integerField(rollout, "timeoutSeconds", `${path}.consumerRollout`);
const pollIntervalSeconds = rollout === null ? 0 : integerField(rollout, "pollIntervalSeconds", `${path}.consumerRollout`);
if (rollout !== null && (deployments.length === 0 || timeoutSeconds <= 0 || pollIntervalSeconds <= 0)) {
throw new Error(`${path}.consumerRollout requires deployments, a positive timeoutSeconds, and a positive pollIntervalSeconds`);
}
return {
id: simpleId(stringField(record, "id", path), `${path}.id`),
route: stringField(record, "route", path),
namespace: kubernetesNameField(record, "namespace", path),
scope: simpleId(stringField(record, "scope", path), `${path}.scope`),
enabled: booleanField(record, "enabled", path),
consumerRollout: rollout === null ? null : { deployments, timeoutSeconds, pollIntervalSeconds },
};
}
@@ -414,7 +431,8 @@ function validateDistributionConfig(config: SecretDistributionConfig): void {
for (const item of secret.data) {
const source = sources.get(item.sourceRef);
if (source === undefined) throw new Error(`${config.configPath}.kubernetesSecrets.${secret.name} references undeclared sourceRef ${item.sourceRef}`);
if (!source.requiredKeys.includes(item.sourceKey)) throw new Error(`${config.configPath}.kubernetesSecrets.${secret.name} maps undeclared source key ${item.sourceRef}.${item.sourceKey}`);
const requiredKeys: readonly string[] = source.requiredKeys;
if (!requiredKeys.includes(item.sourceKey)) throw new Error(`${config.configPath}.kubernetesSecrets.${secret.name} maps undeclared source key ${item.sourceRef}.${item.sourceKey}`);
if (targetKeys.has(item.targetKey)) throw new Error(`${config.configPath}.kubernetesSecrets.${secret.name} maps duplicate target key ${item.targetKey}`);
targetKeys.add(item.targetKey);
}
@@ -429,7 +447,7 @@ function inspectSources(config: SecretDistributionConfig, materialize: boolean,
const root = secretRoot(config);
const materials = new Map<string, SourceMaterial>();
const entries = allSourceFiles(config).filter((source) => selectedRefs.has(source.sourceRef)).map((source) => {
const sourcePath = source.type === "env" ? join(root, source.sourceRef) : externalSourcePath(source.sourceRef);
const sourcePath = sourcePathFor(root, source);
const exists = existsSync(sourcePath);
const existingText = exists ? readFileSync(sourcePath, "utf8") : "";
const existing = exists ? source.type === "env" ? parseEnvFile(existingText) : { contents: existingText } : {};
@@ -502,6 +520,8 @@ function inspectSources(config: SecretDistributionConfig, materialize: boolean,
presence,
bytes,
fingerprint,
ownerOnly: true,
fileMode: "0600",
valuesPrinted: false,
};
}
@@ -517,6 +537,8 @@ function inspectSources(config: SecretDistributionConfig, materialize: boolean,
generatedKeys,
unmaterializedGeneratedKeys,
fingerprint: material.fingerprint,
ownerOnly: true,
fileMode: "0600",
valuesPrinted: false,
};
});
@@ -590,10 +612,17 @@ async function applyTargetSecrets(config: UniDeskConfig, target: DistributionTar
const yaml = renderSecretManifest(target, secrets);
const result = await capture(config, target.route, ["sh"], applySecretScript(target, secrets, yaml));
const parsed = parseJsonOutput(result.stdout);
const applied = result.exitCode === 0 && boolField(parsed, "ok", false);
const consumerRolloutStatus = applied && target.consumerRollout !== null
? await waitForConsumerRollout(config, target)
: null;
return {
ok: result.exitCode === 0 && boolField(parsed, "ok", false),
ok: applied && (consumerRolloutStatus === null || consumerRolloutStatus.ok === true),
target: targetSummary(target),
summary: parsed,
summary: {
...parsed,
...(consumerRolloutStatus === null ? {} : { consumerRolloutStatus }),
},
remote: secretCaptureSummary(result),
...(options.raw ? { rawCaptureOmitted: true, rawPolicy: "Secret distribution never returns raw SSH capture because remote output can contain credential-bearing diagnostics." } : {}),
};
@@ -640,6 +669,9 @@ ${Object.entries(secret.data).sort(([a], [b]) => a.localeCompare(b)).map(([key,
function applySecretScript(target: DistributionTarget, secrets: DesiredSecret[], yaml: string): string {
const manifestB64 = Buffer.from(yaml, "utf8").toString("base64");
const summaryB64 = Buffer.from(JSON.stringify(secrets.map(remoteSecretSummary)), "utf8").toString("base64");
const rollout = target.consumerRollout;
const rolloutDeployments = rollout?.deployments.join(" ") ?? "";
const rolloutSummaryB64 = Buffer.from(JSON.stringify(rollout), "utf8").toString("base64");
return `
set -u
tmp="$(mktemp -d)"
@@ -656,22 +688,38 @@ else
printf '%s\\n' 'skipped because namespace sync failed' >"$tmp/apply.err"
apply_rc=1
fi
python3 - "$ns_rc" "$apply_rc" "$tmp/ns.out" "$tmp/ns.err" "$tmp/apply.out" "$tmp/apply.err" <<'PY'
rollout_restart_rc=0
: >"$tmp/rollout-restart.out"
: >"$tmp/rollout-restart.err"
if [ "$apply_rc" -eq 0 ] && [ -n '${rolloutDeployments}' ]; then
for deployment in ${rolloutDeployments}; do
kubectl -n ${target.namespace} rollout restart "deployment/$deployment" >>"$tmp/rollout-restart.out" 2>>"$tmp/rollout-restart.err" || {
rollout_restart_rc=$?
break
}
done
elif [ "$apply_rc" -ne 0 ] && [ -n '${rolloutDeployments}' ]; then
rollout_restart_rc=1
printf '%s\\n' 'skipped because Secret apply failed' >"$tmp/rollout-restart.err"
fi
python3 - "$ns_rc" "$apply_rc" "$rollout_restart_rc" "$tmp/ns.out" "$tmp/ns.err" "$tmp/apply.out" "$tmp/apply.err" "$tmp/rollout-restart.out" "$tmp/rollout-restart.err" <<'PY'
import base64, json, os, sys
ns_rc, apply_rc = int(sys.argv[1]), int(sys.argv[2])
ns_rc, apply_rc, rollout_restart_rc = [int(value) for value in sys.argv[1:4]]
def byte_count(path):
try:
return os.path.getsize(path)
except FileNotFoundError:
return 0
payload = {
"ok": ns_rc == 0 and apply_rc == 0,
"ok": ns_rc == 0 and apply_rc == 0 and rollout_restart_rc == 0,
"namespace": "${target.namespace}",
"secrets": json.loads(base64.b64decode("${summaryB64}").decode("utf-8")),
"consumerRollout": json.loads(base64.b64decode("${rolloutSummaryB64}").decode("utf-8")),
"valuesPrinted": False,
"steps": {
"namespace": {"exitCode": ns_rc, "stdoutBytes": byte_count(sys.argv[3]), "stderrBytes": byte_count(sys.argv[4]), "outputOmitted": True},
"apply": {"exitCode": apply_rc, "stdoutBytes": byte_count(sys.argv[5]), "stderrBytes": byte_count(sys.argv[6]), "outputOmitted": True},
"namespace": {"exitCode": ns_rc, "stdoutBytes": byte_count(sys.argv[4]), "stderrBytes": byte_count(sys.argv[5]), "outputOmitted": True},
"apply": {"exitCode": apply_rc, "stdoutBytes": byte_count(sys.argv[6]), "stderrBytes": byte_count(sys.argv[7]), "outputOmitted": True},
"consumerRolloutRestart": {"exitCode": rollout_restart_rc, "stdoutBytes": byte_count(sys.argv[8]), "stderrBytes": byte_count(sys.argv[9]), "outputOmitted": True},
},
}
print(json.dumps(payload, ensure_ascii=False, indent=2))
@@ -680,6 +728,105 @@ PY
`;
}
async function waitForConsumerRollout(config: UniDeskConfig, target: DistributionTarget): Promise<Record<string, unknown>> {
const rollout = target.consumerRollout;
if (rollout === null) return { ok: true, mode: "not-declared", mutation: false };
const startedAt = Date.now();
const deadline = startedAt + rollout.timeoutSeconds * 1000;
let attempts = 0;
let lastSummary: Record<string, unknown> | null = null;
let lastRemote: Record<string, unknown> | null = null;
while (true) {
attempts += 1;
const result = await capture(config, target.route, ["sh"], consumerRolloutStatusScript(target));
const parsed = parseJsonOutput(result.stdout);
lastSummary = parsed;
lastRemote = secretCaptureSummary(result);
if (result.exitCode === 0 && boolField(parsed, "ok", false)) {
return {
ok: true,
mutation: false,
attempts,
elapsedSeconds: Math.ceil((Date.now() - startedAt) / 1000),
summary: parsed,
remote: lastRemote,
valuesPrinted: false,
};
}
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) {
return {
ok: false,
mutation: false,
attempts,
elapsedSeconds: Math.ceil((Date.now() - startedAt) / 1000),
reason: "consumer-rollout-timeout",
summary: lastSummary,
remote: lastRemote,
valuesPrinted: false,
};
}
await Bun.sleep(Math.min(rollout.pollIntervalSeconds * 1000, remainingMs));
}
}
function consumerRolloutStatusScript(target: DistributionTarget): string {
const rollout = target.consumerRollout;
if (rollout === null) throw new Error(`target ${target.id} does not declare consumerRollout`);
const commands = rollout.deployments.map((deployment, index) => [
`kubectl -n ${target.namespace} get deployment ${deployment} -o json >"$tmp/deployment.${index}.json" 2>"$tmp/deployment.${index}.err"`,
`printf '%s' "$?" >"$tmp/deployment.${index}.rc"`,
].join("\n")).join("\n");
const expectedB64 = Buffer.from(JSON.stringify(rollout.deployments), "utf8").toString("base64");
return `
set -u
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
${commands}
python3 - "$tmp" <<'PY'
import base64, json, os, sys
tmp = sys.argv[1]
expected = json.loads(base64.b64decode("${expectedB64}").decode("utf-8"))
items = []
ok = True
for index, name in enumerate(expected):
try:
rc = int(open(os.path.join(tmp, f"deployment.{index}.rc"), encoding="utf-8").read() or "1")
except FileNotFoundError:
rc = 1
try:
observed = json.load(open(os.path.join(tmp, f"deployment.{index}.json"), encoding="utf-8"))
except Exception:
observed = None
metadata = (observed or {}).get("metadata") or {}
spec = (observed or {}).get("spec") or {}
status = (observed or {}).get("status") or {}
generation = int(metadata.get("generation") or 0)
observed_generation = int(status.get("observedGeneration") or 0)
desired = int(spec.get("replicas") or 0)
updated = int(status.get("updatedReplicas") or 0)
ready = int(status.get("readyReplicas") or 0)
available = int(status.get("availableReplicas") or 0)
item_ok = rc == 0 and desired > 0 and observed_generation >= generation and updated >= desired and ready >= desired and available >= desired
ok = ok and item_ok
items.append({
"name": name,
"exists": rc == 0,
"generation": generation,
"observedGeneration": observed_generation,
"desiredReplicas": desired,
"updatedReplicas": updated,
"readyReplicas": ready,
"availableReplicas": available,
"ready": item_ok,
})
payload = {"ok": ok, "namespace": "${target.namespace}", "deployments": items, "valuesPrinted": False}
print(json.dumps(payload, ensure_ascii=False, indent=2))
sys.exit(0 if ok else 1)
PY
`;
}
function statusSecretScript(target: DistributionTarget, secrets: DesiredSecret[]): string {
const summaryB64 = Buffer.from(JSON.stringify(secrets.map(remoteSecretSummary)), "utf8").toString("base64");
const commands = secrets.map((secret, index) => [
@@ -761,6 +908,22 @@ function sourceSummary(sources: SourceInspection): Record<string, unknown> {
return { ok: sources.ok, root: sources.root, entries: sources.entries, valuesPrinted: false };
}
function compactSourceSummary(sources: SourceInspection): Record<string, unknown> {
return {
ok: sources.ok,
entries: sources.entries.map((entry) => ({
sourceRef: entry.sourceRef,
type: entry.type,
presence: entry.presence ?? entry.exists,
missingKeys: entry.missingKeys ?? [],
action: entry.action ?? "none",
fingerprint: entry.fingerprint ?? null,
valuesPrinted: false,
})),
valuesPrinted: false,
};
}
function desiredSecretSummary(secret: DesiredSecret): Record<string, unknown> {
return {
name: secret.name,
@@ -775,6 +938,17 @@ function desiredSecretSummary(secret: DesiredSecret): Record<string, unknown> {
};
}
function compactDesiredSecretSummary(secret: DesiredSecret): Record<string, unknown> {
return {
name: secret.name,
secretName: secret.secretName,
keys: Object.keys(secret.data).sort(),
missingKeys: secret.missingKeys,
fingerprint: secret.fingerprint,
valuesPrinted: false,
};
}
function remoteSecretSummary(secret: DesiredSecret): Record<string, unknown> {
return {
name: secret.name,
@@ -786,7 +960,7 @@ function remoteSecretSummary(secret: DesiredSecret): Record<string, unknown> {
}
function targetSummary(target: DistributionTarget): Record<string, unknown> {
return { id: target.id, route: target.route, namespace: target.namespace, scope: target.scope };
return { id: target.id, route: target.route, namespace: target.namespace, scope: target.scope, consumerRollout: target.consumerRollout };
}
function readOptionValue(args: string[], index: number, option: string): string {
@@ -967,6 +1141,12 @@ function externalSourcePath(sourceRef: string): string {
throw new Error(`external sourceRef ${sourceRef} must be an absolute path or a ~/ home path`);
}
function sourcePathFor(root: string, source: SourceFileConfig): string {
return source.sourceRef.startsWith("~/") || isAbsolute(source.sourceRef)
? externalSourcePath(source.sourceRef)
: join(root, source.sourceRef);
}
function sourceDataKeyField(obj: Record<string, unknown>, key: string, path: string): string {
const value = stringField(obj, key, path);
if (!/^[A-Za-z_][A-Za-z0-9._-]*$/u.test(value)) throw new Error(`${path}.${key} must be a source data key`);
@@ -982,6 +1162,10 @@ function kubernetesNameField(obj: Record<string, unknown>, key: string, path: st
return yamlKubernetesNameField(obj, key, path, "");
}
function kubernetesNameValue(value: string, path: string): string {
return yamlKubernetesNameField({ value }, "value", path, "");
}
function kubernetesSecretKeyField(obj: Record<string, unknown>, key: string, path: string): string {
const value = stringField(obj, key, path);
if (!/^[A-Za-z0-9._-]+$/u.test(value)) throw new Error(`${path}.${key} must be a Kubernetes Secret key`);
scripts/src/selfmedia-config.ts
+24
View File
@@ -158,6 +158,7 @@ function parseDeliveryTarget(id: string, value: Record<string, unknown>): SelfMe
if (boolean(ci.serviceAccountAutomount, `${path}.ci.serviceAccountAutomount`) !== false) throw new Error(`${path}.ci.serviceAccountAutomount must be false`);
if (boolean(publicExposure.enabled, `${path}.deployment.publicExposure.enabled`) !== true) throw new Error(`${path}.deployment.publicExposure.enabled must be true`);
if (integer(publicExposure.hostPort, `${path}.deployment.publicExposure.hostPort`) !== 4317) throw new Error(`${path}.deployment.publicExposure.hostPort must be 4317`);
validateAccessSecret(deployment, `${path}.deployment`);
const gitopsWriteUrl = text(gitops.writeUrl, `${path}.gitops.writeUrl`);
if (!gitopsWriteUrl.startsWith("http://gitea-http.")) throw new Error(`${path}.gitops.writeUrl must use internal Gitea authority`);
return {
@@ -206,6 +207,29 @@ function parseDeliveryTarget(id: string, value: Record<string, unknown>): SelfMe
};
}
function validateAccessSecret(deployment: Record<string, unknown>, path: string): void {
const secrets = record(deployment.secrets, `${path}.secrets`);
const access = record(secrets.access, `${path}.secrets.access`);
const target = record(access.target, `${path}.secrets.access.target`);
absolutePath(access.sourceRef, `${path}.secrets.access.sourceRef`);
kubernetesName(target.secretName, `${path}.secrets.access.target.secretName`);
if (boolean(target.readOnly, `${path}.secrets.access.target.readOnly`) !== true) {
throw new Error(`${selfMediaConfigRef}.${path}.secrets.access.target.readOnly must be true`);
}
if (!Array.isArray(target.files) || target.files.length === 0) throw new Error(`${selfMediaConfigRef}.${path}.secrets.access.target.files must be a non-empty array`);
const targetKeys = new Set<string>();
const mountPaths = new Set<string>();
target.files.forEach((value, index) => {
const file = record(value, `${path}.secrets.access.target.files[${index}]`);
const targetKey = text(file.targetKey, `${path}.secrets.access.target.files[${index}].targetKey`);
const mountPath = absolutePath(file.mountPath, `${path}.secrets.access.target.files[${index}].mountPath`);
if (targetKeys.has(targetKey)) throw new Error(`${selfMediaConfigRef}.${path}.secrets.access.target.files contains duplicate targetKey ${targetKey}`);
if (mountPaths.has(mountPath)) throw new Error(`${selfMediaConfigRef}.${path}.secrets.access.target.files contains duplicate mountPath ${mountPath}`);
targetKeys.add(targetKey);
mountPaths.add(mountPath);
});
}
function parseCutoverTarget(id: string, value: Record<string, unknown>): SelfMediaCutoverTarget {
const path = `cutover.targets.${id}`;
const source = record(value.source, `${path}.source`);
scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js
Vendored
+9
View File
File diff suppressed because one or more lines are too long
scripts/vendor/ajv-dist/8.17.1/manifest.json
Vendored
+8
View File
@@ -0,0 +1,8 @@
{
"package": "ajv-dist",
"version": "8.17.1",
"sourcePath": "node_modules/ajv-dist/dist/ajv2020.min.js",
"vendorPath": "scripts/vendor/ajv-dist/8.17.1/ajv2020.min.js",
"sha256": "d2f97a24636c44135e1ffffeea29656c06a1c1f71b315e958c88d0e8126c2100",
"bytes": 132117
}