diff --git a/docs/reference/artifact-registry.md b/docs/reference/artifact-registry.md index 4d0d4f51..77538094 100644 --- a/docs/reference/artifact-registry.md +++ b/docs/reference/artifact-registry.md @@ -104,7 +104,7 @@ bun scripts/cli.ts artifact-registry deploy-service --env prod --service claudeq bun scripts/cli.ts artifact-registry deploy-service --env dev --service code-queue --commit --run-now ``` -dry-run 输出会暴露 registry probe URL、required labels、目标 image、部署形态、目标 Deployment 列表和回滚信息。`baidu-netdisk` 的 Compose 路径会通过 provider-gateway Host SSH 把 `unidesk/baidu-netdisk:` 流式拉到 master server，retag 为 `baidu-netdisk` 和 `baidu-netdisk:`，写入 `UNIDESK_BAIDU_NETDISK_DEPLOY_*`，只 recreate `baidu-netdisk` service，并验证容器 image label 与 `/health.deploy.commit`。`findjob`、`pipeline` 和 `met-nonlinear` 的 D601 direct Compose 路径在 D601 本机验证 registry manifest、pull image、retag stable image、写入 `UNIDESK_*_DEPLOY_*` labels/env，并用 `docker compose up -d --no-build --no-deps --force-recreate ` 重新拉起对应 compose service；其中 `met-nonlinear` 当前因为 registered Dockerfile 和 long-running service contract 不一致而 live deploy blocked。`k3sctl-adapter` 是基础设施控制桥，只做 plan/dry-run，真实生产部署需要 supervisor 单独确认。`frontend --env prod` 使用同一 Compose artifact consumer，流式拉取 `unidesk/frontend:`，retag 为 `unidesk-frontend` 和 `unidesk-frontend:`，写入 `UNIDESK_FRONTEND_DEPLOY_*`，只 recreate `frontend` service，并验证 image label 与 `/health.deploy.commit`。`frontend --env dev`、`decision-center`、`mdtodo`、`claudeqq` 和 dev `code-queue` 的 k3s 路径会在 D601 上验证 commit image、导入 native k3s containerd、更新 Deployment image/env/annotations，并通过 Kubernetes API service proxy 验证 `/health` 中的 live commit 与 requested commit；dev frontend 还会在 rollout 前把主 server `config.json.auth` 同步到 `unidesk-dev` Secret/ConfigMap。`decision-center --env dev` 落到 `unidesk-dev/decision-center-dev`，prod 落到 `unidesk/decision-center`；`mdtodo` 和 `claudeqq` 使用同样的 dev 后 prod k3s consumer 结构。`code-queue --env dev` 只更新 `unidesk-dev` 中的 scheduler/read/write/provider-egress-proxy dev Deployments，prod 没有 consumer target。D601 direct Compose consumer 与 k3s-managed consumer 的区别是：前者只接触 D601 Docker/Compose 项目和私有 backend health，不创建 Kubernetes 对象；后者只通过 native k3s Deployment/Service、containerd import 和 Kubernetes API service proxy 验证 live commit。回滚信息通过同一 artifact consumer 的 `rollback` 字段暴露，提示操作者重新对一个旧 commit 运行相同命令，而不是切回 legacy maintenance-channel 构建。 +dry-run 输出会暴露 registry probe URL、required labels、目标 image、部署形态、目标 Deployment 列表和回滚信息，但不得读取或打印运行时密钥。`baidu-netdisk` 是 PGDATA 备份链路依赖服务；它的 Compose artifact 路径会通过 provider-gateway Host SSH 把 `unidesk/baidu-netdisk:` 流式拉到 master server，retag 为 `baidu-netdisk` 和 `baidu-netdisk:`，在 canonical UniDesk 根目录使用 `providerGateway.upgrade.composeEnvFile` 指向的受控 env 文件写入 `UNIDESK_BAIDU_NETDISK_DEPLOY_*`，只 recreate `baidu-netdisk` service，并验证容器 image label 与 `/health.deploy.commit`。live apply 在 recreate 前必须确认受控 env 文件中存在 `UNIDESK_BAIDU_NETDISK_CLIENT_ID`、`UNIDESK_BAIDU_NETDISK_CLIENT_SECRET` 和 `UNIDESK_BAIDU_NETDISK_TOKEN_KEY`，输出只能包含 present/length/boolean；recreate 后必须验收 `/health.auth.configured`、`clientIdConfigured`、`clientSecretConfigured`、`tokenKeyConfigured` 和 `loggedIn` 全部为 true，否则返回失败或 degraded，并提示先恢复 env、单服务 recreate、再验证 `microservice health baidu-netdisk`。`findjob`、`pipeline` 和 `met-nonlinear` 的 D601 direct Compose 路径在 D601 本机验证 registry manifest、pull image、retag stable image、写入 `UNIDESK_*_DEPLOY_*` labels/env，并用 `docker compose up -d --no-build --no-deps --force-recreate ` 重新拉起对应 compose service；其中 `met-nonlinear` 当前因为 registered Dockerfile 和 long-running service contract 不一致而 live deploy blocked。`k3sctl-adapter` 是基础设施控制桥，只做 plan/dry-run，真实生产部署需要 supervisor 单独确认。`frontend --env prod` 使用同一 Compose artifact consumer，流式拉取 `unidesk/frontend:`，retag 为 `unidesk-frontend` 和 `unidesk-frontend:`，写入 `UNIDESK_FRONTEND_DEPLOY_*`，只 recreate `frontend` service，并验证 image label 与 `/health.deploy.commit`。`frontend --env dev`、`decision-center`、`mdtodo`、`claudeqq` 和 dev `code-queue` 的 k3s 路径会在 D601 上验证 commit image、导入 native k3s containerd、更新 Deployment image/env/annotations，并通过 Kubernetes API service proxy 验证 `/health` 中的 live commit 与 requested commit；dev frontend 还会在 rollout 前把主 server `config.json.auth` 同步到 `unidesk-dev` Secret/ConfigMap。`decision-center --env dev` 落到 `unidesk-dev/decision-center-dev`，prod 落到 `unidesk/decision-center`；`mdtodo` 和 `claudeqq` 使用同样的 dev 后 prod k3s consumer 结构。`code-queue --env dev` 只更新 `unidesk-dev` 中的 scheduler/read/write/provider-egress-proxy dev Deployments，prod 没有 consumer target。D601 direct Compose consumer 与 k3s-managed consumer 的区别是：前者只接触 D601 Docker/Compose 项目和私有 backend health，不创建 Kubernetes 对象；后者只通过 native k3s Deployment/Service、containerd import 和 Kubernetes API service proxy 验证 live commit。回滚信息通过同一 artifact consumer 的 `rollback` 字段暴露，提示操作者重新对一个旧 commit 运行相同命令，而不是切回 legacy maintenance-channel 构建。 `status` 和 `health` 通过： diff --git a/docs/reference/deploy.md b/docs/reference/deploy.md index 71fd2105..15dfbd7a 100644 --- a/docs/reference/deploy.md +++ b/docs/reference/deploy.md @@ -193,7 +193,7 @@ Code Queue health and diagnostics must cover its k3s dependencies, not only sche Existing service-specific commands such as Code Queue deploy are disabled as direct D601 deploy paths. Their build/import/rollout semantics should converge later into one controlled target-side deployment path instead of keeping parallel implementations. -Baidu Netdisk is the main-server `unidesk-direct` sample for artifact CD. Controlled dev validation and prod CD use the D601 registry artifact consumer: it verifies `unidesk/baidu-netdisk:` exists in the registry, streams the image to the main server through provider-gateway Host SSH, retags `baidu-netdisk` and `baidu-netdisk:`, stamps `UNIDESK_BAIDU_NETDISK_DEPLOY_*`, recreates only Compose service `baidu-netdisk`, and verifies container health, image labels, service id, and `/health.deploy.commit`. It must not use `server rebuild baidu-netdisk`, mutable tags, dirty worktrees, hand-built images, or public `4244` exposure as deployment truth. +Baidu Netdisk is the main-server `unidesk-direct` sample for artifact CD and a dependency of the PGDATA-to-Baidu-Netdisk backup path. Controlled dev validation and prod CD use the D601 registry artifact consumer: it verifies `unidesk/baidu-netdisk:` exists in the registry, streams the image to the main server through provider-gateway Host SSH, retags `baidu-netdisk` and `baidu-netdisk:`, stamps `UNIDESK_BAIDU_NETDISK_DEPLOY_*` in the canonical Compose env file, recreates only Compose service `baidu-netdisk`, and verifies container health, image labels, service id, `/health.deploy.commit`, and `/health.auth`. Live apply must fail or return degraded before success if `UNIDESK_BAIDU_NETDISK_CLIENT_ID`, `UNIDESK_BAIDU_NETDISK_CLIENT_SECRET`, or `UNIDESK_BAIDU_NETDISK_TOKEN_KEY` is absent from the controlled env source, or if `/health.auth.configured`, `clientIdConfigured`, `clientSecretConfigured`, `tokenKeyConfigured`, or `loggedIn` is not true after recreate. Dry-run only reports that these secret presences and auth fields are required and pending live check; it must not read or print secret values. It must not use `server rebuild baidu-netdisk`, mutable tags, dirty worktrees, hand-built images, or public `4244` exposure as deployment truth. Decision Center is a standard `k3sctl-managed` service in this model, but D601 maintenance-channel direct apply must not deploy it. Controlled CD for Decision Center uses the D601 registry artifact consumer in both dev and prod: it verifies `unidesk/decision-center:` exists in the registry, imports `unidesk-decision-center:` into native k3s containerd, applies the appropriate Decision Center manifest, stamps the Deployment, and verifies health through `/api/microservices/decision-center/health` while proving the live and requested commit match. It must not add a main-server Compose service, NodePort, hostPort, or provider-gateway direct HTTP backend for Decision Center. diff --git a/docs/reference/microservices.md b/docs/reference/microservices.md index 9ffb88b0..f5e51ba9 100644 --- a/docs/reference/microservices.md +++ b/docs/reference/microservices.md @@ -122,8 +122,9 @@ Project Manager 的标准发布是 `bun scripts/cli.ts ci publish-user-service - - Provider：`main-server`，由 backend-core 直接访问同一 Compose 网络内的 `http://baidu-netdisk:4244`，公网不发布 `4244`。 - 代码引用：`https://github.com/pikasTech/unidesk` 与配置中的 `repository.commitId`；服务源码位于 `src/components/microservices/baidu-netdisk`，属于 UniDesk 自有主 server 用户服务。 - 部署引用：UniDesk 根仓库 `docker-compose.yml` 中的 `baidu-netdisk` service，Dockerfile 为 `src/components/microservices/baidu-netdisk/Dockerfile`，容器名为 `baidu-netdisk-backend`。 -- 标准发布：`baidu-netdisk` 是主 server 直管微服务的镜像化样板。CI 使用 `bun scripts/cli.ts ci publish-user-service --service baidu-netdisk --commit ` 在 D601 registry 发布 `127.0.0.1:5000/unidesk/baidu-netdisk:`；dev 验证使用 `bun scripts/cli.ts deploy apply --env dev --service baidu-netdisk`；prod 发布使用 `bun scripts/cli.ts deploy apply --env prod --service baidu-netdisk`。两条 CD 都必须消费同一类 commit-pinned artifact、只 recreate `baidu-netdisk-backend`、并验证 image label 与 `/health.deploy.commit`；`server rebuild baidu-netdisk` 只保留为维护/非标准路径。 +- 标准发布：`baidu-netdisk` 是主 server 直管微服务的镜像化样板，也是 PGDATA 到百度网盘日备份链路依赖服务。CI 使用 `bun scripts/cli.ts ci publish-user-service --service baidu-netdisk --commit ` 在 D601 registry 发布 `127.0.0.1:5000/unidesk/baidu-netdisk:`；dev 验证使用 `bun scripts/cli.ts deploy apply --env dev --service baidu-netdisk`；prod 发布使用 `bun scripts/cli.ts deploy apply --env prod --service baidu-netdisk`。两条 CD 都必须消费同一类 commit-pinned artifact、只 recreate `baidu-netdisk-backend`、并验证 image label、`/health.deploy.commit` 和 `/health.auth` 门禁；`server rebuild baidu-netdisk` 只保留为维护/非标准路径。 - 配置密钥：Compose 只透传 `UNIDESK_BAIDU_NETDISK_CLIENT_ID`、`UNIDESK_BAIDU_NETDISK_CLIENT_SECRET`、`UNIDESK_BAIDU_NETDISK_TOKEN_KEY` 与可选 `UNIDESK_BAIDU_NETDISK_APP_ROOT`；当前默认工作根目录为 `/`，如需收回到应用目录可显式设为 `/apps/`；不得把百度 AppSecret、token key、access token 或 refresh token 写入仓库文件。 +- artifact CD 密钥与健康门禁：dev/prod `deploy apply` 或 `artifact-registry deploy-service` 在 live apply 前必须从 canonical `.state/docker-compose.env` 确认 `UNIDESK_BAIDU_NETDISK_CLIENT_ID`、`UNIDESK_BAIDU_NETDISK_CLIENT_SECRET`、`UNIDESK_BAIDU_NETDISK_TOKEN_KEY` 存在，只能输出 present/length/boolean，不得打印值。单服务 recreate 后必须确认 `/health.auth.configured`、`clientIdConfigured`、`clientSecretConfigured`、`tokenKeyConfigured`、`loggedIn` 全部为 true；否则发布不能视为成功，需先恢复 env 注入并重新执行受控单服务 recreate。 - 配置步骤：`UNIDESK_BAIDU_NETDISK_TOKEN_KEY` 可由本机生成；百度 `client_id` 和 `client_secret` 必须由账号拥有者在百度网盘开放平台创建应用后提供，操作清单见 `docs/issue/baidu-netdisk-env-setup.md`。该环境配置说明只用于密钥、登录和维护验证；按 commit 发布或恢复 desired state 时仍必须使用上面的 CI artifact 加 dev/prod `deploy apply` 路径，不能把本地 `server rebuild baidu-netdisk` 当作镜像化交付证据。 - 数据库：OAuth 设备码会话、账号摘要、加密 token、传输任务和事件写入主 PostgreSQL 表 `baidu_netdisk_*`；服务启动时自动创建/补齐 schema，不依赖仅首次生效的 database init SQL。 - 文件边界：v1 只支持容器 staging 目录 `/data/staging` 与百度网盘配置工作根之间的后台上传/下载任务；staging 目录由主 server `.state/baidu-netdisk/staging` 挂载，`.state/` 只保存可重建文件缓存，不作为 token 或任务权威状态。当前授权账号已实测可对百度网盘根目录 `/` 执行列表、上传、获取 dlink、下载和删除临时探针，因此 `UNIDESK_BAIDU_NETDISK_APP_ROOT` 默认直接设为 `/`；仅当该值配置为 `/apps/...` 时，后端才会确保应用目录存在，目录已存在时必须返回/记录 `errno=-8` 并继续，禁止使用会重命名的策略重复创建 `_YYYYMMDD_...` 目录。 diff --git a/scripts/baidu-netdisk-artifact-guard-contract-test.ts b/scripts/baidu-netdisk-artifact-guard-contract-test.ts new file mode 100644 index 00000000..667d3237 --- /dev/null +++ b/scripts/baidu-netdisk-artifact-guard-contract-test.ts @@ -0,0 +1,70 @@ +import { + baiduNetdiskAuthHealthGateStatus, + baiduNetdiskRuntimeSecretRequirements, + runtimeSecretPresenceFromEnvText, +} from "./src/artifact-registry"; + +function assertCondition(condition: unknown, message: string, detail: unknown = {}): void { + if (!condition) throw new Error(`${message}: ${JSON.stringify(detail)}`); +} + +const secretEnvText = [ + "UNIDESK_BAIDU_NETDISK_CLIENT_ID=clientid-clientid-clientid-0000", + "UNIDESK_BAIDU_NETDISK_CLIENT_SECRET=\"clientsecret-clientsecret-0001\"", + "UNIDESK_BAIDU_NETDISK_TOKEN_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", +].join("\n"); + +const present = runtimeSecretPresenceFromEnvText(secretEnvText, baiduNetdiskRuntimeSecretRequirements); +assertCondition(present.every((item) => item.present), "all baidu-netdisk secrets should be present", present); +assertCondition(present.map((item) => item.length).join(",") === "31,30,64", "presence reports only lengths", present); +assertCondition(!JSON.stringify(present).includes("0123456789abcdef"), "secret values must not be exposed", present); + +const missing = runtimeSecretPresenceFromEnvText( + "UNIDESK_BAIDU_NETDISK_CLIENT_ID=clientid-clientid-clientid-0000\n", + baiduNetdiskRuntimeSecretRequirements, +); +assertCondition(!missing.every((item) => item.present), "missing env should fail presence contract", missing); +assertCondition( + missing.filter((item) => !item.present).map((item) => item.sourceEnvName).join(",") === "UNIDESK_BAIDU_NETDISK_CLIENT_SECRET,UNIDESK_BAIDU_NETDISK_TOKEN_KEY", + "missing env should identify absent keys without values", + missing, +); + +const healthy = baiduNetdiskAuthHealthGateStatus({ + auth: { + configured: true, + clientIdConfigured: true, + clientSecretConfigured: true, + tokenKeyConfigured: true, + loggedIn: true, + }, +}); +assertCondition(healthy.ok, "healthy auth fields should pass", healthy); + +const degraded = baiduNetdiskAuthHealthGateStatus({ + auth: { + configured: false, + clientIdConfigured: true, + clientSecretConfigured: true, + tokenKeyConfigured: false, + loggedIn: true, + }, +}); +assertCondition(!degraded.ok, "missing auth fields should fail", degraded); +assertCondition( + degraded.failedFields.join(",") === "configured,tokenKeyConfigured", + "auth gate should report failed boolean fields", + degraded, +); + +process.stdout.write(`${JSON.stringify({ + ok: true, + checks: [ + "runtime secret presence reports booleans and lengths only", + "missing Baidu Netdisk env cannot pass the deploy contract", + "auth health gate requires configured/clientId/clientSecret/tokenKey/loggedIn", + ], + present, + missing: missing.map((item) => ({ ...item, length: item.length })), + degraded, +}, null, 2)}\n`); diff --git a/scripts/src/artifact-registry.ts b/scripts/src/artifact-registry.ts index a5f5744b..4db093d9 100644 --- a/scripts/src/artifact-registry.ts +++ b/scripts/src/artifact-registry.ts @@ -1,8 +1,8 @@ import { createHash } from "node:crypto"; -import { readFileSync, writeFileSync } from "node:fs"; +import { existsSync, readFileSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; import { runCommand, type CommandResult } from "./command"; import { readConfig, type UniDeskConfig, repoRoot, rootPath } from "./config"; -import { resolveComposeCommand, writeComposeEnv } from "./docker"; import { startJob } from "./jobs"; type ArtifactRegistryAction = "plan" | "render" | "status" | "health" | "install" | "deploy-backend-core" | "deploy-service"; @@ -118,6 +118,8 @@ interface ArtifactConsumerTarget { workDir?: string; composeFile?: string; projectHint?: string; + requiredRuntimeSecrets?: RuntimeSecretRequirement[]; + authHealthGate?: AuthHealthGate; }; k3s?: { namespace: string; @@ -133,10 +135,46 @@ interface ArtifactConsumerTarget { }; } +export interface RuntimeSecretRequirement { + sourceEnvName: string; + containerEnvName: string; +} + +interface AuthHealthGate { + requiredFields: string[]; + recoveryHint: string; +} + +interface ComposeArtifactRuntime { + workDir: string; + composeFile: string; + envFile: string; + project: string; + command: string[]; +} + +export interface RuntimeSecretPresence { + sourceEnvName: string; + containerEnvName: string; + present: boolean; + length: number; +} + function todoNoteHealthProbeCommand(): string { return "bun -e \"fetch('http://127.0.0.1:4211/api/health').then(async r=>{const text=await r.text(); let body; try{body=JSON.parse(text)}catch{body={ok:r.ok,raw:text}}; const deploy=body.deploy&&typeof body.deploy==='object'&&!Array.isArray(body.deploy)?body.deploy:{}; body.deploy={...deploy,serviceId:deploy.serviceId||process.env.UNIDESK_DEPLOY_SERVICE_ID||'todo-note',ref:deploy.ref||process.env.UNIDESK_DEPLOY_REF||'',repo:deploy.repo||process.env.UNIDESK_DEPLOY_REPO||'',commit:deploy.commit||process.env.UNIDESK_DEPLOY_COMMIT||'',requestedCommit:deploy.requestedCommit||process.env.UNIDESK_DEPLOY_REQUESTED_COMMIT||''}; console.log(JSON.stringify(body)); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\""; } +export const baiduNetdiskRuntimeSecretRequirements: RuntimeSecretRequirement[] = [ + { sourceEnvName: "UNIDESK_BAIDU_NETDISK_CLIENT_ID", containerEnvName: "BAIDU_NETDISK_CLIENT_ID" }, + { sourceEnvName: "UNIDESK_BAIDU_NETDISK_CLIENT_SECRET", containerEnvName: "BAIDU_NETDISK_CLIENT_SECRET" }, + { sourceEnvName: "UNIDESK_BAIDU_NETDISK_TOKEN_KEY", containerEnvName: "BAIDU_NETDISK_TOKEN_KEY" }, +]; + +const baiduNetdiskAuthHealthGate: AuthHealthGate = { + requiredFields: ["configured", "clientIdConfigured", "clientSecretConfigured", "tokenKeyConfigured", "loggedIn"], + recoveryHint: "Restore UNIDESK_BAIDU_NETDISK_CLIENT_ID, UNIDESK_BAIDU_NETDISK_CLIENT_SECRET, and UNIDESK_BAIDU_NETDISK_TOKEN_KEY in the canonical .state/docker-compose.env, recreate only baidu-netdisk with the canonical env file, then verify microservice health baidu-netdisk.", +}; + const artifactConsumerSpecs: Record = { "backend-core": { serviceId: "backend-core", @@ -178,6 +216,8 @@ const artifactConsumerSpecs: Record = { deployEnvPrefix: "UNIDESK_BAIDU_NETDISK_DEPLOY", healthProbeCommand: "bun -e \"fetch('http://127.0.0.1:4244/health').then(async r=>{const text=await r.text(); console.log(text); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\"", requireHealthCommit: true, + requiredRuntimeSecrets: baiduNetdiskRuntimeSecretRequirements, + authHealthGate: baiduNetdiskAuthHealthGate, }, }, prod: { @@ -190,6 +230,8 @@ const artifactConsumerSpecs: Record = { deployEnvPrefix: "UNIDESK_BAIDU_NETDISK_DEPLOY", healthProbeCommand: "bun -e \"fetch('http://127.0.0.1:4244/health').then(async r=>{const text=await r.text(); console.log(text); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\"", requireHealthCommit: true, + requiredRuntimeSecrets: baiduNetdiskRuntimeSecretRequirements, + authHealthGate: baiduNetdiskAuthHealthGate, }, }, }, @@ -1347,6 +1389,95 @@ function composeLockScript(innerScript: string): string { ].join("; "); } +function parseEnvFile(raw: string): Record { + const values: Record = {}; + for (const line of raw.split(/\r?\n/u)) { + if (!line.trim() || line.trimStart().startsWith("#")) continue; + const index = line.indexOf("="); + if (index <= 0) continue; + const key = line.slice(0, index); + let value = line.slice(index + 1); + if (value.startsWith("\"") && value.endsWith("\"")) { + try { + value = JSON.parse(value) as string; + } catch { + value = value.slice(1, -1); + } + } + values[key] = value; + } + return values; +} + +export function runtimeSecretPresenceFromEnvText(envText: string, requirements: RuntimeSecretRequirement[]): RuntimeSecretPresence[] { + const values = parseEnvFile(envText); + return requirements.map((requirement) => { + const value = values[requirement.sourceEnvName] ?? ""; + return { + sourceEnvName: requirement.sourceEnvName, + containerEnvName: requirement.containerEnvName, + present: value.length > 0, + length: value.length, + }; + }); +} + +export function baiduNetdiskAuthHealthGateStatus(health: unknown): { ok: boolean; requiredFields: string[]; failedFields: string[] } { + const record = typeof health === "object" && health !== null && !Array.isArray(health) ? health as Record : {}; + const auth = typeof record.auth === "object" && record.auth !== null && !Array.isArray(record.auth) ? record.auth as Record : {}; + const failedFields = baiduNetdiskAuthHealthGate.requiredFields.filter((field) => auth[field] !== true); + return { + ok: failedFields.length === 0, + requiredFields: baiduNetdiskAuthHealthGate.requiredFields, + failedFields, + }; +} + +function composeArtifactRuntime(config: UniDeskConfig, target: ArtifactConsumerTarget): ComposeArtifactRuntime { + if (target.compose === undefined) throw new Error("compose artifact target missing compose config"); + const compose = target.compose; + const workDir = compose.workDir ?? config.providerGateway.upgrade.hostProjectRoot; + const composeFile = compose.composeFile ?? config.providerGateway.upgrade.composeFile; + const envFile = join(workDir, config.providerGateway.upgrade.composeEnvFile); + const project = compose.projectHint ?? (config.providerGateway.upgrade.composeProject || config.docker.projectName); + const command = ["docker", "compose", "--env-file", envFile, "-f", join(workDir, composeFile), "-p", project]; + return { workDir, composeFile, envFile, project, command }; +} + +function composeArtifactSecretPreflight(envFile: string, requirements: RuntimeSecretRequirement[] | undefined): { ok: boolean; envFile: string; requirements: RuntimeSecretPresence[]; missing: RuntimeSecretPresence[]; valuesLogged: false } { + const requirementList = requirements ?? []; + const envText = existsSync(envFile) ? readFileSync(envFile, "utf8") : ""; + const presence = runtimeSecretPresenceFromEnvText(envText, requirementList); + const missing = presence.filter((item) => !item.present); + return { + ok: requirementList.length === 0 || (existsSync(envFile) && missing.length === 0), + envFile, + requirements: presence, + missing, + valuesLogged: false, + }; +} + +function authHealthGateScript(gate: AuthHealthGate | undefined, healthJsonShellArg: string): string[] { + if (gate === undefined) return []; + return [ + `auth_health_gate_fields=${shellQuote(gate.requiredFields.join(","))}`, + `python3 - ${healthJsonShellArg} "$auth_health_gate_fields" <<'PY'`, + "import json, sys", + "path, fields_text = sys.argv[1:]", + "fields = [item for item in fields_text.split(',') if item]", + "body = json.load(open(path, encoding='utf-8'))", + "auth = body.get('auth') if isinstance(body, dict) else None", + "auth = auth if isinstance(auth, dict) else {}", + "failed = [field for field in fields if auth.get(field) is not True]", + "if failed:", + " print('baidu_netdisk_auth_health_gate_failed=' + ','.join(failed))", + " raise SystemExit(1)", + "print('baidu_netdisk_auth_health_gate=ok')", + "PY", + ]; +} + function upsertEnvFileValues(path: string, values: Record): void { const existing = readFileSync(path, "utf8"); const seen = new Set(); @@ -1469,6 +1600,24 @@ async function deployComposeArtifactNow(options: ArtifactRegistryOptions, spec: const commit = options.commit; if (commit === null) throw new Error("artifact-registry deploy-service requires --commit "); if (target.compose === undefined) throw new Error(`${spec.serviceId} missing compose artifact consumer config`); + const config = readConfig(); + const runtime = composeArtifactRuntime(config, target); + const secretPreflight = composeArtifactSecretPreflight(runtime.envFile, target.compose.requiredRuntimeSecrets); + if (!secretPreflight.ok) { + return { + ok: false, + supported: true, + serviceId: spec.serviceId, + step: "runtime-secret-preflight", + error: "runtime-secret-presence-missing", + environment: options.environment ?? "prod", + envFile: runtime.envFile, + requirements: secretPreflight.requirements, + missing: secretPreflight.missing, + valuesLogged: false, + recoveryHint: target.compose.authHealthGate?.recoveryHint ?? "Restore the required runtime secrets in the canonical Compose env file, then rerun artifact deploy.", + }; + } const health = runReadonlyStatus(options, true); if (health.ok !== true) { return { ok: false, serviceId: spec.serviceId, error: "D601 artifact registry is not healthy", health }; @@ -1509,14 +1658,11 @@ async function deployComposeArtifactNow(options: ArtifactRegistryOptions, spec: if (tagCommit.exitCode !== 0 || tagCommit.timedOut) { return { ok: false, step: "docker-tag-commit", targetImage: commitImage, tag: commandTail(tagCommit), registryProbe: commandTail(registryProbe) }; } - const config = readConfig(); - const runtimeEnv = writeComposeEnv(config, false); - upsertEnvFileValues(runtimeEnv.envFile, { + upsertEnvFileValues(runtime.envFile, { ...composeArtifactEnvValues(spec, target, options, commit), }); - const compose = resolveComposeCommand(config, runtimeEnv.envFile); - const projectIndex = compose.indexOf("-p"); - const composeProject = projectIndex >= 0 && compose[projectIndex + 1] !== undefined ? compose[projectIndex + 1] : "unidesk"; + const compose = runtime.command; + const composeProject = runtime.project; const serviceName = target.compose.serviceName; const containerName = target.compose.containerName; const serviceLogPrefix = spec.serviceId.replace(/-/gu, "_"); @@ -1544,6 +1690,7 @@ async function deployComposeArtifactNow(options: ArtifactRegistryOptions, spec: `health_json=/tmp/unidesk-${safeName(spec.serviceId)}-health.json`, `docker exec "$cid" sh -lc ${shellQuote(target.compose.healthProbeCommand)} > "$health_json"`, "cat \"$health_json\"", + ...authHealthGateScript(target.compose.authHealthGate, "\"$health_json\""), ...(target.compose.requireHealthCommit ? [ "health_commit=$(python3 -c 'import json,sys; print(((json.load(open(sys.argv[1])).get(\"deploy\") or {}).get(\"commit\") or \"\"))' \"$health_json\")", "health_requested_commit=$(python3 -c 'import json,sys; print(((json.load(open(sys.argv[1])).get(\"deploy\") or {}).get(\"requestedCommit\") or \"\"))' \"$health_json\")", @@ -1560,7 +1707,9 @@ async function deployComposeArtifactNow(options: ArtifactRegistryOptions, spec: sourceImage, targetImage: composeImage, registryProbe: commandTail(registryProbe), + runtimeSecretPreflight: secretPreflight, deploy: commandTail(deploy), + recoveryHint: target.compose.authHealthGate?.recoveryHint ?? undefined, }; } const running = runCommand(["docker", "inspect", containerName, "--format", "image={{.Config.Image}} imageId={{.Image}} status={{.State.Status}} health={{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}"], repoRoot); @@ -1584,6 +1733,11 @@ async function deployComposeArtifactNow(options: ArtifactRegistryOptions, spec: imageLabelCommit: commit, serviceHealthCommit: target.compose.requireHealthCommit ? commit : "not-required", serviceHealthRequestedCommit: target.compose.requireHealthCommit ? commit : "not-required", + requiredRuntimeSecrets: secretPreflight.requirements, + authHealthGate: target.compose.authHealthGate === undefined ? "not-required" : { + requiredFields: target.compose.authHealthGate.requiredFields, + passed: true, + }, healthyOldVersionAccepted: false, }, rollback: { @@ -1696,6 +1850,7 @@ function d601ComposeArtifactDeployScript(options: ArtifactRegistryOptions, spec: "health_probe=$(printf '%s' \"$health_probe_b64\" | base64 -d)", "docker exec \"$cid\" sh -lc \"$health_probe\" >/tmp/unidesk-artifact-health.out", "cat /tmp/unidesk-artifact-health.out", + ...authHealthGateScript(compose.authHealthGate, shellQuote("/tmp/unidesk-artifact-health.out")), "printf 'artifact_cd_service=%s\\nartifact_cd_source_repo=%s\\nartifact_cd_deploy_ref=%s\\nartifact_cd_source_image=%s\\nartifact_cd_stable_image=%s\\nartifact_cd_runtime_image=%s\\nartifact_cd_commit=%s\\nartifact_cd_container=%s\\nartifact_cd_container_id=%s\\nartifact_cd_image_label_commit=%s\\nartifact_cd_container_label_commit=%s\\n' \"$service_id\" \"$source_repo\" \"$deploy_ref\" \"$registry_image\" \"$stable_image\" \"$commit_image\" \"$commit\" \"$container\" \"$cid\" \"$actual_commit\" \"$container_commit\"", ].join("\n"); } @@ -1866,7 +2021,28 @@ function dryRunArtifactConsumerPlan(options: ArtifactRegistryOptions, spec: Arti : target.compose.requireHealthCommit ? `${spec.serviceId} runtime health probe succeeds and reports deploy.commit/deploy.requestedCommit matching the artifact commit` : `${spec.serviceId} /health succeeds for the recreated container`, + ...(target.compose.requiredRuntimeSecrets === undefined ? [] : [ + `${spec.serviceId} live apply checks required runtime secret presence in the canonical Compose env file without reading or printing values`, + ]), + ...(target.compose.authHealthGate === undefined ? [] : [ + `${spec.serviceId} live apply gates success on /health.auth fields: ${target.compose.authHealthGate.requiredFields.join(", ")}`, + ]), ], + runtimeSecrets: target.compose.requiredRuntimeSecrets === undefined ? undefined : { + check: "live-apply-preflight", + valuesPrinted: false, + requirements: target.compose.requiredRuntimeSecrets.map((item) => ({ + sourceEnvName: item.sourceEnvName, + containerEnvName: item.containerEnvName, + presence: "not-read-during-dry-run", + })), + dryRunDisposition: "pending-live-check", + }, + authHealthGate: target.compose.authHealthGate === undefined ? undefined : { + path: "/health", + requiredAuthFields: target.compose.authHealthGate.requiredFields, + dryRunDisposition: "pending-live-check", + }, rollback: rollbackInfo(spec, target, environment, commit), }; } diff --git a/scripts/src/check.ts b/scripts/src/check.ts index 6d23dac7..1f1cbe36 100644 --- a/scripts/src/check.ts +++ b/scripts/src/check.ts @@ -300,12 +300,14 @@ export function runChecks(config: UniDeskConfig, options: CheckOptions = default items.push(commandItem("code-queue:stale-active-owner-expired", ["bun", "scripts/code-queue-liveness-diagnostics-test.ts", "--only", "code-queue:stale-active-owner-expired"], 30_000)); items.push(commandItem("code-queue:control-plane-split-brain-diagnostics", ["bun", "scripts/code-queue-liveness-diagnostics-test.ts", "--only", "code-queue:control-plane-split-brain-diagnostics"], 30_000)); items.push(commandItem("code-queue:oa-publisher-degraded-visible", ["bun", "scripts/code-queue-liveness-diagnostics-test.ts", "--only", "code-queue:oa-publisher-degraded-visible"], 30_000)); + items.push(commandItem("baidu-netdisk:artifact-guard-contract", ["bun", "scripts/baidu-netdisk-artifact-guard-contract-test.ts"], 30_000)); } else { items.push(skippedItem("typescript:scripts", "scripts TypeScript typecheck is opt-in", "--scripts-typecheck or --full")); items.push(skippedItem("code-queue:prompt-observation-contract", "prompt observation contract is opt-in with script checks", "--scripts-typecheck or --full")); items.push(skippedItem("code-queue:issue3-diagnostics-and-image-preflight", "Code Queue issue #3 regression fixtures are opt-in with script checks", "--scripts-typecheck or --full")); items.push(skippedItem("code-queue:trace-summary-contract", "Code Queue trace summary contract is opt-in with script checks", "--scripts-typecheck or --full")); items.push(skippedItem("code-queue:liveness-diagnostics-fixtures", "Code Queue liveness diagnostics fixtures are opt-in with script checks", "--scripts-typecheck or --full")); + items.push(skippedItem("baidu-netdisk:artifact-guard-contract", "Baidu Netdisk artifact guard contract is opt-in with script checks", "--scripts-typecheck or --full")); } if (options.logs) { items.push(unifiedLogRotationItem());