docs: define cicd drift contract

2026-05-21 12:16:36 +00:00
parent a2c469ee8c
commit 92dff00028
2 changed files with 279 additions and 0 deletions
@@ -16,6 +16,43 @@ The standard release shape is:

 `backend-core` and D601 `code-queue` may be validated only in dev in this phase. This document must not be used to introduce production deploy validation for either service.

+## Phase-One Deploy.json Consolidation Contract
+
+Issue #60 moves CI/CD image deployment toward one release-intent source. The implemented `deploy.json` schema still carries only `id`, `repo` and `commitId`; this section defines the phase-one migration contract before changing runtime behavior.
+
+Current duplicated configuration surfaces are:
+
+- Service source and version: `deploy.json.environments.*.services[]`, `config.json.microservices[].repository`, `CI.json.artifacts[].source`, runtime `UNIDESK_DEPLOY_*` metadata and service health payloads.
+- Image repository, tag and digest: `CI.json.artifacts[].image`, artifact-registry consumer specs, Compose/Kubernetes image fields, registry digest probes and docs tables.
+- Runtime target metadata: `config.json` provider/deployment entries, Compose service/container names, k3s namespaces, Kubernetes Deployment/Service names, artifact-registry target specs and runtime manifests.
+- Deployment parameters: runtime env prefixes, required secret keys, health deploy metadata requirements, resource requests/limits, rollout strategy and dev/prod support gates.
+- Consumer scope: `deploy.json` dev/prod service lists, deploy/artifact-registry allowlists, dev/prod Kubernetes manifests and service-specific documentation.
+
+Phase one extends the desired-state model in this order:
+
+1. Keep existing `deploy.json` `id`, `repo` and `commitId` as the deployed source/version authority. All environment-ref deploy commands continue reading `origin/master:deploy.json`.
+2. Add optional deploy-owned artifact identity fields after the static drift guard is green: `artifact.kind`, `artifact.repository`, `artifact.tag`, optional `artifact.digestRef`, and `artifact.upstreamDigestRef` for upstream-image services. For source-build services, `artifact.tag` must equal the selected commit unless a future schema explicitly records a digest-only release.
+3. Add consumer intent fields: `consumer.kind`, `consumer.dev.enabled`, `consumer.prod.enabled`, `consumer.supportLevel`, `consumer.targetRef` and `consumer.noRuntimeSourceBuild`. These describe whether the service is `main-server-compose`, `d601-k3s-managed`, `d601-direct-compose`, `upstream-digest`, `dry-run-only` or `unsupported`; low-level object names still come from config/manifests until renderers exist.
+4. Add deployment-time metadata that must be common across CI, dev CD and prod CD: deploy env prefix, health deploy-metadata requirement, required runtime secret key names, rollback policy, and resource profile identifiers. The planned first keys are `health.deployMetadataRequired` and `runtime.requiredSecretKeys`. Secret values, credentials, volumes and host paths do not move into `deploy.json`.
+5. Teach CI producer dry-run, deploy plan and artifact-registry dry-run to render from `deploy.json` first and compare mirrored `CI.json`/`config.json`/manifest values as derived copies. A mismatch is drift, not an alternate source of truth.
+
+Fields that stay outside `deploy.json` during phase one:
+
+- `CI.json`: producer pipeline name, source root, Dockerfile path and success summary shape. Once artifact identity is in `deploy.json`, `CI.json.artifacts[].image.repository` becomes a compatibility mirror checked for drift.
+- `config.json`: provider id, proxy route and port policy, backend health path, Compose file/service/container names, development SSH/worktree details, and secret source locations.
+- Compose and Kubernetes manifests: concrete container specs, volumes, PVCs, security context, rollout strategy and raw resource requests/limits until a renderer owns those files.
+- Artifact-registry executor code: low-level pull/import/retag/recreate commands, registry probes and platform-specific verification scripts. The executor must consume the normalized deploy plan instead of hardcoding release intent.
+
+The drift contract is:
+
+- `deploy.json` wins for release intent. `config.json.microservices[].repository.commitId` is compatibility data only and must not drive env-ref deploys.
+- `deploy.json.repo` and `CI.json.artifacts[].source.repo` must match for every source-build service that appears in `deploy.json`.
+- Image tags used by CI/CD must be commit-pinned or digest-pinned. Mutable tags such as `latest` are never valid release evidence.
+- Runtime topology may be mirrored in dry-run output, but deployment commands must treat it as derived target metadata and keep `noRuntimeSourceBuild=true` for reviewed artifact consumers.
+- Upstream-image services must stay `blocked` in CI source-build publishing until an upstream digest or mirror digest consumer exists.
+
+Lightweight evidence for this contract is `bun scripts/issue-60-cicd-drift-contract-test.ts`. The test parses local JSON/docs only; it does not publish images, deploy services, restart containers, run Playwright or run full e2e.
+
 ## Artifact Catalog

 Root `CI.json` is the CI producer catalog. It is not a deployment manifest.
@@ -127,6 +164,7 @@ Minimum evidence for this lane is:
 | k3s control bridge dry-run | `bun scripts/cli.ts deploy apply --env prod --service k3sctl-adapter --dry-run` |
 | CI producer preflight | `bun scripts/cli.ts ci publish-user-service --service <service> --commit <full-sha> --dry-run` |
 | Decision Center desired/live no-build drift guard | `bun scripts/decision-center-desired-state-contract-test.ts` |
+| Issue #60 phase-one deploy.json drift guard | `bun scripts/issue-60-cicd-drift-contract-test.ts` |

 ### Upstream Image Evidence

@@ -0,0 +1,241 @@
+import { readFileSync } from "node:fs";
+import { rootPath } from "./src/config";
+
+type JsonRecord = Record<string, unknown>;
+type Environment = "dev" | "prod";
+
+const reviewedSourceBuildServices = new Set([
+  "backend-core",
+  "frontend",
+  "baidu-netdisk",
+  "decision-center",
+  "project-manager",
+  "oa-event-flow",
+  "todo-note",
+  "code-queue-mgr",
+  "findjob",
+  "pipeline",
+  "met-nonlinear",
+  "k3sctl-adapter",
+  "mdtodo",
+  "claudeqq",
+  "code-queue",
+]);
+
+const reviewedArtifactConsumers = new Set([
+  "backend-core",
+  "frontend",
+  "baidu-netdisk",
+  "decision-center",
+  "project-manager",
+  "oa-event-flow",
+  "todo-note",
+  "code-queue-mgr",
+  "findjob",
+  "pipeline",
+  "met-nonlinear",
+  "k3sctl-adapter",
+  "mdtodo",
+  "claudeqq",
+  "code-queue",
+]);
+
+const plannedDeployJsonFields = [
+  "artifact.kind",
+  "artifact.repository",
+  "artifact.tag",
+  "artifact.digestRef",
+  "consumer.kind",
+  "consumer.dev.enabled",
+  "consumer.prod.enabled",
+  "consumer.supportLevel",
+  "consumer.noRuntimeSourceBuild",
+  "health.deployMetadataRequired",
+  "runtime.requiredSecretKeys",
+];
+
+function assertCondition(condition: unknown, message: string, detail: unknown = {}): void {
+  if (!condition) throw new Error(`${message}: ${JSON.stringify(detail)}`);
+}
+
+function asRecord(value: unknown, label: string): JsonRecord {
+  assertCondition(typeof value === "object" && value !== null && !Array.isArray(value), `${label} must be an object`, value);
+  return value as JsonRecord;
+}
+
+function asArray(value: unknown, label: string): unknown[] {
+  assertCondition(Array.isArray(value), `${label} must be an array`, value);
+  return value as unknown[];
+}
+
+function stringField(value: unknown, label: string): string {
+  assertCondition(typeof value === "string" && value.length > 0, `${label} must be a non-empty string`, value);
+  return value as string;
+}
+
+function loadJson(path: string): JsonRecord {
+  return asRecord(JSON.parse(readFileSync(rootPath(path), "utf8")) as unknown, path);
+}
+
+function isFullSha(value: string): boolean {
+  return /^[0-9a-f]{40}$/u.test(value);
+}
+
+function deployServicesByEnvironment(deploy: JsonRecord, environment: Environment): JsonRecord[] {
+  const environments = asRecord(deploy.environments, "deploy.json.environments");
+  const env = asRecord(environments[environment], `deploy.json.environments.${environment}`);
+  return asArray(env.services, `deploy.json.environments.${environment}.services`).map((item, index) => asRecord(item, `${environment}.services[${index}]`));
+}
+
+function serviceKey(environment: Environment, serviceId: string): string {
+  return `${environment}/${serviceId}`;
+}
+
+function artifactByService(ciCatalog: JsonRecord): Map<string, JsonRecord> {
+  const artifacts = asArray(ciCatalog.artifacts, "CI.json.artifacts").map((item, index) => asRecord(item, `CI.json.artifacts[${index}]`));
+  return new Map(artifacts.map((artifact) => [stringField(artifact.serviceId, "CI.json artifact serviceId"), artifact]));
+}
+
+function configByService(config: JsonRecord): Map<string, JsonRecord> {
+  const microservices = asArray(config.microservices, "config.json.microservices").map((item, index) => asRecord(item, `config.json.microservices[${index}]`));
+  return new Map(microservices.map((service) => [stringField(service.id, "config service id"), service]));
+}
+
+function assertDeployJsonReleaseIntent(deploy: JsonRecord): Array<{ environment: Environment; serviceId: string; repo: string; commitId: string }> {
+  assertCondition(deploy.schemaVersion === 2, "deploy.json must remain schemaVersion=2", deploy);
+  const seen = new Set<string>();
+  const services: Array<{ environment: Environment; serviceId: string; repo: string; commitId: string }> = [];
+  for (const environment of ["dev", "prod"] as const) {
+    for (const service of deployServicesByEnvironment(deploy, environment)) {
+      const keys = Object.keys(service).sort();
+      assertCondition(keys.join(",") === "commitId,id,repo", "phase-one deploy.json service entries must remain only id/repo/commitId until schema migration lands", { environment, service, keys });
+      const serviceId = stringField(service.id, `${environment} service id`);
+      const repo = stringField(service.repo, `${environment}/${serviceId}.repo`);
+      const commitId = stringField(service.commitId, `${environment}/${serviceId}.commitId`).toLowerCase();
+      assertCondition(isFullSha(commitId), "deploy.json phase-one commit pins must be full 40-character SHAs", { environment, serviceId, commitId });
+      const key = serviceKey(environment, serviceId);
+      assertCondition(!seen.has(key), "deploy.json service must not be duplicated within an environment", { key });
+      seen.add(key);
+      services.push({ environment, serviceId, repo, commitId });
+    }
+  }
+  return services;
+}
+
+function assertCiCatalogAlignment(services: Array<{ environment: Environment; serviceId: string; repo: string; commitId: string }>, ciCatalog: JsonRecord): void {
+  const artifacts = artifactByService(ciCatalog);
+  const defaults = asRecord(ciCatalog.defaults, "CI.json.defaults");
+  assertCondition(defaults.mutableTagsAllowed === false, "CI.json must continue rejecting mutable tags", defaults);
+  assertCondition(defaults.tagTemplate === "{{sourceCommit}}", "CI.json tag template must remain commit-pinned", defaults);
+
+  for (const service of services) {
+    const artifact = artifacts.get(service.serviceId);
+    assertCondition(artifact !== undefined, "every deploy.json service must have a CI catalog entry or a reviewed exception", service);
+    assertCondition(reviewedSourceBuildServices.has(service.serviceId), "deploy.json service must be in the reviewed source-build/drift set for phase one", service);
+    const kind = stringField(artifact?.kind, `${service.serviceId}.kind`);
+    const status = stringField(artifact?.status, `${service.serviceId}.status`);
+    assertCondition(kind === "source-build", "deploy.json source services must stay source-build in CI.json during phase one", { service, artifact });
+    assertCondition(status === "supported", "deploy.json source services must stay supported in CI.json during phase one", { service, artifact });
+    const source = asRecord(artifact?.source, `${service.serviceId}.source`);
+    const image = asRecord(artifact?.image, `${service.serviceId}.image`);
+    assertCondition(source.repo === service.repo, "deploy.json repo and CI.json source.repo must not drift", { service, source });
+    assertCondition(typeof source.dockerfile === "string" && source.dockerfile.length > 0, "CI.json must keep the producer Dockerfile until deploy.json owns artifact identity", { service, source });
+    assertCondition(image.repository === `unidesk/${service.serviceId}`, "CI image repository must remain service-id derived before deploy.json artifact.repository migration", { service, image });
+  }
+}
+
+function assertConfigCompatibility(services: Array<{ environment: Environment; serviceId: string; repo: string; commitId: string }>, config: JsonRecord): Array<{ environment: Environment; serviceId: string; deployCommitId: string; configCommitId: string }> {
+  const configs = configByService(config);
+  const commitMirrors: Array<{ environment: Environment; serviceId: string; deployCommitId: string; configCommitId: string }> = [];
+  for (const service of services) {
+    if (service.serviceId === "backend-core" || service.serviceId === "frontend") continue;
+    const configService = configs.get(service.serviceId);
+    assertCondition(configService !== undefined, "deploy.json service must exist in config.json until renderers replace config lookups", service);
+    const repository = asRecord(configService?.repository, `${service.serviceId}.repository`);
+    assertCondition(repository.url === service.repo, "deploy.json repo and config.json repository.url must not drift", { service, repository });
+    assertCondition(typeof repository.dockerfile === "string" && repository.dockerfile.length > 0, "config.json keeps Dockerfile target metadata in phase one", { service, repository });
+    assertCondition(typeof repository.composeService === "string" && repository.composeService.length > 0, "config.json keeps compose/k3s service name in phase one", { service, repository });
+    assertCondition(typeof repository.containerName === "string" && repository.containerName.length > 0, "config.json keeps container name in phase one", { service, repository });
+    const configCommitId = stringField(repository.commitId, `${service.serviceId}.repository.commitId`);
+    if (configCommitId === service.commitId) {
+      commitMirrors.push({ environment: service.environment, serviceId: service.serviceId, deployCommitId: service.commitId, configCommitId });
+    }
+  }
+  return commitMirrors;
+}
+
+function assertConsumerCoverage(services: Array<{ environment: Environment; serviceId: string; repo: string; commitId: string }>): void {
+  for (const service of services) {
+    assertCondition(reviewedArtifactConsumers.has(service.serviceId), "deploy.json service must be covered by the phase-one consumer drift set", service);
+    if (service.environment === "prod" && service.serviceId === "code-queue") {
+      continue;
+    }
+    if (service.serviceId === "k3sctl-adapter" || service.serviceId === "met-nonlinear") {
+      continue;
+    }
+    assertCondition(
+      service.serviceId !== "code-agent-sandbox",
+      "code-agent-sandbox must not become a deploy.json artifact consumer before the schema contract is reviewed",
+      service,
+    );
+  }
+}
+
+function assertUpstreamImageBoundary(ciCatalog: JsonRecord, config: JsonRecord): void {
+  const artifacts = artifactByService(ciCatalog);
+  const configs = configByService(config);
+  for (const serviceId of ["filebrowser", "filebrowser-d601"]) {
+    const artifact = asRecord(artifacts.get(serviceId), `CI.json ${serviceId}`);
+    const upstream = asRecord(artifact.upstream, `CI.json ${serviceId}.upstream`);
+    const configService = asRecord(configs.get(serviceId), `config.json ${serviceId}`);
+    const repository = asRecord(configService.repository, `config.json ${serviceId}.repository`);
+    const artifactSource = asRecord(repository.artifactSource, `config.json ${serviceId}.repository.artifactSource`);
+    assertCondition(artifact.kind === "upstream-image", "File Browser services must stay upstream-image catalog entries", artifact);
+    assertCondition(artifact.status === "blocked", "File Browser upstream-image services must stay blocked until mirror consumer exists", artifact);
+    assertCondition(upstream.imageRef === artifactSource.imageRef, "CI upstream imageRef and config artifactSource imageRef must not drift", { serviceId, upstream, artifactSource });
+    assertCondition(artifactSource.ciDockerfileBuild === false, "File Browser must not be treated as a CI Dockerfile build", { serviceId, artifactSource });
+    assertCondition(artifactSource.pullOnlyCd === true, "File Browser must remain pull-only CD", { serviceId, artifactSource });
+    assertCondition(typeof upstream.digestRef === "string" && String(upstream.digestRef).includes("@sha256:"), "upstream image entry must keep a digest pin", { serviceId, upstream });
+  }
+}
+
+function assertDocsContract(): void {
+  const docs = readFileSync(rootPath("docs/reference/cicd-standardization.md"), "utf8");
+  for (const phrase of [
+    "Phase-One Deploy.json Consolidation Contract",
+    "Current duplicated configuration surfaces",
+    "Fields that stay outside `deploy.json` during phase one",
+    "The drift contract is:",
+    "bun scripts/issue-60-cicd-drift-contract-test.ts",
+  ]) {
+    assertCondition(docs.includes(phrase), "CI/CD standardization docs must include the issue #60 phase-one contract", { phrase });
+  }
+  for (const field of plannedDeployJsonFields) {
+    assertCondition(docs.includes(field), "phase-one docs must name the planned deploy.json schema field", { field });
+  }
+}
+
+const deploy = loadJson("deploy.json");
+const ciCatalog = loadJson("CI.json");
+const config = loadJson("config.json");
+const services = assertDeployJsonReleaseIntent(deploy);
+assertCiCatalogAlignment(services, ciCatalog);
+const configCommitMirrors = assertConfigCompatibility(services, config);
+assertConsumerCoverage(services);
+assertUpstreamImageBoundary(ciCatalog, config);
+assertDocsContract();
+
+process.stdout.write(`${JSON.stringify({
+  ok: true,
+  checks: [
+    "deploy.json remains the phase-one release-intent source with full commit pins",
+    "deploy.json repo values match CI.json source-build producer repos",
+    "CI.json keeps commit-tag producer policy and service-id image repositories",
+    "config.json runtime topology remains compatibility metadata and does not own env-ref commits",
+    "File Browser upstream-image entries stay blocked from Dockerfile CI",
+    "docs/reference/cicd-standardization.md names the phase-one schema and drift contract",
+  ],
+  deployServicesChecked: services.length,
+  compatibilityCommitMirrors: configCommitMirrors,
+  plannedDeployJsonFields,
+}, null, 2)}\n`);