ci: standardize artifact summary catalog
This commit is contained in:
Codex
@@ -6,6 +6,8 @@ backend-core 和 reviewed user services 的长期分工是：CI 在 D601 构建
|
||||
|
||||
Production CI/CD runtime pinning and release-line boundaries follow `docs/reference/release-governance.md` and [GitHub issue #6](https://github.com/pikasTech/unidesk/issues/6). The registry may cache commit-pinned artifacts, but it must not become a floating replacement for `deploy.json`, `release/v1`, or `master` source history.
|
||||
|
||||
The CI-side artifact catalog is root `CI.json`. That file describes only artifact producer inputs and naming; registry consumers still verify the real image labels, manifest digest and live runtime separately. The producer summary contract is owned by `docs/reference/ci.md` and includes `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`.
|
||||
|
||||
## Architecture
|
||||
|
||||
registry 运行在 D601 host/WSL OS 上，由 systemd 管理 Docker Compose 项目：
|
||||
@@ -111,7 +113,7 @@ docker compose -p unidesk-artifact-registry -f /home/ubuntu/.unidesk/artifact-re
|
||||
|
||||
1. D601 artifact registry 已安装并通过 `health`。
|
||||
2. D601 CI 从 pushed Git checkout 构建 `unidesk/<service-id>:<commit>`。
|
||||
3. CI 将镜像 push 到 `127.0.0.1:5000/unidesk/<service-id>:<commit>`，并记录 image ref 与 digest。
|
||||
3. CI 将镜像 push 到 `127.0.0.1:5000/unidesk/<service-id>:<commit>`，并记录完整 artifact summary，包括 service id、source repo、source commit、Dockerfile、image ref、tag、digest 和 digest ref。
|
||||
4. CD 在运行目标上确认目标 commit/tag/digest 存在。
|
||||
5. Compose runtime 通过 provider-gateway Host SSH 从 D601 registry 流式读取 commit-pinned 镜像 tar，不开放 registry 端口，也不使用第三方镜像托管。
|
||||
6. Compose runtime retag 为 Compose 使用的镜像名，并执行 `docker compose up -d --no-build --no-deps --force-recreate <service>`。
|
||||
|
||||
+20
-2
@@ -8,6 +8,7 @@ UniDesk CI is hosted on the D601 native k3s cluster with Tekton Pipelines and Te
|
||||
- Tekton Triggers: `v0.34.0`.
|
||||
- UniDesk CI namespace: `unidesk-ci`.
|
||||
- Manifests: `src/components/microservices/k3sctl-adapter/k3s/ci/`.
|
||||
- Artifact catalog: root `CI.json`, which is CI artifact catalog only. It describes build inputs, image naming and summary fields; runtime topology, rollout target, ports, namespaces and desired service commits remain in `config.json`, service manifests and `deploy.json`.
|
||||
- CLI entry: `bun scripts/cli.ts ci install|status|run|publish-backend-core|publish-user-service|run-dev-e2e|logs`.
|
||||
- Dev namespace e2e runner: `bun scripts/cli.ts ci run-dev-e2e`; authoritative runner path, manifest contract and safety boundary are in `docs/reference/dev-ci-runner.md`.
|
||||
- Rust backend-core check/build boundary: CI may run `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust` on D601; the master server must not compile Rust for backend-core iteration. The authoritative dev environment rule is `docs/reference/dev-environment.md`.
|
||||
@@ -32,6 +33,23 @@ Git clone and dependency downloads inside the repo check task use `d601-provider
|
||||
|
||||
Private repository source authentication is part of the CI contract and follows `docs/reference/devops-hygiene.md`. If the repo-check task fails at `git clone` because credentials are unavailable, treat it as a CI infrastructure/auth gap, not as an application test result.
|
||||
|
||||
## Artifact Catalog And Summary Contract
|
||||
|
||||
`CI.json` is the reusable CI artifact catalog. It must remain artifact-only: `serviceId`, `sourceRepo`, `dockerfile`, registry repository naming, tag policy and summary-field semantics are allowed; provider ids, runtime namespaces, ports, compose services, Kubernetes Services, health paths, env, volumes and desired deploy commits are not allowed. `deploy.json` remains the version intent for deployments and must not be replaced by `CI.json`.
|
||||
|
||||
Every successful image-producing CI task must expose a common `artifactSummary` contract:
|
||||
|
||||
- `serviceId`: stable UniDesk service id.
|
||||
- `sourceCommit`: full 40-character Git commit used as source and tag.
|
||||
- `sourceRepo`: Git repository URL used to materialize the source.
|
||||
- `dockerfile`: repo-relative Dockerfile path.
|
||||
- `imageRef`: commit-tagged image reference pushed by CI.
|
||||
- `tag`: commit-pinned image tag; mutable `latest` is invalid.
|
||||
- `digest`: registry manifest digest for the pushed image.
|
||||
- `digestRef`: immutable `repository@digest` image reference.
|
||||
|
||||
Tekton artifact tasks write these values as TaskRun results and also print the legacy `*_artifact_*` log lines for operator diagnostics. The CLI must read TaskRun results first, fall back to pod logs only for older runs, derive `imageRef`/`digestRef` from repository, tag and digest where possible, and report exact missing fields such as `digest` or `digestRef`. It must not turn a succeeded PipelineRun into a generic `incomplete` failure.
|
||||
|
||||
## CI/CD Runtime Governance
|
||||
|
||||
CI/CD server and control-plane runtime is production-like infrastructure. Its service version must be pinned by `deploy.json` and verified through runtime commit metadata; it must not float with the latest `master` just because the operator's CLI is newer.
|
||||
@@ -72,7 +90,7 @@ The CI artifact task must follow these rules:
|
||||
- The image is tagged with the source commit, for example `unidesk/backend-core:<commit>`, and pushed to the D601 artifact registry as `127.0.0.1:5000/unidesk/backend-core:<commit>`.
|
||||
- The image must carry at least `unidesk.ai/service-id=backend-core`, `unidesk.ai/source-repo`, `unidesk.ai/source-commit` and `unidesk.ai/dockerfile=src/components/backend-core/Dockerfile`.
|
||||
- Publication must fail if the D601 artifact registry is not healthy. It must not fall back to a third-party registry or a mutable `latest` tag.
|
||||
- CI may output the image ref and digest as deployment input, but it must not restart production Compose services, call production `deploy apply`, mutate production namespaces, or change `deploy.json`.
|
||||
- CI output must include the common `artifactSummary` fields defined above. `artifactSummary.imageRef` and `artifactSummary.digestRef` are deployment inputs for later CD, but CI must not restart production Compose services, call production `deploy apply`, mutate production namespaces, or change `deploy.json`.
|
||||
|
||||
The artifact registry contract and CD consumption path are defined in `docs/reference/artifact-registry.md`. CI is the producer of the backend-core image artifact; CD is only the consumer.
|
||||
|
||||
@@ -86,7 +104,7 @@ The CI user-service artifact task must follow these rules:
|
||||
- D601 prepares a commit-pinned source export under `/home/ubuntu/.unidesk/ci/user-service-artifacts/<service-id>/<commit>` using the existing GitHub SSH deploy identity and node-local provider-gateway WS egress proxy. Tekton consumes that export through a read-only hostPath.
|
||||
- The image is tagged only with the source commit and pushed to the D601 registry as `127.0.0.1:5000/unidesk/<service-id>:<commit>`. The producer must reject third-party registries and must not publish or consume a mutable `latest` tag.
|
||||
- The image must carry `unidesk.ai/service-id`, `unidesk.ai/source-repo`, `unidesk.ai/source-commit` and `unidesk.ai/dockerfile` labels.
|
||||
- The command output must include the image ref, tag, digest, source commit and service id. The digest ref is suitable as immutable input for later dev/prod deployment work.
|
||||
- The command output must include the common `artifactSummary` fields: `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`. The digest ref is suitable as immutable input for later dev/prod deployment work.
|
||||
- CI is an artifact producer only. It must not restart production services, call production `deploy apply`, mutate the production namespace, or change `deploy.json`.
|
||||
|
||||
Publish a Baidu Netdisk artifact:
|
||||
|
||||
@@ -29,7 +29,7 @@ CLI 可以从 `master` 快速演进，但必须兼容 `deploy.json` 固定的 CI
|
||||
- `dev-env validate [--manifest path] [--kubectl-dry-run]` 离线校验 D601 `unidesk-dev` namespace、dev PostgreSQL 底座和 dev workload manifest。默认检查 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-foundation.k8s.yaml`；也可显式校验 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml` 或 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-code-queue.k8s.yaml`。所有 namespaced 对象必须只落到 `unidesk-dev`，foundation manifest 必须包含 `postgres-dev` StatefulSet/Service、dev secret/config、迁移 Job 和 DB URL guard，core manifest 必须包含 `backend-core-dev`/`frontend-dev` Deployment/Service，Code Queue dev manifest 必须包含 `code-queue-scheduler-dev`、`code-queue-read-dev`、`code-queue-write-dev` 和 dev provider egress proxy。加 `--kubectl-dry-run` 时额外执行 `kubectl apply --dry-run=client --validate=false -f <manifest>`，仍不 apply 资源。
|
||||
- `dev-env prewarm-images [--image image] [--provider-id D601] [--no-pull] [--proxy-url URL] [--pull-timeout-ms N] [--dry-run]` 创建异步 job，通过 UniDesk SSH 维护桥在 D601 上把开发底座依赖镜像从 Docker 缓存导入原生 k3s containerd。默认镜像是 `postgres:16-alpine` 和 `rancher/mirrored-library-busybox:1.36.1`，用于避免 `postgres-dev` 与 local-path helper pod 卡在外部 registry 拉取。该命令固定验证 `/etc/rancher/k3s/k3s.yaml` 指向的 native k3s 上下文，并输出 `dev_env_containerd_image_ready=...` 作为成功判据；它不 apply manifest、不修改生产 `unidesk` namespace。
|
||||
- `artifact-registry plan|render|status|health|install|deploy-backend-core|deploy-service` 管理 D601 host-managed CNCF Distribution registry 的声明、安装、只读检查和 pull-only artifact CD。该 registry 固定为 D601 loopback `127.0.0.1:5000`，由 systemd + Docker Compose 管理，位于 native k3s 故障域外；`deploy-backend-core` 和 `deploy-service` 只拉取 CI 已发布的 commit-pinned 镜像、retag/recreate 或导入 native k3s，并做 live commit 验证，不构建 runtime source。长期规则见 `docs/reference/artifact-registry.md`。
|
||||
- `ci install|status|run|publish-backend-core|publish-user-service|run-dev-e2e|logs` 管理 D601 原生 k3s 上的 Tekton CI。`run` 手动创建每 commit 检查和 Code Queue 只读性能门禁；`publish-backend-core` 与 `publish-user-service` 从 pushed Git commit 构建并发布 `127.0.0.1:5000/unidesk/<service>:<commit>`，但不部署生产；`run-dev-e2e` 的 Git 控制 runner、短 launcher、host fetch 边界、临时 smoke namespace 和 no-CD 规则只在 `docs/reference/dev-ci-runner.md` 定义；Tekton CI 通用规则见 `docs/reference/ci.md`。
|
||||
- `ci install|status|run|publish-backend-core|publish-user-service|run-dev-e2e|logs` 管理 D601 原生 k3s 上的 Tekton CI。`run` 手动创建每 commit 检查和 Code Queue 只读性能门禁；`publish-backend-core` 与 `publish-user-service` 从 pushed Git commit 构建并发布 `127.0.0.1:5000/unidesk/<service>:<commit>` commit-pinned artifacts，输出 `artifactSummary`（含 `serviceId`、`sourceCommit`、`sourceRepo`、`dockerfile`、`imageRef`、`tag`、`digest`、`digestRef`），但不部署生产；`run-dev-e2e` 的 Git 控制 runner、短 launcher、host fetch 边界、临时 smoke namespace 和 no-CD 规则只在 `docs/reference/dev-ci-runner.md` 定义；Tekton CI 通用规则见 `docs/reference/ci.md`。
|
||||
- `codex deploy <commitId>` 是旧 Code Queue 兼容部署入口，已禁用以防止维护通道直连 D601 部署 Code Queue；当前 dev 自动化只做 `ci run-dev-e2e` smoke，不提供 Code Queue CD，详细规则见 `docs/reference/codex-deploy.md`。
|
||||
- `codex submit [prompt] [--prompt-file path|--prompt-stdin] [--queue queueId] [--provider-id id] [--cwd path] [--model model] [--reasoning-effort effort] [--execution-mode mode] [--max-attempts N] [--reference-task-id id] [--dry-run]` 通过 backend-core 私有代理向稳定 `code-queue` 用户服务路径提交任务；prompt 必须且只能来自位置参数、文件或 stdin 之一，`--dry-run` 只返回结构化请求且不实际入队。提交确认和 dry-run 必须返回完整 prompt、字符数和 `truncated=false`，不能套用任务详情的预览截断策略，否则长任务 prompt 无法被人工验收。backend-core 默认把提交、队列 CRUD、已读状态、历史摘要和轻量 Trace 读取分流到主 server `code-queue-mgr`，由它写入主 PostgreSQL；D601 scheduler 只轮询并执行已入库任务。
|
||||
- `codex task <taskId>` 通过 Code Queue 私有代理按任务 ID 查询结构化执行摘要；默认只返回有界 prompt/response 预览、执行 Provider、工作目录、最后 assistant message、最近工具调用摘要、attempt、judge、错误、耗时和 trace 翻页提示，适合在新队列任务中引用历史 session 且避免噪声爆炸。该摘要读取默认由主 server `code-queue-mgr` 从 PostgreSQL 返回，不依赖 D601 `code-queue-read` Service 可用。
|
||||
|
||||
@@ -32,7 +32,8 @@ The default release flow for a user-service change is:
|
||||
- No user-service artifact may rely on a third-party registry as source of truth.
|
||||
- No production deploy may rebuild the source from a dirty worktree.
|
||||
- Commit-pinned image tags are the deployment truth; mutable `latest` tags are not.
|
||||
- The standard CI artifact producer is `bun scripts/cli.ts ci publish-user-service --service <id> --commit <full-sha>`. It accepts only a pushed Git commit and a registered service id, and reports image ref, tag, digest, source commit and service id.
|
||||
- Root `CI.json` is an artifact catalog only. It can list user-service CI build inputs such as `serviceId`, `sourceRepo`, `dockerfile`, image repository naming and the required artifact summary fields; it must not carry runtime topology or replace `deploy.json`.
|
||||
- The standard CI artifact producer is `bun scripts/cli.ts ci publish-user-service --service <id> --commit <full-sha>`. It accepts only a pushed Git commit and a registered service id, and reports `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`.
|
||||
- The CI artifact producer is not a deploy executor. It must not mutate the production namespace, restart production services, or update `deploy.json`.
|
||||
- Every production release must finish with a manual acceptance step after the automated checks pass.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user