mirrors/pikasTech-unidesk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e8b3c3ef320c5419e15cd824e2205b5cd00aa87d
pikasTech-unidesk/docs/reference/artifact-registry.md
T
Codex e8b3c3ef32 chore: pin code queue mgr stats dry run
2026-05-20 20:35:54 +00:00

18 KiB
Raw Blame History

D601 Artifact Registry

D601 artifact registry 是为 backend-core 和标准用户服务轻量 CD 准备的本地镜像制品入口。它使用开源、成熟的 CNCF Distribution Docker registry，不自定义镜像协议，也不把镜像托管到第三方服务。

backend-core 和 reviewed user services 的长期分工是：CI 在 D601 构建并发布 commit-pinned 镜像，CD 在运行目标只拉取、替换/导入和验证该镜像。registry 是这条链路的本地制品缓存，不是构建编排器，也不是生产部署控制面。

Production CI/CD runtime pinning and release-line boundaries follow docs/reference/release-governance.md and GitHub issue #6. The registry may cache commit-pinned artifacts, but it must not become a floating replacement for deploy.json, release/v1, or master source history.

The CI-side artifact catalog is root CI.json. That file describes only artifact producer inputs and naming; registry consumers still verify the real image labels, manifest digest and live runtime separately. The producer summary contract is owned by docs/reference/ci.md and includes serviceId, sourceCommit, sourceRepo, dockerfile, imageRef, tag, digest and digestRef.

CI.json may also record image-only upstream services as upstream-image entries with upstream digest and future mirror naming. Those entries are catalog coverage only until a mirror producer exists. Registry CD must not infer a deployable artifact from an upstream-image entry unless the corresponding D601 registry manifest already exists and a reviewed consumer supports that service.

Architecture

registry 运行在 D601 host/WSL OS 上，由 systemd 管理 Docker Compose 项目：

  • systemd unit：unidesk-artifact-registry.service
  • Compose project：unidesk-artifact-registry
  • container：unidesk-artifact-registry
  • image：registry:2.8.3
  • config：/home/ubuntu/.unidesk/artifact-registry/config.yml
  • compose：/home/ubuntu/.unidesk/artifact-registry/compose.yml
  • storage：/home/ubuntu/.unidesk/registry-storage
  • endpoint：http://127.0.0.1:5000

它不是 k3s workload，不使用 NodePort、hostPort、LoadBalancer 或公开端口。Docker 端口映射必须绑定到 D601 loopback：127.0.0.1:5000:5000。容器内部 registry 可监听 :5000，安全边界由 host loopback 绑定保证。

这个服务和 k3sctl-adapter 一样位于 k3s 故障域外，但职责不同：

  • k3sctl-adapter 是 UniDesk 到 native k3s 的控制桥，属于 UniDesk 直管服务。
  • artifact registry 是 D601 host-managed 制品缓存基础设施，D601 CI 向它推送 commit-pinned 镜像，master server Compose CD 和 D601 k3s CD 从它消费镜像。

Dependency Boundary

registry 运行期依赖应保持低且可手动维护：

  • D601 host Docker Engine。
  • Docker Compose plugin。
  • systemd。
  • registry:2.8.3 镜像，后续可升级为 digest pin。
  • D601 本地持久化目录。
  • provider-gateway Host SSH 只用于 UniDesk CLI 的只读 status / health 检查。

它不得依赖：

  • k3s 控制面或任意 k3s namespace。
  • 第三方镜像托管服务作为 artifact source of truth。
  • main server backend-core 本地 Rust 编译或 artifact-consumer 服务本地 Docker build。
  • 公开 registry 端口或公网反向代理。
  • k3s Service、NodePort、Ingress 或任何长期暴露的 registry 代理。

CLI

入口是：

bun scripts/cli.ts artifact-registry plan
bun scripts/cli.ts artifact-registry render
bun scripts/cli.ts artifact-registry install
bun scripts/cli.ts artifact-registry status
bun scripts/cli.ts artifact-registry health
bun scripts/cli.ts artifact-registry deploy-service --service baidu-netdisk --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service frontend --env prod --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service frontend --env dev --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service decision-center --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --env prod --service project-manager --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --env prod --service oa-event-flow --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --env prod --service code-queue-mgr --commit <full-sha> --dry-run
bun scripts/cli.ts artifact-registry deploy-service --env prod --service todo-note --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --env dev --service findjob --commit <full-sha> --dry-run
bun scripts/cli.ts artifact-registry deploy-service --env dev --service pipeline --commit <full-sha> --dry-run
bun scripts/cli.ts artifact-registry deploy-service --env dev --service met-nonlinear --commit <full-sha> --dry-run
bun scripts/cli.ts artifact-registry deploy-service --env prod --service k3sctl-adapter --commit <full-sha> --dry-run
bun scripts/cli.ts artifact-registry deploy-service --service mdtodo --env dev --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service mdtodo --env prod --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service claudeqq --env dev --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service claudeqq --env prod --commit <full-sha>
bun scripts/cli.ts artifact-registry deploy-service --service code-queue --env dev --commit <full-sha>

plan 输出架构边界、依赖项、默认路径和 backend-core artifact CD 流程。render 输出 systemd unit、Compose 文件和 registry config 的完整内容与 SHA-256。install --dry-run 只列出将要执行的远端动作，不写 D601 文件、不启动容器、不 reload systemd。

真实 install 必须是幂等动作：创建远端目录，写入 CLI 渲染出的 config/compose/unit，执行 systemctl daemon-reload，启用并启动 unidesk-artifact-registry.service，然后运行与 health 相同的验收检查。若远端文件 hash 与期望不一致，install 可以覆盖由本 CLI 管理的 unit/config/compose，但不得删除 registry storage。

deploy-backend-core 是旧兼容入口，当前作为标准路径已禁用。Production backend-core CD 必须使用 bun scripts/cli.ts deploy apply --env prod --service backend-core --commit <full-sha>，由 deploy reconciler 先执行共同的 artifact-consumer guardrail，再调用通用 deploy-service consumer。旧入口只能返回 structured deprecated 结果，不得绕过 deploy apply --env prod。

deploy-service 是标准化后的最小通用 artifact consumer。它目前支持 backend-core、baidu-netdisk、prod/dev frontend、decision-center、mdtodo、claudeqq、project-manager、oa-event-flow、code-queue-mgr、todo-note、findjob、pipeline、met-nonlinear 和 k3sctl-adapter，以及 dev-only code-queue。所有路径都必须先通过 D601 registry 的 commit-pinned manifest 校验，再执行拉取、导入/retag、部署和健康验证；code-queue --env prod 必须返回 structured unsupported，不能回退到生产 artifact deploy、rollout 或 manifest 变更；code-queue-mgr 的 prod live apply 仍需 supervisor 单独确认，met-nonlinear 与 k3sctl-adapter 当前只提供 plan/dry-run：

bun scripts/cli.ts artifact-registry deploy-service --service baidu-netdisk --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --service frontend --env prod --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --service frontend --env dev --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env dev --service decision-center --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --service decision-center --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service project-manager --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service oa-event-flow --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service todo-note --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service findjob --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service pipeline --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env dev --service mdtodo --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service mdtodo --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env dev --service claudeqq --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env prod --service claudeqq --commit <full-sha> --run-now
bun scripts/cli.ts artifact-registry deploy-service --env dev --service code-queue --commit <full-sha> --run-now

code-queue-mgr 是主 server Compose sidecar，不是 D601 Code Queue scheduler/runner。artifact-registry deploy-service --env prod --service code-queue-mgr --commit <full-sha> --dry-run 必须显示 target 仅为 composeService=code-queue-mgr / containerName=code-queue-mgr-backend，并在 excludedTargets 中说明不会触碰 code-queue scheduler、runner、任务、interrupt 或 cancel 状态。真实 prod apply 仍受 supervisor-only gate 保护；未经单服务授权不得执行非 dry-run apply。

dry-run 输出会暴露 registry probe URL、required labels、目标 image、部署形态、目标 Deployment 列表和回滚信息，但不得读取或打印运行时密钥。baidu-netdisk 是 PGDATA 备份链路依赖服务；它的 Compose artifact 路径会通过 provider-gateway Host SSH 把 unidesk/baidu-netdisk:<commit> 流式拉到 master server，retag 为 baidu-netdisk 和 baidu-netdisk:<commit>，在 canonical UniDesk 根目录使用 providerGateway.upgrade.composeEnvFile 指向的受控 env 文件写入 UNIDESK_BAIDU_NETDISK_DEPLOY_*，只 recreate baidu-netdisk service，并验证容器 image label 与 /health.deploy.commit。live apply 在 recreate 前必须确认受控 env 文件中存在 UNIDESK_BAIDU_NETDISK_CLIENT_ID、UNIDESK_BAIDU_NETDISK_CLIENT_SECRET 和 UNIDESK_BAIDU_NETDISK_TOKEN_KEY，输出只能包含 present/length/boolean；recreate 后必须验收 /health.auth.configured、clientIdConfigured、clientSecretConfigured、tokenKeyConfigured 和 loggedIn 全部为 true，否则返回失败或 degraded，并提示先恢复 env、单服务 recreate、再验证 microservice health baidu-netdisk。findjob、pipeline 和 met-nonlinear 的 D601 direct Compose 路径在 D601 本机验证 registry manifest、pull image、retag stable image、写入 UNIDESK_*_DEPLOY_* labels/env，并用 docker compose up -d --no-build --no-deps --force-recreate <service> 重新拉起对应 compose service；其中 met-nonlinear 当前因为 registered Dockerfile 和 long-running service contract 不一致而 live deploy blocked。k3sctl-adapter 是基础设施控制桥，只做 plan/dry-run，真实生产部署需要 supervisor 单独确认。frontend --env prod 使用同一 Compose artifact consumer，流式拉取 unidesk/frontend:<commit>，retag 为 unidesk-frontend 和 unidesk-frontend:<commit>，写入 UNIDESK_FRONTEND_DEPLOY_*，只 recreate frontend service，并验证 image label 与 /health.deploy.commit。frontend --env dev、decision-center、mdtodo、claudeqq 和 dev code-queue 的 k3s 路径会在 D601 上验证 commit image、导入 native k3s containerd、更新 Deployment image/env/annotations，并通过 Kubernetes API service proxy 验证 /health 中的 live commit 与 requested commit；dev frontend 还会在 rollout 前把主 server config.json.auth 同步到 unidesk-dev Secret/ConfigMap。decision-center --env dev 落到 unidesk-dev/decision-center-dev，prod 落到 unidesk/decision-center；mdtodo 和 claudeqq 使用同样的 dev 后 prod k3s consumer 结构。code-queue --env dev 只更新 unidesk-dev 中的 scheduler/read/write/provider-egress-proxy dev Deployments，prod 没有 consumer target。D601 direct Compose consumer 与 k3s-managed consumer 的区别是：前者只接触 D601 Docker/Compose 项目和私有 backend health，不创建 Kubernetes 对象；后者只通过 native k3s Deployment/Service、containerd import 和 Kubernetes API service proxy 验证 live commit。回滚信息通过同一 artifact consumer 的 rollback 字段暴露，提示操作者重新对一个旧 commit 运行相同命令，而不是切回 legacy maintenance-channel 构建。

status 和 health 通过：

bun scripts/cli.ts ssh D601 argv bash -lc '<readonly script>'

只读检查 D601 状态。检查项包括：

  • systemd unit 是否存在、active、enabled。
  • Docker 是否可用，registry container 是否 running。
  • host 端口是否只监听 127.0.0.1:5000 或 [::1]:5000。
  • curl http://127.0.0.1:5000/v2/ 是否返回 200。
  • storage 目录是否存在。
  • 远端 config、compose、unit 的 hash 是否匹配 CLI 渲染结果。
  • 运行镜像是否匹配期望版本。

status 表示只读查询是否成功；未安装时仍可 ok=true 并报告 installed=false。health 表示 registry 是否已按期望运行；未安装或不健康时返回 ok=false。两个只读命令都应输出 decision、retryable、healthyScopes、failedScopes 和 runtimeApiHealthy，方便上层 provider triage 判断局部退化范围。

registry health 的 decision=service-degraded 不等同于 D601 全局离线。特别是当 systemd unit inactive 或 unit hash drift，但 Docker container running、loopback listener 正常、/v2/ 返回 200 时，runtime registry API 仍可用；这种状态应作为 registry 服务治理问题处理，不能覆盖 provider heartbeat、Host SSH、k3sctl-adapter、Code Queue scheduler 或业务 API 的健康证据。

Manual Maintenance

服务采用 systemd + Docker Compose 管理，目的是让 D601 host 管理员可以用标准工具维护：

systemctl status unidesk-artifact-registry.service
systemctl restart unidesk-artifact-registry.service
docker compose -p unidesk-artifact-registry -f /home/ubuntu/.unidesk/artifact-registry/compose.yml ps

手动维护必须遵守 Git-backed deployment truth：运行态可以临时修复，但长期配置应回到 artifact-registry render 的声明文件和本文档。若 D601 runtime 文件 hash 与 CLI 渲染结果不一致，status / health 会显示 mismatch，操作者需要确认这是受控升级还是漂移。

Artifact CD

目标流程是：

  1. D601 artifact registry 已安装并通过 health。
  2. D601 CI 从 pushed Git checkout 构建 unidesk/<service-id>:<commit>。
  3. CI 将镜像 push 到 127.0.0.1:5000/unidesk/<service-id>:<commit>，并记录完整 artifact summary，包括 service id、source repo、source commit、Dockerfile、image ref、tag、digest 和 digest ref。
  4. CD 在运行目标上确认目标 commit/tag/digest 存在。
  5. Compose runtime 通过 provider-gateway Host SSH 从 D601 registry 流式读取 commit-pinned 镜像 tar；D601 k3s runtime 通过 host-local registry/containerd import 消费同一 commit-pinned 镜像。不开放 registry 端口，也不使用第三方镜像托管。
  6. Compose runtime retag 为 Compose 使用的镜像名，并执行 docker compose up -d --no-build --no-deps --force-recreate <service>；k3s runtime 设置 Deployment image/env/annotations 并等待 rollout。
  7. 部署后通过 image label、runtime env、health payload 验证 live commit。

Baidu Netdisk is the first main-server direct user-service sample in this flow. Its dev validation command and prod CD command both consume 127.0.0.1:5000/unidesk/baidu-netdisk:<commit> and must not build source on the master server. Frontend is the standard UniDesk UI artifact sample: CI publishes 127.0.0.1:5000/unidesk/frontend:<commit>, production CD consumes that artifact into the master-server Compose frontend service, and dev CD consumes the same artifact into D601 native k3s frontend-dev. Project Manager, OA Event Flow and Todo Note use the same master-server Compose artifact-consumer shape as Baidu Netdisk, with project-manager-backend, oa-event-flow-backend and todo-note-backend as the runtime containers; Todo Note keeps its external source repository and requires a pre-existing 127.0.0.1:5000/unidesk/todo-note:<commit> artifact. Its consumer injects UNIDESK_TODO_NOTE_DEPLOY_* runtime metadata and the container health probe combines /api/health with that metadata before enforcing deploy.commit / deploy.requestedCommit. Code Queue Manager is supported as an artifact consumer for validation, but prod live apply is supervisor-gated. Neither path may use server rebuild frontend, dirty source, mutable latest, or target-side frontend source builds as release truth. Decision Center, MDTODO and ClaudeQQ follow the same artifact-consumer pattern in both dev and prod, except the runtime target is native k3s on D601 instead of the master-server Compose stack. Dev code-queue is a deliberately narrower consumer: it validates only the unidesk-dev Code Queue execution slice and has no prod target. The k3s consumer must check the registry manifest, pull the commit-pinned image, import it into /run/k3s/containerd/containerd.sock, set the Deployment image to the commit tag, stamp UNIDESK_DEPLOY_* env/annotations, verify health through the Kubernetes API service proxy, and reject an old healthy revision if the live commit or requested commit does not match.

这个 CD 路径必须满足：

  • source commit 来自 pushed Git，不来自 dirty worktree。
  • 镜像 tag 必须 commit-pinned，不能用 mutable latest 作为部署真相。
  • provider-gateway SSH image stream 是临时控制动作，不开放长期公网 registry。
  • CI 可以有较多依赖；CD 只做拉取、retag、recreate 和 live commit 验证。
  • CD 不执行 cargo build、docker build、docker compose build <service> 或任何等价构建动作。
  • k3s-managed artifact consumers must not add NodePort, hostPort, public business ports or provider-gateway direct business backends.
  • artifact-registry deploy-backend-core、server rebuild backend-core 和 server rebuild baidu-netdisk 仍是维护/兼容/非标准路径，不得作为 artifact CD 完成证据。
Reference in New Issue View Git Blame Copy Permalink