feat: expand ci artifact catalog

This commit is contained in:
Codex
2026-05-20 01:20:03 +00:00
parent e0d38d7172
commit 5bb44c9a30
11 changed files with 810 additions and 216 deletions
+2
View File
@@ -8,6 +8,8 @@ Production CI/CD runtime pinning and release-line boundaries follow `docs/refere
The CI-side artifact catalog is root `CI.json`. That file describes only artifact producer inputs and naming; registry consumers still verify the real image labels, manifest digest and live runtime separately. The producer summary contract is owned by `docs/reference/ci.md` and includes `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`.
`CI.json` may also record image-only upstream services as `upstream-image` entries with upstream digest and future mirror naming. Those entries are catalog coverage only until a mirror producer exists. Registry CD must not infer a deployable artifact from an upstream-image entry unless the corresponding D601 registry manifest already exists and a reviewed consumer supports that service.
## Architecture
registry 运行在 D601 host/WSL OS 上,由 systemd 管理 Docker Compose 项目:
+18 -3
View File
@@ -35,7 +35,22 @@ Private repository source authentication is part of the CI contract and follows
## Artifact Catalog And Summary Contract
`CI.json` is the reusable CI artifact catalog. It must remain artifact-only: `serviceId`, `sourceRepo`, `dockerfile`, registry repository naming, tag policy and summary-field semantics are allowed; provider ids, runtime namespaces, ports, compose services, Kubernetes Services, health paths, env, volumes and desired deploy commits are not allowed. `deploy.json` remains the version intent for deployments and must not be replaced by `CI.json`.
`CI.json` is the reusable CI artifact catalog. It must remain artifact-only: `serviceId`, artifact `kind`, producer command, source repository URL, optional repo root, repo-relative Dockerfile path, registry repository naming, upstream image digest/mirror metadata and summary-field semantics are allowed; provider ids, runtime namespaces, ports, compose services, Kubernetes Services, health paths, env, volumes and desired deploy commits are not allowed. `deploy.json` remains the version intent for deployments and must not be replaced by `CI.json`.
`CI.json` schema version 2 uses these artifact kinds:
- `source-build`: CI builds a Dockerfile from a pushed Git commit. UniDesk repo Dockerfiles, external Git repositories and Dockerfiles in repository subdirectories all use this kind.
- `upstream-image`: CI records an image-only service that comes from an upstream image digest and optional D601 mirror rule. It is not a Dockerfile build producer.
Each catalog artifact also has a `status`. `supported` means the matching producer command may start a dry-run or real CI producer action. `blocked` means the service is intentionally listed for coverage but the producer must return a structured blocked result instead of silently building, skipping or falling back. `filebrowser` and `filebrowser-d601` are `upstream-image` blocked entries pinned to `docker.io/filebrowser/filebrowser@sha256:289c5dd677c56662440f26eeb44266ed9746fe563d2e9100f546bff558534d70`; they must not be represented as source-build services.
Current catalog coverage:
- `source-build/supported`: `backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `project-manager`, `oa-event-flow`, `todo-note`, `code-queue-mgr`, `findjob`, `pipeline`, `met-nonlinear`, `k3sctl-adapter`, `mdtodo`, `claudeqq`.
- `source-build/blocked`: `code-queue`.
- `upstream-image/blocked`: `filebrowser`, `filebrowser-d601`.
`publish-user-service` reads `source.repo` and `source.dockerfile` from `CI.json`. The command rejects ad hoc `--repo` overrides; the catalog is the only source for producer build inputs. `publish-backend-core` also reads its producer inputs from `CI.json`, while preserving the dedicated backend-core command and Rust/D601 build boundary.
Every successful image-producing CI task must expose a common `artifactSummary` contract:
@@ -96,11 +111,11 @@ The artifact registry contract and CD consumption path are defined in `docs/refe
## User-Service Artifact Publication
User-service image creation uses the same CI producer boundary as backend-core. Most service identities and Dockerfiles come from the registered `config.json.microservices[]` entry; `frontend` is the reviewed UniDesk UI artifact sample and uses `src/components/frontend/Dockerfile`. The reviewed sample services are `baidu-netdisk`, `decision-center` and `frontend`.
User-service image creation uses the same CI producer boundary as backend-core. Service identities, source repositories, Dockerfiles and image repositories come from root `CI.json`; runtime topology still comes from `config.json`, `deploy.json` and existing manifests. The reviewed sample services are `baidu-netdisk`, `decision-center` and `frontend`, and the catalog now also covers the other source-build services listed above.
The CI user-service artifact task must follow these rules:
- Inputs are a pushed full 40-character Git commit and a registered service id. Dirty worktrees, operator-uploaded source trees and local-only commits are not valid artifact sources.
- Inputs are a pushed full 40-character Git commit and a service id registered in `CI.json`. Dirty worktrees, operator-uploaded source trees, command-line repo overrides and local-only commits are not valid artifact sources.
- D601 prepares a commit-pinned source export under `/home/ubuntu/.unidesk/ci/user-service-artifacts/<service-id>/<commit>` using the existing GitHub SSH deploy identity and node-local provider-gateway WS egress proxy. Tekton consumes that export through a read-only hostPath.
- The image is tagged only with the source commit and pushed to the D601 registry as `127.0.0.1:5000/unidesk/<service-id>:<commit>`. The producer must reject third-party registries and must not publish or consume a mutable `latest` tag.
- The image must carry `unidesk.ai/service-id`, `unidesk.ai/source-repo`, `unidesk.ai/source-commit` and `unidesk.ai/dockerfile` labels.
+65 -14
View File
@@ -1,6 +1,6 @@
# CI/CD Standardization Precheck
# CI/CD Standardization
This document classifies CI/CD paths while UniDesk converges on image-artifact delivery. It is a precheck and guardrail document, not a mass-deletion plan.
This document defines the stable split between CI artifact producers, artifact catalog data, legacy guardrails and CD consumers. Detailed Tekton rules remain in `docs/reference/ci.md`; registry runtime and consumers remain in `docs/reference/artifact-registry.md`; user-service delivery order remains in `docs/reference/user-service-delivery.md`.
## Target Shape
@@ -14,16 +14,66 @@ The standard release shape is:
`backend-core` and D601 `code-queue` may be validated only in dev in this phase. This document must not be used to introduce production deploy validation for either service.
## Artifact Catalog
Root `CI.json` is the CI producer catalog. It is not a deployment manifest.
Allowed catalog data:
- stable `serviceId`;
- artifact `kind`: `source-build` or `upstream-image`;
- producer command: `ci publish-backend-core` or `ci publish-user-service`;
- source repository URL, optional repo root and repo-relative Dockerfile path;
- image repository naming and commit tag policy;
- upstream image digest, upstream source revision and D601 mirror intent for image-only services;
- the required success summary contract.
Forbidden catalog data:
- provider IDs;
- runtime namespace, Compose service, Kubernetes Service or health path;
- ports, environment variables, replicas or volumes;
- desired deployment commits or rollout targets.
Runtime topology belongs to `config.json`, `deploy.json`, existing Kubernetes manifests and the artifact-registry executor.
## Producer Contract
`bun scripts/cli.ts ci publish-user-service --service <id> --commit <full-sha>` reads `source.repo`, `source.dockerfile` and image repository naming from `CI.json`. It rejects command-line repo overrides. Successful source-build producers must label the image with:
- `unidesk.ai/service-id`;
- `unidesk.ai/source-commit`;
- `unidesk.ai/source-repo`;
- `unidesk.ai/dockerfile`.
The successful `artifactSummary` must contain `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`.
`blocked` catalog entries must return a structured blocked result. They must not silently skip, build from a dirty worktree, fall back to target-side maintenance deployment, or mutate production.
## Current Coverage
Supported source-build artifact producers:
- `backend-core` through `ci publish-backend-core`;
- `frontend`, `baidu-netdisk`, `decision-center`, `project-manager`, `oa-event-flow`, `todo-note`, `code-queue-mgr`, `findjob`, `pipeline`, `met-nonlinear`, `k3sctl-adapter`, `mdtodo`, `claudeqq` through `ci publish-user-service`.
Cataloged but blocked:
- `code-queue`: source input is known, but this phase allows only dev image validation and not prod-oriented artifact publication.
- `filebrowser` and `filebrowser-d601`: upstream image-only services pinned to `docker.io/filebrowser/filebrowser@sha256:289c5dd677c56662440f26eeb44266ed9746fe563d2e9100f546bff558534d70`; they need a future upstream mirror producer before CI can publish them.
`code-queue-mgr` is a supported CI producer because the source-build input is known and the remote consumer commit already added a reviewed artifact consumer shape. Its production live apply remains supervisor-gated by deploy/artifact-registry and is not authorized by `CI.json`.
## Upstream Image Consumers
`filebrowser` and `filebrowser-d601` are upstream-image consumers, not source-built UniDesk services.
| Service | Upstream image | Source revision | Catalog home | CI Dockerfile build | Digest / mirror strategy | CD validation |
| --- | --- | --- | --- | --- | --- | --- |
| `filebrowser` | `docker.io/filebrowser/filebrowser:v2.63.3` | `ca5e249e3c0c94159c2136a0cd431a424eb18472` | `CI.json.upstreamImageConsumers[]` and `config.json.microservices[].repository.artifactSource` | forbidden | resolve tag to `docker.io/filebrowser/filebrowser@sha256:<manifest-digest>`, then optionally mirror to `127.0.0.1:5000/upstream/filebrowser/filebrowser@sha256:<manifest-digest>` | pull by digest or mirror digest, verify OCI labels, container image id/digest, and private proxy health |
| `filebrowser-d601` | `docker.io/filebrowser/filebrowser:v2.63.3` | `ca5e249e3c0c94159c2136a0cd431a424eb18472` | `CI.json.upstreamImageConsumers[]` and `config.json.microservices[].repository.artifactSource` | forbidden | same as `filebrowser` | same as `filebrowser` |
| `filebrowser` | `docker.io/filebrowser/filebrowser:v2.63.3` | `ca5e249e3c0c94159c2136a0cd431a424eb18472` | `CI.json.artifacts[]` with `kind=upstream-image` plus `config.json.microservices[].repository.artifactSource` | forbidden | resolve tag to `docker.io/filebrowser/filebrowser@sha256:<manifest-digest>`, then optionally mirror to `127.0.0.1:5000/upstream/filebrowser/filebrowser@sha256:<manifest-digest>` | pull by digest or mirror digest, verify OCI labels, container image id/digest, and private proxy health |
| `filebrowser-d601` | `docker.io/filebrowser/filebrowser:v2.63.3` | `ca5e249e3c0c94159c2136a0cd431a424eb18472` | `CI.json.artifacts[]` with `kind=upstream-image` plus `config.json.microservices[].repository.artifactSource` | forbidden | same as `filebrowser` | same as `filebrowser` |
The current precheck could inspect the locally cached image labels and image id, but Docker Hub and registry HTTP requests timed out from this container. Therefore the catalog records `digestPin.status=pending-network-verification`; rollout must remain blocked until a reachable registry path resolves the manifest digest and records the mirror digest. The locally cached image shows `org.opencontainers.image.version=2.63.3`, `org.opencontainers.image.revision=ca5e249e3c0c94159c2136a0cd431a424eb18472`, `linux/amd64`, and image id `sha256:6a4d051140ef9313ad87b443f55ccb1cd6331e7463b4becbec2174b494ea533c`, but a local image id is not a registry digest pin.
The catalog records the resolved upstream digest for the current image. If a future tag refresh cannot resolve the registry manifest digest, rollout must remain blocked until a reachable registry path resolves the manifest digest and records the mirror digest. A local Docker image id is supporting evidence only and not a registry digest pin.
### Upstream Image Evidence
@@ -31,11 +81,10 @@ The catalog expression is intentionally minimal and parseable:
| Evidence command | Required result shape |
| --- | --- |
| `jq '.upstreamImageConsumers[] | {serviceId, upstreamImageRef, digestPin, mirrorStrategy, ciBuild, pullOnlyCdValidation}' CI.json` | both File Browser services show `upstreamImageRef=docker.io/filebrowser/filebrowser:v2.63.3`, `digestPin.required=true`, `digestPin.status=pending-network-verification`, `mirrorStrategy.mode=mirror-after-digest-verification`, `ciBuild.dockerfileBuild=false`, and `publishCommand=null` |
| `jq '.artifacts[] | select(.kind=="upstream-image") | {serviceId, upstream, status}' CI.json` | both File Browser services show `upstream.imageRef=docker.io/filebrowser/filebrowser:v2.63.3`, a sha256 `upstream.digestRef`, `sourceRevision=ca5e249e3c0c94159c2136a0cd431a424eb18472`, mirror intent under `upstream/filebrowser/filebrowser`, and `status=blocked` |
| `bun scripts/cli.ts config show` with the File Browser `artifactSource` projection | both services parse as `kind=upstream-image`, `digestPinRequired=true`, `mirrorRepository=127.0.0.1:5000/upstream/filebrowser/filebrowser`, `ciDockerfileBuild=false`, and `pullOnlyCd=true` |
| `docker image inspect filebrowser/filebrowser:v2.63.3` | local cache evidence may show the image id and OCI labels for version `2.63.3` and revision `ca5e249e3c0c94159c2136a0cd431a424eb18472`; this is not a registry digest |
| `docker manifest inspect --verbose docker.io/filebrowser/filebrowser:v2.63.3` | must resolve an upstream manifest digest before rollout; when the registry request times out, rollout remains blocked and the catalog stays at `pending-network-verification` |
| `bun scripts/cli.ts ci publish-user-service --service filebrowser --commit <full-sha> --dry-run` | returns `ok=false` with the message that File Browser is an upstream image consumer and must not be built by Dockerfile CI |
| `docker manifest inspect --verbose docker.io/filebrowser/filebrowser:v2.63.3` | must resolve the upstream manifest digest before rollout; if the registry request times out, rollout remains blocked |
| `bun scripts/cli.ts ci publish-user-service --service filebrowser --commit <full-sha> --dry-run` | returns `ok=false` with `status=blocked`, upstream digest/mirror metadata, and no Dockerfile source build |
The digest/mirror dry-run contract is:
@@ -70,8 +119,8 @@ Pull-only CD validation must be expressed as concrete checks:
## Guardrails Added
- Upstream-image services are represented in `CI.json.upstreamImageConsumers[]` and in `config.json.microservices[].repository.artifactSource`; they are explicitly outside `CI.json.artifacts[]`.
- `ci publish-user-service` rejects registered `upstream-image` services instead of trying to interpret `repository.dockerfile` as a source Dockerfile.
- Upstream-image services are represented in `CI.json.artifacts[]` with `kind=upstream-image` and in `config.json.microservices[].repository.artifactSource`; they are explicitly outside source-build producers.
- `ci publish-user-service` returns a structured blocked result for registered `upstream-image` services instead of trying to interpret `repository.dockerfile` as a source Dockerfile.
- Local-manifest production deploy for reviewed artifact consumers is blocked before source materialization/build, so prod cannot silently fall back to target-side source build or a dirty worktree.
- `artifact-registry deploy-backend-core` is demoted to a structured deprecated result; backend-core production CD must enter through `deploy apply --env prod`.
@@ -79,7 +128,7 @@ Pull-only CD validation must be expressed as concrete checks:
| Guardrail name / result key | Command evidence | Legacy path covered | Deletion status |
| --- | --- | --- | --- |
| `upstream-image` CI publish rejection | `bun scripts/cli.ts ci publish-user-service --service filebrowser --commit <full-sha> --dry-run` returns `ok=false` and says not to build an upstream image consumer | File Browser accidentally entering `CI.json.artifacts[]` or Dockerfile CI | keep; deletion of docker-run repair waits for digest/mirror CD |
| `upstream-image` CI publish rejection | `bun scripts/cli.ts ci publish-user-service --service filebrowser --commit <full-sha> --dry-run` returns `ok=false`, `status=blocked`, and upstream metadata | File Browser accidentally entering Dockerfile CI | keep; deletion of docker-run repair waits for digest/mirror CD |
| `prod-artifact-consumer-local-manifest-blocked` | `bun scripts/cli.ts deploy apply --file deploy.json --service frontend --dry-run` returns `ok=false`, this error key, and points to `deploy apply --env prod --service <service-id> --commit <full-sha>` | prod source-build fallback for reviewed artifact consumers | keep; local manifest mode may still be needed for non-prod/recovery until runbooks are replaced |
| `artifact-registry deploy-backend-core` deprecated result | `bun scripts/cli.ts artifact-registry deploy-backend-core --commit <full-sha>` returns `ok=false`, `deprecated=true`, and replacement `deploy apply --env prod --service backend-core --commit <full-sha>` | backend-core prod CD bypassing deploy reconciler guardrails | keep name only as compatibility until all callers stop using it |
| prod unsupported result for services without artifact consumers | `deploy apply --env prod --service <unsupported-service> --dry-run` must return unsupported instead of falling back to source build | target-side source build/maintenance-channel prod deploy | keep disabled until service-specific artifact consumers exist |
@@ -87,9 +136,11 @@ Pull-only CD validation must be expressed as concrete checks:
The guarded-but-not-deletable paths are: `server rebuild backend-core`, `server rebuild frontend`, `server rebuild baidu-netdisk`, provider-gateway protected upgrade, native k3s bootstrap, k3sctl-adapter bridge repair, File Browser provider-local docker-run repair, and D601 dev/backend target-side rollout. They remain because they are bootstrap, recovery, diagnostic, or controlled dev paths; deleting them requires replacement runbooks or reviewed artifact consumers.
## Not Removed Yet
## Safety Boundary
Bootstrap and repair paths remain because they still protect recovery: native k3s initialization, provider-gateway protected upgrade, k3sctl-adapter control bridge repair, main-server Compose maintenance rebuilds, and File Browser docker-run operations. These paths must be replaced by reviewed artifact consumers or explicit recovery runbooks before deletion.
CI may build images, push to the D601 loopback registry and report immutable digests. CI must not run production CD, call `deploy apply` for production, mutate production namespaces, recreate production Compose services or update `deploy.json`.
backend-core and D601 `code-queue` remain restricted to dev image validation in this phase. Any future production rollout for them must be implemented as an explicit CD consumer change, not as a CI producer side effect.
## Validation Boundary
+1 -1
View File
@@ -144,7 +144,7 @@ The registry contract is defined in `docs/reference/artifact-registry.md`; the C
## Upstream Image Exception
`filebrowser` and `filebrowser-d601` are not source-built UniDesk services and must not be moved into `CI.json.artifacts[]`. Their minimal catalog expression is `CI.json.upstreamImageConsumers[]` plus `config.json.microservices[].repository.artifactSource`:
`filebrowser` and `filebrowser-d601` are not source-built UniDesk services and must not be modeled as Dockerfile producers. Their minimal catalog expression is `CI.json.artifacts[]` entries with `kind=upstream-image` plus `config.json.microservices[].repository.artifactSource`:
- upstream image: `docker.io/filebrowser/filebrowser:v2.63.3`;
- upstream source revision: `ca5e249e3c0c94159c2136a0cd431a424eb18472`;
+8 -5
View File
@@ -34,9 +34,10 @@ The default release flow for a user-service change is:
- No user-service artifact may rely on a third-party registry as source of truth.
- No production deploy may rebuild the source from a dirty worktree.
- Commit-pinned image tags are the deployment truth; mutable `latest` tags are not.
- Root `CI.json` is an artifact catalog only. It can list user-service CI build inputs such as `serviceId`, `sourceRepo`, `dockerfile`, image repository naming and the required artifact summary fields; it must not carry runtime topology or replace `deploy.json`.
- The standard CI artifact producer is `bun scripts/cli.ts ci publish-user-service --service <id> --commit <full-sha>`. It accepts only a pushed Git commit and a registered service id, and reports `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`.
- Root `CI.json` is an artifact catalog only. It lists CI producer inputs such as `serviceId`, artifact kind, source repository, repo-relative Dockerfile, image repository naming, upstream image digest/mirror metadata and the required artifact summary fields; it must not carry runtime topology or replace `deploy.json`.
- The standard CI artifact producer is `bun scripts/cli.ts ci publish-user-service --service <id> --commit <full-sha>`. It accepts only a pushed Git commit and a service id registered in `CI.json`, reads `source.repo` and `source.dockerfile` from that catalog, rejects ad hoc `--repo` overrides, and reports `serviceId`, `sourceCommit`, `sourceRepo`, `dockerfile`, `imageRef`, `tag`, `digest` and `digestRef`.
- The CI artifact producer is not a deploy executor. It must not mutate the production namespace, restart production services, or update `deploy.json`.
- `CI.json` may list `blocked` source-build entries when the source input is known but the publish/CD boundary is not yet reviewed. It may also list `upstream-image` entries for image-only services such as File Browser; those entries pin upstream digest and mirror intent but must not be treated as Dockerfile builds.
- Every production release must finish with a manual acceptance step after the automated checks pass.
- Multi-service delivery programs may use Code Queue parallelization, but the supervisor must follow `docs/reference/code-queue-supervision.md`: tasks need self-contained prompts, isolated worktrees, bounded queue concurrency, explicit acceptance evidence, and infrastructure defects split into separate follow-up tasks when they block several lanes.
@@ -44,11 +45,13 @@ The default release flow for a user-service change is:
Some registered user services are intentionally upstream-image consumers instead of source-built services. `filebrowser` and `filebrowser-d601` are in this class.
- They must be cataloged as upstream images, not as `CI.json.artifacts[]` Dockerfile producers.
- `ci publish-user-service` must reject them; there is no UniDesk Dockerfile build for these services.
- They must be cataloged as `CI.json.artifacts[]` entries with `kind=upstream-image`, not as Dockerfile `source-build` producers.
- `ci publish-user-service` must return a structured blocked result for them; there is no UniDesk Dockerfile build for these services.
- The release input is an upstream manifest digest or a digest-verified mirror in the D601 registry, not a Git commit tag built by Tekton.
- CD must be pull-only and must verify the image identity, OCI labels and service health through the UniDesk private proxy.
- Until the upstream digest has been resolved and mirrored or pinned, File Browser remains a recovery/diagnostic image-only path rather than a standard release path.
- Until the upstream digest has been resolved and mirrored or pinned for a future mirror producer, File Browser remains a recovery/diagnostic image-only path rather than a standard release path.
The current catalog covers `frontend`, `baidu-netdisk`, `decision-center`, `project-manager`, `oa-event-flow`, `todo-note`, `code-queue-mgr`, `findjob`, `pipeline`, `met-nonlinear`, `k3sctl-adapter`, `mdtodo` and `claudeqq` as supported `publish-user-service` source-build services. `code-queue` is cataloged but blocked by the D601 dev/prod boundary. `filebrowser` and `filebrowser-d601` are cataloged as pinned upstream images, not source builds.
## Frontend Pairing