mirrors/pikasTech-unidesk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: add dev backend-core artifact consumer

Browse Source
This commit is contained in:
Codex
2026-05-21 09:02:16 +00:00
parent 9977f00621
commit add3b8d3f2
14 changed files with 238 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/deploy.md
+10 -9
View File
@@ -40,9 +40,9 @@ The root `deploy.json` is the single desired-state source for both prod and dev.
The optional non-service execution declaration under `environments.dev` is intentionally not specified here. The only currently allowed declaration is `ci`, and its authoritative `repo`, `scriptPath`, `timeoutMs`, short launcher, host fetch boundary and no-CD rules are defined only in `docs/reference/dev-ci-runner.md`.
Environment mode never reads the local dirty working tree manifest. `deploy check --env ...`, `deploy plan --env ...` and `deploy apply --env ...` fetch `origin/master`, read `origin/master:deploy.json`, select `environments.<env>`, and report the manifest commit/blob, service commit IDs, target namespace, database fingerprint and Provider identity. `deploy apply --env dev` is currently enabled for persistent D601 dev `backend-core` target-side rollout and for reviewed artifact consumers `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq`, dev-only `code-queue`, `project-manager`, `oa-event-flow`, `code-queue-mgr`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear`. `deploy apply --env prod` exposes reviewed registry artifact consumers (`backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq`, `project-manager`, `oa-event-flow`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear`), while `code-queue` must report unsupported, `code-queue-mgr` remains supervisor-gated and `k3sctl-adapter` is plan/dry-run only. Production backend-core artifact CD is a separate executor because its build target is D601 CI while its runtime target is the master server. The default user-service delivery policy, including CI build, registry publication, dev validation, production CD and manual acceptance, is documented in `docs/reference/user-service-delivery.md`.
Environment mode never reads the local dirty working tree manifest. `deploy check --env ...`, `deploy plan --env ...` and `deploy apply --env ...` fetch `origin/master`, read `origin/master:deploy.json`, select `environments.<env>`, and report the manifest commit/blob, service commit IDs, target namespace, database fingerprint and Provider identity. `deploy apply --env dev` is currently enabled for reviewed artifact consumers `backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq`, dev-only `code-queue`, `project-manager`, `oa-event-flow`, `code-queue-mgr`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear`. `deploy apply --env prod` exposes reviewed registry artifact consumers (`backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq`, `project-manager`, `oa-event-flow`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear`), while `code-queue` must report unsupported, `code-queue-mgr` remains supervisor-gated and `k3sctl-adapter` is plan/dry-run only. Backend-core artifact CD is a pull-only consumer in both dev and prod; the build target is D601 CI, while dev runtime target is D601 native k3s and prod runtime target is the master server Compose stack. The default user-service delivery policy, including CI build, registry publication, dev validation, production CD and manual acceptance, is documented in `docs/reference/user-service-delivery.md`.
`--commit <full-sha>` is allowed only with `--env dev|prod --service <id>` for reviewed artifact consumers. It overrides the selected service commit for that one artifact consumer while still using the Git-backed environment manifest for target, namespace, repo, deploy ref and guardrails. It is the supported temporary shape for release-line frontend validation and rollback when the artifact was produced from a pushed `release/v1` commit but `origin/master:deploy.json#environments.<env>.services.frontend.commitId` has not been repinned. It must not be used for local-manifest mode, multi-service apply, or target-side source-build services such as dev `backend-core`.
`--commit <full-sha>` is allowed only with `--env dev|prod --service <id>` for reviewed artifact consumers. It overrides the selected service commit for that one artifact consumer while still using the Git-backed environment manifest for target, namespace, repo, deploy ref and guardrails. It is the supported temporary shape for release-line frontend/backend-core validation and rollback when the artifact was produced from a pushed `release/v1` commit but `origin/master:deploy.json#environments.<env>.services.<id>.commitId` has not been repinned. It must not be used for local-manifest mode, multi-service apply, or target-side source-build services.
For services with reviewed production artifact consumers, local-manifest `deploy apply --file ...` is not a production fallback. The CLI blocks `backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq` and other reviewed pull-only consumers before source materialization or Docker build and directs operators to `deploy apply --env prod --service <id> --commit <full-sha>`. This prevents a dirty worktree, local manifest or target-side source build from bypassing the pull-only artifact CD guardrails. The broader precheck and legacy-path classification live in `docs/reference/cicd-standardization.md`.
@@ -87,7 +87,7 @@ Phase 3 introduces the dev backend/frontend manifest at `src/components/microser
`backend-core-dev` must use `unidesk-dev-runtime-config` and `unidesk-dev-runtime-secrets`, connect to `postgres-dev.../unidesk_dev`, expose HTTP on 8080 and provider ingress on 8081, and write logs under `/var/log/unidesk-dev`. `frontend-dev` must set `CORE_INTERNAL_URL=http://backend-core-dev.unidesk-dev.svc.cluster.local:8080` and must not proxy to production backend-core.
The manifest keeps placeholder image tags and deploy commit values in source control. The controlled `deploy apply --env dev --service backend-core` path fetches `origin/master:deploy.json`, materializes the requested source commit on D601, narrows the dev core control manifest to the selected Service/Deployment pair, replaces placeholders with the requested commit and dev image tag, builds on D601, imports the image into native k3s containerd, applies only the `unidesk-dev` objects and stamps the Deployment. `deploy apply --env dev --service frontend` uses the same selected dev manifest objects but consumes the existing D601 registry artifact `127.0.0.1:5000/unidesk/frontend:<commit>` instead of building frontend source on the target. Decision Center, MDTODO and ClaudeQQ use the same dev namespace but follow the D601 registry artifact consumer path instead of a source build: each verifies the commit-pinned image in D601 registry, imports it into native k3s containerd, applies its dev manifest, stamps the Deployment and verifies live commit/requestedCommit through the Kubernetes API service proxy. `project-manager`, `oa-event-flow`, `code-queue-mgr`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear` consume existing D601 registry artifacts for direct Docker/Compose validation rather than separate parallel k3s dev instances; `code-queue-mgr` live prod apply remains supervisor-gated. Client dry-run and static validation remain useful checks before controlled apply:
The manifest keeps placeholder image tags and deploy commit values in source control. The controlled `deploy apply --env dev --service backend-core` path consumes the existing D601 registry artifact `127.0.0.1:5000/unidesk/backend-core:<commit>` produced by `ci publish-backend-core`; it does not compile Rust or build a Docker image during CD. Backend-core and frontend use the same selected dev core manifest objects: CD verifies the commit-pinned registry image and labels, imports the artifact into native k3s containerd, applies only the selected `unidesk-dev` objects, stamps the Deployment, and verifies live commit/requestedCommit through the Kubernetes API service proxy. Decision Center, MDTODO and ClaudeQQ use the same dev namespace and D601 registry artifact consumer path. `project-manager`, `oa-event-flow`, `code-queue-mgr`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear` consume existing D601 registry artifacts for direct Docker/Compose validation rather than separate parallel k3s dev instances; `code-queue-mgr` live prod apply remains supervisor-gated. Client dry-run and static validation remain useful checks before controlled apply:
- `bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply --dry-run=client --validate=false -f src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`
@@ -112,11 +112,11 @@ Production `code-queue-mgr` is a separate main-server Compose sidecar artifact c
`bun scripts/cli.ts deploy plan --env dev [--service <id>]` reads `origin/master:deploy.json#environments.dev` and prints a dry-run environment plan without checking or mutating live runtime resources. `deploy check --env dev` uses the same dry-run environment plan. `--env prod` is available for parity as a dry-run planning path; it reads `origin/master:deploy.json#environments.prod` and must not use a dirty local `deploy.json`.
Environment plan output must be sufficient to review the artifact matrix without running a live apply. Each service item includes `deploymentPath`, `artifactConsumer.consumerKind`, `artifactConsumer.registryImage`, `artifactConsumer.noRuntimeSourceBuild`, `artifactConsumer.dryRunOnly`, `target`, `validation` and `liveApply` where relevant. `consumerKind=d601-direct-compose` means the reviewed consumer touches only the D601 Docker/Compose service and private health path; `consumerKind=d601-k3s-managed` means the reviewed consumer imports the artifact into native k3s/containerd and verifies through the Kubernetes API service proxy; `consumerKind=main-server-compose` means the reviewed consumer streams or loads the D601 artifact into the main-server Compose service; `consumerKind=d601-dev-target-side-build` is reserved for the controlled dev backend-core source-build exception. Artifact consumer plan items must explicitly report `noRuntimeSourceBuild=true` and list forbidden build/public exposure actions. Blocked or gated services must keep structured `dryRunOnly` / `blockedReason` output, for example `met-nonlinear` `runtime-verification-blocked` and `k3sctl-adapter` supervisor-only production apply.
Environment plan output must be sufficient to review the artifact matrix without running a live apply. Each service item includes `deploymentPath`, `artifactConsumer.consumerKind`, `artifactConsumer.registryImage`, `artifactConsumer.registry`, `artifactConsumer.source`, `artifactConsumer.build`, `artifactConsumer.noRuntimeSourceBuild`, `artifactConsumer.dryRunOnly`, `target`, `validation` and `liveApply` where relevant. `consumerKind=d601-direct-compose` means the reviewed consumer touches only the D601 Docker/Compose service and private health path; `consumerKind=d601-k3s-managed` means the reviewed consumer imports the artifact into native k3s/containerd and verifies through the Kubernetes API service proxy; `consumerKind=main-server-compose` means the reviewed consumer streams or loads the D601 artifact into the main-server Compose service; `consumerKind=d601-dev-target-side-build` is retained only as a legacy classification and should not appear for backend-core. Artifact consumer plan items must explicitly report `noRuntimeSourceBuild=true`, expose registry/source/build boundaries including digest provenance, and list forbidden build/public exposure actions. Blocked or gated services must keep structured `dryRunOnly` / `blockedReason` output, for example `met-nonlinear` `runtime-verification-blocked` and `k3sctl-adapter` supervisor-only production apply.
For `--env dev --service code-queue`, the environment plan must also expose a `boundary` block that separates the CI producer from the dev CD consumer. CI is allowed to publish only `127.0.0.1:5000/unidesk/code-queue:<commit>` plus digest/label evidence. DEV CD may consume that artifact only for `unidesk-dev` Code Queue scheduler/read/write/provider-egress-proxy objects after an operator reviews the dry-run. For `--env prod --service code-queue`, the service item must remain `deploymentPath=unsupported`, `artifactConsumer.consumerKind=unsupported`, `target.deployCommandShape=none` and `liveApply.allowed=false`; it must not expose production k3s as an executable target. The prod boundary must state that production Code Queue CD needs a future supervisor-approved design and that this runner cannot self-deploy, mutate the production namespace, restart scheduler/runner, or interrupt/cancel tasks.
`bun scripts/cli.ts deploy apply [--file deploy.json | --env dev|prod] [--service <id>] [--commit <full-sha>] [--dry-run] [--force]` starts an asynchronous job only for supported targets. Use `bun scripts/cli.ts job status <jobId> --tail-bytes 30000` to observe progress. `--dry-run` resolves the same plan but does not build or replace runtime objects. `--force` rebuilds even when the live commit matches. Environment apply is not the dev e2e trigger; use `bun scripts/cli.ts ci run-dev-e2e` for the Git-controlled temporary namespace smoke flow. `--env dev` apply is enabled for persistent D601 `backend-core` target-side rollout and for `frontend`/`baidu-netdisk`/`decision-center`/`mdtodo`/`claudeqq`/dev-only `code-queue`/`project-manager`/`oa-event-flow`/`code-queue-mgr`/`todo-note`/`findjob`/`pipeline`/`met-nonlinear` artifact consumers. `--env prod` apply exposes the D601 registry artifact consumer for `backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq`, `project-manager`, `oa-event-flow`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear`; `code-queue-mgr` prod live apply is supervisor-gated and `k3sctl-adapter` is plan/dry-run only. `--commit` may override one selected reviewed artifact consumer in either dev or prod, for example `deploy apply --env dev --service frontend --commit <release-v1-full-sha>`, and the image must already exist as `127.0.0.1:5000/unidesk/<service-id>:<commit>`. Unsupported prod services, especially `code-queue`, return a structured `unsupported` payload instead of silently falling back to a maintenance-channel source build.
`bun scripts/cli.ts deploy apply [--file deploy.json | --env dev|prod] [--service <id>] [--commit <full-sha>] [--dry-run] [--force]` starts an asynchronous job only for supported targets. Use `bun scripts/cli.ts job status <jobId> --tail-bytes 30000` to observe progress. `--dry-run` resolves the same plan but does not build or replace runtime objects. `--force` redeploys even when the live commit matches. Environment apply is not the dev e2e trigger; use `bun scripts/cli.ts ci run-dev-e2e` for the Git-controlled temporary namespace smoke flow. `--env dev` apply is enabled for `backend-core`/`frontend`/`baidu-netdisk`/`decision-center`/`mdtodo`/`claudeqq`/dev-only `code-queue`/`project-manager`/`oa-event-flow`/`code-queue-mgr`/`todo-note`/`findjob`/`pipeline`/`met-nonlinear` artifact consumers. `--env prod` apply exposes the D601 registry artifact consumer for `backend-core`, `frontend`, `baidu-netdisk`, `decision-center`, `mdtodo`, `claudeqq`, `project-manager`, `oa-event-flow`, `todo-note`, `findjob`, `pipeline` and `met-nonlinear`; `code-queue-mgr` prod live apply is supervisor-gated and `k3sctl-adapter` is plan/dry-run only. `--commit` may override one selected reviewed artifact consumer in either dev or prod, for example `deploy apply --env dev --service backend-core --commit <full-sha>` or `deploy apply --env dev --service frontend --commit <release-v1-full-sha>`, and the image must already exist as `127.0.0.1:5000/unidesk/<service-id>:<commit>`. Unsupported prod services, especially `code-queue`, return a structured `unsupported` payload instead of silently falling back to a maintenance-channel source build.
All deploy commands output JSON. Long operations must use `.state/jobs/` and bounded log tails; no deploy path may succeed with missing progress output.
@@ -131,16 +131,17 @@ Target-side build is the standard deployment mode. The controller may run on the
The reconciler distributes only repository URL, commit ID, Dockerfile path, build context and the existing deployment manifest/compose declaration. It must not distribute large Docker images between hosts as the default path, and it must not accept `docker commit` images, dirty worktrees or hand-mutated runtime containers as deployment truth.
Each target fetches the remote repository, resolves the requested commit to a full 40 character SHA and exports tracked files with `git archive`. Build contexts are created from that archive, not from the operator's current working tree. Environment applies such as `deploy apply --env dev` must not upload Kubernetes manifests or source files from the master server worktree; the target-side materialized commit is the source for Dockerfile, build context and k3s control manifests. The master server side may only do lightweight CLI orchestration, environment ref reading and remote command dispatch.
Target-side source-build services fetch the remote repository, resolve the requested commit to a full 40 character SHA and export tracked files with `git archive`. Build contexts are created from that archive, not from the operator's current working tree. Artifact consumers such as dev/prod backend-core do not materialize source during CD; they use the Git-backed environment manifest only for commit intent, target selection and deploy ref metadata. The master server side may only do lightweight CLI orchestration, environment ref reading and remote command dispatch.
## Artifact Consumer Exception
Production backend-core and reviewed user-service samples are explicit exceptions to standard target-side build. The runtime target can be the master server Compose stack, but the build target is D601 CI; CD then consumes only commit-pinned images from the D601 artifact registry.
Backend-core and reviewed user-service samples are explicit exceptions to standard target-side build. The runtime target can be the master server Compose stack or D601 native k3s dev namespace, but the build target is D601 CI; CD then consumes only commit-pinned images from the D601 artifact registry.
The exception is narrow:
- CI on D601 builds `src/components/backend-core/Dockerfile` from a pushed commit, stamps image labels and publishes `127.0.0.1:5000/unidesk/backend-core:<commit>` to the D601 artifact registry.
- CD on the master server pulls that existing image through the controlled artifact-registry relay, retags it for the Compose service, recreates only `backend-core` with `--no-build --no-deps --force-recreate`, and verifies the running commit.
- Dev CD on D601 imports that existing image into native k3s containerd, sets `backend-core-dev` to the commit image, stamps deploy metadata, and verifies `/health.deploy.commit` and `deploy.requestedCommit`.
- Prod CD on the master server pulls that existing image through the controlled artifact-registry relay, retags it for the Compose service, recreates only `backend-core` with `--no-build --no-deps --force-recreate`, and verifies the running commit.
- CD must not run Rust compilation, Docker build, Compose build or `server rebuild backend-core`.
- The legacy `artifact-registry deploy-backend-core` compatibility entry is deprecated and disabled as a standard entrypoint; use `deploy apply --env prod --service backend-core --commit <full-sha>` so the common artifact-consumer guardrails execute first.
- The pushed Git commit remains the version source of truth. The image registry is a content cache and transfer boundary, not a replacement for `deploy.json` or Git.
@@ -191,7 +192,7 @@ The reconciler selects the executor from `config.json`:
- `deployment.mode=internal-sidecar` on `main-server`: use the same main-server target-side source export, Docker build, image label stamping, fixed Compose project replacement and live commit verification as direct Compose services. This class is for private sidecars such as `code-queue-mgr`; it is still versioned by `deploy.json.commitId`, not by the operator's current worktree, and prod live apply remains supervisor-gated.
- `deployment.mode=unidesk-direct` on a provider: this executor is disabled for D601 service deployment. The historical behavior dispatched `host.ssh` to the provider, built on the provider, then used the service's provider-local compose file and project; that shape must not remain a second deployment control plane.
- Control bridges that UniDesk needs in order to inspect or repair an orchestrator must stay in this direct class. In particular, `k3sctl-adapter` is a UniDesk-managed bridge to native k3s and must remain outside k3s; Docker packaging on Docker Desktop/WSL must create an explicit host-local bridge, currently an adapter-container SSH local tunnel, to reach `/etc/rancher/k3s/k3s.yaml` and WSL `127.0.0.1:6443`.
- `deployment.mode=k3sctl-managed`: the target behavior is to build on the active control target unless the service has a reviewed artifact-consumer exception, verify native k3s on the host OS/WSL distro, import the image into native k3s/containerd, apply the existing Kubernetes manifest, stamp the Deployment and wait for rollout. On D601, persistent dev apply is currently allowed for `backend-core` target-side build plus `frontend`, `decision-center`, `mdtodo`, `claudeqq` and dev-only `code-queue` artifact consumption in `unidesk-dev`; production artifact consumers are limited to reviewed services and exclude Code Queue. Normal production services still cannot use a maintenance-channel direct rollout. The executor must use the native kubeconfig and containerd socket, for example `/etc/rancher/k3s/k3s.yaml` and `/run/k3s/containerd/containerd.sock`; running k3s itself in Docker is forbidden for both control-plane and worker nodes. A `rancher/k3s` image or legacy container may only be used as a temporary artifact source during migration, and any active containerized k3s control plane must be stopped before verification succeeds. The executor must preload a valid `rancher/mirrored-pause:3.6` sandbox image into native k3s containerd through the provider-gateway one-shot egress path, verify its entrypoint is `/pause`, and reject fake or sleep-based replacement images. k3s-managed deploys must use ClusterIP Services and Kubernetes API service proxy health checks; they must not add NodePort, hostPort, public business ports or provider-gateway direct business backends.
- `deployment.mode=k3sctl-managed`: the target behavior is to build on the active control target unless the service has a reviewed artifact-consumer exception, verify native k3s on the host OS/WSL distro, import the image into native k3s/containerd, apply the existing Kubernetes manifest, stamp the Deployment and wait for rollout. On D601, persistent dev apply uses artifact consumption for `backend-core`, `frontend`, `decision-center`, `mdtodo`, `claudeqq` and dev-only `code-queue` in `unidesk-dev`; production artifact consumers are limited to reviewed services and exclude Code Queue. Normal production services still cannot use a maintenance-channel direct rollout. The executor must use the native kubeconfig and containerd socket, for example `/etc/rancher/k3s/k3s.yaml` and `/run/k3s/containerd/containerd.sock`; running k3s itself in Docker is forbidden for both control-plane and worker nodes. A `rancher/k3s` image or legacy container may only be used as a temporary artifact source during migration, and any active containerized k3s control plane must be stopped before verification succeeds. The executor must preload a valid `rancher/mirrored-pause:3.6` sandbox image into native k3s containerd through the provider-gateway one-shot egress path, verify its entrypoint is `/pause`, and reject fake or sleep-based replacement images. k3s-managed deploys must use ClusterIP Services and Kubernetes API service proxy health checks; they must not add NodePort, hostPort, public business ports or provider-gateway direct business backends.
D601 Docker local images are not the source of truth for k3s runtime availability. For Code Queue, the deploy gate must verify `unidesk-code-queue:d601` exists in native k3s containerd after import with `ctr --address /run/k3s/containerd/containerd.sock -n k8s.io images ls`, and it must fail before rollout if the tag is missing. The same gate must verify every production Code Queue Deployment that uses the image (`code-queue`, `code-queue-read`, `code-queue-write`, `d601-provider-egress-proxy`, `d601-tcp-egress-gateway`) still references exactly `unidesk-code-queue:d601`; otherwise kubelet may attempt an external registry pull and leave base gateways in `ImagePullBackOff`.