feat: enable dev core deploy apply

This commit is contained in:
Codex
2026-05-18 00:53:11 +00:00
parent bfaea963ad
commit b605c78875
7 changed files with 312 additions and 38 deletions
+1 -1
View File
@@ -36,7 +36,7 @@ UniDesk 是一个以主 server 为统一入口的分布式工作平台;本文
- `bun scripts/cli.ts ssh <providerId> [ssh-like args...]`:通过 provider-gateway 的 Host SSH / WSL SSH 维护桥打开近似原生 ssh 的交互会话或远端命令,并在远端 PATH 注入 `apply_patch``glob``skill-discover``apply-patch``py``skills`、结构化 `find``glob``argv` 子命令用于避免远端补丁、Python stdin、skill 发现与常用只读命令的嵌套转义问题,使用规则见 `docs/reference/cli.md``docs/reference/provider-gateway.md`
- `bun scripts/cli.ts microservice list/status/health/diagnostics/tunnel-self-test/proxy`:管理和验证挂载在主 server、计算节点 Docker 或 k3s 控制面上的用户服务,`proxy` 支持受控 JSON bodyOA Event Flow/Todo Note/Baidu Netdisk/Code Queue Manager on main-server、k3s Control/Code Queue 执行面/MDTODO/Decision Center/FindJob/Pipeline/MET Nonlinear on D601 的规则见 `docs/reference/microservices.md`
- `bun scripts/cli.ts decision upload/list/show/health`:通过 backend-core 用户服务代理上传会议记录/决议 Markdown、列出记录和查看详情;Decision Center 运行在 D601 k3s,规则见 `docs/reference/microservices.md`
- `bun scripts/cli.ts deploy check/plan/apply [--file deploy.json] [--service <id>]`:按根目录 `deploy.json` 的服务 repo 和 commit 期望状态校验或更新用户服务,目标侧自行 fetch、构建、部署和 live commit 验证;规则见 `docs/reference/deploy.md`
- `bun scripts/cli.ts deploy check/plan/apply [--file deploy.json|--env dev|prod] [--service <id>]`:按根目录或固定环境 ref 的服务 repo 和 commit 期望状态校验或更新用户服务`--env dev` 当前只开放 backend-core/frontend 开发环境部署,目标侧自行 fetch、构建、部署和 live commit 验证;规则见 `docs/reference/deploy.md`
- `bun scripts/cli.ts dev-env validate [--manifest path] [--kubectl-dry-run]` / `dev-env prewarm-images`:离线校验 D601 `unidesk-dev` 生产隔离护栏,或把开发底座基础镜像预热到 D601 原生 k3s containerd,规则见 `docs/reference/deploy.md``docs/reference/microservices.md`
- `bun scripts/cli.ts ci install/status/run/logs`:在 D601 原生 k3s 上安装和运行 Tekton CI,只做每 commit 检查和 Code Queue 只读性能门禁,不部署 CD;规则见 `docs/reference/ci.md`
- `bun scripts/cli.ts codex deploy <commitId>`:Code Queue 兼容部署入口,会生成临时 desired manifest 并调用 `deploy apply --service code-queue` 的同一条 target-side build 与 live commit 验证路径;规则见 `docs/reference/codex-deploy.md`
+2 -1
View File
@@ -190,6 +190,7 @@
- dev frontend `/health` 返回 ok,并且只代理到 dev backend-core。
- dev backend/frontend 重部署期间,生产 `bun scripts/cli.ts server status` 仍健康。
- 重建 dev backend/frontend 不触碰主 server Docker Compose 容器。
- 当前实施切片:`deploy apply --env dev --service backend-core|frontend` 先支持 dev core 两个服务;`deploy/dev` 只保存 `deploy.json`,不保存 k8s manifest 或源码。
## 阶段 4code-queue-mgr-dev
@@ -342,4 +343,4 @@
5. Phase 7:把生产部署迁移到 `deploy/prod`
6. Phase 8:强化操作员和 LLM 安全检查。
第一个里程碑完成条件:`deploy apply --env dev` 可以根据 `origin/deploy/dev:deploy.json` 声明的 commit id,把 backend-core、frontend、code-queue-mgr 以及 Code Queue read/write/scheduler 部署进 `unidesk-dev`;反复 dev redeploy 不改变生产主 server status,也不改变生产 Code Queue state。
第一个完整里程碑完成条件:`deploy apply --env dev` 可以根据 `origin/deploy/dev:deploy.json` 声明的 commit id,把 backend-core、frontend、code-queue-mgr 以及 Code Queue read/write/scheduler 部署进 `unidesk-dev`;反复 dev redeploy 不改变生产主 server status,也不改变生产 Code Queue state。阶段 3 的阶段性里程碑先以 `--service backend-core``--service frontend` 分别部署成功为准。
+1 -1
View File
@@ -21,7 +21,7 @@ UniDesk 的统一 CLI 入口是根目录 `scripts/cli.ts`,运行方式固定
- `ssh <providerId> skills [--scope all|wsl|windows] [--limit N]` 发现目标节点上的 WSL/Linux skill 根目录;当 provider 是 WSL 时同一次调用还会扫描 Windows 用户目录下的 `.agents/skills``.codex/skills`
- `microservice list/status/health/diagnostics/tunnel-self-test/proxy` 通过 backend-core 内网 API 管理挂载在计算节点 Docker 或 k3s 控制面中的用户服务(底层命令名仍为 microservice);`health``diagnostics``tunnel-self-test``proxy` 会走真实 backend-core -> provider-gateway 或 k3sctl-adapter -> 节点服务链路,`proxy` 支持受控 JSON 请求体并对超大响应 body 默认输出有界预览,规则见 `docs/reference/microservices.md`
- `decision upload/list/show/health` 通过 backend-core 用户服务代理访问 D601 k3s Decision Center,用于上传会议记录/决议 Markdown、列出权威记录、查看详情和健康检查;它不得直连 D601 Service、NodePort 或 provider-gateway 业务 HTTP。
- `deploy check/plan/apply` 默认从根目录 `deploy.json` 读取服务 repo 与 commit 期望状态,join `config.json` 和现有 manifest 后使用 target-side build 单一路径校验或更新直管服务与 k3s 代管服务;`deploy plan --env dev|prod` 在 Phase 0 只从固定 Git ref 读取 manifest 并输出 dry-run 环境计划,不使用本地 dirty worktree;规则见 `docs/reference/deploy.md`
- `deploy check/plan/apply` 默认从根目录 `deploy.json` 读取服务 repo 与 commit 期望状态,join `config.json` 和现有 manifest 后使用 target-side build 单一路径校验或更新直管服务与 k3s 代管服务;`deploy plan --env dev|prod` 只从固定 Git ref 读取 manifest 并输出 dry-run 环境计划,不使用本地 dirty worktree`deploy apply --env dev --service backend-core|frontend` 可按 `origin/deploy/dev:deploy.json` 部署第一版 dev core`--env prod` apply 仍禁用;规则见 `docs/reference/deploy.md`
- `dev-env validate [--manifest path] [--kubectl-dry-run]` 离线校验 D601 `unidesk-dev` namespace、dev PostgreSQL 底座和 dev backend/frontend manifest。默认检查 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-foundation.k8s.yaml`;也可显式校验 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`。所有 namespaced 对象必须只落到 `unidesk-dev`foundation manifest 必须包含 `postgres-dev` StatefulSet/Service、dev secret/config、迁移 Job 和 DB URL guardcore manifest 必须包含 `backend-core-dev`/`frontend-dev` Deployment/Service。加 `--kubectl-dry-run` 时额外执行 `kubectl apply --dry-run=client --validate=false -f <manifest>`,仍不 apply 资源。
- `dev-env prewarm-images [--image image] [--provider-id D601] [--no-pull] [--proxy-url URL] [--pull-timeout-ms N] [--dry-run]` 创建异步 job,通过 UniDesk SSH 维护桥在 D601 上把开发底座依赖镜像从 Docker 缓存导入原生 k3s containerd。默认镜像是 `postgres:16-alpine``rancher/mirrored-library-busybox:1.36.1`,用于避免 `postgres-dev` 与 local-path helper pod 卡在外部 registry 拉取。该命令固定验证 `/etc/rancher/k3s/k3s.yaml` 指向的 native k3s 上下文,并输出 `dev_env_containerd_image_ready=...` 作为成功判据;它不 apply manifest、不修改生产 `unidesk` namespace。
- `codex deploy <commitId>` 是 Code Queue 兼容部署入口,会生成临时 desired manifest 并调用 `deploy apply --service code-queue` 的同一条 target-side build、k3s import、rollout 和 live commit 验证路径;详细规则见 `docs/reference/codex-deploy.md`
+6 -4
View File
@@ -29,7 +29,9 @@ Environment mode never reads the local working tree manifest. The mapping is fix
- `dev -> origin/deploy/dev`
- `prod -> origin/deploy/prod`
The current Phase 0 implementation enables only dry-run `check` and `plan` for `--env`. It fetches the fixed ref, reads `deploy.json` from that ref, validates the declared environment, and reports the manifest commit/blob, service commit IDs, target namespace, database fingerprint and Provider identity. `deploy apply --env ...` is intentionally rejected until the dev infrastructure executors exist.
`deploy check --env ...` and `deploy plan --env ...` fetch the fixed ref, read `deploy.json` from that ref, validate the declared environment, and report the manifest commit/blob, service commit IDs, target namespace, database fingerprint and Provider identity without mutating runtime resources. `deploy apply --env dev` is enabled only for the first dev-core slice, currently `backend-core` and `frontend`. If no `--service` is given and the dev manifest still includes unsupported later-stage services such as Code Queue, the command fails before changing runtime resources. `deploy apply --env prod` remains disabled until the production environment executor and authorization policy are explicitly added.
The `deploy/dev` and `deploy/prod` branches are environment desired-state branches, not source branches. They should contain only `deploy.json`; Kubernetes manifests, Dockerfiles and executor code continue to live on `master` and are selected through the commit IDs declared in the environment manifest.
`config.json.microservices[].repository.commitId` is retained for catalog compatibility, but `deploy.json` is the deployment version authority for the reconciler.
@@ -60,7 +62,7 @@ Phase 3 introduces the dev backend/frontend manifest at `src/components/microser
`backend-core-dev` must use `unidesk-dev-runtime-config` and `unidesk-dev-runtime-secrets`, connect to `postgres-dev.../unidesk_dev`, expose HTTP on 8080 and provider ingress on 8081, and write logs under `/var/log/unidesk-dev`. `frontend-dev` must set `CORE_INTERNAL_URL=http://backend-core-dev.unidesk-dev.svc.cluster.local:8080` and must not proxy to production backend-core.
The manifest uses placeholder image tags and deploy commit values until `deploy apply --env dev` supports target-side dev builds. A controller or operator must replace those placeholders from `origin/deploy/dev:deploy.json` before real rollout. Client dry-run and static validation are the required checks before any controlled apply:
The manifest keeps placeholder image tags and deploy commit values in source control. `deploy apply --env dev --service backend-core|frontend` fetches `origin/deploy/dev:deploy.json`, materializes the requested source commit on D601, copies the dev core control manifest, narrows it to the selected Service/Deployment pair, replaces placeholders with the requested commit and dev image tag, builds on D601, imports the image into native k3s containerd, applies only the `unidesk-dev` objects and stamps the Deployment. Client dry-run and static validation are the required checks before any controlled apply:
- `bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply --dry-run=client --validate=false -f src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`
@@ -73,9 +75,9 @@ backend-core and frontend keep their production health payload shape by default.
`bun scripts/cli.ts deploy plan [--file deploy.json] [--service <id>]` prints the same live state plus the intended action: `noop`, `deploy` or `unsupported`.
`bun scripts/cli.ts deploy plan --env dev [--service <id>]` reads `origin/deploy/dev:deploy.json` and prints a dry-run environment plan without checking or mutating live runtime resources. `deploy check --env dev` uses the same Phase 0 dry-run path. `--env prod` is available for parity but is also dry-run only in Phase 0; it reads `origin/deploy/prod:deploy.json` and must not use a dirty local `deploy.json`.
`bun scripts/cli.ts deploy plan --env dev [--service <id>]` reads `origin/deploy/dev:deploy.json` and prints a dry-run environment plan without checking or mutating live runtime resources. `deploy check --env dev` uses the same dry-run environment plan. `--env prod` is available for parity as a dry-run planning path; it reads `origin/deploy/prod:deploy.json` and must not use a dirty local `deploy.json`.
`bun scripts/cli.ts deploy apply [--file deploy.json] [--service <id>] [--dry-run] [--force]` starts an asynchronous job. Use `bun scripts/cli.ts job status <jobId> --tail-bytes 30000` to observe progress. `--dry-run` resolves the same plan but does not build or replace runtime objects. `--force` rebuilds even when the live commit matches.
`bun scripts/cli.ts deploy apply [--file deploy.json | --env dev] [--service <id>] [--dry-run] [--force]` starts an asynchronous job. Use `bun scripts/cli.ts job status <jobId> --tail-bytes 30000` to observe progress. `--dry-run` resolves the same plan but does not build or replace runtime objects. `--force` rebuilds even when the live commit matches. Environment apply is currently limited to `--env dev --service backend-core` and `--env dev --service frontend`; `--env prod` apply is rejected.
All deploy commands output JSON. Long operations must use `.state/jobs/` and bounded log tails; no deploy path may succeed with missing progress output.
+1 -9
View File
@@ -54,7 +54,7 @@ function help(): unknown {
{ command: "decision upload <markdown-file> [--title text] [--type meeting|decision] [--level G0|G1|G2|G3|P0|P1|P2|P3|none] [--status active|blocked|parked|done] [--linked-goal-id id] [--evidence url]", description: "Upload a meeting note or decision record through backend-core -> decision-center user-service proxy." },
{ command: "decision list [--type ...] [--status ...] [--level ...] [--linked-goal-id id] [--limit N]", description: "List Decision Center records through the user-service proxy." },
{ command: "decision show <id>", description: "Show one Decision Center record." },
{ command: "deploy check|plan|apply [--file deploy.json|--env dev|prod] [--service id] [--dry-run] [--force]", description: "Reconcile services from a repo+commit manifest; --env uses fixed environment refs for dry-run planning in Phase 0." },
{ command: "deploy check|plan|apply [--file deploy.json|--env dev|prod] [--service id] [--dry-run] [--force]", description: "Reconcile services from a repo+commit manifest; --env reads fixed environment refs and can apply supported dev services." },
{ command: "dev-env validate|prewarm-images", description: "Validate D601 unidesk-dev guardrails or prewarm dev foundation images into native k3s containerd through a bounded async job." },
{ command: "schedule list|get|runs|run|delete", description: "Manage backend-core scheduled tasks and run history; schedule run <id> supports --wait-ms N." },
{ command: "schedule upsert-pgdata-backup [--time HH:MM] [--remote-base /SERVER_DATA/UNIDESK_PG_DATA]", description: "Create or update the daily PGDATA physical backup task that uploads monthly rotated archives to Baidu Netdisk." },
@@ -149,14 +149,6 @@ async function main(): Promise<void> {
return;
}
if (top === "deploy" && (args.includes("--env") || args.includes("--environment"))) {
const result = await runDeployCommand(null, args.slice(1));
const ok = (result as { ok?: unknown }).ok !== false;
emitJson(commandName, result, ok);
if (!ok) process.exitCode = 1;
return;
}
if (top === "dev-env") {
const result = runDevEnvCommand(args.slice(1));
const ok = (result as { ok?: unknown }).ok !== false;
+257 -20
View File
@@ -131,6 +131,7 @@ const nativeK3sInstallVersion = "v1.34.1+k3s1";
const nativeK3sImage = "rancher/k3s:v1.34.1-k3s1";
const nativeK3sCtrAddress = "/run/k3s/containerd/containerd.sock";
const unideskRepoUrl = "https://github.com/pikasTech/unidesk";
const devApplySupportedServiceIds = new Set(["backend-core", "frontend"]);
const deployEnvironmentTargets: Record<DeployEnvironment, DeployEnvironmentTarget> = {
dev: {
environment: "dev",
@@ -184,7 +185,7 @@ function deployHelp(action: string | undefined = undefined): Record<string, unkn
usage: {
check: "bun scripts/cli.ts deploy check [--file deploy.json | --env dev|prod] [--service id]",
plan: "bun scripts/cli.ts deploy plan [--file deploy.json | --env dev|prod] [--service id]",
apply: "bun scripts/cli.ts deploy apply [--file deploy.json] [--service id] [--dry-run] [--force] [--timeout-ms N] [--run-now]",
apply: "bun scripts/cli.ts deploy apply [--file deploy.json | --env dev] [--service id] [--dry-run] [--force] [--timeout-ms N] [--run-now]",
},
actions: {
check: "Validate desired repo+commit state against live service health and commit markers.",
@@ -193,7 +194,7 @@ function deployHelp(action: string | undefined = undefined): Record<string, unkn
},
options: [
{ name: "--file <path>", default: defaultDeployFile, description: "Desired-state manifest path relative to the repo root. JSON and ESM JS manifests are supported, for example deploy.json or develop.js." },
{ name: "--env <dev|prod>", description: "Read deploy.json from the fixed environment ref: dev=origin/deploy/dev, prod=origin/deploy/prod. Phase 0 supports check/plan only." },
{ name: "--env <dev|prod>", description: "Read deploy.json from the fixed environment ref: dev=origin/deploy/dev, prod=origin/deploy/prod. Apply is currently enabled for supported dev services only." },
{ name: "--service <id>", description: "Limit reconcile to one service from the manifest." },
{ name: "--dry-run", description: "Prepare and validate without mutating the target service." },
{ name: "--force", description: "Redeploy even when the live commit appears up to date." },
@@ -567,7 +568,96 @@ function frontendCoreDeployService(config: UniDeskConfig): UniDeskMicroserviceCo
};
}
function coreDeployService(config: UniDeskConfig, id: string): UniDeskMicroserviceConfig | undefined {
function devCoreDeployService(id: string): UniDeskMicroserviceConfig | undefined {
const specs: Record<string, {
name: string;
description: string;
dockerfile: string;
composeService: string;
containerName: string;
nodeBaseUrl: string;
nodePort: number;
healthPath: string;
route: string;
allowedMethods: string[];
allowedPathPrefixes: string[];
}> = {
"backend-core": {
name: "UniDesk Dev Backend Core",
description: "Isolated dev backend-core deployed into D601 native k3s namespace unidesk-dev.",
dockerfile: "src/components/backend-core/Dockerfile",
composeService: "backend-core-dev",
containerName: "k3s:backend-core-dev",
nodeBaseUrl: "k3s://backend-core-dev",
nodePort: 8080,
healthPath: "/health",
route: "/dev/backend-core",
allowedMethods: ["GET", "HEAD", "POST", "PUT", "PATCH", "DELETE"],
allowedPathPrefixes: ["/", "/api/", "/logs"],
},
frontend: {
name: "UniDesk Dev Frontend",
description: "Isolated dev frontend deployed into D601 native k3s namespace unidesk-dev.",
dockerfile: "src/components/frontend/Dockerfile",
composeService: "frontend-dev",
containerName: "k3s:frontend-dev",
nodeBaseUrl: "k3s://frontend-dev",
nodePort: 8080,
healthPath: "/health",
route: "/dev/frontend",
allowedMethods: ["GET", "HEAD"],
allowedPathPrefixes: ["/"],
},
};
const spec = specs[id];
if (spec === undefined) return undefined;
return {
id,
name: spec.name,
providerId: "D601",
description: spec.description,
repository: {
url: unideskRepoUrl,
commitId: "deploy-dev",
dockerfile: spec.dockerfile,
composeFile: "src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml",
composeService: spec.composeService,
containerName: spec.containerName,
},
backend: {
nodeBaseUrl: spec.nodeBaseUrl,
nodeBindHost: `k3s://unidesk-dev/${spec.composeService}`,
nodePort: spec.nodePort,
proxyMode: "dev-k3s-direct",
frontendOnly: true,
public: false,
allowedMethods: spec.allowedMethods,
allowedPathPrefixes: spec.allowedPathPrefixes,
healthPath: spec.healthPath,
timeoutMs: 30_000,
},
deployment: {
mode: "k3sctl-managed",
adapterServiceId: "k3sctl-adapter",
k3sServiceId: spec.composeService,
namespace: "unidesk-dev",
expectedNodeIds: ["D601"],
activeNodeId: "D601",
},
development: {
providerId: "D601",
sshPassthrough: true,
worktreePath: `/home/ubuntu/unidesk-dev-core-deploy/${id}`,
},
frontend: {
route: spec.route,
integrated: false,
},
};
}
function coreDeployService(config: UniDeskConfig, id: string, environment: DeployEnvironment | null): UniDeskMicroserviceConfig | undefined {
if (environment === "dev") return devCoreDeployService(id);
if (id === "frontend") return frontendCoreDeployService(config);
return undefined;
}
@@ -576,6 +666,12 @@ function isCoreDeployService(service: UniDeskMicroserviceConfig): boolean {
return service.id === "frontend" && service.providerId === "main-server" && service.backend.proxyMode === "core-direct";
}
function isDevK3sCoreService(service: UniDeskMicroserviceConfig): boolean {
return service.deployment.mode === "k3sctl-managed"
&& service.deployment.namespace === deployEnvironmentTargets.dev.namespace
&& devApplySupportedServiceIds.has(service.id);
}
function isDirectComposeDeployMode(service: UniDeskMicroserviceConfig): boolean {
return service.deployment.mode === "unidesk-direct" || service.deployment.mode === "internal-sidecar";
}
@@ -588,7 +684,14 @@ function selectServices(config: UniDeskConfig, manifest: DeployManifest, service
const selected = serviceId === null ? manifest.services : manifest.services.filter((service) => service.id === serviceId);
if (serviceId !== null && selected.length === 0) throw new Error(`deploy manifest does not contain service: ${serviceId}`);
return selected.map((desired) => {
const service = configById.get(desired.id) ?? coreDeployService(config, desired.id);
if (manifest.environment === "dev") {
const service = devCoreDeployService(desired.id);
if (service === undefined) {
throw new Error(`deploy --env dev service ${desired.id} is not enabled in this executor yet; currently supported: ${[...devApplySupportedServiceIds].join(", ")}`);
}
return { desired, config: service };
}
const service = configById.get(desired.id) ?? coreDeployService(config, desired.id, manifest.environment);
if (service === undefined) throw new Error(`deploy manifest service ${desired.id} is not present in config.json microservices or supported core deploy services`);
return { desired, config: service };
});
@@ -643,6 +746,7 @@ function targetExportDir(service: UniDeskMicroserviceConfig, runId: string): str
}
function targetWorkDir(service: UniDeskMicroserviceConfig): string {
if (isDevK3sCoreService(service)) return service.development.worktreePath;
if (service.deployment.mode === "k3sctl-managed") return k3sDeployDir;
if (targetIsMain(service) && isUnideskRepo(service.repository.url)) {
return rootPath(".state", "deploy", "work", safeId(service.id));
@@ -676,6 +780,7 @@ function sourceBuildContext(service: UniDeskMicroserviceConfig): string {
}
function buildImageTag(service: UniDeskMicroserviceConfig): string {
if (isDevK3sCoreService(service)) return `unidesk-${service.id}:dev`;
if (service.deployment.mode === "k3sctl-managed") return `unidesk-${service.id}:d601`;
if (targetIsMain(service)) {
if (["project-manager", "baidu-netdisk", "oa-event-flow"].includes(service.repository.composeService)) return service.repository.composeService;
@@ -706,6 +811,7 @@ function directDockerfileOverride(service: UniDeskMicroserviceConfig): string {
function k8sManifestPath(service: UniDeskMicroserviceConfig): string {
const composeFile = service.repository.composeFile;
if (composeFile.endsWith(".k8s.yaml")) return composeFile;
if (!composeFile.endsWith(".k3s.json")) throw new Error(`${service.id} k3s service composeFile must point to *.k3s.json`);
return composeFile.replace(/\.k3s\.json$/u, ".k8s.yaml");
}
@@ -916,8 +1022,24 @@ function claudeqqDeployAssetOverlayCommands(): string[] {
}
function syncK8sControlManifestsScript(service: UniDeskMicroserviceConfig): string {
if (isDevK3sCoreService(service)) {
const manifest = k8sManifestPath(service);
if (!existsSync(rootPath(manifest))) throw new Error(`${service.id} dev k3s control manifest missing: ${manifest}`);
const encoded = Buffer.from(readFileSync(rootPath(manifest), "utf8"), "utf8").toString("base64");
return [
"set -euo pipefail",
`target_root=${shellQuote(targetWorkDir(service))}`,
`relative_path=${shellQuote(manifest)}`,
"target_file=\"$target_root/$relative_path\"",
"mkdir -p \"$(dirname \"$target_file\")\"",
`printf %s ${shellQuote(encoded)} | base64 -d > "$target_file"`,
"printf 'synced_k3s_control_manifest=%s\\n' \"$target_file\"",
].join("\n");
}
if (service.deployment.mode !== "k3sctl-managed" || isUnideskRepo(service.repository.url)) return "";
const manifests = [service.repository.composeFile, k8sManifestPath(service)];
const manifests = service.repository.composeFile.endsWith(".k8s.yaml")
? [service.repository.composeFile]
: [service.repository.composeFile, k8sManifestPath(service)];
const commands = [
"set -euo pipefail",
`target_root=${shellQuote(targetWorkDir(service))}`,
@@ -970,7 +1092,14 @@ function buildImageScript(service: UniDeskMicroserviceConfig, desired: DeployMan
sourceProxyPrelude(service),
`image=${shellQuote(image)}`,
`dockerfile=${shellQuote(dockerfile)}`,
"docker buildx version >/dev/null",
"if ! docker buildx version >/dev/null 2>&1; then",
" if ! docker buildx inspect default >/dev/null 2>&1; then echo target_build_builder=missing >&2; exit 1; fi",
"fi",
...(isDevK3sCoreService(service) ? [
"if ! docker image inspect oven/bun:1-alpine >/dev/null 2>&1; then",
" docker pull oven/bun:1-alpine",
"fi",
] : []),
"builder_args=()",
"if docker buildx inspect --builder default >/dev/null 2>&1; then builder_args=(--builder default); echo target_build_builder=default; else echo target_build_builder=implicit; fi",
"docker buildx inspect \"${builder_args[@]}\" --bootstrap || true",
@@ -981,6 +1110,54 @@ function buildImageScript(service: UniDeskMicroserviceConfig, desired: DeployMan
].filter((line) => line.length > 0).join("\n");
}
function patchDevK3sCoreManifestScript(service: UniDeskMicroserviceConfig, desired: DeployManifestService, resolvedCommit: string): string {
const manifest = `${targetWorkDir(service)}/${k8sManifestPath(service)}`;
const image = buildImageTag(service);
const serviceId = service.id;
const deploymentName = service.repository.composeService;
const manifestServiceId = deploymentName;
return [
"set -euo pipefail",
`manifest=${shellQuote(manifest)}`,
`service_id=${shellQuote(serviceId)}`,
`deployment_name=${shellQuote(deploymentName)}`,
`image=${shellQuote(image)}`,
`repo=${shellQuote(desired.repo)}`,
`commit=${shellQuote(resolvedCommit)}`,
`requested_commit=${shellQuote(desired.commitId)}`,
`manifest_service_id=${shellQuote(manifestServiceId)}`,
`python3 - "$manifest" "$service_id" "$deployment_name" "$image" "$repo" "$commit" "$requested_commit" "$manifest_service_id" <<'PY'`,
"import re",
"import sys",
"path, service_id, deployment_name, image, repo, commit, requested_commit, manifest_service_id = sys.argv[1:]",
"text = open(path, encoding='utf-8').read()",
"segments = re.split(r'(?m)^---\\s*$', text)",
"kept = []",
"for segment in segments:",
" if not segment.strip():",
" continue",
" if f'\\n name: {deployment_name}\\n' in ('\\n' + segment + '\\n'):",
" kept.append(segment)",
"if not kept:",
" raise SystemExit(f'deployment/service {deployment_name} not found in {path}')",
"patched = []",
"for segment in kept:",
" segment = segment.replace('unidesk.ai/image-source: deploy-dev-commit', 'unidesk.ai/image-source: deploy-env-commit')",
" segment = re.sub(r'image: unidesk-[^\\n]+:dev-placeholder', f'image: {image}', segment)",
" segment = re.sub(r'value: replace-with-deploy-dev-commit', f'value: {commit}', segment)",
" segment = segment.replace(f'value: {manifest_service_id}', f'value: {service_id}')",
" patched.append(segment.strip() + '\\n')",
"out = '---\\n'.join(patched)",
"open(path, 'w', encoding='utf-8').write(out)",
"print(f'dev_k3s_manifest_patched={path}')",
"print(f'dev_k3s_manifest_service={service_id}')",
"print(f'dev_k3s_manifest_deployment={deployment_name}')",
"print(f'dev_k3s_manifest_image={image}')",
"print(f'dev_k3s_manifest_commit={commit}')",
"PY",
].join("\n");
}
function directComposeResolveScript(service: UniDeskMicroserviceConfig): string {
const projectHint = targetIsMain(service) ? "unidesk" : "";
return [
@@ -1322,6 +1499,10 @@ function cleanupLegacyDirectCodeQueueScript(service: UniDeskMicroserviceConfig):
].join("\n");
}
function k8sNamespaceForService(service: UniDeskMicroserviceConfig): string {
return service.deployment.namespace ?? k8sNamespace;
}
function k8sDeploymentsForService(service: UniDeskMicroserviceConfig): string[] {
if (service.id === "code-queue") return ["d601-provider-egress-proxy", "d601-tcp-egress-gateway", "code-queue", "code-queue-read", "code-queue-write"];
return [service.repository.composeService];
@@ -1329,11 +1510,12 @@ function k8sDeploymentsForService(service: UniDeskMicroserviceConfig): string[]
function applyK8sScript(service: UniDeskMicroserviceConfig): string {
const manifest = `${targetWorkDir(service)}/${k8sManifestPath(service)}`;
const namespace = k8sNamespaceForService(service);
const cleanup = service.id === "code-queue"
? [
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} delete endpointslice d601-provider-egress-proxy --ignore-not-found`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} delete deployment code-queue-d518 --ignore-not-found`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} delete service code-queue-d518 --ignore-not-found`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} delete endpointslice d601-provider-egress-proxy --ignore-not-found`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} delete deployment code-queue-d518 --ignore-not-found`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} delete service code-queue-d518 --ignore-not-found`,
].join("\n")
: "";
return [
@@ -1344,6 +1526,7 @@ function applyK8sScript(service: UniDeskMicroserviceConfig): string {
function stampK8sScript(service: UniDeskMicroserviceConfig, desired: DeployManifestService, resolvedCommit: string): string {
const deployments = k8sDeploymentsForService(service).map((name) => `deployment/${name}`);
const namespace = k8sNamespaceForService(service);
const envPairs = [
`UNIDESK_DEPLOY_SERVICE_ID=${service.id}`,
`UNIDESK_DEPLOY_REPO=${desired.repo}`,
@@ -1362,9 +1545,9 @@ function stampK8sScript(service: UniDeskMicroserviceConfig, desired: DeployManif
];
return [
"set -euo pipefail",
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} set env ${deployments.map(shellQuote).join(" ")} ${envPairs.map(shellQuote).join(" ")}`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} annotate ${deployments.map(shellQuote).join(" ")} ${annotatePairs.map(shellQuote).join(" ")} --overwrite`,
`actual=$(KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} get deployment ${shellQuote(service.repository.composeService)} -o jsonpath='{.metadata.annotations.unidesk\\.ai/deploy-commit}')`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} set env ${deployments.map(shellQuote).join(" ")} ${envPairs.map(shellQuote).join(" ")}`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} annotate ${deployments.map(shellQuote).join(" ")} ${annotatePairs.map(shellQuote).join(" ")} --overwrite`,
`actual=$(KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} get deployment ${shellQuote(service.repository.composeService)} -o jsonpath='{.metadata.annotations.unidesk\\.ai/deploy-commit}')`,
`test "$actual" = ${shellQuote(resolvedCommit)}`,
"printf 'k8s_deploy_commit=%s\\n' \"$actual\"",
].join("\n");
@@ -1372,18 +1555,20 @@ function stampK8sScript(service: UniDeskMicroserviceConfig, desired: DeployManif
function rolloutK8sScript(service: UniDeskMicroserviceConfig): string {
const deployments = k8sDeploymentsForService(service);
const namespace = k8sNamespaceForService(service);
return [
"set -euo pipefail",
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} rollout restart ${deployments.map((name) => shellQuote(`deployment/${name}`)).join(" ")}`,
...deployments.map((name) => `KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} rollout status ${shellQuote(`deployment/${name}`)} --timeout=180s`),
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} get deploy ${deployments.map(shellQuote).join(" ")} -o wide`,
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} rollout restart ${deployments.map((name) => shellQuote(`deployment/${name}`)).join(" ")}`,
...deployments.map((name) => `KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} rollout status ${shellQuote(`deployment/${name}`)} --timeout=180s`),
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} get deploy ${deployments.map(shellQuote).join(" ")} -o wide`,
].join("\n");
}
function k8sCommitProbeScript(service: UniDeskMicroserviceConfig): string {
const namespace = k8sNamespaceForService(service);
return [
"set -euo pipefail",
`commit=$(KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(k8sNamespace)} get deployment ${shellQuote(service.repository.composeService)} -o jsonpath='{.metadata.annotations.unidesk\\.ai/deploy-commit}' 2>/dev/null || true)`,
`commit=$(KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n ${shellQuote(namespace)} get deployment ${shellQuote(service.repository.composeService)} -o jsonpath='{.metadata.annotations.unidesk\\.ai/deploy-commit}' 2>/dev/null || true)`,
"printf '%s\\n' \"$commit\"",
].join("\n");
}
@@ -1444,10 +1629,41 @@ async function directHttpJson(url: string, timeoutMs: number): Promise<unknown>
}
}
function devK3sServiceHealthScript(service: UniDeskMicroserviceConfig): string {
const namespace = k8sNamespaceForService(service);
const deployment = service.repository.composeService;
const path = service.backend.healthPath;
return [
"set -euo pipefail",
`namespace=${shellQuote(namespace)}`,
`deployment=${shellQuote(deployment)}`,
`path=${shellQuote(path)}`,
`pod=$(KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n "$namespace" get pod -l app.kubernetes.io/name=${shellQuote(service.id)},unidesk.ai/environment=dev -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)`,
"if [ -z \"$pod\" ]; then echo '{\"ok\":false,\"error\":\"dev deployment pod not found\"}'; exit 0; fi",
`KUBECONFIG=${shellQuote(k8sKubeconfig)} kubectl -n "$namespace" exec "$pod" -- wget -q -T 5 -O - "http://127.0.0.1:8080$path"`,
].join("\n");
}
async function serviceHealth(config: UniDeskConfig, service: UniDeskMicroserviceConfig): Promise<unknown> {
if (isCoreDeployService(service)) {
return await directHttpJson(`${service.backend.nodeBaseUrl}${service.backend.healthPath}`, service.backend.timeoutMs);
}
if (isDevK3sCoreService(service)) {
const result = await runTargetCommand(config, service, devK3sServiceHealthScript(service), "/home/ubuntu", 30_000, 20_000);
let body: unknown = null;
try {
body = result.stdout.trim().length > 0 ? JSON.parse(result.stdout.trim()) : null;
} catch {
body = { text: result.stdout.trim() };
}
return {
ok: result.ok,
status: result.ok ? 200 : 502,
body,
raw: result.raw,
error: result.ok ? undefined : result.stderr || result.stdout || "dev k3s service health failed",
};
}
return coreInternalFetch(`/api/microservices/${encodeURIComponent(service.id)}/health`);
}
@@ -1699,6 +1915,7 @@ async function step(
}
async function readDockerImageCommit(config: UniDeskConfig, service: UniDeskMicroserviceConfig): Promise<string | null> {
if (isDevK3sCoreService(service)) return null;
const result = await runTargetCommand(config, service, dockerCommitProbeScript(service), targetIsMain(service) ? repoRoot : "/home/ubuntu", 30_000, 20_000);
const commit = parseFullCommit(result.stdout);
return commit.length > 0 ? commit : null;
@@ -1824,7 +2041,7 @@ async function applyOneService(config: UniDeskConfig, service: UniDeskMicroservi
const runId = `${Date.now().toString(36)}-${Math.random().toString(16).slice(2, 8)}`;
const exportDir = targetExportDir(service, runId);
if (service.id === "code-queue" && !targetIsMain(service)) {
if (!targetIsMain(service) && isUnideskRepo(desired.repo)) {
const identity = await ensureGithubSshIdentityStep(config, service);
if (!pushStep(steps, identity)) return { ok: false, serviceId: service.id, startedAt, finishedAt: nowIso(), before, steps };
}
@@ -1846,6 +2063,11 @@ async function applyOneService(config: UniDeskConfig, service: UniDeskMicroservi
if (!pushStep(steps, controlManifestSync)) return { ok: false, serviceId: service.id, startedAt, finishedAt: nowIso(), resolvedCommit, before, steps };
}
if (isDevK3sCoreService(service)) {
const patchManifest = await step(config, service, "patch-dev-k3s-manifest", patchDevK3sCoreManifestScript(service, desired, resolvedCommit), targetWorkDir(service), 60_000, false);
if (!pushStep(steps, patchManifest)) return { ok: false, serviceId: service.id, startedAt, finishedAt: nowIso(), resolvedCommit, before, steps };
}
const buildScript = isDirectComposeDeployMode(service)
? buildDirectImageScript(service, desired, resolvedCommit)
: buildImageScript(service, desired, resolvedCommit);
@@ -1965,6 +2187,12 @@ function environmentDryRunPlan(
};
}
function unsupportedDevApplyServices(manifest: DeployManifest, serviceId: string | null): string[] {
if (manifest.environment !== "dev") return [];
const services = serviceId === null ? manifest.services : manifest.services.filter((service) => service.id === serviceId);
return services.map((service) => service.id).filter((id) => !devApplySupportedServiceIds.has(id));
}
async function runApplyNow(config: UniDeskConfig, manifest: DeployManifest, options: DeployOptions): Promise<Record<string, unknown>> {
const selected = selectServices(config, manifest, options.serviceId);
const startedAt = nowIso();
@@ -1989,7 +2217,8 @@ async function runApplyNow(config: UniDeskConfig, manifest: DeployManifest, opti
function applyJob(config: UniDeskConfig, args: string[], options: DeployOptions): Record<string, unknown> {
const runArgs = args.includes("--run-now") ? args : [...args, "--run-now"];
const command = [process.execPath, rootPath("scripts", "cli.ts"), "deploy", ...runArgs];
const job = startJob("deploy_apply", command, `Reconcile services from ${options.file}${options.serviceId === null ? "" : ` service=${options.serviceId}`}`);
const source = options.environment === null ? options.file : `${deployEnvironmentTargets[options.environment].gitRef}:deploy.json`;
const job = startJob("deploy_apply", command, `Reconcile services from ${source}${options.serviceId === null ? "" : ` service=${options.serviceId}`}`);
return {
ok: true,
mode: "async-job",
@@ -2008,9 +2237,17 @@ export async function runDeployCommand(config: UniDeskConfig | null, args: strin
const action = actionRaw as DeployAction;
const options = parseOptions(args.slice(1));
if (options.environment !== null) {
if (action === "apply") throw new Error("deploy apply --env is not enabled in Phase 0; use deploy plan --env dev|prod for dry-run only");
const { manifest, source } = readEnvironmentDeployManifest(options.environment);
return environmentDryRunPlan(manifest, source, options, action);
if (action === "check" || action === "plan") return environmentDryRunPlan(manifest, source, options, action);
if (options.environment !== "dev") throw new Error("deploy apply --env prod is not enabled yet");
const unsupported = unsupportedDevApplyServices(manifest, options.serviceId);
if (unsupported.length > 0) {
throw new Error(`deploy apply --env dev currently supports only ${[...devApplySupportedServiceIds].join(", ")}; unsupported selected services: ${unsupported.join(", ")}`);
}
if (config === null) throw new Error("deploy apply --env dev requires config.json");
const resolved = resolveManifestCommits(manifest, options.serviceId);
if (!options.runNow) return applyJob(config, args, options);
return await runApplyNow(config, resolved, options);
}
if (config === null) throw new Error("deploy local manifest mode requires config.json");
const manifest = resolveManifestCommits(await readDeployManifest(options.file), options.serviceId);
@@ -34,6 +34,7 @@ metadata:
annotations:
unidesk.ai/deploy-ref: origin/deploy/dev
unidesk.ai/image-source: deploy-dev-commit
unidesk.ai/deploy-service-id: backend-core
spec:
replicas: 1
strategy:
@@ -77,10 +78,30 @@ spec:
value: "8080"
- name: PROVIDER_PORT
value: "8081"
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: unidesk-dev-runtime-secrets
key: DATABASE_URL
- name: PROVIDER_TOKEN
valueFrom:
secretKeyRef:
name: unidesk-dev-runtime-secrets
key: PROVIDER_TOKEN
- name: DATABASE_VOLUME_NAME
valueFrom:
configMapKeyRef:
name: unidesk-dev-runtime-config
key: DATABASE_VOLUME_NAME
- name: DATABASE_VOLUME_SIZE
valueFrom:
configMapKeyRef:
name: unidesk-dev-runtime-config
key: DATABASE_VOLUME_SIZE
- name: UNIDESK_DATABASE_NAME
value: unidesk_dev
- name: UNIDESK_DEPLOY_SERVICE_ID
value: backend-core-dev
value: backend-core
- name: UNIDESK_DEPLOY_REPO
value: https://github.com/pikasTech/unidesk
- name: UNIDESK_DEPLOY_COMMIT
@@ -156,6 +177,7 @@ metadata:
annotations:
unidesk.ai/deploy-ref: origin/deploy/dev
unidesk.ai/image-source: deploy-dev-commit
unidesk.ai/deploy-service-id: frontend
spec:
replicas: 1
strategy:
@@ -201,10 +223,30 @@ spec:
value: http://frontend-dev.unidesk-dev.svc.cluster.local:8080
- name: PROVIDER_INGRESS_PUBLIC_URL
value: ws://backend-core-dev.unidesk-dev.svc.cluster.local:8081/ws/provider
- name: AUTH_USERNAME
valueFrom:
secretKeyRef:
name: unidesk-dev-runtime-secrets
key: AUTH_USERNAME
- name: AUTH_PASSWORD
valueFrom:
secretKeyRef:
name: unidesk-dev-runtime-secrets
key: AUTH_PASSWORD
- name: SESSION_SECRET
valueFrom:
secretKeyRef:
name: unidesk-dev-runtime-secrets
key: SESSION_SECRET
- name: SESSION_TTL_SECONDS
valueFrom:
configMapKeyRef:
name: unidesk-dev-runtime-config
key: SESSION_TTL_SECONDS
- name: UNIDESK_DATABASE_NAME
value: unidesk_dev
- name: UNIDESK_DEPLOY_SERVICE_ID
value: frontend-dev
value: frontend
- name: UNIDESK_DEPLOY_REPO
value: https://github.com/pikasTech/unidesk
- name: UNIDESK_DEPLOY_COMMIT