feat: add dry-run server cleanup plan
This commit is contained in:
@@ -32,6 +32,7 @@ UniDesk 是一个以主 server 为统一入口的分布式工作平台;本文
|
||||
- `bun scripts/cli.ts server status`:查询固定端口、swap 摘要、容器状态、健康检查和访问 URL,包含生产 frontend、dev frontend proxy 和 provider ingress,判定标准见 `docs/reference/deployment.md` 与 `docs/reference/dev-environment.md`。
|
||||
- `bun scripts/cli.ts server swap status|ensure [--path /swapfile] [--size 2GiB] [--dry-run]`:以 JSON 查看或幂等创建主 server swapfile,`ensure` 输出 before/after、动作、持久化状态和 degraded/failed 详情,规则见 `docs/reference/deployment.md`。
|
||||
- `bun scripts/cli.ts server logs [--tail-bytes N]`:分页返回文件日志与 Docker 日志尾部并带截断元数据,日志规则见 `docs/reference/observability.md`。
|
||||
- `bun scripts/cli.ts server cleanup plan [--min-age-hours N] [--limit N]`:只读/干跑生成主 server Docker 镜像清理计划,默认只列出至少 24 小时前创建的非保护镜像,输出 active/protected images、stale candidates、预计释放空间、风险等级和必须人工确认的 `docker image rm` 命令;禁止默认删除、禁止 prune、禁止触碰 database volume、registry storage 或 Baidu Netdisk 状态。
|
||||
- `bun scripts/cli.ts server rebuild <backend-core|frontend|dev-frontend-proxy|provider-gateway|todo-note|code-queue-mgr|project-manager|baidu-netdisk|oa-event-flow>`:以 build-first、Compose lock、no-deps force-recreate 和 post-up validation 的异步 job 重建主 server Compose 内单个服务;对 database、File Browser、Code Queue 执行面、k3sctl-adapter 或未知对象返回结构化 `unsupported-server-rebuild`,规则见 `docs/reference/deployment.md` 与 `docs/reference/cicd-standardization.md`。
|
||||
- `bun scripts/cli.ts provider attach <providerId> [--master-server URL] [--up] [--force]` / `bun scripts/cli.ts provider triage <providerId> [--observed-error text] [--observed-scope scope] [--microservice id ...]`:前者在新增计算节点上生成两项配置的 provider-gateway 挂载包;后者是只读多信号健康裁决入口,输出 `decision`、`healthyScopes`、`failedScopes` 和 `retryable`,用来把单路径 `provider is not online`、SSH 超时、registry 失败或 proxy 失败归类为 `retryable-transient`、`service-degraded` 或 `global-offline`,规则见 `docs/reference/provider-gateway.md` 和 `docs/reference/code-queue-supervision.md`。
|
||||
- `bun scripts/cli.ts ssh <providerId> [ssh-like args...]`:通过 provider-gateway 的 Host SSH / WSL SSH 维护桥打开近似原生 ssh 的交互会话或远端命令,并在远端 PATH 注入 `apply_patch`、`glob` 与 `skill-discover`;`apply-patch`、`py`、`skills`、结构化 `find`、`glob` 和 `argv` 子命令用于避免远端补丁、Python stdin、skill 发现与常用只读命令的嵌套转义问题,使用规则见 `docs/reference/cli.md` 和 `docs/reference/provider-gateway.md`。
|
||||
|
||||
@@ -16,6 +16,7 @@ CLI 可以从 `master` 快速演进,但必须兼容 `deploy.json` 固定的 CI
|
||||
- `server status` 查询公开端口、受限宿主端口、内部端口、主机 swap 摘要、Compose 容器、core/frontend/dev-frontend/provider/database 健康检查和访问 URL;D601 Code Queue 使用的 PostgreSQL/OA Event Flow host mapping 必须出现在受限宿主端口而不是无条件公开入口中。低内存主 server 上 `swap.warning` 非空时,先执行 `server swap status` 或 `server swap ensure`。
|
||||
- `server swap status|ensure [--path /swapfile] [--size 2GiB] [--dry-run]` 是主 server swap 管理入口。`status` 仅读 `/proc/meminfo`、`/proc/swaps` 和 `/etc/fstab` 并返回 JSON;`ensure` 在已有任何 active swap 时只报告 no-op,在无 active swap 时创建固定 swapfile、`chmod 600`、`mkswap`、`swapon` 并尽量写入 `/etc/fstab`。输出必须包含 `before`、`after`、total memory、active swap、持久化状态、关键动作和错误详情;若 swap 已启用但 fstab 写入失败,状态为 `degraded`,调用者需按返回的 detail 修复持久化。
|
||||
- `server logs` 返回 `logs/` 文件日志和 Docker 容器日志的尾部,默认限制输出大小,避免日志爆炸。实现必须只读取文件末尾字节,不得为了 tail 先把巨大日志完整读入 CLI 内存。
|
||||
- `server cleanup plan [--min-age-hours N] [--limit N]` 只生成主 server Docker 镜像清理 dry-run 计划,不执行删除;默认 `--min-age-hours 24`,避免把刚发布或刚验证的镜像列为 stale。输出必须包含 `dryRun=true`、`mutation=false`、`policy.deletionExecuted=false`、active containers/images、受保护镜像、candidate stale images、估算释放空间、风险等级、`commandsToReview` 和人工审批清单。计划必须保守白名单:保留 running containers 使用的 image ID,保留 stopped containers 引用的 image ID 直到人工先复核容器,保留 `deploy.json`/`CI.json` 当前 commit-pinned artifact、Compose stable image、上游 digest pin 和 provider-gateway runner image;`protectedStorage` 必须显式列出 PostgreSQL named volume、Baidu Netdisk `.state`、D601 registry storage 和 Docker volumes/host data policy。该入口禁止生成或执行 `docker system prune`、`docker image prune`、`docker builder prune`、`docker volume rm`、`docker compose down -v`、数据库清理或 host data `rm` 命令;未来若增加真实删除,必须另设显式审批参数并先复核 dry-run 输出。
|
||||
- `server rebuild <backend-core|frontend|dev-frontend-proxy|provider-gateway|todo-note|code-queue-mgr|project-manager|baidu-netdisk|oa-event-flow>` 创建异步 job,先构建目标服务镜像,随后在 `.state/locks/server-compose.lock` 串行保护下用 `--no-deps --force-recreate` 替换目标 service 并等待容器 `healthy/running`;该命令用于替代手工删除容器的兜底流程,其中 `dev-frontend-proxy` 只更新主 server dev 入口薄代理,`todo-note`、`code-queue-mgr`、`project-manager`、`baidu-netdisk` 和 `oa-event-flow` 只重建主 server 承载的对应后端,不会重建或删除 database 命名卷。D601 Code Queue 执行面不由 `server rebuild` 管理,Rust backend-core 迭代不得用 `server rebuild backend-core` 在 master server 编译,规则见 `docs/reference/dev-environment.md`。
|
||||
- `provider attach <providerId> [--master-server URL] [--up] [--force]` 在新计算节点生成两项配置的 provider-gateway 挂载包:`.state/provider-<ID>.env` 默认只包含 `UNIDESK_MASTER_SERVER` 与 `PROVIDER_ID`,`provider-<ID>.yml` 固定 Docker socket、`pid: "host"`、`restart: always`、只读 `/workspace` 和 SSH 维护私钥挂载;`--up` 会立即执行生成的 `docker compose up -d --build`。`provider triage <providerId> [--observed-error text] [--observed-scope scope] [--microservice id ...]` 是只读多信号健康裁决入口,会把单路径 `provider is not online`、SSH 超时、registry 失败和 service proxy 失败归类成 `runner-local-observation-gap`、`service-degraded`、`provider-degraded` 或 `global-blocker`,且默认提供 `debug health`、`debug dispatch <providerId> host.ssh --wait-ms 15000`、`ssh <providerId> argv true`、`artifact-registry health --provider-id <providerId>`、`microservice health k3sctl-adapter`、`microservice health code-queue` 和 `codex tasks --view supervisor --limit 20` 作为推荐交叉验证命令。
|
||||
- `ssh <providerId> [ssh-like args...]` 通过 backend-core 内网 WebSocket broker 和 provider-gateway 的 Host SSH / WSL SSH 维护桥连接目标节点;无后续参数时进入远端登录 shell,有后续参数时按 ssh 远端命令体验执行并返回远端 exit code。
|
||||
|
||||
@@ -40,6 +40,8 @@ Compose v2 安装后仍然必须遵守 UniDesk 的服务控制入口:全栈生
|
||||
|
||||
swap 管理不能被强塞进所有热路径。`server start/status` 可以暴露 warning 或摘要,但不会自动创建 swap;需要变更主机 swap 时必须显式运行 `server swap ensure`,并用返回的 `before`/`after` 和 `fstab.persisted` 作为验收记录。
|
||||
|
||||
根分区 Docker 镜像高水位治理必须先走 `bun scripts/cli.ts server cleanup plan` 的只读 dry-run。该计划只针对 Docker image inventory:默认只把创建时间超过 24 小时且不在保护集里的镜像列为 stale,输出 active containers/images、protected images、candidate stale images、风险、估算释放空间和人工复核命令,但不删除、不 prune、不改容器、不碰 volume。候选必须从白名单保护集中排除:running container image ID、stopped container 引用 image ID、Compose stable image、`deploy.json`/`CI.json` 当前 commit artifact、上游 digest pin 和 provider-gateway runner image。计划还必须显式保护 PostgreSQL named volume、Baidu Netdisk `.state`/staging、D601 registry storage 和所有 Docker volume/host data 目录。任何真实清理必须作为未来显式授权操作实现,且不得用 `docker system prune`、`docker image prune`、`docker builder prune` 或数据库清理替代 dry-run 审批;数据库清理前必须先确认可用备份。
|
||||
|
||||
## Start And Stop
|
||||
|
||||
`bun scripts/cli.ts server start` 与 `bun scripts/cli.ts server stop` 都是异步 job。启动 job 只执行固定 Compose project 的 `up -d --build --remove-orphans`,不得先 `down`,避免在 provider-gateway 旧容器或网络冲突时把长驻控制面容器先删掉又启动失败;停止 job 才允许执行 `down --remove-orphans`。启动和停止流程都禁止删除 Docker named volume。所有会改变主 server Compose 状态的 job 必须通过 `.state/locks/server-compose.lock` 串行化;连续 `server rebuild` 命令只代表连续创建异步 job,不能代表第一个 job 已结束,实际容器变更仍必须由 Compose lock 串行执行。
|
||||
|
||||
@@ -22,6 +22,7 @@ import { runAuthBrokerCommand } from "./src/auth-broker";
|
||||
import { runGhCommand } from "./src/gh";
|
||||
import { runCommanderCommand } from "./src/commander";
|
||||
import { isHelpToken, rootHelp, serverHelp, sshHelp, staticNamespaceHelp } from "./src/help";
|
||||
import { runServerCleanupCommand } from "./src/server-cleanup";
|
||||
|
||||
const remoteOptions = extractRemoteCliOptions(process.argv.slice(2));
|
||||
const args = remoteOptions.args;
|
||||
@@ -269,6 +270,13 @@ async function main(): Promise<void> {
|
||||
emitJson(commandName, stackLogs(config, boundedNumberOption("--tail-bytes", 3000, 500_000)));
|
||||
return;
|
||||
}
|
||||
if (sub === "cleanup") {
|
||||
const result = await runServerCleanupCommand(config, args.slice(2));
|
||||
const ok = (result as { ok?: unknown }).ok !== false;
|
||||
emitJson(commandName, result, ok);
|
||||
if (!ok) process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
if (sub === "rebuild") {
|
||||
if (!isRebuildableService(third)) {
|
||||
const result = unsupportedRebuildService(third);
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
import { buildDockerCleanupPlan, type DockerCleanupInventory } from "./src/server-cleanup";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function assertCondition(condition: unknown, message: string, detail: unknown = {}): void {
|
||||
if (!condition) throw new Error(`${message}: ${JSON.stringify(detail)}`);
|
||||
}
|
||||
|
||||
const observedAt = "2026-05-21T14:00:00.000Z";
|
||||
|
||||
const fixture: DockerCleanupInventory = {
|
||||
observedAt,
|
||||
images: [
|
||||
{
|
||||
id: "sha256:1111111111111111111111111111111111111111111111111111111111111111",
|
||||
repoTags: ["unidesk-backend-core:latest"],
|
||||
repoDigests: [],
|
||||
sizeBytes: 110 * 1024 * 1024,
|
||||
createdAt: "2026-05-21T10:00:00.000Z",
|
||||
labels: { "unidesk.ai/service-id": "backend-core", "unidesk.ai/source-commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" },
|
||||
},
|
||||
{
|
||||
id: "sha256:2222222222222222222222222222222222222222222222222222222222222222",
|
||||
repoTags: ["127.0.0.1:5000/unidesk/frontend:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"],
|
||||
repoDigests: [],
|
||||
sizeBytes: 120 * 1024 * 1024,
|
||||
createdAt: "2026-05-21T08:00:00.000Z",
|
||||
labels: { "unidesk.ai/service-id": "frontend", "unidesk.ai/source-commit": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" },
|
||||
},
|
||||
{
|
||||
id: "sha256:3333333333333333333333333333333333333333333333333333333333333333",
|
||||
repoTags: ["old-test-image:local"],
|
||||
repoDigests: ["old-test-image@sha256:3333333333333333333333333333333333333333333333333333333333333333"],
|
||||
sizeBytes: 500 * 1024 * 1024,
|
||||
createdAt: "2026-05-19T14:00:00.000Z",
|
||||
labels: {},
|
||||
},
|
||||
{
|
||||
id: "sha256:4444444444444444444444444444444444444444444444444444444444444444",
|
||||
repoTags: [],
|
||||
repoDigests: [],
|
||||
sizeBytes: 1024 * 1024 * 1024,
|
||||
createdAt: "2026-05-18T14:00:00.000Z",
|
||||
labels: {},
|
||||
},
|
||||
],
|
||||
containers: [
|
||||
{
|
||||
id: "container-running-backend-core",
|
||||
name: "unidesk-backend-core",
|
||||
imageRef: "unidesk-backend-core:latest",
|
||||
imageId: "sha256:1111111111111111111111111111111111111111111111111111111111111111",
|
||||
state: "running",
|
||||
status: "running",
|
||||
labels: {},
|
||||
},
|
||||
],
|
||||
desiredImageRefs: [
|
||||
{
|
||||
ref: "127.0.0.1:5000/unidesk/frontend:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
source: "CI.json + deploy.json",
|
||||
serviceId: "frontend",
|
||||
reason: "current commit-pinned registry artifact",
|
||||
},
|
||||
],
|
||||
desiredCommitsByService: {
|
||||
"backend-core": ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"],
|
||||
frontend: ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"],
|
||||
},
|
||||
protectedStorage: [
|
||||
{
|
||||
kind: "docker-volume",
|
||||
ref: "unidesk_pgdata_10gb",
|
||||
risk: "blocked",
|
||||
reason: "database named volume",
|
||||
},
|
||||
{
|
||||
kind: "path",
|
||||
ref: "/workspace/unidesk/.state/baidu-netdisk",
|
||||
risk: "blocked",
|
||||
reason: "Baidu Netdisk state",
|
||||
},
|
||||
{
|
||||
kind: "path",
|
||||
ref: "/home/ubuntu/.unidesk/registry-storage",
|
||||
risk: "blocked",
|
||||
reason: "registry storage",
|
||||
},
|
||||
],
|
||||
collection: {
|
||||
dockerAvailable: true,
|
||||
readOnlyCommands: [["docker", "image", "ls", "-q", "--no-trunc"]],
|
||||
errors: [],
|
||||
},
|
||||
};
|
||||
|
||||
export function runServerCleanupPlanContract(): JsonRecord {
|
||||
const plan = buildDockerCleanupPlan(fixture, { minAgeHours: 24, limit: 20 });
|
||||
const candidateIds = plan.candidateStaleImages.map((image) => image.id);
|
||||
const protectedIds = plan.protectedImages.map((image) => image.id);
|
||||
|
||||
assertCondition(plan.ok === true, "plan should be ok", plan);
|
||||
assertCondition(plan.dryRun === true && plan.mutation === false, "plan must be dry-run and non-mutating", plan.policy);
|
||||
assertCondition(plan.policy.deletionExecuted === false, "plan must not execute deletion", plan.policy);
|
||||
assertCondition(plan.policy.dockerPruneUsed === false, "plan must not use docker prune", plan.policy);
|
||||
assertCondition(plan.policy.dockerVolumesTouched === false, "plan must not touch docker volumes", plan.policy);
|
||||
assertCondition(plan.policy.databaseCleanupIncluded === false, "plan must not include database cleanup", plan.policy);
|
||||
|
||||
assertCondition(candidateIds.includes("sha256:3333333333333333333333333333333333333333333333333333333333333333"), "tagged stale image should be a candidate", plan.candidateStaleImages);
|
||||
assertCondition(candidateIds.includes("sha256:4444444444444444444444444444444444444444444444444444444444444444"), "dangling stale image should be a candidate", plan.candidateStaleImages);
|
||||
assertCondition(protectedIds.includes("sha256:1111111111111111111111111111111111111111111111111111111111111111"), "running image should be protected", plan.protectedImages);
|
||||
assertCondition(protectedIds.includes("sha256:2222222222222222222222222222222222222222222222222222222222222222"), "desired deploy image should be protected", plan.protectedImages);
|
||||
|
||||
assertCondition(plan.candidateStaleImages.length === 2, "only stale non-protected images should be candidates", plan.candidateStaleImages);
|
||||
assertCondition(plan.risk.medium === 1, "tagged stale candidate should be medium risk", plan.risk);
|
||||
assertCondition(plan.risk.low === 1, "dangling stale candidate should be low risk", plan.risk);
|
||||
assertCondition(plan.commandsToReview.length === 2, "commandsToReview should include candidate commands", plan.commandsToReview);
|
||||
assertCondition(plan.commandsToReview.every((command) => command.requiresManualApproval === true), "commands must require manual approval", plan.commandsToReview);
|
||||
const taggedCommand = plan.commandsToReview.find((command) => command.imageId === "sha256:3333333333333333333333333333333333333333333333333333333333333333");
|
||||
assertCondition(taggedCommand?.command.includes("old-test-image:local"), "tagged candidate command should include reviewed tag", plan.commandsToReview);
|
||||
assertCondition(taggedCommand?.command.includes("old-test-image@sha256:3333333333333333333333333333333333333333333333333333333333333333"), "tagged candidate command should include reviewed digest", plan.commandsToReview);
|
||||
assertCondition(plan.commandsToReview.some((command) => command.command.includes("sha256:4444444444444444444444444444444444444444444444444444444444444444")), "dangling candidate command should use image id", plan.commandsToReview);
|
||||
assertCondition(!JSON.stringify(plan.commandsToReview).includes("docker image prune"), "plan must not recommend image prune", plan.commandsToReview);
|
||||
assertCondition(!JSON.stringify(plan.commandsToReview).includes("docker system prune"), "plan must not recommend system prune", plan.commandsToReview);
|
||||
assertCondition(plan.prohibitedCommands.includes("docker image prune"), "image prune should be explicitly prohibited", plan.prohibitedCommands);
|
||||
assertCondition(plan.prohibitedCommands.includes("docker system prune"), "system prune should be explicitly prohibited", plan.prohibitedCommands);
|
||||
assertCondition(plan.protectedStorage.some((item) => item.ref === "unidesk_pgdata_10gb"), "database volume must be protected", plan.protectedStorage);
|
||||
assertCondition(plan.protectedStorage.some((item) => String(item.ref).includes("baidu-netdisk")), "Baidu Netdisk state must be protected", plan.protectedStorage);
|
||||
assertCondition(plan.protectedStorage.some((item) => String(item.ref).includes("registry-storage")), "registry storage must be protected", plan.protectedStorage);
|
||||
assertCondition(plan.estimatedReclaimBytes === (500 * 1024 * 1024) + (1024 * 1024 * 1024), "estimated reclaim should sum candidate image sizes", plan.estimates);
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
checks: [
|
||||
"dry-run non-mutating policy",
|
||||
"active image protected",
|
||||
"deploy/CI desired image protected",
|
||||
"stale candidates emitted",
|
||||
"risk levels emitted",
|
||||
"manual commandsToReview emitted",
|
||||
"database/registry/baidu storage protected",
|
||||
"prune commands absent",
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
process.stdout.write(`${JSON.stringify(runServerCleanupPlanContract(), null, 2)}\n`);
|
||||
}
|
||||
@@ -24,6 +24,7 @@ const syntaxFiles = [
|
||||
"scripts/src/e2e.ts",
|
||||
"scripts/src/help.ts",
|
||||
"scripts/src/commander.ts",
|
||||
"scripts/src/server-cleanup.ts",
|
||||
"scripts/src/remote.ts",
|
||||
"scripts/host-codex-commander-contract-test.ts",
|
||||
"scripts/host-codex-commander-no-daemon-smoke-contract-test.ts",
|
||||
@@ -316,7 +317,9 @@ export function runChecks(config: UniDeskConfig, options: CheckOptions = default
|
||||
fileItem("scripts/gh-cli-pr-contract-test.ts"),
|
||||
fileItem("scripts/code-queue-pr-preflight-example.ts"),
|
||||
fileItem("scripts/schedule-cli-contract-test.ts"),
|
||||
fileItem("scripts/server-cleanup-plan-contract-test.ts"),
|
||||
fileItem("scripts/src/artifact-registry.ts"),
|
||||
fileItem("scripts/src/server-cleanup.ts"),
|
||||
fileItem("scripts/src/auth-broker.ts"),
|
||||
fileItem("scripts/auth-broker-contract-test.ts"),
|
||||
fileItem("src/components/microservices/auth-broker/Cargo.toml"),
|
||||
@@ -348,6 +351,7 @@ export function runChecks(config: UniDeskConfig, options: CheckOptions = default
|
||||
items.push(commandItem("baidu-netdisk:artifact-guard-contract", ["bun", "scripts/baidu-netdisk-artifact-guard-contract-test.ts"], 30_000));
|
||||
items.push(commandItem("artifact-registry:direct-compose-dry-run-matrix", ["bun", "scripts/artifact-consumer-dry-run-matrix-test.ts"], 30_000));
|
||||
items.push(commandItem("schedule:cli-contract", ["bun", "scripts/schedule-cli-contract-test.ts"], 30_000));
|
||||
items.push(commandItem("server:cleanup-plan-contract", ["bun", "scripts/server-cleanup-plan-contract-test.ts"], 30_000));
|
||||
items.push(commandItem("gh:issue-guard-contract", ["bun", "scripts/gh-cli-issue-guard-contract-test.ts"], 30_000));
|
||||
items.push(commandItem("gh:pr-contract", ["bun", "scripts/gh-cli-pr-contract-test.ts"], 30_000));
|
||||
items.push(commandItem("auth-broker:p0-contract", ["bun", "scripts/auth-broker-contract-test.ts"], 30_000));
|
||||
@@ -367,6 +371,7 @@ export function runChecks(config: UniDeskConfig, options: CheckOptions = default
|
||||
items.push(skippedItem("baidu-netdisk:artifact-guard-contract", "Baidu Netdisk artifact guard contract is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
items.push(skippedItem("artifact-registry:direct-compose-dry-run-matrix", "main-server direct artifact consumer dry-run matrix is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
items.push(skippedItem("schedule:cli-contract", "Schedule CLI contract is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
items.push(skippedItem("server:cleanup-plan-contract", "Server cleanup dry-run contract is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
items.push(skippedItem("gh:issue-guard-contract", "GitHub issue CLI contract is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
items.push(skippedItem("gh:pr-contract", "GitHub PR CLI contract is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
items.push(skippedItem("auth-broker:p0-contract", "Auth Broker P0 skeleton and CLI adapter contract is opt-in with script checks", "--scripts-typecheck or --full"));
|
||||
|
||||
+16
-1
@@ -15,6 +15,7 @@ export function rootHelp(): unknown {
|
||||
{ command: "server status", description: "Show fixed ports, containers, service health, and public URLs." },
|
||||
{ command: "server swap status|ensure [--path /swapfile] [--size 2GiB] [--dry-run]", description: "Inspect or idempotently create host swap for low-memory main-server operation." },
|
||||
{ command: "server logs [--tail-bytes N]", description: "Return bounded tails from file logs and docker logs." },
|
||||
{ command: "server cleanup plan [--min-age-hours N] [--limit N]", description: "Dry-run Docker image cleanup plan only: list active/protected images, stale candidates older than the default 24h threshold, risk, estimated reclaim, and manual review commands without deleting anything." },
|
||||
{ command: "server rebuild <backend-core|frontend|dev-frontend-proxy|provider-gateway|todo-note|code-queue-mgr|project-manager|baidu-netdisk|oa-event-flow>", description: "Maintenance-only local Compose rebuild for reviewed main-server services; frontend standard release must use CI artifact plus deploy apply dev/prod artifact consumers." },
|
||||
{ command: "provider attach <providerId> [--master-server URL] [--up] [--force] | provider triage <providerId> [--observed-error text] [--observed-scope scope] [--microservice id ...]", description: "Generate the minimal external provider-gateway env/compose bundle or run the read-only provider health triage contract." },
|
||||
{ command: "ssh <providerId> [ssh-like args...]", description: "Open a Host SSH / WSL SSH maintenance session through the provider-gateway bridge with built-in remote helper tools in PATH." },
|
||||
@@ -80,7 +81,7 @@ export function isHelpToken(value: string | undefined): boolean {
|
||||
|
||||
export function serverHelp(action: string | undefined = undefined): unknown {
|
||||
return {
|
||||
command: action === undefined || isHelpToken(action) ? "server start|stop|status|swap|logs|rebuild" : `server ${action}`,
|
||||
command: action === undefined || isHelpToken(action) ? "server start|stop|status|swap|logs|cleanup|rebuild" : `server ${action}`,
|
||||
output: "json",
|
||||
description: "Manage the fixed main-server Docker Compose stack without exposing backend-core REST publicly.",
|
||||
usage: {
|
||||
@@ -89,8 +90,22 @@ export function serverHelp(action: string | undefined = undefined): unknown {
|
||||
status: "bun scripts/cli.ts server status",
|
||||
swap: "bun scripts/cli.ts server swap status|ensure [--path /swapfile] [--size 2GiB] [--dry-run]",
|
||||
logs: "bun scripts/cli.ts server logs [--tail-bytes N]",
|
||||
cleanup: "bun scripts/cli.ts server cleanup plan [--min-age-hours N] [--limit N]",
|
||||
rebuild: "bun scripts/cli.ts server rebuild <backend-core|frontend|dev-frontend-proxy|provider-gateway|todo-note|code-queue-mgr|project-manager|baidu-netdisk|oa-event-flow>",
|
||||
},
|
||||
cleanupPlan: {
|
||||
dryRunOnly: true,
|
||||
mutation: false,
|
||||
scope: "docker images only",
|
||||
defaultMinAgeHours: 24,
|
||||
protectedByDefault: [
|
||||
"running containers and their image IDs",
|
||||
"stopped containers and their image IDs until the container is reviewed separately",
|
||||
"deploy.json/CI.json current commit-pinned images",
|
||||
"database named volume, registry storage, Baidu Netdisk state and staging directories",
|
||||
],
|
||||
forbidden: ["docker prune", "docker volume rm", "compose down -v", "database cleanup without verified backup"],
|
||||
},
|
||||
publicEntrypoints: {
|
||||
frontend: "prod UniDesk frontend",
|
||||
devFrontend: "dev UniDesk frontend proxy to D601 unidesk-dev/frontend-dev",
|
||||
|
||||
@@ -0,0 +1,839 @@
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
|
||||
import { runCommand } from "./command";
|
||||
import { type UniDeskConfig, repoRoot, rootPath } from "./config";
|
||||
import { loadCiCatalog, type CiCatalogArtifact } from "./ci-catalog";
|
||||
|
||||
type RiskLevel = "low" | "medium" | "high" | "blocked";
|
||||
|
||||
export interface DockerImageInventoryItem {
|
||||
id: string;
|
||||
repoTags: string[];
|
||||
repoDigests: string[];
|
||||
sizeBytes: number;
|
||||
createdAt: string | null;
|
||||
labels: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface DockerContainerInventoryItem {
|
||||
id: string;
|
||||
name: string;
|
||||
imageRef: string;
|
||||
imageId: string;
|
||||
state: string;
|
||||
status: string;
|
||||
labels: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface DesiredImageRef {
|
||||
ref: string;
|
||||
source: string;
|
||||
serviceId?: string;
|
||||
reason: string;
|
||||
}
|
||||
|
||||
export interface ProtectedStorageRef {
|
||||
kind: "docker-volume" | "path" | "policy";
|
||||
ref: string;
|
||||
risk: RiskLevel;
|
||||
reason: string;
|
||||
}
|
||||
|
||||
export interface DockerCleanupInventory {
|
||||
observedAt: string;
|
||||
images: DockerImageInventoryItem[];
|
||||
containers: DockerContainerInventoryItem[];
|
||||
desiredImageRefs: DesiredImageRef[];
|
||||
desiredCommitsByService: Record<string, string[]>;
|
||||
protectedStorage: ProtectedStorageRef[];
|
||||
collection: {
|
||||
dockerAvailable: boolean;
|
||||
readOnlyCommands: string[][];
|
||||
errors: Array<{ command?: string[]; message: string; exitCode?: number | null; stderrTail?: string }>;
|
||||
};
|
||||
}
|
||||
|
||||
export interface ServerCleanupPlanOptions {
|
||||
minAgeHours: number;
|
||||
limit: number;
|
||||
}
|
||||
|
||||
export interface CleanupImageSummary {
|
||||
id: string;
|
||||
shortId: string;
|
||||
repoTags: string[];
|
||||
repoDigests: string[];
|
||||
sizeBytes: number;
|
||||
createdAt: string | null;
|
||||
}
|
||||
|
||||
export interface ProtectedImageSummary extends CleanupImageSummary {
|
||||
risk: "blocked";
|
||||
reasons: string[];
|
||||
containers: Array<{ id: string; name: string; state: string; status: string; imageRef: string }>;
|
||||
}
|
||||
|
||||
export interface CandidateImageSummary extends CleanupImageSummary {
|
||||
risk: Exclude<RiskLevel, "blocked">;
|
||||
ageHours: number | null;
|
||||
reasons: string[];
|
||||
commandsToReview: string[][];
|
||||
reviewChecklist: string[];
|
||||
}
|
||||
|
||||
export interface CleanupCommandReview {
|
||||
kind: "docker-image-remove";
|
||||
risk: Exclude<RiskLevel, "blocked">;
|
||||
imageId: string;
|
||||
estimatedReclaimBytes: number;
|
||||
command: string[];
|
||||
commandText: string;
|
||||
requiresManualApproval: true;
|
||||
reviewChecklist: string[];
|
||||
}
|
||||
|
||||
export interface ServerCleanupPlan {
|
||||
ok: boolean;
|
||||
dryRun: true;
|
||||
mutation: false;
|
||||
action: "server cleanup plan";
|
||||
scope: "docker-images-only";
|
||||
observedAt: string;
|
||||
options: ServerCleanupPlanOptions;
|
||||
policy: {
|
||||
deletionExecuted: false;
|
||||
deleteCommandsExecuted: [];
|
||||
dockerPruneUsed: false;
|
||||
dockerVolumesTouched: false;
|
||||
dataDirectoriesTouched: false;
|
||||
databaseCleanupIncluded: false;
|
||||
liveCleanupImplemented: false;
|
||||
note: string;
|
||||
};
|
||||
inventory: {
|
||||
dockerAvailable: boolean;
|
||||
imageCount: number;
|
||||
containerCount: number;
|
||||
activeContainerCount: number;
|
||||
candidateImageCount: number;
|
||||
protectedImageCount: number;
|
||||
omittedCandidateCount: number;
|
||||
collectionErrors: DockerCleanupInventory["collection"]["errors"];
|
||||
readOnlyCommands: string[][];
|
||||
};
|
||||
activeContainers: Array<{ id: string; name: string; imageRef: string; imageId: string; status: string }>;
|
||||
activeImages: ProtectedImageSummary[];
|
||||
protectedImages: ProtectedImageSummary[];
|
||||
protectedDesiredImageRefs: DesiredImageRef[];
|
||||
protectedStorage: ProtectedStorageRef[];
|
||||
candidateStaleImages: CandidateImageSummary[];
|
||||
estimatedReclaimBytes: number;
|
||||
estimates: {
|
||||
candidateImageCount: number;
|
||||
estimatedReclaimBytes: number;
|
||||
estimatedReclaimBytesUpperBound: number;
|
||||
basis: string;
|
||||
warning: string;
|
||||
};
|
||||
risk: {
|
||||
overall: "manual-review-required" | "no-candidates";
|
||||
low: number;
|
||||
medium: number;
|
||||
high: number;
|
||||
blocked: number;
|
||||
};
|
||||
commandsToReview: CleanupCommandReview[];
|
||||
manualApprovalRequired: string[];
|
||||
prohibitedCommands: string[];
|
||||
}
|
||||
|
||||
interface DeployServiceCommit {
|
||||
environment: string;
|
||||
serviceId: string;
|
||||
commitId: string;
|
||||
}
|
||||
|
||||
const defaultOptions: ServerCleanupPlanOptions = {
|
||||
minAgeHours: 24,
|
||||
limit: 200,
|
||||
};
|
||||
|
||||
export function parseServerCleanupOptions(args: string[]): ServerCleanupPlanOptions {
|
||||
const options = { ...defaultOptions };
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const arg = args[index];
|
||||
if (arg === "--min-age-hours") {
|
||||
const raw = args[index + 1];
|
||||
const value = Number(raw);
|
||||
if (!Number.isFinite(value) || value < 0) throw new Error("--min-age-hours must be a non-negative number");
|
||||
options.minAgeHours = value;
|
||||
index += 1;
|
||||
} else if (arg === "--limit") {
|
||||
const raw = args[index + 1];
|
||||
const value = Number(raw);
|
||||
if (!Number.isInteger(value) || value <= 0) throw new Error("--limit must be a positive integer");
|
||||
options.limit = Math.min(value, 1000);
|
||||
index += 1;
|
||||
} else {
|
||||
throw new Error(`unknown server cleanup plan option: ${arg}`);
|
||||
}
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
export async function runServerCleanupCommand(config: UniDeskConfig, args: string[]): Promise<unknown> {
|
||||
const [action = "plan", ...rest] = args;
|
||||
if (action === "plan" || action === "dry-run") {
|
||||
return serverCleanupPlan(config, parseServerCleanupOptions(rest));
|
||||
}
|
||||
return {
|
||||
ok: false,
|
||||
error: "unsupported-server-cleanup-action",
|
||||
action,
|
||||
supportedActions: ["plan"],
|
||||
dryRunOnly: true,
|
||||
mutation: false,
|
||||
policy: "This task implements only server cleanup plan. Real image deletion is intentionally not implemented and must require a future explicit approval parameter.",
|
||||
};
|
||||
}
|
||||
|
||||
export function serverCleanupPlan(config: UniDeskConfig, options: ServerCleanupPlanOptions = defaultOptions): ServerCleanupPlan {
|
||||
const inventory = collectDockerCleanupInventory(config);
|
||||
return buildDockerCleanupPlan(inventory, options);
|
||||
}
|
||||
|
||||
export function buildDockerCleanupPlan(inventory: DockerCleanupInventory, options: ServerCleanupPlanOptions = defaultOptions): ServerCleanupPlan {
|
||||
const normalizedDesiredRefs = new Map<string, DesiredImageRef[]>();
|
||||
for (const desired of inventory.desiredImageRefs) {
|
||||
for (const variant of imageRefVariants(desired.ref)) {
|
||||
const existing = normalizedDesiredRefs.get(variant) ?? [];
|
||||
existing.push(desired);
|
||||
normalizedDesiredRefs.set(variant, existing);
|
||||
}
|
||||
}
|
||||
|
||||
const containersByImage = new Map<string, DockerContainerInventoryItem[]>();
|
||||
for (const container of inventory.containers) {
|
||||
if (container.imageId.length === 0) continue;
|
||||
const existing = containersByImage.get(container.imageId) ?? [];
|
||||
existing.push(container);
|
||||
containersByImage.set(container.imageId, existing);
|
||||
}
|
||||
|
||||
const runningImageIds = new Set(
|
||||
inventory.containers
|
||||
.filter((container) => container.state === "running")
|
||||
.map((container) => container.imageId)
|
||||
.filter(Boolean),
|
||||
);
|
||||
|
||||
const protectedImages: ProtectedImageSummary[] = [];
|
||||
const candidateImages: CandidateImageSummary[] = [];
|
||||
const commandsToReview: CleanupCommandReview[] = [];
|
||||
|
||||
for (const image of inventory.images) {
|
||||
const relatedContainers = containersByImage.get(image.id) ?? [];
|
||||
const reasons = protectedReasonsForImage(image, relatedContainers, runningImageIds, normalizedDesiredRefs, inventory.desiredCommitsByService);
|
||||
if (reasons.length > 0) {
|
||||
protectedImages.push({
|
||||
...imageSummary(image),
|
||||
risk: "blocked",
|
||||
reasons,
|
||||
containers: relatedContainers.map(containerSummary),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const ageHours = imageAgeHours(image.createdAt, inventory.observedAt);
|
||||
if (ageHours !== null && ageHours < options.minAgeHours) {
|
||||
protectedImages.push({
|
||||
...imageSummary(image),
|
||||
risk: "blocked",
|
||||
reasons: [`younger-than-min-age-hours:${options.minAgeHours}`],
|
||||
containers: [],
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const risk = candidateRisk(image, ageHours);
|
||||
const reviewChecklist = reviewChecklistForCandidate(risk, image);
|
||||
const command = imageRemoveCommand(image);
|
||||
const candidate: CandidateImageSummary = {
|
||||
...imageSummary(image),
|
||||
risk,
|
||||
ageHours,
|
||||
reasons: candidateReasons(image, ageHours),
|
||||
commandsToReview: [command],
|
||||
reviewChecklist,
|
||||
};
|
||||
candidateImages.push(candidate);
|
||||
commandsToReview.push({
|
||||
kind: "docker-image-remove",
|
||||
risk,
|
||||
imageId: image.id,
|
||||
estimatedReclaimBytes: image.sizeBytes,
|
||||
command,
|
||||
commandText: command.map(shellQuote).join(" "),
|
||||
requiresManualApproval: true,
|
||||
reviewChecklist,
|
||||
});
|
||||
}
|
||||
|
||||
candidateImages.sort((left, right) => right.sizeBytes - left.sizeBytes || left.shortId.localeCompare(right.shortId));
|
||||
commandsToReview.sort((left, right) => right.estimatedReclaimBytes - left.estimatedReclaimBytes || left.imageId.localeCompare(right.imageId));
|
||||
protectedImages.sort((left, right) => right.sizeBytes - left.sizeBytes || left.shortId.localeCompare(right.shortId));
|
||||
|
||||
const visibleCandidates = candidateImages.slice(0, options.limit);
|
||||
const visibleCommands = commandsToReview.slice(0, options.limit);
|
||||
const activeImages = protectedImages.filter((image) => runningImageIds.has(image.id));
|
||||
const estimatedReclaimBytes = candidateImages.reduce((total, image) => total + image.sizeBytes, 0);
|
||||
const riskCounts = candidateImages.reduce(
|
||||
(counts, image) => ({ ...counts, [image.risk]: counts[image.risk] + 1 }),
|
||||
{ low: 0, medium: 0, high: 0 },
|
||||
);
|
||||
|
||||
return {
|
||||
ok: inventory.collection.dockerAvailable && inventory.collection.errors.length === 0,
|
||||
dryRun: true,
|
||||
mutation: false,
|
||||
action: "server cleanup plan",
|
||||
scope: "docker-images-only",
|
||||
observedAt: inventory.observedAt,
|
||||
options,
|
||||
policy: {
|
||||
deletionExecuted: false,
|
||||
deleteCommandsExecuted: [],
|
||||
dockerPruneUsed: false,
|
||||
dockerVolumesTouched: false,
|
||||
dataDirectoriesTouched: false,
|
||||
databaseCleanupIncluded: false,
|
||||
liveCleanupImplemented: false,
|
||||
note: "This command only inventories Docker images and builds a dry-run review plan. It never runs docker image rm, docker prune, docker volume rm, rm, or database cleanup.",
|
||||
},
|
||||
inventory: {
|
||||
dockerAvailable: inventory.collection.dockerAvailable,
|
||||
imageCount: inventory.images.length,
|
||||
containerCount: inventory.containers.length,
|
||||
activeContainerCount: inventory.containers.filter((container) => container.state === "running").length,
|
||||
candidateImageCount: candidateImages.length,
|
||||
protectedImageCount: protectedImages.length,
|
||||
omittedCandidateCount: Math.max(0, candidateImages.length - visibleCandidates.length),
|
||||
collectionErrors: inventory.collection.errors,
|
||||
readOnlyCommands: inventory.collection.readOnlyCommands,
|
||||
},
|
||||
activeContainers: inventory.containers
|
||||
.filter((container) => container.state === "running")
|
||||
.map((container) => ({
|
||||
id: shortContainerId(container.id),
|
||||
name: container.name,
|
||||
imageRef: container.imageRef,
|
||||
imageId: container.imageId,
|
||||
status: container.status,
|
||||
})),
|
||||
activeImages,
|
||||
protectedImages,
|
||||
protectedDesiredImageRefs: inventory.desiredImageRefs,
|
||||
protectedStorage: inventory.protectedStorage,
|
||||
candidateStaleImages: visibleCandidates,
|
||||
estimatedReclaimBytes,
|
||||
estimates: {
|
||||
candidateImageCount: candidateImages.length,
|
||||
estimatedReclaimBytes,
|
||||
estimatedReclaimBytesUpperBound: estimatedReclaimBytes,
|
||||
basis: "sum of docker image inspect .Size for candidate image IDs, counted once per image ID",
|
||||
warning: "Docker layers may be shared; actual reclaim can be lower. Review docker system df before any future approved deletion.",
|
||||
},
|
||||
risk: {
|
||||
overall: candidateImages.length === 0 ? "no-candidates" : "manual-review-required",
|
||||
low: riskCounts.low,
|
||||
medium: riskCounts.medium,
|
||||
high: riskCounts.high,
|
||||
blocked: protectedImages.length,
|
||||
},
|
||||
commandsToReview: visibleCommands,
|
||||
manualApprovalRequired: [
|
||||
"Review every candidate image ID, tag, age, and risk before running any command outside this dry-run.",
|
||||
"Confirm the target is not a rollback artifact, emergency diagnostic image, or active provider/runtime dependency.",
|
||||
"Do not clean PostgreSQL, Baidu Netdisk state, registry storage, Docker volumes, or host data directories from this plan.",
|
||||
"Any future database cleanup requires a verified backup first; this plan deliberately excludes database cleanup.",
|
||||
],
|
||||
prohibitedCommands: [
|
||||
"docker system prune",
|
||||
"docker image prune",
|
||||
"docker builder prune",
|
||||
"docker volume rm",
|
||||
"docker compose down -v",
|
||||
"rm -rf .state",
|
||||
"rm -rf /home/ubuntu/.unidesk/registry-storage",
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
function collectDockerCleanupInventory(config: UniDeskConfig): DockerCleanupInventory {
|
||||
const observedAt = new Date().toISOString();
|
||||
const desired = collectDesiredImagePolicy(config);
|
||||
const readOnlyCommands: string[][] = [];
|
||||
const errors: DockerCleanupInventory["collection"]["errors"] = [];
|
||||
const images = collectDockerImages(readOnlyCommands, errors);
|
||||
const containers = collectDockerContainers(readOnlyCommands, errors);
|
||||
return {
|
||||
observedAt,
|
||||
images,
|
||||
containers,
|
||||
desiredImageRefs: desired.refs,
|
||||
desiredCommitsByService: desired.commitsByService,
|
||||
protectedStorage: protectedStorageRefs(config),
|
||||
collection: {
|
||||
dockerAvailable: errors.length === 0 || images.length > 0 || containers.length > 0,
|
||||
readOnlyCommands,
|
||||
errors,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function collectDockerImages(
|
||||
readOnlyCommands: string[][],
|
||||
errors: DockerCleanupInventory["collection"]["errors"],
|
||||
): DockerImageInventoryItem[] {
|
||||
const listCommand = ["docker", "image", "ls", "-q", "--no-trunc"];
|
||||
readOnlyCommands.push(listCommand);
|
||||
const list = runCommand(listCommand, repoRoot, { timeoutMs: 30_000 });
|
||||
if (list.exitCode !== 0) {
|
||||
errors.push(commandError(listCommand, "failed to list docker images", list.exitCode, list.stderr));
|
||||
return [];
|
||||
}
|
||||
const imageIds = [...new Set(list.stdout.split("\n").map((line) => line.trim()).filter(Boolean))];
|
||||
const images: DockerImageInventoryItem[] = [];
|
||||
for (const chunk of chunks(imageIds, 50)) {
|
||||
const inspectCommand = ["docker", "image", "inspect", ...chunk];
|
||||
readOnlyCommands.push(["docker", "image", "inspect", `<${chunk.length} image ids>`]);
|
||||
const inspect = runCommand(inspectCommand, repoRoot, { timeoutMs: 60_000 });
|
||||
if (inspect.exitCode !== 0) {
|
||||
errors.push(commandError(inspectCommand, "failed to inspect docker images", inspect.exitCode, inspect.stderr));
|
||||
continue;
|
||||
}
|
||||
const parsed = parseJsonArray(inspect.stdout, "docker image inspect");
|
||||
for (const item of parsed) images.push(imageFromInspect(item));
|
||||
}
|
||||
return images;
|
||||
}
|
||||
|
||||
function collectDockerContainers(
|
||||
readOnlyCommands: string[][],
|
||||
errors: DockerCleanupInventory["collection"]["errors"],
|
||||
): DockerContainerInventoryItem[] {
|
||||
const psCommand = ["docker", "ps", "-a", "--no-trunc", "--format", "{{json .}}"];
|
||||
readOnlyCommands.push(psCommand);
|
||||
const ps = runCommand(psCommand, repoRoot, { timeoutMs: 30_000 });
|
||||
if (ps.exitCode !== 0) {
|
||||
errors.push(commandError(psCommand, "failed to list docker containers", ps.exitCode, ps.stderr));
|
||||
return [];
|
||||
}
|
||||
const ids = ps.stdout
|
||||
.split("\n")
|
||||
.map((line) => {
|
||||
const record = parseJsonRecord(line, "docker ps");
|
||||
return stringValue(record.ID);
|
||||
})
|
||||
.filter(Boolean);
|
||||
if (ids.length === 0) return [];
|
||||
const containers: DockerContainerInventoryItem[] = [];
|
||||
for (const chunk of chunks(ids, 50)) {
|
||||
const inspectCommand = ["docker", "container", "inspect", ...chunk];
|
||||
readOnlyCommands.push(["docker", "container", "inspect", `<${chunk.length} container ids>`]);
|
||||
const inspect = runCommand(inspectCommand, repoRoot, { timeoutMs: 60_000 });
|
||||
if (inspect.exitCode !== 0) {
|
||||
errors.push(commandError(inspectCommand, "failed to inspect docker containers", inspect.exitCode, inspect.stderr));
|
||||
continue;
|
||||
}
|
||||
const parsed = parseJsonArray(inspect.stdout, "docker container inspect");
|
||||
for (const item of parsed) containers.push(containerFromInspect(item));
|
||||
}
|
||||
return containers;
|
||||
}
|
||||
|
||||
function collectDesiredImagePolicy(config: UniDeskConfig): { refs: DesiredImageRef[]; commitsByService: Record<string, string[]> } {
|
||||
const refs: DesiredImageRef[] = [];
|
||||
const commits = deployServiceCommits();
|
||||
const commitsByService = commits.reduce<Record<string, string[]>>((accumulator, item) => {
|
||||
accumulator[item.serviceId] = [...(accumulator[item.serviceId] ?? []), item.commitId];
|
||||
return accumulator;
|
||||
}, {});
|
||||
const add = (ref: string | undefined, source: string, reason: string, serviceId?: string): void => {
|
||||
if (ref === undefined || ref.trim().length === 0) return;
|
||||
refs.push({ ref: ref.trim(), source, reason, ...(serviceId === undefined ? {} : { serviceId }) });
|
||||
};
|
||||
|
||||
for (const service of composeServiceImageRefs(config)) {
|
||||
add(service.ref, "docker-compose.yml", service.reason, service.serviceId);
|
||||
}
|
||||
add(config.providerGateway.upgrade.runnerImage, "config.providerGateway.upgrade.runnerImage", "provider-gateway remote upgrade runner image", "provider-gateway");
|
||||
|
||||
for (const microservice of config.microservices) {
|
||||
add(microservice.repository.artifactSource?.imageRef, "config.microservices.repository.artifactSource", "upstream image pin", microservice.id);
|
||||
add(microservice.repository.artifactSource?.mirrorRepository, "config.microservices.repository.artifactSource", "upstream mirror repository", microservice.id);
|
||||
if (looksLikeImageRef(microservice.repository.dockerfile)) {
|
||||
add(microservice.repository.dockerfile, "config.microservices.repository.dockerfile", "repository dockerfile field is an upstream image reference", microservice.id);
|
||||
}
|
||||
if (microservice.providerId === config.providerGateway.id || microservice.deployment.mode === "internal-sidecar") {
|
||||
add(microservice.repository.composeService, "config.microservices.repository.composeService", "main-server or internal-sidecar stable image", microservice.id);
|
||||
add(`unidesk-${microservice.repository.composeService}`, "config.microservices.repository.composeService", "main-server or internal-sidecar Compose default image", microservice.id);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const catalog = loadCiCatalog();
|
||||
for (const artifact of catalog.artifacts) {
|
||||
addCatalogArtifactRefs(artifact, catalog.defaults.registry, commitsByService[artifact.serviceId] ?? [], add);
|
||||
}
|
||||
} catch (error) {
|
||||
add("ci-catalog-unavailable-placeholder", "CI.json", `CI catalog could not be loaded: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
|
||||
for (const item of commits) {
|
||||
for (const stable of stableImageRefsForService(item.serviceId, config)) {
|
||||
add(stable, `deploy.json#environments.${item.environment}`, "current deploy desired stable image", item.serviceId);
|
||||
if (!hasTagOrDigest(stable)) {
|
||||
add(`${stable}:${item.commitId}`, `deploy.json#environments.${item.environment}`, "current deploy desired commit-tag image", item.serviceId);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { refs: dedupeDesiredRefs(refs), commitsByService };
|
||||
}
|
||||
|
||||
function addCatalogArtifactRefs(
|
||||
artifact: CiCatalogArtifact,
|
||||
registry: string,
|
||||
commits: string[],
|
||||
add: (ref: string | undefined, source: string, reason: string, serviceId?: string) => void,
|
||||
): void {
|
||||
if (artifact.kind === "source-build") {
|
||||
for (const commit of commits) {
|
||||
add(`${registry}/${artifact.image.repository}:${commit}`, "CI.json + deploy.json", "current commit-pinned registry artifact", artifact.serviceId);
|
||||
add(`${artifact.image.repository}:${commit}`, "CI.json + deploy.json", "current commit-pinned local artifact tag", artifact.serviceId);
|
||||
}
|
||||
return;
|
||||
}
|
||||
add(artifact.upstream.imageRef, "CI.json upstream image", "blocked upstream image pin", artifact.serviceId);
|
||||
add(artifact.upstream.digestRef, "CI.json upstream image", "blocked upstream image digest pin", artifact.serviceId);
|
||||
add(`${registry}/${artifact.upstream.mirrorRepository}:${artifact.upstream.mirrorTag}`, "CI.json upstream image", "upstream mirror tag", artifact.serviceId);
|
||||
add(artifact.upstream.mirrorDigestRef, "CI.json upstream image", "upstream mirror digest pin", artifact.serviceId);
|
||||
}
|
||||
|
||||
function composeServiceImageRefs(config: UniDeskConfig): Array<{ ref: string; serviceId?: string; reason: string }> {
|
||||
const composePath = rootPath(config.docker.composeFile);
|
||||
if (!existsSync(composePath)) return [];
|
||||
const raw = readFileSync(composePath, "utf8");
|
||||
const refs: Array<{ ref: string; serviceId?: string; reason: string }> = [];
|
||||
let inServices = false;
|
||||
let currentService: string | null = null;
|
||||
let currentHasBuild = false;
|
||||
for (const line of raw.split("\n")) {
|
||||
if (/^services:\s*$/u.test(line)) {
|
||||
inServices = true;
|
||||
continue;
|
||||
}
|
||||
if (inServices && /^[A-Za-z0-9_-]+:\s*$/u.test(line)) break;
|
||||
if (!inServices) continue;
|
||||
const serviceMatch = line.match(/^ ([A-Za-z0-9_.-]+):\s*$/u);
|
||||
if (serviceMatch) {
|
||||
if (currentService !== null && currentHasBuild) {
|
||||
refs.push({ ref: `${config.docker.projectName}-${currentService}`, serviceId: serviceIdForComposeService(currentService, config), reason: "Compose default build image" });
|
||||
}
|
||||
currentService = serviceMatch[1];
|
||||
currentHasBuild = false;
|
||||
continue;
|
||||
}
|
||||
if (currentService === null) continue;
|
||||
if (/^\s{4}build:\s*$/u.test(line)) currentHasBuild = true;
|
||||
const imageMatch = line.match(/^\s{4}image:\s*(?:"([^"]+)"|'([^']+)'|([^#\s]+))/u);
|
||||
if (imageMatch) {
|
||||
refs.push({ ref: imageMatch[1] ?? imageMatch[2] ?? imageMatch[3] ?? "", serviceId: serviceIdForComposeService(currentService, config), reason: "Compose explicit image" });
|
||||
}
|
||||
}
|
||||
if (currentService !== null && currentHasBuild) {
|
||||
refs.push({ ref: `${config.docker.projectName}-${currentService}`, serviceId: serviceIdForComposeService(currentService, config), reason: "Compose default build image" });
|
||||
}
|
||||
return refs.filter((item) => item.ref.length > 0);
|
||||
}
|
||||
|
||||
function serviceIdForComposeService(composeService: string, config: UniDeskConfig): string | undefined {
|
||||
if (composeService === "backend-core") return "backend-core";
|
||||
if (composeService === "frontend") return "frontend";
|
||||
if (composeService === "provider-gateway") return "provider-gateway";
|
||||
const microservice = config.microservices.find((item) => item.repository.composeService === composeService);
|
||||
return microservice?.id;
|
||||
}
|
||||
|
||||
function deployServiceCommits(): DeployServiceCommit[] {
|
||||
const path = rootPath("deploy.json");
|
||||
if (!existsSync(path)) return [];
|
||||
const parsed = asRecord(JSON.parse(readFileSync(path, "utf8")) as unknown, "deploy.json");
|
||||
const environments = asRecord(parsed.environments, "deploy.json.environments");
|
||||
const commits: DeployServiceCommit[] = [];
|
||||
for (const [environment, envValue] of Object.entries(environments)) {
|
||||
const env = asRecord(envValue, `deploy.json.environments.${environment}`);
|
||||
const services = Array.isArray(env.services) ? env.services : [];
|
||||
for (const [index, serviceValue] of services.entries()) {
|
||||
const service = asRecord(serviceValue, `deploy.json.environments.${environment}.services[${index}]`);
|
||||
const serviceId = stringValue(service.id);
|
||||
const commitId = stringValue(service.commitId);
|
||||
if (serviceId.length > 0 && /^[0-9a-f]{40}$/iu.test(commitId)) commits.push({ environment, serviceId, commitId });
|
||||
}
|
||||
}
|
||||
return commits;
|
||||
}
|
||||
|
||||
function stableImageRefsForService(serviceId: string, config: UniDeskConfig): string[] {
|
||||
if (serviceId === "backend-core") return ["unidesk-backend-core"];
|
||||
if (serviceId === "frontend") return ["unidesk-frontend", "unidesk-frontend:dev"];
|
||||
if (serviceId === "code-queue") return ["unidesk-code-queue", "unidesk-code-queue:dev", "unidesk-code-queue:d601"];
|
||||
if (serviceId === "k3sctl-adapter") return ["unidesk-k3sctl-adapter", "unidesk-k3sctl-adapter:d601"];
|
||||
if (serviceId === "decision-center") return ["unidesk-decision-center", "unidesk-decision-center:dev", "unidesk-decision-center:d601"];
|
||||
if (serviceId === "mdtodo") return ["unidesk-mdtodo", "unidesk-mdtodo:dev", "unidesk-mdtodo:d601"];
|
||||
if (serviceId === "claudeqq") return ["unidesk-claudeqq", "unidesk-claudeqq:dev", "unidesk-claudeqq:d601"];
|
||||
if (serviceId === "findjob") return ["findjob-server"];
|
||||
if (serviceId === "pipeline") return ["pipeline-v2-control"];
|
||||
if (serviceId === "met-nonlinear") return ["met-nonlinear-ml", "met-nonlinear-ml:tf26"];
|
||||
const microservice = config.microservices.find((item) => item.id === serviceId);
|
||||
if (microservice === undefined) return [];
|
||||
return [microservice.repository.composeService, `unidesk-${microservice.repository.composeService}`];
|
||||
}
|
||||
|
||||
function protectedStorageRefs(config: UniDeskConfig): ProtectedStorageRef[] {
|
||||
return [
|
||||
{
|
||||
kind: "docker-volume",
|
||||
ref: config.database.volume,
|
||||
risk: "blocked",
|
||||
reason: "PostgreSQL PGDATA named volume; database cleanup requires a verified backup and is outside this image-only plan.",
|
||||
},
|
||||
{
|
||||
kind: "path",
|
||||
ref: rootPath(".state", "baidu-netdisk"),
|
||||
risk: "blocked",
|
||||
reason: "Baidu Netdisk OAuth, transfer, and staging state must not be removed by Docker image cleanup.",
|
||||
},
|
||||
{
|
||||
kind: "path",
|
||||
ref: "/home/ubuntu/.unidesk/registry-storage",
|
||||
risk: "blocked",
|
||||
reason: "D601 artifact registry storage is protected; registry cleanup is not part of main-server Docker image cleanup.",
|
||||
},
|
||||
{
|
||||
kind: "policy",
|
||||
ref: "docker-volumes-and-host-data",
|
||||
risk: "blocked",
|
||||
reason: "This plan must not generate docker volume rm, compose down -v, database cleanup, or host data directory removal commands.",
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
function protectedReasonsForImage(
|
||||
image: DockerImageInventoryItem,
|
||||
relatedContainers: DockerContainerInventoryItem[],
|
||||
runningImageIds: Set<string>,
|
||||
desiredRefs: Map<string, DesiredImageRef[]>,
|
||||
desiredCommitsByService: Record<string, string[]>,
|
||||
): string[] {
|
||||
const reasons: string[] = [];
|
||||
if (runningImageIds.has(image.id)) reasons.push("used-by-running-container");
|
||||
if (relatedContainers.some((container) => container.state !== "running")) reasons.push("referenced-by-stopped-container-review-container-first");
|
||||
for (const ref of [...image.repoTags, ...image.repoDigests]) {
|
||||
const matches = desiredRefs.get(normalizeImageRef(ref)) ?? [];
|
||||
for (const match of matches) reasons.push(`desired-image-ref:${match.source}:${match.ref}`);
|
||||
}
|
||||
const labelService = image.labels["unidesk.ai/service-id"] ?? image.labels["unidesk.service.id"];
|
||||
const labelCommit = image.labels["unidesk.ai/source-commit"] ?? image.labels["unidesk.source.commit"] ?? image.labels["org.opencontainers.image.revision"];
|
||||
if (labelService !== undefined && labelCommit !== undefined && (desiredCommitsByService[labelService] ?? []).includes(labelCommit)) {
|
||||
reasons.push(`desired-label:${labelService}@${labelCommit}`);
|
||||
}
|
||||
return [...new Set(reasons)];
|
||||
}
|
||||
|
||||
function imageSummary(image: DockerImageInventoryItem): CleanupImageSummary {
|
||||
return {
|
||||
id: image.id,
|
||||
shortId: image.id.replace(/^sha256:/u, "").slice(0, 12),
|
||||
repoTags: image.repoTags,
|
||||
repoDigests: image.repoDigests,
|
||||
sizeBytes: image.sizeBytes,
|
||||
createdAt: image.createdAt,
|
||||
};
|
||||
}
|
||||
|
||||
function containerSummary(container: DockerContainerInventoryItem): { id: string; name: string; state: string; status: string; imageRef: string } {
|
||||
return {
|
||||
id: shortContainerId(container.id),
|
||||
name: container.name,
|
||||
state: container.state,
|
||||
status: container.status,
|
||||
imageRef: container.imageRef,
|
||||
};
|
||||
}
|
||||
|
||||
function candidateRisk(image: DockerImageInventoryItem, ageHours: number | null): Exclude<RiskLevel, "blocked"> {
|
||||
const refs = [...image.repoTags, ...image.repoDigests].join(" ");
|
||||
const hasServiceLabel = image.labels["unidesk.ai/service-id"] !== undefined || image.labels["unidesk.service.id"] !== undefined;
|
||||
if (hasServiceLabel || /(^|[/:-])unidesk([/:_-]|$)/iu.test(refs)) return "high";
|
||||
if (image.repoTags.length > 0 || ageHours === null || ageHours < 24) return "medium";
|
||||
return "low";
|
||||
}
|
||||
|
||||
function candidateReasons(image: DockerImageInventoryItem, ageHours: number | null): string[] {
|
||||
const reasons = ["not-used-by-any-container", "not-present-in-deploy-or-ci-protected-refs"];
|
||||
if (image.repoTags.length === 0) reasons.push("dangling-or-untagged-image");
|
||||
else reasons.push("tagged-image-requires-extra-review");
|
||||
if (ageHours !== null) reasons.push(`age-hours:${Math.round(ageHours * 10) / 10}`);
|
||||
return reasons;
|
||||
}
|
||||
|
||||
function reviewChecklistForCandidate(risk: Exclude<RiskLevel, "blocked">, image: DockerImageInventoryItem): string[] {
|
||||
const checklist = [
|
||||
"Confirm docker ps -a still shows no container using this image ID.",
|
||||
"Confirm the image is not a rollback target in deploy.json, CI.json, or recent deployment notes.",
|
||||
];
|
||||
if (risk === "high") checklist.push("High risk because the image looks UniDesk/service-related; require explicit operator approval before removal.");
|
||||
if (image.repoTags.length > 0) checklist.push("Tagged image: review every tag in the command before approving deletion.");
|
||||
return checklist;
|
||||
}
|
||||
|
||||
function imageRemoveCommand(image: DockerImageInventoryItem): string[] {
|
||||
const refs = [...image.repoTags, ...image.repoDigests].filter((tag) => tag !== "<none>:<none>" && tag !== "<none>@<none>");
|
||||
if (refs.length > 0) return ["docker", "image", "rm", ...refs];
|
||||
return ["docker", "image", "rm", image.id];
|
||||
}
|
||||
|
||||
function imageAgeHours(createdAt: string | null, observedAt: string): number | null {
|
||||
if (createdAt === null) return null;
|
||||
const created = Date.parse(createdAt);
|
||||
const observed = Date.parse(observedAt);
|
||||
if (!Number.isFinite(created) || !Number.isFinite(observed)) return null;
|
||||
return Math.max(0, (observed - created) / 3_600_000);
|
||||
}
|
||||
|
||||
function imageFromInspect(value: unknown): DockerImageInventoryItem {
|
||||
const record = asRecord(value, "docker image inspect item");
|
||||
const config = asRecord(record.Config ?? {}, "docker image inspect item.Config");
|
||||
return {
|
||||
id: stringValue(record.Id),
|
||||
repoTags: stringArrayValue(record.RepoTags).filter((tag) => tag !== "<none>:<none>"),
|
||||
repoDigests: stringArrayValue(record.RepoDigests).filter((digest) => digest !== "<none>@<none>"),
|
||||
sizeBytes: numberValue(record.Size),
|
||||
createdAt: stringValue(record.Created) || null,
|
||||
labels: stringRecordValue(config.Labels),
|
||||
};
|
||||
}
|
||||
|
||||
function containerFromInspect(value: unknown): DockerContainerInventoryItem {
|
||||
const record = asRecord(value, "docker container inspect item");
|
||||
const config = asRecord(record.Config ?? {}, "docker container inspect item.Config");
|
||||
const state = asRecord(record.State ?? {}, "docker container inspect item.State");
|
||||
return {
|
||||
id: stringValue(record.Id),
|
||||
name: stringValue(record.Name).replace(/^\//u, ""),
|
||||
imageRef: stringValue(config.Image),
|
||||
imageId: stringValue(record.Image),
|
||||
state: stringValue(state.Status),
|
||||
status: stringValue(state.Status),
|
||||
labels: stringRecordValue(config.Labels),
|
||||
};
|
||||
}
|
||||
|
||||
function parseJsonArray(text: string, label: string): unknown[] {
|
||||
const trimmed = text.trim();
|
||||
if (trimmed.length === 0) return [];
|
||||
const parsed = JSON.parse(trimmed) as unknown;
|
||||
if (!Array.isArray(parsed)) throw new Error(`${label} did not return a JSON array`);
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function parseJsonRecord(text: string, label: string): Record<string, unknown> {
|
||||
const trimmed = text.trim();
|
||||
if (trimmed.length === 0) return {};
|
||||
return asRecord(JSON.parse(trimmed) as unknown, label);
|
||||
}
|
||||
|
||||
function asRecord(value: unknown, label: string): Record<string, unknown> {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value)) throw new Error(`${label} must be an object`);
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function stringValue(value: unknown): string {
|
||||
return typeof value === "string" ? value : "";
|
||||
}
|
||||
|
||||
function numberValue(value: unknown): number {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : 0;
|
||||
}
|
||||
|
||||
function stringArrayValue(value: unknown): string[] {
|
||||
if (!Array.isArray(value)) return [];
|
||||
return value.filter((item): item is string => typeof item === "string" && item.length > 0);
|
||||
}
|
||||
|
||||
function stringRecordValue(value: unknown): Record<string, string> {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value)) return {};
|
||||
const result: Record<string, string> = {};
|
||||
for (const [key, entry] of Object.entries(value)) {
|
||||
if (typeof entry === "string") result[key] = entry;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function chunks<T>(values: T[], size: number): T[][] {
|
||||
const result: T[][] = [];
|
||||
for (let index = 0; index < values.length; index += size) result.push(values.slice(index, index + size));
|
||||
return result;
|
||||
}
|
||||
|
||||
function imageRefVariants(ref: string): string[] {
|
||||
const normalized = normalizeImageRef(ref);
|
||||
const variants = new Set<string>([normalized]);
|
||||
if (!hasTagOrDigest(normalized)) variants.add(`${normalized}:latest`);
|
||||
if (normalized.startsWith("docker.io/library/")) variants.add(normalized.replace(/^docker\.io\/library\//u, ""));
|
||||
if (normalized.startsWith("docker.io/")) variants.add(normalized.replace(/^docker\.io\//u, ""));
|
||||
return [...variants];
|
||||
}
|
||||
|
||||
function normalizeImageRef(ref: string): string {
|
||||
return ref.trim().replace(/^docker\.io\/library\//u, "");
|
||||
}
|
||||
|
||||
function hasTagOrDigest(ref: string): boolean {
|
||||
if (ref.includes("@")) return true;
|
||||
const slashIndex = ref.lastIndexOf("/");
|
||||
const colonIndex = ref.lastIndexOf(":");
|
||||
return colonIndex > slashIndex;
|
||||
}
|
||||
|
||||
function looksLikeImageRef(value: string): boolean {
|
||||
if (value.endsWith("Dockerfile") || value.endsWith(".yml") || value.endsWith(".yaml") || value.endsWith(".json")) return false;
|
||||
return value.includes(":") || value.includes("@");
|
||||
}
|
||||
|
||||
function dedupeDesiredRefs(refs: DesiredImageRef[]): DesiredImageRef[] {
|
||||
const seen = new Set<string>();
|
||||
const result: DesiredImageRef[] = [];
|
||||
for (const ref of refs) {
|
||||
const key = `${ref.ref}\0${ref.source}\0${ref.reason}\0${ref.serviceId ?? ""}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
result.push(ref);
|
||||
}
|
||||
return result.sort((left, right) => left.ref.localeCompare(right.ref) || left.source.localeCompare(right.source));
|
||||
}
|
||||
|
||||
function commandError(command: string[], message: string, exitCode: number | null, stderr: string): DockerCleanupInventory["collection"]["errors"][number] {
|
||||
return { command, message, exitCode, stderrTail: stderr.slice(-1200) };
|
||||
}
|
||||
|
||||
function shortContainerId(id: string): string {
|
||||
return id.slice(0, 12);
|
||||
}
|
||||
|
||||
function shellQuote(value: string): string {
|
||||
if (/^[A-Za-z0-9_./:@=-]+$/u.test(value)) return value;
|
||||
return `'${value.replace(/'/g, `'\\''`)}'`;
|
||||
}
|
||||
Reference in New Issue
Block a user