feat: add D601 dev backend path
This commit is contained in:
@@ -25,7 +25,7 @@
|
||||
- Run all user services as Docker containers; these user-facing services are mounted onto the UniDesk core and the core can still run without them
|
||||
- Includes frontend gateway, task scheduler, project management, provider ingress, and other stateless modules
|
||||
- Instances can scale horizontally; failure recovery requires no state synchronization
|
||||
- Only the frontend gateway and provider ingress are public; core REST APIs and PostgreSQL remain on the Docker internal network
|
||||
- Only the production frontend gateway, dev frontend proxy and provider ingress are unrestricted public entries; core REST APIs and PostgreSQL remain on the Docker internal network or explicitly restricted host mappings. The dev frontend proxy rule is owned by `docs/reference/dev-environment.md`.
|
||||
- Frontend Time Zone Policy
|
||||
- All UniDesk frontend timestamps, dates, clocks, update times, heartbeat times, Trace times, Gantt axis labels, export date stamps, and `datetime-local` values must render as Beijing time.
|
||||
- Beijing time means IANA timezone `Asia/Shanghai` / UTC+8, regardless of the browser timezone, host system timezone, container timezone, or server-side `project.timezone` value.
|
||||
|
||||
@@ -10,6 +10,7 @@ UniDesk CI is hosted on the D601 native k3s cluster with Tekton Pipelines and Te
|
||||
- Manifests: `src/components/microservices/k3sctl-adapter/k3s/ci/`.
|
||||
- CLI entry: `bun scripts/cli.ts ci install|status|run|run-dev-e2e|logs`.
|
||||
- Dev namespace e2e runner: `bun scripts/cli.ts ci run-dev-e2e`; authoritative runner path, manifest contract and safety boundary are in `docs/reference/dev-ci-runner.md`.
|
||||
- Rust backend-core check/build boundary: CI may run `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust` on D601; the master server must not compile Rust for backend-core iteration. The authoritative dev environment rule is `docs/reference/dev-environment.md`.
|
||||
|
||||
## Pipeline Scope
|
||||
|
||||
@@ -17,7 +18,7 @@ Each commit CI run performs:
|
||||
|
||||
- `git clone` and checkout of the requested repository revision.
|
||||
- `bun install --frozen-lockfile` at the repo root and `src/`, because `bun scripts/cli.ts check` compiles all `src/components` and needs the component workspace lockfile for frontend React dependencies.
|
||||
- `bun scripts/cli.ts check`.
|
||||
- `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust`, so Rust backend-core is checked only inside the D601 CI execution boundary.
|
||||
- Temporary `code-queue-ci-read` Deployment and ClusterIP Service in `unidesk-ci`.
|
||||
- Code Queue read performance checks against the production PostgreSQL through `d601-tcp-egress-gateway`.
|
||||
|
||||
@@ -93,4 +94,4 @@ bun scripts/cli.ts ci logs <runId>
|
||||
|
||||
## Trigger Boundary
|
||||
|
||||
`unidesk-ci.triggers.yaml` installs the EventListener, TriggerBinding and TriggerTemplate, but the EventListener remains a normal in-cluster Service. Do not expose it through NodePort, LoadBalancer or an unrestricted public ingress. If GitHub or another Git remote needs webhook delivery, add a UniDesk-controlled frontend/backend route with secret verification and then proxy to the EventListener; keep frontend and provider ingress as the only unrestricted public entry points.
|
||||
`unidesk-ci.triggers.yaml` installs the EventListener, TriggerBinding and TriggerTemplate, but the EventListener remains a normal in-cluster Service. Do not expose it through NodePort, LoadBalancer or an unrestricted public ingress. If GitHub or another Git remote needs webhook delivery, add a UniDesk-controlled frontend/backend route with secret verification and then proxy to the EventListener; keep only the documented main-server public entrypoints: production frontend, dev frontend proxy and provider ingress. The dev frontend public port is defined in `docs/reference/dev-environment.md`.
|
||||
|
||||
@@ -7,13 +7,13 @@ UniDesk 的统一 CLI 入口是根目录 `scripts/cli.ts`,运行方式固定
|
||||
- `help` 输出命令索引,适合作为交互式入口。
|
||||
- `--main-server-ip <ip> <command>` 默认通过公网 frontend 登录态调用主 server 的同源 API 代理,不要求计算节点持有主 server SSH key;显式提供 `--main-server-key` 或 `--main-server-transport ssh` 时才使用旧 SSH 传输。
|
||||
- `config show` 读取并校验根目录 `config.json`,不从环境变量、默认值或隐藏文件静默补配置。
|
||||
- `check` 默认只执行轻量配置校验、Bun 版本检查和 Bun Transpiler 语法解析(覆盖 CLI 入口、主要 `scripts/` 模块和核心组件入口,不做类型推导);关键文件存在性、`scripts/` TypeScript 类型检查、`src/components/` TypeScript 类型检查、Docker Compose config 和日志轮转策略扫描默认不启用,分别通过 `--files`、`--scripts-typecheck`、`--components`、`--compose`、`--logs` 开启,或用 `--full` 一次性开启。
|
||||
- `check` 默认只执行轻量配置校验、Bun 版本检查和 Bun Transpiler 语法解析(覆盖 CLI 入口、主要 `scripts/` 模块和核心组件入口,不做类型推导);关键文件存在性、`scripts/` TypeScript 类型检查、`src/components/` TypeScript 类型检查、Docker Compose config 和日志轮转策略扫描默认不启用,分别通过 `--files`、`--scripts-typecheck`、`--components`、`--compose`、`--logs` 开启,或用 `--full` 一次性开启。`--rust` 只允许在 D601 CI/dev execution 中配合 `UNIDESK_D601_RUST_CHECK=1` 使用,长期规则见 `docs/reference/dev-environment.md`。
|
||||
- `server start` 创建异步 job,在后台执行 Docker 构建和启动;命令本身只负责返回 job id、日志路径和启动命令。
|
||||
- `server stop` 创建异步 job,在后台停止固定 Compose project 中的全部 UniDesk 服务。
|
||||
- `server status` 查询公开端口、受限宿主端口、内部端口、主机 swap 摘要、Compose 容器、core/frontend/provider/database 健康检查和访问 URL;D601 Code Queue 使用的 PostgreSQL/OA Event Flow host mapping 必须出现在受限宿主端口而不是无条件公开入口中。低内存主 server 上 `swap.warning` 非空时,先执行 `server swap status` 或 `server swap ensure`。
|
||||
- `server status` 查询公开端口、受限宿主端口、内部端口、主机 swap 摘要、Compose 容器、core/frontend/dev-frontend/provider/database 健康检查和访问 URL;D601 Code Queue 使用的 PostgreSQL/OA Event Flow host mapping 必须出现在受限宿主端口而不是无条件公开入口中。低内存主 server 上 `swap.warning` 非空时,先执行 `server swap status` 或 `server swap ensure`。
|
||||
- `server swap status|ensure [--path /swapfile] [--size 2GiB] [--dry-run]` 是主 server swap 管理入口。`status` 仅读 `/proc/meminfo`、`/proc/swaps` 和 `/etc/fstab` 并返回 JSON;`ensure` 在已有任何 active swap 时只报告 no-op,在无 active swap 时创建固定 swapfile、`chmod 600`、`mkswap`、`swapon` 并尽量写入 `/etc/fstab`。输出必须包含 `before`、`after`、total memory、active swap、持久化状态、关键动作和错误详情;若 swap 已启用但 fstab 写入失败,状态为 `degraded`,调用者需按返回的 detail 修复持久化。
|
||||
- `server logs` 返回 `logs/` 文件日志和 Docker 容器日志的尾部,默认限制输出大小,避免日志爆炸。实现必须只读取文件末尾字节,不得为了 tail 先把巨大日志完整读入 CLI 内存。
|
||||
- `server rebuild <backend-core|frontend|provider-gateway|todo-note|code-queue-mgr|project-manager|baidu-netdisk|oa-event-flow>` 创建异步 job,先构建目标服务镜像,随后在 `.state/locks/server-compose.lock` 串行保护下用 `--no-deps --force-recreate` 替换目标 service 并等待容器 `healthy/running`;该命令用于替代手工删除容器的兜底流程,其中 `todo-note`、`code-queue-mgr`、`project-manager`、`baidu-netdisk` 和 `oa-event-flow` 只重建主 server 承载的对应后端,不会重建或删除 database 命名卷。D601 Code Queue 执行面不由 `server rebuild` 管理。
|
||||
- `server rebuild <backend-core|frontend|dev-frontend-proxy|provider-gateway|todo-note|code-queue-mgr|project-manager|baidu-netdisk|oa-event-flow>` 创建异步 job,先构建目标服务镜像,随后在 `.state/locks/server-compose.lock` 串行保护下用 `--no-deps --force-recreate` 替换目标 service 并等待容器 `healthy/running`;该命令用于替代手工删除容器的兜底流程,其中 `dev-frontend-proxy` 只更新主 server dev 入口薄代理,`todo-note`、`code-queue-mgr`、`project-manager`、`baidu-netdisk` 和 `oa-event-flow` 只重建主 server 承载的对应后端,不会重建或删除 database 命名卷。D601 Code Queue 执行面不由 `server rebuild` 管理,Rust backend-core 迭代不得用 `server rebuild backend-core` 在 master server 编译,规则见 `docs/reference/dev-environment.md`。
|
||||
- `provider attach <providerId> [--master-server URL] [--up] [--force]` 在新计算节点生成两项配置的 provider-gateway 挂载包:`.state/provider-<ID>.env` 默认只包含 `UNIDESK_MASTER_SERVER` 与 `PROVIDER_ID`,`provider-<ID>.yml` 固定 Docker socket、`pid: "host"`、`restart: always`、只读 `/workspace` 和 SSH 维护私钥挂载;`--up` 会立即执行生成的 `docker compose up -d --build`。
|
||||
- `ssh <providerId> [ssh-like args...]` 通过 backend-core 内网 WebSocket broker 和 provider-gateway 的 Host SSH / WSL SSH 维护桥连接目标节点;无后续参数时进入远端登录 shell,有后续参数时按 ssh 远端命令体验执行并返回远端 exit code。
|
||||
- `ssh <providerId> apply-patch [tool args...] < patch.diff` 直接调用远端注入的 `apply_patch` 工具,并把本地 stdin 中的标准 `*** Begin Patch` / `*** End Patch` patch 流透传给目标节点。
|
||||
@@ -22,7 +22,7 @@ UniDesk 的统一 CLI 入口是根目录 `scripts/cli.ts`,运行方式固定
|
||||
- `microservice list/status/health/diagnostics/tunnel-self-test/proxy` 通过 backend-core 内网 API 管理挂载在计算节点 Docker 或 k3s 控制面中的用户服务(底层命令名仍为 microservice);`health`、`diagnostics`、`tunnel-self-test` 和 `proxy` 会走真实 backend-core -> provider-gateway 或 k3sctl-adapter -> 节点服务链路,`proxy` 支持受控 JSON 请求体并对超大响应 body 默认输出有界预览,规则见 `docs/reference/microservices.md`。
|
||||
- `decision upload/list/show/health` 通过 backend-core 用户服务代理访问 D601 k3s Decision Center,用于上传会议记录/决议 Markdown、列出权威记录、查看详情和健康检查;它不得直连 D601 Service、NodePort 或 provider-gateway 业务 HTTP。
|
||||
- `decision diary import <markdown-file>` 将带 `# YYYY年M月D日`、`# YYYY-MM-DD` 或 `# YYYY/M/D` 标题的工作日志拆成每天一篇 Markdown 日记,按 `YYYY-MM/YYYY-MM-DD.md` 虚拟路径写入 Decision Center PostgreSQL;`decision diary list/months/show` 分别用于按月/日期查询、列出月份和查看单日正文。
|
||||
- `deploy check/plan/apply` 默认从根目录 `deploy.json` 读取服务 repo 与 commit 期望状态,join `config.json` 和现有 manifest 后使用 target-side build 单一路径校验或更新已支持目标;`deploy plan --env dev|prod` 只从 `origin/master:deploy.json#environments.<env>` 读取 manifest 并输出 dry-run 环境计划,不使用本地 dirty worktree;当前 `deploy apply --env dev` 不做 D601 服务 rollout,dev desired-state smoke 使用 `ci run-dev-e2e`;规则见 `docs/reference/deploy.md` 和 `docs/reference/dev-ci-runner.md`。
|
||||
- `deploy check/plan/apply` 默认从根目录 `deploy.json` 读取服务 repo 与 commit 期望状态,join `config.json` 和现有 manifest 后使用 target-side build 单一路径校验或更新已支持目标;`deploy plan --env dev|prod` 只从 `origin/master:deploy.json#environments.<env>` 读取 manifest 并输出 dry-run 环境计划,不使用本地 dirty worktree;当前 `deploy apply --env dev` 只支持 D601 `backend-core` 与 `frontend` persistent dev rollout,dev desired-state smoke 使用 `ci run-dev-e2e`;规则见 `docs/reference/deploy.md`、`docs/reference/dev-environment.md` 和 `docs/reference/dev-ci-runner.md`。
|
||||
- `dev-env validate [--manifest path] [--kubectl-dry-run]` 离线校验 D601 `unidesk-dev` namespace、dev PostgreSQL 底座和 dev workload manifest。默认检查 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-foundation.k8s.yaml`;也可显式校验 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml` 或 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-code-queue.k8s.yaml`。所有 namespaced 对象必须只落到 `unidesk-dev`,foundation manifest 必须包含 `postgres-dev` StatefulSet/Service、dev secret/config、迁移 Job 和 DB URL guard,core manifest 必须包含 `backend-core-dev`/`frontend-dev` Deployment/Service,Code Queue dev manifest 必须包含 `code-queue-scheduler-dev`、`code-queue-read-dev`、`code-queue-write-dev` 和 dev provider egress proxy。加 `--kubectl-dry-run` 时额外执行 `kubectl apply --dry-run=client --validate=false -f <manifest>`,仍不 apply 资源。
|
||||
- `dev-env prewarm-images [--image image] [--provider-id D601] [--no-pull] [--proxy-url URL] [--pull-timeout-ms N] [--dry-run]` 创建异步 job,通过 UniDesk SSH 维护桥在 D601 上把开发底座依赖镜像从 Docker 缓存导入原生 k3s containerd。默认镜像是 `postgres:16-alpine` 和 `rancher/mirrored-library-busybox:1.36.1`,用于避免 `postgres-dev` 与 local-path helper pod 卡在外部 registry 拉取。该命令固定验证 `/etc/rancher/k3s/k3s.yaml` 指向的 native k3s 上下文,并输出 `dev_env_containerd_image_ready=...` 作为成功判据;它不 apply manifest、不修改生产 `unidesk` namespace。
|
||||
- `ci install|status|run|run-dev-e2e|logs` 管理 D601 原生 k3s 上的 Tekton CI。`run` 手动创建每 commit 检查和 Code Queue 只读性能门禁;`run-dev-e2e` 的 Git 控制 runner、短 launcher、host fetch 边界、临时 smoke namespace 和 no-CD 规则只在 `docs/reference/dev-ci-runner.md` 定义;Tekton CI 通用规则见 `docs/reference/ci.md`。
|
||||
@@ -37,15 +37,15 @@ UniDesk 的统一 CLI 入口是根目录 `scripts/cli.ts`,运行方式固定
|
||||
- 所有 `codex` 查询和管理命令必须走与 WebUI 相同的 backend-core 私有代理路径 `/api/microservices/code-queue/proxy/...`;CLI 不得为了提交、移动、中断、取消或队列管理直接调用 D601 内部 Service、数据库、pod curl 或 k3sctl scheduler 子服务。若该路径失败,应先修复 CLI/backend/provider tunnel 链路,而不是绕过控制面。
|
||||
- `job list [--limit N] [--include-command]` 与 `job status <jobId|latest> [--tail-bytes N]` 查询 `.state/jobs/` 文件系统状态,是异步命令的可观测入口。`job list` 默认只返回最新 50 条摘要;`job status` 默认只返回 stdout/stderr 末尾 12000 字节,并带 `tailPolicy` 与完整日志路径。
|
||||
- `debug health`、`debug dispatch` 与 `debug task` 走真实内部 core、WebSocket、数据库、provider、系统指标、Docker 状态和 Host SSH 维护桥流程,只用于开发调试,不写入 `TEST.md` 的正式验收步骤。
|
||||
- `e2e run [--only pattern[,pattern...]] [--skip pattern[,pattern...]]` 使用 publicHost 派生的公开 frontend/provider ingress URL,并通过 Docker 内网验证 core API、PostgreSQL、provider self-connection、系统指标曲线、Docker 状态快照、provider.upgrade 预检和 Playwright 前端页面,是交付前的自动化 E2E 门禁;CLI 默认输出 check 状态摘要,完整诊断写入 `resultPath`,日常迭代应优先用 `--only` / `--skip` 跑最小必要集合。
|
||||
- `e2e run [--only pattern[,pattern...]] [--skip pattern[,pattern...]]` 使用 publicHost 派生的公开 production frontend/dev frontend/provider ingress URL,并通过 Docker 内网验证 core API、PostgreSQL、provider self-connection、系统指标曲线、Docker 状态快照、provider.upgrade 预检和 Playwright 前端页面,是交付前的自动化 E2E 门禁;CLI 默认输出 check 状态摘要,完整诊断写入 `resultPath`,日常迭代应优先用 `--only` / `--skip` 跑最小必要集合。
|
||||
|
||||
## Async Job State
|
||||
|
||||
长时操作采用 Fire-and-Forget 模式:CLI 创建 `.state/jobs/{jobId}.json`,后台进程执行真实命令,并将 stdout、stderr 分别写入 `.state/jobs/{jobId}.stdout.log` 与 `.state/jobs/{jobId}.stderr.log`。调用者通过 `bun scripts/cli.ts job status <jobId>` 查询进度和尾部输出。
|
||||
|
||||
`server rebuild` 与 `server start`、`server stop` 一样必须通过返回的 job id 确认结果;不要把连续 `server rebuild` 命令理解成“前一个重建已完成”,因为两个命令只是在快速创建异步 job。重建 frontend 的标准流程是运行 `bun scripts/cli.ts server rebuild frontend`,随后轮询 `bun scripts/cli.ts job status <jobId>` 到 `succeeded`,再用 `server status` 或 `e2e run` 验证公网 frontend;重建 Todo Note 后端使用 `bun scripts/cli.ts server rebuild todo-note`,随后用 `microservice health todo-note` 和 `microservice proxy todo-note /api/instances` 验证;重建 Code Queue Manager 使用 `bun scripts/cli.ts server rebuild code-queue-mgr`,随后用 `microservice health code-queue-mgr`、`microservice health code-queue` 和 `codex submit --dry-run` 验证主 server 控制面路径;重建 Project Manager 后端使用 `bun scripts/cli.ts server rebuild project-manager`,随后用 `microservice health project-manager` 和 `microservice proxy project-manager /api/projects` 验证;重建 Baidu Netdisk 后端使用 `bun scripts/cli.ts server rebuild baidu-netdisk`,随后用 `microservice health baidu-netdisk` 和 `microservice proxy baidu-netdisk /api/transfers` 验证;重建 OA Event Flow 后端使用 `bun scripts/cli.ts server rebuild oa-event-flow`,随后用 `microservice health oa-event-flow` 和 `microservice proxy oa-event-flow /api/diagnostics` 验证。D601 Code Queue 执行面和 Decision Center 后端由 D601 k3s/k8s 控制面代管,但当前不得通过维护通道直连 D601 做部署;正式 CD 控制路径另行设计,当前只允许 `ci run-dev-e2e` 启动一次性 smoke runner。不得把 `docker rm` 手工兜底当成正式交付步骤。
|
||||
`server rebuild` 与 `server start`、`server stop` 一样必须通过返回的 job id 确认结果;不要把连续 `server rebuild` 命令理解成“前一个重建已完成”,因为两个命令只是在快速创建异步 job。重建 frontend 的标准流程是运行 `bun scripts/cli.ts server rebuild frontend`,随后轮询 `bun scripts/cli.ts job status <jobId>` 到 `succeeded`,再用 `server status` 或 `e2e run` 验证公网 frontend;重建 dev 入口薄代理使用 `bun scripts/cli.ts server rebuild dev-frontend-proxy`,随后验证 `server status` 的 `urls.devFrontend` 和 `http://127.0.0.1:18083/health`;重建 Todo Note 后端使用 `bun scripts/cli.ts server rebuild todo-note`,随后用 `microservice health todo-note` 和 `microservice proxy todo-note /api/instances` 验证;重建 Code Queue Manager 使用 `bun scripts/cli.ts server rebuild code-queue-mgr`,随后用 `microservice health code-queue-mgr`、`microservice health code-queue` 和 `codex submit --dry-run` 验证主 server 控制面路径;重建 Project Manager 后端使用 `bun scripts/cli.ts server rebuild project-manager`,随后用 `microservice health project-manager` 和 `microservice proxy project-manager /api/projects` 验证;重建 Baidu Netdisk 后端使用 `bun scripts/cli.ts server rebuild baidu-netdisk`,随后用 `microservice health baidu-netdisk` 和 `microservice proxy baidu-netdisk /api/transfers` 验证;重建 OA Event Flow 后端使用 `bun scripts/cli.ts server rebuild oa-event-flow`,随后用 `microservice health oa-event-flow` 和 `microservice proxy oa-event-flow /api/diagnostics` 验证。D601 Code Queue 执行面和 Decision Center 后端由 D601 k3s/k8s 控制面代管;persistent dev backend-core/frontend 只通过 `deploy apply --env dev --service ...`,当前 Code Queue/Decision Center 仍不得通过维护通道直连 D601 做部署。不得把 `docker rm` 手工兜底当成正式交付步骤。
|
||||
|
||||
新部署入口优先使用 `deploy apply`,但当前 D601 维护直连 apply 不承担服务部署。旧的 `codex deploy` 已禁用;后续 Code Queue、Decision Center、backend-core dev、frontend dev 等 D601 服务部署应收敛到一条受控 target-side CD 路径:从 remote commit 导出源码,在目标节点一次性代理构建镜像,部署后用 live commit 校验证明不是旧服务。
|
||||
新部署入口优先使用 `deploy apply`。`deploy apply --env dev --service backend-core|frontend` 已收敛到 D601 target-side dev 路径;旧的 `codex deploy` 已禁用;后续 Code Queue、Decision Center 等 D601 服务部署应另行收敛到同一类受控 target-side CD 路径:从 remote commit 导出源码,在目标节点一次性代理构建镜像,部署后用 live commit 校验证明不是旧服务。
|
||||
|
||||
## Output Contract
|
||||
|
||||
@@ -59,11 +59,11 @@ UniDesk 的统一 CLI 入口是根目录 `scripts/cli.ts`,运行方式固定
|
||||
|
||||
## Debug Contract
|
||||
|
||||
`debug` 子命令必须复用真实模块与真实端点,禁止维护平行实现。`debug health` 会摘要展示 `/api/nodes/system-status` 和 `/api/nodes/docker-status`,避免输出完整快照造成信息爆炸。`debug dispatch` 会通过 frontend 容器的 Docker 内网访问 backend-core `/api/dispatch`,core 再通过 WebSocket 将 `docker.ps`、`provider.upgrade`、`host.ssh`、`microservice.http` 或 `echo` 任务下发给 provider gateway,因此它可以验证核心调度闭环,同时不需要公开 core REST API,也不要求 backend-core 运行镜像携带 Bun CLI。`provider.upgrade` 默认使用 `mode: "plan"` 预检;需要验证一键升级时必须显式加 `--mode schedule`,并通过 `--wait-ms` 或 `debug task` 确认任务进入 `succeeded`、result 中包含 updater 容器信息和 `policy: "always-enabled"`。`host.ssh` 默认使用 `mode: "probe"` 做短超时维护桥自检;需要执行明确命令时使用 `--ssh-command` 进入 `mode: "exec"`,并配合 `--wait-ms` 和 `debug task` 查看 stdout、stderr、exitCode 与 probeLine。`microservice.http` 只用于开发调试 provider-gateway 私有 HTTP 代理,正式用户入口应使用 `microservice` CLI 或 frontend 的用户服务页面。
|
||||
`debug` 子命令必须复用真实模块与真实端点,禁止维护平行实现。`debug health` 会摘要展示 `/api/nodes/system-status` 和 `/api/nodes/docker-status`,避免输出完整快照造成信息爆炸。`debug dispatch` 会通过 backend-core 容器内置 helper 访问 backend-core `/api/dispatch`,core 再通过 WebSocket 将 `docker.ps`、`provider.upgrade`、`host.ssh`、`microservice.http` 或 `echo` 任务下发给 provider gateway,因此它可以验证核心调度闭环,同时不需要公开 core REST API,也不要求 frontend 容器携带调试 broker。`provider.upgrade` 默认使用 `mode: "plan"` 预检;需要验证一键升级时必须显式加 `--mode schedule`,并通过 `--wait-ms` 或 `debug task` 确认任务进入 `succeeded`、result 中包含 updater 容器信息和 `policy: "always-enabled"`。`host.ssh` 默认使用 `mode: "probe"` 做短超时维护桥自检;需要执行明确命令时使用 `--ssh-command` 进入 `mode: "exec"`,并配合 `--wait-ms` 和 `debug task` 查看 stdout、stderr、exitCode 与 probeLine。`microservice.http` 只用于开发调试 provider-gateway 私有 HTTP 代理,正式用户入口应使用 `microservice` CLI 或 frontend 的用户服务页面。
|
||||
|
||||
## SSH Command
|
||||
|
||||
`ssh <providerId> [ssh-like args...]` 是面向人的终端透传入口,不包装 JSON 输出。CLI 会在宿主机启动一个 `docker exec -i unidesk-frontend bun -e ...` broker,broker 只连接 backend-core 的 Docker 内网 `/ws/ssh`,core 再把 stdin/stdout/stderr 流量通过目标 provider 的既有 WebSocket 转发到 provider-gateway,provider-gateway 最终执行维护用 SSH 连接宿主或 WSL sshd。TTY 策略固定为交互登录 shell 使用 `ssh -tt`,带远端命令的会话使用 `ssh -T`;脚本 stdin、`apply-patch` 和 `py` 这类命令模式不得被伪终端回显或注入控制字符。该入口不新增 core 公网端口,不暴露 database,也不改变 frontend/provider ingress 之外的公网边界。
|
||||
`ssh <providerId> [ssh-like args...]` 是面向人的终端透传入口,不包装 JSON 输出。CLI 会在宿主机启动 `docker exec -i unidesk-backend-core backend-core --ssh-broker ...`,broker 只连接 backend-core 的 Docker 内网 `/ws/ssh`,core 再把 stdin/stdout/stderr 流量通过目标 provider 的既有 WebSocket 转发到 provider-gateway,provider-gateway 最终执行维护用 SSH 连接宿主或 WSL sshd。TTY 策略固定为交互登录 shell 使用 `ssh -tt`,带远端命令的会话使用 `ssh -T`;脚本 stdin、`apply-patch` 和 `py` 这类命令模式不得被伪终端回显或注入控制字符。该入口不新增 core 公网端口,不暴露 database,也不改变 frontend/dev frontend/provider ingress 之外的公网边界。
|
||||
|
||||
`bun scripts/cli.ts ssh --help` 和 `bun scripts/cli.ts ssh <providerId> --help` 是本地 JSON 帮助命令,必须快速返回;不能把 `--help` 解析成 Provider ID,不能打开交互 shell,也不能等待 provider 会话。
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
`bun scripts/cli.ts codex deploy <commitId>` 是旧兼容入口,现已禁用。原因是它会通过 backend-core `host.ssh` 维护通道直连 D601 部署 Code Queue,把维护入口扩张成第二套部署系统。
|
||||
|
||||
Code Queue 后续正式部署必须走一条受控 target-side CD 路径:读取 `origin/master:deploy.json#environments.dev` 或生产 desired-state,在目标节点执行 source fetch、build、k3s image import、rollout、stamp 和 live commit 验证。当前阶段不提供 Code Queue CD;只提供 `ci run-dev-e2e` dev smoke runner,规则见 `docs/reference/dev-ci-runner.md`。
|
||||
Code Queue 后续正式部署必须走一条受控 target-side CD 路径:读取 `origin/master:deploy.json#environments.dev` 或生产 desired-state,在目标节点执行 source fetch、build、k3s image import、rollout、stamp 和 live commit 验证。当前阶段不提供 Code Queue CD;persistent dev apply 只支持 backend-core/frontend,规则见 `docs/reference/dev-environment.md`,Code Queue smoke 仍通过 `ci run-dev-e2e`,规则见 `docs/reference/dev-ci-runner.md`。
|
||||
|
||||
## Command
|
||||
|
||||
|
||||
@@ -4,11 +4,11 @@
|
||||
|
||||
## Runtime
|
||||
|
||||
TypeScript 运行时固定为 Bun。根目录 CLI、backend-core、frontend 和 provider-gateway 都直接运行 `.ts` 入口;Docker 镜像使用 `oven/bun` 基础镜像,本机命令使用 `bun scripts/cli.ts`。
|
||||
TypeScript 运行时固定为 Bun for CLI, frontend, provider-gateway and TypeScript user services. backend-core is a Rust service; Rust build/check must follow the D601 boundary in `docs/reference/dev-environment.md`. 本机命令使用 `bun scripts/cli.ts`。
|
||||
|
||||
## Network Ports
|
||||
|
||||
`config.json` 中保留 core、frontend、database 和 providerIngress 的端口字段。frontend 与 providerIngress 是面向浏览器/Provider 的公开入口;core 不允许映射公网。database 的 `port` 字段用于 D601 Code Queue 的受限 PostgreSQL 端口映射和公网阻断测试,必须配合 `network.restrictedHostAccess` 的来源白名单限制到 D601 出口地址,不能作为任意公网数据库入口。
|
||||
`config.json` 中保留 core、frontend、devFrontend、database 和 providerIngress 的端口字段。frontend、devFrontend 与 providerIngress 是面向浏览器/Provider 的公开入口;devFrontend 是 D601 `unidesk-dev` 的薄代理入口,权威规则见 `docs/reference/dev-environment.md`。core 不允许映射公网。database 的 `port` 字段用于 D601 Code Queue 的受限 PostgreSQL 端口映射和公网阻断测试,必须配合 `network.restrictedHostAccess` 的来源白名单限制到 D601 出口地址,不能作为任意公网数据库入口。
|
||||
|
||||
## Auth
|
||||
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
UniDesk deployment is driven by a desired-state manifest. The manifest answers only one question: which service should run which repository commit. Runtime topology, ports, providers, compose files, Kubernetes manifests, health paths and proxy policy remain in `config.json` and the existing service manifests.
|
||||
|
||||
Persistent D601 dev environment rules, including the public dev frontend port, `deploy apply --env dev` service scope and Rust backend-core build boundary, are owned by `docs/reference/dev-environment.md`. This document owns the generic desired-state reconciler and target-side build contract.
|
||||
|
||||
## Manifest
|
||||
|
||||
The root `deploy.json` is the single desired-state source for both prod and dev. Environment branches such as `deploy/dev` and `deploy/prod` are deprecated because they create a second control plane for version intent.
|
||||
@@ -38,13 +40,15 @@ The root `deploy.json` is the single desired-state source for both prod and dev.
|
||||
|
||||
The optional non-service execution declaration under `environments.dev` is intentionally not specified here. The only currently allowed declaration is `ci`, and its authoritative `repo`, `scriptPath`, `timeoutMs`, short launcher, host fetch boundary and no-CD rules are defined only in `docs/reference/dev-ci-runner.md`.
|
||||
|
||||
Environment mode never reads the local dirty working tree manifest. `deploy check --env ...`, `deploy plan --env ...` and `deploy apply --env ...` fetch `origin/master`, read `origin/master:deploy.json`, select `environments.<env>`, and report the manifest commit/blob, service commit IDs, target namespace, database fingerprint and Provider identity. Maintenance-channel direct D601 apply is intentionally narrow: no D601 backend-core/frontend/code-queue/Decision Center/k3sctl-adapter service deployment may use that path. `deploy apply --env prod` remains disabled until the production environment executor and authorization policy are explicitly added.
|
||||
Environment mode never reads the local dirty working tree manifest. `deploy check --env ...`, `deploy plan --env ...` and `deploy apply --env ...` fetch `origin/master`, read `origin/master:deploy.json`, select `environments.<env>`, and report the manifest commit/blob, service commit IDs, target namespace, database fingerprint and Provider identity. `deploy apply --env dev` is currently enabled only for persistent D601 dev `backend-core` and `frontend`; all other D601 services remain rejected before runtime mutation. `deploy apply --env prod` remains disabled until the production environment executor and authorization policy are explicitly added.
|
||||
|
||||
`config.json.microservices[].repository.commitId` is retained for catalog compatibility, but `deploy.json` is the deployment version authority for the reconciler.
|
||||
|
||||
## Dev CI Runner
|
||||
|
||||
Dev desired-state smoke verification is not a deploy executor. Use `bun scripts/cli.ts ci run-dev-e2e` for the Git-controlled temporary namespace runner described in `docs/reference/dev-ci-runner.md`. `deploy apply --env dev` must not roll out D601 direct or k3s-managed services in the current CI-only phase.
|
||||
Dev desired-state smoke verification is not a deploy executor. Use `bun scripts/cli.ts ci run-dev-e2e` for the Git-controlled temporary namespace runner described in `docs/reference/dev-ci-runner.md`; that command must not roll out persistent D601 services.
|
||||
|
||||
Persistent dev `backend-core` and `frontend` rollout is separate from that smoke runner and is described in `docs/reference/dev-environment.md`.
|
||||
|
||||
## D601 Dev Foundation
|
||||
|
||||
@@ -69,11 +73,11 @@ On D601, dev/prod k3s verification must use the native k3s kubeconfig explicitly
|
||||
|
||||
## D601 Dev Core
|
||||
|
||||
Phase 3 introduces the dev backend/frontend manifest at `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`. It may create only `backend-core-dev` and `frontend-dev` Deployment/Service objects in `unidesk-dev`.
|
||||
Phase 3 introduces the dev backend/frontend manifest at `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`. It may create only `backend-core-dev` and `frontend-dev` Deployment/Service objects in `unidesk-dev`; the persistent rollout contract and Rust build boundary are owned by `docs/reference/dev-environment.md`.
|
||||
|
||||
`backend-core-dev` must use `unidesk-dev-runtime-config` and `unidesk-dev-runtime-secrets`, connect to `postgres-dev.../unidesk_dev`, expose HTTP on 8080 and provider ingress on 8081, and write logs under `/var/log/unidesk-dev`. `frontend-dev` must set `CORE_INTERNAL_URL=http://backend-core-dev.unidesk-dev.svc.cluster.local:8080` and must not proxy to production backend-core.
|
||||
|
||||
The manifest keeps placeholder image tags and deploy commit values in source control. Maintenance-channel direct D601 apply must not deploy `backend-core-dev` or `frontend-dev`; the CLI rejects `deploy apply --env dev --service backend-core|frontend` before runtime mutation. Dev core deployment must be implemented later as a controlled D601 target-side action that fetches `origin/master:deploy.json`, selects `environments.dev`, materializes the requested source commit on D601, narrows the dev core control manifest to the selected Service/Deployment pair, replaces placeholders with the requested commit and dev image tag, builds on D601, imports the image into native k3s containerd, applies only the `unidesk-dev` objects and stamps the Deployment. Client dry-run and static validation are the required checks before any controlled apply:
|
||||
The manifest keeps placeholder image tags and deploy commit values in source control. The controlled `deploy apply --env dev --service backend-core|frontend` path fetches `origin/master:deploy.json`, materializes the requested source commit on D601, narrows the dev core control manifest to the selected Service/Deployment pair, replaces placeholders with the requested commit and dev image tag, builds on D601, imports the image into native k3s containerd, applies only the `unidesk-dev` objects and stamps the Deployment. Client dry-run and static validation remain useful checks before controlled apply:
|
||||
|
||||
- `bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`
|
||||
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply --dry-run=client --validate=false -f src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`
|
||||
@@ -96,7 +100,7 @@ Maintenance-channel direct D601 apply must not deploy dev Code Queue; the CLI re
|
||||
|
||||
`bun scripts/cli.ts deploy plan --env dev [--service <id>]` reads `origin/master:deploy.json#environments.dev` and prints a dry-run environment plan without checking or mutating live runtime resources. `deploy check --env dev` uses the same dry-run environment plan. `--env prod` is available for parity as a dry-run planning path; it reads `origin/master:deploy.json#environments.prod` and must not use a dirty local `deploy.json`.
|
||||
|
||||
`bun scripts/cli.ts deploy apply [--file deploy.json | --env dev] [--service <id>] [--dry-run] [--force]` starts an asynchronous job only for supported targets. Use `bun scripts/cli.ts job status <jobId> --tail-bytes 30000` to observe progress. `--dry-run` resolves the same plan but does not build or replace runtime objects. `--force` rebuilds even when the live commit matches. Environment apply is not the dev e2e trigger; use `bun scripts/cli.ts ci run-dev-e2e` for the Git-controlled temporary namespace smoke flow. `--env prod` apply is rejected, and D601 service apply is rejected before any runtime mutation.
|
||||
`bun scripts/cli.ts deploy apply [--file deploy.json | --env dev] [--service <id>] [--dry-run] [--force]` starts an asynchronous job only for supported targets. Use `bun scripts/cli.ts job status <jobId> --tail-bytes 30000` to observe progress. `--dry-run` resolves the same plan but does not build or replace runtime objects. `--force` rebuilds even when the live commit matches. Environment apply is not the dev e2e trigger; use `bun scripts/cli.ts ci run-dev-e2e` for the Git-controlled temporary namespace smoke flow. `--env dev` apply is enabled only for `backend-core` and `frontend`; `--env prod` apply is rejected.
|
||||
|
||||
All deploy commands output JSON. Long operations must use `.state/jobs/` and bounded log tails; no deploy path may succeed with missing progress output.
|
||||
|
||||
@@ -137,7 +141,7 @@ The reconciler selects the executor from `config.json`:
|
||||
- `deployment.mode=internal-sidecar` on `main-server`: use the same main-server target-side source export, Docker build, image label stamping, fixed Compose project replacement and live commit verification as direct Compose services. This class is for private sidecars such as `code-queue-mgr`; it is still versioned by `deploy.json.commitId`, not by the operator's current worktree.
|
||||
- `deployment.mode=unidesk-direct` on a provider: this executor is disabled for D601 service deployment. The historical behavior dispatched `host.ssh` to the provider, built on the provider, then used the service's provider-local compose file and project; that shape must not remain a second deployment control plane.
|
||||
- Control bridges that UniDesk needs in order to inspect or repair an orchestrator must stay in this direct class. In particular, `k3sctl-adapter` is a UniDesk-managed bridge to native k3s and must remain outside k3s; Docker packaging on Docker Desktop/WSL must create an explicit host-local bridge, currently an adapter-container SSH local tunnel, to reach `/etc/rancher/k3s/k3s.yaml` and WSL `127.0.0.1:6443`.
|
||||
- `deployment.mode=k3sctl-managed`: the target behavior is to build on the active control target, verify native k3s on the host OS/WSL distro, import the image into native k3s/containerd, apply the existing Kubernetes manifest, stamp the Deployment and wait for rollout. On D601, maintenance-channel direct execution of this behavior is not allowed for normal services; the current work is CI-only and must not roll out services. The executor must use the native kubeconfig and containerd socket, for example `/etc/rancher/k3s/k3s.yaml` and `/run/k3s/containerd/containerd.sock`; running k3s itself in Docker is forbidden for both control-plane and worker nodes. A `rancher/k3s` image or legacy container may only be used as a temporary artifact source during migration, and any active containerized k3s control plane must be stopped before verification succeeds. The executor must preload a valid `rancher/mirrored-pause:3.6` sandbox image into native k3s containerd through the provider-gateway one-shot egress path, verify its entrypoint is `/pause`, and reject fake or sleep-based replacement images. Code Queue's k3s migration executor must also stop/remove the legacy direct Docker `code-queue-backend` after k3s rollout, so there is never a second scheduler running beside the native k3s scheduler.
|
||||
- `deployment.mode=k3sctl-managed`: the target behavior is to build on the active control target, verify native k3s on the host OS/WSL distro, import the image into native k3s/containerd, apply the existing Kubernetes manifest, stamp the Deployment and wait for rollout. On D601, persistent dev apply is currently allowed only for `backend-core` and `frontend` in `unidesk-dev`; normal production services still cannot use a maintenance-channel direct rollout. The executor must use the native kubeconfig and containerd socket, for example `/etc/rancher/k3s/k3s.yaml` and `/run/k3s/containerd/containerd.sock`; running k3s itself in Docker is forbidden for both control-plane and worker nodes. A `rancher/k3s` image or legacy container may only be used as a temporary artifact source during migration, and any active containerized k3s control plane must be stopped before verification succeeds. The executor must preload a valid `rancher/mirrored-pause:3.6` sandbox image into native k3s containerd through the provider-gateway one-shot egress path, verify its entrypoint is `/pause`, and reject fake or sleep-based replacement images. Code Queue's k3s migration executor must also stop/remove the legacy direct Docker `code-queue-backend` after k3s rollout, so there is never a second scheduler running beside the native k3s scheduler.
|
||||
|
||||
Existing service-specific commands such as Code Queue deploy are disabled as direct D601 deploy paths. Their build/import/rollout semantics should converge later into one controlled target-side deployment path instead of keeping parallel implementations.
|
||||
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
# UniDesk Deployment Reference
|
||||
|
||||
主 server 使用根目录 `docker-compose.yml` 统一编排 database、backend-core、frontend、provider-gateway 以及必须留在主 server 的用户服务。当前环境本身就是主 server,因此 provider-gateway 也在同一台机器上启动,用与普通计算节点相同的 WebSocket 方式接入 core。Code Queue 按“master 低资源低抖动控制面、D601 高资源高抖动执行面”拆分:队列 CRUD、任务提交、历史摘要和轻量 Trace 读取由主 server Compose 中的 `code-queue-mgr` 直管 PostgreSQL;Codex/OpenCode scheduler、runner、dev-container、active run steer/interrupt 和执行态写回仍由 D601 原生 k3s/k8s Code Queue 执行面承担。
|
||||
主 server 使用根目录 `docker-compose.yml` 统一编排 database、backend-core、frontend、dev-frontend-proxy、provider-gateway 以及必须留在主 server 的用户服务。当前环境本身就是主 server,因此 provider-gateway 也在同一台机器上启动,用与普通计算节点相同的 WebSocket 方式接入 core。Code Queue 按“master 低资源低抖动控制面、D601 高资源高抖动执行面”拆分:队列 CRUD、任务提交、历史摘要和轻量 Trace 读取由主 server Compose 中的 `code-queue-mgr` 直管 PostgreSQL;Codex/OpenCode scheduler、runner、dev-container、active run steer/interrupt 和执行态写回仍由 D601 原生 k3s/k8s Code Queue 执行面承担。
|
||||
|
||||
## Services
|
||||
|
||||
- `database` 使用 `postgres:16-alpine`,数据保存到 named volume `unidesk_pgdata_10gb`,当前容量预算为 15 GB,初始化 SQL 位于 `src/components/database/init/`。
|
||||
- `backend-core` 是无状态核心服务,提供 Docker 内网 REST API、provider ingress WebSocket、任务调度入口和数据库访问层,Compose restart policy 必须为 `always`,确保进程崩溃或 Docker daemon 重启后自动恢复。源码拆分为 15 个职责单一的模块(`index.ts` 只做路由和启动),模块结构见 `docs/reference/repo-tree.md`。
|
||||
- `frontend` 是唯一公开 Web 控制台,提供登录、从 TSX 转译出的 React 应用资产和到 backend-core 的同源代理。
|
||||
- `backend-core` 是无状态核心服务,提供 Docker 内网 REST API、provider ingress WebSocket、任务调度入口和数据库访问层。当前 backend-core 是 Rust 服务;Rust 迭代编译边界见 `docs/reference/dev-environment.md`。源码模块结构见 `docs/reference/repo-tree.md`。
|
||||
- `frontend` 是 production 公开 Web 控制台,提供登录、从 TSX 转译出的 React 应用资产和到 backend-core 的同源代理。
|
||||
- `dev-frontend-proxy` 是主 server 上的 18083 dev UI 薄入口,代理到 D601 `unidesk-dev/frontend-dev`,详细边界见 `docs/reference/dev-environment.md`。
|
||||
- `provider-gateway` 是当前主 server 的本机计算节点代理,通过 WebSocket 主动连到 provider ingress,挂载 `/var/run/docker.sock` 作为自动任务执行主路径,使用 `pid: "host"` 读取节点级进程资源,并周期性上报系统资源指标、进程占用与 Docker daemon 状态;计算节点 provider-gateway 还必须把 egress proxy 仅发布到宿主 loopback `127.0.0.1:18789` 供本节点执行环境出网,维护用 Host SSH / WSL SSH 私钥目录只读挂载到 `/run/host-ssh`,不得作为自动任务调度主路径。
|
||||
- `todo-note` 是主 server 承载的 Todo Note 纯后端用户服务,容器名 `todo-note-backend`,只在 Compose 内网暴露 `4211/tcp`,使用主 PostgreSQL 存储迁移后的 Todo Note 数据。
|
||||
- `code-queue-mgr` 是主 server 承载的轻量 Code Queue 控制面用户服务,容器名 `code-queue-mgr-backend`,只在 Compose 内网暴露 `4278/tcp`,只连接主 PostgreSQL,负责 `code-queue` 稳定代理路径下的队列管理、任务提交、历史摘要、已读状态和轻量 Trace 读取;不得引入 Codex/OpenCode/Playwright/Chromium/Docker socket/runner 依赖。
|
||||
@@ -17,7 +18,7 @@
|
||||
|
||||
## Public Exposure Boundary
|
||||
|
||||
Docker Compose 对最终用户只公开 frontend host port 和 provider ingress host port。backend-core REST API 仍不得映射公网;PostgreSQL 和 OA Event Flow 只允许为了受控 Code Queue active/standby 节点使用受限端口映射,并必须由 `network.restrictedHostAccess.allowedSourceCidrs` 生成的 `DOCKER-USER` 规则限制到 D601 出口地址,不能作为浏览器或任意公网客户端入口。浏览器访问 core API 必须通过 frontend 的同源代理完成。
|
||||
Docker Compose 对最终用户只公开 production frontend host port、dev frontend proxy host port 和 provider ingress host port。backend-core REST API 仍不得映射公网;PostgreSQL 和 OA Event Flow 只允许为了受控 Code Queue active/standby 节点使用受限端口映射,并必须由 `network.restrictedHostAccess.allowedSourceCidrs` 生成的 `DOCKER-USER` 规则限制到 D601 出口地址,不能作为浏览器或任意公网客户端入口。浏览器访问 core API 必须通过 frontend 的同源代理完成;dev UI 访问路径见 `docs/reference/dev-environment.md`。
|
||||
|
||||
计算节点上的用户服务后端也遵守同一边界:业务容器端口只绑定节点本机地址,并由 provider-gateway 主动连出 WebSocket 后接受 backend-core 的 `microservice.http` 调度访问。主 server 不为用户服务新增面向浏览器的公网反向代理端口;最终用户只通过 UniDesk frontend 的 React 页面访问结构化业务控件。
|
||||
|
||||
@@ -27,7 +28,7 @@ CLI 会优先使用 `docker compose` v2 plugin;当 v2 plugin 不存在时才
|
||||
|
||||
Compose v2 安装后仍然必须遵守 UniDesk 的服务控制入口:全栈生命周期用 `server start` / `server stop`,单服务重建用 `server rebuild <service>`。不要因为 v2 可用就直接在生产栈上手工执行未纳入 CLI 的 `up --build`、`down -v` 或跨项目清理命令;所有会影响容器的动作都应保持 job 可观测、Compose project 固定、database named volume 保留。主 server Compose 命令必须从 `providerGateway.upgrade.hostProjectRoot` 指定的 canonical UniDesk 根目录运行,临时 worktree、Code Queue 导出目录或实验分支不得复用生产 `-p unidesk` 和固定 `container_name` 去替换生产容器。
|
||||
|
||||
版本化用户服务部署优先使用 `bun scripts/cli.ts deploy apply` 已支持的受控路径;当前 D601 维护通道直连 apply 不承担服务部署,dev desired-state smoke 使用 `ci run-dev-e2e`。`deploy.json` 只声明服务 `id`、`repo` 和 `commitId`;目标节点、Dockerfile、Compose、Kubernetes manifest、健康检查和代理路径继续来自 `config.json` 与现有 manifest。主 server 直管微服务和内部 sidecar,例如 `code-queue-mgr`,也必须支持这一路径:`deploy apply --service code-queue-mgr` 从 `deploy.json` 指定 commit 导出源码、构建镜像、替换固定 Compose service 并验证运行中镜像/健康信息的 commit。部署必须遵循 target-side build:服务部署到哪台 target,就在哪台 target 从 remote commit 导出源码、一次性代理构建镜像并部署;不得把中心构建镜像作为默认分发路径,也不得用 `docker commit` 或脏 worktree 作为部署输入。完整规则见 `docs/reference/deploy.md`。
|
||||
版本化用户服务部署优先使用 `bun scripts/cli.ts deploy apply` 已支持的受控路径;D601 persistent dev apply 当前只支持 `backend-core` 和 `frontend`,dev desired-state smoke 使用 `ci run-dev-e2e`。`deploy.json` 只声明服务 `id`、`repo` 和 `commitId`;目标节点、Dockerfile、Compose、Kubernetes manifest、健康检查和代理路径继续来自 `config.json` 与现有 manifest。主 server 直管微服务和内部 sidecar,例如 `code-queue-mgr`,也必须支持这一路径:`deploy apply --service code-queue-mgr` 从 `deploy.json` 指定 commit 导出源码、构建镜像、替换固定 Compose service 并验证运行中镜像/健康信息的 commit。部署必须遵循 target-side build:服务部署到哪台 target,就在哪台 target 从 remote commit 导出源码、一次性代理构建镜像并部署;不得把中心构建镜像作为默认分发路径,也不得用 `docker commit` 或脏 worktree 作为部署输入。完整规则见 `docs/reference/deploy.md`,D601 dev/Rust 边界见 `docs/reference/dev-environment.md`。
|
||||
|
||||
## Main Server Swap
|
||||
|
||||
@@ -43,7 +44,7 @@ swap 管理不能被强塞进所有热路径。`server start/status` 可以暴
|
||||
|
||||
## Single Service Rebuild
|
||||
|
||||
前端、backend-core、本机 provider-gateway 或主 server 承载的 Todo Note/Code Queue Manager/Project Manager/Baidu Netdisk/OA Event Flow 用户服务需要非版本化本地重建时,统一使用 `bun scripts/cli.ts server rebuild <service>`,其中 `<service>` 只能是 `backend-core`、`frontend`、`provider-gateway`、`todo-note`、`code-queue-mgr`、`project-manager`、`baidu-netdisk` 或 `oa-event-flow`。需要按 commit 上线或恢复到 desired-state 时必须改用 `bun scripts/cli.ts deploy apply --service <id>`;直管微服务也不能把脏工作树或手工重建作为部署真相。D601 Code Queue 执行面、File Browser、FindJob、Pipeline、MET Nonlinear 和 ClaudeQQ 部署在计算节点,不属于主 server Compose 可重建服务;其中 D601 Code Queue 执行面不得再通过 `codex deploy` 或维护通道直连 D601 部署;未来正式 CD 必须经受控 target-side 路径执行 build-first、rollout 和 live commit 验证。
|
||||
前端、backend-core、本机 provider-gateway、dev-frontend-proxy 或主 server 承载的 Todo Note/Code Queue Manager/Project Manager/Baidu Netdisk/OA Event Flow 用户服务需要非版本化本地重建时,统一使用 `bun scripts/cli.ts server rebuild <service>`,其中 `<service>` 只能是 `backend-core`、`frontend`、`dev-frontend-proxy`、`provider-gateway`、`todo-note`、`code-queue-mgr`、`project-manager`、`baidu-netdisk` 或 `oa-event-flow`。需要按 commit 上线或恢复到 desired-state 时必须改用 `bun scripts/cli.ts deploy apply --service <id>`;直管微服务也不能把脏工作树或手工重建作为部署真相。Rust backend-core 迭代不得在 master server 用 `server rebuild backend-core` 编译,必须走 D601 dev deploy/CI。D601 Code Queue 执行面、File Browser、FindJob、Pipeline、MET Nonlinear 和 ClaudeQQ 部署在计算节点,不属于主 server Compose 可重建服务;其中 D601 Code Queue 执行面不得再通过 `codex deploy` 或维护通道直连 D601 部署;未来正式 CD 必须经受控 target-side 路径执行 build-first、rollout 和 live commit 验证。
|
||||
|
||||
frontend 改动必须明确上线到公网:修改 `src/components/frontend/src/`、`src/components/frontend/public/style.css`、frontend 使用的共享 TSX/TS 模块或 WebUI 导航后,必须在同一变更集中执行 `bun scripts/cli.ts server rebuild frontend`,并等待 job 成功。公网 WebUI 的 `/app.js` 是 `unidesk-frontend` 容器启动时从镜像内源码转译生成的运行时 bundle;只改工作区文件、只跑 `bun run check`、只跑 `Bun.build` 或只刷新浏览器都不会替换已经运行的容器。
|
||||
|
||||
@@ -61,7 +62,7 @@ frontend 的 Docker 上线顺序为:先运行必要的本地校验,例如 `b
|
||||
|
||||
## Health Criteria
|
||||
|
||||
服务跑通的最低标准是:backend-core 内网 `/health` 返回 ok,frontend 公网 `/health` 返回 ok,provider ingress 公网 `/health` 返回 ok,database 在容器内 `pg_isready` 可用,Todo Note 后端 `/api/health` 返回 `storage=postgres`,`code-queue-mgr` `/health` 返回 `role=master-control-plane`、`schemaReady=true` 和资源预算字段,`bun scripts/cli.ts microservice health code-queue` 与 `/api/tasks/overview` 默认经主 server `code-queue-mgr` 返回 PostgreSQL 队列总览,`k3sctl-adapter` `/api/control-plane` 可见 `unidesk-k3s` Kubernetes API service proxy 状态、D601 scheduler serving healthy、D601 read/write Service healthy、`presentNodeIds` 包含 `D601`、`missingNodeIds=[]` 和 no-fallback 路径,D601 Code Queue scheduler `/health` 返回轻量 readiness、默认模型、`queue.storage` 和 `egressProxy.connected=true`,提交类入口经 `code-queue-mgr` 入库后由 D601 `code-queue-scheduler` 轮询并执行,Project Manager `/health` 返回 `storage.primary=postgres` 和项目数量,backend-core `/api/performance` 返回性能指标,`/api/nodes` 中出现 `main-server`、`D601` 和 `D518` provider 且状态为 `online`,`/api/nodes/system-status` 中出现对应 CPU/内存/硬盘采样,`/api/nodes/docker-status` 中能看到主 server、D601 与 D518 Docker 快照。D601/D518 上不得存在 active `rancher/k3s` 容器;D518 只有在原生 k3s-agent 与稳定 Kubernetes 网络完成验证后才可加入 Code Queue expected nodes。交付前还必须运行 `bun scripts/cli.ts e2e run`,并以 `docs/reference/e2e.md` 的门禁作为最终判定。
|
||||
服务跑通的最低标准是:backend-core 内网 `/health` 返回 ok,frontend 公网 `/health` 返回 ok,dev frontend proxy 公网 `/health` 在 D601 dev frontend 已部署后返回 ok,provider ingress 公网 `/health` 返回 ok,database 在容器内 `pg_isready` 可用,Todo Note 后端 `/api/health` 返回 `storage=postgres`,`code-queue-mgr` `/health` 返回 `role=master-control-plane`、`schemaReady=true` 和资源预算字段,`bun scripts/cli.ts microservice health code-queue` 与 `/api/tasks/overview` 默认经主 server `code-queue-mgr` 返回 PostgreSQL 队列总览,`k3sctl-adapter` `/api/control-plane` 可见 `unidesk-k3s` Kubernetes API service proxy 状态、D601 scheduler serving healthy、D601 read/write Service healthy、`presentNodeIds` 包含 `D601`、`missingNodeIds=[]` 和 no-fallback 路径,D601 Code Queue scheduler `/health` 返回轻量 readiness、默认模型、`queue.storage` 和 `egressProxy.connected=true`,提交类入口经 `code-queue-mgr` 入库后由 D601 `code-queue-scheduler` 轮询并执行,Project Manager `/health` 返回 `storage.primary=postgres` 和项目数量,backend-core `/api/performance` 返回性能指标,`/api/nodes` 中出现 `main-server`、`D601` 和 `D518` provider 且状态为 `online`,`/api/nodes/system-status` 中出现对应 CPU/内存/硬盘采样,`/api/nodes/docker-status` 中能看到主 server、D601 与 D518 Docker 快照。D601/D518 上不得存在 active `rancher/k3s` 容器;D518 只有在原生 k3s-agent 与稳定 Kubernetes 网络完成验证后才可加入 Code Queue expected nodes。交付前还必须运行 `bun scripts/cli.ts e2e run`,并以 `docs/reference/e2e.md` 的门禁作为最终判定。
|
||||
|
||||
## Code Queue Control/Execution Resource Budget
|
||||
|
||||
|
||||
@@ -44,13 +44,24 @@ Do not add a long-lived DevOps service, run broker, webhook listener or second d
|
||||
"scriptPath": "scripts/ci/dev-e2e.sh",
|
||||
"timeoutMs": 1800000
|
||||
},
|
||||
"services": []
|
||||
"services": [
|
||||
{
|
||||
"id": "backend-core",
|
||||
"repo": "https://github.com/pikasTech/unidesk",
|
||||
"commitId": "<pushed-commit>"
|
||||
},
|
||||
{
|
||||
"id": "frontend",
|
||||
"repo": "https://github.com/pikasTech/unidesk",
|
||||
"commitId": "<pushed-commit>"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`scriptPath` must be a repo-relative `scripts/ci/*.sh` path. Inline shell bodies, arbitrary script paths, local dirty scripts and separate `develop.json` or CI manifest files are forbidden. The script is fetched from the same full 40-character manifest commit that supplied `deploy.json`, so the runner logic is auditable and rollbackable with the desired state.
|
||||
`scriptPath` must be a repo-relative `scripts/ci/*.sh` path. Inline shell bodies, arbitrary script paths, local dirty scripts and separate `develop.json` or CI manifest files are forbidden. The script is fetched from the same full 40-character manifest commit that supplied `deploy.json`, so the runner logic is auditable and rollbackable with the desired state. Persistent dev rollout service scope is owned by `docs/reference/dev-environment.md`; this runner only consumes the dev service list for smoke verification and must not deploy it.
|
||||
|
||||
## Execution Path
|
||||
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
# D601 Dev Environment
|
||||
|
||||
This document is the authoritative source for the persistent `unidesk-dev` environment, the public dev frontend port, `deploy apply --env dev`, and the Rust backend-core build boundary. If the same dev-environment rule would need edits in `AGENTS.md`, `docs/reference/cli.md`, `docs/reference/deploy.md`, `docs/reference/ci.md`, `docs/reference/deployment.md`, `docs/reference/e2e.md` or `TEST.md`, keep the detailed rule here and leave only a short cross-reference elsewhere.
|
||||
|
||||
## Goal
|
||||
|
||||
The dev environment lets users experience the next UniDesk version without interrupting production:
|
||||
|
||||
- Production stays on the normal public frontend at `http://74.48.78.17:18081/`.
|
||||
- Dev is exposed through a separate main-server public port at `http://74.48.78.17:18083/`.
|
||||
- Dev backend/frontend workloads run in D601 native k3s namespace `unidesk-dev`, not in the main-server Compose stack.
|
||||
- Dev backend/frontend build and rollout use pushed Git commits from `origin/master:deploy.json#environments.dev`, not dirty local worktrees.
|
||||
- Rust backend-core check/build must run on D601 through CI or dev deploy; the master server must not compile Rust for this iteration path.
|
||||
|
||||
## Public Dev Frontend Port
|
||||
|
||||
The main server owns one extra public entrypoint for dev UI:
|
||||
|
||||
```text
|
||||
browser -> main-server:18083 -> dev-frontend-proxy -> prod backend-core microservice proxy -> k3sctl-adapter -> D601 k3s Service frontend-dev -> frontend-dev -> backend-core-dev
|
||||
```
|
||||
|
||||
`dev-frontend-proxy` is an nginx sidecar in the main-server Compose project. It proxies requests to `backend-core:8080/api/microservices/k3sctl-adapter/proxy/api/services/frontend-dev/proxy...`, so D601 does not expose a new public port and the dev UI still crosses the existing UniDesk/k3sctl-adapter control boundary. The proxy is intentionally thin: it does not build frontend assets, does not talk to D601 directly, and does not contain DevOps logic.
|
||||
|
||||
The dev public port is configured in `config.json` as `network.devFrontend.port=18083`, surfaced by `server status` as `urls.devFrontend`, and managed by `server rebuild dev-frontend-proxy`. The proxy health depends on `frontend-dev`; it can be unhealthy until the D601 dev frontend has been deployed.
|
||||
|
||||
The unrestricted public network entries are therefore production frontend, dev frontend, and provider ingress. backend-core REST, PostgreSQL, user-service backend ports, k3s Services, NodePorts and D601 host ports remain private or explicitly restricted.
|
||||
|
||||
## Desired State
|
||||
|
||||
`deploy.json` remains the only version intent file. Dev entries live under `environments.dev` and are read from `origin/master:deploy.json`, never from a dirty local file, when using `--env dev`.
|
||||
|
||||
The persistent dev rollout currently supports only:
|
||||
|
||||
- `backend-core`
|
||||
- `frontend`
|
||||
|
||||
`code-queue`, Decision Center, k3sctl-adapter and other D601 services are not part of persistent dev apply yet. Their smoke validation stays under `ci run-dev-e2e` or service-specific future designs. The `environments.dev.ci` declaration and short launcher runner are owned by `docs/reference/dev-ci-runner.md`.
|
||||
|
||||
## Rust Backend-Core Boundary
|
||||
|
||||
backend-core is implemented as a Rust service for the dev path. The master server may inspect files, run TypeScript CLI checks, render Compose config, dispatch jobs and proxy traffic, but it must not run Rust compilation for backend-core iteration.
|
||||
|
||||
Allowed on the master server:
|
||||
|
||||
- `bun scripts/cli.ts check --files --scripts-typecheck --compose --logs`
|
||||
- `bun scripts/cli.ts check --help`
|
||||
- `bun scripts/cli.ts deploy plan --env dev --service backend-core`
|
||||
- `bun scripts/cli.ts deploy apply --env dev --service backend-core`
|
||||
- `bun scripts/cli.ts ci run --revision <commit>`
|
||||
- `bun scripts/cli.ts ci run-dev-e2e --wait-ms <ms>`
|
||||
|
||||
Not allowed on the master server for this path:
|
||||
|
||||
- `cargo check`, `cargo build`, `cargo test` or `rustfmt` against backend-core.
|
||||
- `bun scripts/cli.ts check --rust` without the D601 CI guard.
|
||||
- `bun scripts/cli.ts server rebuild backend-core` as a way to iterate Rust backend-core, because it would build the Rust image in the main-server Docker daemon.
|
||||
- Ad-hoc `docker build` of `src/components/backend-core/Dockerfile` on the master server.
|
||||
|
||||
Rust checking is enabled only when the process is already running inside the D601 CI/dev execution boundary: `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust`. `check --rust` deliberately fails outside that guard with an explicit explanation instead of silently compiling on the wrong host.
|
||||
|
||||
## Dev Deploy Path
|
||||
|
||||
`deploy apply --env dev --service backend-core|frontend` is the controlled persistent dev rollout path. The controller runs on the master server, but the heavy work runs on D601:
|
||||
|
||||
1. Fetch `origin/master:deploy.json#environments.dev` and resolve the requested service commit to a full SHA.
|
||||
2. Dispatch to D601 through the existing provider-gateway/Host SSH maintenance capability.
|
||||
3. On D601, fetch/export the requested Git commit through the node-local provider-gateway egress proxy `http://127.0.0.1:18789`.
|
||||
4. Use the target-side commit as the source for Dockerfile, build context and dev k3s manifest.
|
||||
5. Build the service image on D601 Docker, importing any required base images through the same egress boundary.
|
||||
6. Import the image into native k3s containerd at `/run/k3s/containerd/containerd.sock`.
|
||||
7. Apply only the selected `unidesk-dev` Service/Deployment objects from the dev manifest.
|
||||
8. Stamp the Deployment with `UNIDESK_DEPLOY_*` env and `unidesk.ai/deploy-*` annotations.
|
||||
9. Verify health through the Kubernetes API service proxy and require the live commit to match the requested commit.
|
||||
|
||||
The dev path is not a fallback system. If GitHub fetch, provider-gateway egress, Docker build, native k3s, containerd import, kubectl apply or live health verification fails, the job fails with logs. It must not fall back to building on the master server, using a dirty worktree, direct D601 public ports, NodePort, or another deployment command.
|
||||
|
||||
## Standard Workflow
|
||||
|
||||
Use this sequence for backend-core Rust and frontend dev work:
|
||||
|
||||
1. Develop in the current `master` worktree and keep unrelated parallel changes separated with `git status`/`git diff`.
|
||||
2. Run local non-Rust checks on the master server, for example `bun scripts/cli.ts check --files --scripts-typecheck --compose --logs`.
|
||||
3. Commit and push the code to `origin master`; `deploy apply --env dev` cannot deploy unpushed local changes.
|
||||
4. Update `deploy.json` `environments.dev.services` so `backend-core` and `frontend` point at the pushed commit, then commit and push that manifest update.
|
||||
5. Run `bun scripts/cli.ts deploy apply --env dev --service backend-core` and observe the returned job with `bun scripts/cli.ts job status <jobId> --tail-bytes 30000`.
|
||||
6. Run `bun scripts/cli.ts deploy apply --env dev --service frontend` and observe the job the same way.
|
||||
7. Rebuild or verify `dev-frontend-proxy` on the main server with `bun scripts/cli.ts server rebuild dev-frontend-proxy` when the proxy config or port changes.
|
||||
8. Manually test `http://74.48.78.17:18083/` and the dev health endpoints.
|
||||
9. Run D601 CI for the commit and the dev smoke runner: `bun scripts/cli.ts ci run --revision <commit> --wait-ms <ms>` and `bun scripts/cli.ts ci run-dev-e2e --wait-ms <ms>`.
|
||||
|
||||
## Validation Commands
|
||||
|
||||
Useful read-only or bounded validation commands:
|
||||
|
||||
```bash
|
||||
bun scripts/cli.ts server status
|
||||
bun scripts/cli.ts deploy plan --env dev
|
||||
bun scripts/cli.ts deploy plan --env dev --service backend-core
|
||||
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml
|
||||
bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/backend-core-dev/proxy/health --raw --full
|
||||
bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/frontend-dev/proxy/health --raw --full
|
||||
curl -fsS http://127.0.0.1:18083/health
|
||||
```
|
||||
|
||||
When validating on D601 directly, always use the native kubeconfig explicitly: `KUBECONFIG=/etc/rancher/k3s/k3s.yaml`. The default `kubectl` context may point at Docker Desktop and is not valid for UniDesk native k3s verification.
|
||||
@@ -1,6 +1,6 @@
|
||||
# UniDesk E2E Reference
|
||||
|
||||
UniDesk delivery is not complete until the public frontend, public provider ingress, internal core API, PostgreSQL database, local provider-gateway self-connection, and frontend Playwright flow pass one end-to-end check. The canonical automated command is `bun scripts/cli.ts e2e run`.
|
||||
UniDesk delivery is not complete until the public production frontend, public dev frontend proxy when deployed, public provider ingress, internal core API, PostgreSQL database, local provider-gateway self-connection, and frontend Playwright flow pass the relevant end-to-end checks. The canonical automated command is `bun scripts/cli.ts e2e run`.
|
||||
|
||||
## Required Preconditions
|
||||
|
||||
@@ -31,7 +31,7 @@ Typical targeted commands:
|
||||
- `bun scripts/cli.ts e2e run --only frontend --skip frontend:todo-note-integrated-visible,frontend:findjob-integrated-visible`
|
||||
- `bun scripts/cli.ts e2e run --only network,provider-ingress`
|
||||
|
||||
- Public exposure: Docker port summary must not show core REST, Code Queue NodePort, or Code Queue public host mappings; frontend and provider ingress are the only browser/provider public entries. PostgreSQL `15432` and OA Event Flow `4255` may be host-mapped only for controlled Code Queue nodes and must be protected by the `DOCKER-USER` source restrictions generated from `network.restrictedHostAccess`; E2E treats either an unreachable generic probe or a verified restricted rule as passing. Known private user-service ports such as FindJob `3254`, MET Nonlinear `3288`, Todo Note `4211`, legacy Code Queue host ports and File Browser provider port `4251` probes must fail.
|
||||
- Public exposure: Docker port summary must not show core REST, Code Queue NodePort, or Code Queue public host mappings; the only unrestricted public entries are production frontend, dev frontend proxy and provider ingress. PostgreSQL `15432` and OA Event Flow `4255` may be host-mapped only for controlled Code Queue nodes and must be protected by the `DOCKER-USER` source restrictions generated from `network.restrictedHostAccess`; E2E treats either an unreachable generic probe or a verified restricted rule as passing. Known private user-service ports such as FindJob `3254`, MET Nonlinear `3288`, Todo Note `4211`, legacy Code Queue host ports and File Browser provider port `4251` probes must fail. The dev frontend proxy rule is owned by `docs/reference/dev-environment.md`.
|
||||
- Core API: `docker exec unidesk-backend-core` calls internal `GET /api/overview`, which must report `dbReady: true`, `pgdata.volumeName=unidesk_pgdata_10gb`, a positive PostgreSQL database byte count, and at least one online node; internal `GET /api/performance` must report component request statistics, internal operation statistics, PGDATA usage and Code Queue PostgreSQL storage metadata.
|
||||
- Provider self-connection: internal `GET /api/nodes` must contain `main-server` with `status: online`, `labels.providerGatewayVersion` equal to `src/components/provider-gateway/package.json`, `labels.providerGatewayUpgradePolicy: "always-enabled"`, `labels.providerGatewayRestartPolicyOk: true`, `labels.providerGatewayPidModeOk: true`, and `labels.providerGatewayRuntimeGuardOk: true`; internal `GET /api/nodes/system-status` must contain CPU/memory/disk samples plus a non-empty process resource list sorted by `memoryBytes` by default, where `memoryBytes` should use PSS when `/proc/[pid]/smaps_rollup` is available, otherwise `rssBytes - statm.shared` before raw RSS, and must retain `rssBytes` for diagnostics; internal `GET /api/nodes/docker-status` must contain a Docker snapshot for `main-server`; every running `provider-gateway` container visible in Docker snapshots must report `restartPolicy: "always"` and `pidMode: "host"`; public provider ingress `/health` must return ok.
|
||||
- Provider remote control: internal `/api/dispatch` must successfully complete a real `provider.upgrade` task in `mode: "plan"` so the upgrade path is validated without recreating the running gateway during E2E.
|
||||
@@ -59,7 +59,7 @@ User service pages are covered by the same rule. `Todo Note` must show lists, ta
|
||||
|
||||
## Public Boundary Rule
|
||||
|
||||
The public frontend URL and provider ingress URL are the only unrestricted public network interfaces. backend-core REST API remains Docker-internal only; PostgreSQL and OA Event Flow may expose restricted host mappings solely for controlled Code Queue nodes, and E2E must prove those mappings are unreachable to generic clients or protected by explicit source rules.
|
||||
The production frontend URL, dev frontend proxy URL and provider ingress URL are the only unrestricted public network interfaces. backend-core REST API remains Docker-internal only; PostgreSQL and OA Event Flow may expose restricted host mappings solely for controlled Code Queue nodes, and E2E must prove those mappings are unreachable to generic clients or protected by explicit source rules.
|
||||
|
||||
## Database Persistence Rule
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ UniDesk 用户服务是挂载到 UniDesk 核心服务上的、面向用户使用
|
||||
|
||||
- 用户服务后端端口默认只绑定计算节点本机地址,例如 `127.0.0.1:<port>`,不得直接暴露公网。
|
||||
- 浏览器只访问 UniDesk frontend;frontend 通过同源 `/api/microservices/*` 代理到 backend-core。`deployment.mode=unidesk-direct` 的用户服务由 backend-core 通过目标 provider-gateway 的 `microservice.http` 能力访问计算节点本机后端;`deployment.mode=internal-sidecar` 的主 server 内置控制面服务由 backend-core 直接访问同一 Compose 网络内的显式服务名;`deployment.mode=k3sctl-managed` 的用户服务只允许经 `k3sctl-adapter` 微服务进入 k3s 标准服务路由,backend-core 不得直接向业务容器所在 provider 下发 `microservice.http`。
|
||||
- backend-core REST API、database 和计算节点用户服务后端都不得新增公网端口;公网入口仍只有 frontend 和 provider ingress。
|
||||
- backend-core REST API、database 和计算节点用户服务后端都不得新增公网端口;公网入口仅限 production frontend、dev frontend proxy 和 provider ingress。dev frontend proxy 的唯一权威规则见 `docs/reference/dev-environment.md`。
|
||||
- `microservice.http` 只允许 provider-gateway 访问 `http://127.0.0.1`、`http://localhost`、`http://host.docker.internal` 这类节点本地地址,或明确登记为同一私有 Docker network 内的服务名;主 server 内置用户服务可使用同一 Compose 网络内的显式服务名,例如 `todo-note:4211`。k3s 代管服务不得把业务容器地址登记成 provider-gateway 直连目标,`backend.proxyMode` 必须使用 `k3sctl-adapter-http`,`backend.nodeBaseUrl` 可使用 `k3s://<service>` 这类逻辑服务名。backend-core 还必须用 `allowedPathPrefixes` 和 `allowedMethods` 同时限制可代理路径和 HTTP 方法。
|
||||
|
||||
## Config Contract
|
||||
@@ -155,7 +155,7 @@ Baidu Netdisk 在 UniDesk 语境中按纯后端服务管理:不得暴露百度
|
||||
|
||||
D601 开发环境底座只允许创建 `unidesk-dev` namespace 与 dev 专用对象,manifest 固定为 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-foundation.k8s.yaml`。该 manifest 包含 `postgres-dev` 独立 PostgreSQL StatefulSet/Service/PVC、dev-only secret/config 模板、dev DB 初始化 SQL 和迁移 Job、ResourceQuota/LimitRange,以及 `unidesk-dev-db-guard`。它不得修改生产 `unidesk` namespace、生产 PostgreSQL、生产 PVC、生产 Deployment/Service/Secret 或主 server Docker Compose。
|
||||
|
||||
`postgres-dev` 是 dev backend-core 与 dev Code Queue 状态的默认唯一数据库。dev 运行时必须使用 `postgres-dev.unidesk-dev.svc.cluster.local:5432/unidesk_dev` 和 dev Provider 身份 `D601-dev`;不得共享生产 `d601-tcp-egress-gateway.../unidesk`。当前 Phase 2 只提供 manifest 脚本和 `dev-env validate` 的静态护栏;backend-core、Code Queue 和 Code Queue Manager 的运行时启动护栏需在后续阶段单独评审后接入。
|
||||
`postgres-dev` 是 dev backend-core 与未来 dev Code Queue 状态的默认唯一数据库。dev 运行时必须使用 `postgres-dev.unidesk-dev.svc.cluster.local:5432/unidesk_dev` 和 dev Provider 身份 `D601-dev`;不得共享生产 `d601-tcp-egress-gateway.../unidesk`。Persistent dev backend-core/frontend rollout、public dev frontend port and Rust build boundary are defined in `docs/reference/dev-environment.md`; Code Queue dev rollout remains future work.
|
||||
|
||||
验收入口:先运行 `bun scripts/cli.ts dev-env validate` 做静态资源与 DB URL 护栏检查;具备 D601 kubeconfig 时运行 `bun scripts/cli.ts dev-env validate --kubectl-dry-run` 做 Kubernetes client dry-run。首次或镜像缓存不确定时,先运行 `bun scripts/cli.ts dev-env prewarm-images`,把 `postgres:16-alpine` 和 local-path helper 所需的 `rancher/mirrored-library-busybox:1.36.1` 导入 D601 原生 k3s containerd;否则 D601 的 Docker 代理/缓存正常也不能保证 k3s/containerd 能实时拉到外部镜像。若实际 apply,只能 apply 到 `unidesk-dev`,随后用 `kubectl -n unidesk-dev get pods,svc,pvc` 验证 dev DB ready,并对比 apply 前后的 `kubectl -n unidesk get deploy,sts,svc,secret,pvc -o name` 证明生产 workload 未变化。
|
||||
|
||||
@@ -163,7 +163,7 @@ D601 上必须显式使用原生 k3s kubeconfig:`KUBECONFIG=/etc/rancher/k3s/k
|
||||
|
||||
### D601 Dev Core Services
|
||||
|
||||
`backend-core-dev` 与 `frontend-dev` 的第一版 manifest 固定为 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`。该 manifest 只允许创建 `unidesk-dev` 内的 `backend-core-dev`、`frontend-dev` Deployment/Service;不得修改生产主 server Compose、生产 `unidesk` namespace 或生产 backend/frontend。
|
||||
`backend-core-dev` 与 `frontend-dev` 的第一版 manifest 固定为 `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml`。该 manifest 只允许创建 `unidesk-dev` 内的 `backend-core-dev`、`frontend-dev` Deployment/Service;不得修改生产主 server Compose、生产 `unidesk` namespace 或生产 backend/frontend。Rollout uses `deploy apply --env dev --service backend-core|frontend` under the rules in `docs/reference/dev-environment.md`.
|
||||
|
||||
`backend-core-dev` 必须从 `unidesk-dev-runtime-config` 和 `unidesk-dev-runtime-secrets` 注入 dev-only 配置,使用 `postgres-dev.../unidesk_dev`、dev Provider token、dev log path 和 `UNIDESK_DEPLOY_REF=origin/master:deploy.json#environments.dev`。`frontend-dev` 必须把 `CORE_INTERNAL_URL` 指向 `backend-core-dev.unidesk-dev.svc.cluster.local:8080`,页面在 dev identity 下显示 DEV 标记,`/health` 返回 dev namespace、database、service id、deploy ref 和 commit metadata。生产环境未设置 dev identity 时,backend-core 和 frontend health payload 保持生产兼容形状。
|
||||
|
||||
|
||||
@@ -8,13 +8,13 @@ UniDesk 的可观测性优先级高于静默成功。CLI、服务日志、Docker
|
||||
|
||||
## Service Logs
|
||||
|
||||
服务日志位于 `logs/{YYYYMMDD}/`,每次 `server start` 都生成新的本地时间戳前缀。新写入的 UniDesk JSONL 日志必须按小时切片:`logs/{YYYYMMDD}/{startStamp}_{YYYYMMDD}_{HH}_{service}.jsonl`,一天一个目录,禁止长期追加到单个巨大 JSONL。所有 UniDesk Bun 服务(backend-core、frontend、provider-gateway、Code Queue、project-manager、baidu-netdisk 以及后续新增服务)必须复用 `src/components/shared/src/rotating-jsonl.ts` 中的 `createHourlyJsonlWriter`;`LOG_FILE` 只作为推导 `logs` 根目录、启动前缀和 service 后缀的 base path,不得直接 `appendFileSync(LOG_FILE, ...)` 长期追加。database 通过 PostgreSQL logging collector 写入同一日期目录。
|
||||
服务日志位于 `logs/{YYYYMMDD}/`,每次 `server start` 都生成新的本地时间戳前缀。新写入的 UniDesk JSONL 日志必须按小时切片:`logs/{YYYYMMDD}/{startStamp}_{YYYYMMDD}_{HH}_{service}.jsonl`,一天一个目录,禁止长期追加到单个巨大 JSONL。所有 UniDesk Bun 服务(frontend、provider-gateway、Code Queue、project-manager、baidu-netdisk 以及后续新增 Bun 服务)必须复用 `src/components/shared/src/rotating-jsonl.ts` 中的 `createHourlyJsonlWriter`;Rust backend-core 必须提供等价的 hourly rotation and retention behavior in `src/components/backend-core/src/logger.rs`。`LOG_FILE` 只作为推导 `logs` 根目录、启动前缀和 service 后缀的 base path,不得长期追加到单个文件。database 通过 PostgreSQL logging collector 写入同一日期目录。
|
||||
|
||||
日志保留默认按日志族限制为 `1GiB`:服务写入或 Code Queue 导出日志时必须扫描同一 service 后缀的历史文件,超过上限后自动删除最旧切片;当前活跃切片不能被保留清理删除。全局上限由 `UNIDESK_LOG_RETENTION_BYTES` 控制,服务级上限使用 `UNIDESK_<SERVICE>_LOG_MAX_BYTES`(如 `UNIDESK_FRONTEND_LOG_MAX_BYTES`、`UNIDESK_PROVIDER_GATEWAY_LOG_MAX_BYTES`),历史兼容变量只允许作为过渡入口。Codex app-server 的 `logs_*.sqlite` 仅作为 Codex 上游运行时的短暂缓冲,Code Queue 必须周期性导出为同样按小时切片的 `codex-app-server` JSONL,并删除/压缩已导出的 SQLite 行,避免 `logs_2.sqlite` 成为长期大文件。
|
||||
|
||||
新增或迁移服务的长期规范:Dockerfile 必须把 `src/components/shared` 复制到与仓库相同的相对路径,TypeScript 配置必须能解析 shared 引用,Compose 必须传入 `LOG_FILE` 和 `UNIDESK_LOG_RETENTION_BYTES`;如果服务需要在内存中暴露 `/logs`,可以继续维护有限 `recentLogs`,但落盘只能通过统一 hourly writer。业务归档日志(例如 Code Queue task output archive)可以保留 append-only 文件,但不得复用 UniDesk service JSONL 命名族,也不得替代 `/logs` 的结构化服务日志。
|
||||
|
||||
`bun scripts/cli.ts check` 必须包含日志轮转门禁:核心 Bun 服务源码不得直接向 `LOG_FILE` append,且必须引用统一 hourly writer。新增服务如果进入主 Compose,也要纳入该门禁的 checked file 列表。
|
||||
`bun scripts/cli.ts check` 必须包含日志轮转门禁:核心 Bun 服务源码不得直接向 `LOG_FILE` append,且必须引用统一 hourly writer;Rust backend-core logger must expose equivalent rotation/pruning markers for the same check. 新增服务如果进入主 Compose,也要纳入该门禁的 checked file 列表。
|
||||
|
||||
## Log Access
|
||||
|
||||
|
||||
+24
-19
@@ -2,7 +2,7 @@
|
||||
- AGENTS.md (Top-level agent index and `scripts/cli.ts` usage guide)
|
||||
- TEST.md (Manual CLI test plan following cli-spec expectations)
|
||||
- config.json (Single source of truth for ports, tokens, runtime, paths, and provider identity)
|
||||
- docker-compose.yml (Main server orchestration for database, backend-core, frontend, provider-gateway, and managed main-server user services such as Todo Note)
|
||||
- docker-compose.yml (Main server orchestration for database, backend-core, frontend, dev-frontend-proxy, provider-gateway, and managed main-server user services such as Todo Note)
|
||||
- package.json / bun.lock (Root Bun tooling for CLI checks)
|
||||
- .gitignore
|
||||
- reference -> docs/reference (Compatibility symlink for older references)
|
||||
@@ -17,7 +17,7 @@
|
||||
- debug.ts (Real-flow debug helpers)
|
||||
- command.ts (Bounded command execution helpers)
|
||||
- output.ts (JSON output helpers)
|
||||
- e2e.ts (Public frontend/provider ingress, internal core/database, and Playwright frontend E2E checks)
|
||||
- e2e.ts (Public production frontend/dev frontend/provider ingress, internal core/database, and Playwright frontend E2E checks)
|
||||
- ci.ts (D601 k3s Tekton CI install/status/manual-run/dev-namespace-e2e/logs helpers; CI only, no CD)
|
||||
- logs/ (Generated service logs; ignored by git)
|
||||
- .state/ (Generated job state and compose env; ignored by git)
|
||||
@@ -34,7 +34,8 @@
|
||||
- observability.md (Logs and status visibility)
|
||||
- e2e.md (Delivery gate, Playwright frontend E2E, and database persistence checks)
|
||||
- ci.md (D601 k3s Tekton CI, read-only production database performance gate, master deploy.json dev namespace e2e harness, and trigger boundary)
|
||||
- src/ (TypeScript component monorepo)
|
||||
- dev-environment.md (D601 persistent dev environment, public dev frontend proxy, deploy dev scope, and Rust build boundary)
|
||||
- src/ (Component monorepo)
|
||||
- package.json (Component workspace metadata)
|
||||
- bun.lock (Component dependency lockfile)
|
||||
- tsconfig.base.json (Project references for component checks)
|
||||
@@ -45,24 +46,28 @@
|
||||
- tsconfig.json
|
||||
- src/index.ts
|
||||
- backend-core/ (UniDesk stateless core service container)
|
||||
- Cargo.toml / Cargo.lock (Rust backend-core crate and pinned dependency graph)
|
||||
- package.json
|
||||
- tsconfig.json
|
||||
- Dockerfile
|
||||
- src/index.ts (Server setup, HTTP/WebSocket routing, startup orchestration)
|
||||
- src/types.ts (All interfaces and type aliases)
|
||||
- src/config.ts (Environment variable reading and config validation)
|
||||
- src/context.ts (Shared mutable state container: db client, config ref, dbReady flag)
|
||||
- src/logger.ts (Structured logger factory)
|
||||
- src/http.ts (HTTP/JSON utilities, ISO formatting, nested value extraction)
|
||||
- src/db.ts (Database init, schema creation, all CRUD queries)
|
||||
- src/performance.ts (Request/operation performance sampling and database size reporting)
|
||||
- src/provider-registry.ts (Provider WebSocket lifecycle: register, heartbeat, status, disconnect)
|
||||
- src/task-dispatcher.ts (Task creation, dispatch to providers, wait-for-result)
|
||||
- src/ssh-bridge.ts (SSH session bridging over provider WebSocket)
|
||||
- src/egress-tcp.ts (Egress TCP proxy for provider outbound connections)
|
||||
- src/scheduler.ts (Scheduled task system: cron-like recurring tasks)
|
||||
- src/microservice-proxy.ts (Microservice routing, HTTP proxy, health aggregation)
|
||||
- src/overview.ts (Overview endpoint, node/task/service summary, load test)
|
||||
- src/main.rs (Server setup, HTTP/WebSocket routing, startup orchestration)
|
||||
- src/cli.rs (Container-local helpers for JSON fetch and SSH broker)
|
||||
- src/types.rs (Runtime config, provider, task, microservice and performance data structures)
|
||||
- src/config.rs (Environment variable reading, config validation and dev identity)
|
||||
- src/state.rs (Shared application state, database pool, logger and in-memory task/provider indexes)
|
||||
- src/logger.rs (Structured JSONL logger with hourly rotation and retention pruning)
|
||||
- src/http.rs (HTTP routing, JSON utilities, health payload and API handlers)
|
||||
- src/db.rs (Database init, schema creation and CRUD queries)
|
||||
- src/performance.rs (Request/operation performance sampling and database size reporting)
|
||||
- src/provider_registry.rs (Provider WebSocket lifecycle: register, heartbeat, status, disconnect)
|
||||
- src/task_dispatcher.rs (Task creation, dispatch to providers, wait-for-result)
|
||||
- src/ssh_bridge.rs (SSH session bridging over provider WebSocket)
|
||||
- src/egress_tcp.rs (Egress TCP proxy for provider outbound connections)
|
||||
- src/scheduler.rs (Scheduled task system: cron-like recurring tasks)
|
||||
- src/microservice_proxy.rs (Microservice routing, HTTP proxy, health aggregation)
|
||||
- src/overview.rs (Overview endpoint, node/task/service summary, load test)
|
||||
- dev-frontend-proxy/ (Main-server nginx proxy exposing D601 `frontend-dev` on the dev public port)
|
||||
- Dockerfile
|
||||
- nginx.conf
|
||||
- frontend/ (Frontend web application container)
|
||||
- package.json
|
||||
- tsconfig.json
|
||||
|
||||
Reference in New Issue
Block a user