mirrors/pikasTech-unidesk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
27ed8a261db26917ce978e585dc7274972e2afa4
pikasTech-unidesk/docs/reference/dev-environment.md
T
Codex 27ed8a261d Improve ssh tran file transfer reliability
2026-05-27 04:08:11 +00:00

149 lines
16 KiB
Markdown
Raw Blame History

# D601 Dev Environment
This document is the authoritative source for the persistent `unidesk-dev` environment, the public dev frontend port, `deploy apply --env dev`, and the Rust backend-core build boundary. Release-line and dev-lane governance is owned by `docs/reference/release-governance.md` and [GitHub issue #6](https://github.com/pikasTech/unidesk/issues/6). If the same dev-environment rule would need edits in `AGENTS.md`, `docs/reference/cli.md`, `docs/reference/deploy.md`, `docs/reference/ci.md`, `docs/reference/deployment.md`, `docs/reference/e2e.md` or `TEST.md`, keep the detailed rule here and leave only a short cross-reference elsewhere.
## Goal
The dev environment lets users experience the next UniDesk version without interrupting production:
- Production stays on the normal public frontend at `http://74.48.78.17:18081/`.
- Dev is exposed through a separate main-server public port at `http://74.48.78.17:18083/`.
- Dev backend/frontend workloads run in D601 native k3s namespace `unidesk-dev`, not in the main-server Compose stack.
- Dev backend-core and frontend rollout use pushed Git commits from `origin/master:deploy.json#environments.dev`, not dirty local worktrees.
- Rust backend-core check/build must run on D601 CI through `ci publish-backend-core`; dev CD consumes the published image and must not compile Rust.
## D601 UniDesk Workspace
`D601:UniDesk` 的固定开发 workspace 是 D601 节点上的 `/home/ubuntu/workspace/unidesk-dev`，固定使用 `master` 分支和 `origin git@github.com:pikasTech/unidesk.git`。所有需要在 D601 上改 UniDesk 代码、跑轻量合同测试、验证 `tran`/Code Queue runner、收敛分布式敏捷实验补丁的工作，都应先进入这个目录。
每次开始 D601 UniDesk 分布式开发、切换任务、恢复中断或上下文压缩后，先通过 UniDesk SSH 维护桥执行：
```bash
tran D601:/home/ubuntu/workspace/unidesk-dev git status --short --branch
tran D601:/home/ubuntu/workspace/unidesk-dev git remote -v
```
若路径、分支或 remote 不符合预期，先修正 fixed workspace，再继续。`/home/ubuntu/cq-deploy`、Code Queue pod 内 `/root/unidesk`/`/app`、D601 上的 `/root/unidesk` 和 `/tmp/unidesk-*` 只作为部署副本、运行副本或临时实验面；运行面热修可以直接作用在 pod/容器，但必须随后把持久化修复提交到 Git remote，并在 fixed workspace 中复验。
固定 workspace 只作为 source truth 预检、fetch、worktree 管理和最终同步入口。实际开发、文档修改、测试补丁和 PR 准备应在固定 repo 下的独立 worktree 中完成，例如 `/home/ubuntu/workspace/unidesk-dev/.worktree/<task>`；该 worktree 必须从最新 `origin/master` 创建，使用任务专属分支或按当前 master-only 规则完成提交，结束前用 `git status` 确认只包含本任务文件。不要把 `/home/ubuntu/workspace/unidesk-dev` 根目录当作并行任务 scratch 区，也不要复用其他任务遗留 worktree。
Master server 不作为 UniDesk 重型验证机。仓库级 check、Playwright/browser smoke、镜像构建、Rust/Go 编译和 Code Queue runner 实测必须放到 D601、CI runner 或其他获批执行面；master server 只做轻量源码编辑、Git 操作、状态观察和受控调度。
`scripts/cli.ts`、`scripts/tran`、`scripts/src/ssh.ts` 和相邻的 `tran`/SSH 透传 helper 是主 server 上人工与 Codex 高频使用的控制入口；这类客户端工具链改进可以直接在 master server `/root/unidesk` 轻量修改、提交并推送到 `origin/master`。该例外只覆盖 CLI/tran 客户端源码、帮助、合同测试和对应 reference 文档，不覆盖 `src/components/provider-gateway` 行为变更、镜像构建、仓库级 check、浏览器 smoke 或其他重型验证。涉及 provider-gateway 代码时仍必须遵循 provider-gateway 版本和远程升级规则。
当 `tran`/SSH 透传的文件传输、stdin、chunk、编码、timeout 或 route/operation 解析出现高频摩擦时，先优化 CLI 客户端的分块、校验、重试、可观测输出和帮助文档，并用目标 provider/pod/Windows route 的最小闭环证明；只有证据显示 client 侧无法规避 provider-gateway 边界时，才进入 provider-gateway 变更流程。
## Public Dev Frontend Port
The main server owns one extra public entrypoint for dev UI:
```text
browser -> main-server:18083 -> dev-frontend-proxy -> prod backend-core microservice proxy -> k3sctl-adapter -> D601 k3s Service frontend-dev -> frontend-dev -> backend-core-dev
```
`dev-frontend-proxy` is an nginx sidecar in the main-server Compose project. It proxies requests to `backend-core:8080/api/microservices/k3sctl-adapter/proxy/api/services/frontend-dev/proxy...`, so D601 does not expose a new public port and the dev UI still crosses the existing UniDesk/k3sctl-adapter control boundary. The proxy is intentionally thin: it does not build frontend assets, does not talk to D601 directly, and does not contain DevOps logic.
`frontend-dev` and `backend-core-dev` are registered with `k3sctl-adapter` through the managed-service catalog `src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k3s.json`. The public dev port must use that cataloged Kubernetes API service-proxy path; it must not add a direct D601 public port, NodePort, or backend-core catalog entry for arbitrary k3s services.
The dev public port is configured in `config.json` as `network.devFrontend.port=18083`, surfaced by `server status` as `urls.devFrontend`, and managed by `server rebuild dev-frontend-proxy`. The proxy health depends on `frontend-dev`; it can be unhealthy until the D601 dev frontend has been deployed.
The unrestricted public network entries are therefore production frontend, dev frontend, and provider ingress. backend-core REST, PostgreSQL, user-service backend ports, k3s Services, NodePorts and D601 host ports remain private or explicitly restricted.
Dev and production frontend authentication must use the same `config.json.auth` username, password, session secret and session TTL. The public dev and production entrypoints share the same IP/host with different ports, so the `unidesk_session` cookie is host-scoped rather than port-scoped. `deploy apply --env dev --service frontend` must sync `unidesk-dev-runtime-secrets` and `unidesk-dev-runtime-config` from the main-server config before rolling out `frontend-dev`; dev manifests may contain placeholders but must not establish a separate dev login identity.
## Desired State
`deploy.json` remains the only version intent file. Dev entries live under `environments.dev` and are read from `origin/master:deploy.json`, never from a dirty local file, when using `--env dev` or `ci run-dev-e2e`.
The target release-governance model is to split stable maintenance validation from high-risk master integration, for example with explicit lanes such as `dev-v1` and `dev-master`. That split is not active until `deploy.json`, deploy commands, CI, frontend labels and diagnostics support it explicitly. Until then, the documented `environments.dev` behavior remains the active contract and must not be emulated through local manifest edits or hidden branch conventions.
During a `release/v1` stabilization window, this same implemented `environments.dev` lane is temporarily reserved for v1 validation. Use pushed commits and explicit `deploy.json` pins to validate v1 fixes; do not add a parallel `dev-v1`/`dev-master` schema in the middle of the stabilization window unless that split is the blocker for v1. The detailed phase rule lives in `docs/reference/release-governance.md`.
The persistent dev rollout currently supports:
- `backend-core`, `frontend`, `decision-center`, `mdtodo`, `claudeqq` and `code-queue` as D601 registry artifact consumers in `unidesk-dev`.
- `baidu-netdisk` as the main-server direct user-service artifact validation sample.
`deploy apply --env dev --service baidu-netdisk` consumes the D601 registry artifact and validates the main-server Compose service; it is not a persistent D601 k3s dev workload. `deploy apply --env dev --service code-queue` is intentionally dev-only and may mutate only `unidesk-dev` Code Queue scheduler/read/write/provider-egress-proxy objects. `deploy apply --env prod --service code-queue` must remain unsupported. The `environments.dev.ci` declaration and short launcher runner are owned by `docs/reference/dev-ci-runner.md`.
## Rust Backend-Core Boundary
backend-core is implemented as a Rust service for the dev path. The master server may inspect files, run TypeScript CLI checks, render Compose config, dispatch jobs and proxy traffic, but it must not run Rust compilation for backend-core iteration.
Allowed on the master server:
- `bun scripts/cli.ts check --files --scripts-typecheck --compose --logs`
- `bun scripts/cli.ts check --help`
- `bun scripts/cli.ts deploy plan --env dev --service backend-core`
- `bun scripts/cli.ts deploy apply --env dev --service backend-core`
- `bun scripts/cli.ts ci run --revision <commit>`
- `bun scripts/cli.ts ci run-dev-e2e --wait-ms <ms>`
Not allowed on the master server for this path:
- `cargo check`, `cargo build`, `cargo test` or `rustfmt` against backend-core.
- `bun scripts/cli.ts check --rust` without the D601 CI guard.
- `bun scripts/cli.ts server rebuild backend-core` as a way to iterate Rust backend-core, because it would build the Rust image in the main-server Docker daemon.
- Ad-hoc `docker build` of `src/components/backend-core/Dockerfile` on the master server.
Rust checking is enabled only when the process is already running inside the D601 CI execution boundary: `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust`. `check --rust` deliberately fails outside that guard with an explicit explanation instead of silently compiling on the wrong host.
## Dev Deploy Path
`deploy apply --env dev --service backend-core|frontend|decision-center|mdtodo|claudeqq|code-queue` is the controlled persistent dev rollout path for D601 dev workloads. The controller runs on the master server, but dev CD is a pull-only artifact consumer. All listed services consume CI-published commit-pinned artifacts from the D601 registry.
1. Fetch `origin/master:deploy.json#environments.dev` and resolve the requested service commit to a full SHA.
2. Dispatch to D601 through the existing provider-gateway/Host SSH maintenance capability.
3. Require the existing registry image `127.0.0.1:5000/unidesk/<service-id>:<commit>` produced by `ci publish-backend-core` for backend-core or `ci publish-user-service --service <service-id>` for user services.
4. Verify the registry manifest and image labels for service id, source commit and Dockerfile. The dry-run plan exposes the registry image, planned digest source, target, source commit and `build.willCompile=false`.
5. Import the registry artifact into native k3s containerd at `/run/k3s/containerd/containerd.sock`.
6. Apply only the selected `unidesk-dev` Service/Deployment objects from the dev manifest.
7. For `frontend`, sync auth/session settings from main-server `config.json.auth` into the dev runtime Secret/ConfigMap before rollout.
8. Stamp the Deployment with `UNIDESK_DEPLOY_*` env and `unidesk.ai/deploy-*` annotations.
9. Verify health through the Kubernetes API service proxy and require the live commit to match the requested commit.
The dev path is not a fallback system. If artifact lookup, label verification, native k3s, containerd import, kubectl apply or live health verification fails, the job fails with logs. It must not fall back to `cargo build`, `docker build`, building on the master server, building source on the dev target, using a dirty worktree, direct D601 public ports, NodePort, hostPort, provider-gateway business HTTP, or another deployment command. k3s-managed health verification must use the Kubernetes API service proxy.
## DevOps Hygiene
The persistent dev environment follows the shared Git-backed deployment hygiene rules in `docs/reference/devops-hygiene.md`. In particular, D601 runtime edits, dirty-worktree builds, copied scripts/images/source, direct D601 ports, NodePorts and manual smoke checks without live commit agreement must not become deployment truth.
## Standard Workflow
Use this sequence for backend-core Rust and frontend dev work:
1. Preflight the fixed workspace, then develop in a task-scoped `.worktree/<task>` created from the latest `origin/master`; keep unrelated parallel changes separated with `git status`/`git diff`.
2. Run local non-Rust checks on the master server, for example `bun scripts/cli.ts check --files --scripts-typecheck --compose --logs`.
3. Commit and push the code to `origin master`; `deploy apply --env dev` cannot deploy unpushed local changes.
4. Update `deploy.json` `environments.dev.services` so `backend-core` and `frontend` point at the pushed commit, then commit and push that manifest update.
5. Preflight backend-core publication: `bun scripts/cli.ts ci publish-backend-core --commit <full-sha> --dry-run`. The result must have no `blockedScopes`, `wouldBuildOnD601=true`, D601 `unidesk-ci` Tekton runner metadata, D601 registry target `127.0.0.1:5000/unidesk/backend-core`, required labels for service id/source repo/source commit/Dockerfile, and `recommendedAction` pointing to the real publish command.
6. Publish the artifact first: `bun scripts/cli.ts ci publish-backend-core --commit <full-sha> --wait-ms 1200000` for backend-core, or `bun scripts/cli.ts ci publish-user-service --service <frontend|decision-center|mdtodo|claudeqq|code-queue> --commit <full-sha> --wait-ms 1200000` for user services.
7. Verify the publish output contains non-empty `artifactSummary.digest` and `artifactSummary.digestRef`, and that the pushed image labels match `backend-core`, the source repo, source commit and Dockerfile. This verification can use the publish output and the D601 registry manifest HEAD; it must not rebuild.
8. Run `bun scripts/cli.ts deploy apply --env dev --service backend-core --dry-run` and confirm `artifactConsumer.noRuntimeSourceBuild=true`, `build.willCompile=false`, the registry image is `127.0.0.1:5000/unidesk/backend-core:<commit>`, and the target is `unidesk-dev/backend-core-dev`.
9. Run `bun scripts/cli.ts deploy apply --env dev --service backend-core` and observe the returned job with `bun scripts/cli.ts job status <jobId> --tail-bytes 30000`.
10. Run `bun scripts/cli.ts deploy apply --env dev --service <frontend|decision-center|mdtodo|claudeqq|code-queue>` and observe the job the same way; this must consume the registry artifact and verify live deploy metadata through the service health path.
11. If the dev service catalog changes, deploy the pushed `k3sctl-adapter` commit through the controlled local manifest exception, then verify `/api/control-plane` lists `k3s/dev/unidesk-dev-core.k3s.json`.
12. Rebuild or verify `dev-frontend-proxy` on the main server with `bun scripts/cli.ts server rebuild dev-frontend-proxy` when the proxy config or port changes.
13. Manually test `http://74.48.78.17:18083/` and the dev health endpoints.
14. Run D601 CI for the commit and the dev smoke runner: `bun scripts/cli.ts ci run --revision <commit> --wait-ms <ms>` and `bun scripts/cli.ts ci run-dev-e2e --wait-ms <ms>`. When Code Queue behavior changes, update the `code-queue` entry in `environments.dev.services` to the pushed commit before running dev artifact validation or the temporary dev smoke.
## Validation Commands
Useful read-only or bounded validation commands:
```bash
bun scripts/cli.ts server status
bun scripts/cli.ts deploy plan --env dev
bun scripts/cli.ts deploy plan --env dev --service backend-core
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-mdtodo.k8s.yaml
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-claudeqq.k8s.yaml
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-code-queue.k8s.yaml
bun scripts/cli.ts ci run-dev-e2e --wait-ms 600000
bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/backend-core-dev/proxy/health --raw --full
bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/frontend-dev/proxy/health --raw --full
curl -fsS http://127.0.0.1:18083/health
```
When validating on D601 directly, always use the native kubeconfig explicitly: `KUBECONFIG=/etc/rancher/k3s/k3s.yaml`. The default `kubectl` context may point at Docker Desktop and is not valid for UniDesk native k3s verification.
Reference in New Issue View Git Blame Copy Permalink