From 27ed8a261db26917ce978e585dc7274972e2afa4 Mon Sep 17 00:00:00 2001 From: Codex Date: Wed, 27 May 2026 04:08:11 +0000 Subject: [PATCH] Improve ssh tran file transfer reliability --- AGENTS.md | 4 +- docs/reference/cli.md | 8 +- docs/reference/dev-environment.md | 4 + scripts/src/help.ts | 6 + scripts/src/remote.ts | 22 +- scripts/src/ssh-file-transfer.ts | 409 +++++++++++++++++++++ scripts/src/ssh.ts | 38 +- scripts/ssh-argv-guidance-contract-test.ts | 126 +++++++ 8 files changed, 607 insertions(+), 10 deletions(-) create mode 100644 scripts/src/ssh-file-transfer.ts diff --git a/AGENTS.md b/AGENTS.md index a4a2ac93..73151821 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -66,6 +66,7 @@ UniDesk 是一个以主 server 为统一入口的分布式工作平台；本文 - P0: `D601:UniDesk` 的固定开发 workspace 是 D601 节点上的 `/home/ubuntu/workspace/unidesk-dev`，固定使用 `master` 分支和 `origin git@github.com:pikasTech/unidesk.git`；所有需要在 D601 上改 UniDesk 代码、跑轻量合同测试、做分布式敏捷实验补丁收敛或验证 Code Queue runner/tran 的工作，都必须优先使用这个目录。 - P0: 每次开始 `D601:UniDesk` 分布式开发、切换任务、恢复中断或上下文压缩后，必须重新读取目标 workspace 的 `/home/ubuntu/workspace/unidesk-dev/AGENTS.md`，并以该文件和其引用的 UniDesk repo 内规则为当前任务约束；禁止只凭压缩摘要或主 server 记忆继续改代码。 +- P0: UniDesk CLI/tran/SSH 透传客户端工具链改进可以直接在 master server `/root/unidesk` 做轻量源码修改、提交和推送；这不允许在 master server 运行仓库级 check、browser smoke、镜像构建或编译，细则见 `docs/reference/dev-environment.md`。 - `/home/ubuntu/cq-deploy`、`/root/unidesk`、`/app`、Code Queue pod 内 `/root/unidesk` 和 `/tmp/unidesk-*` 都是运行副本、部署副本或一次性实验面，不是 `D601:UniDesk` 日常开发 source truth；运行面热修可以直接作用于 pod/容器，但必须随后把持久化修复落回 fixed workspace 和 Git remote。 - 每次开始 `D601:UniDesk` 工作前必须通过 UniDesk SSH 桥执行 `cd /home/ubuntu/workspace/unidesk-dev && git status --short --branch && git remote -v`；若路径、分支、remote 或权限不符合预期，必须先修正固定 workspace，再继续开发、测试或发布。 - D601 UniDesk 开发必须先以 `/home/ubuntu/workspace/unidesk-dev` 做固定 repo 预检，再在 `/home/ubuntu/workspace/unidesk-dev/.worktree/` 从最新 `origin/master` 创建独立 worktree 修改和提交；不要把固定 repo 当作并行任务 scratch 区，细则见 `docs/reference/dev-environment.md`。 @@ -115,6 +116,7 @@ UniDesk 是一个以主 server 为统一入口的分布式工作平台；本文 ## Critical Tran Shell Boundary Rule - P0: `tran ...` 后面禁止裸放本地 shell 续接控制符，包括 `&&`、`;` 和 `|`；需要在远端执行多步命令时，必须使用 `tran script -- '远端完整脚本'`、`tran script <<'SCRIPT'` 或等价的单一 stdin/script 参数，避免后半段被本地 shell 执行。`script -- '<单个字符串>'` 会按远端 shell one-liner 执行；`script -- <多个 argv>` 才是 direct argv。`apply-patch`、`apply-patch-v1`、`script`、`py` 等 stdin/capture-backed operation 可以使用 heredoc 或 `< patch.diff` 作为本地输入。 +- P0: 新增或扩展 `ssh`/`tran` 高频 operation 不得把完整实现继续堆进 `scripts/src/ssh.ts`；`ssh.ts` 只保留 route/parser/broker/dispatch，共享能力拆到 `scripts/src/ssh-*.ts` 专门模块，细则见 `docs/reference/cli.md`。 ## CLI @@ -129,7 +131,7 @@ UniDesk 是一个以主 server 为统一入口的分布式工作平台；本文 - `bun scripts/cli.ts server cleanup plan [--min-age-hours N] [--limit N]`：只读/干跑生成主 server Docker 镜像清理计划，默认只列出至少 24 小时前创建的非保护镜像，输出 active/protected images、stale candidates、预计释放空间、风险等级和必须人工确认的 `docker image rm` 命令；禁止默认删除、禁止 prune、禁止触碰 database volume、registry storage 或 Baidu Netdisk 状态。 - `bun scripts/cli.ts server rebuild `：以 build-first、Compose lock、no-deps force-recreate 和 post-up validation 的异步 job 重建主 server Compose 内单个服务；对 database、File Browser、Code Queue 执行面、k3sctl-adapter 或未知对象返回结构化 `unsupported-server-rebuild`，规则见 `docs/reference/deployment.md` 与 `docs/reference/cicd-standardization.md`。 - `bun scripts/cli.ts provider attach [--master-server URL] [--up] [--force]` / `bun scripts/cli.ts provider triage [--observed-error text] [--observed-scope scope] [--microservice id ...] [--full|--raw]`：前者在新增计算节点上生成两项配置的 provider-gateway 挂载包；后者是只读多信号健康裁决入口，默认低噪声输出 `decision`、`healthyScopes`、`failedScopes`、`retryable` 和异常信号摘要，用来把单路径 `provider is not online`、SSH 超时、registry 失败或 proxy 失败归类为 `retryable-transient`、`service-degraded` 或 `global-offline`，完整 evidence 需显式 `--full|--raw`，规则见 `docs/reference/provider-gateway.md` 和 `docs/reference/code-queue-supervision.md`。 -- `bun scripts/cli.ts ssh [operation args...]` / `tran [operation args...]`：通过 provider-gateway 的 Host SSH / WSL SSH 维护桥进入 provider、host workspace、Windows cmd route、k3s 控制面或 pod workspace；主 server 人工/Codex 分布式操作必须用本机 `tran` wrapper，CLI 参考和可移植脚本可保留完整命令，细则见 `docs/reference/cli.md`、`docs/reference/windows-passthrough.md` 和 `docs/reference/provider-gateway.md`。 +- `bun scripts/cli.ts ssh [operation args...]` / `tran [operation args...]`：通过 provider-gateway 的 Host SSH / WSL SSH 维护桥进入 provider、host workspace、Windows cmd route、k3s 控制面或 pod workspace，并提供带 SHA-256 校验的 `upload`/`download` 文件传输；主 server 人工/Codex 分布式操作必须用本机 `tran` wrapper，CLI 参考和可移植脚本可保留完整命令，细则见 `docs/reference/cli.md`、`docs/reference/windows-passthrough.md` 和 `docs/reference/provider-gateway.md`。 - `bun scripts/cli.ts microservice list/status/health/diagnostics/tunnel-self-test/proxy`：管理和验证挂载在主 server、计算节点 Docker 或 k3s 控制面上的用户服务，`status/health/diagnostics` 默认 compact summary 并用 `--full|--raw` 展开完整 body，`proxy` 支持受控 JSON body，OA Event Flow/Todo Note/Baidu Netdisk/Code Queue Manager on main-server、k3s Control/Code Queue 执行面/MDTODO/Decision Center/FindJob/Pipeline/MET Nonlinear on D601 的规则见 `docs/reference/microservices.md`。 - `bun scripts/cli.ts microservice health/diagnostics/proxy code-agent-sandbox`：验证独立 Code Agent Sandbox 的 health、只读 diagnostics、trace 和 adapter/mode/credential boundary 契约，规则见 `docs/reference/code-agent-sandbox.md`。 - `bun scripts/cli.ts decision upload/list/show/health`：通过 backend-core 用户服务代理上传会议记录/需求/决议 Markdown、列出记录和查看详情；Decision Center 运行在 D601 k3s，规则见 `docs/reference/microservices.md`。 diff --git a/docs/reference/cli.md b/docs/reference/cli.md index c52e1639..cd2ba959 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -19,12 +19,12 @@ CLI 可以从 `master` 快速演进，但必须兼容 `deploy.json` 固定的 CI - `server cleanup plan [--min-age-hours N] [--limit N]` 只生成主 server Docker 镜像清理 dry-run 计划，不执行删除；默认 `--min-age-hours 24`，避免把刚发布或刚验证的镜像列为 stale。输出必须包含 `dryRun=true`、`mutation=false`、`policy.deletionExecuted=false`、active containers/images、受保护镜像、candidate stale images、估算释放空间、风险等级、`commandsToReview` 和人工审批清单。计划必须保守白名单：保留 running containers 使用的 image ID，保留 stopped containers 引用的 image ID 直到人工先复核容器，保留 `deploy.json`/`CI.json` 当前 commit-pinned artifact、Compose stable image、上游 digest pin 和 provider-gateway runner image；`protectedStorage` 必须显式列出 PostgreSQL named volume、Baidu Netdisk `.state`、D601 registry storage 和 Docker volumes/host data policy。该入口禁止生成或执行 `docker system prune`、`docker image prune`、`docker builder prune`、`docker volume rm`、`docker compose down -v`、数据库清理或 host data `rm` 命令；未来若增加真实删除，必须另设显式审批参数并先复核 dry-run 输出。 - `server rebuild ` 创建异步 job，先构建目标服务镜像，随后在 `.state/locks/server-compose.lock` 串行保护下用 `--no-deps --force-recreate` 替换目标 service 并等待容器 `healthy/running`；该命令用于替代手工删除容器的兜底流程，其中 `dev-frontend-proxy` 只更新主 server dev 入口薄代理，`todo-note`、`code-queue-mgr`、`project-manager`、`baidu-netdisk` 和 `oa-event-flow` 只重建主 server 承载的对应后端，不会重建或删除 database 命名卷。D601 Code Queue 执行面不由 `server rebuild` 管理，Rust backend-core 迭代不得用 `server rebuild backend-core` 在 master server 编译，规则见 `docs/reference/dev-environment.md`。 - `provider attach [--master-server URL] [--up] [--force]` 在新计算节点生成两项配置的 provider-gateway 挂载包：`.state/provider-.env` 默认只包含 `UNIDESK_MASTER_SERVER` 与 `PROVIDER_ID`，`provider-.yml` 固定 Docker socket、`pid: "host"`、`restart: always`、只读 `/workspace` 和 SSH 维护私钥挂载；`--up` 会立即执行生成的 `docker compose up -d --build`。`provider triage [--observed-error text] [--observed-scope scope] [--microservice id ...] [--full|--raw]` 是只读多信号健康裁决入口，会把单路径 `provider is not online`、SSH 超时、registry 失败和 service proxy 失败归类成 `runner-local-observation-gap`、`service-degraded`、`provider-degraded` 或 `global-blocker`。默认输出只返回裁决、scope、失败/降级/未知信号和有界 evidence 摘要，完整 evidence 必须显式加 `--full` 或 `--raw`；推荐交叉验证命令仍包含 `debug health`、`debug dispatch host.ssh --wait-ms 15000`、`ssh argv true`、`artifact-registry health --provider-id `、`microservice health k3sctl-adapter`、`microservice health code-queue` 和 `codex tasks --view supervisor --limit 20`。 -- `ssh [operation args...]` / `tran [operation args...]` 通过 backend-core 内网 WebSocket broker 和 provider-gateway 的 Host SSH / WSL SSH 维护桥连接目标节点；`route` 基础形态是 provider id，例如 `D601` 或 `G14`，也可以扩展为纯定位路径 `provider:plane[:namespace:resource[:container]]`，例如 `D601:win`、`D601:win/c/test`、`G14:k3s`、`D601:k3s` 或 `G14:k3s::`。WSL provider 的 Windows cmd 入口固定写 `tran D601:win cmd `，需要 Windows cwd 时用 `tran D601:win/c/test cmd cd`，由 CLI 自动设置 `chcp 65001`、`PYTHONUTF8=1` 和 `PYTHONIOENCODING=utf-8`；命名只允许 `win`，不得使用 `win32`。非交互远端命令优先使用 `ssh argv ...`；需要 shell 脚本、管道、变量或循环时优先使用 quoted heredoc 单步传输，例如 `tran G14 script <<'SCRIPT'`、`tran G14:k3s script <<'SCRIPT'` 或 `tran G14:k3s:: script <<'SCRIPT'`，把脚本走 stdin。`script -- '<单个字符串>'` 是无需 stdin 的远端 shell one-liner，例如 `tran G14:/root/hwlab script -- 'cd /root/hwlab && git status --short --branch'`；`script -- <多个 argv>` 才是 direct argv，适合 `tran D601:/path script -- sed -n '1,20p' file` 这类带短横线的单进程命令。顶层 remote option parser 必须保留命令已经开始后的 `--`，不得把它吞成全局选项结束符。需要远端改文本文件时默认优先使用 ` apply-patch < patch.diff`；需要旧 helper 时显式使用 `:k3s:: apply-patch-v1` 或 ` apply-patch-v1`。ssh-like 命令遇到 timeout/kex/255 类失败时，CLI 会在 stderr 追加一行 `UNIDESK_SSH_HINT` JSON，提示 stdin script/argv 重试和 provider triage 交叉验证。 +- `ssh [operation args...]` / `tran [operation args...]` 通过 backend-core 内网 WebSocket broker 和 provider-gateway 的 Host SSH / WSL SSH 维护桥连接目标节点；`route` 基础形态是 provider id，例如 `D601` 或 `G14`，也可以扩展为纯定位路径 `provider:plane[:namespace:resource[:container]]`，例如 `D601:win`、`D601:win/c/test`、`G14:k3s`、`D601:k3s` 或 `G14:k3s::`。WSL provider 的 Windows cmd 入口固定写 `tran D601:win cmd `，需要 Windows cwd 时用 `tran D601:win/c/test cmd cd`，由 CLI 自动设置 `chcp 65001`、`PYTHONUTF8=1` 和 `PYTHONIOENCODING=utf-8`；命名只允许 `win`，不得使用 `win32`。非交互远端命令优先使用 `ssh argv ...`；需要 shell 脚本、管道、变量或循环时优先使用 quoted heredoc 单步传输，例如 `tran G14 script <<'SCRIPT'`、`tran G14:k3s script <<'SCRIPT'` 或 `tran G14:k3s:: script <<'SCRIPT'`，把脚本走 stdin。`script -- '<单个字符串>'` 是无需 stdin 的远端 shell one-liner，例如 `tran G14:/root/hwlab script -- 'cd /root/hwlab && git status --short --branch'`；`script -- <多个 argv>` 才是 direct argv，适合 `tran D601:/path script -- sed -n '1,20p' file` 这类带短横线的单进程命令。顶层 remote option parser 必须保留命令已经开始后的 `--`，不得把它吞成全局选项结束符。需要远端改文本文件时默认优先使用 ` apply-patch < patch.diff`；需要可靠传输非文本或整文件时使用 ` upload ` 和 ` download `，CLI 会按字节数与 SHA-256 自动校验并在 provider-gateway stdin/argv 限制下切换客户端分块策略；需要旧 helper 时显式使用 `:k3s:: apply-patch-v1` 或 ` apply-patch-v1`。ssh-like 命令遇到 timeout/kex/255 类失败时，CLI 会在 stderr 追加一行 `UNIDESK_SSH_HINT` JSON，提示 stdin script/argv 重试和 provider triage 交叉验证。 - `ssh apply-patch < patch.diff` 是默认推荐的远端 patch 入口：本地 TypeScript line-based engine 解析和计算新文件内容，远端 route 只负责读写文件；支持 host workspace、k3s pod workspace 和 frontend transport，并优先处理长中文/Unicode、低上下文插入、重复块 `@@` 定位等旧 helper 容易失败的场景。`ssh apply-patch-v1 [tool args...] < patch.diff` 保留为 v1 fallback，直接调用远端注入的 `apply_patch` sh/perl helper；只有默认 v2 引擎出现问题、需要复用旧 helper 行为或人工确认 `--allow-loose` 时才优先使用 v1。 - `ssh py [script-args...] < script.py` 把本地 stdin 落到远端临时 `.py` 文件后再以 `python3 -u` 执行并自动清理，避免再手写 `'python3 -'`、heredoc 或多层引号；`script-args` 会按 argv 安全透传给远端脚本。 - `ssh skills [--scope all|wsl|windows] [--limit N]` 发现目标节点上的 WSL/Linux skill 根目录；当 provider 是 WSL 时同一次调用还会扫描 Windows 用户目录下的 `.agents/skills` 与 `.codex/skills`。 - `ssh :k3s[:namespace:workload[:container]] ...` 是原生 k3s 结构化 route 入口，route 只定位控制面或 workload，`kubectl`、`logs`、`exec`、`script`、`apply-patch`、旧 `apply-patch-v1` fallback 和普通容器命令作为 operation 放在 route 之后；CLI 固定注入 `KUBECONFIG=/etc/rancher/k3s/k3s.yaml` 并把 kubectl、workload exec、logs 和 pod workspace 读写参数组装成 argv，避免在 Host SSH、bash、kubectl exec 和容器 shell 之间反复手写多层引号；D601 与 G14 都有 provider-specific guard，分别校验 `d601` 和 G14 k3s 节点身份。 -- Code Queue runner 镜像必须在 PATH 上提供 `/usr/local/bin/tran`。runner 内的 `tran` 检测到 `CODE_QUEUE_*` 或 `KUBERNETES_SERVICE_HOST` 后，默认执行 `bun /root/unidesk/scripts/cli.ts --main-server-ip ssh ...`，其中 `` 优先来自 `UNIDESK_MAIN_SERVER_IP` / `UNIDESK_MAIN_SERVER_HOST` / `CODE_QUEUE_DEV_CONTAINER_MASTER_HOST`。runner remote frontend HTTP 客户端默认使用 `curl` 后端，降低 Bun 在部分 runner 内读取非 SSH HTTP response body 时触发 native crash 的风险；显式 `UNIDESK_REMOTE_HTTP_CLIENT=fetch` 可用于诊断。runner 内跨 D601/G14 的分布式访问应优先使用结构化 route/operation，例如 `tran D601 argv ...`、`tran G14 argv ...`、`tran D601:k3s kubectl ...`、`tran D601:k3s:: argv ...` 和 `tran G14:/absolute/workspace apply-patch ...`；`apply-patch`、`script`、`py` 和旧 `apply-patch-v1` fallback 经 frontend `/ws/ssh` 通道执行，stdout/stderr 也必须完整直通，不得退回 `/api/dispatch` task JSON。 +- Code Queue runner 镜像必须在 PATH 上提供 `/usr/local/bin/tran`。runner 内的 `tran` 检测到 `CODE_QUEUE_*` 或 `KUBERNETES_SERVICE_HOST` 后，默认执行 `bun /root/unidesk/scripts/cli.ts --main-server-ip ssh ...`，其中 `` 优先来自 `UNIDESK_MAIN_SERVER_IP` / `UNIDESK_MAIN_SERVER_HOST` / `CODE_QUEUE_DEV_CONTAINER_MASTER_HOST`。runner remote frontend HTTP 客户端默认使用 `curl` 后端，降低 Bun 在部分 runner 内读取非 SSH HTTP response body 时触发 native crash 的风险；显式 `UNIDESK_REMOTE_HTTP_CLIENT=fetch` 可用于诊断。runner 内跨 D601/G14 的分布式访问应优先使用结构化 route/operation，例如 `tran D601 argv ...`、`tran G14 argv ...`、`tran D601:k3s kubectl ...`、`tran D601:k3s:: argv ...`、`tran G14:/absolute/workspace apply-patch ...` 和 `tran upload|download ...`；`apply-patch`、`upload`、`download`、`script`、`py` 和旧 `apply-patch-v1` fallback 经 frontend `/ws/ssh` 通道执行，stdout/stderr 也必须完整直通，不得退回 `/api/dispatch` task JSON。 - `microservice list/status/health/diagnostics/tunnel-self-test/proxy` 通过 backend-core 内网 API 管理挂载在计算节点 Docker 或 k3s 控制面中的用户服务（底层命令名仍为 microservice）；`health`、`status` 和 `diagnostics` 默认返回 compact summary、body 字节数和 `--full|--raw` 展开命令，只有小 body 或无法抽取 summary 时才带有界 body preview，避免 Code Queue/k3s 诊断一次性输出爆炸；`tunnel-self-test` 和 `proxy` 会走真实 backend-core -> provider-gateway 或 k3sctl-adapter -> 节点服务链路。`microservice health code-queue` 使用 commander-safe 专用摘要，必须保留 ok/status、service id、running count、queue count、heartbeat freshness/risk、split-brain/live/degraded 解释和 raw drill-down 命令；需要完整健康 JSON 时显式加 `--raw` 或 `--full`，等价深挖路径是 `microservice proxy code-queue /health --raw --full`。`proxy` 支持受控 JSON 请求体并对超大响应 body 默认输出有界预览，规则见 `docs/reference/microservices.md`。 - `decision upload/list/show/health` 通过 backend-core 用户服务代理访问 D601 k3s Decision Center，用于上传会议记录/决议 Markdown、列出权威记录、查看详情和健康检查；`decision list` 默认只返回摘要并省略完整 Markdown body，需要排查大正文时显式加 `--include-body`。正式文书字段通过 records 模型一等字段返回和查询：`--doc-no DC-...`、`--doc-type DCSN|GOAL|PLAN|RPRT|ACTN|ISSU|RETR|RQST|RESP|MINS`、`--doc-priority P0|P1|P2|P3`、`--year YYYY`、`--signer`、`--issued-at`、`--effective-scope`、`--supersedes`、`--superseded-by`；`show` 和 `requirement update` 可使用 `id` 或 `docNo`。`decision requirement list/create/upsert/update/show` 在同一 records 模型上管理 `goal|decision|blocker|debt|experiment` 需求记录，`docNo` 唯一，未传 `--doc-no` 但提供 `--doc-type/--doc-priority/--year` 时由服务分配下一个序号。它们不得直连 D601 Service、NodePort 或 provider-gateway 业务 HTTP。 - `decision diary import ` 将带 `# YYYY年M月D日`、`# YYYY-MM-DD` 或 `# YYYY/M/D` 标题的工作日志拆成每天一篇 Markdown 日记，按 `YYYY-MM/YYYY-MM-DD.md` 虚拟路径写入 Decision Center PostgreSQL；`decision diary list/history` 默认只返回摘要，需要完整 Markdown 时显式加 `--include-body`；`decision diary show [--source-file path]` 查看单日正文，`--source-file` 用于同一天存在多个导入来源时精确选择；`decision diary edit|upsert --body-file [--title text] [--source-file path] [--tag tag]` 通过 `PUT /api/diary/entries/:idOrDate` 创建当天或历史条目并编辑既有条目。 @@ -135,6 +135,10 @@ exec /root/unidesk/scripts/tran "$@" `bun scripts/cli.ts ssh D518` 应表现为登录 D518 WSL 的 shell；`bun scripts/cli.ts ssh D518 hostname` 应像 `ssh D518 hostname` 一样只输出远端命令结果并返回远端 exit code。Provider ID 前的目标选择由 UniDesk 节点清单决定，`-p`、`-i`、`-l`、`-o` 等传统 ssh 传输参数由 provider-gateway 部署配置统一管理，CLI 会兼容性消费这些参数但不会覆盖节点侧维护桥配置。指挥官、CI 预检和其他非交互流程不要依赖 ssh-like 自由拼接；单进程标准写法是 `bun scripts/cli.ts ssh D601 argv true`，多行 shell 逻辑标准写法是 quoted heredoc 单步调用 `bun scripts/cli.ts ssh D601 script <<'SCRIPT'`。 +UniDesk CLI/tran 客户端改进本身是 master server 高频控制入口维护，可以直接在 `/root/unidesk` 轻量开发、提交并推送 `origin/master`；不要为这类客户端小改强制迁移到 D601 worktree。该例外不改变 master server 禁重型验证规则：仓库级 check、Playwright/browser smoke、镜像构建、Rust/Go 编译、Code Queue runner 实测仍必须在 D601、CI runner 或目标运行面执行。若 `tran`/SSH 文件传输遇到 provider-gateway 单次 stdin、argv 或 stdout 限制，先在 CLI 客户端做分块、SHA-256 校验、失败可观测输出和最小真实闭环；只有 client 侧不能解决且有证据时，才改 provider-gateway。 + +`scripts/src/ssh.ts` 只承担 route/operation parser、共享远端命令构造、broker 调用和顶层 dispatch。新增或扩展高频 operation 不得继续把完整实现堆进 `ssh.ts`；应按能力拆到专门模块，例如整文件传输放在 `scripts/src/ssh-file-transfer.ts`，再由 `ssh.ts` 和 frontend remote transport 传入共享 command builder/bridge executor。后续新增 operation 也按 `scripts/src/ssh-.ts` 或等价专门模块组织，帮助文本、合同测试和 reference 与代码同一变更集更新。 + core 只允许声明了 `host.ssh` capability 的 provider 使用 `ssh` 透传或 `host.ssh` dispatch；旧 provider 不支持该能力时必须快速失败并输出错误，不能把未知命令误判成 `echo` 成功。 本地 broker 默认等待 provider SSH 会话打开 60000ms，以便在目标节点同时有较多 microservice.http 任务时仍能建立维护会话；需要诊断慢连接时可用 `UNIDESK_SSH_OPEN_TIMEOUT_MS=` 临时调大，但最小有效值固定为 15000ms，避免把真实离线误判为长时间阻塞。注意 open timeout 只控制“会话打开”阶段，不能绕过 60 秒最外层运行时硬超时。 diff --git a/docs/reference/dev-environment.md b/docs/reference/dev-environment.md index 6be5f5be..cb291125 100644 --- a/docs/reference/dev-environment.md +++ b/docs/reference/dev-environment.md @@ -29,6 +29,10 @@ tran D601:/home/ubuntu/workspace/unidesk-dev git remote -v Master server 不作为 UniDesk 重型验证机。仓库级 check、Playwright/browser smoke、镜像构建、Rust/Go 编译和 Code Queue runner 实测必须放到 D601、CI runner 或其他获批执行面；master server 只做轻量源码编辑、Git 操作、状态观察和受控调度。 +`scripts/cli.ts`、`scripts/tran`、`scripts/src/ssh.ts` 和相邻的 `tran`/SSH 透传 helper 是主 server 上人工与 Codex 高频使用的控制入口；这类客户端工具链改进可以直接在 master server `/root/unidesk` 轻量修改、提交并推送到 `origin/master`。该例外只覆盖 CLI/tran 客户端源码、帮助、合同测试和对应 reference 文档，不覆盖 `src/components/provider-gateway` 行为变更、镜像构建、仓库级 check、浏览器 smoke 或其他重型验证。涉及 provider-gateway 代码时仍必须遵循 provider-gateway 版本和远程升级规则。 + +当 `tran`/SSH 透传的文件传输、stdin、chunk、编码、timeout 或 route/operation 解析出现高频摩擦时，先优化 CLI 客户端的分块、校验、重试、可观测输出和帮助文档，并用目标 provider/pod/Windows route 的最小闭环证明；只有证据显示 client 侧无法规避 provider-gateway 边界时，才进入 provider-gateway 变更流程。 + ## Public Dev Frontend Port The main server owns one extra public entrypoint for dev UI: diff --git a/scripts/src/help.ts b/scripts/src/help.ts index 334eda25..47f768b1 100644 --- a/scripts/src/help.ts +++ b/scripts/src/help.ts @@ -21,6 +21,7 @@ export function rootHelp(): unknown { { command: "provider attach [--master-server URL] [--up] [--force] | provider triage [--observed-error text] [--observed-scope scope] [--microservice id ...] [--full|--raw]", description: "Generate the minimal external provider-gateway env/compose bundle or run the low-noise read-only provider health triage contract." }, { command: "ssh [operation args...]", description: "Open a Host SSH / WSL SSH maintenance session through the provider-gateway bridge; route syntax such as `G14:k3s` or `D601:win/c/test` only locates distributed targets." }, { command: "ssh apply-patch < patch.diff", description: "Default remote text patch entry: apply a standard patch with the local TypeScript v2 engine while the remote route only reads and writes files." }, + { command: "ssh upload | ssh download ", description: "Transfer whole files through SSH passthrough with remote temp files, byte-count checks, SHA-256 verification, and client-side chunk fallback." }, { command: "ssh apply-patch-v1 [tool args...] < patch.diff", description: "Fallback to the injected legacy remote apply_patch helper directly over SSH passthrough and stream the patch from local stdin." }, { command: "ssh py [script-args...] < script.py", description: "Run remote Python from local stdin through SSH passthrough without nested shell quoting; extra args become script argv." }, { command: "ssh script [--shell sh|bash] [script-args...] <<'SCRIPT' ...", description: "Run a remote shell script from local stdin using shell -s; the default sh inherits provider proxy env, so --shell bash is only for bash-specific syntax." }, @@ -150,6 +151,8 @@ export function sshHelp(): unknown { "bun scripts/cli.ts ssh ", "bun scripts/cli.ts ssh argv [args...]", "bun scripts/cli.ts ssh :/absolute/workspace apply-patch < patch.diff", + "bun scripts/cli.ts ssh upload ", + "bun scripts/cli.ts ssh download ", "bun scripts/cli.ts ssh apply-patch-v1 [--allow-loose] < patch.diff", "bun scripts/cli.ts ssh py [script-args...] < script.py", "bun scripts/cli.ts ssh script [--shell sh|bash] [script-args...] <<'SCRIPT'", @@ -170,6 +173,8 @@ export function sshHelp(): unknown { "bun scripts/cli.ts ssh D601:k3s:hwlab-dev:hwlab-cloud-api/app apply-patch <<'PATCH'", "bun scripts/cli.ts ssh D601:k3s:hwlab-dev:hwlab-cloud-api apply-patch-v1 <<'PATCH'", "tar -C /path/to/files -cf - . | bun scripts/cli.ts ssh D601:k3s:unidesk:code-queue/root/unidesk exec --stdin -- tar -xf - -C /root/unidesk", + "bun scripts/cli.ts ssh D601:win upload ./tool.mjs F:\\Work\\hwlab\\.tmp\\tool.mjs", + "bun scripts/cli.ts ssh D601:win download F:\\Work\\hwlab\\.tmp\\tool.mjs ./tool.mjs", "bun scripts/cli.ts ssh D601:k3s:hwlab-dev:hwlab-cloud-api node -e 'console.log(process.version)'", "bun scripts/cli.ts ssh D601:k3s:hwlab-dev:hwlab-cloud-api script <<'SCRIPT'", "bun scripts/cli.ts ssh D601:k3s:hwlab-dev:hwlab-cloud-api logs --tail 80", @@ -181,6 +186,7 @@ export function sshHelp(): unknown { "When a one-line shell command is easier to type through the script path, `script -- ''` runs that single string through the remote shell without waiting for stdin. When `script --` is followed by multiple tokens, it stays a direct argv form for commands such as `tran D601:/work script -- sed -n '1,20p' file`.", "For arbitrary stdin streams into a workload command, use a workload route plus `exec --stdin -- ...`; this keeps the route as location-only and avoids heredoc/base64/tar shell wrapping.", "`apply-patch` is the default remote text patch entry and uses the v2 local line-based patch engine with remote read/write operations, so long Unicode/Chinese lines and pure insertion hunks avoid the legacy remote shell hunk parser.", + "`upload` and `download` are the default whole-file transfer entries for non-text and generated files. They write through remote temp files, verify byte count and SHA-256 on both sides, and fall back from a single stdin payload to bounded client-side chunks before treating provider-gateway limits as a server-side problem.", "`apply-patch-v1` is the only legacy fallback entry: it rejects low-context update hunks by default, reports the matched file:line for each hunk on stderr, and only accepts --allow-loose when the caller has manually reviewed an intentionally ambiguous insertion.", "script defaults to target /bin/sh and inherits provider proxy variables such as HTTP_PROXY/HTTPS_PROXY/ALL_PROXY/NO_PROXY; use --shell bash only for bash syntax such as pipefail, arrays, or [[ ... ]], not as a proxy workaround.", "Route syntax is `{provider}:{plane}[:{scope...}] {operation} [operation-args...]`: the first argv token locates a distributed target only, and every following token belongs to the operation parser. Host workspace routes use `:/absolute/workspace`; WSL providers can use `:win cmd ` to run Windows host cmd.exe with UTF-8 defaults, and `:win/c/test cmd cd` maps the Windows cwd to `C:\\test`; native k3s providers such as D601 and G14 use :k3s for the control plane, :k3s:: for a workload, and :k3s::/ for a pod workspace.", diff --git a/scripts/src/remote.ts b/scripts/src/remote.ts index 5acdd834..735ee7d0 100644 --- a/scripts/src/remote.ts +++ b/scripts/src/remote.ts @@ -7,6 +7,7 @@ import { type DebugDispatchCommand, isDebugDispatchCommand } from "./debug"; import { summarizeMicroserviceHealthResponse, summarizeMicroserviceObservation, summarizeMicroserviceProxyResponse } from "./microservices"; import { parseNetworkPerfOptions, runNetworkPerf } from "./network-perf"; import { + buildWindowsPowerShellInvocation, formatSshFailureHint, formatSshRuntimeTimeoutHint, formatSshRuntimeTimingHint, @@ -20,6 +21,7 @@ import { wrapSshRemoteCommand, type SshCaptureResult, } from "./ssh"; +import { isSshFileTransferOperation, runSshFileTransferOperation, type SshRemoteCommandExecutor } from "./ssh-file-transfer"; import { runApplyPatchV2, type ApplyPatchV2Executor } from "./apply-patch-v2"; import { codexJudgeQueryAsync, codexOutputQueryAsync, codexPrPreflightQueryAsync, codexQueuesQueryAsync, codexTaskQueryAsync, codexTasksQueryAsync, codexUnreadTriageAsync } from "./code-queue"; import { runDecisionCenterCommandAsync } from "./decision-center"; @@ -1085,7 +1087,16 @@ async function runRemoteSshWebSocketCapture( command: string[], input?: string, ): Promise { - const remoteCommand = remoteCommandForRoute(invocation.route, command); + const remoteCommand = remoteCommandForRoute(invocation.route, command, { stdin: input !== undefined }); + return await runRemoteSshWebSocketCaptureRemoteCommand(session, invocation, remoteCommand, input); +} + +async function runRemoteSshWebSocketCaptureRemoteCommand( + session: FrontendSession, + invocation: ReturnType, + remoteCommand: string, + input?: string, +): Promise { const captureInvocation = { ...invocation, parsed: { ...invocation.parsed, remoteCommand, requiresStdin: input !== undefined, invocationKind: "helper" as const }, @@ -1253,6 +1264,15 @@ export function remoteSshFrontendPlanForTest(target: string, args: string[]): Re async function runRemoteSshOverFrontend(session: FrontendSession, target: string | undefined, args: string[]): Promise { if (!target) throw new Error("remote ssh requires a route, for example: bun scripts/cli.ts --main-server-ip 74.48.78.17 ssh D601 hostname"); const invocation = parseSshInvocation(target, args); + if (isSshFileTransferOperation(args)) { + const executor: SshRemoteCommandExecutor = { + runRemoteCommand: (remoteCommand, input) => runRemoteSshWebSocketCaptureRemoteCommand(session, invocation, remoteCommand, input), + }; + return await runSshFileTransferOperation(invocation, args, executor, { + buildRouteCommand: remoteCommandForRoute, + buildWindowsPowerShellCommand: buildWindowsPowerShellInvocation, + }); + } if ((args[0] ?? "") === "apply-patch") { const executor: ApplyPatchV2Executor = { run: (command, input) => runRemoteSshWebSocketCapture(session, invocation, command, input), diff --git a/scripts/src/ssh-file-transfer.ts b/scripts/src/ssh-file-transfer.ts new file mode 100644 index 00000000..60af6a55 --- /dev/null +++ b/scripts/src/ssh-file-transfer.ts @@ -0,0 +1,409 @@ +import { createHash, randomBytes } from "node:crypto"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import path from "node:path"; +import type { ParsedSshInvocation, ParsedSshRoute, SshCaptureResult } from "./ssh"; + +export interface SshRemoteCommandExecutor { + runRemoteCommand(remoteCommand: string, input?: string): Promise; +} + +export interface SshFileTransferCommandBuilders { + buildRouteCommand(route: ParsedSshRoute, command: string[], options?: { stdin?: boolean }): string; + buildWindowsPowerShellCommand(script: string): string; +} + +type SshFileTransferOperation = + | "stat" + | "read-b64-block" + | "write-b64-argv" + | "write-b64-stdin" + | "write-b64-begin" + | "write-b64-append-stdin" + | "write-b64-commit"; + +interface SshFileTransferCliOptions { + action: "upload" | "download"; + localPath: string; + remotePath: string; + readBlockBytes: number; +} + +interface SshFileTransferStat { + bytes: number; + sha256: string; +} + +interface SshFileTransferWriteResult { + strategy: "argv" | "stdin" | "chunked-stdin"; + chunks: number; +} + +class SshFileTransferError extends Error { + constructor(message: string, public readonly details: Record = {}) { + super(message); + this.name = "SshFileTransferError"; + } +} + +const fileTransferReadBlockBytes = 45_000; +const fileTransferWriteB64ArgvLimit = 48_000; +const fileTransferWriteB64ChunkChars = 12_000; + +export function isSshFileTransferOperation(args: string[]): boolean { + const subcommand = args[0] ?? ""; + return subcommand === "upload" || subcommand === "download"; +} + +export async function runSshFileTransferOperation( + invocation: ParsedSshInvocation, + args: string[], + executor: SshRemoteCommandExecutor, + builders: SshFileTransferCommandBuilders, +): Promise { + const options = parseSshFileTransferCliOptions(args); + const localPath = path.resolve(options.localPath); + if (options.action === "upload") { + const content = await readFile(localPath); + const expected = { bytes: content.length, sha256: sha256HexBuffer(content) }; + const write = await writeRemoteFileVerified(invocation, executor, builders, options.remotePath, content); + const remote = await statRemoteFile(invocation, executor, builders, options.remotePath); + assertTransferStat("upload final remote verification", options.remotePath, expected, remote); + process.stdout.write(`${JSON.stringify({ + ok: true, + command: "ssh upload", + route: invocation.route.raw, + providerId: invocation.providerId, + localPath, + remotePath: options.remotePath, + bytes: expected.bytes, + sha256: expected.sha256, + verified: true, + transfer: write, + }, null, 2)}\n`); + return 0; + } + + const read = await readRemoteFileVerified(invocation, executor, builders, options.remotePath, options.readBlockBytes); + await mkdir(path.dirname(localPath), { recursive: true }); + await writeFile(localPath, read.content); + const local = await readFile(localPath); + const localStat = { bytes: local.length, sha256: sha256HexBuffer(local) }; + assertTransferStat("download final local verification", localPath, read.remote, localStat); + process.stdout.write(`${JSON.stringify({ + ok: true, + command: "ssh download", + route: invocation.route.raw, + providerId: invocation.providerId, + remotePath: options.remotePath, + localPath, + bytes: read.remote.bytes, + sha256: read.remote.sha256, + verified: true, + transfer: { + strategy: "chunked-read", + chunks: read.chunks, + chunkBytes: options.readBlockBytes, + }, + }, null, 2)}\n`); + return 0; +} + +function parseSshFileTransferCliOptions(args: string[]): SshFileTransferCliOptions { + const action = args[0]; + if (action !== "upload" && action !== "download") throw new Error("ssh file transfer requires upload or download"); + const positionals: string[] = []; + let readBlockBytes = fileTransferReadBlockBytes; + for (let index = 1; index < args.length; index += 1) { + const arg = args[index] ?? ""; + if (arg === "--") { + positionals.push(...args.slice(index + 1)); + break; + } + if (arg === "--chunk-bytes" || arg === "--block-bytes") { + const value = args[index + 1]; + if (value === undefined) throw new Error(`ssh ${action} ${arg} requires a value`); + readBlockBytes = boundedTransferChunkBytes(value, `ssh ${action} ${arg}`); + index += 1; + continue; + } + if (arg === "--help" || arg === "-h" || arg === "help") { + throw new Error(`ssh ${action} usage: ssh ${action} ${action === "upload" ? " " : " "} [--chunk-bytes N]`); + } + if (arg.startsWith("-")) throw new Error(`unsupported ssh ${action} option: ${arg}`); + positionals.push(arg); + } + if (positionals.length !== 2) { + throw new Error(`ssh ${action} requires exactly two paths: ${action === "upload" ? " " : " "}`); + } + const [first, second] = positionals; + if (!first || !second) throw new Error(`ssh ${action} paths must be non-empty`); + return action === "upload" + ? { action, localPath: first, remotePath: second, readBlockBytes } + : { action, remotePath: first, localPath: second, readBlockBytes }; +} + +function boundedTransferChunkBytes(raw: string, option: string): number { + const value = Number(raw); + if (!Number.isInteger(value) || value <= 0) throw new Error(`${option} must be a positive integer`); + return Math.min(96_000, Math.max(1024, value)); +} + +async function writeRemoteFileVerified( + invocation: ParsedSshInvocation, + executor: SshRemoteCommandExecutor, + builders: SshFileTransferCommandBuilders, + remotePath: string, + content: Buffer, +): Promise { + const encoded = content.toString("base64"); + const expectedBytes = String(content.length); + const expectedSha256 = sha256HexBuffer(content); + if (invocation.route.plane !== "win" && encoded.length <= fileTransferWriteB64ArgvLimit) { + await checkedFileTransfer(invocation, executor, builders, "write-b64-argv", [remotePath, expectedBytes, expectedSha256, ...chunkString(encoded, fileTransferWriteB64ChunkChars)]); + return { strategy: "argv", chunks: encoded.length === 0 ? 0 : Math.ceil(encoded.length / fileTransferWriteB64ChunkChars) }; + } + try { + await checkedFileTransfer(invocation, executor, builders, "write-b64-stdin", [remotePath, expectedBytes, expectedSha256], encoded); + return { strategy: "stdin", chunks: 1 }; + } catch { + const token = `${process.pid}-${Date.now()}-${randomBytes(4).toString("hex")}-${expectedSha256.slice(0, 12)}`; + const chunks = chunkString(encoded, fileTransferWriteB64ChunkChars); + await checkedFileTransfer(invocation, executor, builders, "write-b64-begin", [remotePath, token]); + for (const chunk of chunks) { + await checkedFileTransfer(invocation, executor, builders, "write-b64-append-stdin", [remotePath, token], chunk); + } + await checkedFileTransfer(invocation, executor, builders, "write-b64-commit", [remotePath, token, expectedBytes, expectedSha256]); + return { strategy: "chunked-stdin", chunks: encoded.length === 0 ? 0 : chunks.length }; + } +} + +async function readRemoteFileVerified( + invocation: ParsedSshInvocation, + executor: SshRemoteCommandExecutor, + builders: SshFileTransferCommandBuilders, + remotePath: string, + readBlockBytes: number, +): Promise<{ remote: SshFileTransferStat; content: Buffer; chunks: number }> { + const remote = await statRemoteFile(invocation, executor, builders, remotePath); + const chunks: Buffer[] = []; + let actualBytes = 0; + let chunkCount = 0; + for (let blockIndex = 0; actualBytes < remote.bytes; blockIndex += 1) { + const read = await checkedFileTransfer(invocation, executor, builders, "read-b64-block", [remotePath, String(blockIndex), String(readBlockBytes)]); + const encoded = read.stdout.replace(/\s+/gu, ""); + const chunk = encoded.length === 0 ? Buffer.alloc(0) : Buffer.from(encoded, "base64"); + if (chunk.length === 0) { + throw new SshFileTransferError("remote download returned an empty block before EOF", { + route: invocation.route.raw, + remotePath, + blockIndex, + expectedBytes: remote.bytes, + actualBytes, + }); + } + chunks.push(chunk); + actualBytes += chunk.length; + chunkCount += 1; + } + const content = Buffer.concat(chunks); + const actual = { bytes: content.length, sha256: sha256HexBuffer(content) }; + assertTransferStat("download remote read verification", remotePath, remote, actual); + return { remote, content, chunks: chunkCount }; +} + +async function statRemoteFile( + invocation: ParsedSshInvocation, + executor: SshRemoteCommandExecutor, + builders: SshFileTransferCommandBuilders, + remotePath: string, +): Promise { + const stat = await checkedFileTransfer(invocation, executor, builders, "stat", [remotePath]); + return parseFileTransferStat(stat.stdout, remotePath); +} + +async function checkedFileTransfer( + invocation: ParsedSshInvocation, + executor: SshRemoteCommandExecutor, + builders: SshFileTransferCommandBuilders, + operation: SshFileTransferOperation, + args: string[], + input?: string, +): Promise { + const remoteCommand = buildFileTransferRemoteCommand(invocation.route, builders, operation, args, input !== undefined); + const result = await executor.runRemoteCommand(remoteCommand, input); + if (result.exitCode === 0) return result; + throw new SshFileTransferError("remote ssh file transfer operation failed", { + route: invocation.route.raw, + operation, + args: args.slice(0, 4), + exitCode: result.exitCode, + stdout: result.stdout.slice(-2000), + stderr: result.stderr.slice(-4000), + }); +} + +function buildFileTransferRemoteCommand( + route: ParsedSshRoute, + builders: SshFileTransferCommandBuilders, + operation: SshFileTransferOperation, + args: string[], + hasInput: boolean, +): string { + if (route.plane === "win") return buildWindowsFileTransferCommand(route, builders, operation, args); + return builders.buildRouteCommand(route, ["sh", "-c", posixFileTransferScript(), "unidesk-file-transfer", operation, ...args], { stdin: hasInput }); +} + +function parseFileTransferStat(stdout: string, remotePath: string): SshFileTransferStat { + const [bytesText, sha256] = stdout.trim().split(/\s+/u); + const bytes = Number(bytesText); + if (!Number.isSafeInteger(bytes) || bytes < 0 || !/^[0-9a-f]{64}$/u.test(sha256 ?? "")) { + throw new SshFileTransferError("remote file transfer stat returned invalid metadata", { + remotePath, + stdout: stdout.slice(0, 500), + }); + } + return { bytes, sha256: sha256! }; +} + +function assertTransferStat(label: string, pathName: string, expected: SshFileTransferStat, actual: SshFileTransferStat): void { + if (expected.bytes === actual.bytes && expected.sha256 === actual.sha256) return; + throw new SshFileTransferError(`${label} failed`, { + path: pathName, + expected, + actual, + }); +} + +function sha256HexBuffer(value: Buffer): string { + return createHash("sha256").update(value).digest("hex"); +} + +function chunkString(value: string, chunkSize: number): string[] { + const chunks: string[] = []; + for (let index = 0; index < value.length; index += chunkSize) { + chunks.push(value.slice(index, index + chunkSize)); + } + return chunks.length > 0 ? chunks : [""]; +} + +function posixFileTransferScript(): string { + return [ + "set -eu", + "sha256_file() {", + " if command -v sha256sum >/dev/null 2>&1; then sha256sum -- \"$1\" | awk '{print $1}'; return; fi", + " if command -v shasum >/dev/null 2>&1; then shasum -a 256 -- \"$1\" | awk '{print $1}'; return; fi", + " if command -v openssl >/dev/null 2>&1; then openssl dgst -sha256 -- \"$1\" | awk '{print $NF}'; return; fi", + " printf 'missing sha256 tool\\n' >&2; return 127", + "}", + "ensure_parent() { case \"$1\" in */*) parent=${1%/*}; mkdir -p -- \"$parent\";; esac; }", + "verify_tmp() {", + " target=$1; tmp=$2; expected_bytes=$3; expected_sha256=$4", + " actual_bytes=$(wc -c < \"$tmp\" | tr -d '[:space:]')", + " if [ \"$actual_bytes\" != \"$expected_bytes\" ]; then rm -f -- \"$tmp\"; printf 'transfer byte count mismatch for %s: expected=%s actual=%s\\n' \"$target\" \"$expected_bytes\" \"$actual_bytes\" >&2; exit 23; fi", + " actual_sha256=$(sha256_file \"$tmp\")", + " if [ \"$actual_sha256\" != \"$expected_sha256\" ]; then rm -f -- \"$tmp\"; printf 'transfer sha256 mismatch for %s: expected=%s actual=%s\\n' \"$target\" \"$expected_sha256\" \"$actual_sha256\" >&2; exit 24; fi", + "}", + "set_tmp_paths() {", + " target=$1; token=$2", + " case \"$token\" in ''|*[!a-zA-Z0-9_.-]*) printf 'invalid transfer temp token\\n' >&2; exit 2;; esac", + " base=${target##*/}; dir=.", + " case \"$target\" in */*) dir=${target%/*};; esac", + " tmp=\"$dir/.${base}.unidesk-transfer-${token}.tmp\"", + " tmp_b64=\"$tmp.b64\"", + "}", + "op=$1; shift", + "case \"$op\" in", + " stat)", + " target=$1; bytes=$(wc -c < \"$target\" | tr -d '[:space:]'); digest=$(sha256_file \"$target\"); printf '%s %s\\n' \"$bytes\" \"$digest\"", + " ;;", + " read-b64-block)", + " target=$1; block_index=$2; block_size=$3", + " case \"$block_index:$block_size\" in *[!0-9:]*|:*) printf 'invalid read block args\\n' >&2; exit 2;; esac", + " dd if=\"$target\" bs=\"$block_size\" skip=\"$block_index\" count=1 2>/dev/null | base64 | tr -d '\\n'", + " ;;", + " write-b64-argv)", + " target=$1; expected_bytes=$2; expected_sha256=$3; shift 3", + " ensure_parent \"$target\"; base=${target##*/}; dir=.; case \"$target\" in */*) dir=${target%/*};; esac", + " tmp=\"$dir/.${base}.unidesk-transfer-$$.tmp\"; tmp_b64=\"$tmp.b64\"; : > \"$tmp_b64\"", + " for chunk in \"$@\"; do printf '%s' \"$chunk\" >> \"$tmp_b64\"; done", + " if ! base64 -d < \"$tmp_b64\" > \"$tmp\"; then rm -f -- \"$tmp\" \"$tmp_b64\"; printf 'transfer base64 decode failed for %s\\n' \"$target\" >&2; exit 22; fi", + " rm -f -- \"$tmp_b64\"; verify_tmp \"$target\" \"$tmp\" \"$expected_bytes\" \"$expected_sha256\"; mv -f -- \"$tmp\" \"$target\"", + " actual_sha256=$(sha256_file \"$target\"); if [ \"$actual_sha256\" != \"$expected_sha256\" ]; then printf 'transfer final sha256 mismatch for %s\\n' \"$target\" >&2; exit 25; fi", + " ;;", + " write-b64-stdin)", + " target=$1; expected_bytes=$2; expected_sha256=$3", + " ensure_parent \"$target\"; base=${target##*/}; dir=.; case \"$target\" in */*) dir=${target%/*};; esac", + " tmp=\"$dir/.${base}.unidesk-transfer-$$.tmp\"", + " if ! base64 -d > \"$tmp\"; then rm -f -- \"$tmp\"; printf 'transfer base64 decode failed for %s\\n' \"$target\" >&2; exit 22; fi", + " verify_tmp \"$target\" \"$tmp\" \"$expected_bytes\" \"$expected_sha256\"; mv -f -- \"$tmp\" \"$target\"", + " actual_sha256=$(sha256_file \"$target\"); if [ \"$actual_sha256\" != \"$expected_sha256\" ]; then printf 'transfer final sha256 mismatch for %s\\n' \"$target\" >&2; exit 25; fi", + " ;;", + " write-b64-begin)", + " target=$1; token=$2; ensure_parent \"$target\"; set_tmp_paths \"$target\" \"$token\"; : > \"$tmp_b64\"", + " ;;", + " write-b64-append-stdin)", + " target=$1; token=$2; set_tmp_paths \"$target\" \"$token\"; cat >> \"$tmp_b64\"", + " ;;", + " write-b64-commit)", + " target=$1; token=$2; expected_bytes=$3; expected_sha256=$4; set_tmp_paths \"$target\" \"$token\"", + " if ! base64 -d < \"$tmp_b64\" > \"$tmp\"; then rm -f -- \"$tmp\" \"$tmp_b64\"; printf 'transfer base64 decode failed for %s\\n' \"$target\" >&2; exit 22; fi", + " rm -f -- \"$tmp_b64\"; verify_tmp \"$target\" \"$tmp\" \"$expected_bytes\" \"$expected_sha256\"; mv -f -- \"$tmp\" \"$target\"", + " actual_sha256=$(sha256_file \"$target\"); if [ \"$actual_sha256\" != \"$expected_sha256\" ]; then printf 'transfer final sha256 mismatch for %s\\n' \"$target\" >&2; exit 25; fi", + " ;;", + " *) printf 'unsupported transfer op: %s\\n' \"$op\" >&2; exit 2;;", + "esac", + ].join("\n"); +} + +function buildWindowsFileTransferCommand( + route: ParsedSshRoute, + builders: SshFileTransferCommandBuilders, + operation: SshFileTransferOperation, + args: string[], +): string { + return builders.buildWindowsPowerShellCommand(windowsFileTransferScript(route.workspace, operation, args)); +} + +function windowsFileTransferScript(basePath: string | null, operation: SshFileTransferOperation, args: string[]): string { + const target = args[0] ?? ""; + const tokenOrBlock = args[1] ?? ""; + const third = args[2] ?? ""; + const fourth = args[3] ?? ""; + const argvChunks = args.slice(3).map(powerShellSingleQuote).join(", "); + return [ + "$ErrorActionPreference = 'Stop';", + "$ProgressPreference = 'SilentlyContinue';", + "[Console]::InputEncoding = [System.Text.UTF8Encoding]::new();", + "[Console]::OutputEncoding = [System.Text.UTF8Encoding]::new();", + "$OutputEncoding = [System.Text.UTF8Encoding]::new();", + `$basePath = ${powerShellSingleQuote(basePath ?? "")};`, + `$operation = ${powerShellSingleQuote(operation)};`, + `$targetArg = ${powerShellSingleQuote(target)};`, + `$arg1 = ${powerShellSingleQuote(tokenOrBlock)};`, + `$arg2 = ${powerShellSingleQuote(third)};`, + `$arg3 = ${powerShellSingleQuote(fourth)};`, + `$argvChunks = @(${argvChunks});`, + "function Fail([string]$Message, [int]$Code) { [Console]::Error.WriteLine($Message); exit $Code }", + "function Resolve-UnideskPath([string]$Raw) { if ([string]::IsNullOrWhiteSpace($Raw)) { Fail 'empty transfer path' 2 }; if ([System.IO.Path]::IsPathRooted($Raw)) { return [System.IO.Path]::GetFullPath($Raw) }; if (-not [string]::IsNullOrWhiteSpace($basePath)) { return [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($basePath, $Raw)) }; return [System.IO.Path]::GetFullPath($Raw) }", + "function Ensure-Parent([string]$Target) { $parent = [System.IO.Path]::GetDirectoryName($Target); if (-not [string]::IsNullOrWhiteSpace($parent)) { [System.IO.Directory]::CreateDirectory($parent) | Out-Null } }", + "function Get-Sha256([string]$Path) { return (Get-FileHash -LiteralPath $Path -Algorithm SHA256).Hash.ToLowerInvariant() }", + "function Set-TmpPaths([string]$Target, [string]$Token) { if ($Token -notmatch '^[A-Za-z0-9_.-]+$') { Fail 'invalid transfer temp token' 2 }; $dir = [System.IO.Path]::GetDirectoryName($Target); if ([string]::IsNullOrWhiteSpace($dir)) { $dir = (Get-Location).ProviderPath }; $base = [System.IO.Path]::GetFileName($Target); $script:tmp = [System.IO.Path]::Combine($dir, '.' + $base + '.unidesk-transfer-' + $Token + '.tmp'); $script:tmpB64 = $script:tmp + '.b64' }", + "function Verify-Temp([string]$Target, [string]$Tmp, [Int64]$ExpectedBytes, [string]$ExpectedSha256) { $actualBytes = ([System.IO.FileInfo]$Tmp).Length; if ($actualBytes -ne $ExpectedBytes) { Remove-Item -LiteralPath $Tmp -Force -ErrorAction SilentlyContinue; Fail ('transfer byte count mismatch for ' + $Target + ': expected=' + $ExpectedBytes + ' actual=' + $actualBytes) 23 }; $actualSha256 = Get-Sha256 $Tmp; if ($actualSha256 -ne $ExpectedSha256) { Remove-Item -LiteralPath $Tmp -Force -ErrorAction SilentlyContinue; Fail ('transfer sha256 mismatch for ' + $Target + ': expected=' + $ExpectedSha256 + ' actual=' + $actualSha256) 24 } }", + "function Decode-ToTarget([string]$Target, [string]$Encoded, [Int64]$ExpectedBytes, [string]$ExpectedSha256) { Ensure-Parent $Target; Set-TmpPaths $Target ([guid]::NewGuid().ToString('N')); try { $bytes = [Convert]::FromBase64String(($Encoded -replace '\\s','')) } catch { Fail ('transfer base64 decode failed for ' + $Target + ': ' + $_.Exception.Message) 22 }; [System.IO.File]::WriteAllBytes($script:tmp, $bytes); Verify-Temp $Target $script:tmp $ExpectedBytes $ExpectedSha256; Move-Item -LiteralPath $script:tmp -Destination $Target -Force; $actualSha256 = Get-Sha256 $Target; if ($actualSha256 -ne $ExpectedSha256) { Fail ('transfer final sha256 mismatch for ' + $Target) 25 } }", + "$target = Resolve-UnideskPath $targetArg;", + "switch ($operation) {", + " 'stat' { if (-not (Test-Path -LiteralPath $target -PathType Leaf)) { Fail ('file not found: ' + $target) 1 }; $bytes = ([System.IO.FileInfo]$target).Length; $digest = Get-Sha256 $target; [Console]::Out.WriteLine(([string]$bytes) + ' ' + $digest); break }", + " 'read-b64-block' { $blockIndex = [Int64]$arg1; $blockSize = [Int32]$arg2; if ($blockIndex -lt 0 -or $blockSize -le 0) { Fail 'invalid read block args' 2 }; $fs = [System.IO.File]::Open($target, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite); try { [void]$fs.Seek($blockIndex * $blockSize, [System.IO.SeekOrigin]::Begin); $buffer = New-Object byte[] $blockSize; $read = $fs.Read($buffer, 0, $blockSize); if ($read -gt 0) { [Console]::Out.Write([Convert]::ToBase64String($buffer, 0, $read)) } } finally { $fs.Dispose() }; break }", + " 'write-b64-argv' { Decode-ToTarget $target ([string]::Concat($argvChunks)) ([Int64]$arg1) $arg2; break }", + " 'write-b64-stdin' { Decode-ToTarget $target ([Console]::In.ReadToEnd()) ([Int64]$arg1) $arg2; break }", + " 'write-b64-begin' { Ensure-Parent $target; Set-TmpPaths $target $arg1; [System.IO.File]::WriteAllText($script:tmpB64, '', [System.Text.Encoding]::ASCII); break }", + " 'write-b64-append-stdin' { Set-TmpPaths $target $arg1; $chunk = ([Console]::In.ReadToEnd()) -replace '\\s',''; [System.IO.File]::AppendAllText($script:tmpB64, $chunk, [System.Text.Encoding]::ASCII); break }", + " 'write-b64-commit' { Set-TmpPaths $target $arg1; $encoded = [System.IO.File]::ReadAllText($script:tmpB64, [System.Text.Encoding]::ASCII); Remove-Item -LiteralPath $script:tmpB64 -Force -ErrorAction SilentlyContinue; Decode-ToTarget $target $encoded ([Int64]$arg2) $arg3; break }", + " default { Fail ('unsupported transfer op: ' + $operation) 2 }", + "}", + ].join(" "); +} + +function powerShellSingleQuote(value: string): string { + return `'${value.replace(/'/g, "''")}'`; +} diff --git a/scripts/src/ssh.ts b/scripts/src/ssh.ts index 87c6bda9..793aae1c 100644 --- a/scripts/src/ssh.ts +++ b/scripts/src/ssh.ts @@ -1,6 +1,7 @@ import { spawn } from "node:child_process"; import { type UniDeskConfig, repoRoot } from "./config"; import { isApplyPatchV2HelpArgs, runApplyPatchV2, type ApplyPatchV2Executor } from "./apply-patch-v2"; +import { isSshFileTransferOperation, runSshFileTransferOperation, type SshRemoteCommandExecutor } from "./ssh-file-transfer"; export interface ParsedSshArgs { remoteCommand: string | null; @@ -824,6 +825,9 @@ export function isSshSkillDiscoveryArgs(args: string[]): boolean { export function parseSshArgs(args: string[]): ParsedSshArgs { const subcommand = args[0] ?? ""; + if (isSshFileTransferOperation(args)) { + return { remoteCommand: null, requiresStdin: false, invocationKind: "helper" }; + } if (isSshSkillDiscoveryArgs(args)) { const toolArgs = subcommand === "skill" ? ["skill-discover", ...args.slice(2)] : ["skill-discover", ...args.slice(1)]; return { remoteCommand: shellArgv(toolArgs), requiresStdin: false, invocationKind: "helper", requiredHelpers: ["skill-discover"] }; @@ -981,6 +985,9 @@ function parseWinRouteArgs(route: ParsedSshRoute, args: string[]): ParsedSshArgs if (operation.length === 0) { throw new Error(`ssh ${route.raw} requires a Windows operation, for example: ssh ${route.providerId}:win cmd ver or ssh ${route.providerId}:win skills`); } + if (operation === "upload" || operation === "download") { + return { remoteCommand: null, requiresStdin: false, invocationKind: "helper" }; + } if (operation === "skills" || operation === "skill-discover" || operation === "discover-skills") { return { remoteCommand: buildWindowsPowerShellInvocation(buildWindowsSkillsDiscoveryScript(args.slice(1))), @@ -1103,7 +1110,7 @@ function buildWindowsCmdLauncherScript(cmdLine: string): string { ].join(" "); } -function buildWindowsPowerShellInvocation(script: string): string { +export function buildWindowsPowerShellInvocation(script: string): string { return shellArgv([ windowsPowerShellExePath, "-NoProfile", @@ -1292,6 +1299,9 @@ function parseK3sRouteArgs(route: ParsedSshRoute, args: string[]): ParsedSshArgs function parseK3sControlPlaneOperation(route: ParsedSshRoute, args: string[]): ParsedSshArgs { const operation = args[0] ?? "guard"; + if (operation === "upload" || operation === "download") { + throw new Error(`ssh ${route.providerId}:k3s ${operation} requires a workload route: ssh ${route.providerId}:k3s:: ${operation} ...`); + } if (operation === "apply-patch" || operation === "apply-patch-v1") { throw new Error(`ssh ${route.providerId}:k3s apply-patch requires a workload route: ssh ${route.providerId}:k3s:: apply-patch`); } @@ -1323,6 +1333,9 @@ function parseK3sTargetOperation(route: ParsedSshRoute, args: string[]): ParsedS } const operation = args[0] ?? ""; const operationArgs = args.slice(1); + if (operation === "upload" || operation === "download") { + return { remoteCommand: null, requiresStdin: false, invocationKind: "helper" }; + } if (operation === "apply-patch") { if (isApplyPatchV2HelpArgs(operationArgs)) { return { remoteCommand: null, requiresStdin: false, invocationKind: "helper" }; @@ -1361,8 +1374,8 @@ function buildK3sTargetObjectCommand(action: "get" | "describe", route: ParsedSs return shellArgv(["env", `KUBECONFIG=${nativeK3sKubeconfig}`, "kubectl", action, "-n", route.namespace, normalizeK3sRouteResource(route.resource), ...args]); } -function buildK3sTargetCommand(route: ParsedSshRoute, command: string[]): string { - return buildK3sExecCommand([...k3sRouteTargetArgs(route), "--", ...command]); +function buildK3sTargetCommand(route: ParsedSshRoute, command: string[], options: { stdin?: boolean } = {}): string { + return buildK3sExecCommand([...k3sRouteTargetArgs(route), ...(options.stdin === true ? ["--stdin"] : []), "--", ...command]); } function k3sRouteTargetArgs(route: ParsedSshRoute): string[] { @@ -2072,15 +2085,19 @@ function terminalSize(): { cols: number; rows: number } { }; } -export function remoteCommandForRoute(route: ParsedSshRoute, command: string[]): string { - if (route.plane === "k3s") return buildK3sTargetCommand(route, command); +export function remoteCommandForRoute(route: ParsedSshRoute, command: string[], options: { stdin?: boolean } = {}): string { + if (route.plane === "k3s") return buildK3sTargetCommand(route, command, options); if (route.plane === "win") throw new Error(`ssh apply-patch does not support win routes yet: ${route.raw}`); return shellArgv(command); } async function runSshCaptureCommand(config: UniDeskConfig, invocation: ParsedSshInvocation, command: string[], input?: string): Promise { + const remoteCommand = remoteCommandForRoute(invocation.route, command, { stdin: input !== undefined }); + return await runSshCaptureRemoteCommand(config, invocation, remoteCommand, input); +} + +async function runSshCaptureRemoteCommand(config: UniDeskConfig, invocation: ParsedSshInvocation, remoteCommand: string, input?: string): Promise { const startedAtMs = Date.now(); - const remoteCommand = remoteCommandForRoute(invocation.route, command); const size = terminalSize(); const runtimeTimeoutMs = sshRuntimeTimeoutMs(); const payload = { @@ -2184,6 +2201,15 @@ export async function runSsh(config: UniDeskConfig, providerId: string, args: st const invocation = parseSshInvocation(providerId, args); const parsed = invocation.parsed; const operationName = args[0] ?? ""; + if (isSshFileTransferOperation(args)) { + const executor: SshRemoteCommandExecutor = { + runRemoteCommand: (remoteCommand, input) => runSshCaptureRemoteCommand(config, invocation, remoteCommand, input), + }; + return await runSshFileTransferOperation(invocation, args, executor, { + buildRouteCommand: remoteCommandForRoute, + buildWindowsPowerShellCommand: buildWindowsPowerShellInvocation, + }); + } if (operationName === "apply-patch") { const executor: ApplyPatchV2Executor = { run: (command, input) => runSshCaptureCommand(config, invocation, command, input) }; return await runApplyPatchV2({ diff --git a/scripts/ssh-argv-guidance-contract-test.ts b/scripts/ssh-argv-guidance-contract-test.ts index 61cb785c..954203aa 100644 --- a/scripts/ssh-argv-guidance-contract-test.ts +++ b/scripts/ssh-argv-guidance-contract-test.ts @@ -8,6 +8,7 @@ import { sshHelp } from "./src/help"; import { runApplyPatchV2 } from "./src/apply-patch-v2"; import { providerTriageRecommendedCrossChecks } from "./src/provider-triage"; import { extractRemoteCliOptions, remoteSshFrontendPlanForTest } from "./src/remote"; +import { runSshFileTransferOperation, type SshFileTransferCommandBuilders, type SshRemoteCommandExecutor } from "./src/ssh-file-transfer"; import { formatSshFailureHint, formatSshRuntimeTimeoutHint, @@ -42,6 +43,27 @@ function sha256Hex(value: string): string { return createHash("sha256").update(Buffer.from(value, "utf8")).digest("hex"); } +function sha256BufferHex(value: Buffer): string { + return createHash("sha256").update(value).digest("hex"); +} + +async function captureStdout(fn: () => Promise): Promise<{ exitCode: number; stdout: string }> { + const originalWrite = process.stdout.write; + let stdout = ""; + process.stdout.write = ((chunk: unknown, ...args: unknown[]) => { + stdout += Buffer.isBuffer(chunk) ? chunk.toString("utf8") : String(chunk); + const callback = args.find((arg): arg is () => void => typeof arg === "function"); + if (callback) callback(); + return true; + }) as typeof process.stdout.write; + try { + const exitCode = await fn(); + return { exitCode, stdout }; + } finally { + process.stdout.write = originalWrite; + } +} + function decodeWinEncodedCommand(remoteCommand: string | null | undefined): string { const text = String(remoteCommand ?? ""); const match = /'-EncodedCommand' '([^']+)'/u.exec(text); @@ -243,6 +265,78 @@ async function applyPatchV2Fixture(patch: string, files: Record) return { stdout: result.stdout, files: result.files, commands: result.commands }; } +function fileTransferFixture(initial: Record = {}): { + state: Map; + commands: Array<{ operation: string; stdin: boolean }>; + executor: SshRemoteCommandExecutor; + builders: SshFileTransferCommandBuilders; +} { + const state = new Map(Object.entries(initial)); + const pending = new Map(); + const commands: Array<{ operation: string; stdin: boolean }> = []; + const builders: SshFileTransferCommandBuilders = { + buildRouteCommand(route, command, options) { + return JSON.stringify({ route: route.raw, command, stdin: options?.stdin === true }); + }, + buildWindowsPowerShellCommand(script) { + return JSON.stringify({ route: "win", command: ["powershell", script], stdin: false }); + }, + }; + const executor: SshRemoteCommandExecutor = { + async runRemoteCommand(remoteCommand, input) { + const payload = JSON.parse(remoteCommand) as { command: string[]; stdin?: boolean }; + const command = payload.command; + const operation = command[4] ?? ""; + const target = command[5] ?? ""; + commands.push({ operation, stdin: payload.stdin === true }); + if (operation === "stat") { + const content = state.get(target); + if (content === undefined) return { exitCode: 1, stdout: "", stderr: "missing" }; + return { exitCode: 0, stdout: `${content.length} ${sha256BufferHex(content)}\n`, stderr: "" }; + } + if (operation === "read-b64-block") { + const content = state.get(target); + if (content === undefined) return { exitCode: 1, stdout: "", stderr: "missing" }; + const blockIndex = Number(command[6] ?? "-1"); + const blockSize = Number(command[7] ?? "-1"); + const start = blockIndex * blockSize; + return { exitCode: 0, stdout: content.subarray(start, start + blockSize).toString("base64"), stderr: "" }; + } + if (operation === "write-b64-argv" || operation === "write-b64-stdin") { + const expectedBytes = Number(command[6] ?? "-1"); + const expectedSha256 = command[7] ?? ""; + const encoded = operation === "write-b64-argv" ? command.slice(8).join("") : String(input ?? ""); + const content = Buffer.from(encoded, "base64"); + if (content.length !== expectedBytes || sha256BufferHex(content) !== expectedSha256) return { exitCode: 23, stdout: "", stderr: "integrity mismatch" }; + state.set(target, content); + return { exitCode: 0, stdout: "", stderr: "" }; + } + if (operation === "write-b64-begin") { + pending.set(`${target}\0${command[6] ?? ""}`, ""); + return { exitCode: 0, stdout: "", stderr: "" }; + } + if (operation === "write-b64-append-stdin") { + const key = `${target}\0${command[6] ?? ""}`; + if (!pending.has(key)) return { exitCode: 2, stdout: "", stderr: "missing pending write" }; + pending.set(key, `${pending.get(key) ?? ""}${String(input ?? "")}`); + return { exitCode: 0, stdout: "", stderr: "" }; + } + if (operation === "write-b64-commit") { + const key = `${target}\0${command[6] ?? ""}`; + const expectedBytes = Number(command[7] ?? "-1"); + const expectedSha256 = command[8] ?? ""; + const content = Buffer.from(pending.get(key) ?? "", "base64"); + if (content.length !== expectedBytes || sha256BufferHex(content) !== expectedSha256) return { exitCode: 23, stdout: "", stderr: "integrity mismatch" }; + state.set(target, content); + pending.delete(key); + return { exitCode: 0, stdout: "", stderr: "" }; + } + return { exitCode: 2, stdout: "", stderr: `unsupported op ${operation}` }; + }, + }; + return { state, commands, executor, builders }; +} + export async function runSshArgvGuidanceContract(): Promise { const argv = parseSshArgs(["argv", "true"]); assertCondition(argv.invocationKind === "argv", "argv subcommand must be classified as argv", argv); @@ -285,6 +379,31 @@ export async function runSshArgvGuidanceContract(): Promise { const winSkillsScript = decodeWinEncodedCommand(winSkills.parsed.remoteCommand); assertCondition(winSkillsScript.includes(".agents\\skills") && winSkillsScript.includes(".codex\\skills") && winSkillsScript.includes("$limit = 20") && winSkillsScript.includes("ConvertTo-Json"), "win skills must discover Windows user skill roots as JSON", { winSkills, winSkillsScript }); + const hostUploadParse = parseSshInvocation("D601", ["upload", "/tmp/local.bin", "/tmp/remote.bin"]); + assertCondition(hostUploadParse.parsed.remoteCommand === null && hostUploadParse.parsed.invocationKind === "helper", "host upload must be a structured local operation, not an ssh-like command string", hostUploadParse); + const winDownloadParse = parseSshInvocation("D601:win", ["download", String.raw`F:\Work\hwlab\.tmp\tool.mjs`, "/tmp/tool.mjs"]); + assertCondition(winDownloadParse.route.plane === "win" && winDownloadParse.parsed.remoteCommand === null, "win download must be handled by the file-transfer operation module", winDownloadParse); + const podUploadParse = parseSshInvocation("D601:k3s:unidesk:code-queue/root/unidesk", ["upload", "/tmp/local.bin", "/root/unidesk/.tmp/remote.bin"]); + assertCondition(podUploadParse.route.plane === "k3s" && podUploadParse.parsed.remoteCommand === null, "pod upload must keep k3s route as location-only and defer transfer execution to the operation module", podUploadParse); + + const transferRoot = mkdtempSync(path.join(os.tmpdir(), "unidesk-transfer-contract-")); + try { + const localSource = path.join(transferRoot, "local-source.bin"); + const localDownload = path.join(transferRoot, "downloaded", "local-copy.bin"); + const payload = Buffer.from("hello 中文\n\0binary tail", "utf8"); + writeFileSync(localSource, payload); + const transfer = fileTransferFixture(); + const uploadResult = await captureStdout(() => runSshFileTransferOperation(hostUploadParse, ["upload", localSource, "/tmp/remote.bin"], transfer.executor, transfer.builders)); + assertCondition(uploadResult.exitCode === 0 && JSON.parse(uploadResult.stdout).verified === true, "upload should report verified JSON success", uploadResult); + assertCondition(transfer.state.get("/tmp/remote.bin")?.equals(payload), "upload must preserve binary and UTF-8 bytes in the mock remote file", transfer.commands); + const downloadResult = await captureStdout(() => runSshFileTransferOperation(parseSshInvocation("D601", ["download", "/tmp/remote.bin", localDownload]), ["download", "/tmp/remote.bin", localDownload], transfer.executor, transfer.builders)); + assertCondition(downloadResult.exitCode === 0 && JSON.parse(downloadResult.stdout).sha256 === sha256BufferHex(payload), "download should report the verified sha256", downloadResult); + assertCondition(readFileSync(localDownload).equals(payload), "download must preserve binary and UTF-8 bytes locally", { commands: transfer.commands }); + assertCondition(transfer.commands.some((item) => item.operation === "stat") && transfer.commands.some((item) => item.operation === "read-b64-block"), "file transfer should use stat plus chunked verified reads", transfer.commands); + } finally { + rmSync(transferRoot, { recursive: true, force: true }); + } + assertThrows( () => parseSshInvocation("D601:win32", ["cmd", "ver"]), /use D601:win/u, @@ -800,6 +919,7 @@ export async function runSshArgvGuidanceContract(): Promise { assertCondition(helpText.includes("ssh D601:k3s:hwlab-dev:hwlab-cloud-api/app pwd"), "ssh help must document k3s pod workspace route", helpText); assertCondition(helpText.includes("ssh D601:k3s script <<'SCRIPT'"), "ssh help must document k3s control-plane script operation", helpText); assertCondition(helpText.includes("ssh D601:k3s:hwlab-dev:hwlab-cloud-api/app apply-patch <<'PATCH'"), "ssh help must document k3s pod apply-patch operation", helpText); + assertCondition(helpText.includes("ssh upload ") && helpText.includes("ssh download "), "ssh help must document verified file transfer operations", helpText); assertCondition(helpText.includes("ssh D601:k3s:unidesk:code-queue/root/unidesk exec --stdin -- tar -xf - -C /root/unidesk"), "ssh help must document one-step stdin file streaming into pod exec", helpText); assertCondition(helpText.includes("apply-patch-v1 [--allow-loose]") && helpText.includes("low-context update hunks"), "ssh help must document apply-patch-v1 loose-context guard", helpText); assertCondition(helpText.includes("ssh D601:k3s:hwlab-dev:hwlab-cloud-api script <<'SCRIPT'"), "ssh help must document k3s script operation", helpText); @@ -848,6 +968,11 @@ export async function runSshArgvGuidanceContract(): Promise { assertCondition(!remoteSource.includes("remote frontend transport does not stream stdin"), "remote frontend ssh must not reject stdin-backed helpers", remoteSource); assertCondition(!remoteSource.includes("source: \"cli-remote-ssh\""), "remote frontend ssh must not use host.ssh dispatch task polling", remoteSource); + const sshSource = readFileSync(new URL("./src/ssh.ts", import.meta.url), "utf8"); + const sshFileTransferSource = readFileSync(new URL("./src/ssh-file-transfer.ts", import.meta.url), "utf8"); + assertCondition(sshFileTransferSource.includes("runSshFileTransferOperation") && sshFileTransferSource.includes("write-b64-commit"), "file transfer operation implementation must live in the dedicated ssh-file-transfer module", {}); + assertCondition(!sshSource.includes("type SshFileTransferOperation") && !sshSource.includes("posixFileTransferScript"), "ssh.ts must not accumulate the full upload/download implementation", {}); + const frontendSource = readFileSync(new URL("../src/components/frontend/src/index.ts", import.meta.url), "utf8"); assertCondition(frontendSource.includes('url.pathname === "/ws/ssh"') && frontendSource.includes("proxySshWebSocket"), "frontend must expose an authenticated /ws/ssh proxy", frontendSource); assertCondition(frontendSource.includes("coreSshWebSocketUrl") && frontendSource.includes('url.searchParams.set("token"'), "frontend /ws/ssh proxy must connect to backend-core ssh bridge with the provider token", frontendSource); @@ -870,6 +995,7 @@ export async function runSshArgvGuidanceContract(): Promise { "script -- single-string runs as a remote shell one-liner while multi-token form keeps dash-prefixed argv", "pod apply-patch operation uses the v2 local engine and apply-patch-v1 injects the legacy helper", "pod exec --stdin streams arbitrary local stdin through workload routes without shell wrapping", + "upload/download file transfer operations use a dedicated module with byte-count and sha256 verification", "apply-patch-v1 uses one sh helper for host and pod paths and rejects low-context hunks unless --allow-loose is explicit", "legacy operation-in-route forms are rejected in any k3s route segment with canonical route-plus-operation guidance", "post-provider k3s shorthand is rejected so location and operation stay separated",