fix: support windows route downloads
Pipelines as Code CI / hwlab-web-probe-sentinel-nc01- Success

This commit is contained in:
Codex
2026-07-09 12:07:26 +02:00
parent e02d820678
commit 258340f84d
4 changed files with 112 additions and 7 deletions
@@ -7,4 +7,6 @@ Prefer direct, bounded operations:
- `logs`: tail bounded logs, not full dumps.
- `py`, `upload`, `download`: one-off support operations when patch is not appropriate.
`download` returns verified bytes/SHA-256 evidence. On Windows routes, drive-letter paths are automatically mapped through the same provider's WSL `/mnt/<drive>` host route for binary streaming; for example `trans D518:win download 'D:\tmp\image.png' /tmp/image.png` does not require manually rewriting the source path to `/mnt/d/tmp/image.png`. Relative Windows downloads use the `win/<drive>/<path>` route workspace, such as `trans D518:win/d/tmp download image.png /tmp/image.png`. UNC paths still need to be copied to a drive-letter path first.
Ordinary trans/SSH work must fit the short connection budget. Long build, CI/CD, trace, probe or rollout work must use submit-and-poll through a controlled job/status surface.
+2
View File
@@ -30,6 +30,8 @@ trans <PROVIDER_ID>:win skills --limit 20
`trans <PROVIDER_ID>:win ps` 会把用户脚本写入 Windows 临时 `.ps1`,并在脚本前注入 UTF-8 与 JSON 安全 prelude。该 prelude 只在本次脚本作用域内覆盖 `ConvertTo-Json`,先把常见 PowerShell ETS 扩展属性剥离为普通 JSON 值,再调用 `Microsoft.PowerShell.Utility\ConvertTo-Json` 输出;这样 `Get-Content -Tail` 得到的字符串放入 `[pscustomobject]` 后不会把 `PSPath``PSDrive``PSProvider` 和 provider 类型对象图一起序列化。若确实需要原始 PowerShell cmdlet 行为,可在脚本中显式调用 `Microsoft.PowerShell.Utility\ConvertTo-Json`
`trans <PROVIDER_ID>:win download <windows-path> <local-path>` 支持 drive-letter Windows 路径的二进制 verified transfer。CLI 会先通过 Windows file-transfer helper 对原始 Windows 路径做 bytes/SHA-256 stat,再把同一文件自动映射为该 provider 的 WSL host 路径 `/mnt/<drive>/...``host.ssh.tcp-pool` stdout stream 下载,最终返回 `verification.verified=true``verification.match.bytes=true``verification.match.sha256=true`。例如 `trans D518:win download 'D:\tmp\image.png' /tmp/image.png` 不需要人工改写成 `/mnt/d/tmp/image.png``trans D518:win/d/tmp download image.png /tmp/image.png` 也会按 route workspace 解析。UNC 路径和没有 drive-letter 的路径无法自动映射,需先复制到可由 WSL `/mnt/<drive>` 访问的位置。
## Windows Long-Lived Process Detach
`trans <PROVIDER_ID>:win cmd ...``trans <PROVIDER_ID>:win ps ...` 适合短命令、只读探测和有界 skill 调用,不适合直接启动 Windows 长驻进程。Windows `cmd start``cmd /c ... &`、PowerShell `Start-Process -PassThru` 或带 stdout/stderr 重定向的子进程,仍可能被 provider-gateway/SSH broker 按子进程树或继承句柄等待;结果是远端进程其实已启动,但 `trans` 会持续占用 provider session,后续 D601/G14 高频调用被 provider session lock 串行排队。
+1 -1
View File
@@ -234,7 +234,7 @@ export function sshHelp(): unknown {
"sh and bash helper modes inject a tiny POSIX-compatible printf wrapper before user shell text, so portable printf headings such as `printf \"--- section ---\\n\"` work consistently under dash/sh and bash. Direct argv commands are unchanged.",
"For arbitrary stdin streams into a workload command, use a workload route plus `exec --stdin -- <command> ...`; this keeps the route as location-only and avoids heredoc/base64/tar shell wrapping.",
"`apply-patch` is the default remote text patch entry and uses the v2 local line-based patch engine with remote read/write operations, including Windows routes such as `D601:win/c/test`, so long Unicode/Chinese lines and pure insertion hunks avoid the legacy remote shell hunk parser. Plain multi-file Update File patches on POSIX host/k3s and Windows workspace routes use bulk read/write operations to avoid per-file SSH round trips. Its stdout follows Codex apply_patch text output rather than UniDesk JSON output; stderr keeps Codex-style failure text and appends one `UNIDESK_APPLY_PATCH_TIMING` JSON summary with durationMs, patchBytes, fileCount, hunkCount, changedCount, remoteOperationCount, remoteOperationCounts and remoteElapsedMs so slow patch runs can be attributed without changing success stdout.",
"`upload` and `download` are the default whole-file transfer entries for non-text and generated files. They write through remote temp files, verify byte count and SHA-256 on both sides, and return `verification.automatic=true`, `verification.verified=true`, and `verification.match.{bytes,sha256}=true`; this JSON is the transfer integrity proof, so callers do not need a separate manual `sha256sum` check. Downloads stream over `host.ssh.tcp-pool`, emit progress JSON, and may receive a caller-supplied `--inactivity-timeout-ms` from async artifact/deploy jobs so active large transfers are not killed by the generic short-command budget.",
"`upload` and `download` are the default whole-file transfer entries for non-text and generated files. They write through remote temp files, verify byte count and SHA-256 on both sides, and return `verification.automatic=true`, `verification.verified=true`, and `verification.match.{bytes,sha256}=true`; this JSON is the transfer integrity proof, so callers do not need a separate manual `sha256sum` check. Downloads stream over `host.ssh.tcp-pool`, emit progress JSON, and may receive a caller-supplied `--inactivity-timeout-ms` from async artifact/deploy jobs so active large transfers are not killed by the generic short-command budget. Windows route downloads such as `trans D601:win download C:\\Temp\\tool.mjs ./tool.mjs` automatically map drive-letter paths to the same provider's WSL `/mnt/<drive>` host route and still return verified bytes/SHA-256 evidence.",
"`apply-patch-v1` is the only legacy fallback entry: it rejects low-context update hunks by default, reports the matched file:line for each hunk on stderr, and only accepts --allow-loose when the caller has manually reviewed an intentionally ambiguous insertion.",
"`sh` inherits provider proxy variables such as HTTP_PROXY/HTTPS_PROXY/ALL_PROXY/NO_PROXY and runs target `/bin/sh`; use `bash` only for Bash syntax such as `pipefail`, arrays, substring expansion, or `[[ ... ]]`, not as a proxy workaround.",
"Route syntax is `{provider}:{plane}[:{scope...}] {operation} [operation-args...]`: the first argv token locates a distributed target only, and every following token belongs to the operation parser. Host workspace routes use `<provider>:/absolute/workspace`; WSL providers can use `<provider>:win ps` for Windows PowerShell and `<provider>:win cmd` for Windows cmd.exe, with `<provider>:win/c/test ...` mapping the Windows cwd to `C:\\test`; native k3s providers such as D601 and G14 use `<provider>:k3s` for the control plane and `<provider>:k3s:<namespace>:<workload>[:<container>]` for a workload/container. In k3s routes, `:` is the distributed route separator; `/...` is only an in-container filesystem cwd and never selects a container. Prefer operation `--cwd /path` when a container is also specified.",
+107 -6
View File
@@ -60,6 +60,8 @@ interface SshFileTransferDownloadResult {
local: SshFileTransferStat;
strategy: "tcp-pool-stdout-stream";
transport: "host.ssh.tcp-pool";
sourceRoute: string;
sourcePath: string;
chunks: number;
elapsedMs: number;
throughputBytesPerSecond: number;
@@ -76,6 +78,8 @@ export interface SshVerifiedDownloadResult {
transfer: {
strategy: SshFileTransferDownloadResult["strategy"];
transport: SshFileTransferDownloadResult["transport"];
sourceRoute: string;
sourcePath: string;
chunks: number;
elapsedMs: number;
throughputBytesPerSecond: number;
@@ -174,6 +178,8 @@ export async function downloadSshFileVerified(
transfer: {
strategy: read.strategy,
transport: read.transport,
sourceRoute: read.sourceRoute,
sourcePath: read.sourcePath,
chunks: read.chunks,
elapsedMs: read.elapsedMs,
throughputBytesPerSecond: read.throughputBytesPerSecond,
@@ -293,19 +299,24 @@ async function downloadRemoteFileVerified(
blocker: "streamRemoteCommand unavailable",
});
}
const streamSource = invocation.route.plane === "win"
? windowsDownloadStreamSource(invocation.route, remotePath)
: { route: invocation.route, path: remotePath };
if (invocation.route.plane === "win") {
throw new SshFileTransferError("ssh download requires tcp-pool stdout streaming; win routes are not supported by the safe binary stream path", {
process.stderr.write(`${JSON.stringify({
event: "unidesk.ssh.download.win-route-mapped",
at: new Date().toISOString(),
route: invocation.route.raw,
providerId: invocation.providerId,
remotePath,
localPath,
requiredTransport: "host.ssh.tcp-pool",
blocker: "win route has no verified binary stdout stream path",
});
sourceRoute: streamSource.route.raw,
sourcePath: streamSource.path,
transport: "host.ssh.tcp-pool",
})}\n`);
}
const remote = await statRemoteFile(invocation, executor, builders, remotePath);
await mkdir(path.dirname(localPath), { recursive: true });
const remoteCommand = buildStreamDownloadRemoteCommand(invocation.route, builders, remotePath);
const remoteCommand = buildStreamDownloadRemoteCommand(streamSource.route, builders, streamSource.path);
const startedAtMs = Date.now();
const hash = createHash("sha256");
const output = createWriteStream(localPath, { flags: "w", mode: 0o600 });
@@ -343,6 +354,8 @@ async function downloadRemoteFileVerified(
local,
strategy: "tcp-pool-stdout-stream",
transport: "host.ssh.tcp-pool",
sourceRoute: streamSource.route.raw,
sourcePath: streamSource.path,
chunks: chunkCount,
elapsedMs,
throughputBytesPerSecond: Math.round((actualBytes * 1000) / elapsedMs),
@@ -355,6 +368,94 @@ async function downloadRemoteFileVerified(
}
}
function windowsDownloadStreamSource(route: ParsedSshRoute, remotePath: string): { route: ParsedSshRoute; path: string } {
const mappedPath = windowsPathToWslMountPath(remotePath, route.workspace);
return {
route: {
providerId: route.providerId,
plane: "host",
entry: null,
namespace: null,
resource: null,
container: null,
workspace: null,
raw: `${route.providerId}:${mappedPath}`,
},
path: mappedPath,
};
}
function windowsPathToWslMountPath(rawPath: string, basePath: string | null): string {
const absolute = resolveWindowsTransferPath(rawPath, basePath);
const match = absolute.match(/^([A-Za-z]):\\(.*)$/u);
if (match === null) {
throw new SshFileTransferError("ssh win download can only auto-map drive-letter paths to WSL /mnt/<drive>", {
remotePath: rawPath,
basePath,
supported: "D:\\path\\file or route workspace D518:win/d/path plus relative file",
unsupported: "UNC paths and paths without a drive letter",
fallback: "copy the file to a drive-mounted path, then retry the same download command",
});
}
const drive = match[1]!.toLowerCase();
const rest = match[2] ?? "";
const segments = normalizeWindowsPathSegments(rest.split(/[\\/]+/u));
return `/mnt/${drive}${segments.length === 0 ? "" : `/${segments.join("/")}`}`;
}
function resolveWindowsTransferPath(rawPath: string, basePath: string | null): string {
const normalizedRaw = stripWindowsLongPathPrefix(rawPath.trim()).replace(/\//gu, "\\");
if (normalizedRaw.length === 0) {
throw new SshFileTransferError("ssh win download requires a non-empty remote path");
}
if (/^\\\\/u.test(normalizedRaw)) {
throw new SshFileTransferError("ssh win download cannot auto-map UNC paths to WSL /mnt/<drive>", {
remotePath: rawPath,
fallback: "copy the file to a drive-letter path such as D:\\tmp, then retry download",
});
}
if (/^[A-Za-z]:/u.test(normalizedRaw)) return normalizeWindowsAbsolutePath(normalizedRaw);
const base = basePath === null ? "" : stripWindowsLongPathPrefix(basePath.trim()).replace(/\//gu, "\\");
const baseMatch = base.match(/^([A-Za-z]):\\?(.*)$/u);
if (baseMatch === null) {
throw new SshFileTransferError("ssh win download with a relative path requires a drive-letter route workspace", {
remotePath: rawPath,
routeWorkspace: basePath,
example: "trans D518:win/d/tmp download image.png /tmp/image.png",
});
}
const drive = baseMatch[1]!;
const baseRest = baseMatch[2] ?? "";
return normalizeWindowsAbsolutePath(`${drive}:\\${[baseRest, normalizedRaw].filter((part) => part.length > 0).join("\\")}`);
}
function stripWindowsLongPathPrefix(value: string): string {
if (value.startsWith("\\\\?\\")) return value.slice(4);
return value;
}
function normalizeWindowsAbsolutePath(value: string): string {
const match = value.match(/^([A-Za-z]):\\?(.*)$/u);
if (match === null) return value;
const drive = match[1]!.toUpperCase();
const segments = normalizeWindowsPathSegments((match[2] ?? "").split(/[\\/]+/u));
return `${drive}:\\${segments.join("\\")}`;
}
function normalizeWindowsPathSegments(segments: string[]): string[] {
const normalized: string[] = [];
for (const segment of segments) {
if (segment.length === 0 || segment === ".") continue;
if (segment === "..") {
if (normalized.length === 0) throw new SshFileTransferError("ssh win download path escapes the drive root");
normalized.pop();
continue;
}
normalized.push(segment);
}
return normalized;
}
function emitDownloadProgress(
invocation: ParsedSshInvocation,
remotePath: string,