diff --git a/.agents/skills/unidesk-trans/SKILL.md b/.agents/skills/unidesk-trans/SKILL.md index fe8e21a9..eaa4f83f 100644 --- a/.agents/skills/unidesk-trans/SKILL.md +++ b/.agents/skills/unidesk-trans/SKILL.md @@ -29,15 +29,28 @@ Host workspace、k3s、Windows、GitHub issue/PR route 见 [references/routes.md - 远端文本修改优先 `trans apply-patch`；不要 download/upload/sed 拼临时 diff 代替 patch。 - Host/WSL 与 Windows route 的 `apply-patch` 优先走 fs adapter bulk update path；不要为了规避旧的 `read-b64-block` / `write-b64-argv` 慢路径改用临时脚本写文件。 -- `apply-patch v2` 的 localized replacement 必须保留纯 CRLF 文件的 CRLF 风格；original/final integrity mismatch 会重新只读采集当前文件并输出 adapter、stage、expected/current bytes、SHA-256、newline 和 encoding 摘要，不披露文件正文，也不回退 `apply-patch-v1`。 +- `apply-patch v2` 字节与完整性边界： + - localized replacement 必须保留纯 CRLF 和 `legacy-single-byte` 文件的原始字节风格。 + - legacy 文件只允许可保持的单字节 replacement。 + - original/final integrity mismatch 会重新只读采集当前文件并输出 adapter、stage、expected/current bytes、SHA-256、newline 和 encoding 摘要。 + - 诊断不披露文件正文，也不回退 `apply-patch-v1`。 - 当前 HWLAB node/lane source workspace 的正式预检/同步入口是 `bun scripts/cli.ts hwlab nodes control-plane source-workspace sync --node --lane --confirm` 和 `source-workspace status`；不要把裸 `trans : git fetch ...` 当成当前 HWLAB node/lane workspace 预检/修复入口。 - `sh`/`bash` 必须显式声明 shell；单进程命令优先 direct argv 或已知 operation。 - 普通 trans/ssh 短连接硬预算 60s；长 CI/CD、trace、logs、build、硬件流程必须 submit-and-poll。 - `/mnt/` WSL host workspace 由 root 透传执行时，CLI 自动把 workspace owner uid 桥接到进程级 `SUDO_UID`，让 Git 信任 Windows 挂载目录而不写全局 `safe.directory`；非 Git 命令和普通 Linux workspace 不改变所有权信任。 - Windows route 的 `win` 是 route plane，operation 直接写 `ps`、`cmd`、`git` 或只读 fs 操作 `pwd|ls|cat|head|tail|stat|wc|rg`；不要写成 `trans D601:win/... win ps`，也不要把 POSIX shell 当 Windows shell。 -- Windows `rg` 调用 Windows PATH 中的原生 `rg.exe`，支持 `--files`、`-g|--glob`、`--max-count`、`--max-files` 和 `--timeout-ms`；stderr 固定输出 `UNIDESK_WINDOWS_RG_SUMMARY`，其中包含 `matched`、`fileCount`、`elapsedMs` 和 `timeout`，默认预算来自 `config/unidesk-cli.yaml#trans.windowsFs.rg`。超时额外输出 `UNIDESK_WINDOWS_RG_HINT code=timeout exitCode=124 action=narrow-scope-or-add-glob`；缺少 `rg.exe` 时明确返回 127 和安装/PowerShell 替代提示。 +- Windows `rg`： + - 调用 Windows PATH 中的原生 `rg.exe`。 + - 支持一个或多个 `-e|--regexp`、`--files`、`-g|--glob`、`--max-count`、`--max-files` 和 `--timeout-ms`。 + - stderr 固定输出 `UNIDESK_WINDOWS_RG_SUMMARY`，包含 `matched`、`fileCount`、`elapsedMs`、`timeout` 和 `encoding`。 + - 默认预算来自 `config/unidesk-cli.yaml#trans.windowsFs.rg`。 + - 超时输出 `UNIDESK_WINDOWS_RG_HINT code=timeout exitCode=124 action=narrow-scope-or-add-glob`。 + - 缺少 `rg.exe` 时返回 127 和安装或 PowerShell 替代提示。 - Windows `wc` 支持 `-l|--lines`、`-w|--words`、`-m|--chars` 和 `-c|--bytes`；`skills` 支持 `--name <精确名称>` 与 `--filter <文本>`，单个 skill 查询不应先拉取全量列表。 -- Windows `head` / `tail` 支持 `-n N`、`-nN`、`--lines=N` 和 GNU 简写 `-N`；PowerShell prelude 将 `Get-Content` 默认编码固定为 UTF-8，读取 UTF-8 Markdown 时应与 `cat` helper 保持一致。 +- Windows `head` / `tail`： + - 支持 `-n N`、`-nN`、`--lines=N` 和 GNU 简写 `-N`。 + - 两者按行流式裁剪，不能在裁剪前因整文件超过通用读取门禁而失败。 + - PowerShell prelude 将 `Get-Content` 默认编码固定为 UTF-8，读取 UTF-8 Markdown 时应与 `cat` helper 保持一致。 - Windows `ps ''` 的单个参数按 PowerShell 源码执行；`ps ` 的多个参数按 argv 边界执行，Windows native application 通过显式命令行引用保留 `python -c` 代码、路径、encoding 和字典 key 中的嵌套引号。 - `trans` / `tran` 是 PATH wrapper，不是 `bun scripts/cli.ts` 的根子命令；误写 `bun scripts/cli.ts trans ...` 时应读取结构化 `trans-root-cli-entrypoint-moved` 错误，并改用 `trans [args...]`。 - Direct argv 会保留调用端已经形成的每个 argv token，包括空格和中文；需要 shell 语法时仍显式使用 `sh`、`bash`、Windows `cmd` 或 `ps`。 diff --git a/docs/reference/cli.md b/docs/reference/cli.md index b601f3b0..0a60dfb9 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -443,7 +443,21 @@ PS `:win skills [--scope agents|codex|all] [--limit N] [--name <精确名称>|--filter <文本>]` 是 Windows 用户 skill 发现入口，默认只读取当前 Windows 用户的 `%USERPROFILE%\.agents\skills`，输出 JSON 中包含 `roots`、`counts`、`query` 和每个 skill 的 `name`、`path`、`skillFile`、`description`。查单个 skill 时优先 `--name`，模糊查找时使用 `--filter`，避免全量结果触发 stdout dump；需要同时检查 `%USERPROFILE%\.codex\skills` 时显式加 `--scope all`。 -`:win// rg` 是有界 Windows 文件搜索入口，调用 Windows PATH 中的原生 `rg.exe`，支持 `--files`、`-g|--glob`、`--max-count`、`--max-files` 和 `--timeout-ms`。每次搜索都在 stderr 输出 `UNIDESK_WINDOWS_RG_SUMMARY`，稳定披露 `matched`、`fileCount`、`elapsedMs`、`timeout` 和截断状态；默认预算由 `config/unidesk-cli.yaml#trans.windowsFs.rg` 唯一控制。超时返回 124，无匹配返回 1，缺少 `rg.exe` 返回 127 和可执行替代提示，均不会以空输出伪装成功。Windows `wc` 同时支持 `-l|--lines`、`-w|--words`、`-m|--chars` 与 `-c|--bytes`。 +- Windows `rg`： + - `:win// rg` 调用 Windows PATH 中的原生 `rg.exe`。 + - 支持一个或多个 `-e|--regexp`，以及 `--files`、`-g|--glob`、`--max-count`、`--max-files` 和 `--timeout-ms`。 + - 搜索强制输出文件名，`UNIDESK_WINDOWS_RG_SUMMARY` 披露 `matched`、`fileCount`、`elapsedMs`、`timeout`、`encoding` 和截断状态。 + - legacy 或无效 UTF-8 输出必须标记 `encoding=legacy-or-invalid-utf8`，不能静默当作 UTF-8 正常文本。 + - 默认预算由 `config/unidesk-cli.yaml#trans.windowsFs.rg` 唯一控制。 + - 超时返回 124，无匹配返回 1，缺少 `rg.exe` 返回 127 和可执行替代提示。 +- Windows 有界读取： + - `wc` 支持 `-l|--lines`、`-w|--words`、`-m|--chars` 与 `-c|--bytes`。 + - `head` 和 `tail` 在行裁剪阶段使用流式读取，不得先以整文件大小门禁拒绝已经有界的 `-n N` 请求。 +- `apply-patch v2` legacy 文本： + - bulk read、plan 和 apply 必须共享同一原始字节 fingerprint。 + - 非 UTF-8 单字节文本标记为 `legacy-single-byte`。 + - ASCII localized replacement 必须保持原 legacy 字节和换行风格。 + - legacy replacement 包含无法用单字节保持的新字符时必须显式拒绝，不能转码整个文件。 `D601:k3s` 或 `G14:k3s` 定位到对应 provider 的原生 k3s 控制面；`:k3s::[:container]` 定位到 namespace 下的一个默认 deployment workload；若目标是具体 Pod，标准 workload 段写成 `pod:`，例如 `D601:k3s:hwlab-v03:pod:hwlab-cloud-api-abc:hwlab-cloud-api`。`pod/` 不是合法 route 写法，因为 `:` 是分布式路由分隔符，`/` 只表示目标容器里的文件系统 cwd；如果调用者写成 `pod//`，CLI 必须在连接运行面前报错并提示改用 `pod::` 或 `--container `。若目标是 Deployment，也可以显式写 `deployment:` 或简写 ``；容器选择写 `:` 或 operation 参数 `--container `。pod 内 cwd 推荐用 operation 参数 `--cwd /path`，例如 `D601:k3s:hwlab-dev:hwlab-cloud-api:hwlab-cloud-api exec --cwd /app -- pwd`。`kubectl`、`logs`、`sh`、`bash`、`apply-patch`、旧 `apply-patch-v1` fallback、`exec` 和普通容器命令都是 route 后面的 operation，这样路由子模块和操作子模块可以独立扩展。 diff --git a/scripts/src/apply-patch-v2.test.ts b/scripts/src/apply-patch-v2.test.ts index bcfe2279..68bc3cab 100644 --- a/scripts/src/apply-patch-v2.test.ts +++ b/scripts/src/apply-patch-v2.test.ts @@ -167,6 +167,50 @@ describe("apply-patch v2 fs bulk update path", () => { } }); + test("preserves legacy single-byte content during ASCII localized replacement", async () => { + const original = Buffer.concat([ + Buffer.from("static int old_value = 1;\r\n// legacy ", "ascii"), + Buffer.from([0xca]), + Buffer.from(" comment\r\n", "ascii"), + ]); + let captured: ApplyPatchV2BulkReplacementWritePlan | undefined; + const fs: ApplyPatchV2FileSystem = { + async stat() { throw new Error("stat should not be used"); }, + async readBlock() { throw new Error("readBlock should not be used"); }, + async writeFile() { throw new Error("writeFile should not be used"); }, + async deleteFile() { throw new Error("deleteFile should not be used"); }, + async readFiles(paths) { + return new Map(paths.map((path) => [path, { + content: original.toString("latin1"), + bytes: original, + encodingStyle: "legacy-single-byte" as const, + }])); + }, + async applyReplacementsBulk(_paths, plans) { + captured = plans.get("src/legacy.c"); + }, + }; + const exitCode = await runApplyPatchV2({ + executor: { fs }, + stdin: Readable.from([[ + "*** Begin Patch", + "*** Update File: src/legacy.c", + "@@", + "-static int old_value = 1;", + "+static int new_value = 2;", + "*** End Patch", + ].join("\n")]), + stdout: new CaptureWritable(), + stderr: new CaptureWritable(), + }); + + expect(exitCode).toBe(0); + expect(captured?.encodingStyle).toBe("legacy-single-byte"); + expect(captured?.newlineStyle).toBe("crlf"); + expect(captured?.originalBytes).toBe(original.length); + expect(captured?.originalSha256).toBe(createHash("sha256").update(original).digest("hex")); + }); + test("bulk read failure falls back to block reads but still writes through replacement bulk", async () => { const files = new Map([ ["a.md", "first\nold-a\nlast\n"], diff --git a/scripts/src/apply-patch-v2.ts b/scripts/src/apply-patch-v2.ts index c2411c42..26b03a41 100644 --- a/scripts/src/apply-patch-v2.ts +++ b/scripts/src/apply-patch-v2.ts @@ -63,10 +63,16 @@ export interface ApplyPatchV2FileSystem { readBlock(path: string, blockIndex: number, blockBytes: number): Promise; writeFile(path: string, content: Buffer): Promise; deleteFile(path: string): Promise; - readFiles?(paths: string[]): Promise>; + readFiles?(paths: string[]): Promise>; applyReplacementsBulk?(paths: Iterable, plans: Map): Promise; } +export interface ApplyPatchV2ReadFile { + content: string; + bytes: Buffer; + encodingStyle: "utf8" | "utf8-bom" | "legacy-single-byte"; +} + export interface ApplyPatchV2FileStat { bytes: number; sha256: string; @@ -114,7 +120,7 @@ export class ApplyPatchV2Error extends Error { } } -type PlannedFileState = { exists: true; content: string } | { exists: false; content: "" }; +type PlannedFileState = { exists: true; content: string; bytes: Buffer; encodingStyle: ApplyPatchV2ReadFile["encodingStyle"] } | { exists: false; content: "" }; type PlannedOperation = { kind: "write"; path: string; content: string } | { kind: "delete"; path: string }; export type ApplyPatchV2Replacement = [start: number, oldLength: number, newLines: string[]]; type Replacement = ApplyPatchV2Replacement; @@ -125,7 +131,7 @@ export interface ApplyPatchV2BulkReplacementWritePlan { finalBytes: number; finalSha256: string; newlineStyle: "lf" | "crlf" | "mixed" | "none"; - encodingStyle: "utf8" | "utf8-bom"; + encodingStyle: ApplyPatchV2ReadFile["encodingStyle"]; replacements: ApplyPatchV2Replacement[]; } type BulkReplacementWritePlan = ApplyPatchV2BulkReplacementWritePlan; @@ -653,7 +659,7 @@ async function applyPatchV2UpdateHunksBulk(executor: ApplyPatchV2Executor, hunks const paths = uniqueStrings(hunks.map((hunk) => hunk.path)); const originalFiles = await readRemoteTextsBulk(executor, paths); const states = new Map(); - for (const [filePath, content] of originalFiles) states.set(filePath, { exists: true, content }); + for (const [filePath, file] of originalFiles) states.set(filePath, { exists: true, ...file }); const changed: string[] = []; const outcomes: ApplyPatchV2Outcome[] = []; @@ -667,12 +673,12 @@ async function applyPatchV2UpdateHunksBulk(executor: ApplyPatchV2Executor, hunks const state = states.get(hunk.path); if (state === undefined || !state.exists) throw new ApplyPatchV2Error("cannot update a missing file in bulk patch", { path: hunk.path }); const originalLines = splitContentLines(state.content); - const textFormat = detectApplyPatchTextFormat(state.content); + const textFormat = detectApplyPatchTextFormat(state.content, state.encodingStyle); const replacements = preserveReplacementNewlines(computeReplacements(hunk.path, originalLines, hunk.chunks), textFormat.newlineStyle); const newContent = joinLinesWithFinalNewline(applyReplacements(originalLines, replacements)); - const originalBuffer = Buffer.from(state.content, "utf8"); - const finalBuffer = Buffer.from(newContent, "utf8"); - states.set(hunk.path, { exists: true, content: newContent }); + const originalBuffer = state.bytes; + const finalBuffer = encodeApplyPatchText(newContent, state.encodingStyle, hunk.path); + states.set(hunk.path, { exists: true, content: newContent, bytes: finalBuffer, encodingStyle: state.encodingStyle }); replacementPlans.set(hunk.path, { path: hunk.path, originalBytes: originalBuffer.length, @@ -1033,7 +1039,7 @@ async function readRemoteText(executor: ApplyPatchV2Executor, target: string): P return contentBuffer.toString("utf8"); } -async function readRemoteTextsBulk(executor: ApplyPatchV2Executor, targets: string[]): Promise> { +async function readRemoteTextsBulk(executor: ApplyPatchV2Executor, targets: string[]): Promise> { if (targets.length === 0) return new Map(); if (executor.fs?.readFiles !== undefined) { try { @@ -1041,14 +1047,14 @@ async function readRemoteTextsBulk(executor: ApplyPatchV2Executor, targets: stri for (const target of targets) { if (!files.has(target)) throw new ApplyPatchV2Error("remote apply-patch v2 fs bulk read omitted target", { target }); } - return files; + return new Map(Array.from(files, ([path, file]) => [path, normalizeApplyPatchV2ReadFile(file)])); } catch (error) { if (!isBulkReadFailure(error)) throw error; } } if (executor.fs || targets.length === 1) { - const files = new Map(); - for (const target of targets) files.set(target, await readRemoteText(executor, target)); + const files = new Map(); + for (const target of targets) files.set(target, decodeApplyPatchV2ReadFile(Buffer.from(await readRemoteText(executor, target), "utf8"))); return files; } const result = await checkedRemoteV2(executor, "read-bulk-b64", targets); @@ -1099,7 +1105,7 @@ function decodeRemoteReadBlock(stdout: string, target: string, blockIndex: numbe return decoded.length > expectedChunkBytes ? decoded.subarray(0, expectedChunkBytes) : decoded; } -export function decodeApplyPatchV2BulkRead(stdout: string, targets: string[]): Map { +export function decodeApplyPatchV2BulkRead(stdout: string, targets: string[]): Map { const lines = stdout.split(/\r?\n/u); const markerIndex = lines.findIndex((line) => line.startsWith(`${remoteBulkReadMarker} `)); if (markerIndex < 0) { @@ -1113,7 +1119,7 @@ export function decodeApplyPatchV2BulkRead(stdout: string, targets: string[]): M if (records.length !== declaredCount) { throw new ApplyPatchV2Error("remote apply-patch v2 bulk read record count mismatch", { expectedTargets: targets.length, records: records.length }); } - const files = new Map(); + const files = new Map(); for (const record of records) { const [pathB64, bytesText, sha256, contentB64] = record.split(/\s+/u); const target = Buffer.from(pathB64, "base64").toString("utf8"); @@ -1132,7 +1138,7 @@ export function decodeApplyPatchV2BulkRead(stdout: string, targets: string[]): M if (actualSha256 !== sha256) { throw new ApplyPatchV2Error(`remote apply-patch v2 bulk read sha256 mismatch for ${target}`, { target, expectedSha256: sha256, actualSha256 }); } - files.set(target, contentBuffer.toString("utf8")); + files.set(target, decodeApplyPatchV2ReadFile(contentBuffer)); } for (const target of targets) { if (!files.has(target)) throw new ApplyPatchV2Error("remote apply-patch v2 bulk read omitted target", { target }); @@ -1188,7 +1194,7 @@ async function enrichBulkReplacementIntegrityFailure(error: unknown, fs: ApplyPa const details = error instanceof ApplyPatchV2Error ? error.details : {}; const serialized = `${message}\n${JSON.stringify(details)}`; if (fs.readFiles === undefined || !serialized.includes("integrity mismatch")) return error instanceof ApplyPatchV2Error ? error : new ApplyPatchV2Error(message); - let currentFiles: Map; + let currentFiles: Map; try { currentFiles = await fs.readFiles(targets); } catch (readError) { @@ -1202,10 +1208,11 @@ async function enrichBulkReplacementIntegrityFailure(error: unknown, fs: ApplyPa const adapter = route.includes(":win") ? "windows" : "posix"; const integrityDiagnostics = targets.flatMap((target) => { const plan = plans.get(target); - const current = currentFiles.get(target); - if (plan === undefined || current === undefined) return []; - const currentBuffer = Buffer.from(current, "utf8"); - const currentFormat = detectApplyPatchTextFormat(current); + const currentValue = currentFiles.get(target); + if (plan === undefined || currentValue === undefined) return []; + const current = normalizeApplyPatchV2ReadFile(currentValue); + const currentBuffer = current.bytes; + const currentFormat = detectApplyPatchTextFormat(current.content, current.encodingStyle); return [{ adapter, stage, @@ -1245,6 +1252,7 @@ export function formatApplyPatchV2BulkReplacementPayload(paths: Iterable plan.originalSha256, String(plan.finalBytes), plan.finalSha256, + plan.encodingStyle, replacementFields.join(";"), ].join(" ")); } @@ -1364,11 +1372,14 @@ export function applyPatchV2RemoteCommand(operation: ApplyPatchV2RemoteOperation " python3 - \"$tmp_payload\" \"$tmp_dir\" \"$expected_count\" <<'PY'", "import base64, hashlib, pathlib, shutil, sys", "def text_profile(source):", - " encoding = 'utf8-bom' if source.startswith(b'\\xef\\xbb\\xbf') else 'utf8'", - " text = source.decode('utf-8')", - " crlf = text.count('\\r\\n')", - " bare_lf = text.count('\\n') - crlf", - " lone_cr = text.count('\\r') - crlf", + " try:", + " source.decode('utf-8')", + " encoding = 'utf8-bom' if source.startswith(b'\\xef\\xbb\\xbf') else 'utf8'", + " except UnicodeDecodeError:", + " encoding = 'legacy-single-byte'", + " crlf = source.count(b'\\r\\n')", + " bare_lf = source.count(b'\\n') - crlf", + " lone_cr = source.count(b'\\r') - crlf", " newline = 'crlf' if crlf > 0 and bare_lf == 0 and lone_cr == 0 else ('lf' if bare_lf > 0 and crlf == 0 and lone_cr == 0 else ('none' if crlf == 0 and bare_lf == 0 and lone_cr == 0 else 'mixed'))", " return encoding, newline", "def integrity_failure(adapter, stage, target, expected_bytes, expected_sha256, source):", @@ -1384,16 +1395,17 @@ export function applyPatchV2RemoteCommand(operation: ApplyPatchV2RemoteOperation "plans = []", "for index, record in enumerate(records, start=1):", " fields = record.split()", - " if len(fields) != 6:", + " if len(fields) != 7:", " raise SystemExit('bulk replacement malformed record')", - " path_b64, original_bytes_text, original_sha256, final_bytes_text, final_sha256, replacements_text = fields", + " path_b64, original_bytes_text, original_sha256, final_bytes_text, final_sha256, encoding_style, replacements_text = fields", " target = base64.b64decode(path_b64).decode('utf-8')", " original_bytes = int(original_bytes_text)", " final_bytes = int(final_bytes_text)", " source = pathlib.Path(target).read_bytes()", " if len(source) != original_bytes or hashlib.sha256(source).hexdigest() != original_sha256:", " integrity_failure('posix', 'original', target, original_bytes, original_sha256, source)", - " lines = source.decode('utf-8').split('\\n')", + " source_encoding = 'latin-1' if encoding_style == 'legacy-single-byte' else 'utf-8'", + " lines = source.decode(source_encoding).split('\\n')", " if lines and lines[-1] == '':", " lines.pop()", " replacements = []", @@ -1410,7 +1422,7 @@ export function applyPatchV2RemoteCommand(operation: ApplyPatchV2RemoteOperation " if start < 0 or old_len < 0 or start + old_len > len(lines):", " raise SystemExit(f'bulk replacement out of bounds for {target}')", " lines[start:start + old_len] = new_lines", - " output = (('\\n'.join(lines) + '\\n') if lines else '').encode('utf-8')", + " output = (('\\n'.join(lines) + '\\n') if lines else '').encode(source_encoding)", " if len(output) != final_bytes or hashlib.sha256(output).hexdigest() != final_sha256:", " integrity_failure('posix', 'final', target, final_bytes, final_sha256, output)", " tmp_path = pathlib.Path(tmp_dir) / f'file-{index}.tmp'", @@ -1636,7 +1648,7 @@ function splitContentLines(content: string): string[] { return lines; } -function detectApplyPatchTextFormat(content: string): Pick { +function detectApplyPatchTextFormat(content: string, encodingStyle: ApplyPatchV2ReadFile["encodingStyle"] = content.startsWith("\uFEFF") ? "utf8-bom" : "utf8"): Pick { const crlfCount = (content.match(/\r\n/gu) ?? []).length; const bareLfCount = (content.match(/(? 0xff) { + throw new ApplyPatchV2Error("legacy single-byte localized replacement only supports byte-preserving text", { + path, + encodingStyle, + replacementRequirement: "ASCII or existing single-byte characters", + }); + } + } + return Buffer.from(content, "latin1"); +} + function preserveReplacementNewlines(replacements: Replacement[], newlineStyle: ApplyPatchV2BulkReplacementWritePlan["newlineStyle"]): Replacement[] { if (newlineStyle !== "crlf") return replacements; return replacements.map(([start, oldLength, newLines]) => [ diff --git a/scripts/src/ssh-runtime.ts b/scripts/src/ssh-runtime.ts index c8805824..c3b9d5c2 100644 --- a/scripts/src/ssh-runtime.ts +++ b/scripts/src/ssh-runtime.ts @@ -860,7 +860,7 @@ function windowsApplyPatchFsScript(basePath: string | null, operation: WindowsAp const arg1 = args[1] ?? ""; const arg2 = args[2] ?? ""; const arg3 = args[3] ?? ""; - return [ + const scriptParts = [ "$ErrorActionPreference = 'Stop';", "$ProgressPreference = 'SilentlyContinue';", "[Console]::InputEncoding = [System.Text.UTF8Encoding]::new();", @@ -895,6 +895,41 @@ function windowsApplyPatchFsScript(basePath: string | null, operation: WindowsAp " 'delete' { Remove-Item -LiteralPath $target -Force -ErrorAction SilentlyContinue; break }", " default { Fail ('unsupported apply-patch fs op: ' + $operation) 2 }", "}", + ]; + return scriptParts + .map((line) => line.startsWith(" 'apply-replacements-bulk-stdin' ") ? windowsBulkReplacementScript() : line) + .join(" "); +} + +function windowsBulkReplacementScript(): string { + return [ + " 'apply-replacements-bulk-stdin' {", + " $expectedCount = [Int32]$targetArg;", + " $records = @([Console]::In.ReadToEnd() -split \"`r?`n\" | Where-Object { -not [string]::IsNullOrWhiteSpace($_) });", + " if ($records.Count -ne $expectedCount) { Fail ('bulk replacement record count mismatch: expected=' + $expectedCount + ' actual=' + $records.Count) 23 };", + " $plans = New-Object System.Collections.Generic.List[object];", + " foreach ($record in $records) {", + " $fields = $record -split '\\s+'; if ($fields.Count -ne 7) { Fail 'bulk replacement malformed record' 23 };", + " $targetRelative = Decode-Utf8 $fields[0]; $resolved = Resolve-UnideskPath $targetRelative;", + " $originalBytes = [Int64]$fields[1]; $originalSha256 = $fields[2]; $finalBytes = [Int64]$fields[3]; $finalSha256 = $fields[4]; $encodingStyle = $fields[5]; $replacementsText = $fields[6];", + " if (-not (Test-Path -LiteralPath $resolved -PathType Leaf)) { Fail ('file not found: ' + $resolved) 1 };", + " $source = [System.IO.File]::ReadAllBytes($resolved); $actualOriginalBytes = [Int64]$source.Length; $actualOriginalSha256 = Get-Sha256Bytes $source;", + " if ($actualOriginalBytes -ne $originalBytes) { Fail ('bulk replacement original integrity mismatch for ' + $resolved) 23 };", + " if (-not [string]::Equals($actualOriginalSha256, $originalSha256, [StringComparison]::OrdinalIgnoreCase)) { Fail ('bulk replacement original integrity mismatch for ' + $resolved) 23 };", + " $textEncoding = if ($encodingStyle -eq 'legacy-single-byte') { [System.Text.Encoding]::GetEncoding(28591) } else { [System.Text.UTF8Encoding]::new($false, $true) };", + " try { $sourceText = $textEncoding.GetString($source) } catch { Fail ('bulk replacement source decode failed encoding=' + $encodingStyle + ' for ' + $resolved) 25 };", + " $lines = New-Object System.Collections.Generic.List[string]; foreach ($line in ($sourceText -split \"`n\", -1)) { $lines.Add($line) | Out-Null }; if ($lines.Count -gt 0 -and $lines[$lines.Count - 1] -eq '') { $lines.RemoveAt($lines.Count - 1) };", + " $replacements = New-Object System.Collections.Generic.List[object];", + " if (-not [string]::IsNullOrEmpty($replacementsText)) { foreach ($item in ($replacementsText -split ';')) { if ([string]::IsNullOrWhiteSpace($item)) { continue }; $parts = $item -split ',', 3; if ($parts.Count -ne 3) { Fail 'bulk replacement malformed replacement' 23 }; $newText = Decode-Utf8 $parts[2]; $newLines = New-Object System.Collections.Generic.List[string]; foreach ($line in ($newText -split \"`n\", -1)) { $newLines.Add($line) | Out-Null }; if ($newLines.Count -gt 0 -and $newLines[$newLines.Count - 1] -eq '') { $newLines.RemoveAt($newLines.Count - 1) }; $replacements.Add([pscustomobject]@{ start = [Int32]$parts[0]; oldLength = [Int32]$parts[1]; newLines = [string[]]$newLines }) | Out-Null } };", + " foreach ($replacement in @($replacements | Sort-Object -Property start -Descending)) { if ($replacement.start -lt 0 -or $replacement.oldLength -lt 0 -or ($replacement.start + $replacement.oldLength) -gt $lines.Count) { Fail ('bulk replacement out of bounds for ' + $resolved) 23 }; if ($replacement.oldLength -gt 0) { $lines.RemoveRange($replacement.start, $replacement.oldLength) }; if ($replacement.newLines.Count -gt 0) { $lines.InsertRange($replacement.start, [string[]]$replacement.newLines) } };", + " $outputText = if ($lines.Count -eq 0) { '' } else { ([string]::Join(\"`n\", [string[]]$lines) + \"`n\") };", + " try { $outputBytes = $textEncoding.GetBytes($outputText) } catch { Fail ('bulk replacement final encode failed encoding=' + $encodingStyle + ' for ' + $resolved) 25 };", + " if ($outputBytes.Length -ne $finalBytes -or (Get-Sha256Bytes $outputBytes) -ne $finalSha256) { Fail ('bulk replacement final integrity mismatch for ' + $resolved) 23 };", + " $tmp = [System.IO.Path]::Combine([System.IO.Path]::GetDirectoryName($resolved), ('.' + [System.IO.Path]::GetFileName($resolved) + '.unidesk-v2-bulk-' + [guid]::NewGuid().ToString('N') + '.tmp')); [System.IO.File]::WriteAllBytes($tmp, $outputBytes); $plans.Add([pscustomobject]@{ target = $resolved; tmp = $tmp; sha256 = $finalSha256 }) | Out-Null;", + " };", + " foreach ($plan in $plans) { Ensure-Parent $plan.target; Move-Item -LiteralPath $plan.tmp -Destination $plan.target -Force; if ((Get-Sha256 $plan.target) -ne $plan.sha256) { Fail ('bulk replacement final sha256 mismatch for ' + $plan.target) 25 } };", + " break", + " }", ].join(" "); } diff --git a/scripts/src/ssh-windows-fs/head-tail.ps1 b/scripts/src/ssh-windows-fs/head-tail.ps1 index 054432a4..ccf88e46 100644 --- a/scripts/src/ssh-windows-fs/head-tail.ps1 +++ b/scripts/src/ssh-windows-fs/head-tail.ps1 @@ -10,7 +10,26 @@ if ($operation -in @('head', 'tail')) { $paths.Add($arg) } if ($paths.Count -ne 1) { Fail ($operation + ' requires exactly one file path on Windows routes') 2 } - $lines = Text-Lines (Read-StrictText (Resolve-UnideskPath $paths[0]) 1048576) - if ($operation -eq 'head') { Print-Lines ([string[]]($lines | Select-Object -First $linesWanted)) } else { Print-Lines ([string[]]($lines | Select-Object -Last $linesWanted)) } + $resolved = Resolve-UnideskPath $paths[0] + Need-File $resolved + $selected = [Collections.Generic.List[string]]::new() + $reader = [IO.StreamReader]::new($resolved, [Text.UTF8Encoding]::new($false, $true), $true) + try { + if ($operation -eq 'head') { + while ($selected.Count -lt $linesWanted -and -not $reader.EndOfStream) { $selected.Add($reader.ReadLine()) } + } else { + $queue = [Collections.Generic.Queue[string]]::new() + while (-not $reader.EndOfStream) { + $queue.Enqueue($reader.ReadLine()) + if ($queue.Count -gt $linesWanted) { [void]$queue.Dequeue() } + } + foreach ($line in $queue) { $selected.Add($line) } + } + } catch { + Fail ('file is not valid UTF-8: ' + $resolved + ': ' + $_.Exception.Message) 25 + } finally { + $reader.Dispose() + } + Print-Lines ([string[]]$selected) exit 0 } diff --git a/scripts/src/ssh-windows-fs/rg.ps1 b/scripts/src/ssh-windows-fs/rg.ps1 index 8df2365e..ce05b73a 100644 --- a/scripts/src/ssh-windows-fs/rg.ps1 +++ b/scripts/src/ssh-windows-fs/rg.ps1 @@ -1,30 +1,32 @@ if ($operation -ne 'rg') { Fail ('unsupported Windows fs operation: ' + $operation) 2 } -$ignoreCase = $false; $fixed = $false; $filesOnly = $false; $pattern = $null; $endOptions = $false +$ignoreCase = $false; $fixed = $false; $filesOnly = $false; $endOptions = $false $maxCount = [int]$rgPolicy.maxMatches; $maxFiles = [int]$rgPolicy.maxFiles; $maxBytes = [int]$rgPolicy.maxBytesPerFile; $timeoutMs = [int]$rgPolicy.timeoutMs -$paths = [Collections.Generic.List[string]]::new(); $globs = [Collections.Generic.List[string]]::new() +$paths = [Collections.Generic.List[string]]::new(); $globs = [Collections.Generic.List[string]]::new(); $patterns = [Collections.Generic.List[string]]::new() for ($i = 0; $i -lt $toolArgs.Count; $i++) { $arg = $toolArgs[$i] - if ($null -eq $pattern -and -not $endOptions -and $arg -eq '--') { $endOptions = $true; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -in @('-i', '--ignore-case')) { $ignoreCase = $true; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -in @('-F', '--fixed-strings')) { $fixed = $true; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -in @('-n', '--line-number')) { continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -eq '--files') { $filesOnly = $true; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -in @('-g', '--glob')) { $i += 1; if ($i -ge $toolArgs.Count) { Fail ($arg + ' requires a value') 2 }; $globs.Add($toolArgs[$i]); continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('--glob=')) { $globs.Add($arg.Substring(7)); continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -in @('-m', '--max-count')) { $i += 1; if ($i -ge $toolArgs.Count) { Fail ($arg + ' requires a value') 2 }; $maxCount = Parse-NonNegativeInt $arg $toolArgs[$i] 10000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('-m') -and $arg.Length -gt 2) { $maxCount = Parse-NonNegativeInt '-m' $arg.Substring(2) 10000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('--max-count=')) { $maxCount = Parse-NonNegativeInt '--max-count' $arg.Substring(12) 10000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -eq '--max-files') { $i += 1; if ($i -ge $toolArgs.Count) { Fail '--max-files requires a value' 2 }; $maxFiles = Parse-NonNegativeInt '--max-files' $toolArgs[$i] 50000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('--max-files=')) { $maxFiles = Parse-NonNegativeInt '--max-files' $arg.Substring(12) 50000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -eq '--max-bytes') { $i += 1; if ($i -ge $toolArgs.Count) { Fail '--max-bytes requires a value' 2 }; $maxBytes = Parse-NonNegativeInt '--max-bytes' $toolArgs[$i] 16777216; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('--max-bytes=')) { $maxBytes = Parse-NonNegativeInt '--max-bytes' $arg.Substring(12) 16777216; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg -eq '--timeout-ms') { $i += 1; if ($i -ge $toolArgs.Count) { Fail '--timeout-ms requires a value' 2 }; $timeoutMs = Parse-NonNegativeInt '--timeout-ms' $toolArgs[$i] 60000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('--timeout-ms=')) { $timeoutMs = Parse-NonNegativeInt '--timeout-ms' $arg.Substring(13) 60000; continue } - if ($null -eq $pattern -and -not $endOptions -and $arg.StartsWith('-')) { Fail ('unsupported rg option on Windows route: ' + $arg + '; supported: --files -g/--glob -i -F -n -m/--max-count --max-files --max-bytes --timeout-ms') 2 } - if ($filesOnly) { $paths.Add($arg) } elseif ($null -eq $pattern) { $pattern = $arg } else { $paths.Add($arg) } + if (-not $endOptions -and $arg -eq '--') { $endOptions = $true; continue } + if (-not $endOptions -and $arg -in @('-i', '--ignore-case')) { $ignoreCase = $true; continue } + if (-not $endOptions -and $arg -in @('-F', '--fixed-strings')) { $fixed = $true; continue } + if (-not $endOptions -and $arg -in @('-n', '--line-number')) { continue } + if (-not $endOptions -and $arg -eq '--files') { $filesOnly = $true; continue } + if (-not $endOptions -and $arg -in @('-e', '--regexp')) { $i += 1; if ($i -ge $toolArgs.Count) { Fail ($arg + ' requires a value') 2 }; $patterns.Add($toolArgs[$i]); continue } + if (-not $endOptions -and $arg.StartsWith('--regexp=')) { $patterns.Add($arg.Substring(9)); continue } + if (-not $endOptions -and $arg -in @('-g', '--glob')) { $i += 1; if ($i -ge $toolArgs.Count) { Fail ($arg + ' requires a value') 2 }; $globs.Add($toolArgs[$i]); continue } + if (-not $endOptions -and $arg.StartsWith('--glob=')) { $globs.Add($arg.Substring(7)); continue } + if (-not $endOptions -and $arg -in @('-m', '--max-count')) { $i += 1; if ($i -ge $toolArgs.Count) { Fail ($arg + ' requires a value') 2 }; $maxCount = Parse-NonNegativeInt $arg $toolArgs[$i] 10000; continue } + if (-not $endOptions -and $arg.StartsWith('-m') -and $arg.Length -gt 2) { $maxCount = Parse-NonNegativeInt '-m' $arg.Substring(2) 10000; continue } + if (-not $endOptions -and $arg.StartsWith('--max-count=')) { $maxCount = Parse-NonNegativeInt '--max-count' $arg.Substring(12) 10000; continue } + if (-not $endOptions -and $arg -eq '--max-files') { $i += 1; if ($i -ge $toolArgs.Count) { Fail '--max-files requires a value' 2 }; $maxFiles = Parse-NonNegativeInt '--max-files' $toolArgs[$i] 50000; continue } + if (-not $endOptions -and $arg.StartsWith('--max-files=')) { $maxFiles = Parse-NonNegativeInt '--max-files' $arg.Substring(12) 50000; continue } + if (-not $endOptions -and $arg -eq '--max-bytes') { $i += 1; if ($i -ge $toolArgs.Count) { Fail '--max-bytes requires a value' 2 }; $maxBytes = Parse-NonNegativeInt '--max-bytes' $toolArgs[$i] 16777216; continue } + if (-not $endOptions -and $arg.StartsWith('--max-bytes=')) { $maxBytes = Parse-NonNegativeInt '--max-bytes' $arg.Substring(12) 16777216; continue } + if (-not $endOptions -and $arg -eq '--timeout-ms') { $i += 1; if ($i -ge $toolArgs.Count) { Fail '--timeout-ms requires a value' 2 }; $timeoutMs = Parse-NonNegativeInt '--timeout-ms' $toolArgs[$i] 60000; continue } + if (-not $endOptions -and $arg.StartsWith('--timeout-ms=')) { $timeoutMs = Parse-NonNegativeInt '--timeout-ms' $arg.Substring(13) 60000; continue } + if (-not $endOptions -and $arg.StartsWith('-')) { Fail ('unsupported rg option on Windows route: ' + $arg + '; supported: --files -e/--regexp -g/--glob -i -F -n -m/--max-count --max-files --max-bytes --timeout-ms') 2 } + if ($filesOnly) { $paths.Add($arg) } elseif ($patterns.Count -eq 0) { $patterns.Add($arg) } else { $paths.Add($arg) } } -if (-not $filesOnly -and $null -eq $pattern) { Fail 'rg requires a pattern on Windows routes (or use --files)' 2 } +if (-not $filesOnly -and $patterns.Count -eq 0) { Fail 'rg requires a pattern on Windows routes (or use --files)' 2 } if ($paths.Count -eq 0) { $paths.Add('.') } $nativeRg = Get-Command rg.exe -ErrorAction SilentlyContinue | Select-Object -First 1 @@ -36,12 +38,12 @@ if ($null -ne $nativeRg) { return $Value } $nativeArgs = [Collections.Generic.List[string]]::new() - $nativeArgs.Add('--color=never'); $nativeArgs.Add('--no-heading'); $nativeArgs.Add('--line-number') + $nativeArgs.Add('--color=never'); $nativeArgs.Add('--no-heading'); $nativeArgs.Add('--line-number'); $nativeArgs.Add('--with-filename') $nativeArgs.Add('--max-filesize'); $nativeArgs.Add([string]$maxBytes) if ($ignoreCase) { $nativeArgs.Add('--ignore-case') } if ($fixed) { $nativeArgs.Add('--fixed-strings') } foreach ($glob in $globs) { $nativeArgs.Add('--glob'); $nativeArgs.Add($glob) } - if ($filesOnly) { $nativeArgs.Add('--files') } else { $nativeArgs.Add($pattern) } + if ($filesOnly) { $nativeArgs.Add('--files') } else { foreach ($item in $patterns) { $nativeArgs.Add('--regexp'); $nativeArgs.Add($item) } } foreach ($raw in $paths) { $nativeArgs.Add((Resolve-UnideskPath $raw)) } $startInfo = [Diagnostics.ProcessStartInfo]::new() $startInfo.FileName = $nativeRg.Source @@ -59,9 +61,10 @@ if ($null -ne $nativeRg) { if (-not [string]::IsNullOrWhiteSpace($stderrText)) { [Console]::Error.Write($stderrText) } $truncated = $lines.Count -gt $selected.Count $matchedFiles = [Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) - if (-not $filesOnly) { foreach ($line in $selected) { if ($line -match '^([A-Za-z]:\\.*?):\d+:') { [void]$matchedFiles.Add($Matches[1]) } } } + if (-not $filesOnly) { foreach ($line in $selected) { if ($line -match '^(.+?):\d+:') { [void]$matchedFiles.Add($Matches[1]) } } } $fileCount = if ($filesOnly) { $selected.Count } else { $matchedFiles.Count } - [Console]::Error.WriteLine('UNIDESK_WINDOWS_RG_SUMMARY matched=' + $selected.Count + ' fileCount=' + $fileCount + ' elapsedMs=' + $stopwatch.ElapsedMilliseconds + ' timeout=' + $timedOut.ToString().ToLowerInvariant() + ' skipped=0 fileLimit=' + ($filesOnly -and $truncated).ToString().ToLowerInvariant() + ' matchLimit=' + ((-not $filesOnly) -and $truncated).ToString().ToLowerInvariant() + ' engine=native-rg config=config/unidesk-cli.yaml#trans.windowsFs.rg') + $encodingState = if ($stdoutText.Contains([char]0xfffd)) { 'legacy-or-invalid-utf8' } else { 'utf8-or-ascii' } + [Console]::Error.WriteLine('UNIDESK_WINDOWS_RG_SUMMARY matched=' + $selected.Count + ' fileCount=' + $fileCount + ' elapsedMs=' + $stopwatch.ElapsedMilliseconds + ' timeout=' + $timedOut.ToString().ToLowerInvariant() + ' encoding=' + $encodingState + ' skipped=0 fileLimit=' + ($filesOnly -and $truncated).ToString().ToLowerInvariant() + ' matchLimit=' + ((-not $filesOnly) -and $truncated).ToString().ToLowerInvariant() + ' engine=native-rg config=config/unidesk-cli.yaml#trans.windowsFs.rg') if ($timedOut) { [Console]::Error.WriteLine('UNIDESK_WINDOWS_RG_HINT code=timeout exitCode=124 action=narrow-scope-or-add-glob message=retry-with-a-narrower-path-or-g/--glob'); exit 124 } if ($process.ExitCode -ne 0) { exit $process.ExitCode } exit 0 diff --git a/scripts/src/ssh.test.ts b/scripts/src/ssh.test.ts index 6d659a9d..e955bd0d 100644 --- a/scripts/src/ssh.test.ts +++ b/scripts/src/ssh.test.ts @@ -103,6 +103,20 @@ describe("ssh windows fs read-only operations", () => { expect(rgScript).toContain("config/unidesk-cli.yaml#trans.windowsFs.rg"); expect(rgScript).toContain("--files"); expect(rgScript).toContain("--glob"); + expect(rgScript).toContain("--with-filename"); + expect(rgScript).toContain("encoding="); + }); + + test("supports repeated rg patterns and streams large-file tail", () => { + const rgScript = windowsFsReadOnlyScript("F:\\Work\\demo", "rg", ["-n", "-e", "alpha", "--regexp", "beta", "src/test.c"]); + const tailScript = windowsFsReadOnlyScript("F:\\Work\\demo", "tail", ["-n", "40", ".state/request_history.jsonl"]); + + expect(rgScript).toContain("$patterns.Add($toolArgs[$i])"); + expect(rgScript).toContain("$nativeArgs.Add('--regexp')"); + expect(rgScript).toContain("fileCount="); + expect(tailScript).toContain("[IO.StreamReader]::new"); + expect(tailScript).toContain("[Collections.Generic.Queue[string]]::new()"); + expect(tailScript).not.toContain("Read-StrictText (Resolve-UnideskPath $paths[0]) 1048576"); }); test("supports wc flags, exact skill lookup, and Windows cmd argv boundaries", () => { @@ -236,7 +250,7 @@ describe("ssh host apply-patch fs backend", () => { }, ]])); - expect(files.get("docs/test.md")).toBe(text); + expect(files.get("docs/test.md")?.content).toBe(text); expect(calls.map((call) => call.operation)).toEqual(["read-bulk-b64", "apply-replacements-bulk-stdin"]); expect(calls[0]?.args).toEqual(["docs/test.md"]); expect(calls[1]?.args).toEqual(["1"]);