diff --git a/.agents/skills/unidesk-webdev/SKILL.md b/.agents/skills/unidesk-webdev/SKILL.md index 21221d83..3ee356e1 100644 --- a/.agents/skills/unidesk-webdev/SKILL.md +++ b/.agents/skills/unidesk-webdev/SKILL.md @@ -97,11 +97,11 @@ bun scripts/cli.ts hwlab nodes web-probe observe analyze webobs-xxxx 约束： - `web-probe script` 不运行默认探针，必须通过 stdin heredoc 或 `--script-file ` 提供脚本；只需要 repo-owned 标准 DOM probe 时使用 `web-probe run`。 -- `web-probe run|script|observe start` 的默认 URL 和 browser proxy mode 必须来自 `config/hwlab-node-lanes.yaml` 的 `webProbe`；需要排除公网/FRP/跨国 proxy 抖动时，在 YAML 里把目标 node/lane 的 `webProbe.defaultOrigin` 配成内部 Service ClusterIP origin，不要在命令行长期手写 `--url` 或裸 Playwright。 +- `web-probe run|script|observe start` 的默认 URL、browser proxy mode 和 observe/analyze 报警阈值必须来自 `config/hwlab-node-lanes.yaml` 的 `webProbe`；需要排除公网/FRP/跨国 proxy 抖动时，在 YAML 里把目标 node/lane 的 `webProbe.defaultOrigin` 配成内部 Service ClusterIP origin，不要在命令行长期手写 `--url` 或裸 Playwright。 - `web-probe observe start` 默认是被动观测：记录 DOM 摘要、自然页面 request/response/requestfailed、截图和 performance 样本，不主动 fetch Workbench API、不切换 control session、不拦截路由、不调用 repair helper。长程 Workbench 观测必须保留 control/observer 双页面模型：control 页面执行显式 command，observer 页面只同步到同一 session URL 后被动采样，并按默认 180000ms 周期整页刷新同一 session 来模拟用户往返；周期刷新只作用于 observer，不得改变 control active session 或作为通过条件。两页的 `pageRole`、`pageId`、`sampleGroupSeq` 必须进入样本和 analyzer 报表。任何 `newSession`、`selectProvider`、`sendPrompt`、`goto`、`screenshot`、`mark`、`stop` 都必须通过 `observe command` 显式下发，并进入 `control.jsonl`；长 prompt 必须优先用 `sendPrompt --text-stdin`，不要为了绕开 shell quoting 退回裸 Playwright 或临时脚本。 - `web-probe observe` 的 issue evidence 优先记录 observer id、stateDir、report JSON/Markdown SHA、samples/control/network/artifact 计数、routeSessionId、activeSessionId、prompt hash/textBytes、traceId、AgentRun runId/commandId、最终 status 和必要摘要；不要把 prompt 原文、assistant 大段正文、完整 stdout/stderr 或 provider payload 粘贴到 issue。 - 多轮 Workbench 采样必须证明同一个 `sessionId` 连续承载所有轮次；每轮至少记录 prompt hash、traceId、终态、最终回答摘要和性能/产物表。若 Web UI 投影卡住但 Code Agent/AgentRun result 已 terminal，应同时登记“执行终态”和“Workbench 投影未收敛”，不得用 `goto`、reload、切 session 或 result polling 把 UI 失败伪装成通过。 -- `observe analyze` 是离线分析，只读取 artifact JSONL 并写 `analysis/report.md` 与 `analysis/report.json`，不访问 Workbench API、不驱动浏览器。`observe start` 每次启动必须先把同一 stateDir 中已有的根目录 JSONL 轮转到带时间戳的 `archive/` 文件；`observe analyze` 默认只分析当前根目录 JSONL，不扫描历史 archive，只有显式指定 archive prefix 时才分析历史轮转窗口。报告必须输出采样点 vs 每个 turn 的总耗时/最近更新时间表、可见“加载中”的数量/归属/并发 owner/连续出现区间、DOM diagnostic/HTTP/console/requestfailed/runtime execution error 分组、page asset provenance segment、同源 API Resource Timing 分位表和超过 5s 的慢路径 finding；页面/API 加载超过 5s 视为不可用级性能红线，可见“加载中”持续超过 5s 也必须作为真实慢加载证据登记到上游问题。修复必须降低真实请求、投影、渲染或后端路径耗时，禁止为了减少“加载中”出现时间而提前展示未加载完的内容，也不能靠下游 retry/reload/fallback 掩盖。报告里的 `final-response-flicker`、`uncommanded-visible-state-change`、session changed、network 503 等 finding 是排障线索；用于 closeout 时必须结合原始 session/trace/DOM 证据解释，避免把采样噪声直接当作业务结论。 +- `observe analyze` 是离线分析，只读取 artifact JSONL 并写 `analysis/report.md` 与 `analysis/report.json`，不访问 Workbench API、不驱动浏览器。`observe start` 每次启动必须先把同一 stateDir 中已有的根目录 JSONL 轮转到带时间戳的 `archive/` 文件；`observe analyze` 默认只分析当前根目录 JSONL，不扫描历史 archive，只有显式指定 archive prefix 时才分析历史轮转窗口。报告必须输出采样点 vs 每个 turn 的总耗时/最近更新时间表、可见“加载中”的数量/归属/并发 owner/连续出现区间、DOM diagnostic/HTTP/console/requestfailed/runtime execution error 分组、page asset provenance segment、同源 API Resource Timing 分位表和超过 YAML `webProbe.alertThresholds` budget 的慢路径 finding；页面/API 加载、可见“加载中”、长连接打开耗时、turn timing 跳变和 session fallback 标题比例的报警阈值只能改 YAML，不能在 analyzer/renderer 中写死。修复必须降低真实请求、投影、渲染或后端路径耗时，禁止为了减少“加载中”出现时间而提前展示未加载完的内容，也不能靠下游 retry/reload/fallback 掩盖。报告里的 `final-response-flicker`、`uncommanded-visible-state-change`、session changed、network 503 等 finding 是排障线索；用于 closeout 时必须结合原始 session/trace/DOM 证据解释，避免把采样噪声直接当作业务结论。 - 自定义 `web-probe script` 仍运行在 UniDesk `trans` 60s 最外层短连接约束内；能在一轮内完成的 P4 验收优先把 `--command-timeout-seconds` 控制在 55 秒以内，并减少无界 selector/network 等待。确需等待更久时，改用 `web-probe run` 的异步 job/status 语义，或把动作拆成“提交/采样/截图/状态读取”多次短 probe。若输出出现 `UNIDESK_SSH_RUNTIME_TIMEOUT` 但同时恢复了 `reportPath`、`reportSha256`、screenshots 或 DOM steps，先按远端报告判断脚本/页面实际状态；最终关闭证据仍优先用一次未触发短连接超时的 bounded rerun。 - issue closeout 优先引用 `web-probe script` 输出的顶层 `issueEvidence` 或 `summary.issueEvidence`；只有需要展开调查时才粘贴 `probe.script.result`、`probe.steps` 或完整 `reportPath`，避免 stdout、summary 和 report 多层重复同一证据。 - stdin heredoc 与 `--script-file` 都按 ES module 加载，脚本必须导出 `export default async ({ page, gotoStable, recordStep, ... }) => { ... }`；不要在模块顶层直接写 `return`。失败为 `Illegal return statement`、`does not provide an export named default` 或 finalUrl 仍是 `about:blank` 且 stepCount=0 时，先按 probe 脚本入口误用处理，不要归因成 Cloud Web 行为失败。 diff --git a/config/hwlab-node-lanes.yaml b/config/hwlab-node-lanes.yaml index 4410b54a..80c382e3 100644 --- a/config/hwlab-node-lanes.yaml +++ b/config/hwlab-node-lanes.yaml @@ -172,6 +172,13 @@ lanes: namespace: hwlab-v03 port: 8080 scheme: http + alertThresholds: + sameOriginApiSlowMs: 10000 + partialApiSlowMs: 10000 + longLivedStreamOpenSlowMs: 10000 + visibleLoadingSlowMs: 10000 + turnTimingSampleSlackSeconds: 3 + sessionRailFallbackRatio: 0.5 tektonDir: tekton-v03 argoApplicationFile: application-v03.yaml registryPrefix: 127.0.0.1:5000/hwlab diff --git a/scripts/src/hwlab-node-help.ts b/scripts/src/hwlab-node-help.ts index 63fe7322..1ae872a2 100644 --- a/scripts/src/hwlab-node-help.ts +++ b/scripts/src/hwlab-node-help.ts @@ -90,7 +90,7 @@ export function hwlabNodeWebProbeHelp(): Record { observe: "Start, inspect, control, stop, collect, and analyze a pure-client long-running Workbench observer on the target host. The observer runs a control page plus a passive observer page in a shared-auth browser context, receives commands through stateDir/commands files, writes JSONL artifacts, and does not expose any inbound service API.", }, notes: [ - "The default probe URL and browser proxy mode come from config/hwlab-node-lanes.yaml webProbe; pass --url only when intentionally overriding the YAML-selected origin.", + "The default probe URL, browser proxy mode, and observe/analyze alert thresholds come from config/hwlab-node-lanes.yaml webProbe; pass --url only when intentionally overriding the YAML-selected origin.", "Prefer --script-file for reusable probes; stdin heredocs remain supported for one-off probes.", "Issue-ready evidence is available under issueEvidence and summary.issueEvidence; full script report is persisted under probe.reportPath with a SHA-256 fingerprint.", "observe sampling is passive by default: it records DOM summaries and natural page request/response/requestfailed events with observerInitiated=false; it does not actively fetch Workbench APIs, reload, switch sessions, route/intercept, or call repair helpers.", diff --git a/scripts/src/hwlab-node-impl.ts b/scripts/src/hwlab-node-impl.ts index bfed6ef8..55ce7f0a 100644 --- a/scripts/src/hwlab-node-impl.ts +++ b/scripts/src/hwlab-node-impl.ts @@ -8,7 +8,7 @@ import { runCommand, type CommandResult } from "./command"; import { startJob } from "./jobs"; import { classifySshTcpPoolFailure } from "./ssh"; import { HWLAB_NODE_CONTROL_PLANE_CONFIG_PATH, hwlabNodeControlPlaneInfraHelp, runHwlabNodeControlPlaneInfra } from "./hwlab-node-control-plane"; -import { hwlabRuntimeLaneConfigPath, hwlabRuntimeLaneIds, hwlabRuntimeLaneSpec, hwlabRuntimeLaneSpecForNode, hwlabRuntimeNodeIds, isHwlabRuntimeLane, type HwlabRuntimeLane, type HwlabRuntimeLaneSpec, type HwlabRuntimeObservabilityRecordingRuleSpec, type HwlabRuntimeObservabilitySpec, type HwlabRuntimeObservabilityWarningAlertSpec, type HwlabRuntimePublicExposureSpec } from "./hwlab-node-lanes"; +import { hwlabRuntimeLaneConfigPath, hwlabRuntimeLaneIds, hwlabRuntimeLaneSpec, hwlabRuntimeLaneSpecForNode, hwlabRuntimeNodeIds, isHwlabRuntimeLane, type HwlabRuntimeLane, type HwlabRuntimeLaneSpec, type HwlabRuntimeObservabilityRecordingRuleSpec, type HwlabRuntimeObservabilitySpec, type HwlabRuntimeObservabilityWarningAlertSpec, type HwlabRuntimePublicExposureSpec, type HwlabRuntimeWebProbeAlertThresholdsSpec } from "./hwlab-node-lanes"; import { nodeWebProbeScriptRunnerSource } from "./hwlab-node-web-probe-runner-source"; import { nodeWebObserveAnalyzerSource, nodeWebObserveRunnerSource } from "./hwlab-node-web-observe-runner-source"; import { hwlabNodeHelp, hwlabNodeObservabilityHelp, hwlabNodeWebProbeHelp } from "./hwlab-node-help"; @@ -7309,6 +7309,14 @@ function webProbeCredential(secretSpec: RuntimeSecretSpec, material: BootstrapAd }; } +function nodeWebProbeAlertThresholds(spec: HwlabRuntimeLaneSpec): HwlabRuntimeWebProbeAlertThresholdsSpec { + const thresholds = spec.webProbe?.alertThresholds; + if (thresholds === undefined) { + throw new Error(`${hwlabRuntimeLaneConfigPath()} node=${spec.nodeId} lane=${spec.lane} requires webProbe.alertThresholds for web-probe observe`); + } + return thresholds; +} + interface NodeWebProbeHostProxyEnv { readonly envAssignments: string[]; readonly summary: Record; @@ -7451,6 +7459,7 @@ function runNodeWebProbeObserveStart( const stateDir = options.stateDir ?? defaultStateDir; const runnerB64 = Buffer.from(nodeWebObserveRunnerSource(), "utf8").toString("base64"); const webProbeProxy = nodeWebProbeHostProxyEnv(spec); + const alertThresholds = nodeWebProbeAlertThresholds(spec); const runnerEnvAssignments = [ ...webProbeProxy.envAssignments, `HWLAB_WEB_BASE_URL=${shellQuote(options.url)}`, @@ -7465,6 +7474,7 @@ function runNodeWebProbeObserveStart( `UNIDESK_WEB_OBSERVE_MAX_SAMPLES=${shellQuote(String(options.maxSamples))}`, `UNIDESK_WEB_OBSERVE_VIEWPORT=${shellQuote(options.viewport)}`, `UNIDESK_WEB_OBSERVE_BROWSER_PROXY_MODE=${shellQuote(options.browserProxyMode)}`, + `UNIDESK_WEB_OBSERVE_ALERT_THRESHOLDS_JSON=${shellQuote(JSON.stringify(alertThresholds))}`, ].join(" "); const script = [ "set -eu", @@ -7507,6 +7517,7 @@ function runNodeWebProbeObserveStart( workspace: spec.workspace, url: options.url, network: webProbeProxy.summary, + alertThresholds, targetPath: options.targetPath, id: observerId, credential, @@ -8017,6 +8028,7 @@ function renderWebObserveCollectTable(value: Record): string { function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec: HwlabRuntimeLaneSpec): Record | RenderedCliResult { const analyzerB64 = Buffer.from(nodeWebObserveAnalyzerSource(), "utf8").toString("base64"); + const alertThresholds = nodeWebProbeAlertThresholds(spec); const script = [ "set -eu", nodeWebObserveResolveStateDirShell(options), @@ -8033,7 +8045,8 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec "analysis_stderr=\"$state_dir/analysis/analyzer-stderr.log\"", "set +e", `UNIDESK_WEB_OBSERVE_ANALYZE_ARCHIVE_PREFIX=${shellQuote(options.analyzeArchivePrefix ?? "")}`, - "UNIDESK_WEB_OBSERVE_STATE_DIR=\"$state_dir\" UNIDESK_WEB_OBSERVE_ANALYZE_ARCHIVE_PREFIX=\"$UNIDESK_WEB_OBSERVE_ANALYZE_ARCHIVE_PREFIX\" node \"$analyzer\" >\"$analysis_stdout\" 2>\"$analysis_stderr\"", + `UNIDESK_WEB_OBSERVE_ALERT_THRESHOLDS_JSON=${shellQuote(JSON.stringify(alertThresholds))}`, + "UNIDESK_WEB_OBSERVE_STATE_DIR=\"$state_dir\" UNIDESK_WEB_OBSERVE_ANALYZE_ARCHIVE_PREFIX=\"$UNIDESK_WEB_OBSERVE_ANALYZE_ARCHIVE_PREFIX\" UNIDESK_WEB_OBSERVE_ALERT_THRESHOLDS_JSON=\"$UNIDESK_WEB_OBSERVE_ALERT_THRESHOLDS_JSON\" node \"$analyzer\" >\"$analysis_stdout\" 2>\"$analysis_stderr\"", "analyzer_exit=$?", "set -e", "report_json=\"$state_dir/analysis/report.json\"", @@ -8066,7 +8079,7 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec "const slimRound = (item) => { const v = objectOrNull(item) || {}; return { promptIndex: v.promptIndex ?? null, sampleCount: v.sampleCount ?? null, loadingSamples: v.loadingSamples ?? null, maxLoadingCount: v.maxLoadingCount ?? null, loadingOwnerCount: v.loadingOwnerCount ?? null, lastTotalElapsedSeconds: v.lastTotalElapsedSeconds ?? null, lastRecentUpdateSeconds: v.lastRecentUpdateSeconds ?? null, diagnosticSamples: v.diagnosticSamples ?? null, terminalSamples: v.terminalSamples ?? null, turnTimingTotalElapsedForwardJumpCount: v.turnTimingTotalElapsedForwardJumpCount ?? null, turnTimingTotalElapsedForwardJumpMaxSeconds: v.turnTimingTotalElapsedForwardJumpMaxSeconds ?? null, promptTextHash: clip(v.promptTextHash, 80) }; };", "const slimTurnColumn = (item) => { const v = objectOrNull(item) || {}; return { label: clip(v.label, 24), promptIndex: v.promptIndex ?? null, lastPromptIndex: v.lastPromptIndex ?? null, traceId: clip(v.traceId, 48), firstSeq: v.firstSeq ?? null, lastSeq: v.lastSeq ?? null, source: clip(v.source, 48) }; };", "const slimSlowSample = (item) => { const v = objectOrNull(item) || {}; return { ts: v.ts ?? null, seq: v.seq ?? null, path: clip(v.path ?? v.rawPath, 96), initiatorType: clip(v.initiatorType, 24), durationMs: v.durationMs ?? null, requestToResponseStartMs: v.requestToResponseStartMs ?? v.streamOpenMs ?? null, responseTransferMs: v.responseTransferMs ?? null, nextHopProtocol: clip(v.nextHopProtocol, 24), timingStatus: clip(v.timingStatus, 16), serverTimingNames: Array.isArray(v.serverTimingNames) ? v.serverTimingNames.slice(0, 4).map((x) => clip(x, 32)) : [], otelTraceId: clip(v.otelTraceId, 32) }; };", - "const slimSlowApi = (item) => { const v = objectOrNull(item) || {}; return { path: clip(v.path ?? v.route, 96), route: clip(v.route ?? v.path, 96), sampleCount: v.sampleCount ?? null, p95Ms: v.p95Ms ?? v.p95 ?? null, maxMs: v.maxMs ?? v.max ?? null, overFiveSecondCount: v.overFiveSecondCount ?? null, slowSamples: Array.isArray(v.slowSamples) ? v.slowSamples.slice(0, 3).map(slimSlowSample) : [] }; };", + "const slimSlowApi = (item) => { const v = objectOrNull(item) || {}; return { path: clip(v.path ?? v.route, 96), route: clip(v.route ?? v.path, 96), sampleCount: v.sampleCount ?? null, p95Ms: v.p95Ms ?? v.p95 ?? null, maxMs: v.maxMs ?? v.max ?? null, budgetMs: v.budgetMs ?? null, overBudgetCount: v.overBudgetCount ?? null, overFiveSecondCount: v.overFiveSecondCount ?? null, slowSamples: Array.isArray(v.slowSamples) ? v.slowSamples.slice(0, 3).map(slimSlowSample) : [] }; };", "const slimFinding = (item) => { const v = objectOrNull(item) || {}; return { kind: clip(v.kind ?? v.id ?? v.code, 48), code: clip(v.code ?? v.id ?? v.kind, 48), severity: clip(v.severity ?? v.level, 24), level: clip(v.level ?? v.severity, 24), count: v.count ?? v.sampleCount ?? null, sampleCount: v.sampleCount ?? v.count ?? null, summary: clip(v.summary ?? v.message, 180), message: clip(v.message ?? v.summary, 180) }; };", "const slimDomGroup = (item) => { const v = objectOrNull(item) || {}; return { count: v.count ?? null, firstAt: v.firstAt ?? null, lastAt: v.lastAt ?? null, text: clip(v.text ?? v.preview, 180) }; };", "const slimNetworkGroup = (item) => { const v = objectOrNull(item) || {}; return { count: v.count ?? null, method: clip(v.method, 12), status: v.status ?? null, path: clip(v.path ?? v.urlPath, 96), firstAt: v.firstAt ?? null, lastAt: v.lastAt ?? null, promptIndexes: Array.isArray(v.promptIndexes) ? v.promptIndexes.slice(0, 6) : [], failureKinds: Array.isArray(v.failureKinds) ? v.failureKinds.slice(0, 4).map((x) => clip(x, 48)) : [] }; };", @@ -8115,8 +8128,8 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec "const fullPagePerformance = objectOrNull(fullRecentWindow?.pagePerformance) || objectOrNull(fullSource?.pagePerformance);", "const fullArchivePagePerformance = objectOrNull(fullSource?.pagePerformance);", "const isLongLivedApi = (item) => item?.isLongLivedStream === true || item?.routeKind === 'same-origin-api-stream' || item?.budgetMetric === 'streamOpenMs';", - "const sourceSlowApi = Array.isArray(source?.pagePerformanceSlowApi) ? source.pagePerformanceSlowApi.filter((item) => !isLongLivedApi(item) && Number(item?.overFiveSecondCount ?? 0) > 0) : (Array.isArray(fullPagePerformance?.sameOriginApiByPath) ? fullPagePerformance.sameOriginApiByPath.filter((item) => !isLongLivedApi(item) && Number(item?.overFiveSecondCount ?? 0) > 0) : []);", - "const archiveSlowApi = firstNonEmptyArray(source?.archivePagePerformanceSlowApi, Array.isArray(fullArchivePagePerformance?.sameOriginApiByPath) ? fullArchivePagePerformance.sameOriginApiByPath : []).filter((item) => !isLongLivedApi(item) && Number(item?.overFiveSecondCount ?? 0) > 0);", + "const sourceSlowApi = Array.isArray(source?.pagePerformanceSlowApi) ? source.pagePerformanceSlowApi.filter((item) => !isLongLivedApi(item) && Number(item?.overBudgetCount ?? item?.overFiveSecondCount ?? 0) > 0) : (Array.isArray(fullPagePerformance?.sameOriginApiByPath) ? fullPagePerformance.sameOriginApiByPath.filter((item) => !isLongLivedApi(item) && Number(item?.overBudgetCount ?? item?.overFiveSecondCount ?? 0) > 0) : []);", + "const archiveSlowApi = firstNonEmptyArray(source?.archivePagePerformanceSlowApi, Array.isArray(fullArchivePagePerformance?.sameOriginApiByPath) ? fullArchivePagePerformance.sameOriginApiByPath : []).filter((item) => !isLongLivedApi(item) && Number(item?.overBudgetCount ?? item?.overFiveSecondCount ?? 0) > 0);", "const sourceSseStreams = Array.isArray(source?.pagePerformanceSseStreams) ? source.pagePerformanceSseStreams : (Array.isArray(fullPagePerformance?.sameOriginApiByPath) ? fullPagePerformance.sameOriginApiByPath.filter((item) => isLongLivedApi(item)) : []);", "const combinedFindingSource = firstNonEmptyArray(source?.findings, fullSource?.findings, fullRecentWindow?.findings, source?.archiveSummary?.redFindings, fullSource?.archiveSummary?.redFindings);", "const combinedFindings = sortFindings(combinedFindingSource);", @@ -8138,7 +8151,7 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec " promptNetwork,", " pagePerformanceSlowApi: takeHead(sourceSlowApi, 4).map(slimSlowApi),", " archivePagePerformanceSlowApi: takeHead(archiveSlowApi, 8).map(slimSlowApi),", - " pagePerformanceSseStreams: takeHead(sourceSseStreams, 4).map((item) => ({ path: item?.path ?? item?.route ?? null, route: item?.route ?? null, sampleCount: item?.sampleCount ?? null, streamOpenSampleCount: item?.streamOpenSampleCount ?? null, streamOpenP95Ms: item?.streamOpenP95Ms ?? null, streamOpenMaxMs: item?.streamOpenMaxMs ?? null, streamOpenOverFiveSecondCount: item?.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item?.streamLifetimeOverFiveSecondCount ?? null })),", + " pagePerformanceSseStreams: takeHead(sourceSseStreams, 4).map((item) => ({ path: item?.path ?? item?.route ?? null, route: item?.route ?? null, sampleCount: item?.sampleCount ?? null, streamOpenSampleCount: item?.streamOpenSampleCount ?? null, streamOpenP95Ms: item?.streamOpenP95Ms ?? null, streamOpenMaxMs: item?.streamOpenMaxMs ?? null, streamOpenBudgetMs: item?.streamOpenBudgetMs ?? null, streamOpenOverBudgetCount: item?.streamOpenOverBudgetCount ?? null, streamOpenOverFiveSecondCount: item?.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item?.streamLifetimeOverFiveSecondCount ?? null })),", " findings: takeHead(combinedFindings, 8).map(slimFinding),", " archiveRedFindings: takeHead(archiveRedFindings, 8).map(slimFinding),", " httpErrorGroups: takeHead(firstArray(source.httpErrorGroups, fullRecentWindow?.runtimeAlerts?.networkHttpErrorsByPath, fullSource?.httpErrorGroups, fullSource?.runtimeAlerts?.networkHttpErrorsByPath), 4).map(slimNetworkGroup),", @@ -8222,7 +8235,7 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec " archiveRedFindings: Array.isArray(compact.archiveRedFindings) ? compact.archiveRedFindings.slice(0, 4) : [],", " pagePerformanceSlowApi: Array.isArray(compact.pagePerformanceSlowApi) ? compact.pagePerformanceSlowApi.slice(0, 2).map((item) => ({ ...item, slowSamples: Array.isArray(item.slowSamples) ? item.slowSamples.slice(0, 1) : [] })) : [],", " archivePagePerformanceSlowApi: Array.isArray(compact.archivePagePerformanceSlowApi) ? compact.archivePagePerformanceSlowApi.slice(0, 4).map((item) => ({ ...item, slowSamples: Array.isArray(item.slowSamples) ? item.slowSamples.slice(0, 1) : [] })) : [],", - " pagePerformanceSseStreams: Array.isArray(compact.pagePerformanceSseStreams) ? compact.pagePerformanceSseStreams.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, streamOpenP95Ms: item.streamOpenP95Ms ?? null, streamOpenMaxMs: item.streamOpenMaxMs ?? null, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount ?? null })) : [],", + " pagePerformanceSseStreams: Array.isArray(compact.pagePerformanceSseStreams) ? compact.pagePerformanceSseStreams.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, streamOpenP95Ms: item.streamOpenP95Ms ?? null, streamOpenMaxMs: item.streamOpenMaxMs ?? null, streamOpenBudgetMs: item.streamOpenBudgetMs ?? null, streamOpenOverBudgetCount: item.streamOpenOverBudgetCount ?? null, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount ?? null })) : [],", " httpErrorGroups: Array.isArray(compact.httpErrorGroups) ? compact.httpErrorGroups.slice(0, 3) : [],", " requestFailedGroups: Array.isArray(compact.requestFailedGroups) ? compact.requestFailedGroups.slice(0, 3) : [],", " domDiagnosticGroups: Array.isArray(compact.domDiagnosticGroups) ? compact.domDiagnosticGroups.slice(0, 2) : [],", @@ -8276,8 +8289,8 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec " pagePerformance: compact.pagePerformance ?? null,", " findings: Array.isArray(compact.findings) ? compact.findings.slice(0, 4) : [],", " archiveRedFindings: Array.isArray(compact.archiveRedFindings) ? compact.archiveRedFindings.slice(0, 4) : [],", - " pagePerformanceSlowApi: Array.isArray(compact.pagePerformanceSlowApi) ? compact.pagePerformanceSlowApi.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? item.p95 ?? null, maxMs: item.maxMs ?? item.max ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null, slowSamples: Array.isArray(item.slowSamples) ? item.slowSamples.slice(0, 1) : [] })) : [],", - " archivePagePerformanceSlowApi: Array.isArray(compact.archivePagePerformanceSlowApi) ? compact.archivePagePerformanceSlowApi.slice(0, 4).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? item.p95 ?? null, maxMs: item.maxMs ?? item.max ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null, slowSamples: Array.isArray(item.slowSamples) ? item.slowSamples.slice(0, 1) : [] })) : [],", + " pagePerformanceSlowApi: Array.isArray(compact.pagePerformanceSlowApi) ? compact.pagePerformanceSlowApi.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? item.p95 ?? null, maxMs: item.maxMs ?? item.max ?? null, budgetMs: item.budgetMs ?? null, overBudgetCount: item.overBudgetCount ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null, slowSamples: Array.isArray(item.slowSamples) ? item.slowSamples.slice(0, 1) : [] })) : [],", + " archivePagePerformanceSlowApi: Array.isArray(compact.archivePagePerformanceSlowApi) ? compact.archivePagePerformanceSlowApi.slice(0, 4).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? item.p95 ?? null, maxMs: item.maxMs ?? item.max ?? null, budgetMs: item.budgetMs ?? null, overBudgetCount: item.overBudgetCount ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null, slowSamples: Array.isArray(item.slowSamples) ? item.slowSamples.slice(0, 1) : [] })) : [],", " pagePerformanceSseStreams: Array.isArray(compact.pagePerformanceSseStreams) ? compact.pagePerformanceSseStreams.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, route: item.route ?? item.path ?? null, sampleCount: item.sampleCount ?? null, streamOpenP95Ms: item.streamOpenP95Ms ?? null, streamOpenMaxMs: item.streamOpenMaxMs ?? null, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount ?? null })) : [],", " httpErrorGroups: Array.isArray(compact.httpErrorGroups) ? compact.httpErrorGroups.slice(0, 2) : [],", " requestFailedGroups: Array.isArray(compact.requestFailedGroups) ? compact.requestFailedGroups.slice(0, 2) : [],", @@ -8316,9 +8329,9 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec " httpErrorGroups: Array.isArray(compact.httpErrorGroups) ? compact.httpErrorGroups.slice(0, 2).map((item) => ({ count: item.count ?? null, method: item.method ?? null, status: item.status ?? null, path: item.path ?? null, lastAt: item.lastAt ?? null })) : [],", " requestFailedGroups: Array.isArray(compact.requestFailedGroups) ? compact.requestFailedGroups.slice(0, 2).map((item) => ({ count: item.count ?? null, method: item.method ?? null, path: item.path ?? null, failureKinds: item.failureKinds ?? null })) : [],", " commandFailures: Array.isArray(compact.commandFailures) ? compact.commandFailures.slice(-3).map((item) => ({ ts: item.ts ?? null, type: item.type ?? null, commandId: item.commandId ?? null, durationMs: item.durationMs ?? null, sampleSeq: item.sampleSeq ?? null, beforePath: item.beforePath ?? null, afterPath: item.afterPath ?? null, message: clip(item.message ?? item.failureKind ?? item.name, 120) })) : [],", - " pagePerformanceSlowApi: Array.isArray(compact.pagePerformanceSlowApi) ? compact.pagePerformanceSlowApi.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? null, maxMs: item.maxMs ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null })) : [],", - " archivePagePerformanceSlowApi: Array.isArray(compact.archivePagePerformanceSlowApi) ? compact.archivePagePerformanceSlowApi.slice(0, 4).map((item) => ({ path: item.path ?? item.route ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? null, maxMs: item.maxMs ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null })) : [],", - " pagePerformanceSseStreams: Array.isArray(compact.pagePerformanceSseStreams) ? compact.pagePerformanceSseStreams.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, sampleCount: item.sampleCount ?? null, streamOpenP95Ms: item.streamOpenP95Ms ?? null, streamOpenMaxMs: item.streamOpenMaxMs ?? null, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount ?? null })) : [],", + " pagePerformanceSlowApi: Array.isArray(compact.pagePerformanceSlowApi) ? compact.pagePerformanceSlowApi.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? null, maxMs: item.maxMs ?? null, budgetMs: item.budgetMs ?? null, overBudgetCount: item.overBudgetCount ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null })) : [],", + " archivePagePerformanceSlowApi: Array.isArray(compact.archivePagePerformanceSlowApi) ? compact.archivePagePerformanceSlowApi.slice(0, 4).map((item) => ({ path: item.path ?? item.route ?? null, sampleCount: item.sampleCount ?? null, p95Ms: item.p95Ms ?? null, maxMs: item.maxMs ?? null, budgetMs: item.budgetMs ?? null, overBudgetCount: item.overBudgetCount ?? null, overFiveSecondCount: item.overFiveSecondCount ?? null })) : [],", + " pagePerformanceSseStreams: Array.isArray(compact.pagePerformanceSseStreams) ? compact.pagePerformanceSseStreams.slice(0, 2).map((item) => ({ path: item.path ?? item.route ?? null, sampleCount: item.sampleCount ?? null, streamOpenP95Ms: item.streamOpenP95Ms ?? null, streamOpenMaxMs: item.streamOpenMaxMs ?? null, streamOpenBudgetMs: item.streamOpenBudgetMs ?? null, streamOpenOverBudgetCount: item.streamOpenOverBudgetCount ?? null, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount ?? null, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount ?? null })) : [],", " reportJsonPath: compact.reportJsonPath ?? reportJsonPath,", " reportJsonSha256: compact.reportJsonSha256 ?? sha256(reportJsonPath),", " reportMdPath: compact.reportMdPath ?? reportMdPath,", @@ -8348,6 +8361,7 @@ function runNodeWebProbeObserveAnalyze(options: NodeWebProbeObserveOptions, spec lane: options.lane, workspace: spec.workspace, analysis, + alertThresholds, result: analysis === null ? compactCommandResultWithStdoutTail(result) : compactCommandResult(result), valuesRedacted: true, }); @@ -8375,6 +8389,17 @@ function renderWebObserveAnalyzeTable(value: Record): string { const archivePagePerformance = record(archiveSummary?.pagePerformance); const runtimeAlerts = record(analysis?.runtimeAlerts); const pagePerformance = record(analysis?.pagePerformance); + const pagePerformanceSummary = record(pagePerformance?.summary); + const alertThresholds = record(analysis?.alertThresholds ?? pagePerformanceSummary?.alertThresholds ?? value.alertThresholds); + const budgetLabel = (rawValue: unknown, fallbackMs: number): string => { + const parsed = Number(rawValue); + const ms = Number.isFinite(parsed) && parsed > 0 ? parsed : fallbackMs; + return ms % 1000 === 0 ? `${Math.round(ms / 1000)}s` : `${ms}ms`; + }; + const sameOriginApiBudgetLabel = budgetLabel(alertThresholds?.sameOriginApiSlowMs ?? pagePerformanceSummary?.budgetMs, 10000); + const partialApiBudgetLabel = budgetLabel(alertThresholds?.partialApiSlowMs, 10000); + const streamOpenBudgetLabel = budgetLabel(alertThresholds?.longLivedStreamOpenSlowMs, 10000); + const loadingBudgetLabel = budgetLabel(alertThresholds?.visibleLoadingSlowMs, 10000); const promptNetwork = record(analysis?.promptNetwork); const loading = record(sampleMetrics?.loading); const loadingSummary = record(loading?.summary); @@ -8586,7 +8611,7 @@ function renderWebObserveAnalyzeTable(value: Record): string { ]) : [["-", "-", "-", "-", "-", "-", "-", "-", "-"]]), "", "Loading visibility:", - webObserveTable(["SAMPLES", "LOADING_SAMPLES", "MAX_COUNT", "MAX_OWNERS", "OWNERS", "LONGEST_S", "CURRENT_S", "OVER5S"], [[ + webObserveTable(["SAMPLES", "LOADING_SAMPLES", "MAX_COUNT", "MAX_OWNERS", "OWNERS", "LONGEST_S", "CURRENT_S", `OVER_${loadingBudgetLabel}`], [[ webObserveText(loadingSummary?.sampleCount ?? sampleMetrics?.sampleCount), webObserveText(loadingSummary?.loadingSampleCount ?? sampleMetrics?.loadingSampleCount), webObserveText(loadingSummary?.maxSimultaneousCount ?? sampleMetrics?.loadingMaxCount), @@ -8594,7 +8619,7 @@ function renderWebObserveAnalyzeTable(value: Record): string { webObserveText(loadingSummary?.ownerCount ?? sampleMetrics?.loadingOwnerCount), webObserveText(loadingSummary?.longestContinuousSeconds ?? sampleMetrics?.loadingLongestContinuousSeconds), webObserveText(loadingSummary?.currentContinuousSeconds ?? sampleMetrics?.loadingCurrentContinuousSeconds), - webObserveText(loadingSummary?.overFiveSecondSegmentCount ?? sampleMetrics?.loadingOverFiveSecondSegmentCount), + webObserveText(loadingSummary?.overBudgetSegmentCount ?? loadingSummary?.overFiveSecondSegmentCount ?? sampleMetrics?.loadingOverFiveSecondSegmentCount), ]]), "", "Session rail titles:", @@ -8702,16 +8727,16 @@ function renderWebObserveAnalyzeTable(value: Record): string { "Request failed groups:", webObserveTable(["COUNT", "METHOD", "STATUS", "PATH", "LAST", "FAILURE"], requestFailedRows.length > 0 ? requestFailedRows : [["-", "-", "-", "-", "-", "-"]]), "", - "Slow API (>5s):", - webObserveTable(["PATH", "SAMPLES", "P95", "MAX", "OVER5S"], slowApis.length > 0 ? slowApis.map((item) => [ + `Slow API (>${sameOriginApiBudgetLabel}):`, + webObserveTable(["PATH", "SAMPLES", "P95", "MAX", "OVER_BUDGET"], slowApis.length > 0 ? slowApis.map((item) => [ webObserveShort(webObserveText(item.path ?? item.route), 52), webObserveText(item.sampleCount), webObserveText(item.p95Ms ?? item.p95), webObserveText(item.maxMs ?? item.max), - webObserveText(item.overFiveSecondCount), + webObserveText(item.overBudgetCount ?? item.overFiveSecondCount), ]) : [["-", "-", "-", "-", "-"]]), "", - "Slow API samples (>5s):", + `Slow API samples (>${sameOriginApiBudgetLabel}):`, webObserveTable(["TS", "SEQ", "PATH", "DURATION", "REQ_WAIT", "RESP_XFER", "TIMING", "INIT", "PROTO", "SERVER", "OTEL"], (() => { const rows = slowApis.flatMap((item) => (Array.isArray(item.slowSamples) ? item.slowSamples : []).map((sample) => ({ ...sample, path: sample.path ?? item.path ?? item.route }))).slice(0, 8).map((sample) => [ webObserveShort(webObserveText(sample.ts), 24), @@ -8729,13 +8754,13 @@ function renderWebObserveAnalyzeTable(value: Record): string { return rows.length > 0 ? rows : [["-", "-", "-", "-", "-", "-", "-", "-", "-", "-", "-"]]; })()), "", - "Partial API timing (>5s, not counted as completed slow API):", - webObserveTable(["PATH", "SAMPLES", "COMPLETE", "PARTIAL", "PARTIAL>5S"], partialApis.length > 0 ? partialApis.map((item) => [ + `Partial API timing (>${partialApiBudgetLabel}, not counted as completed slow API):`, + webObserveTable(["PATH", "SAMPLES", "COMPLETE", "PARTIAL", "PARTIAL_OVER_BUDGET"], partialApis.length > 0 ? partialApis.map((item) => [ webObserveShort(webObserveText(item.path ?? item.route), 52), webObserveText(item.sampleCount), webObserveText(item.completeTimingSampleCount), webObserveText(item.partialTimingSampleCount), - webObserveText(item.partialOverFiveSecondCount), + webObserveText(item.partialOverBudgetCount ?? item.partialOverFiveSecondCount), ]) : [["-", "-", "-", "-", "-"]]), "", "Partial API samples:", @@ -8753,12 +8778,12 @@ function renderWebObserveAnalyzeTable(value: Record): string { })()), "", "Long-lived streams (SSE):", - webObserveTable(["PATH", "SAMPLES", "OPEN_P95", "OPEN_MAX", "OPEN>5S", "LIFE>5S"], sseStreams.length > 0 ? sseStreams.map((item) => [ + webObserveTable(["PATH", "SAMPLES", "OPEN_P95", "OPEN_MAX", `OPEN>${streamOpenBudgetLabel}`, "LIFE>5S"], sseStreams.length > 0 ? sseStreams.map((item) => [ webObserveShort(webObserveText(item.path ?? item.route), 52), webObserveText(item.sampleCount), webObserveText(item.streamOpenP95Ms), webObserveText(item.streamOpenMaxMs), - webObserveText(item.streamOpenOverFiveSecondCount), + webObserveText(item.streamOpenOverBudgetCount ?? item.streamOpenOverFiveSecondCount), webObserveText(item.streamLifetimeOverFiveSecondCount), ]) : [["-", "-", "-", "-", "-", "-"]]), "", @@ -8770,16 +8795,16 @@ function renderWebObserveAnalyzeTable(value: Record): string { webObserveText(archivePagePerformance?.worstP95Ms), ]]), "", - "Archive slow API (>5s):", - webObserveTable(["PATH", "SAMPLES", "P95", "MAX", "OVER5S"], archiveSlowApis.length > 0 ? archiveSlowApis.map((item) => [ + `Archive slow API (>${sameOriginApiBudgetLabel}):`, + webObserveTable(["PATH", "SAMPLES", "P95", "MAX", "OVER_BUDGET"], archiveSlowApis.length > 0 ? archiveSlowApis.map((item) => [ webObserveShort(webObserveText(item.path ?? item.route), 52), webObserveText(item.sampleCount), webObserveText(item.p95Ms ?? item.p95), webObserveText(item.maxMs ?? item.max), - webObserveText(item.overFiveSecondCount), + webObserveText(item.overBudgetCount ?? item.overFiveSecondCount), ]) : [["-", "-", "-", "-", "-"]]), "", - "Archive slow API samples (>5s):", + `Archive slow API samples (>${sameOriginApiBudgetLabel}):`, webObserveTable(["TS", "SEQ", "PATH", "DURATION", "REQ_WAIT", "TIMING", "OTEL"], (() => { const rows = archiveSlowApis.flatMap((item) => (Array.isArray(item.slowSamples) ? item.slowSamples : []).map((sample) => ({ ...sample, path: sample.path ?? item.path ?? item.route }))).slice(0, 8).map((sample) => [ webObserveShort(webObserveText(sample.ts), 24), @@ -9365,7 +9390,7 @@ function renderWebObserveAnalyzeResult(result: Record): Record< [ ["sameOriginApiPaths", pagePerformance.sameOriginApiPathCount], ["longLivedStreamPaths", pagePerformance.longLivedStreamPathCount], - ["streamOpenOver5Samples", pagePerformance.longLivedStreamOpenOverFiveSecondSampleCount], + ["streamOpenOverBudgetSamples", pagePerformance.longLivedStreamOpenOverBudgetSampleCount ?? pagePerformance.longLivedStreamOpenOverFiveSecondSampleCount], ["slowPathCount", pagePerformance.slowPathCount], ["slowSampleCount", pagePerformance.slowSampleCount], ["worstP95Ms", pagePerformance.worstP95Ms], @@ -9375,15 +9400,15 @@ function renderWebObserveAnalyzeResult(result: Record): Record< slowApis.length === 0 ? "SLOW_API\n-" : webObserveTable( - ["SLOW_API", "P50", "P75", "P95", ">5S", "COUNT"], - slowApis.map((item) => [item.path, item.p50Ms, item.p75Ms, item.p95Ms, item.overFiveSecondCount, item.sampleCount]), + ["SLOW_API", "P50", "P75", "P95", "OVER_BUDGET", "COUNT"], + slowApis.map((item) => [item.path, item.p50Ms, item.p75Ms, item.p95Ms, item.overBudgetCount ?? item.overFiveSecondCount, item.sampleCount]), ), "", sseStreams.length === 0 ? "SSE_STREAMS\n-" : webObserveTable( - ["SSE_STREAM", "OPEN_P95", "OPEN_MAX", "OPEN>5S", "LIFE>5S", "COUNT"], - sseStreams.map((item) => [item.path, item.streamOpenP95Ms, item.streamOpenMaxMs, item.streamOpenOverFiveSecondCount, item.streamLifetimeOverFiveSecondCount, item.sampleCount]), + ["SSE_STREAM", "OPEN_P95", "OPEN_MAX", "OPEN_OVER_BUDGET", "LIFE>5S", "COUNT"], + sseStreams.map((item) => [item.path, item.streamOpenP95Ms, item.streamOpenMaxMs, item.streamOpenOverBudgetCount ?? item.streamOpenOverFiveSecondCount, item.streamLifetimeOverFiveSecondCount, item.sampleCount]), ), "", findings.length === 0 diff --git a/scripts/src/hwlab-node-lanes.ts b/scripts/src/hwlab-node-lanes.ts index 4ffb388d..bd20a47b 100644 --- a/scripts/src/hwlab-node-lanes.ts +++ b/scripts/src/hwlab-node-lanes.ts @@ -125,6 +125,16 @@ export type HwlabRuntimeWebProbeOriginSpec = HwlabRuntimeWebProbeServiceOriginSp export interface HwlabRuntimeWebProbeSpec { readonly browserProxyMode?: "auto" | "direct"; readonly defaultOrigin?: HwlabRuntimeWebProbeOriginSpec; + readonly alertThresholds?: HwlabRuntimeWebProbeAlertThresholdsSpec; +} + +export interface HwlabRuntimeWebProbeAlertThresholdsSpec { + readonly sameOriginApiSlowMs: number; + readonly partialApiSlowMs: number; + readonly longLivedStreamOpenSlowMs: number; + readonly visibleLoadingSlowMs: number; + readonly turnTimingSampleSlackSeconds: number; + readonly sessionRailFallbackRatio: number; } export interface HwlabRuntimeBuildkitSpec { @@ -692,6 +702,21 @@ function webProbeConfig(value: unknown, path: string): HwlabRuntimeWebProbeSpec return { ...(browserProxyMode === undefined ? {} : { browserProxyMode }), ...(raw.defaultOrigin === undefined ? {} : { defaultOrigin: webProbeOriginConfig(raw.defaultOrigin, `${path}.defaultOrigin`) }), + ...(raw.alertThresholds === undefined ? {} : { alertThresholds: webProbeAlertThresholdsConfig(raw.alertThresholds, `${path}.alertThresholds`) }), + }; +} + +function webProbeAlertThresholdsConfig(value: unknown, path: string): HwlabRuntimeWebProbeAlertThresholdsSpec { + const raw = asRecord(value, path); + const sessionRailFallbackRatio = positiveNumberField(raw, "sessionRailFallbackRatio", path); + if (sessionRailFallbackRatio > 1) throw new Error(`${path}.sessionRailFallbackRatio must be <= 1`); + return { + sameOriginApiSlowMs: positiveNumberField(raw, "sameOriginApiSlowMs", path), + partialApiSlowMs: positiveNumberField(raw, "partialApiSlowMs", path), + longLivedStreamOpenSlowMs: positiveNumberField(raw, "longLivedStreamOpenSlowMs", path), + visibleLoadingSlowMs: positiveNumberField(raw, "visibleLoadingSlowMs", path), + turnTimingSampleSlackSeconds: positiveNumberField(raw, "turnTimingSampleSlackSeconds", path), + sessionRailFallbackRatio, }; } diff --git a/scripts/src/hwlab-node-web-observe-runner-source.ts b/scripts/src/hwlab-node-web-observe-runner-source.ts index 64d9fdec..20b1f76d 100644 --- a/scripts/src/hwlab-node-web-observe-runner-source.ts +++ b/scripts/src/hwlab-node-web-observe-runner-source.ts @@ -23,6 +23,7 @@ const maxSamples = positiveInteger(process.env.UNIDESK_WEB_OBSERVE_MAX_SAMPLES, const observerRefreshIntervalMs = positiveInteger(process.env.UNIDESK_WEB_OBSERVE_OBSERVER_REFRESH_INTERVAL_MS, 180000); const viewport = parseViewport(process.env.UNIDESK_WEB_OBSERVE_VIEWPORT || "1440x900"); const browserProxyMode = parseBrowserProxyMode(process.env.UNIDESK_WEB_OBSERVE_BROWSER_PROXY_MODE || "auto"); +const alertThresholds = parseAlertThresholds(process.env.UNIDESK_WEB_OBSERVE_ALERT_THRESHOLDS_JSON); const playwrightProxy = proxyConfigFromEnv(baseUrl); const chromiumLaunchOptions = chromiumLaunchOptionsForProxy(playwrightProxy); const pageId = "control-" + randomBytes(4).toString("hex"); @@ -172,6 +173,7 @@ async function writeManifest(extra = {}) { pageAuthority: { browser: "chromium", context: "shared-auth", pageMode: "dual-control-observer", controlPageId: pageId, observerPageId, continuityBreaksRecorded: true }, pageProvenance: compactPageProvenance(currentPageProvenance), sampling: { mode: "passive", sampleIntervalMs, screenshotIntervalMs, maxSamples, observerRefreshIntervalMs, observerInitiatedDefault: false, responseBodyReadDefault: false }, + alertThresholds, jsonlRotation, commandDirs: dirs, artifacts: files, @@ -1874,6 +1876,35 @@ function positiveInteger(value, fallback) { return Number.isFinite(parsed) && parsed >= 0 ? Math.floor(parsed) : fallback; } +function positiveNumber(value, fallback) { + const parsed = Number(value); + return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback; +} + +function parseAlertThresholds(value) { + const raw = (() => { + try { return value ? JSON.parse(value) : {}; } catch { return {}; } + })(); + const fallback = { + sameOriginApiSlowMs: 10000, + partialApiSlowMs: 10000, + longLivedStreamOpenSlowMs: 10000, + visibleLoadingSlowMs: 10000, + turnTimingSampleSlackSeconds: 3, + sessionRailFallbackRatio: 0.5, + }; + const sessionRailFallbackRatio = positiveNumber(raw.sessionRailFallbackRatio, fallback.sessionRailFallbackRatio); + return { + sameOriginApiSlowMs: positiveNumber(raw.sameOriginApiSlowMs, fallback.sameOriginApiSlowMs), + partialApiSlowMs: positiveNumber(raw.partialApiSlowMs, fallback.partialApiSlowMs), + longLivedStreamOpenSlowMs: positiveNumber(raw.longLivedStreamOpenSlowMs, fallback.longLivedStreamOpenSlowMs), + visibleLoadingSlowMs: positiveNumber(raw.visibleLoadingSlowMs, fallback.visibleLoadingSlowMs), + turnTimingSampleSlackSeconds: positiveNumber(raw.turnTimingSampleSlackSeconds, fallback.turnTimingSampleSlackSeconds), + sessionRailFallbackRatio: Math.min(1, sessionRailFallbackRatio), + source: value ? "yaml-env" : "runner-default", + }; +} + function sha256Text(value) { return "sha256:" + createHash("sha256").update(String(value)).digest("hex"); } @@ -1926,6 +1957,7 @@ import { createInterface } from "node:readline"; const stateDir = path.resolve(process.env.UNIDESK_WEB_OBSERVE_STATE_DIR || process.argv[2] || ".state/web-observe/manual"); const archivePrefix = safeArchivePrefix(process.env.UNIDESK_WEB_OBSERVE_ANALYZE_ARCHIVE_PREFIX || ""); +const alertThresholds = parseAlertThresholds(process.env.UNIDESK_WEB_OBSERVE_ALERT_THRESHOLDS_JSON); const dataDir = archivePrefix ? path.join(stateDir, "archive") : stateDir; const dataFile = (name) => path.join(dataDir, archivePrefix ? archivePrefix + "-" + name : name); const analysisDir = path.join(stateDir, "analysis"); @@ -1999,6 +2031,7 @@ const report = { generatedAt: new Date().toISOString(), stateDir, jsonlScope: { mode: archivePrefix ? "archive" : "current", archivePrefix: archivePrefix || null, dataDir, valuesRedacted: true }, + alertThresholds, manifest: compactManifest(manifest), heartbeat: compactHeartbeat(heartbeat), counts: { samples: samples.length, control: control.length, network: network.length, console: consoleEvents.length, errors: errors.length, artifacts: artifacts.length }, @@ -2159,10 +2192,10 @@ console.log(JSON.stringify({ excessiveIncreaseSeconds: item.excessiveIncreaseSeconds ?? null, anomaly: item.anomaly ?? null, })), - pagePerformanceSlowApi: recentWindow.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true && item.overFiveSecondCount > 0).slice(0, 5).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, p95Ms: item.p95Ms, maxMs: item.maxMs, overFiveSecondCount: item.overFiveSecondCount, slowSamples: item.slowSamples })), - archivePagePerformanceSlowApi: pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true && item.overFiveSecondCount > 0).slice(0, 8).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, p95Ms: item.p95Ms, maxMs: item.maxMs, overFiveSecondCount: item.overFiveSecondCount, slowSamples: item.slowSamples })), - pagePerformancePartialApi: recentWindow.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true && Number(item.partialOverFiveSecondCount ?? 0) > 0).slice(0, 5).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, completeTimingSampleCount: item.completeTimingSampleCount, partialTimingSampleCount: item.partialTimingSampleCount, partialOverFiveSecondCount: item.partialOverFiveSecondCount, partialSamples: item.partialSamples })), - pagePerformanceSseStreams: recentWindow.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream === true).slice(0, 5).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, streamOpenSampleCount: item.streamOpenSampleCount, streamOpenP95Ms: item.streamOpenP95Ms, streamOpenMaxMs: item.streamOpenMaxMs, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount, slowSamples: item.slowSamples })), + pagePerformanceSlowApi: recentWindow.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true && Number(item.overBudgetCount ?? item.overFiveSecondCount ?? 0) > 0).slice(0, 5).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, p95Ms: item.p95Ms, maxMs: item.maxMs, overBudgetCount: item.overBudgetCount, budgetMs: item.budgetMs, overFiveSecondCount: item.overFiveSecondCount, slowSamples: item.slowSamples })), + archivePagePerformanceSlowApi: pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true && Number(item.overBudgetCount ?? item.overFiveSecondCount ?? 0) > 0).slice(0, 8).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, p95Ms: item.p95Ms, maxMs: item.maxMs, overBudgetCount: item.overBudgetCount, budgetMs: item.budgetMs, overFiveSecondCount: item.overFiveSecondCount, slowSamples: item.slowSamples })), + pagePerformancePartialApi: recentWindow.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true && Number(item.partialOverBudgetCount ?? item.partialOverFiveSecondCount ?? 0) > 0).slice(0, 5).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, completeTimingSampleCount: item.completeTimingSampleCount, partialTimingSampleCount: item.partialTimingSampleCount, partialOverBudgetCount: item.partialOverBudgetCount, budgetMs: item.partialBudgetMs, partialOverFiveSecondCount: item.partialOverFiveSecondCount, partialSamples: item.partialSamples })), + pagePerformanceSseStreams: recentWindow.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream === true).slice(0, 5).map((item) => ({ path: item.path, route: item.route, sampleCount: item.sampleCount, streamOpenSampleCount: item.streamOpenSampleCount, streamOpenP95Ms: item.streamOpenP95Ms, streamOpenMaxMs: item.streamOpenMaxMs, streamOpenOverBudgetCount: item.streamOpenOverBudgetCount, streamOpenBudgetMs: item.streamOpenBudgetMs, streamOpenOverFiveSecondCount: item.streamOpenOverFiveSecondCount, streamLifetimeOverFiveSecondCount: item.streamLifetimeOverFiveSecondCount, slowSamples: item.slowSamples })), findings: prioritizeFindings(recentWindow.findings).slice(0, 8).map((item) => ({ kind: item.id ?? item.kind ?? item.code, severity: item.severity, count: item.count ?? item.sampleCount ?? null, summary: String(item.summary ?? item.message ?? "").slice(0, 180) })), valuesRedacted: true, })); @@ -2178,6 +2211,35 @@ function safeArchivePrefix(value) { return text; } +function positiveNumber(value, fallback) { + const parsed = Number(value); + return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback; +} + +function parseAlertThresholds(value) { + const raw = (() => { + try { return value ? JSON.parse(value) : {}; } catch { return {}; } + })(); + const fallback = { + sameOriginApiSlowMs: 10000, + partialApiSlowMs: 10000, + longLivedStreamOpenSlowMs: 10000, + visibleLoadingSlowMs: 10000, + turnTimingSampleSlackSeconds: 3, + sessionRailFallbackRatio: 0.5, + }; + const sessionRailFallbackRatio = positiveNumber(raw.sessionRailFallbackRatio, fallback.sessionRailFallbackRatio); + return { + sameOriginApiSlowMs: positiveNumber(raw.sameOriginApiSlowMs, fallback.sameOriginApiSlowMs), + partialApiSlowMs: positiveNumber(raw.partialApiSlowMs, fallback.partialApiSlowMs), + longLivedStreamOpenSlowMs: positiveNumber(raw.longLivedStreamOpenSlowMs, fallback.longLivedStreamOpenSlowMs), + visibleLoadingSlowMs: positiveNumber(raw.visibleLoadingSlowMs, fallback.visibleLoadingSlowMs), + turnTimingSampleSlackSeconds: positiveNumber(raw.turnTimingSampleSlackSeconds, fallback.turnTimingSampleSlackSeconds), + sessionRailFallbackRatio: Math.min(1, sessionRailFallbackRatio), + source: value ? "yaml-env" : "analyzer-default", + }; +} + async function readJsonl(file, options = {}) { const rows = []; try { @@ -2469,10 +2531,11 @@ function buildFindings(samples, control, network, errors, sampleMetrics, promptN : []; if (recentUpdateSawtoothJumps.length > 0) findings.push({ id: "turn-timing-recent-update-sawtooth-jump", severity: "amber", summary: "最近更新 value jumped faster than sample interval; expected sawtooth increase-or-reset", count: recentUpdateSawtoothJumps.length, samples: recentUpdateSawtoothJumps.slice(0, 20) }); const loadingSummary = sampleMetrics?.loading?.summary || {}; - if (Number(loadingSummary.longestContinuousSeconds ?? 0) > 5) findings.push({ id: "page-loading-visible-over-5s", severity: "red", summary: "visible 加载中 stayed on screen longer than 5s; fix real loading latency instead of revealing incomplete content early", count: loadingSummary.overFiveSecondSegmentCount ?? 1, longestContinuousSeconds: loadingSummary.longestContinuousSeconds, segments: sampleMetrics.loading.segments.slice(0, 20), owners: sampleMetrics.loading.owners.slice(0, 20) }); + const visibleLoadingSlowSeconds = alertThresholds.visibleLoadingSlowMs / 1000; + if (Number(loadingSummary.longestContinuousSeconds ?? 0) > visibleLoadingSlowSeconds) findings.push({ id: "page-loading-visible-over-budget", severity: "red", summary: "visible 加载中 stayed on screen longer than configured YAML budget; fix real loading latency instead of revealing incomplete content early", count: loadingSummary.overBudgetSegmentCount ?? loadingSummary.overFiveSecondSegmentCount ?? 1, longestContinuousSeconds: loadingSummary.longestContinuousSeconds, budgetSeconds: visibleLoadingSlowSeconds, segments: sampleMetrics.loading.segments.slice(0, 20), owners: sampleMetrics.loading.owners.slice(0, 20) }); if (Number(loadingSummary.maxSimultaneousCount ?? 0) > 1) findings.push({ id: "page-loading-concurrent", severity: "info", summary: "multiple 加载中 indicators were visible in the same sampled DOM point", count: loadingSummary.concurrentLoadingSampleCount ?? 0, maxSimultaneousCount: loadingSummary.maxSimultaneousCount, owners: sampleMetrics.loading.owners.slice(0, 20) }); const sessionRailTitleSummary = sampleMetrics?.sessionRailTitles?.summary || {}; - if (Number(sessionRailTitleSummary.majorityFallbackSampleCount ?? 0) > 0) findings.push({ id: "session-rail-title-fallback-majority", severity: "red", summary: "more than half of visible session list rows used fallback Session ses_... titles", count: sessionRailTitleSummary.majorityFallbackSampleCount, maxFallbackRatio: sessionRailTitleSummary.maxFallbackRatio, maxFallbackTitleCount: sessionRailTitleSummary.maxFallbackTitleCount, samples: sampleMetrics.sessionRailTitles.samples.slice(0, 20), examples: sampleMetrics.sessionRailTitles.examples.slice(0, 20) }); + if (Number(sessionRailTitleSummary.overThresholdSampleCount ?? sessionRailTitleSummary.majorityFallbackSampleCount ?? 0) > 0) findings.push({ id: "session-rail-title-fallback-over-threshold", severity: "red", summary: "visible session list rows exceeded configured YAML fallback-title ratio", count: sessionRailTitleSummary.overThresholdSampleCount ?? sessionRailTitleSummary.majorityFallbackSampleCount, thresholdRatio: alertThresholds.sessionRailFallbackRatio, maxFallbackRatio: sessionRailTitleSummary.maxFallbackRatio, maxFallbackTitleCount: sessionRailTitleSummary.maxFallbackTitleCount, samples: sampleMetrics.sessionRailTitles.samples.slice(0, 20), examples: sampleMetrics.sessionRailTitles.examples.slice(0, 20) }); if ((runtimeAlerts?.summary?.httpErrorCount ?? 0) > 0) findings.push({ id: "runtime-http-errors", severity: "amber", summary: "natural page requests returned HTTP error status during observation", count: runtimeAlerts.summary.httpErrorCount, groups: runtimeAlerts.networkHttpErrorsByPath.slice(0, 12) }); if ((runtimeAlerts?.summary?.significantRequestFailedCount ?? runtimeAlerts?.summary?.requestFailedCount ?? 0) > 0) findings.push({ id: "runtime-requestfailed", severity: "amber", summary: "browser requestfailed events were captured during observation", count: runtimeAlerts.summary.significantRequestFailedCount ?? runtimeAlerts.summary.requestFailedCount, groups: (runtimeAlerts.networkSignificantRequestFailedByPath ?? runtimeAlerts.networkRequestFailedByPath).slice(0, 12) }); if ((runtimeAlerts?.summary?.domDiagnosticSampleCount ?? 0) > 0) findings.push({ id: "runtime-dom-diagnostics", severity: "amber", summary: "diagnostic/error/warning-like text was visible in sampled DOM", count: runtimeAlerts.summary.domDiagnosticSampleCount, groupCount: runtimeAlerts.summary.domDiagnosticGroupCount ?? 0, groups: runtimeAlerts.domDiagnosticsByText.slice(0, 12), samples: runtimeAlerts.domDiagnostics.slice(0, 12) }); @@ -2491,11 +2554,11 @@ function buildFindings(samples, control, network, errors, sampleMetrics, promptN const turnTraceMissing = detectTurnTraceIdMissing(samples); if (turnTraceMissing.length > 0) findings.push({ id: "turn-trace-id-missing", severity: "red", summary: "Code Agent turn/card was visible without a trace id, so historical trace hydration cannot be reliable", count: turnTraceMissing.length, samples: turnTraceMissing.slice(0, 20) }); const pagePerformanceItems = Array.isArray(pagePerformance?.sameOriginApiByPath) ? pagePerformance.sameOriginApiByPath : []; - const slowApi = pagePerformanceItems.filter((item) => item.isLongLivedStream !== true && item.overFiveSecondCount > 0); - if (slowApi.length > 0) findings.push({ id: "page-performance-slow-same-origin-api", severity: "red", summary: "same-origin API resource timing exceeded 5s usability budget", count: slowApi.length, groups: slowApi.slice(0, 20) }); + const slowApi = pagePerformanceItems.filter((item) => item.isLongLivedStream !== true && Number(item.overBudgetCount ?? item.overFiveSecondCount ?? 0) > 0); + if (slowApi.length > 0) findings.push({ id: "page-performance-slow-same-origin-api", severity: "red", summary: "same-origin API resource timing exceeded configured YAML usability budget", count: slowApi.length, budgetMs: alertThresholds.sameOriginApiSlowMs, groups: slowApi.slice(0, 20) }); const longLivedStreams = pagePerformanceItems.filter((item) => item.isLongLivedStream); - const slowStreamOpen = longLivedStreams.filter((item) => Number(item.streamOpenOverFiveSecondCount ?? 0) > 0); - if (slowStreamOpen.length > 0) findings.push({ id: "page-performance-slow-long-lived-stream-open", severity: "red", summary: "long-lived stream open latency exceeded 5s usability budget; stream lifetime is still reported separately", count: slowStreamOpen.length, groups: slowStreamOpen.slice(0, 20) }); + const slowStreamOpen = longLivedStreams.filter((item) => Number(item.streamOpenOverBudgetCount ?? item.streamOpenOverFiveSecondCount ?? 0) > 0); + if (slowStreamOpen.length > 0) findings.push({ id: "page-performance-slow-long-lived-stream-open", severity: "red", summary: "long-lived stream open latency exceeded configured YAML usability budget; stream lifetime is still reported separately", count: slowStreamOpen.length, budgetMs: alertThresholds.longLivedStreamOpenSlowMs, groups: slowStreamOpen.slice(0, 20) }); if (longLivedStreams.length > 0) findings.push({ id: "page-performance-long-lived-streams", severity: "info", summary: "same-origin long-lived streams are reported separately; lifetime is not treated as API load latency", count: longLivedStreams.length, groups: longLivedStreams.slice(0, 20) }); if ((pageProvenance?.summary?.segmentCount ?? 0) > 1) findings.push({ id: "page-provenance-segments", severity: "info", summary: "observer crossed page asset provenance segments; interpret runtime findings by segment", segmentCount: pageProvenance.summary.segmentCount, segments: pageProvenance.segments.slice(0, 20) }); const naturalApi = network.filter((item) => item.observerInitiated === false && item.type === "response" && /\/v1\/|\/auth\//u.test(String(item.url || ""))); @@ -2647,9 +2710,12 @@ function buildPagePerformanceReport(samples, manifest) { durationsMs: [], streamOpenDurationsMs: [], overFiveSecondCount: 0, + overBudgetCount: 0, partialOverFiveSecondCount: 0, + partialOverBudgetCount: 0, streamLifetimeOverFiveSecondCount: 0, streamOpenOverFiveSecondCount: 0, + streamOpenOverBudgetCount: 0, firstAt: entryTs, lastAt: entryTs, firstSeq: sample.seq ?? null, @@ -2665,8 +2731,9 @@ function buildPagePerformanceReport(samples, manifest) { let overBudget = false; if (partialOrdinaryTiming) { group.partialTimingSampleCount += 1; - if (durationMs > 5000) { - group.partialOverFiveSecondCount += 1; + if (durationMs > 5000) group.partialOverFiveSecondCount += 1; + if (durationMs > alertThresholds.partialApiSlowMs) { + group.partialOverBudgetCount += 1; if (group.partialSamples.length < 80) group.partialSamples.push(compactPagePerformanceSlowSample({ sample, entry, entryTs, normalizedPath, rawPath: parsed.path, durationMs, streamOpenMs })); } } else { @@ -2679,12 +2746,19 @@ function buildPagePerformanceReport(samples, manifest) { if (streamOpenMs > 5000) { group.streamOpenOverFiveSecondCount += 1; group.overFiveSecondCount += 1; + } + if (streamOpenMs > alertThresholds.longLivedStreamOpenSlowMs) { + group.streamOpenOverBudgetCount += 1; + group.overBudgetCount += 1; overBudget = true; } } - } else if (durationMs > 5000) { - group.overFiveSecondCount += 1; - overBudget = true; + } else { + if (durationMs > 5000) group.overFiveSecondCount += 1; + if (durationMs > alertThresholds.sameOriginApiSlowMs) { + group.overBudgetCount += 1; + overBudget = true; + } } } if (overBudget && group.slowSamples.length < 80) group.slowSamples.push(compactPagePerformanceSlowSample({ sample, entry, entryTs, normalizedPath, rawPath: parsed.path, durationMs, streamOpenMs })); @@ -2706,6 +2780,9 @@ function buildPagePerformanceReport(samples, manifest) { isLongLivedStream: group.isLongLivedStream === true, budgetMetric: group.budgetMetric, sampleCount: group.sampleCount, + budgetMs: group.isLongLivedStream === true ? alertThresholds.longLivedStreamOpenSlowMs : alertThresholds.sameOriginApiSlowMs, + partialBudgetMs: alertThresholds.partialApiSlowMs, + streamOpenBudgetMs: alertThresholds.longLivedStreamOpenSlowMs, completeTimingSampleCount: group.completeTimingSampleCount, partialTimingSampleCount: group.partialTimingSampleCount, p50Ms: percentile(durations, 50), @@ -2718,10 +2795,14 @@ function buildPagePerformanceReport(samples, manifest) { streamOpenP95Ms: percentile(streamOpenDurations, 95), streamOpenMaxMs: streamOpenDurations.length > 0 ? streamOpenDurations[streamOpenDurations.length - 1] : null, streamOpenOverFiveSecondCount: group.streamOpenOverFiveSecondCount, + streamOpenOverBudgetCount: group.streamOpenOverBudgetCount, streamLifetimeOverFiveSecondCount: group.streamLifetimeOverFiveSecondCount, overFiveSecondCount: group.overFiveSecondCount, + overBudgetCount: group.overBudgetCount, partialOverFiveSecondCount: group.partialOverFiveSecondCount, + partialOverBudgetCount: group.partialOverBudgetCount, overFiveSecondRatio: group.sampleCount > 0 ? Number((group.overFiveSecondCount / group.sampleCount).toFixed(3)) : 0, + overBudgetRatio: group.sampleCount > 0 ? Number((group.overBudgetCount / group.sampleCount).toFixed(3)) : 0, firstAt: group.firstAt, lastAt: group.lastAt, firstSeq: group.firstSeq, @@ -2739,30 +2820,40 @@ function buildPagePerformanceReport(samples, manifest) { .slice(0, 12), valuesRedacted: true }; - }).sort((a, b) => (b.overFiveSecondCount - a.overFiveSecondCount) || (Number(b.p95Ms ?? 0) - Number(a.p95Ms ?? 0)) || a.path.localeCompare(b.path)); + }).sort((a, b) => (Number(b.overBudgetCount ?? b.overFiveSecondCount ?? 0) - Number(a.overBudgetCount ?? a.overFiveSecondCount ?? 0)) || (Number(b.p95Ms ?? 0) - Number(a.p95Ms ?? 0)) || a.path.localeCompare(b.path)); const longLivedStreams = sameOriginApiByPath.filter((item) => item.isLongLivedStream); const ordinaryApi = sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true); - const slow = ordinaryApi.filter((item) => item.overFiveSecondCount > 0); - const partialSlow = ordinaryApi.filter((item) => Number(item.partialOverFiveSecondCount ?? 0) > 0); - const slowStreamOpen = longLivedStreams.filter((item) => Number(item.streamOpenOverFiveSecondCount ?? 0) > 0); + const slow = ordinaryApi.filter((item) => Number(item.overBudgetCount ?? item.overFiveSecondCount ?? 0) > 0); + const slowFiveSecond = ordinaryApi.filter((item) => Number(item.overFiveSecondCount ?? 0) > 0); + const partialSlow = ordinaryApi.filter((item) => Number(item.partialOverBudgetCount ?? item.partialOverFiveSecondCount ?? 0) > 0); + const partialFiveSecond = ordinaryApi.filter((item) => Number(item.partialOverFiveSecondCount ?? 0) > 0); + const slowStreamOpen = longLivedStreams.filter((item) => Number(item.streamOpenOverBudgetCount ?? item.streamOpenOverFiveSecondCount ?? 0) > 0); + const slowStreamOpenFiveSecond = longLivedStreams.filter((item) => Number(item.streamOpenOverFiveSecondCount ?? 0) > 0); const budgetP95Values = sameOriginApiByPath .map((item) => Number(item.isLongLivedStream ? (item.streamOpenP95Ms ?? 0) : (item.p95Ms ?? 0))) .filter((value) => Number.isFinite(value)); return { summary: { - budgetMs: 5000, + budgetMs: alertThresholds.sameOriginApiSlowMs, + alertThresholds, sameOriginApiPathCount: sameOriginApiByPath.length, sameOriginApiSampleCount: sameOriginApiByPath.reduce((sum, item) => sum + item.sampleCount, 0), longLivedStreamPathCount: longLivedStreams.length, longLivedStreamSampleCount: longLivedStreams.reduce((sum, item) => sum + item.sampleCount, 0), - longLivedStreamOpenOverFiveSecondPathCount: slowStreamOpen.length, - longLivedStreamOpenOverFiveSecondSampleCount: slowStreamOpen.reduce((sum, item) => sum + Number(item.streamOpenOverFiveSecondCount ?? 0), 0), + longLivedStreamOpenOverFiveSecondPathCount: slowStreamOpenFiveSecond.length, + longLivedStreamOpenOverFiveSecondSampleCount: slowStreamOpenFiveSecond.reduce((sum, item) => sum + Number(item.streamOpenOverFiveSecondCount ?? 0), 0), + longLivedStreamOpenOverBudgetPathCount: slowStreamOpen.length, + longLivedStreamOpenOverBudgetSampleCount: slowStreamOpen.reduce((sum, item) => sum + Number(item.streamOpenOverBudgetCount ?? item.streamOpenOverFiveSecondCount ?? 0), 0), longLivedStreamLifetimeOverFiveSecondSampleCount: longLivedStreams.reduce((sum, item) => sum + Number(item.streamLifetimeOverFiveSecondCount ?? 0), 0), slowPathCount: slow.length, - slowSampleCount: slow.reduce((sum, item) => sum + item.overFiveSecondCount, 0), + slowSampleCount: slow.reduce((sum, item) => sum + Number(item.overBudgetCount ?? item.overFiveSecondCount ?? 0), 0), + overFiveSecondPathCount: slowFiveSecond.length, + overFiveSecondSampleCount: slowFiveSecond.reduce((sum, item) => sum + Number(item.overFiveSecondCount ?? 0), 0), partialTimingSampleCount: ordinaryApi.reduce((sum, item) => sum + Number(item.partialTimingSampleCount ?? 0), 0), - partialOverFiveSecondPathCount: partialSlow.length, - partialOverFiveSecondSampleCount: partialSlow.reduce((sum, item) => sum + Number(item.partialOverFiveSecondCount ?? 0), 0), + partialOverFiveSecondPathCount: partialFiveSecond.length, + partialOverFiveSecondSampleCount: partialFiveSecond.reduce((sum, item) => sum + Number(item.partialOverFiveSecondCount ?? 0), 0), + partialOverBudgetPathCount: partialSlow.length, + partialOverBudgetSampleCount: partialSlow.reduce((sum, item) => sum + Number(item.partialOverBudgetCount ?? item.partialOverFiveSecondCount ?? 0), 0), worstP95Ms: budgetP95Values.length > 0 ? Math.max(...budgetP95Values) : null, valuesRedacted: true }, @@ -3623,6 +3714,7 @@ function buildSampleMetrics(samples, control) { loadingLongestContinuousSeconds: loading.summary.longestContinuousSeconds, loadingCurrentContinuousSeconds: loading.summary.currentContinuousSeconds, loadingOverFiveSecondSegmentCount: loading.summary.overFiveSecondSegmentCount, + loadingOverBudgetSegmentCount: loading.summary.overBudgetSegmentCount, sessionRailSampleCount: sessionRailTitles.summary.sampleCount, sessionRailVisibleSampleCount: sessionRailTitles.summary.visibleSampleCount, sessionRailFallbackMajoritySampleCount: sessionRailTitles.summary.majorityFallbackSampleCount, @@ -3704,6 +3796,7 @@ function buildSessionRailTitleMetrics(samples, timeline) { fallbackTitleCount: safeFallbackTitleCount, fallbackTitleRatio, majorityFallback: safeVisibleCount > 0 && safeFallbackTitleCount > safeVisibleCount / 2, + overThreshold: safeVisibleCount > 0 && fallbackTitleRatio > alertThresholds.sessionRailFallbackRatio, examples: fallbackItems.slice(0, 5).map((item) => ({ titleHash: item?.titleHash ?? null, titlePreview: limitText(String(item?.titlePreview || ""), 160), @@ -3714,6 +3807,7 @@ function buildSessionRailTitleMetrics(samples, timeline) { } const visibleRows = rows.filter((item) => item.visibleCount > 0); const majorityRows = rows.filter((item) => item.majorityFallback); + const overThresholdRows = rows.filter((item) => item.overThreshold); const fallbackRows = rows.filter((item) => item.fallbackTitleCount > 0); const maxFallbackRatio = rows.length > 0 ? Math.max(...rows.map((item) => Number(item.fallbackTitleRatio) || 0)) : 0; const maxVisibleCount = rows.length > 0 ? Math.max(...rows.map((item) => Number(item.visibleCount) || 0)) : 0; @@ -3724,6 +3818,8 @@ function buildSessionRailTitleMetrics(samples, timeline) { visibleSampleCount: visibleRows.length, fallbackSampleCount: fallbackRows.length, majorityFallbackSampleCount: majorityRows.length, + overThresholdSampleCount: overThresholdRows.length, + thresholdRatio: alertThresholds.sessionRailFallbackRatio, maxFallbackRatio, maxVisibleCount, maxFallbackTitleCount, @@ -3863,6 +3959,8 @@ function buildLoadingMetrics(samples, timeline) { ownerCount: owners.length, segmentCount: segments.length, overFiveSecondSegmentCount: segments.filter((segment) => Number(segment.durationSeconds ?? 0) > 5).length, + overBudgetSegmentCount: segments.filter((segment) => Number(segment.durationSeconds ?? 0) > alertThresholds.visibleLoadingSlowMs / 1000).length, + budgetSeconds: alertThresholds.visibleLoadingSlowMs / 1000, longestContinuousSeconds: segments.length > 0 ? Number(segments[0].durationSeconds ?? 0) : 0, currentContinuousSeconds: currentSegment ? Number(currentSegment.durationSeconds ?? 0) : 0, continuityThresholdMs, @@ -4087,7 +4185,7 @@ function detectTurnTimingNonMonotonic(columns, rows) { if (previous && metric === "totalElapsedSeconds" && current > previous.value) { const sampleDeltaSeconds = elapsedSecondsBetween(previous.ts, row.ts); const delta = current - previous.value; - const allowedIncreaseSeconds = sampleDeltaSeconds + 3; + const allowedIncreaseSeconds = sampleDeltaSeconds + alertThresholds.turnTimingSampleSlackSeconds; if (delta > allowedIncreaseSeconds) { totalElapsedForwardJumps.push({ columnId: column.id, @@ -4147,7 +4245,9 @@ function detectTurnTimingNonMonotonic(columns, rows) { const elapsedMs = Date.parse(String(row.ts ?? "")) - Date.parse(String(previous.ts ?? "")); const elapsedSeconds = Number.isFinite(elapsedMs) && elapsedMs >= 0 ? elapsedMs / 1000 : null; const increase = current - previous.value; - const allowedIncrease = elapsedSeconds === null ? 3 : Math.max(3, elapsedSeconds + 2); + const allowedIncrease = elapsedSeconds === null + ? alertThresholds.turnTimingSampleSlackSeconds + : Math.max(alertThresholds.turnTimingSampleSlackSeconds, elapsedSeconds + alertThresholds.turnTimingSampleSlackSeconds); const excessiveIncrease = increase > allowedIncrease ? increase - allowedIncrease : 0; recentUpdateSteps.push({ columnId: column.id, @@ -4919,11 +5019,13 @@ function renderMarkdown(report) { : "- 无页面 provenance segment。"; const ordinaryPerformanceItems = Array.isArray(report.pagePerformance?.sameOriginApiByPath) ? report.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream !== true) : []; const streamPerformanceItems = Array.isArray(report.pagePerformance?.sameOriginApiByPath) ? report.pagePerformance.sameOriginApiByPath.filter((item) => item.isLongLivedStream === true) : []; + const sameOriginApiBudgetMs = Number(report.alertThresholds?.sameOriginApiSlowMs ?? report.pagePerformance?.summary?.budgetMs ?? 10000); + const streamOpenBudgetMs = Number(report.alertThresholds?.longLivedStreamOpenSlowMs ?? 10000); const performanceLines = ordinaryPerformanceItems.length > 0 - ? ordinaryPerformanceItems.slice(0, 80).map((item) => "- " + item.path + " kind=" + (item.routeKind || "same-origin-api") + " budgetMetric=" + (item.budgetMetric || "durationMs") + " samples=" + item.sampleCount + " p50=" + (item.p50Ms ?? "-") + "ms p75=" + (item.p75Ms ?? "-") + "ms p95=" + (item.p95Ms ?? "-") + "ms max=" + (item.maxMs ?? "-") + "ms >5s=" + (item.overFiveSecondCount ?? 0) + " window=" + (item.firstAt || "-") + ".." + (item.lastAt || "-")).join("\n") + ? ordinaryPerformanceItems.slice(0, 80).map((item) => "- " + item.path + " kind=" + (item.routeKind || "same-origin-api") + " budgetMetric=" + (item.budgetMetric || "durationMs") + " samples=" + item.sampleCount + " p50=" + (item.p50Ms ?? "-") + "ms p75=" + (item.p75Ms ?? "-") + "ms p95=" + (item.p95Ms ?? "-") + "ms max=" + (item.maxMs ?? "-") + "ms >budget=" + (item.overBudgetCount ?? item.overFiveSecondCount ?? 0) + " budgetMs=" + (item.budgetMs ?? sameOriginApiBudgetMs) + " legacy>5s=" + (item.overFiveSecondCount ?? 0) + " window=" + (item.firstAt || "-") + ".." + (item.lastAt || "-")).join("\n") : "- 无同源 API Resource Timing 样本。"; const streamPerformanceLines = streamPerformanceItems.length > 0 - ? streamPerformanceItems.slice(0, 80).map((item) => "- " + item.path + " kind=" + (item.routeKind || "same-origin-api-stream") + " samples=" + item.sampleCount + " streamOpenP50=" + (item.streamOpenP50Ms ?? "-") + "ms streamOpenP75=" + (item.streamOpenP75Ms ?? "-") + "ms streamOpenP95=" + (item.streamOpenP95Ms ?? "-") + "ms streamOpenMax=" + (item.streamOpenMaxMs ?? "-") + "ms streamOpen>5s=" + (item.streamOpenOverFiveSecondCount ?? 0) + " streamLifetime>5s=" + (item.streamLifetimeOverFiveSecondCount ?? 0) + " lifetimeMax=" + (item.maxMs ?? "-") + "ms window=" + (item.firstAt || "-") + ".." + (item.lastAt || "-")).join("\n") + ? streamPerformanceItems.slice(0, 80).map((item) => "- " + item.path + " kind=" + (item.routeKind || "same-origin-api-stream") + " samples=" + item.sampleCount + " streamOpenP50=" + (item.streamOpenP50Ms ?? "-") + "ms streamOpenP75=" + (item.streamOpenP75Ms ?? "-") + "ms streamOpenP95=" + (item.streamOpenP95Ms ?? "-") + "ms streamOpenMax=" + (item.streamOpenMaxMs ?? "-") + "ms streamOpen>budget=" + (item.streamOpenOverBudgetCount ?? item.streamOpenOverFiveSecondCount ?? 0) + " streamOpenBudgetMs=" + (item.streamOpenBudgetMs ?? streamOpenBudgetMs) + " streamOpenLegacy>5s=" + (item.streamOpenOverFiveSecondCount ?? 0) + " streamLifetime>5s=" + (item.streamLifetimeOverFiveSecondCount ?? 0) + " lifetimeMax=" + (item.maxMs ?? "-") + "ms window=" + (item.firstAt || "-") + ".." + (item.lastAt || "-")).join("\n") : "- 无同源长连接 Resource Timing 样本。"; const metricLines = Array.isArray(report.sampleMetrics?.timeline) && report.sampleMetrics.timeline.length > 0 ? report.sampleMetrics.timeline.slice(0, 120).map((item) => "- #" + item.seq + " " + item.ts + " prompt=" + item.promptIndex + " loadingCount=" + (item.loadingCount ?? 0) + " loadingOwners=" + (item.loadingOwnerCount ?? 0) + " totalElapsedSeconds=" + (item.totalElapsedSeconds ?? "-") + " recentUpdateSeconds=" + (item.recentUpdateSeconds ?? "-") + " terminal=" + item.terminalSeen + " finalText=" + item.finalResultTextSeen + " diagnostic=" + item.diagnosticSeen).join("\n") @@ -4981,6 +5083,7 @@ function renderMarkdown(report) { + "- overFiveSecondSegmentCount: " + (loadingSummary.overFiveSecondSegmentCount ?? 0) + "\n" + "- longestContinuousSeconds: " + (loadingSummary.longestContinuousSeconds ?? 0) + "\n" + "- currentContinuousSeconds: " + (loadingSummary.currentContinuousSeconds ?? 0) + "\n" + + "- budgetSeconds: " + (loadingSummary.budgetSeconds ?? (Number(report.alertThresholds?.visibleLoadingSlowMs ?? 10000) / 1000)) + "\n" + "- policy: 该指标只能证明用户真实看到“加载中”的持续时间；修复必须降低真实请求/投影/渲染耗时，禁止提前展示未加载完内容来压低该指标。\n\n" + "#### Loading segments\n\n" + loadingSegmentLines + "\n\n" + "#### Loading owners\n\n" + loadingOwnerLines + "\n\n" @@ -5001,7 +5104,7 @@ function renderMarkdown(report) { + "- controlSegmentCount: " + (report.pageProvenance?.summary?.controlSegmentCount ?? 0) + "\n\n" + provenanceLines + "\n\n" + "### Page performance: same-origin API Resource Timing\n\n" - + "- budgetMs: " + (report.pagePerformance?.summary?.budgetMs ?? 5000) + "\n" + + "- budgetMs: " + (report.pagePerformance?.summary?.budgetMs ?? sameOriginApiBudgetMs) + "\n" + "- sameOriginApiPathCount: " + (report.pagePerformance?.summary?.sameOriginApiPathCount ?? 0) + "\n" + "- sameOriginApiSampleCount: " + (report.pagePerformance?.summary?.sameOriginApiSampleCount ?? 0) + "\n" + "- longLivedStreamPathCount: " + (report.pagePerformance?.summary?.longLivedStreamPathCount ?? 0) + "\n" @@ -5014,7 +5117,7 @@ function renderMarkdown(report) { + "- worstP95Ms: " + (report.pagePerformance?.summary?.worstP95Ms ?? "-") + "\n\n" + performanceLines + "\n\n" + "### Page performance: long-lived streams\n\n" - + "- policy: SSE/long-lived stream lifetime is not ordinary API load latency; only stream open latency is compared with the 5s usability budget, while disconnects remain runtime alerts.\n\n" + + "- policy: SSE/long-lived stream lifetime is not ordinary API load latency; only stream open latency is compared with the YAML usability budget, while disconnects remain runtime alerts.\n\n" + streamPerformanceLines + "\n\n" + "### Prompt network\n\n" + promptNetworkLines + "\n\n" + "### Runtime alerts\n\n"