Merge pull request #121 from pikasTech/fix/issue20-commander-concurrency-closeout
fix: consolidate Code Queue commander concurrency disclosure
This commit is contained in:
Lyon
@@ -47,7 +47,7 @@ CLI 可以从 `master` 快速演进，但必须兼容 `deploy.json` 固定的 CI
|
||||
- `codex submit [prompt] [--prompt-file path|--prompt-stdin] [--queue queueId] [--provider-id id] [--cwd path] [--model model] [--reasoning-effort effort] [--execution-mode mode] [--max-attempts N] [--reference-task-id id] [--dry-run]` 通过 backend-core 私有代理向稳定 `code-queue` 用户服务路径提交任务；prompt 必须且只能来自位置参数、文件或 stdin 之一，`--dry-run` 只返回结构化请求且不实际入队。长 prompt、多行 prompt、含引号/反引号/Markdown 表格/JSON/反斜杠的 prompt 必须优先用 `--prompt-stdin` 或 `--prompt-file`，不要拼进 shell 单个参数；位置参数只适合短单行 smoke prompt。stdin 推荐用 quoted heredoc：`cat <<'PROMPT' | bun scripts/cli.ts codex submit --prompt-stdin --queue <id> --dry-run`，文件路径推荐 `bun scripts/cli.ts codex submit --prompt-file /tmp/code-queue-prompt.md --queue <id> --dry-run`，确认 dry-run 后移除 `--dry-run` 提交同一 payload。dry-run 会额外输出 `routingRecommendation`，包含推荐 route、runner、model、风险信号、prompt 自包含/issue 非唯一来源/prod-secret-DB 禁止/运行态或 release 禁止/证据要求/中等复杂度候选等 guard 状态；同时输出 `policyContract`，固定暴露 GPT-5.5、DeepSeek、MiniMax 的风险分层、并发上限和外部 provider 429 退避处置。该建议只用于指挥官 preflight，不会改写 payload，不改变 runtime admission，也不假设生产 MiniMax 或 DeepSeek 可用。`--dry-run` 必须返回完整 prompt、字符数和 `truncated=false` 用于人工验收；真实提交是写入操作，默认只返回 `accepted=true`、task id、队列、写入保护摘要和后续查看命令，必须标记 `promptOmitted=true` 且不得回显 prompt 或 promptPreview。真实提交会经过本机本地串行化保护和短节流，避免同一指挥端并发 submit 把低内存主机或 `code-queue-mgr` 控制面打抖；返回值会附带低噪声 `submitConcurrencyGuard` 说明本次提交的锁与等待信息。backend-core 默认把提交、队列 CRUD、已读状态、历史摘要和轻量 Trace 读取分流到主 server `code-queue-mgr`，由它写入主 PostgreSQL；D601 scheduler 只轮询并执行已入库任务。
|
||||
- `codex pr-preflight [--remote] [--push-dry-run --push-dry-run-ref refs/heads/probe/<name>] [--pr-create-dry-run --pr-create-dry-run-head <head>] [--issue N] [--full]` 通过稳定 `code-queue` proxy 请求 D601 scheduler `/api/runtime-preflight`，用于 PR 型派单 admission。输出会压缩展示 scheduler/runner 的 token 覆盖、Auth Broker source/capability/nextAction、工具、agent port、Git worktree、GitHub egress、repo/issue/PR 只读探测、可选 push dry-run，以及可选 PR body/create dry-run guard；只报告 `GH_TOKEN`/`GITHUB_TOKEN` 是否存在和来源 key，不打印值。当 auth-broker 配置存在时，`tokenCoverage.source="auth-broker"`、`credentialSource="broker-issued-token"` 且 runner env token 不是成功前提；当仅 env token 存在时，`credentialSource="env-token"` 且 `preflight.authBroker.nextAction="use-env-token-until-auth-broker-live"`；两者都缺失时顶层 `ok=false`、`runnerDisposition=infra-blocked`、`degradedReason=auth-broker-needed`，`tokenCoverage.missing` 同时列出 `GH_TOKEN` 与 `GITHUB_TOKEN`，并输出 `preflight.authBroker.source="broker/auth-broker-needed"`、`capability.source="missing-token"`。该 `auth-missing` 的 scope 是 `scheduler-runner-env`，不能简化成“当前 active runner/dev container 不能创建 PR”；输出必须带 `scopeBoundary` 和 `activeRunnerDevContainer`，要求调用方另跑 `bun scripts/cli.ts gh auth status --repo pikasTech/unidesk` 与 PR dry-run 来确认当前 dev container 能力。`preflight.prCapabilityContract` 是 runner-facing 合同摘要，必须包含目标分支、token/auth 来源、`systemGhBinaryRequiredForWrites=false`、UniDesk REST `bun scripts/cli.ts gh` 可用性、push dry-run/PR create dry-run 的 `writesRemote=false`、expected PR handoff、真实 PR 创建需要 commander 授权和 `gh pr merge` 的 `unsupported-command` 边界；系统 `gh` binary 缺失只进入 `tools.systemGhBinary`，不得误判为 UniDesk REST `gh` CLI 不可用。`--remote` 在 runner-like 环境里不再依赖本地 `unidesk-backend-core`、`unidesk-database`、`baidu-netdisk-backend` 容器存在；这些缺失只作为本地观测证据。若远程控制面可达，则继续走远程控制面结果；若远程控制面不可达，则结构化返回 `failureKind=control-plane-missing` / `degradedReason=remote-control-plane-unreachable`，而不是把本地 `backend-core-container-missing` 当作最终阻塞。`--pr-create-dry-run` 不 POST GitHub，只证明 runner 内 PR body 生成、`scripts/cli.ts gh pr create --dry-run` 和 branch 参数形态可用；服务端创建权限仍以 token/auth broker、repo/issue/PR read、push dry-run 和最终授权后的真实 PR 创建结果为准。
|
||||
- `codex task <taskId>` 通过 Code Queue 私有代理按任务 ID 查询结构化审阅摘要；默认只返回任务身份、执行 Provider、工作目录、attempt 计数、原始 prompt、最终 response、最后错误和渐进披露命令，适合指挥官审阅完成未读任务且避免上下文爆炸。`--detail` 仍是有界详细摘要：默认只返回少量 attempt/tool 行、短 prompt/response/stderr/feedback 预览和 omitted/truncated 元数据；需要完整 prompt/response 文本或更多 tool/attempt 细节时再显式加 `--full`、`--tool-limit N`、`--trace` 或 `codex output`。该摘要读取默认由主 server `code-queue-mgr` 从 PostgreSQL 返回，不依赖 D601 `code-queue-read` Service 可用。
|
||||
- `codex tasks [--view supervisor|full] [--queue id] [--status succeeded|running|queued|failed|canceled|judging|retry_wait[,..]] [--unread|--unread-only] [--limit N] [--before-id id]` 通过同一私有代理输出渐进式披露视图。默认 `supervisor` 是低噪声指挥官视图，只返回 `running`、`completedUnread`、`recentCompleted`、`queued` 和 `executionDiagnostics` 的紧凑行；prompt/body 只给短预览和原始字符数，`running`/`completedUnread`/`queued` 默认只返回一个很小的有界页并通过 section `commands.next` 继续分页，`recentCompleted` 默认也限量且不重复 `completedUnread` 未读终态，不嵌入完整 Trace、final response 或全量 overview。`--limit` 在 supervisor 中主要是扫描/分页预算，不是返回几十条肥行的开关；需要更详细当前页任务行时显式使用 `--view full` 或 `--full`，仍受 `--limit` 和 `--before-id` 分页约束。
|
||||
- `codex tasks [--view supervisor|full] [--queue id] [--status succeeded|running|queued|failed|canceled|judging|retry_wait[,..]] [--unread|--unread-only] [--limit N] [--before-id id]` 通过同一私有代理输出渐进式披露视图。默认 `supervisor` 是低噪声指挥官视图，只返回 `activeRunning`、`running`、`completedUnread`、`recentCompleted`、`queued`、`activity`、`commanderConcurrency` 和 `executionDiagnostics` 的紧凑行；`activeRunning.count` 是 running+judging 的状态计数，`exact=true` 时来自 queue summary counts，`running.returned` 和 `activeRunning.rowPage.returned` 只是本次返回的紧凑行数。`commanderConcurrency.activeRunnerCount` 是并发策略应使用的 active/running 计数，等于 `activity.effectiveActiveTaskCount`；15 并发策略按 `15 - activeRunnerCount` 计算剩余窗口。`commanderConcurrency.splitBrainDisposition=live-count-as-active` 表示 split-brain 有 fresh heartbeat 证据，应继续监督并计入 active；`interventionRequired=true` 才提示介入。prompt/body 只给短预览和原始字符数，`running`/`completedUnread`/`queued` 默认只返回一个有界小页并通过 section `commands.next` 继续分页，`recentCompleted` 默认限量且不重复 `completedUnread` 未读终态，不嵌入完整 Trace、final response 或全量 overview。`--limit` 在 supervisor 中主要是扫描/分页预算，不是返回几十条肥行的开关；CLI 安全上限是 100，输出会在 `filters.requestedLimit`、`filters.effectiveLimit`、`filters.limitCapped` 和 `disclosure.limitPolicy` 说明显式请求是否被 capped；底层 overview 拉取预算独立显示在 `source.requestedLimit` / `source.effectiveLimit`，所以 `--limit 260` 应显示 requested=260、effective=100、source requested/effective=200，而不是只露出一个含糊的 `limit`。`--unread` 是 `--unread-only` 的别名，必须只保留未读终态；`--status` 必须真实过滤支持的状态，未知参数或未知状态必须结构化失败。需要更详细当前页任务行时显式使用 `--view full` 或 `--full`，仍受 `--limit` 和 `--before-id` 分页约束。
|
||||
- `codex task <taskId> --trace --tail|--from-start|--after-seq N|--before-seq N --limit N` 按页拉取 Code Queue 的逻辑 trace；响应会返回 `nextAfterSeq`、`previousBeforeSeq`、`hasMore`、`hasBefore` 和下一页/上一页命令，默认 `--trace` 取最新一页，且仍以分页 trace 为主；需要完整 prompt/最终 response 时加 `--full`，需要详细 task 摘要时加 `--detail`。
|
||||
- `codex output <taskId> --tail|--from-start|--after-seq N|--before-seq N --limit N [--full-text]` 按原始 output seq 分页读取底层记录；当 trace 行提示 `commandOmittedLines`、`bodyOmittedLines` 或 `rawSeqs` 时，用该命令按 seq 补取信息。默认是低噪声 raw-output 摘要：即使传入很大的 `--limit`，非 `--full-text` 也会限制返回行数和单条文本预览，并在 `disclosure.limitCapped`、`requestedLimit`、`effectiveLimit` 和 `commands.fullText` 中说明如何继续展开；显式 `--full-text` 才返回该页全文。
|
||||
- `codex read <taskId>` 在人工审阅后标记单个终态任务已读；列表、overview 和 supervisor 视图只返回这个命令字段，不得自动执行，也不得批量清空未读状态。
|
||||
@@ -56,7 +56,7 @@ CLI 可以从 `master` 快速演进，但必须兼容 `deploy.json` 固定的 CI
|
||||
- `codex steer <taskId> [prompt|--prompt-file path|--prompt-stdin] [--dry-run] [--no-retry|--retry-attempts N]` 通过 Code Queue 私有代理向正在运行的 task 注入纠偏提示，正式替代底层 `microservice proxy code-queue /api/tasks/<taskId>/steer` 调用。prompt 必须且只能来自位置参数、文件或 stdin 之一；`--dry-run` 只输出 `method`、`path`、`stableProxyPath`、retry policy、prompt 字符数、截断预览和 raw proxy 等价命令，不触碰运行中 session，也不得泄露超长 prompt 全文。真实执行是写入操作，成功只返回 `accepted=true`、task id、prompt 字符数、`promptOmitted=true`、有界 task/queue 确认、attempt summary 和后续查看命令，不回显 prompt 或完整 task state；路径固定为 `/api/microservices/code-queue/proxy/api/tasks/<taskId>/steer`，只能作用于 D601 scheduler 上存在 active steerable turn 的 running task。默认对 `stable-proxy-failed` 和 `backend-core-unreachable` 这类 retryable control-plane failures 做一次有界重试；`--retry-attempts N` 最大为 3，`--retry-delay-ms N` 最大为 5000，`--no-retry` 用于复现单次失败。
|
||||
- `codex steer` 非 dry-run 失败仍输出 JSON 且退出非零；`.data.diagnostics.reason` 用于 runner 分流，当前包括 `backend-core-unreachable`、`code-queue-microservice-unregistered`、`proxy-unauthorized`、`proxy-404`、`steer-endpoint-404`、`upstream-runtime-rejected`、`stable-proxy-failed` 和 `invalid-proxy-response`。`scope` 区分 `backend-core`、`stable-proxy`、`code-queue-runtime` 或 `unknown`，并带 `status`、`exitCode`、`retryable`、有界 `upstreamBodyPreview`、`attempts`、`retryPolicy` 和推荐交叉验证命令；若任务不在 running/active-turn 状态，通常归类为 `upstream-runtime-rejected`，不得静默成功。`502 provider HTTP tunnel failed`、`provider-gateway-http-fetch`、`The operation was aborted` 或约 30 秒 tunnel wait abort 会归类为 `stable-proxy-failed`，CLI 会先按 retry policy 重试；如果仍失败，`.data.diagnostics.operatorGuidance.rawProxyEquivalentIsFallback=false` 表示 raw proxy 等价命令走同一条 tunnel，只能用于对照诊断，不应被当作更低噪声 fallback。此时 `.data.steer.deliveryUnconfirmed=true`，指挥官应先看 `codex tasks --view supervisor`、`codex task <taskId>` 和 `microservice health code-queue`，再从主 server CLI 或显式 SSH transport 重试同一个 `codex steer`。
|
||||
- `codex interrupt|cancel <taskId>` 通过 Code Queue 私有代理请求中断；running/judging 任务会请求 D601 当前 agent run 停止，queued/retry_wait 任务的取消也必须保持与 WebUI 相同代理路径，返回有界 task 摘要和后续查询命令。任何需要接触 active run 的动作仍属于 D601 执行面。
|
||||
- Code Queue 多队列 lane 由 `codex` 命令命名空间管理：`queues [--full|--all] [--limit N] [--page N|--offset N]` 列表、`queue create <queueId>` 创建、`queue merge <sourceQueueId> --into <targetQueueId>` 合并、`move <taskId> --queue <queueId>` 迁移；这些队列管理入口默认由主 server `code-queue-mgr` 直管 PostgreSQL，仍通过稳定 `code-queue` 用户服务代理路径访问。`codex queues` 默认只返回 active/nonempty/unread/runnable queue 摘要、全局 counts 和 execution diagnostics；`--full` 或 `--all` 只切换为完整队列行视图的一页，仍受 `--limit`/`--page`/`--offset` 分页约束，不再默认携带 deprecated full array。summary 和 full 的稳定机读路径都是 `.data.queues.items[]`，全局元数据固定在 `.data.queues.counts`、`.data.queues.executionDiagnostics`、`.data.queues.activeTaskIds` 和 `.data.queues.queuedTaskIds`；需要完整 upstream 时使用输出中的 raw command。旧 full 顶层数组语义已作为 deprecated 兼容信息记录，不再作为 `.data.queues` 主形态。同一个 queue 内部串行执行，不同 queue 之间并行执行。迁移只允许尚未被 scheduler claim 的 `queued`/`retry_wait` 任务，必须满足 `startedAt=null`、`currentAttempt=0` 且没有 active thread/turn；已进入 `running`/`judging` 或已有 claim 标记的任务返回 409，不得被 move/merge 回写成 queued。合并会移动可迁移任务归属并自动删除源 queue 记录，只保留合并后的目标 queue；若 source 或 target queue 存在 active/claimed 任务，合并整体返回 409。合并后的目标 queue 按任务原 `queueEnteredAt`/`createdAt` 时间顺序串行，成功迁移 queued/retry_wait 任务后由 D601 scheduler 轮询推进。
|
||||
- Code Queue 多队列 lane 由 `codex` 命令命名空间管理：`queues [--full|--all] [--limit N] [--page N|--offset N]` 列表、`queue create <queueId>` 创建、`queue merge <sourceQueueId> --into <targetQueueId>` 合并、`move <taskId> --queue <queueId>` 迁移；这些队列管理入口默认由主 server `code-queue-mgr` 直管 PostgreSQL，仍通过稳定 `code-queue` 用户服务代理路径访问。`codex queues` 默认只返回 active/nonempty/unread/runnable queue 摘要、activity、commanderConcurrency、全局 counts 和 execution diagnostics；`--full` 或 `--all` 只切换为完整队列行视图的一页，仍受 `--limit`/`--page`/`--offset` 分页约束，不再默认携带 deprecated full array。summary 和 full 的稳定机读路径都是 `.data.queues.items[]`，全局元数据固定在 `.data.queues.commanderConcurrency`、`.data.queues.activity`、`.data.queues.counts`、`.data.queues.executionDiagnostics`、`.data.queues.activeTaskIds` 和 `.data.queues.queuedTaskIds`；需要完整 upstream 时使用输出中的 raw command。`commanderConcurrency.activeRunnerCount` / `activity.effectiveActiveTaskCount` 是指挥官并发判断的有效活跃数，`schedulerLocalActiveQueueCount`/`activeQueueIds` 只描述本地 scheduler active-run slots，不能覆盖数据库 running 计数或 heartbeat-fresh runner 计数。旧 full 顶层数组语义已作为 deprecated 兼容信息记录，不再作为 `.data.queues` 主形态。同一个 queue 内部串行执行，不同 queue 之间并行执行。迁移只允许尚未被 scheduler claim 的 `queued`/`retry_wait` 任务，必须满足 `startedAt=null`、`currentAttempt=0` 且没有 active thread/turn；已进入 `running`/`judging` 或已有 claim 标记的任务返回 409，不得被 move/merge 回写成 queued。合并会移动可迁移任务归属并自动删除源 queue 记录，只保留合并后的目标 queue；若 source 或 target queue 存在 active/claimed 任务，合并整体返回 409。合并后的目标 queue 按任务原 `queueEnteredAt`/`createdAt` 时间顺序串行，成功迁移 queued/retry_wait 任务后由 D601 scheduler 轮询推进。
|
||||
- 所有 `codex` 查询和管理命令必须走与 WebUI 相同的 backend-core 私有代理路径 `/api/microservices/code-queue/proxy/...`；CLI 不得为了提交、移动、中断、取消或队列管理直接调用 D601 内部 Service、数据库、pod curl 或 k3sctl scheduler 子服务。若该路径失败，应先修复 CLI/backend/provider tunnel 链路，而不是绕过控制面。
|
||||
- `job list [--limit N] [--include-command]` 与 `job status <jobId|latest> [--tail-bytes N]` 查询 `.state/jobs/` 文件系统状态，是异步命令的可观测入口。`job list` 默认只返回最新 50 条摘要；`job status` 默认只返回 stdout/stderr 末尾 12000 字节，并带 `tailPolicy` 与完整日志路径。
|
||||
- `debug health`、`debug dispatch` 与 `debug task` 走真实内部 core、WebSocket、数据库、provider、系统指标、Docker 状态和 Host SSH 维护桥流程，只用于开发调试，不写入 `TEST.md` 的正式验收步骤。
|
||||
|
||||
@@ -220,15 +220,15 @@ bun scripts/cli.ts codex pr-preflight --remote --issue <issue-number>
|
||||
|
||||
常用入口：
|
||||
|
||||
- `bun scripts/cli.ts codex tasks --view supervisor --limit N`：查看默认低噪声监督视图，包括 running、完成未读、少量最近完成、queued/runnable、execution diagnostics、任务分类和下一步 drill-down 命令。默认行只保留 task id、队列、短 prompt/body 预览和原始字符数；`--limit` 是扫描/分页预算，不是返回几十条肥行的开关；`show/detail/trace/output/full/read` 放在 section template 中，避免每条任务重复刷屏，需要更多内容再按 taskId 展开。
|
||||
- `bun scripts/cli.ts codex queues`：查看低噪声队列计数、active task id、完成未读队列、runnable 队列和控制面诊断；需要完整队列行视图时加 `--full`，但 `--full` 仍默认分页，继续用 `--limit N`、`--page N` 或 `--offset N` 渐进展开。summary 和 full 都使用稳定 JSON path `.data.queues.items[]` 读取队列行，并从 `.data.queues.counts` 与 `.data.queues.executionDiagnostics` 读取全局计数和执行诊断；完整 upstream 只通过输出中的 raw command 显式获取。
|
||||
- `bun scripts/cli.ts codex tasks --view supervisor --limit N`：查看默认低噪声监督视图，包括 `activeRunning`、running、完成未读、少量最近完成、queued/runnable、activity、commanderConcurrency、execution diagnostics、任务分类和下一步 drill-down 命令。默认行只保留 task id、队列、短 prompt/body 预览和原始字符数；`--limit` 是扫描/分页预算，不是返回几十条肥行的开关，CLI effective limit 安全上限为 100，输出必须用 `filters.requestedLimit`、`filters.effectiveLimit`、`filters.limitCapped`、`source.requestedLimit` 和 `source.effectiveLimit` 区分用户请求、CLI cap 和 overview 源拉取预算；例如 `--limit 260` 应明确显示 requested=260、effective=100、source=200，`running.returned` 只是低噪声返回行数。`show/detail/trace/output/full/read` 放在 section template 中，避免每条任务重复刷屏，需要更多内容再按 taskId 展开。
|
||||
- `bun scripts/cli.ts codex queues`：查看低噪声队列计数、activity、commanderConcurrency、active task id、完成未读队列、runnable 队列和控制面诊断；需要完整队列行视图时加 `--full`，但 `--full` 仍默认分页，继续用 `--limit N`、`--page N` 或 `--offset N` 渐进展开。summary 和 full 都使用稳定 JSON path `.data.queues.items[]` 读取队列行，并从 `.data.queues.commanderConcurrency`、`.data.queues.activity`、`.data.queues.counts` 与 `.data.queues.executionDiagnostics` 读取全局活跃计数和执行诊断；完整 upstream 只通过输出中的 raw command 显式获取。
|
||||
- `bun scripts/cli.ts codex tasks --unread --limit N`：查看完成未读审阅积压；`--unread` 与 `--unread-only` 等价，不能被静默忽略。
|
||||
- `bun scripts/cli.ts codex tasks --status succeeded --unread --limit N`：按具体终态过滤监督结果；不支持的 status filter 必须显式失败，不能扩大为未过滤结果。
|
||||
- `bun scripts/cli.ts codex task <taskId>`：默认只查看原始 prompt、最终 response、最后错误和 drill-down 命令，这是完成未读任务审阅的第一步。
|
||||
- 当默认审阅摘要不足时，再逐级使用 `bun scripts/cli.ts codex task <taskId> --detail`、`bun scripts/cli.ts codex task <taskId> --trace --limit N` 或 `codex output`。
|
||||
- 当 master 控制面状态和 D601 scheduler 状态看起来分裂时，使用 `docs/reference/observability.md` 中的活性规则判断。
|
||||
|
||||
默认 supervisor 视图必须保持低噪声。`running`、`completedUnread` 和 `queued` 即使传入较大的 `--limit`，默认也只返回一个很小的有界页，并通过 section `commands.next` 继续分页；`--limit` 保留为扫描/分页预算和 full view 返回预算，不得让一次 supervisor 调用输出几十条肥行。每个任务行只应带 task id 和必要摘要，`show`、`detail`、`trace`、`output`、`full`、`read` 使用 section template 表达，让下一步渐进披露动作明确且不重复；默认不得嵌入完整 queue 列表、完整 final response、raw output 页或完整 trace 行。`recentCompleted` 必须默认限量，且不得重复 `completedUnread` 里的未读终态，避免完成历史把当前 running、阻塞和未读审阅挤出视野；需要完整当前页时显式使用 `--view full`。`executionDiagnostics` 只能展示有界 task-id/reason 预览、总数、截断标记和 omitted counts；需要全量诊断时使用输出中的 raw command。`commands.read` 只是在人工审阅后的建议命令，listing 命令绝不能自动执行。
|
||||
默认 supervisor 视图必须保持低噪声。`activeRunning.count` 是指挥官 active running 计数，来源是 queue summary 的 status counts 时 `activeRunning.exact=true`，用于 redline 判断；`activeRunning.rowPage.returned` / `running.returned` 只表示本次返回的紧凑任务行。`activeRunning.redline` 必须写明 `countField`、routine target、burst redline、hard redline、`state` 和 `decisionReady`；只有 `decisionReady=true` 时，才能直接用该 count 做红线/补派判断。`running`、`completedUnread` 和 `queued` 即使传入较大的 `--limit`，默认也只返回一个很小的有界页，并通过 section `commands.next` 继续分页；`--limit` 保留为扫描/分页预算和 full view 返回预算，不得让一次 supervisor 调用输出几十条肥行。每个任务行只应带 task id 和必要摘要，`show`、`detail`、`trace`、`output`、`full`、`read` 使用 section template 表达，让下一步渐进披露动作明确且不重复；默认不得嵌入完整 queue 列表、完整 final response、raw output 页或完整 trace 行。`recentCompleted` 必须默认限量，且不得重复 `completedUnread` 里的未读终态，避免完成历史把当前 running、阻塞和未读审阅挤出视野；需要完整当前页时显式使用 `--view full`。`executionDiagnostics` 只能展示有界 task-id/reason 预览、总数、截断标记和 omitted counts；需要全量诊断时使用输出中的 raw command。`commands.read` 只是在人工审阅后的建议命令，listing 命令绝不能自动执行。
|
||||
|
||||
`codex tasks` 中的 `status` 永远是 scheduler/control-plane 原始状态，不因为看到 worker final response 而改写。若某个非终态任务的最后 assistant 文本来自 `finalResponse`，CLI 会额外显示 `statusLabel`、`awaitingTerminalJudge=true`、`closeoutState=awaiting-terminal-or-judge` 或 `awaiting-judge`，并附带 closeout hint。指挥官应把这类行理解为“worker 已经产出最终回复文本，但 Code Queue 还在等待 agent terminal event、scheduler 写回或 judge 结果”；它仍占用 active/running 监督窗口，不能按完成任务 `read` 或验收，直到 `status` 进入 `succeeded`、`failed` 或 `canceled` 并可审阅 judge/terminal 记录。
|
||||
|
||||
@@ -238,6 +238,8 @@ bun scripts/cli.ts codex pr-preflight --remote --issue <issue-number>
|
||||
|
||||
队列诊断中的 `split-brain` 表示控制面/执行面观测分裂，不自动证明任务已经死亡。只要任务 heartbeat 还在刷新、trace 仍在推进，就不能把它判成服务中断或要求立刻 stop；应把它视为 `splitBrainLive=true` 的 live 任务，继续监督并推进 #20 里的已排任务，而不是 interrupt、替换或把 backend 当成已经挂掉。队列摘要应显示 `effectiveLiveness=live`、`splitBrainLive=true` 和 `recommendedAction=continue-supervision`；compact 输出还应在 `executionDiagnostics.liveness` 中重复这些低噪声字段，并突出 `activeHeartbeatCount`、有界 `heartbeatFreshTaskIds`、`databaseActiveTaskCount` 和 `schedulerActiveRunSlotCount`。当 master/control-plane 的 `schedulerActiveRunSlotCount=0` 但 `heartbeatFreshTaskIds` 非空时，active 数应优先按 scheduler heartbeat 摘要解释为 live，而不是按 master 本地 slot 0 解释为执行停摆。只有 heartbeat expired/missing 或满足 stale-recovery 条件时，才应显示 `effectiveLiveness=at-risk` 并进入恢复判断。
|
||||
|
||||
`codex queues` 和默认 supervisor 视图的 `activity` / `commanderConcurrency` 是指挥官并发治理的主读数。并发决策固定使用 `commanderConcurrency.activeRunnerCount`，它等于 `activity.effectiveActiveTaskCount`；15 并发策略的可补窗口按 `15 - activeRunnerCount` 计算，不能用 `activeQueueIds.length` 或 scheduler-local slot 数替代。`effectiveActiveTaskCount` 表示用于调度判断的有效活跃任务数；`databaseRunningTaskCount` 来自 PostgreSQL 中 `running` 状态计数；`databaseActiveTaskCount` 覆盖 running/judging 等数据库活跃任务；`heartbeatFreshActiveTaskCount` 表示 heartbeat-fresh 的有效 runner 数；`schedulerLocalActiveQueueCount` 和 `schedulerLocalActiveRunSlotCount` 只表示当前控制面本地可见 active run slots。`activeQueueIds` 与 `activeQueueCount` 是 scheduler-local 字段，可能在 `counts.running>0` 且 heartbeat 新鲜时为 0；看到这种组合时应按 `activity.effectiveActiveTaskCount`、`activity.heartbeatFreshActiveTaskCount` 和 `splitBrainLive` 决策，不得把空 `activeQueueIds` 当作零并发或停摆证据。`commanderConcurrency.splitBrainDisposition=live-count-as-active` 表示 split-brain 仍是 live 且应计入 active runner；`interventionRequired=true`、heartbeat risk、stale recovery candidates，或非 `continue-supervision` 的 recommended action 才进入人工介入/恢复判断。
|
||||
|
||||
单次 `provider is not online`、SSH 超时、proxy 超时或 registry 请求失败只能证明“当前观察路径失败”，不能单独升级为 D601 全局离线、CI/CD 全局阻塞或业务任务不可推进。指挥官和 runner 必须用多信号裁决运行面状态，至少区分以下观察面：
|
||||
|
||||
- backend-core 节点视图和 provider heartbeat，例如 `debug health` 中的 D601 `status`、`lastHeartbeat` 和 gateway 版本；
|
||||
|
||||
@@ -16,6 +16,10 @@ function asArray(value: unknown): unknown[] {
|
||||
return value as unknown[];
|
||||
}
|
||||
|
||||
function manyIds(prefix: string, count: number): string[] {
|
||||
return Array.from({ length: count }, (_, index) => `${prefix}-${String(index + 1).padStart(2, "0")}`);
|
||||
}
|
||||
|
||||
function fixtureResponse(): JsonRecord {
|
||||
return {
|
||||
ok: true,
|
||||
@@ -76,6 +80,66 @@ function fixtureResponse(): JsonRecord {
|
||||
};
|
||||
}
|
||||
|
||||
function splitBrainLiveResponse(): JsonRecord {
|
||||
const liveTaskIds = manyIds("task-running", 8);
|
||||
return {
|
||||
ok: true,
|
||||
status: 200,
|
||||
body: {
|
||||
ok: true,
|
||||
queue: {
|
||||
total: 8,
|
||||
queueCount: 2,
|
||||
activeQueueIds: [],
|
||||
activeTaskIds: [],
|
||||
queuedTaskIds: [],
|
||||
counts: { running: 8 },
|
||||
unreadTerminal: 0,
|
||||
executionDiagnostics: {
|
||||
state: "split-brain",
|
||||
splitBrain: true,
|
||||
splitBrainLive: true,
|
||||
effectiveLiveness: "live",
|
||||
recommendedAction: "continue-supervision",
|
||||
databaseActiveTaskCount: 8,
|
||||
databaseActiveTaskIds: liveTaskIds,
|
||||
schedulerActiveRunSlotCount: 0,
|
||||
schedulerActiveTaskIds: [],
|
||||
activeHeartbeatCount: 8,
|
||||
activeHeartbeatTaskIds: liveTaskIds,
|
||||
heartbeatFreshTaskIds: liveTaskIds,
|
||||
heartbeatExpiredTaskIds: [],
|
||||
heartbeatMissingTaskIds: [],
|
||||
staleRecoveryCandidateTaskIds: [],
|
||||
heartbeatRiskTaskIds: [],
|
||||
},
|
||||
},
|
||||
queues: [
|
||||
{
|
||||
id: "alpha",
|
||||
name: "Alpha",
|
||||
total: 4,
|
||||
counts: { running: 4 },
|
||||
unreadTerminal: 0,
|
||||
activeTaskId: null,
|
||||
runnableTaskId: null,
|
||||
updatedAt: "2026-05-20T00:00:00.000Z",
|
||||
},
|
||||
{
|
||||
id: "beta",
|
||||
name: "Beta",
|
||||
total: 4,
|
||||
counts: { running: 4 },
|
||||
unreadTerminal: 0,
|
||||
activeTaskId: null,
|
||||
runnableTaskId: null,
|
||||
updatedAt: "2026-05-20T00:01:00.000Z",
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function assertQueuesShape(label: string, result: unknown, expectedView: string): void {
|
||||
const data = asRecord(result);
|
||||
const queues = asRecord(data.queues);
|
||||
@@ -100,15 +164,62 @@ function assertQueuesShape(label: string, result: unknown, expectedView: string)
|
||||
assertCondition(liveness.schedulerActiveRunSlotCount === 0, `${label} liveness summary should keep master active slot zero visible`, liveness);
|
||||
assertCondition(asArray(liveness.heartbeatFreshTaskIds).length === 1, `${label} liveness summary should include bounded fresh heartbeat task ids`, liveness);
|
||||
assertCondition(String(liveness.interpretation ?? "").includes("heartbeat is fresh"), `${label} liveness interpretation should explain slot-zero split-brain`, liveness);
|
||||
const activity = asRecord(queues.activity);
|
||||
assertCondition(activity.effectiveActiveTaskCount === 1, `${label} activity should foreground effective active task count`, activity);
|
||||
assertCondition(activity.databaseRunningTaskCount === 1, `${label} activity should distinguish database running tasks`, activity);
|
||||
assertCondition(activity.heartbeatFreshActiveTaskCount === 1, `${label} activity should distinguish heartbeat-fresh active runners`, activity);
|
||||
assertCondition(activity.schedulerLocalActiveQueueCount === 1, `${label} activity should distinguish scheduler-local active queues`, activity);
|
||||
const commanderConcurrency = asRecord(activity.commanderConcurrency);
|
||||
assertCondition(commanderConcurrency.activeRunnerCount === 1, `${label} activity should expose commander-facing active runner count`, commanderConcurrency);
|
||||
assertCondition(commanderConcurrency.activeRunnerCountField === "activity.effectiveActiveTaskCount", `${label} activity should name the active runner count field`, commanderConcurrency);
|
||||
assertCondition(activity.activeQueueIdsScope === "scheduler-local-active-run-slots", `${label} activity should label activeQueueIds scope`, activity);
|
||||
assertCondition(Array.isArray(queues.activeTaskIds), `${label} activeTaskIds should be present`, queues);
|
||||
assertCondition(Array.isArray(queues.queuedTaskIds), `${label} queuedTaskIds should be present`, queues);
|
||||
}
|
||||
|
||||
function assertSplitBrainLiveActivity(label: string, result: unknown): void {
|
||||
const queues = asRecord(asRecord(result).queues);
|
||||
const totals = asRecord(queues.totals);
|
||||
assertCondition(totals.activeQueueCount === 0, `${label} scheduler-local active queue count should be zero`, totals);
|
||||
assertCondition(totals.schedulerLocalActiveQueueCount === 0, `${label} should preserve zero scheduler-local active queues`, totals);
|
||||
assertCondition(totals.runnableQueueCount === 0, `${label} runnable queue count should be zero`, totals);
|
||||
assertCondition(totals.databaseRunningTaskCount === 8, `${label} should foreground DB running task count`, totals);
|
||||
assertCondition(totals.databaseActiveTaskCount === 8, `${label} should foreground DB active task count`, totals);
|
||||
assertCondition(totals.heartbeatFreshActiveTaskCount === 8, `${label} should foreground heartbeat-effective active runners`, totals);
|
||||
assertCondition(totals.commanderActiveRunnerCount === 8, `${label} should mirror commander active count in totals`, totals);
|
||||
assertCondition(totals.effectiveActiveTaskCount === 8, `${label} should foreground effective active task count`, totals);
|
||||
assertCondition(asArray(queues.activeQueueIds).length === 0, `${label} activeQueueIds should remain the scheduler-local list`, queues);
|
||||
assertCondition(queues.activeQueueIdsScope === "scheduler-local-active-run-slots", `${label} activeQueueIds should be scoped`, queues);
|
||||
assertCondition(String(queues.activeQueueIdsNote ?? "").includes("scheduler-local only"), `${label} activeQueueIds note should explain local-only semantics`, queues);
|
||||
const activity = asRecord(queues.activity);
|
||||
assertCondition(activity.effectiveActiveTaskCount === 8, `${label} activity should expose effective active count`, activity);
|
||||
assertCondition(activity.effectiveActiveSource === "heartbeat-fresh", `${label} activity should choose heartbeat-fresh source`, activity);
|
||||
assertCondition(activity.databaseRunningTaskCount === 8, `${label} activity should expose DB running count`, activity);
|
||||
assertCondition(activity.heartbeatFreshActiveTaskCount === 8, `${label} activity should expose heartbeat-effective active count`, activity);
|
||||
assertCondition(activity.schedulerLocalActiveQueueCount === 0, `${label} activity should expose scheduler-local queue count`, activity);
|
||||
assertCondition(activity.schedulerLocalActiveRunSlotCount === 0, `${label} activity should expose scheduler-local slot count`, activity);
|
||||
assertCondition(activity.runnableQueueCount === 0, `${label} activity should expose runnable queue count`, activity);
|
||||
assertCondition(activity.splitBrainLive === true, `${label} activity should preserve split-brain live`, activity);
|
||||
assertCondition(activity.splitBrainDisposition === "live-count-as-active", `${label} activity should count live split-brain as active`, activity);
|
||||
const commanderConcurrency = asRecord(queues.commanderConcurrency);
|
||||
assertCondition(commanderConcurrency.activeRunnerCount === 8, `${label} should expose commander-facing active runner count`, commanderConcurrency);
|
||||
assertCondition(commanderConcurrency.activeRunnerCountField === "activity.effectiveActiveTaskCount", `${label} should name the active runner count field`, commanderConcurrency);
|
||||
assertCondition(commanderConcurrency.splitBrainDisposition === "live-count-as-active", `${label} should classify live split-brain capacity`, commanderConcurrency);
|
||||
assertCondition(commanderConcurrency.interventionRequired === false, `${label} should not require intervention for fresh split-brain`, commanderConcurrency);
|
||||
assertCondition(String(commanderConcurrency.decisionRule ?? "").includes("15 - activeRunnerCount"), `${label} should give 15-concurrency arithmetic`, commanderConcurrency);
|
||||
assertCondition(String(activity.activeQueueIdsNote ?? "").includes("zero local queue ids does not mean zero active runners"), `${label} activity note should prevent zero-active misread`, activity);
|
||||
assertCondition(String(activity.interpretation ?? "").includes("continue supervision"), `${label} activity interpretation should keep supervision action`, activity);
|
||||
}
|
||||
|
||||
export function runCodeQueueQueuesShapeContract(): JsonRecord {
|
||||
const fetcher = (path: string): JsonRecord => {
|
||||
assertCondition(path === "/api/microservices/code-queue/proxy/api/queues", "codex queues should use stable proxy path", { path });
|
||||
return fixtureResponse();
|
||||
};
|
||||
const splitBrainFetcher = (path: string): JsonRecord => {
|
||||
assertCondition(path === "/api/microservices/code-queue/proxy/api/queues", "codex queues should use stable proxy path", { path });
|
||||
return splitBrainLiveResponse();
|
||||
};
|
||||
|
||||
const summary = codexQueuesQueryForTest([], fetcher);
|
||||
assertQueuesShape("summary", summary, "summary");
|
||||
@@ -140,6 +251,11 @@ export function runCodeQueueQueuesShapeContract(): JsonRecord {
|
||||
assertCondition(offsetFullQueues.hasPrevious === true, "offset full should expose previous page", offsetFullQueues);
|
||||
assertCondition(asRecord(asArray(offsetFullQueues.items)[0]).id === "gamma", "offset full should return second page rows", offsetFullQueues);
|
||||
|
||||
const splitSummary = codexQueuesQueryForTest([], splitBrainFetcher);
|
||||
assertSplitBrainLiveActivity("split-brain summary", splitSummary);
|
||||
const splitFull = codexQueuesQueryForTest(["--full"], splitBrainFetcher);
|
||||
assertSplitBrainLiveActivity("split-brain full", splitFull);
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
checks: [
|
||||
@@ -150,6 +266,8 @@ export function runCodeQueueQueuesShapeContract(): JsonRecord {
|
||||
"deprecated full array omitted from default output",
|
||||
"full explicit limit remains bounded and paged",
|
||||
"offset pagination",
|
||||
"split-brain live activity counts distinguish scheduler-local queues, DB running tasks, and heartbeat-fresh runners",
|
||||
"commander concurrency block names the active runner count and 15-concurrency rule",
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
@@ -82,6 +82,14 @@ function fixtureResponse(path: string): JsonRecord {
|
||||
body: {
|
||||
ok: true,
|
||||
queue: {
|
||||
counts: {
|
||||
running: 15,
|
||||
judging: 0,
|
||||
queued: 4,
|
||||
retry_wait: 1,
|
||||
succeeded: 13,
|
||||
},
|
||||
maxActiveQueues: 12,
|
||||
executionDiagnostics: {
|
||||
state: "split-brain",
|
||||
splitBrain: true,
|
||||
@@ -112,7 +120,7 @@ function fixtureResponse(path: string): JsonRecord {
|
||||
pagination: {
|
||||
limit: 200,
|
||||
returned: 15,
|
||||
total: 15,
|
||||
total: 33,
|
||||
hasMore: false,
|
||||
nextBeforeId: null,
|
||||
includeActive: true,
|
||||
@@ -152,6 +160,13 @@ function manyRunningFixtureResponse(path: string): JsonRecord {
|
||||
body: {
|
||||
ok: true,
|
||||
queue: {
|
||||
counts: {
|
||||
running: 40,
|
||||
judging: 0,
|
||||
queued: 7,
|
||||
retry_wait: 2,
|
||||
},
|
||||
maxActiveQueues: 50,
|
||||
executionDiagnostics: {
|
||||
state: "healthy",
|
||||
databaseActiveTaskCount: 40,
|
||||
@@ -174,11 +189,66 @@ function manyRunningFixtureResponse(path: string): JsonRecord {
|
||||
};
|
||||
}
|
||||
|
||||
function splitBrainLiveSupervisorFixtureResponse(path: string): JsonRecord {
|
||||
if (path.includes("/summary")) return fixtureResponse(path);
|
||||
assertCondition(path.startsWith("/api/microservices/code-queue/proxy/api/tasks/overview"), "unexpected path", { path });
|
||||
const liveTaskIds = manyIds("split-live", 8);
|
||||
const tasks = liveTaskIds.map((taskId, index) => task(
|
||||
taskId,
|
||||
"running",
|
||||
`2026-05-22T01:${String(50 - index).padStart(2, "0")}:00.000Z`,
|
||||
));
|
||||
return {
|
||||
ok: true,
|
||||
status: 200,
|
||||
body: {
|
||||
ok: true,
|
||||
queue: {
|
||||
counts: { running: 8 },
|
||||
activeQueueIds: [],
|
||||
activeTaskIds: [],
|
||||
activeRunSlotCount: 0,
|
||||
databaseActiveTaskCount: 8,
|
||||
executionDiagnostics: {
|
||||
state: "split-brain",
|
||||
splitBrain: true,
|
||||
splitBrainLive: true,
|
||||
effectiveLiveness: "live",
|
||||
recommendedAction: "continue-supervision",
|
||||
databaseActiveTaskCount: 8,
|
||||
databaseActiveTaskIds: liveTaskIds,
|
||||
schedulerActiveRunSlotCount: 0,
|
||||
schedulerActiveTaskIds: [],
|
||||
activeHeartbeatCount: 8,
|
||||
activeHeartbeatTaskIds: liveTaskIds,
|
||||
heartbeatFreshTaskIds: liveTaskIds,
|
||||
heartbeatExpiredTaskIds: [],
|
||||
heartbeatMissingTaskIds: [],
|
||||
staleRecoveryCandidateTaskIds: [],
|
||||
heartbeatRiskTaskIds: [],
|
||||
},
|
||||
},
|
||||
pagination: {
|
||||
limit: 200,
|
||||
returned: 8,
|
||||
total: 8,
|
||||
hasMore: false,
|
||||
nextBeforeId: null,
|
||||
includeActive: true,
|
||||
},
|
||||
tasks,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function runCodeQueueSupervisorDisclosureContract(): JsonRecord {
|
||||
const supervisor = codexTasksQueryForTest(["--view", "supervisor", "--limit", "20"], fixtureResponse);
|
||||
const cappedLimit = codexTasksQueryForTest(["--view", "supervisor", "--limit", "260"], fixtureResponse);
|
||||
const full = codexTasksQueryForTest(["--view", "full", "--limit", "20"], fixtureResponse);
|
||||
const cappedFull = codexTasksQueryForTest(["--view", "full", "--limit", "260"], fixtureResponse);
|
||||
const runningFiltered = codexTasksQueryForTest(["--status", "running", "--limit", "40"], manyRunningFixtureResponse);
|
||||
const unreadFiltered = codexTasksQueryForTest(["--unread", "--limit", "20"], fixtureResponse);
|
||||
const splitBrainLive = codexTasksQueryForTest(["--view", "supervisor", "--limit", "20"], splitBrainLiveSupervisorFixtureResponse);
|
||||
|
||||
const supervisorBody = JSON.stringify(supervisor);
|
||||
const fullBody = JSON.stringify(full);
|
||||
@@ -186,6 +256,7 @@ export function runCodeQueueSupervisorDisclosureContract(): JsonRecord {
|
||||
const unreadFilteredBody = JSON.stringify(unreadFiltered);
|
||||
const supervisorData = asRecord(supervisor);
|
||||
const supervisorView = asRecord(supervisorData.supervisor);
|
||||
const cappedSupervisorView = asRecord(asRecord(cappedLimit).supervisor);
|
||||
const runningFilteredView = asRecord(asRecord(runningFiltered).supervisor);
|
||||
const runningFilteredSection = asRecord(runningFilteredView.running);
|
||||
const unreadFilteredView = asRecord(asRecord(unreadFiltered).supervisor);
|
||||
@@ -197,12 +268,36 @@ export function runCodeQueueSupervisorDisclosureContract(): JsonRecord {
|
||||
const fullItem = asRecord(asArray(asRecord(asRecord(full).tasks).items)[0]);
|
||||
const completedUnread = asRecord(supervisorView.completedUnread);
|
||||
const fullTasks = asRecord(asRecord(full).tasks);
|
||||
const cappedFullTasks = asRecord(asRecord(cappedFull).tasks);
|
||||
const diagnostics = asRecord(supervisorView.executionDiagnostics);
|
||||
const filters = asRecord(supervisorView.filters);
|
||||
const activeRunning = asRecord(supervisorView.activeRunning);
|
||||
const activeRunningRowPage = asRecord(activeRunning.rowPage);
|
||||
const activeRunningRedline = asRecord(activeRunning.redline);
|
||||
const activeRunningCommands = asRecord(activeRunning.commands);
|
||||
const counts = asRecord(supervisorView.counts);
|
||||
const outputBudget = asRecord(asRecord(disclosure.outputBudget));
|
||||
const listBudget = asRecord(diagnostics.listBudget);
|
||||
const omittedCounts = asRecord(listBudget.omittedCounts);
|
||||
const splitBrainLiveView = asRecord(asRecord(splitBrainLive).supervisor);
|
||||
const splitBrainLiveActivity = asRecord(splitBrainLiveView.activity);
|
||||
const splitBrainLiveConcurrency = asRecord(splitBrainLiveView.commanderConcurrency);
|
||||
const splitBrainLiveCounts = asRecord(splitBrainLiveView.counts);
|
||||
const cappedFilters = asRecord(cappedSupervisorView.filters);
|
||||
const cappedSource = asRecord(cappedSupervisorView.source);
|
||||
const cappedLimitPolicy = asRecord(asRecord(cappedSupervisorView.disclosure).limitPolicy);
|
||||
const cappedCommands = asRecord(cappedSupervisorView.commands);
|
||||
const cappedFullFilters = asRecord(cappedFullTasks.filters);
|
||||
const cappedFullSource = asRecord(cappedFullTasks.source);
|
||||
|
||||
assertCondition(supervisorBody.length < fullBody.length * 0.55, "supervisor output should be materially smaller than full output", { supervisorChars: supervisorBody.length, fullChars: fullBody.length });
|
||||
assertCondition(supervisorBody.length < 45_000, "supervisor output should remain bounded even with large diagnostics", { supervisorChars: supervisorBody.length });
|
||||
assertCondition(cappedFilters.requestedLimit === 260 && cappedFilters.effectiveLimit === 100 && cappedFilters.limit === 100 && cappedFilters.limitCapped === true, "supervisor filters should disclose requested and capped effective limit", cappedFilters);
|
||||
assertCondition(cappedSource.requestedLimit === 200 && cappedSource.effectiveLimit === 200 && cappedSource.limit === 200 && cappedSource.returned === 15, "supervisor source should disclose independent overview fetch limit", cappedSource);
|
||||
assertCondition(cappedLimitPolicy.requestedLimit === 260 && cappedLimitPolicy.effectiveLimit === 100 && cappedLimitPolicy.sourceFetchLimit === 200 && cappedLimitPolicy.sourceEffectiveLimit === 200, "supervisor disclosure should summarize requested/effective/source limits", cappedLimitPolicy);
|
||||
assertCondition(String(cappedCommands.refresh ?? "").includes("--limit 260") && String(cappedCommands.byStatus ?? "").includes("--limit 260"), "supervisor follow-up commands should preserve requested limit", cappedCommands);
|
||||
assertCondition(cappedFullFilters.requestedLimit === 260 && cappedFullFilters.effectiveLimit === 100 && cappedFullFilters.limitCapped === true, "full view filters should disclose capped requested limit", cappedFullFilters);
|
||||
assertCondition(cappedFullSource.requestedLimit === 200 && cappedFullSource.effectiveLimit === 200, "full view source should disclose independent overview fetch limit", cappedFullSource);
|
||||
assertCondition(recentItems.length === 3, "recentCompleted should be capped below --limit by default", { returned: recentItems.length });
|
||||
assertCondition(asArray(completedUnread.items).length === 3, "completedUnread should be locally paged and kept separate from recentCompleted", completedUnread);
|
||||
assertCondition(recentItems.every((item) => asRecord(item).unreadTerminal === false), "recentCompleted should not duplicate unread terminal tasks", { recentItems });
|
||||
@@ -229,26 +324,58 @@ export function runCodeQueueSupervisorDisclosureContract(): JsonRecord {
|
||||
assertCondition(fullItem.status === "running" && String(fullItem.statusLabel ?? "").includes("awaiting terminal/judge"), "full view should keep raw status while exposing derived closeout label", fullItem);
|
||||
assertCondition(fullItem.awaitingTerminalJudge === true && fullItem.closeoutState === "awaiting-terminal-or-judge", "full view should expose awaiting terminal/judge state", fullItem);
|
||||
assertCondition(fullTasks.returned === 15, "full view must not inherit supervisor recentCompleted cap", fullTasks);
|
||||
const budget = asRecord(disclosure.outputBudget);
|
||||
assertCondition(budget.recentCompletedReturnedLimit === 3 && budget.sectionReturnedLimit === 3, "supervisor must expose output budget metadata", disclosure);
|
||||
assertCondition(filters.requestedLimit === 20 && filters.limit === 20 && filters.limitCapped === false, "supervisor filters should disclose requested vs effective limit", filters);
|
||||
assertCondition(outputBudget.requestedLimit === 20 && outputBudget.effectiveLimit === 20 && outputBudget.sectionReturnedLimit === 3, "supervisor must expose output budget metadata", outputBudget);
|
||||
assertCondition(activeRunning.count === 15 && activeRunning.exact === true && activeRunning.source === "queue-summary-counts", "activeRunning should expose exact running+judging count from queue summary", activeRunning);
|
||||
assertCondition(activeRunningRowPage.returned === 1 && activeRunningRowPage.returnedLimit === 3 && activeRunningRowPage.distinction.includes("row page"), "activeRunning row page should distinguish returned rows from active count", activeRunningRowPage);
|
||||
assertCondition(activeRunningRedline.countField === "supervisor.activeRunning.count" && activeRunningRedline.hardRedline === 15 && activeRunningRedline.state === "at-or-over-hard-redline", "activeRunning redline should name count field and interpretation", activeRunningRedline);
|
||||
assertCondition(counts.activeRunningCount === 15 && counts.activeRunningExact === true && counts.activeRunningRowsReturned === 1, "supervisor counts should separate active count from returned running rows", counts);
|
||||
assertCondition(String(activeRunningCommands.running ?? "").includes("--status running,judging"), "activeRunning should provide running drilldown", activeRunningCommands);
|
||||
assertCondition(asArray(runningFilteredSection.items).length === 3, "running status filter should be locally paged below --limit", runningFilteredSection);
|
||||
assertCondition(runningFilteredSection.count === 40 && runningFilteredSection.hasMore === true, "running status filter should preserve count and hasMore", runningFilteredSection);
|
||||
assertCondition(String(asRecord(runningFilteredSection.commands).next ?? "").includes("--before-id task-running-03"), "running status filter should provide next page command", runningFilteredSection);
|
||||
assertCondition(runningFilteredBody.length < 14_000, "running status filter output should remain bounded", { chars: runningFilteredBody.length });
|
||||
assertCondition(asArray(unreadFilteredSection.items).length <= 3, "unread list should be locally paged below --limit", unreadFilteredSection);
|
||||
assertCondition(unreadFilteredBody.length < 14_000, "unread output should remain bounded", { chars: unreadFilteredBody.length });
|
||||
assertCondition(splitBrainLiveCounts.running === 8, "split-brain supervisor should preserve DB running task count", splitBrainLiveCounts);
|
||||
assertCondition(splitBrainLiveCounts.commanderActiveRunnerCount === 8, "split-brain supervisor should mirror commander active count in counts", splitBrainLiveCounts);
|
||||
assertCondition(splitBrainLiveCounts.effectiveActive === 8, "split-brain supervisor should foreground effective active count", splitBrainLiveCounts);
|
||||
assertCondition(splitBrainLiveCounts.databaseRunning === 8, "split-brain supervisor should distinguish database running tasks", splitBrainLiveCounts);
|
||||
assertCondition(splitBrainLiveCounts.heartbeatFreshActive === 8, "split-brain supervisor should distinguish heartbeat-effective active runners", splitBrainLiveCounts);
|
||||
assertCondition(splitBrainLiveCounts.schedulerLocalActiveQueues === 0, "split-brain supervisor should preserve zero scheduler-local active queues", splitBrainLiveCounts);
|
||||
assertCondition(splitBrainLiveActivity.effectiveActiveTaskCount === 8, "split-brain supervisor activity should expose effective active count", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.effectiveActiveSource === "heartbeat-fresh", "split-brain supervisor activity should prefer heartbeat-fresh source", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.databaseRunningTaskCount === 8, "split-brain supervisor activity should expose DB running count", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.heartbeatFreshActiveTaskCount === 8, "split-brain supervisor activity should expose heartbeat-fresh active count", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.schedulerLocalActiveQueueCount === 0, "split-brain supervisor activity should expose scheduler-local queue count", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.schedulerLocalActiveRunSlotCount === 0, "split-brain supervisor activity should expose scheduler-local slot count", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.splitBrainLive === true, "split-brain supervisor activity should mark live split-brain", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.splitBrainDisposition === "live-count-as-active", "split-brain supervisor activity should classify live split-brain as active capacity", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveActivity.commanderConcurrency !== undefined, "split-brain supervisor activity should include commander concurrency guidance", splitBrainLiveActivity);
|
||||
assertCondition(splitBrainLiveConcurrency.activeRunnerCount === 8, "split-brain supervisor should expose commander-facing active runner count", splitBrainLiveConcurrency);
|
||||
assertCondition(splitBrainLiveConcurrency.activeRunnerCountField === "activity.effectiveActiveTaskCount", "split-brain supervisor should name the field to use", splitBrainLiveConcurrency);
|
||||
assertCondition(splitBrainLiveConcurrency.splitBrainDisposition === "live-count-as-active", "split-brain supervisor should explain live split-brain disposition", splitBrainLiveConcurrency);
|
||||
assertCondition(splitBrainLiveConcurrency.interventionRequired === false, "fresh split-brain supervisor should not require intervention", splitBrainLiveConcurrency);
|
||||
assertCondition(String(splitBrainLiveConcurrency.decisionRule ?? "").includes("15 - activeRunnerCount"), "split-brain supervisor should give 15-concurrency arithmetic", splitBrainLiveConcurrency);
|
||||
assertCondition(String(splitBrainLiveActivity.activeQueueIdsNote ?? "").includes("zero local queue ids does not mean zero active runners"), "split-brain supervisor activity should explain activeQueueIds are local-only", splitBrainLiveActivity);
|
||||
assertCondition(String(splitBrainLiveActivity.interpretation ?? "").includes("continue supervision"), "split-brain supervisor activity should not imply scheduler stoppage", splitBrainLiveActivity);
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
checks: [
|
||||
"supervisor output materially smaller than full",
|
||||
"recentCompleted capped",
|
||||
"explicit --limit cap disclosed",
|
||||
"running/unread locally paged",
|
||||
"split-brain diagnostics capped",
|
||||
"active running exact count exposed",
|
||||
"requested/effective/returned limits disclosed",
|
||||
"prompt/body previews bounded",
|
||||
"running finalResponse rows labeled awaiting terminal/judge",
|
||||
"drill-down commands preserved",
|
||||
"full view remains detailed",
|
||||
"split-brain live supervisor activity distinguishes scheduler-local, database, and heartbeat counts",
|
||||
"commander concurrency block names the active runner count and 15-concurrency rule",
|
||||
],
|
||||
supervisorChars: supervisorBody.length,
|
||||
fullChars: fullBody.length,
|
||||
|
||||
+395
-22
@@ -14,6 +14,7 @@ const maxOutputPreviewChars = 1200;
|
||||
const defaultTasksLimit = 20;
|
||||
const defaultQueuesLimit = 8;
|
||||
const maxTasksLimit = 100;
|
||||
const tasksOverviewSourceFetchLimit = 200;
|
||||
const supervisorSectionReturnedLimit = 3;
|
||||
const supervisorRecentCompletedLimit = 3;
|
||||
const supervisorPromptPreviewChars = 70;
|
||||
@@ -189,6 +190,7 @@ interface CompactTaskMutationResponseOptions {
|
||||
|
||||
interface CodexTasksOptions {
|
||||
queueId: string | undefined;
|
||||
requestedLimit: number;
|
||||
limit: number;
|
||||
beforeId: string | undefined;
|
||||
unreadOnly: boolean;
|
||||
@@ -288,6 +290,14 @@ interface CodexTasksDegraded {
|
||||
reason: string;
|
||||
}
|
||||
|
||||
interface SupervisorStatusCounts {
|
||||
counts: Record<string, number>;
|
||||
exact: boolean;
|
||||
source: "queue-summary-counts" | "overview-page-fallback";
|
||||
scope: "all-queues" | "queue";
|
||||
queueId: string | null;
|
||||
}
|
||||
|
||||
interface CodexQueuesOptions {
|
||||
full: boolean;
|
||||
limit: number;
|
||||
@@ -433,6 +443,10 @@ function asNumber(value: unknown, fallback = 0): number {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : fallback;
|
||||
}
|
||||
|
||||
function finiteNumber(value: unknown): number | null {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : null;
|
||||
}
|
||||
|
||||
function asBoolean(value: unknown): boolean {
|
||||
return value === true || value === 1 || value === "1" || value === "true";
|
||||
}
|
||||
@@ -1304,6 +1318,149 @@ function compactQueueExecutionDiagnostics(value: unknown): Record<string, unknow
|
||||
};
|
||||
}
|
||||
|
||||
function firstFiniteNumber(...values: unknown[]): number | null {
|
||||
for (const value of values) {
|
||||
const number = finiteNumber(value);
|
||||
if (number !== null) return number;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function stringListCount(value: unknown): number | null {
|
||||
return Array.isArray(value) ? stringList(value).length : null;
|
||||
}
|
||||
|
||||
function compactCodeQueueActivity(
|
||||
queue: Record<string, unknown>,
|
||||
diagnostics: Record<string, unknown> | null,
|
||||
options: { schedulerLocalActiveQueueIds?: string[]; runnableQueueCount?: number | null } = {},
|
||||
): Record<string, unknown> {
|
||||
const rawDiagnostics = asRecord(queue.executionDiagnostics) ?? {};
|
||||
const compactDiagnostics = diagnostics ?? {};
|
||||
const rawLiveness = asRecord(rawDiagnostics.liveness) ?? {};
|
||||
const compactLiveness = asRecord(compactDiagnostics.liveness) ?? {};
|
||||
const counts = asRecord(queue.counts) ?? {};
|
||||
const schedulerLocalActiveQueueIds = options.schedulerLocalActiveQueueIds ?? stringList(queue.activeQueueIds);
|
||||
const databaseRunningTaskCount = firstFiniteNumber(counts.running) ?? 0;
|
||||
const databaseJudgingTaskCount = firstFiniteNumber(counts.judging) ?? 0;
|
||||
const databaseActiveTaskCount = firstFiniteNumber(
|
||||
rawDiagnostics.databaseActiveTaskCount,
|
||||
queue.databaseActiveTaskCount,
|
||||
compactDiagnostics.databaseActiveTaskCount,
|
||||
stringListCount(rawDiagnostics.databaseActiveTaskIds),
|
||||
stringListCount(queue.databaseActiveTaskIds),
|
||||
databaseRunningTaskCount + databaseJudgingTaskCount,
|
||||
) ?? 0;
|
||||
const heartbeatFreshActiveTaskCount = firstFiniteNumber(
|
||||
rawLiveness.heartbeatFreshTaskCount,
|
||||
compactLiveness.heartbeatFreshTaskCount,
|
||||
stringListCount(rawDiagnostics.heartbeatFreshTaskIds),
|
||||
stringListCount(compactDiagnostics.heartbeatFreshTaskIds),
|
||||
) ?? 0;
|
||||
const activeHeartbeatTaskCount = firstFiniteNumber(
|
||||
rawDiagnostics.activeHeartbeatCount,
|
||||
compactDiagnostics.activeHeartbeatCount,
|
||||
stringListCount(rawDiagnostics.activeHeartbeatTaskIds),
|
||||
heartbeatFreshActiveTaskCount,
|
||||
) ?? 0;
|
||||
const heartbeatRiskTaskCount = firstFiniteNumber(
|
||||
rawLiveness.heartbeatRiskTaskCount,
|
||||
compactLiveness.heartbeatRiskTaskCount,
|
||||
stringListCount(rawDiagnostics.heartbeatRiskTaskIds),
|
||||
stringListCount(compactDiagnostics.heartbeatRiskTaskIds),
|
||||
) ?? 0;
|
||||
const staleRecoveryCandidateTaskCount = firstFiniteNumber(
|
||||
rawLiveness.staleRecoveryCandidateTaskCount,
|
||||
compactLiveness.staleRecoveryCandidateTaskCount,
|
||||
stringListCount(rawDiagnostics.staleRecoveryCandidateTaskIds),
|
||||
stringListCount(compactDiagnostics.staleRecoveryCandidateTaskIds),
|
||||
) ?? 0;
|
||||
const schedulerLocalActiveRunSlotCount = firstFiniteNumber(
|
||||
rawDiagnostics.schedulerActiveRunSlotCount,
|
||||
queue.schedulerActiveRunSlotCount,
|
||||
queue.activeRunSlotCount,
|
||||
compactDiagnostics.schedulerActiveRunSlotCount,
|
||||
);
|
||||
const runnableQueueCount = firstFiniteNumber(options.runnableQueueCount, queue.runnableQueueCount);
|
||||
const effectiveActiveTaskCount = Math.max(databaseActiveTaskCount, databaseRunningTaskCount, heartbeatFreshActiveTaskCount);
|
||||
const executionState = asString(rawDiagnostics.state ?? compactDiagnostics.state);
|
||||
const effectiveLiveness = asString(rawDiagnostics.effectiveLiveness ?? compactDiagnostics.effectiveLiveness ?? rawLiveness.effectiveLiveness ?? compactLiveness.effectiveLiveness);
|
||||
const recommendedAction = asString(rawDiagnostics.recommendedAction ?? compactDiagnostics.recommendedAction ?? rawLiveness.recommendedAction ?? compactLiveness.recommendedAction);
|
||||
const splitBrain = asBoolean(rawDiagnostics.splitBrain) || asBoolean(compactDiagnostics.splitBrain) || executionState === "split-brain";
|
||||
const splitBrainLive = splitBrainLiveFromDiagnostics(rawDiagnostics) || splitBrainLiveFromDiagnostics(compactDiagnostics);
|
||||
const effectiveActiveSource = heartbeatFreshActiveTaskCount > 0 && heartbeatFreshActiveTaskCount >= databaseActiveTaskCount
|
||||
? "heartbeat-fresh"
|
||||
: databaseActiveTaskCount > 0
|
||||
? "database-active"
|
||||
: schedulerLocalActiveQueueIds.length > 0 || (schedulerLocalActiveRunSlotCount ?? 0) > 0
|
||||
? "scheduler-local"
|
||||
: "none";
|
||||
const activeQueueIdsNote = schedulerLocalActiveQueueIds.length === 0 && effectiveActiveTaskCount > 0
|
||||
? "activeQueueIds are scheduler-local only; zero local queue ids does not mean zero active runners when database or heartbeat counts are nonzero."
|
||||
: "activeQueueIds are scheduler-local active-run slots; use effectiveActiveTaskCount for commander concurrency decisions.";
|
||||
const recommendedActionIntervenes = recommendedAction.length > 0 && recommendedAction !== "continue-supervision";
|
||||
const interventionRequired = heartbeatRiskTaskCount > 0 || staleRecoveryCandidateTaskCount > 0 || recommendedActionIntervenes || (splitBrain && !splitBrainLive);
|
||||
const splitBrainDisposition = splitBrainLive
|
||||
? "live-count-as-active"
|
||||
: splitBrain
|
||||
? "risk-investigate-before-new-work"
|
||||
: "not-split-brain";
|
||||
const splitBrainReason = splitBrainLive
|
||||
? "database active/running tasks have fresh heartbeat evidence and no heartbeat-risk candidates in the compact summary."
|
||||
: splitBrain
|
||||
? "split-brain without fresh-heartbeat live evidence is risky; inspect heartbeat risk, stale recovery candidates, and raw diagnostics before changing capacity."
|
||||
: "control-plane and execution-plane activity signals are not split-brain.";
|
||||
const interventionReason = splitBrainLive
|
||||
? "fresh heartbeat makes split-brain live; count these runners as active and continue supervision."
|
||||
: heartbeatRiskTaskCount > 0
|
||||
? "heartbeat risk is present; inspect before adding replacement work or recovering tasks."
|
||||
: staleRecoveryCandidateTaskCount > 0
|
||||
? "stale recovery candidates are present; follow the recovery runbook before changing concurrency."
|
||||
: recommendedActionIntervenes
|
||||
? `execution diagnostics recommend ${recommendedAction}; intervene before adding work.`
|
||||
: splitBrain
|
||||
? "split-brain is not proven live; inspect raw diagnostics before treating capacity as available."
|
||||
: "no intervention signal in compact activity summary.";
|
||||
const commanderConcurrency = {
|
||||
activeRunnerCount: effectiveActiveTaskCount,
|
||||
activeRunnerCountField: "activity.effectiveActiveTaskCount",
|
||||
activeRunnerCountSource: effectiveActiveSource,
|
||||
decisionRule: "subtract activeRunnerCount from the commander concurrency target; for a 15-runner policy, remaining slots = 15 - activeRunnerCount.",
|
||||
splitBrainDisposition,
|
||||
splitBrainReason,
|
||||
interventionRequired,
|
||||
interventionReason,
|
||||
};
|
||||
return {
|
||||
effectiveActiveTaskCount,
|
||||
effectiveActiveSource,
|
||||
databaseRunningTaskCount,
|
||||
databaseActiveTaskCount,
|
||||
heartbeatFreshActiveTaskCount,
|
||||
activeHeartbeatTaskCount,
|
||||
heartbeatRiskTaskCount,
|
||||
staleRecoveryCandidateTaskCount,
|
||||
schedulerLocalActiveQueueCount: schedulerLocalActiveQueueIds.length,
|
||||
schedulerLocalActiveRunSlotCount,
|
||||
runnableQueueCount,
|
||||
effectiveLiveness: effectiveLiveness || null,
|
||||
recommendedAction: recommendedAction || null,
|
||||
splitBrainLive,
|
||||
splitBrainDisposition,
|
||||
splitBrainReason,
|
||||
commanderConcurrency,
|
||||
activeQueueIdsScope: "scheduler-local-active-run-slots",
|
||||
activeQueueIdsNote,
|
||||
interpretation: splitBrainLive
|
||||
? "split-brain live: database-active tasks have fresh scheduler heartbeat; continue supervision."
|
||||
: heartbeatRiskTaskCount > 0
|
||||
? "heartbeat risk is present; investigate before retry or recovery."
|
||||
: effectiveActiveTaskCount > 0
|
||||
? "active work is present; compare database, heartbeat, and scheduler-local counts before changing concurrency."
|
||||
: "no active work observed by database, heartbeat, or scheduler-local signals.",
|
||||
};
|
||||
}
|
||||
|
||||
function supervisorExecutionDiagnostics(value: unknown): Record<string, unknown> | null {
|
||||
const diagnostics = compactExecutionDiagnostics(value);
|
||||
if (diagnostics === null) return null;
|
||||
@@ -1704,9 +1861,11 @@ function parseTasksOptions(args: string[]): CodexTasksOptions {
|
||||
const unsupported = statusFilter.filter((status) => !supported.has(status));
|
||||
if (unsupported.length > 0) throw new Error(`unsupported --status value: ${unsupported.join(", ")}; supported: ${Array.from(supported).join(", ")}`);
|
||||
}
|
||||
const requestedLimit = positiveIntegerOption(args, ["--limit"], defaultTasksLimit);
|
||||
return {
|
||||
queueId: optionValue(args, ["--queue", "--queue-id"]),
|
||||
limit: positiveIntegerOption(args, ["--limit"], defaultTasksLimit, maxTasksLimit),
|
||||
requestedLimit,
|
||||
limit: Math.min(requestedLimit, maxTasksLimit),
|
||||
beforeId: optionValue(args, ["--before-id", "--beforeId"]),
|
||||
unreadOnly: hasFlag(args, "--unread-only") || hasFlag(args, "--unread"),
|
||||
statusFilter,
|
||||
@@ -2229,18 +2388,56 @@ function filterTasksForOptions(tasks: Record<string, unknown>[], options: CodexT
|
||||
.filter((task) => !options.unreadOnly || taskUnreadTerminal(task));
|
||||
}
|
||||
|
||||
function baseTaskListOptions(options: CodexTasksOptions): CodexTasksOptions {
|
||||
return {
|
||||
queueId: options.queueId,
|
||||
requestedLimit: options.requestedLimit,
|
||||
limit: options.limit,
|
||||
beforeId: options.beforeId,
|
||||
unreadOnly: options.unreadOnly,
|
||||
statusFilter: options.statusFilter,
|
||||
view: options.view,
|
||||
};
|
||||
}
|
||||
|
||||
function taskListCommand(options: CodexTasksOptions, extra: string[] = []): string {
|
||||
const args = ["codex", "tasks"];
|
||||
if (options.view !== "supervisor") args.push("--view", options.view);
|
||||
if (options.queueId !== undefined) args.push("--queue", options.queueId);
|
||||
if (options.unreadOnly) args.push("--unread");
|
||||
if (options.statusFilter !== null) args.push("--status", options.statusFilter.join(","));
|
||||
if (options.limit !== defaultTasksLimit) args.push("--limit", String(options.limit));
|
||||
if (options.requestedLimit !== defaultTasksLimit) args.push("--limit", String(options.requestedLimit));
|
||||
if (options.beforeId !== undefined) args.push("--before-id", options.beforeId);
|
||||
args.push(...extra);
|
||||
return `bun scripts/cli.ts ${args.join(" ")}`;
|
||||
}
|
||||
|
||||
function codexTasksLimitDisclosure(options: CodexTasksOptions): Record<string, unknown> {
|
||||
return {
|
||||
requestedLimit: options.requestedLimit,
|
||||
effectiveLimit: options.limit,
|
||||
maxLimit: maxTasksLimit,
|
||||
limitCapped: options.requestedLimit > options.limit,
|
||||
};
|
||||
}
|
||||
|
||||
function codexTasksSourceDisclosure(pagination: Record<string, unknown>): Record<string, unknown> {
|
||||
const effectiveLimit = asNumber(pagination.limit, 0) || null;
|
||||
return {
|
||||
endpoint: "/api/tasks/overview",
|
||||
requestedLimit: tasksOverviewSourceFetchLimit,
|
||||
fetchLimit: tasksOverviewSourceFetchLimit,
|
||||
limit: effectiveLimit,
|
||||
effectiveLimit,
|
||||
limitCapped: effectiveLimit !== null && effectiveLimit < tasksOverviewSourceFetchLimit,
|
||||
returned: asNumber(pagination.returned, 0) || null,
|
||||
total: asNumber(pagination.total, 0) || null,
|
||||
hasMore: asBoolean(pagination.hasMore),
|
||||
nextBeforeId: asString(pagination.nextBeforeId) || null,
|
||||
includeActive: asBoolean(pagination.includeActive),
|
||||
};
|
||||
}
|
||||
|
||||
function taskSectionLimit(options: CodexTasksOptions, maxReturned = supervisorSectionReturnedLimit): number {
|
||||
return Math.min(options.limit, maxReturned);
|
||||
}
|
||||
@@ -2259,6 +2456,155 @@ function sectionNextCommand(
|
||||
return sourceNextCommand;
|
||||
}
|
||||
|
||||
function countRecordValues(value: unknown): Record<string, number> {
|
||||
const record = asRecord(value);
|
||||
const counts: Record<string, number> = {};
|
||||
if (record === null) return counts;
|
||||
for (const [status, rawCount] of Object.entries(record)) {
|
||||
const count = typeof rawCount === "number" ? rawCount : typeof rawCount === "string" ? Number(rawCount) : Number.NaN;
|
||||
if (Number.isFinite(count) && count >= 0) counts[status] = Math.floor(count);
|
||||
}
|
||||
return counts;
|
||||
}
|
||||
|
||||
function queueSummaryCountsForOptions(taskPage: CodexTasksTaskPage, options: CodexTasksOptions): SupervisorStatusCounts | null {
|
||||
const queue = taskPage.queue;
|
||||
if (queue === null) return null;
|
||||
if (options.queueId !== undefined) {
|
||||
for (const row of asArray(queue.queues)) {
|
||||
const record = asRecord(row);
|
||||
if (record === null || asString(record.id) !== options.queueId) continue;
|
||||
if (asRecord(record.counts) === null) return null;
|
||||
return {
|
||||
counts: countRecordValues(record.counts),
|
||||
exact: true,
|
||||
source: "queue-summary-counts",
|
||||
scope: "queue",
|
||||
queueId: options.queueId,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}
|
||||
if (asRecord(queue.counts) === null) return null;
|
||||
return {
|
||||
counts: countRecordValues(queue.counts),
|
||||
exact: true,
|
||||
source: "queue-summary-counts",
|
||||
scope: "all-queues",
|
||||
queueId: null,
|
||||
};
|
||||
}
|
||||
|
||||
function fallbackStatusCountsForOptions(taskPage: CodexTasksTaskPage, options: CodexTasksOptions): SupervisorStatusCounts {
|
||||
const counts: Record<string, number> = {};
|
||||
for (const task of taskPage.tasks) {
|
||||
if (options.queueId !== undefined && asString(task.queueId) !== options.queueId) continue;
|
||||
const status = asString(task.status);
|
||||
if (status.length === 0) continue;
|
||||
counts[status] = (counts[status] ?? 0) + 1;
|
||||
}
|
||||
return {
|
||||
counts,
|
||||
exact: false,
|
||||
source: "overview-page-fallback",
|
||||
scope: options.queueId === undefined ? "all-queues" : "queue",
|
||||
queueId: options.queueId ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
function statusCountsForOptions(taskPage: CodexTasksTaskPage, options: CodexTasksOptions): SupervisorStatusCounts {
|
||||
return queueSummaryCountsForOptions(taskPage, options) ?? fallbackStatusCountsForOptions(taskPage, options);
|
||||
}
|
||||
|
||||
function positiveCount(value: unknown): number | null {
|
||||
const count = asNumber(value, Number.NaN);
|
||||
return Number.isFinite(count) && count >= 0 ? Math.floor(count) : null;
|
||||
}
|
||||
|
||||
function runtimeMaxActiveQueues(taskPage: CodexTasksTaskPage): number | null {
|
||||
const value = positiveCount(taskPage.queue?.maxActiveQueues);
|
||||
return value !== null && value > 0 ? value : null;
|
||||
}
|
||||
|
||||
function activeRunningRedlineState(activeCount: number, hardRedline: number, burstRedline: number, routineTarget: number): string {
|
||||
if (activeCount >= hardRedline) return "at-or-over-hard-redline";
|
||||
if (activeCount >= burstRedline) return "burst-redline";
|
||||
if (activeCount >= routineTarget) return "above-routine-target";
|
||||
return "below-routine-target";
|
||||
}
|
||||
|
||||
function supervisorActiveRunningSummary(
|
||||
taskPage: CodexTasksTaskPage,
|
||||
options: CodexTasksOptions,
|
||||
runningSection: CodexTasksSection<CodexTasksSupervisorEntry>,
|
||||
diagnostics: Record<string, unknown> | null,
|
||||
): Record<string, unknown> {
|
||||
const statusCounts = statusCountsForOptions(taskPage, options);
|
||||
const runningCount = statusCounts.counts.running ?? 0;
|
||||
const judgingCount = statusCounts.counts.judging ?? 0;
|
||||
const activeStatusCount = runningCount + judgingCount;
|
||||
const heartbeatActiveCount = positiveCount(diagnostics?.activeHeartbeatCount);
|
||||
const schedulerLocalActiveRunSlotCount = positiveCount(diagnostics?.schedulerActiveRunSlotCount);
|
||||
const effectiveActiveRunnerCount = Math.max(activeStatusCount, heartbeatActiveCount ?? 0, schedulerLocalActiveRunSlotCount ?? 0);
|
||||
const policy = submitPolicyContract().concurrency;
|
||||
const routineTarget = policy.gpt55Routine;
|
||||
const burstRedline = Math.max(policy.gpt55BurstMax, policy.minimaxSimpleMax);
|
||||
const hardRedline = 15;
|
||||
const runtimeLimit = runtimeMaxActiveQueues(taskPage);
|
||||
const redlineState = activeRunningRedlineState(activeStatusCount, hardRedline, burstRedline, routineTarget);
|
||||
const exactInterpretation = statusCounts.exact
|
||||
? "activeRunning.count is exact from queue status counts; row pagination does not change this count"
|
||||
: "activeRunning.count is page-scoped because queue status counts were absent; do not use it for commander redline decisions without a queue summary/raw cross-check";
|
||||
const baseOptions = baseTaskListOptions({ ...options, beforeId: undefined, unreadOnly: false, statusFilter: null });
|
||||
const runningOptions: CodexTasksOptions = { ...baseOptions, statusFilter: ["running", "judging"] };
|
||||
const queuedOptions: CodexTasksOptions = { ...baseOptions, statusFilter: ["queued", "retry_wait"] };
|
||||
return {
|
||||
count: activeStatusCount,
|
||||
exact: statusCounts.exact,
|
||||
source: statusCounts.source,
|
||||
scope: statusCounts.scope,
|
||||
queueId: statusCounts.queueId,
|
||||
statuses: {
|
||||
running: runningCount,
|
||||
judging: judgingCount,
|
||||
},
|
||||
effectiveActiveRunnerCount,
|
||||
effectiveActiveRunnerCountSources: {
|
||||
activeStatusCount,
|
||||
heartbeatActiveCount,
|
||||
schedulerLocalActiveRunSlotCount,
|
||||
},
|
||||
rowPage: {
|
||||
returned: runningSection.returned,
|
||||
availableInCurrentOverview: runningSection.count,
|
||||
returnedLimit: taskSectionLimit(options),
|
||||
truncated: runningSection.truncated,
|
||||
hasMore: runningSection.hasMore,
|
||||
distinction: "returned is the compact row page; count is the active running total used for supervision",
|
||||
},
|
||||
redline: {
|
||||
countField: "supervisor.activeRunning.count",
|
||||
routineTarget,
|
||||
burstRedline,
|
||||
hardRedline,
|
||||
runtimeMaxActiveQueues: runtimeLimit,
|
||||
state: redlineState,
|
||||
remainingToRoutineTarget: Math.max(0, routineTarget - activeStatusCount),
|
||||
remainingToBurstRedline: Math.max(0, burstRedline - activeStatusCount),
|
||||
remainingToHardRedline: Math.max(0, hardRedline - activeStatusCount),
|
||||
decisionReady: statusCounts.exact,
|
||||
interpretation: exactInterpretation,
|
||||
},
|
||||
commands: {
|
||||
running: taskListCommand(runningOptions),
|
||||
runningFull: taskListCommand({ ...runningOptions, view: "full" }),
|
||||
queued: taskListCommand(queuedOptions),
|
||||
queues: "bun scripts/cli.ts codex queues",
|
||||
nextRunningPage: runningSection.commands.next,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function fetchTaskSummaries(taskIds: string[], fetcher: CodexResponseFetcher): { summaries: Map<string, Record<string, unknown>>; degraded: CodexTasksDegraded | null } {
|
||||
const boundedIds = taskIds.slice(0, maxTasksLimit);
|
||||
const summaries = new Map<string, Record<string, unknown>>();
|
||||
@@ -2299,8 +2645,8 @@ type CodexTasksTaskPage = {
|
||||
|
||||
function tasksListQueryString(options: CodexTasksOptions): string {
|
||||
return queryString({
|
||||
limit: 200,
|
||||
priorityLimit: 200,
|
||||
limit: tasksOverviewSourceFetchLimit,
|
||||
priorityLimit: tasksOverviewSourceFetchLimit,
|
||||
queueId: options.queueId,
|
||||
beforeId: options.beforeId,
|
||||
includeActive: 1,
|
||||
@@ -2355,6 +2701,9 @@ function codexTasksOverviewResult(
|
||||
const queuedSection = buildSupervisorTaskSection(queuedTasks, summaries, sectionLimit, sectionNextCommand(queuedTasks, sectionLimit, options, nextCommand), fullCommand);
|
||||
const pagination = taskPage.pagination;
|
||||
const diagnostics = supervisorExecutionDiagnostics(asRecord(taskPage.queue)?.executionDiagnostics);
|
||||
const activity = compactCodeQueueActivity(asRecord(taskPage.queue) ?? {}, diagnostics);
|
||||
const commanderConcurrency = asRecord(activity.commanderConcurrency) ?? {};
|
||||
const activeRunning = supervisorActiveRunningSummary(taskPage, options, runningSection, diagnostics);
|
||||
const visibleSupervisorItems = [...runningSection.items, ...unreadSection.items, ...recentSection.items, ...queuedSection.items];
|
||||
const classifierCounts = visibleSupervisorItems.reduce((counts, item) => {
|
||||
const key = item.kind;
|
||||
@@ -2368,28 +2717,32 @@ function codexTasksOverviewResult(
|
||||
filters: {
|
||||
view: options.view,
|
||||
queueId: options.queueId ?? null,
|
||||
requestedLimit: options.requestedLimit,
|
||||
limit: options.limit,
|
||||
...codexTasksLimitDisclosure(options),
|
||||
unreadOnly: options.unreadOnly,
|
||||
status: options.statusFilter,
|
||||
beforeId: options.beforeId ?? null,
|
||||
},
|
||||
source: {
|
||||
endpoint: "/api/tasks/overview",
|
||||
queueId: options.queueId ?? null,
|
||||
limit: asNumber(pagination.limit, 0) || null,
|
||||
returned: asNumber(pagination.returned, 0) || null,
|
||||
total: asNumber(pagination.total, 0) || null,
|
||||
hasMore: asBoolean(pagination.hasMore),
|
||||
nextBeforeId: asString(pagination.nextBeforeId) || null,
|
||||
includeActive: asBoolean(pagination.includeActive),
|
||||
},
|
||||
source: { queueId: options.queueId ?? null, ...codexTasksSourceDisclosure(pagination) },
|
||||
bounded: true,
|
||||
disclosure: {
|
||||
defaultView: "supervisor",
|
||||
policy: "bounded summary rows only; prompt/body are short previews and raw detail requires explicit task/full/trace/output commands",
|
||||
limitSemantics: "--limit is request/scanning/page budget; supervisor sections still return compact bounded row pages and expose counts separately",
|
||||
limitPolicy: {
|
||||
...codexTasksLimitDisclosure(options),
|
||||
sourceFetchLimit: tasksOverviewSourceFetchLimit,
|
||||
sourceEffectiveLimit: asNumber(pagination.limit, 0) || null,
|
||||
sourceReturned: asNumber(pagination.returned, 0) || null,
|
||||
note: "codex tasks --limit is capped for CLI output safety; supervisor sections also keep small row caps and expose commands.next for pagination",
|
||||
},
|
||||
outputBudget: {
|
||||
promptPreviewChars: supervisorPromptPreviewChars,
|
||||
bodyPreviewChars: supervisorBodyPreviewChars,
|
||||
requestedLimit: options.requestedLimit,
|
||||
effectiveLimit: options.limit,
|
||||
limitCapped: options.limit < options.requestedLimit,
|
||||
sectionReturnedLimit: sectionLimit,
|
||||
recentCompletedReturnedLimit: recentLimit,
|
||||
recentCompletedBodyPreviewChars: supervisorRecentBodyPreviewChars,
|
||||
@@ -2402,17 +2755,28 @@ function codexTasksOverviewResult(
|
||||
counts: {
|
||||
scanned: allTasks.length,
|
||||
running: runningSection.count,
|
||||
activeRunningCount: activeRunning.count,
|
||||
activeRunningExact: activeRunning.exact,
|
||||
activeRunningRowsReturned: runningSection.returned,
|
||||
completedUnread: unreadSection.count,
|
||||
recentCompleted: recentSection.count,
|
||||
queued: queuedSection.count,
|
||||
commanderActiveRunnerCount: asNumber(commanderConcurrency.activeRunnerCount, 0),
|
||||
effectiveActive: asNumber(activity.effectiveActiveTaskCount, 0),
|
||||
databaseRunning: asNumber(activity.databaseRunningTaskCount, 0),
|
||||
heartbeatFreshActive: asNumber(activity.heartbeatFreshActiveTaskCount, 0),
|
||||
schedulerLocalActiveQueues: asNumber(activity.schedulerLocalActiveQueueCount, 0),
|
||||
},
|
||||
classifierCounts,
|
||||
commanderConcurrency,
|
||||
activity,
|
||||
activeRunning,
|
||||
executionDiagnostics: diagnostics,
|
||||
degraded,
|
||||
commands: {
|
||||
refresh: taskListCommand(options),
|
||||
unread: taskListCommand({ ...options, unreadOnly: true }),
|
||||
byStatus: `bun scripts/cli.ts codex tasks --status <status>${options.queueId === undefined ? "" : ` --queue ${options.queueId}`}${options.limit === defaultTasksLimit ? "" : ` --limit ${options.limit}`}`,
|
||||
byStatus: `bun scripts/cli.ts codex tasks --status <status>${options.queueId === undefined ? "" : ` --queue ${options.queueId}`}${options.requestedLimit === defaultTasksLimit ? "" : ` --limit ${options.requestedLimit}`}`,
|
||||
full: fullCommand,
|
||||
next: nextCommand,
|
||||
},
|
||||
@@ -2443,18 +2807,14 @@ function codexTasksFullResult(
|
||||
filters: {
|
||||
view: "full",
|
||||
queueId: options.queueId ?? null,
|
||||
requestedLimit: options.requestedLimit,
|
||||
limit: options.limit,
|
||||
...codexTasksLimitDisclosure(options),
|
||||
unreadOnly: options.unreadOnly,
|
||||
status: options.statusFilter,
|
||||
beforeId: options.beforeId ?? null,
|
||||
},
|
||||
source: {
|
||||
endpoint: "/api/tasks/overview",
|
||||
returned: asNumber(pagination.returned, 0) || null,
|
||||
total: asNumber(pagination.total, 0) || null,
|
||||
hasMore: asBoolean(pagination.hasMore),
|
||||
nextBeforeId,
|
||||
},
|
||||
source: codexTasksSourceDisclosure(pagination),
|
||||
count: filtered.length,
|
||||
returned: visible.length,
|
||||
hasMore: filtered.length > visible.length || sourceHasMore,
|
||||
@@ -2698,6 +3058,8 @@ function compactQueuesResponse(body: Record<string, unknown>, options: CodexQueu
|
||||
const selected = options.full ? queues : Array.from(new Map([...activeQueues, ...unreadQueues, ...runnableQueues, ...nonemptyQueues].map((row) => [String(row.id), row])).values());
|
||||
const visible = selected.slice(options.offset, options.offset + options.limit);
|
||||
const diagnostics = compactQueueExecutionDiagnostics(queue.executionDiagnostics);
|
||||
const activity = compactCodeQueueActivity(queue, diagnostics, { schedulerLocalActiveQueueIds: activeIds, runnableQueueCount: runnableQueues.length });
|
||||
const commanderConcurrency = asRecord(activity.commanderConcurrency) ?? {};
|
||||
const activeTaskIds = boundedUniqueStringList(queue.activeTaskIds, Math.min(options.limit, maxTasksLimit));
|
||||
const queuedTaskIds = boundedUniqueStringList(queue.queuedTaskIds, Math.min(options.limit, maxTasksLimit));
|
||||
const nextOffset = options.offset + visible.length;
|
||||
@@ -2725,11 +3087,22 @@ function compactQueuesResponse(body: Record<string, unknown>, options: CodexQueu
|
||||
totalTasks: queue.total ?? null,
|
||||
queueCount: queue.queueCount ?? queues.length,
|
||||
activeQueueCount: activeIds.length,
|
||||
activeQueueCountScope: "scheduler-local-active-run-slots",
|
||||
schedulerLocalActiveQueueCount: activeIds.length,
|
||||
commanderActiveRunnerCount: commanderConcurrency.activeRunnerCount,
|
||||
effectiveActiveTaskCount: activity.effectiveActiveTaskCount,
|
||||
databaseRunningTaskCount: activity.databaseRunningTaskCount,
|
||||
databaseActiveTaskCount: activity.databaseActiveTaskCount,
|
||||
heartbeatFreshActiveTaskCount: activity.heartbeatFreshActiveTaskCount,
|
||||
nonemptyQueueCount: nonemptyQueues.length,
|
||||
unreadQueueCount: unreadQueues.length,
|
||||
runnableQueueCount: runnableQueues.length,
|
||||
},
|
||||
commanderConcurrency,
|
||||
activity,
|
||||
activeQueueIds: queue.activeQueueIds ?? [],
|
||||
activeQueueIdsScope: "scheduler-local-active-run-slots",
|
||||
activeQueueIdsNote: activity.activeQueueIdsNote,
|
||||
activeTaskIds: activeTaskIds.items,
|
||||
activeTaskIdsCount: activeTaskIds.count,
|
||||
activeTaskIdsTruncated: activeTaskIds.truncated,
|
||||
|
||||
+15
-3
@@ -56,14 +56,14 @@ export function rootHelp(): unknown {
|
||||
{ command: "codex skills-sync --dry-run [--full]", description: "Inspect the controlled runner skills hostPath lifecycle contract without copying files, restarting services, reading secrets, or mutating live runner paths." },
|
||||
{ command: "codex pr-preflight [--remote] [--push-dry-run --push-dry-run-ref refs/heads/probe/<name>] [--pr-create-dry-run --pr-create-dry-run-head <head>] [--issue N]", description: "Read-only PR admission check against the D601 scheduler/runner token, GitHub egress, repo visibility, skills lifecycle health, optional push dry-run, and PR body/create dry-run guard." },
|
||||
{ command: "codex task <taskId> [--detail] [--trace --tail|--from-start|--after-seq N|--before-seq N --limit N] [--full]", description: "Fetch the bounded review view by default; --detail is still capped, while --full/trace/output explicitly expand evidence." },
|
||||
{ command: "codex tasks [--view supervisor|full] [--queue id] [--status status[,status]] [--unread|--unread-only] [--limit N] [--before-id id]", description: "Show the low-noise supervisor view by default: compact task rows, tiny local sections, diagnostics, and drill-down commands; use --view full for detailed rows." },
|
||||
{ command: "codex tasks [--view supervisor|full] [--queue id] [--status status[,status]] [--unread|--unread-only] [--limit N] [--before-id id]", description: "Show the low-noise supervisor view by default: compact task rows, tiny local sections, activity counts, diagnostics, and drill-down commands; use --view full for detailed rows." },
|
||||
{ command: "codex output <taskId> [--tail|--from-start|--after-seq N|--before-seq N --limit N] [--full-text]", description: "Fetch paged raw Code Queue output records; default caps large limits/text previews, --full-text explicitly expands one seq window." },
|
||||
{ command: "codex read <taskId>", description: "Mark one reviewed terminal task read; never run automatically as part of listing." },
|
||||
{ command: "codex dev-ready", description: "Fetch execution-container readiness, including sanitized skill injection status from /api/dev-ready." },
|
||||
{ command: "codex judge <taskId> --attempt N [--dry-run] [--include-prompt]", description: "Replay one stored Code Queue attempt through the same judge context builder and MiniMax judge call path used by the live queue worker." },
|
||||
{ command: "codex steer <taskId> [prompt|--prompt-file path|--prompt-stdin] [--dry-run] [--no-retry|--retry-attempts N]", description: "Push a corrective prompt into a running Code Queue task; retryable tunnel aborts get bounded retry diagnostics, and real success does not echo prompt text." },
|
||||
{ command: "codex interrupt|cancel <taskId>", description: "Request interrupt for a running Code Queue task, or cancel a queued/retry_wait task, through the same private proxy." },
|
||||
{ command: "codex (queues [--full|--all] | queue create <queueId> | queue merge <sourceQueueId> --into <targetQueueId> | move <taskId> --queue <queueId>)", description: "List low-noise queue summaries by default; full queue rows require --full/--all." },
|
||||
{ command: "codex (queues [--full|--all] | queue create <queueId> | queue merge <sourceQueueId> --into <targetQueueId> | move <taskId> --queue <queueId>)", description: "List low-noise queue summaries by default, including effective activity counts that distinguish scheduler-local queues, DB running tasks, and heartbeat-fresh runners; full queue rows require --full/--all." },
|
||||
{ command: "job list [--limit N] [--include-command]", description: "List async jobs from .state/jobs with a bounded default page." },
|
||||
{ command: "job status <jobId|latest> [--tail-bytes N]", description: "Show job state with bounded stdout/stderr tails." },
|
||||
{ command: "debug health", description: "Probe internal core, nodes, system/Docker status, frontend, provider ingress, and public boundary." },
|
||||
@@ -284,7 +284,19 @@ function codexHelp(): unknown {
|
||||
defaultPolicy: "low-noise JSON by default; write commands confirm persistence, list/detail/output commands return bounded summaries with drill-down commands",
|
||||
expand: ["codex task <taskId> --full", "codex task <taskId> --trace --limit N", "codex output <taskId> --after-seq N --limit N --full-text", "codex tasks --view full --limit N", "codex skills-sync --dry-run --full"],
|
||||
},
|
||||
description: "Operate Code Queue through the stable backend-core private proxy path. Real submit/steer success is a low-noise write confirmation and does not echo prompt text.",
|
||||
activityFields: {
|
||||
path: "data.queues.activity and data.supervisor.activity",
|
||||
effectiveActiveTaskCount: "Commander-facing active count derived from database active/running tasks and heartbeat-fresh runners.",
|
||||
commanderConcurrency: "Use data.supervisor.commanderConcurrency.activeRunnerCount or data.queues.commanderConcurrency.activeRunnerCount for concurrency decisions; the block states the 15-runner arithmetic and intervention signal.",
|
||||
schedulerLocalActiveQueueCount: "Only queues currently visible in this scheduler-local active-run slot view; zero does not override DB or heartbeat activity.",
|
||||
heartbeatFreshActiveTaskCount: "Heartbeat-effective active runner count used to avoid split-brain zero-active mistakes.",
|
||||
},
|
||||
supervisorView: {
|
||||
activeRunning: "Use data.supervisor.activeRunning.count for exact active running+judging count when exact=true; data.supervisor.running.returned is only the compact row page.",
|
||||
redline: "data.supervisor.activeRunning.redline names the count field, routine target, burst redline, hard redline, and decisionReady flag.",
|
||||
limitSemantics: "filters.requestedLimit preserves the user input; filters.limit/effectiveLimit shows the capped query budget; section outputBudget/rowPage show returned-row caps.",
|
||||
},
|
||||
description: "Operate Code Queue through the stable backend-core private proxy path with bounded activity summaries for queue and supervisor views. Real submit/steer success is a low-noise write confirmation and does not echo prompt text.",
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user