feat: add in-cluster Kafka replay diagnostics

This commit is contained in:
Codex
2026-07-12 18:12:53 +02:00
parent bdb569d939
commit 01a004bbeb
6 changed files with 442 additions and 17 deletions
+39 -11
View File
@@ -61,18 +61,46 @@
- Remote k3s targets that share the already-initialized PK01 Sub2API database should use YAML-controlled `target.runtime.autoSetup=false` and a rendered runtime config initContainer instead of rerunning Sub2API auto setup. The app's migration check path is latency-sensitive because it checks many migration rows one by one; repeated remote auto setup against PK01 can time out without indicating DB corruption. Required PostgreSQL `pg_hba` entries and host firewall forward allowances must be declared in YAML and applied through the controlled platform DB/server paths rather than by changing DB passwords, connection strings, or data.
- Sub2API account sentinel, public exposure, and HTTPS egress proxy are target-scoped YAML decisions. The active target may run them when YAML enables them; the standby G14 target must stay deployed but inactive until YAML promotion. `sentinel.enabledOnTargets` is the authority for where Codex-pool sentinel image, CronJob, Secret and state resources are expected; disabled targets should report sentinel validation as skipped instead of failing on missing runtime sentinel objects. Do not create a second sentinel, FRP client, public management surface, or edge proxy by hand; enable or move those resources only through the target YAML and the `platform-infra sub2api` / `codex-pool --target` CLI paths.
## Kafka Event Bus Boundary
## Kafka 事件总线边界
- Kafka for the HWLAB v0.3 / AgentRun v0.2 event-bus POC is a target-scoped UniDesk-operated platform service in namespace `platform-infra`; NC01 is the current validation target for pikasTech/HWLAB#2449. It is not owned by `hwlab-v03`, `agentrun-v02`, a per-lane Kafka namespace, or a service repository deployment file.
- The canonical source of truth is `config/platform-infra/kafka.yaml`; target, namespace, Strimzi release URL, cluster name, storage class/size, topic list, client declarations, DLQ names, runtime switch and validation smoke topic must stay in that YAML. Current version numbers and retention values belong only in YAML, not in this reference.
- The canonical entrypoint is `bun scripts/cli.ts platform-infra kafka plan|apply|status|validate|topics|groups|offsets|tail|produce --target <node>`; `--node <node>` is an equivalent selector for node-targeted operations. Consumer-group retirement uses `platform-infra kafka groups cleanup --node <node> --policy <id> [--confirm]`. Formal mutation must use that path; raw `kubectl` is bounded diagnosis only.
- `platform-infra kafka groups` renders a YAML-bounded real group table plus total/shown/omitted counts by default; `--limit` may raise the view only to the YAML maximum, while `--full` and `--raw` are explicit drill-down paths.
- Consumer-group cleanup is plan-only unless `--confirm` is present and accepts only a policy declared under `config/platform-infra/kafka.yaml#management.consumerGroupCleanup.policies`. A group must match that policy's exact prefix and full name pattern, decode a group-name timestamp old enough for the YAML minimum, and be in a YAML-allowed inactive Kafka state. The candidate limit is also the per-command mutation ceiling; `--full` and `--raw` never expand it. Confirmed execution re-reads broker state after deletion and reports deleted, state-changed and failed groups separately.
- HWLAB v0.3 and AgentRun v0.2 are client namespaces. Runtime readiness alone does not prove Workbench projection, SSE or AgentRun command ingestion has migrated. App producer/consumer switchover is owned by the corresponding service repo and must be verified from that service's original entrypoint, not by Kafka topic readiness alone.
- The event-bus authority split is three topics: `codex-stdio.raw.v1` stores AgentRun runner's raw Codex stdio frames; `agentrun.event.v1` stores AgentRun manager durable event mirrors before HWLAB conversion; `hwlab.event.v1` stores HWLAB event stream after HWLAB-side conversion and is the stream that SSE should proxy to Workbench in the event-bus design. Do not mix these schemas into a single topic or let a downstream consumer rewrite the upstream source topic.
- Shadow produce may write Kafka events for observation only when YAML keeps consumer cutover disabled; it must not replace the current business read path until the corresponding service has an explicit consumer switchover decision and replay/refresh semantics.
- The first target-scoped POC is a single-node KRaft broker for observability, ordering and replay investigation. It improves auditability and smoke coverage, but it is not a production high-availability claim; replication, backup, min ISR and app-side transactional inbox/outbox are separate decisions.
- Kafka must stay ClusterIP-only by default. Do not add Ingress, NodePort, LoadBalancer, host networking, public FRP, or browser-facing Kafka access unless a later YAML-controlled platform decision explicitly changes that boundary.
- HWLAB v0.3 / AgentRun v0.2 事件总线 POC 的 Kafka
- 是 UniDesk 运维的 target-scoped 平台服务;
- 位于 `platform-infra` namespace
- 当前验证 target 为 pikasTech/HWLAB#2449 指定的 NC01
- 不属于 `hwlab-v03``agentrun-v02`、lane 独立 Kafka namespace 或服务仓库部署文件。
- 权威配置为 `config/platform-infra/kafka.yaml`
- target、namespace、Strimzi release URL、cluster name、storage、topic、client、DLQ、runtime switch 和 smoke topic 必须保留在该 YAML
- 当前版本号和 retention 值只写 YAML,不写入本参考文档。
- 平台受控入口为 `bun scripts/cli.ts platform-infra kafka plan|apply|status|validate|topics|groups|offsets|tail|replay|produce --target <node>`
- node-targeted 操作可等价使用 `--node <node>`
- consumer group 退役使用 `platform-infra kafka groups cleanup --node <node> --policy <id> [--confirm]`
- 正式 mutation 必须使用受控入口;
- 原生 `kubectl` 只用于有界诊断。
- 应用侧 Kafka replay 的受控入口:
- 命令为 `bun scripts/cli.ts platform-infra kafka replay agentrun --node <node> --lane <lane> --session-id <id> [--trace-id <id>] --no-publish`
- Kafka target 来自 `config/platform-infra/kafka.yaml`
- AgentRun workload 来自 `config/agentrun.yaml`
- 命令进入应用 Pod 调用运行中 commit 自带的正式 reducer
- 命令同时披露 partition offset window、scanned、accepted、typed rejection、produced 和首个 mismatch identity
- 命令固定 `mutation=false``topicAppended=false``valuesPrinted=false`
- scan limit、应用读取 timeout 和短连接 grace 由 `config/platform-infra/kafka.yaml#management.replay` 控制;
- 机器输出使用 `-o json|yaml`
- `platform-infra kafka groups` 默认输出受 YAML 上限约束的真实 group 表及 total/shown/omitted 计数;`--limit` 不能超过 YAML 上限,`--full``--raw` 是显式下钻入口。
- consumer group cleanup
- 未提供 `--confirm` 时只生成计划;
- 只接受 `config/platform-infra/kafka.yaml#management.consumerGroupCleanup.policies` 声明的 policy
- group 必须匹配 prefix、完整名称、YAML 最小年龄和允许的 inactive state
- candidate limit 同时是单次 mutation 上限,`--full``--raw` 不扩大该上限;
- confirmed 执行在删除后重新读取 broker 状态,分别报告 deleted、state-changed 和 failed。
- HWLAB v0.3 和 AgentRun v0.2 是 client namespaceruntime ready 不能证明 Workbench projection、SSE 或 AgentRun command ingestion 已迁移,producer/consumer 切换必须由对应服务仓库拥有并从原入口验证。
- 事件总线按三个 topic 划分权威:
- `codex-stdio.raw.v1` 保存 AgentRun runner 的 Codex stdio 原始 frame
- `agentrun.event.v1` 保存 HWLAB 转换前的 AgentRun manager durable event mirror
- `hwlab.event.v1` 保存 HWLAB 转换后的事件流,也是 SSE 向 Workbench 代理的流;
- 禁止混合 schema 或由下游 consumer 回写上游 source topic。
- 只有 YAML 保持 consumer cutover disabled 时,shadow produce 才可写观察事件;在服务仓库明确完成 consumer 切换与 replay/refresh 语义前,不得替代当前业务读取路径。
- 首个 target-scoped POC 是用于可观测性、顺序和 replay 调查的单节点 KRaft broker;它提高审计与 smoke 覆盖,但不声明生产高可用,replication、backup、min ISR 和应用 transactional inbox/outbox 属于独立决策。
- Kafka 默认必须保持 ClusterIP-only;除非后续 YAML 控制的平台决策明确改变边界,禁止添加 Ingress、NodePort、LoadBalancer、host networking、public FRP 或浏览器直连 Kafka。
## LangBot Deployment Boundary