feat: manage codex pool account pruning
This commit is contained in:
Codex
@@ -20,15 +20,16 @@ profiles:
|
||||
accountName: unidesk-codex-gptclub
|
||||
configFile: config.toml.gptclub
|
||||
authFile: auth.json.gptclub
|
||||
- profile: pinche
|
||||
accountName: unidesk-codex-pinche
|
||||
configFile: config.toml.pinche
|
||||
authFile: auth.json.pinche
|
||||
- profile: fixwikihub
|
||||
accountName: unidesk-codex-fixwikihub
|
||||
configFile: config.toml.fixwikihub
|
||||
authFile: auth.json.fixwikihub
|
||||
openaiResponsesWebSocketsV2Mode: ctx_pool
|
||||
- profile: 17nas
|
||||
accountName: unidesk-codex-17nas
|
||||
configFile: config.toml.17nas
|
||||
authFile: auth.json.17nas
|
||||
upstreamUserAgent: codex_cli_rs/0.137.0 (Ubuntu 22.04; x86_64) xterm-256color
|
||||
publicExposure:
|
||||
enabled: true
|
||||
proxyName: platform-infra-sub2api
|
||||
|
||||
@@ -68,7 +68,7 @@ CI/CD、GitOps、rollout、artifact 发布、PR 合并后的 runtime lane 滚动
|
||||
- `hwlab g14 git-mirror status|apply|sync|flush [--dry-run|--confirm]` 是 `devops-infra` git mirror/relay 的受控维护入口：`apply` 渲染并 server-side apply `devops-infra/git-mirror.yaml`，同时删除遗留 `git-mirror-hwlab-sync` CronJob；`sync` 创建一次性 manual Job，把 GitHub allowlist refs 拉入本地 mirror；`flush` 创建一次性 manual Job，把本地 `v0.2-gitops` 快进推回 GitHub。
|
||||
`status` 返回 read/write URL、last sync/write/flush、本地 ref、GitHub staging ref 和 pending flush 状态，并在 `cache.summary` 给出 `localV02`、`localGitops`、`githubGitops`、`pendingFlush`、`flushNeeded`、`githubInSync` 和下一条受控 `flushCommand`。confirmed `sync` 和 `flush` 默认创建 `.state/jobs/` 异步 job 并立刻返回可查询状态，只有现场同步调试才显式加 `--wait`；mirror 不设置 CronJob。
|
||||
如果 PipelineRun 的 `gitops-promote` 阶段报 git mirror 控制面漂移或 refs 不一致，先执行 `hwlab g14 git-mirror apply --confirm` 重新应用当前 `devops-infra/git-mirror.yaml` hook/ConfigMap，再执行 `hwlab g14 git-mirror sync --confirm --wait` 复核 refs；失败的同名 PipelineRun 只能通过 `hwlab g14 control-plane cleanup-runs --lane <lane> --pipeline-run <name> --confirm` 受控清理后重试，不要用原生 `kubectl delete` 或手工改 mirror hook。修复后仍必须用 `control-plane status --pipeline-run <name>` 和 `git-mirror status` 分别确认 runtime closeout 与 GitHub flush。
|
||||
- `platform-infra sub2api plan|apply|status|validate|codex-pool` 是 G14 `platform-infra` namespace 内 Sub2API 的受控入口。镜像版本由 `config/platform-infra/sub2api.yaml` 控制，Codex 上游池、统一 API key Secret、FRP 公网端口和 master `~/.codex` 消费端由 `config/platform-infra/sub2api-codex-pool.yaml` 控制；`codex-pool sync --confirm` 从 YAML 指定的 `~/.codex/config.toml*`/`auth.json*` 导入 YAML 声明的上游账号到同一 Sub2API group，统一消费 key 固定存放在 `platform-infra/sub2api-codex-pool-api-key.API_KEY`。`profiles.entries[].openaiResponsesWebSocketsV2Mode` 用于对接需要 Responses WebSocket v2 的 OpenAI-compatible 上游，新增或调整上游优先改 YAML 后 `codex-pool sync --confirm`，不改代码、不走 CI/CD。`codex-pool expose --confirm` 只新增明确 FRPS allow port 和 `platform-infra/sub2api-frpc`，不得扩大端口段；`codex-pool configure-local --confirm` 先把当前 `~/.codex/config.toml`、`auth.json` 备份成 `*.pre-sub2api`，再把默认 provider 指向 master 本机 FRP 入口。`validate`/`codex-pool validate` 是按需验收，不是连续可用性探针；完整 namespace 边界、路由和探针口径见 `docs/reference/platform-infra.md`。所有输出只允许打印 key preview/fingerprint、字节数和 Secret 路径，不打印完整 API key。
|
||||
- `platform-infra sub2api plan|apply|status|validate|codex-pool` 是 G14 `platform-infra` namespace 内 Sub2API 的受控入口。镜像版本由 `config/platform-infra/sub2api.yaml` 控制，Codex 上游池、统一 API key Secret、FRP 公网端口和 master `~/.codex` 消费端由 `config/platform-infra/sub2api-codex-pool.yaml` 控制；`codex-pool sync --confirm` 从 YAML 指定的 `~/.codex/config.toml*`/`auth.json*` 导入 YAML 声明的上游账号到同一 Sub2API group，统一消费 key 固定存放在 `platform-infra/sub2api-codex-pool-api-key.API_KEY`。`profiles.entries[].openaiResponsesWebSocketsV2Mode` 用于对接需要 Responses WebSocket v2 的 OpenAI-compatible 上游，`profiles.entries[].upstreamUserAgent` 用于少数需要 Codex CLI User-Agent 的上游；新增、调整或移除上游优先改 YAML 后 `codex-pool sync --confirm`，sync 会删除 YAML 中已移除且带 `unidesk_managed=true` 的 `unidesk-codex-*` account，不走 CI/CD。`codex-pool expose --confirm` 只新增明确 FRPS allow port 和 `platform-infra/sub2api-frpc`，不得扩大端口段；`codex-pool configure-local --confirm` 先把当前 `~/.codex/config.toml`、`auth.json` 备份成 `*.pre-sub2api`，再把默认 provider 指向 master 本机 FRP 入口。`validate`/`codex-pool validate` 是按需验收，不是连续可用性探针；完整 namespace 边界、路由和探针口径见 `docs/reference/platform-infra.md`。所有输出只允许打印 key preview/fingerprint、字节数和 Secret 路径，不打印完整 API key。
|
||||
- `hwlab g14 observability status|apply|query|targets|boundary|closeout [--lane v02] [--promql <expr>] [--expect-count N] [--expect-value V] [--dry-run|--confirm]` 是 G14 `devops-infra` 共享监控基础设施和 HWLAB v0.2 监控 closeout 的受控入口。`apply` 固定安装 Prometheus Operator `v0.91.0`、Prometheus `v3.12.0`、Prometheus 发现 RBAC、`devops-infra` 内 Prometheus 实例和 ClusterIP query Service，并给被允许发现的 workload namespace 打低风险 label；它不把 Prometheus、Grafana 或 Alertmanager 部署到 `hwlab-v02`，也不接管 HWLAB runtime Deployment/Service。`status` 只读汇总 CRD、operator Deployment、Prometheus CR/pod/service、`hwlab-v02` ServiceMonitor/PrometheusRule 和 bounded `up` 查询；`query` 只通过 Kubernetes service proxy 查询 Prometheus，支持 `--expect-count` / `--expect-value` 输出 `assertion`、bad values 和 missing/extra series；`targets` 汇总 ServiceMonitor/PrometheusRule、metrics sidecar readiness/restart、三层指标值和 `metrics.k8s.io` 当前 CPU/内存资源快照；`boundary` 验证 workload namespace 没有 Prometheus/Alertmanager，并对 `19666/19667` 公网 `/metrics` 做负向验证；`closeout` 聚合平台 ready、scrape reachable、sidecar serving、business health probe、resource snapshot、namespace boundary 和 public metrics exposure 语义结论。长期边界见 `docs/reference/g14-observability-infra.md`。
|
||||
- `hwlab g14 tools-image status|build --name ci-node-tools --tag <tag> [--dockerfile deploy/ci/hwlab-ci-node-tools.Dockerfile] [--dry-run|--confirm]` 是 G14 固定 HWLAB CI tools image 的受控 host build/push 入口；构建和 push 只发生在 G14 host 与本地 registry，不在 master server 构建，也不把 `apk add`/runtime install 塞进 Tekton PipelineRun。
|
||||
- `trans gh:/owner/repo ...` 把 GitHub issue/PR 映射成只读/受控写入的虚拟文本目录，适合日报、PR 正文和 issue 正文的小补丁维护：`trans gh:/pikasTech/HWLAB ls` 展示 `pr/` 与 `issue/`，`trans gh:/pikasTech/HWLAB/pr ls [--limit N] [--full]` 和 `trans gh:/pikasTech/HWLAB/issue ls [--limit N] [--full]` 展示条目状态、楼层数、正文长度和标题，`trans gh:/pikasTech/HWLAB/pr/507 ls` 展示单个 PR 的一楼正文文件，`trans gh:/pikasTech/HWLAB/505/1 cat|rg|patch-apply` 兼容旧式 issue/PR number route。`patch-apply` 使用 UniDesk 默认 apply-patch v2 的虚拟文件 executor，把正文一楼映射为 `body.md`，写回仍走 `bun scripts/cli.ts gh issue/pr update` 的 guard/concurrency 规则；`rm` 对正文一楼结构化拒绝，避免误删 issue/PR 正文。大正文读取必须展开 UniDesk gh dump 文件，否则 `cat/rg/patch-apply` 会误读为空，这是 `gh:` 虚拟文件接口的 P0 可见性契约。
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
- `pool.apiKeySecretName` and `pool.apiKeySecretKey` name the k3s Secret that stores the single consumer API key.
|
||||
- `profiles.entries` selects local Codex profile files from `~/.codex/` and maps them to Sub2API account names.
|
||||
- `profiles.entries[].openaiResponsesWebSocketsV2Mode` is the account-level Responses WebSocket v2 switch for OpenAI-compatible upstreams that require WebSocket transport. Allowed values are `off`, `ctx_pool`, and `passthrough`; omit the field unless that upstream needs it.
|
||||
- `profiles.entries[].upstreamUserAgent` is an optional account-level upstream request User-Agent override. Use it only for upstreams that require a Codex CLI compatible User-Agent; keep the value YAML-controlled and newline-free.
|
||||
- `publicExposure` controls the optional FRP bridge from master server to the G14 ClusterIP service.
|
||||
- `localCodex` controls how the master server's current `~/.codex` consumer files are backed up and rewritten.
|
||||
|
||||
@@ -37,7 +38,7 @@ The request path is:
|
||||
4. Sub2API validates the unified key and resolves its `group_id`.
|
||||
5. Accounts listed in `profiles.entries` are bound to the same group via `group_ids`, so Sub2API dispatches through that group using its own account selection semantics.
|
||||
|
||||
Adding an upstream should be a fast YAML operation: create the corresponding local `~/.codex/config.toml.<profile>` and `auth.json.<profile>` inputs, add one `profiles.entries` item, then run `platform-infra sub2api codex-pool plan|sync --confirm`. Do not add code or CI/CD for ordinary pool membership changes. Code changes are only appropriate when the YAML schema needs a new reusable capability such as account-level WebSocket mode.
|
||||
Adding an upstream should be a fast YAML operation: create the corresponding local `~/.codex/config.toml.<profile>` and `auth.json.<profile>` inputs, add one `profiles.entries` item, then run `platform-infra sub2api codex-pool plan|sync --confirm`. Removing an upstream is also a YAML operation: remove the `profiles.entries` item, then run `sync --confirm`; the sync path deletes only matching `unidesk-codex-*` accounts that are marked `unidesk_managed=true` and absent from YAML. Do not add code or CI/CD for ordinary pool membership changes. Code changes are only appropriate when the YAML schema needs a new reusable capability such as account-level WebSocket mode or per-account upstream User-Agent.
|
||||
|
||||
After `codex-pool configure-local --confirm`, the default upstream profile must not recursively import the just-created Sub2API consumer endpoint as an upstream account. Keep the default source profile pointed at `config.toml.<backupSuffix>` and `auth.json.<backupSuffix>`; fallback to the current default files is only for first bootstrap before backups exist.
|
||||
|
||||
|
||||
@@ -45,6 +45,7 @@ interface CodexProfile {
|
||||
apiKey: string | null;
|
||||
apiKeySource: "auth-json" | "env" | null;
|
||||
openaiResponsesWebSocketsV2Mode: OpenAIResponsesWebSocketsV2Mode | null;
|
||||
upstreamUserAgent: string | null;
|
||||
authOpenAIKeyShape: string;
|
||||
ok: boolean;
|
||||
error: string | null;
|
||||
@@ -71,6 +72,7 @@ interface CodexPoolProfileConfig {
|
||||
fallbackConfigFile: string | null;
|
||||
fallbackAuthFile: string | null;
|
||||
openaiResponsesWebSocketsV2Mode: OpenAIResponsesWebSocketsV2Mode | null;
|
||||
upstreamUserAgent: string | null;
|
||||
}
|
||||
|
||||
interface CodexPoolPublicExposureConfig {
|
||||
@@ -192,7 +194,7 @@ function codexPoolPlan(): Record<string, unknown> {
|
||||
publicExposure: pool.publicExposure.enabled
|
||||
? `Master server consumers use ${pool.publicExposure.masterBaseUrl}; FRP proxy ${pool.publicExposure.proxyName} maps public ${pool.publicExposure.publicBaseUrl} to ${pool.publicExposure.localIP}:${pool.publicExposure.localPort}.`
|
||||
: "Public FRP exposure is disabled by YAML.",
|
||||
idempotency: "sync reuses the group, account names, and k3s Secret when they already exist; credentials are updated from the current local Codex files.",
|
||||
idempotency: "sync reuses the group, account names, and k3s Secret when they already exist; credentials are updated from the current local Codex files; UniDesk-managed accounts removed from YAML are deleted from Sub2API.",
|
||||
configPolicy: "UniDesk-owned durable configuration remains YAML-first; local ~/.codex files and runtime Secrets are not committed.",
|
||||
},
|
||||
next: ok
|
||||
@@ -237,6 +239,7 @@ async function codexPoolSync(config: UniDeskConfig, options: SyncOptions): Promi
|
||||
apiKeySource: profile.apiKeySource,
|
||||
apiKeyFingerprint: fingerprint(profile.apiKey ?? ""),
|
||||
openaiResponsesWebSocketsV2Mode: profile.openaiResponsesWebSocketsV2Mode,
|
||||
upstreamUserAgent: profile.upstreamUserAgent,
|
||||
})),
|
||||
};
|
||||
const result = await capture(config, g14K3sRoute, ["script"], syncScript(payload, pool));
|
||||
@@ -418,6 +421,7 @@ function collectCodexProfiles(): CodexProfile[] {
|
||||
apiKey: null,
|
||||
apiKeySource: null,
|
||||
openaiResponsesWebSocketsV2Mode: entry.openaiResponsesWebSocketsV2Mode,
|
||||
upstreamUserAgent: entry.upstreamUserAgent,
|
||||
authOpenAIKeyShape: existsSync(authPath) ? "unknown" : "missing",
|
||||
ok: false,
|
||||
error: null,
|
||||
@@ -481,6 +485,7 @@ function discoverCodexProfileConfigs(codexDir: string): CodexPoolProfileConfig[]
|
||||
fallbackConfigFile: null,
|
||||
fallbackAuthFile: null,
|
||||
openaiResponsesWebSocketsV2Mode: null,
|
||||
upstreamUserAgent: null,
|
||||
};
|
||||
});
|
||||
}
|
||||
@@ -576,6 +581,7 @@ function readProfileConfig(value: unknown, defaults: CodexPoolProfileConfig[]):
|
||||
if (fallbackConfigFile !== null) validateCodexFileName(fallbackConfigFile, `profiles.entries[${index}].fallbackConfigFile`);
|
||||
if (fallbackAuthFile !== null) validateCodexFileName(fallbackAuthFile, `profiles.entries[${index}].fallbackAuthFile`);
|
||||
const openaiResponsesWebSocketsV2Mode = readOpenAIResponsesWebSocketsV2Mode(entry.openaiResponsesWebSocketsV2Mode, `profiles.entries[${index}].openaiResponsesWebSocketsV2Mode`);
|
||||
const upstreamUserAgent = readUpstreamUserAgent(entry.upstreamUserAgent, `profiles.entries[${index}].upstreamUserAgent`);
|
||||
return {
|
||||
profile,
|
||||
accountName,
|
||||
@@ -584,6 +590,7 @@ function readProfileConfig(value: unknown, defaults: CodexPoolProfileConfig[]):
|
||||
fallbackConfigFile,
|
||||
fallbackAuthFile,
|
||||
openaiResponsesWebSocketsV2Mode,
|
||||
upstreamUserAgent,
|
||||
};
|
||||
});
|
||||
}
|
||||
@@ -596,6 +603,15 @@ function readOpenAIResponsesWebSocketsV2Mode(value: unknown, key: string): OpenA
|
||||
throw new Error(`${codexPoolConfigPath}.${key} must be one of off, ctx_pool, passthrough`);
|
||||
}
|
||||
|
||||
function readUpstreamUserAgent(value: unknown, key: string): string | null {
|
||||
if (value === undefined || value === null) return null;
|
||||
const text = stringValue(value);
|
||||
if (text === null) throw new Error(`${codexPoolConfigPath}.${key} must be a string`);
|
||||
if (text.length > 512) throw new Error(`${codexPoolConfigPath}.${key} must be at most 512 characters`);
|
||||
if (/[\r\n]/u.test(text)) throw new Error(`${codexPoolConfigPath}.${key} must not contain newlines`);
|
||||
return text;
|
||||
}
|
||||
|
||||
function readPublicExposureConfig(value: unknown, defaults: CodexPoolPublicExposureConfig): CodexPoolPublicExposureConfig {
|
||||
if (!isRecord(value)) return defaults;
|
||||
const masterFrpsValue = isRecord(value.masterFrps) ? value.masterFrps : {};
|
||||
@@ -696,6 +712,7 @@ function redactProfile(profile: CodexProfile): Record<string, unknown> {
|
||||
envKey: profile.envKey,
|
||||
apiKeySource: profile.apiKeySource,
|
||||
openaiResponsesWebSocketsV2Mode: profile.openaiResponsesWebSocketsV2Mode,
|
||||
upstreamUserAgent: profile.upstreamUserAgent,
|
||||
apiKeyPresent: profile.apiKey !== null && profile.apiKey.length > 0,
|
||||
apiKeyBytes: profile.apiKey === null ? 0 : Buffer.byteLength(profile.apiKey, "utf8"),
|
||||
apiKeyFingerprint: profile.apiKey === null ? null : fingerprint(profile.apiKey),
|
||||
@@ -1450,15 +1467,19 @@ def account_payload(profile, group_id):
|
||||
if ws_mode:
|
||||
extra["openai_apikey_responses_websockets_v2_mode"] = ws_mode
|
||||
extra["openai_apikey_responses_websockets_v2_enabled"] = ws_mode != "off"
|
||||
credentials = {
|
||||
"api_key": profile["apiKey"],
|
||||
"base_url": profile["baseUrl"],
|
||||
}
|
||||
upstream_user_agent = profile.get("upstreamUserAgent")
|
||||
if upstream_user_agent:
|
||||
credentials["user_agent"] = upstream_user_agent
|
||||
return {
|
||||
"name": profile["accountName"],
|
||||
"notes": f"UniDesk-managed Codex profile {profile['profile']} from {profile['configFile']} and {profile['authFile']}; secret source={profile['apiKeySource']}; fingerprint={profile['apiKeyFingerprint']}.",
|
||||
"platform": "openai",
|
||||
"type": "apikey",
|
||||
"credentials": {
|
||||
"api_key": profile["apiKey"],
|
||||
"base_url": profile["baseUrl"],
|
||||
},
|
||||
"credentials": credentials,
|
||||
"extra": extra,
|
||||
"concurrency": 1,
|
||||
"priority": 1,
|
||||
@@ -1469,7 +1490,9 @@ def account_payload(profile, group_id):
|
||||
}
|
||||
|
||||
def ensure_accounts(token, profiles, group_id):
|
||||
existing = {item.get("name"): item for item in list_accounts(token)}
|
||||
existing_accounts = list_accounts(token)
|
||||
existing = {item.get("name"): item for item in existing_accounts}
|
||||
desired_names = {profile["accountName"] for profile in profiles}
|
||||
results = []
|
||||
for profile in profiles:
|
||||
payload = account_payload(profile, group_id)
|
||||
@@ -1493,6 +1516,33 @@ def ensure_accounts(token, profiles, group_id):
|
||||
"apiKeySource": profile["apiKeySource"],
|
||||
"apiKeyFingerprint": profile["apiKeyFingerprint"],
|
||||
"openaiResponsesWebSocketsV2Mode": profile.get("openaiResponsesWebSocketsV2Mode"),
|
||||
"upstreamUserAgentConfigured": bool(profile.get("upstreamUserAgent")),
|
||||
"valuesPrinted": False,
|
||||
})
|
||||
prune_results = prune_removed_accounts(token, existing_accounts, desired_names)
|
||||
return results, prune_results
|
||||
|
||||
def prune_removed_accounts(token, existing_accounts, desired_names):
|
||||
results = []
|
||||
for account in existing_accounts:
|
||||
name = account.get("name")
|
||||
account_id = account.get("id")
|
||||
extra = account.get("extra") if isinstance(account.get("extra"), dict) else {}
|
||||
if not isinstance(name, str) or not name.startswith("unidesk-codex-"):
|
||||
continue
|
||||
if extra.get("unidesk_managed") is not True:
|
||||
continue
|
||||
if name in desired_names:
|
||||
continue
|
||||
if account_id is None:
|
||||
raise RuntimeError(f"removed account {name} has no id")
|
||||
ensure_success(curl_api("DELETE", f"/api/v1/admin/accounts/{account_id}", bearer=token), f"delete removed account {name}")
|
||||
results.append({
|
||||
"accountName": name,
|
||||
"accountId": account_id,
|
||||
"profile": extra.get("unidesk_codex_profile"),
|
||||
"action": "deleted",
|
||||
"reason": "removed-from-yaml",
|
||||
"valuesPrinted": False,
|
||||
})
|
||||
return results
|
||||
@@ -1634,7 +1684,7 @@ def run_sync():
|
||||
group_id = group.get("id") if isinstance(group, dict) else None
|
||||
if group_id is None:
|
||||
raise RuntimeError("pool group id missing after ensure")
|
||||
account_results = ensure_accounts(token, profiles, group_id)
|
||||
account_results, pruned_account_results = ensure_accounts(token, profiles, group_id)
|
||||
api_key, secret_action, secret_apply_stdout = ensure_api_key_secret(group_id)
|
||||
api_key_result = ensure_sub2api_api_key(token, api_key, group_id)
|
||||
owner_balance = ensure_pool_owner_balance(token, api_key_result["userId"])
|
||||
@@ -1651,7 +1701,9 @@ def run_sync():
|
||||
"desired": len(profiles),
|
||||
"created": sum(1 for item in account_results if item["action"] == "created"),
|
||||
"updated": sum(1 for item in account_results if item["action"] == "updated"),
|
||||
"pruned": len(pruned_account_results),
|
||||
"items": account_results,
|
||||
"prunedItems": pruned_account_results,
|
||||
"valuesPrinted": False,
|
||||
},
|
||||
"apiKey": {
|
||||
|
||||
Reference in New Issue
Block a user