fix(cicd): expose selfmedia GitHub hook authority errors
This commit is contained in:
@@ -0,0 +1,69 @@
|
||||
# R1 任务报告
|
||||
|
||||
## 任务回链
|
||||
|
||||
- MDTODO:`docs/MDTODO/selfmedia-production-delivery.md` R1。
|
||||
- Target:`NC01`。
|
||||
- targetWorkspace:`/root/.worktrees/unidesk/selfmedia-production-delivery`。
|
||||
- repository:`pikasTech/unidesk`。
|
||||
- ref:`feat/selfmedia-production-delivery`。
|
||||
- 状态:保持 `in_progress`,尚未取得修复合并后的真实自动链证据。
|
||||
|
||||
## 最小根因
|
||||
|
||||
- `platform-infra gitea mirror webhook status` 的 GitHub hooks 观察器在未校验 HTTP 结果和 JSON 形状时直接迭代响应。
|
||||
- 私有仓库 `pikainc/selfmedia` 对 YAML 声明的 `gh_token.txt` authority 返回 HTTP `404`;错误对象被按 hook list 迭代,最终产生 `AttributeError: 'str' object has no attribute 'get'`,并把权限故障伪装成空 hook。
|
||||
- owning YAML 已包含 `selfmedia-nc01`、GitHub→Gitea hook topology、Gitea repository、PaC consumer、GitOps/Argo、双 PVC、Secret、NetworkPolicy 和公网 `152.53.229.148:4317` 契约;不需要新增 Secret 或手工 apply。
|
||||
- 自动链当前真正阻塞 authority:`config/platform-infra/gitea.yaml` 声明的 GitHub bridge credential `gh_token.txt` 无法读取或管理私有仓库 `pikainc/selfmedia` 的 hooks,也无法读取仓库 head。Git SSH push 可用不能替代 GitHub API hook authority。
|
||||
|
||||
## 源码修复
|
||||
|
||||
- `scripts/src/platform_infra_gitea_status_evaluator.py`
|
||||
- 新增 GitHub hooks HTTP/JSON/list/item 的 fail-closed 解析。
|
||||
- 错误仅输出 `github-hooks-http-<status>`、`github-hooks-response-not-list`、`github-hooks-item-not-object` 等有界 code,不回显 GitHub 错误正文或凭据。
|
||||
- 统一支持 hook `config` 为对象 URL 或字符串 URL 的公开 API 形状。
|
||||
- `scripts/src/platform-infra-gitea-remote.sh`
|
||||
- embedded observer 复用 evaluator,不再直接迭代未校验响应。
|
||||
- HTTP 失败、错误对象和畸形 list 不再伪装成空 hook。
|
||||
- `scripts/src/platform-infra-gitea-status-evaluator.test.ts`
|
||||
- 覆盖 HTTP 错误对象、非 list、含非对象元素、对象 config、字符串 config 和畸形 hook。
|
||||
|
||||
## Target 原入口验证
|
||||
|
||||
- `sh -n scripts/src/platform-infra-gitea-remote.sh`:通过。
|
||||
- `python3 -m py_compile scripts/src/platform_infra_gitea_status_evaluator.py`:通过。
|
||||
- `git diff --check`:通过。
|
||||
- `bun test scripts/src/platform-infra-gitea-status-evaluator.test.ts`:`14 pass / 0 fail / 86 expect()`。
|
||||
- `platform-infra gitea mirror webhook status --target NC01 --repo selfmedia-nc01 --full`:
|
||||
- bridge `ready=true`、`1/1`;
|
||||
- durable inbox `ready=true`,无 pending/failed;
|
||||
- selfmedia `hookReady=false`、`authorityMatches=false`;
|
||||
- topology drift 为 `desired-url-missing`;
|
||||
- 有界错误为 `github-hooks-http-404`;
|
||||
- 不再出现 Python traceback,不打印 token 或 GitHub 错误正文。
|
||||
- `platform-infra pipelines-as-code status --target NC01 --consumer selfmedia-nc01`:
|
||||
- consumer bootstrap ready,runner automount disabled,Argo repository Secret keys ready;
|
||||
- Repository hook `0`,Latest PipelineRun 为空;
|
||||
- diagnosis 仍为 `pac-source-unknown`、registry missing、Argo sync Unknown。
|
||||
- 公网补充证据:`152.53.229.148:4317` 健康但 WebTerm 显示“后端尚未声明内置终端能力”,`/api/v1/system` 无 terminal configRef;宿主 PID 自 01:44 运行。该入口是旧宿主进程证据,不是 K8s 自动发布证明,任务未执行重启或 cutover。
|
||||
- Secret 补充证据:`secrets status --scope selfmedia --target selfmedia-nc01` 显示 runtime TOKEN 与 codex `auth.json` / `config.toml` 均存在且 keys exact、`valuesPrinted=false`;任务未新增或同步 Secret。
|
||||
|
||||
## Primary Workspace 边界
|
||||
|
||||
- primary workspace 仅用于读取 `$dad-dev`、`$unidesk-trans`、`$unidesk-cicd`、`$unidesk-ymalops`、`$cli-spec`、`$git-spec`、`$docs-spec` 与 `$unidesk-gh`。
|
||||
- 所有源码读取、编辑、测试、Git 和运行面只读观察均通过 `trans NC01:/root/.worktrees/unidesk/selfmedia-production-delivery ...` 完成。
|
||||
- 未使用 runner 本地结果替代 Target 原入口证据,未递归派单。
|
||||
|
||||
## 保留与删除的特例
|
||||
|
||||
- `keep-domain-special`:selfmedia 双 PVC、零 runtime RBAC、`automountServiceAccountToken=false`、非特权 runtime、YAML-first Secret、NetworkPolicy、公网 IP 与 GitOps branch 契约保持不变。
|
||||
- `remove-code-default`:无新增代码默认值;修复只校验实际 GitHub API 形状。
|
||||
- `legacy-retire`:不恢复 branch-follower、人工 mirror sync、人工 PipelineRun、Argo refresh/sync、rollout 或 cutover。
|
||||
|
||||
## 合并后自动验收
|
||||
|
||||
1. 由凭据 authority 管理者为 owning YAML 的 `gh_token.txt` 授予 `pikainc/selfmedia` 私有仓库读取与 webhook 管理权限;不得复制 root SSH 私钥或改变仓库可见性。
|
||||
2. 合并本修复 PR 后,仅观察 `platform-infra-gitea-nc01` 自动 PipelineRun 与 Argo 收敛;不得 apply、sync、refresh 或手工 reconciler。
|
||||
3. 待 hook reconciler 自动运行后,只读执行 `platform-infra gitea mirror webhook status --target NC01 --repo selfmedia-nc01`,应看到 hook ready、真实 delivery accepted→committed、GitHub head 与 Gitea branch/snapshot 对齐。
|
||||
4. 合并一条 `pikainc/selfmedia/master` 正常 PR,观察自动产生唯一外层 selfmedia PipelineRun、镜像 digest、GitOps release state、Argo revision、runtime commitId 与 `/healthz` ready。
|
||||
5. 用 `platform-infra pipelines-as-code status|history --target NC01 --consumer selfmedia-nc01` 验证阶段耗时、失败归因和下一步;未取得上述真实自动事件前 R1 保持 `in_progress`。
|
||||
@@ -757,7 +757,7 @@ NODE
|
||||
python3 - "$tmp/repos.json" "$bridge_ready" "$service_exists" "$mirror_status_rc" "$bridge_logs_rc" "$inbox_status_rc" "$tmp/mirror-status.json" "$tmp/mirror-status.err" "$tmp/bridge.log" "$tmp/bridge.err" "$tmp/inbox-status.json" "$tmp/inbox-status.err" "$tmp/hook-topology.json" <<'PY'
|
||||
import hashlib, json, os, sys, urllib.error, urllib.parse, urllib.request
|
||||
sys.path.insert(0, os.environ["tmp"])
|
||||
from platform_infra_gitea_status_evaluator import evaluate_repository, select_committed_head_record, select_target_delivery
|
||||
from platform_infra_gitea_status_evaluator import evaluate_repository, github_hook_url, parse_github_hooks_response, select_committed_head_record, select_target_delivery
|
||||
repos = json.load(open(sys.argv[1], encoding="utf-8"))
|
||||
bridge_ready = sys.argv[2]
|
||||
service_exists = bool(sys.argv[3])
|
||||
@@ -783,12 +783,15 @@ topology_repositories = hook_topology.get("repositories") if isinstance(hook_top
|
||||
def url_hash(value):
|
||||
return "sha256:" + hashlib.sha256(value.encode("utf-8")).hexdigest()[:16]
|
||||
def hook_url(item):
|
||||
config = item.get("config") if isinstance(item.get("config"), dict) else {}
|
||||
return config.get("url") if isinstance(config.get("url"), str) else ""
|
||||
return github_hook_url(item)
|
||||
def is_managed_url(value):
|
||||
return value == managed_url_prefix or value.startswith(managed_url_prefix.rstrip("/") + "/")
|
||||
def hook_config_matches(item):
|
||||
config = item.get("config") if isinstance(item.get("config"), dict) else {}
|
||||
config = item.get("config")
|
||||
if isinstance(config, str):
|
||||
return item.get("name") == "web" and item.get("active") is True and set(item.get("events") or []) == set(desired_events)
|
||||
if not isinstance(config, dict):
|
||||
return False
|
||||
return item.get("name") == "web" and item.get("active") is True and set(item.get("events") or []) == set(desired_events) and config.get("content_type") == "json" and str(config.get("insecure_ssl", "0")) == "0"
|
||||
def topology_observation(repository, hooks):
|
||||
spec = next((item for item in topology_repositories if isinstance(item, dict) and item.get("repository") == repository), {})
|
||||
@@ -989,11 +992,8 @@ for repo in repos:
|
||||
repository = repo["upstream"]["repository"]
|
||||
branch = repo["upstream"]["branch"]
|
||||
result = request(f"/repos/{repository}/hooks")
|
||||
hooks = []
|
||||
try:
|
||||
hooks = json.loads(result.get("body") or "[]")
|
||||
except Exception:
|
||||
hooks = []
|
||||
hooks_result = parse_github_hooks_response(result.get("ok"), result.get("status"), result.get("body"))
|
||||
hooks = hooks_result["hooks"]
|
||||
matches = [item for item in hooks if hook_url(item) == url]
|
||||
topology = topology_observation(repository, hooks)
|
||||
head = github_head(repository, branch)
|
||||
@@ -1044,7 +1044,7 @@ for repo in repos:
|
||||
"latest": latest_delivery,
|
||||
"latestPush": latest_push_delivery,
|
||||
"selectedPush": selected_push_delivery,
|
||||
"error": result.get("body", "")[:500] if not result.get("ok") else (delivery_errors[0] if delivery_errors else topology_reason),
|
||||
"error": hooks_result["error"] or (delivery_errors[0] if delivery_errors else topology_reason),
|
||||
}
|
||||
rows.append(evaluate_repository(repo, hook, head, mirror, bridge_events, inbox_records, inbox_status.get("journal")))
|
||||
bridge = {
|
||||
|
||||
@@ -5,6 +5,33 @@ import { resolve } from "node:path";
|
||||
const evaluator = resolve(import.meta.dir, "platform_infra_gitea_status_evaluator.py");
|
||||
|
||||
describe("Gitea source authority status evaluator", () => {
|
||||
test("fails closed for GitHub hooks error objects and malformed lists", () => {
|
||||
expect(evaluate({
|
||||
action: "parse-github-hooks-response",
|
||||
ok: false,
|
||||
status: 403,
|
||||
body: JSON.stringify({ message: "forbidden" }),
|
||||
})).toEqual({ ok: false, status: 403, hooks: [], error: "github-hooks-http-403" });
|
||||
expect(evaluate({
|
||||
action: "parse-github-hooks-response",
|
||||
ok: true,
|
||||
status: 200,
|
||||
body: JSON.stringify({ message: "not a list" }),
|
||||
}).error).toBe("github-hooks-response-not-list");
|
||||
expect(evaluate({
|
||||
action: "parse-github-hooks-response",
|
||||
ok: true,
|
||||
status: 200,
|
||||
body: JSON.stringify([{ id: 1 }, "malformed"]),
|
||||
}).error).toBe("github-hooks-item-not-object");
|
||||
});
|
||||
|
||||
test("reads GitHub hook URLs from object and string config shapes", () => {
|
||||
expect(evaluate({ action: "github-hook-url", hook: { config: { url: "https://hooks.example/object" } } })).toBe("https://hooks.example/object");
|
||||
expect(evaluate({ action: "github-hook-url", hook: { config: "https://hooks.example/string" } })).toBe("https://hooks.example/string");
|
||||
expect(evaluate({ action: "github-hook-url", hook: "malformed" })).toBe("");
|
||||
});
|
||||
|
||||
test("parses branch and exact snapshot from ls-remote output larger than the display tail budget", () => {
|
||||
const head = "2".repeat(40);
|
||||
const prefix = "refs/unidesk/snapshots/gitea-actions/unidesk-master-nc01";
|
||||
|
||||
@@ -13,6 +13,31 @@ def parse_ls_remote_lines(lines):
|
||||
return refs
|
||||
|
||||
|
||||
def parse_github_hooks_response(ok, status, body):
|
||||
if ok is not True:
|
||||
return {"ok": False, "status": status, "hooks": [], "error": f"github-hooks-http-{status or 'unknown'}"}
|
||||
try:
|
||||
payload = json.loads(body or "[]")
|
||||
except (TypeError, json.JSONDecodeError):
|
||||
return {"ok": False, "status": status, "hooks": [], "error": "github-hooks-response-invalid-json"}
|
||||
if not isinstance(payload, list):
|
||||
return {"ok": False, "status": status, "hooks": [], "error": "github-hooks-response-not-list"}
|
||||
if any(not isinstance(item, dict) for item in payload):
|
||||
return {"ok": False, "status": status, "hooks": [], "error": "github-hooks-item-not-object"}
|
||||
return {"ok": True, "status": status, "hooks": payload, "error": None}
|
||||
|
||||
|
||||
def github_hook_url(item):
|
||||
if not isinstance(item, dict):
|
||||
return ""
|
||||
config = item.get("config")
|
||||
if isinstance(config, str):
|
||||
return config
|
||||
if isinstance(config, dict) and isinstance(config.get("url"), str):
|
||||
return config["url"]
|
||||
return ""
|
||||
|
||||
|
||||
def select_committed_head_record(repo_key, github_head, records):
|
||||
candidates = []
|
||||
for record in records:
|
||||
@@ -255,6 +280,10 @@ def main():
|
||||
action = payload.get("action")
|
||||
if action == "parse-ls-remote":
|
||||
result = parse_ls_remote_lines(str(payload.get("output") or "").splitlines())
|
||||
elif action == "parse-github-hooks-response":
|
||||
result = parse_github_hooks_response(payload.get("ok"), payload.get("status"), payload.get("body"))
|
||||
elif action == "github-hook-url":
|
||||
result = github_hook_url(payload.get("hook"))
|
||||
elif action == "select-committed-head-record":
|
||||
result = select_committed_head_record(payload.get("repoKey"), payload.get("githubHead"), payload.get("records", []))
|
||||
elif action == "select-target-delivery":
|
||||
|
||||
Reference in New Issue
Block a user