diff --git a/.agents/skills/unidesk-cicd/SKILL.md b/.agents/skills/unidesk-cicd/SKILL.md index eaca8f15..bdd723cf 100644 --- a/.agents/skills/unidesk-cicd/SKILL.md +++ b/.agents/skills/unidesk-cicd/SKILL.md @@ -36,7 +36,7 @@ bun scripts/cli.ts cicd branch-follower debug-step --follower web-probe-sentinel - PR monitor 与自动合并: [references/pr-monitor.md](references/pr-monitor.md)。 - Tekton/Argo、node-scoped runtime lane、D601 infra bootstrap: [references/control-plane.md](references/control-plane.md)。 - HWLAB/AgentRun git-mirror source authority 与 flush: [references/git-mirror.md](references/git-mirror.md)。 -- JD01 Gitea mirror + Pipelines-as-Code 正式架构、三 consumer 覆盖矩阵和旧入口处置: [references/gitea-pac.md](references/gitea-pac.md)。 +- JD01/NC01 Gitea mirror + Pipelines-as-Code 正式架构、三 consumer 覆盖矩阵、target 级 webhook path 和旧入口处置: [references/gitea-pac.md](references/gitea-pac.md)。 - AgentRun/HWLAB env reuse 机制、证据解释和 PaC status/history 观察口径: [references/env-reuse.md](references/env-reuse.md)。 - Secret、observability、platform-infra、CI tools image、PipelineRun 清理和 rollout 补记: [references/platform-ops.md](references/platform-ops.md)。 - AgentRun YAML-only lane、v0.1 兼容入口和 AgentRun git-mirror: [references/agentrun.md](references/agentrun.md)。 diff --git a/.agents/skills/unidesk-cicd/references/gitea-pac.md b/.agents/skills/unidesk-cicd/references/gitea-pac.md index dfbb01a4..9fbf9e4f 100644 --- a/.agents/skills/unidesk-cicd/references/gitea-pac.md +++ b/.agents/skills/unidesk-cicd/references/gitea-pac.md @@ -14,6 +14,7 @@ GitHub remains the upstream write authority. A delivery should be triggered by m - PaC controller, Repository CRs, consumer parameters, Tekton pipeline names and Argo applications: `config/platform-infra/pipelines-as-code.yaml`. - Historical snapshot prefixes containing `gitea-actions` are retained only for existing refs. They do not mean Gitea Actions is the active trigger architecture. - Node-level status for these consumers is `bun scripts/cli.ts cicd status --node `. Consumer drill-down is `platform-infra pipelines-as-code status|history|closeout --target --consumer `. +- Multi-node PaC consumers that share a GitHub upstream must not share a webhook bridge URL. Each target node declares its own GitHub webhook public path and FRP remote port in `config/platform-infra/gitea.yaml#targets[].webhookSync`, then `platform-infra gitea apply --target --confirm` renders the node-local bridge, FRPC proxy and PK01 Caddy path. ## Coverage Matrix diff --git a/config/platform-infra/gitea.yaml b/config/platform-infra/gitea.yaml index 0bac14ba..96bb1ff8 100644 --- a/config/platform-infra/gitea.yaml +++ b/config/platform-infra/gitea.yaml @@ -252,6 +252,11 @@ targets: storageClassName: local-path publicExposure: enabled: false + webhookSync: + publicPath: /_unidesk/github-to-gitea/nc01 + frpc: + proxyName: platform-infra-gitea-nc01-github-webhook + remotePort: 22082 app: name: gitea diff --git a/docs/reference/agentrun.md b/docs/reference/agentrun.md index 2958223f..9dbf84ec 100644 --- a/docs/reference/agentrun.md +++ b/docs/reference/agentrun.md @@ -98,7 +98,7 @@ bun scripts/cli.ts platform-infra pipelines-as-code status --target JD01 `status` 只读观察 YAML 选中 lane 的 source authority、对应 PipelineRun、GitOps latest、Argo Application、runtime workload、manager source commit 和 git mirror/Gitea 摘要，并报告 Argo revision 是否对齐该 lane 的 GitOps latest。未迁移的 legacy `v0.2` lane source authority 只来自 k8s git-mirror snapshot：受控 sync 先为 branch tip 创建 `refs/unidesk/snapshots/agentrun-yaml-lane//`，`status` / `trigger-current` / build 只读取该 snapshot 和 `sourceStageRef`，不得把 host source workspace、本地 fetch/pull、可变 branch ref 或 Pipeline 直连 GitHub 当 authoritative source。默认输出是 compact commander 视图：`target` 只保留 node/lane/source/runtime/CI/GitOps/git-mirror/database 摘要，关键结论在 `summary` 和 `alignment`，成功 probe 的 stdout/stderr tail、完整 YAML target、原始 `source`、`runtime`、`gitMirror` payload 默认省略；需要完整展开时使用返回的 `disclosure.fullCommand` 或显式加 `--full`，需要原始调试视图时加 `--raw`。`status` 额外支持 `--pipeline-run ` 与 `--source-commit ` 定点查询；`--pipeline-run` 会读取 PipelineRun `revision` 参数作为 pinned source commit，并在 `alignment.branchDrift` / `summary.branchDrift` 中同时披露当前 snapshot tip、目标 source commit、PipelineRun source commit、是否已被当前 snapshot supersede 以及 `triggerLatest` 下一步。`status` 会向 stderr 输出 `agentrun.control-plane.status.progress` 阶段事件，覆盖 `source`、`runtime` 和 `git-mirror`，避免长时间聚合时无可见进展。legacy `trigger-current` 会先执行 k8s git-mirror sync 并以 snapshot commit 创建 commit-pinned PipelineRun；同名 PipelineRun 正在运行或已经成功时必须拒绝重复触发，只允许在失败态或不存在时创建。该命令只提交 CI/CD 工作，不等待完整 PipelineRun 或 rollout 完成，后续用 `job status` 和 `status --pipeline-run ` 轮询。`refresh` 只对 YAML 声明的 Argo Application 执行 hard refresh，用于 GitOps promotion 已完成但 Argo 仍停留旧 revision 时的受控同步入口；它不直接 patch runtime workload。 -JD01/NC01 `agentrun--v02` PaC 迁移 lane 的正式触发链路是：GitHub PR 合并到 `pikasTech/agentrun@v0.2` -> GitHub webhook bridge 同步到 Gitea controlled mirror 和 immutable snapshot refs -> Gitea repository push webhook 触发 Pipelines-as-Code -> Tekton PipelineRun -> GitOps/Argo -> k8s runtime。该 lane 不再使用 branch-follower、Gitea Actions、act_runner 或自维护 `trigger-current` 作为正式 CI 触发器；对应 source authority、repo visibility、public exposure 和 mirror credentials 属于 `config/platform-infra/gitea.yaml`，PaC controller/Repository/webhook/Tekton 参数属于 `config/platform-infra/pipelines-as-code.yaml`。Gitea 的公开 Web UI 是 `https://gitea.pikapython.com`，但 CI、Argo 和 runtime 内部读取必须使用 `gitea-http.devops-infra.svc.cluster.local:3000` 的 ClusterIP URL，避免公网回环和 legacy git-mirror commit 缺失。 +JD01/NC01 `agentrun--v02` PaC 迁移 lane 的正式触发链路是：GitHub PR 合并到 `pikasTech/agentrun@v0.2` -> GitHub webhook bridge 同步到 Gitea controlled mirror 和 immutable snapshot refs -> Gitea repository push webhook 触发 Pipelines-as-Code -> Tekton PipelineRun -> GitOps/Argo -> k8s runtime。该 lane 不再使用 branch-follower、Gitea Actions、act_runner 或自维护 `trigger-current` 作为正式 CI 触发器；对应 source authority、repo visibility、public exposure 和 mirror credentials 属于 `config/platform-infra/gitea.yaml`，PaC controller/Repository/webhook/Tekton 参数属于 `config/platform-infra/pipelines-as-code.yaml`。多节点共享同一 GitHub upstream 时，每个目标 node 的 webhook bridge 必须使用 target 级 YAML path 和 FRP remote port，不得复用另一节点的 GitHub hook URL 或 bridge。Gitea 的公开 Web UI 是 `https://gitea.pikapython.com`，但 CI、Argo 和 runtime 内部读取必须使用 `gitea-http.devops-infra.svc.cluster.local:3000` 的 ClusterIP URL，避免公网回环和 legacy git-mirror commit 缺失。 AgentRun PaC lane 的 closeout 以 `bun scripts/cli.ts cicd status --node ` 或 `bun scripts/cli.ts platform-infra pipelines-as-code status --target --consumer agentrun--v02` 为首选状态入口。默认输出必须能直接看到 webhook 是否存在、最新 PipelineRun/TaskRun duration、image status、env identity、digest、GitOps commit、Argo revision/health、manager Deployment readiness 和 runtime source/env annotation。env reuse 的通过证据是 `IMAGE_STATUS=reused`、同一 env identity 和稳定 digest；首次 cache miss 可以作为冷启动事实记录，但不能替代后续 env-reuse 秒级收口。若 GitHub PR 已合并但没有新 PipelineRun，应先排查 `platform-infra gitea mirror webhook status|test` 与 `platform-infra pipelines-as-code history`，不得改用 `trigger-current`、直接创建 PipelineRun、手工 push Gitea 或裸 `kubectl` 作为正式补触发。 diff --git a/docs/reference/platform-infra.md b/docs/reference/platform-infra.md index 6c468360..60328272 100644 --- a/docs/reference/platform-infra.md +++ b/docs/reference/platform-infra.md @@ -15,7 +15,7 @@ - Gitea mirror and Pipelines-as-Code are platform-infra CI source/trigger services operated by UniDesk. Their durable configuration lives in `config/platform-infra/gitea.yaml` and `config/platform-infra/pipelines-as-code.yaml`; do not hide repo URLs, mirror repo names, webhook settings, public exposure, FRP/Caddy ports, token sourceRefs or PaC Repository params in helper constants. - The canonical Gitea entrypoints are `bun scripts/cli.ts platform-infra gitea plan|apply|status|validate|mirror --target `, `bun scripts/cli.ts platform-infra gitea mirror plan|bootstrap|sync|status --target `, and `bun scripts/cli.ts platform-infra gitea mirror webhook apply|status|test --target `. Mirror bootstrap/sync must repair declared repo/org visibility such as `publicRead: true`; create-time defaults alone are not enough for long-lived repos. - Gitea mirror webhook sync is GitHub -> Gitea only. GitHub remains the upstream write authority; the webhook bridge may update the YAML-declared Gitea branch and immutable snapshot refs, but it must not write Gitea changes back to GitHub or add a second polling path. -- The Gitea public Web UI and GitHub webhook endpoint may share the YAML-declared HTTPS hostname only through explicit Caddy path routing. The webhook path must route to its own YAML-declared FRP proxy/remote port, while ordinary Web UI traffic continues to route to the Gitea Web UI proxy. Internal k8s consumers still use the ClusterIP service URL, not the public hostname. +- The Gitea public Web UI and GitHub webhook endpoint may share the YAML-declared HTTPS hostname only through explicit Caddy path routing. Each target node that needs GitHub -> Gitea sync must have its own YAML-declared webhook path and FRP proxy/remote port; multiple nodes must not reuse one GitHub hook URL or one bridge. Ordinary Web UI traffic continues to route to the Gitea Web UI proxy. Internal k8s consumers still use the ClusterIP service URL, not the public hostname. - Applying Gitea public exposure or webhook sync must roll the affected connector workloads after Secret or ConfigMap changes. A successful Secret apply is not sufficient evidence that `frpc` loaded a new proxy; closeout should use `platform-infra gitea apply` output plus `platform-infra gitea mirror webhook status|test` rather than assuming unchanged connector Pods picked up mounted config. - The canonical PaC entrypoints are `bun scripts/cli.ts platform-infra pipelines-as-code plan|apply|status|history|webhook-test --target `. PaC status is the operator-facing closeout surface for migrated CI lanes and must expose webhook count, latest PipelineRun/TaskRun duration, image status, env identity, digest, GitOps commit, Argo revision and runtime provenance without requiring raw `kubectl`, `tkn` or Gitea UI inspection. - PaC history is the trigger/timing audit surface for Gitea/PaC-managed lanes. It must query Gitea Repository CR and Tekton PipelineRun/TaskRun live objects on the target node, aggregate there, return Beijing-time display by YAML timezone, expose a detail id for drill-down, and report read errors explicitly; a large namespace or unreadable target object must never be rendered as a successful empty table. diff --git a/scripts/src/platform-infra-gitea-remote.sh b/scripts/src/platform-infra-gitea-remote.sh index 67b655ca..efda528e 100644 --- a/scripts/src/platform-infra-gitea-remote.sh +++ b/scripts/src/platform-infra-gitea-remote.sh @@ -703,13 +703,28 @@ run_mirror_webhook_status() { github_api_env bridge_ready=$(kubectl -n "$UNIDESK_GITEA_NAMESPACE" get deploy "$UNIDESK_GITEA_WEBHOOK_DEPLOYMENT" -o jsonpath='{.status.readyReplicas}/{.spec.replicas}' 2>/dev/null || echo "0/0") service_exists=$(kubectl -n "$UNIDESK_GITEA_NAMESPACE" get service "$UNIDESK_GITEA_WEBHOOK_SERVICE" -o name 2>/dev/null || true) - python3 - "$tmp/repos.json" "$bridge_ready" "$service_exists" <<'PY' -import json, os, sys, urllib.error, urllib.request + run_mirror_status >"$tmp/mirror-status.json" 2>"$tmp/mirror-status.err" + mirror_status_rc=$? + kubectl -n "$UNIDESK_GITEA_NAMESPACE" logs "deployment/$UNIDESK_GITEA_WEBHOOK_DEPLOYMENT" --tail=80 >"$tmp/bridge.log" 2>"$tmp/bridge.err" + bridge_logs_rc=$? + python3 - "$tmp/repos.json" "$bridge_ready" "$service_exists" "$mirror_status_rc" "$bridge_logs_rc" "$tmp/mirror-status.json" "$tmp/mirror-status.err" "$tmp/bridge.log" "$tmp/bridge.err" <<'PY' +import json, os, sys, urllib.error, urllib.parse, urllib.request repos = json.load(open(sys.argv[1], encoding="utf-8")) bridge_ready = sys.argv[2] service_exists = bool(sys.argv[3]) +mirror_status_rc = int(sys.argv[4]) +bridge_logs_rc = int(sys.argv[5]) +mirror_status_path = sys.argv[6] +mirror_status_err_path = sys.argv[7] +bridge_log_path = sys.argv[8] +bridge_log_err_path = sys.argv[9] token = os.environ["UNIDESK_GITEA_GITHUB_TOKEN"] url = os.environ["UNIDESK_GITEA_WEBHOOK_PUBLIC_URL"] +def text(path, limit=1600): + try: + return open(path, encoding="utf-8", errors="replace").read()[-limit:] + except FileNotFoundError: + return "" def request(api_path): req = urllib.request.Request("https://api.github.com" + api_path, method="GET") req.add_header("Authorization", "Bearer " + token) @@ -720,9 +735,60 @@ def request(api_path): return {"ok": True, "status": resp.status, "body": resp.read().decode("utf-8", errors="replace")} except urllib.error.HTTPError as exc: return {"ok": False, "status": exc.code, "body": exc.read().decode("utf-8", errors="replace")[:1000]} +def github_head(repository, branch): + result = request(f"/repos/{repository}/git/ref/heads/{urllib.parse.quote(branch, safe='')}") + try: + payload = json.loads(result.get("body") or "{}") + except Exception: + payload = {} + return {"ok": result.get("ok"), "status": result.get("status"), "sha": payload.get("object", {}).get("sha"), "error": None if result.get("ok") else result.get("body", "")[:300]} +def hook_deliveries(repository, hook_id): + if not hook_id: + return {"ok": False, "status": None, "latest": None, "latestPush": None, "error": "hook not found"} + result = request(f"/repos/{repository}/hooks/{hook_id}/deliveries?per_page=8") + try: + rows = json.loads(result.get("body") or "[]") + except Exception: + rows = [] + def compact(item): + if not isinstance(item, dict): + return None + return { + "id": item.get("id"), + "event": item.get("event"), + "status": item.get("status"), + "statusCode": item.get("status_code"), + "deliveredAt": item.get("delivered_at"), + "duration": item.get("duration"), + } + latest = compact(rows[0]) if rows else None + latest_push = next((compact(item) for item in rows if isinstance(item, dict) and item.get("event") == "push"), None) + return {"ok": result.get("ok"), "status": result.get("status"), "latest": latest, "latestPush": latest_push, "error": None if result.get("ok") else result.get("body", "")[:300]} +try: + mirror_status = json.load(open(mirror_status_path, encoding="utf-8")) +except Exception: + mirror_status = {} +mirror_by_key = {item.get("key"): item for item in mirror_status.get("repositories", []) if isinstance(item, dict)} +bridge_events = [] +for line in text(bridge_log_path, 12000).splitlines(): + try: + event = json.loads(line) + except Exception: + continue + if isinstance(event, dict) and str(event.get("event", "")).startswith("github-to-gitea"): + bridge_events.append({ + "event": event.get("event"), + "repo": event.get("repo"), + "ok": event.get("ok"), + "sourceCommit": event.get("sourceCommit"), + "snapshotRef": event.get("snapshotRef"), + "error": event.get("error"), + "elapsedMs": event.get("elapsedMs"), + }) rows = [] for repo in repos: repository = repo["upstream"]["repository"] + branch = repo["upstream"]["branch"] result = request(f"/repos/{repository}/hooks") hooks = [] try: @@ -730,9 +796,56 @@ for repo in repos: except Exception: hooks = [] match = next((item for item in hooks if (item.get("config") or {}).get("url") == url), None) - rows.append({"key": repo["key"], "repository": repository, "hookId": match.get("id") if match else None, "hookReady": bool(match and match.get("active") is True), "active": match.get("active") if match else None, "status": result.get("status"), "error": None if result.get("ok") else result.get("body", "")[:500], "valuesPrinted": False}) + hook_id = match.get("id") if match else None + head = github_head(repository, branch) + delivery = hook_deliveries(repository, hook_id) + mirror = mirror_by_key.get(repo["key"], {}) + branch_commit = mirror.get("branchCommit") + snapshot_commit = mirror.get("snapshotCommit") + branch_matches_github = bool(head.get("sha") and branch_commit == head.get("sha")) + snapshot_matches_branch = bool(branch_commit and snapshot_commit == branch_commit) + repo_bridge_events = [event for event in bridge_events if event.get("repo") == repo["key"]] + last_bridge_event = repo_bridge_events[-1] if repo_bridge_events else None + error = None + if not result.get("ok"): + error = result.get("body", "")[:500] + elif not head.get("ok"): + error = head.get("error") + elif not delivery.get("ok"): + error = delivery.get("error") + elif not branch_matches_github: + error = "gitea branch does not match GitHub head" + elif not snapshot_matches_branch: + error = "gitea snapshot does not match branch" + rows.append({ + "key": repo["key"], + "repository": repository, + "branch": branch, + "hookId": hook_id, + "hookReady": bool(match and match.get("active") is True), + "active": match.get("active") if match else None, + "status": result.get("status"), + "githubHead": head.get("sha"), + "branchCommit": branch_commit, + "snapshotCommit": snapshot_commit, + "branchMatchesGitHub": branch_matches_github, + "snapshotMatchesBranch": snapshot_matches_branch, + "latestDelivery": delivery.get("latest"), + "latestPushDelivery": delivery.get("latestPush"), + "lastBridgeEvent": last_bridge_event, + "error": error, + "valuesPrinted": False, + }) bridge = {"deployment": os.environ["UNIDESK_GITEA_WEBHOOK_DEPLOYMENT"], "ready": bridge_ready not in ("0/0", "/") and bridge_ready.split("/")[0] == bridge_ready.split("/")[-1], "readyReplicas": bridge_ready, "serviceExists": service_exists} -payload = {"ok": bridge["ready"] and service_exists and all(row["hookReady"] for row in rows), "bridge": bridge, "repositories": rows, "url": url, "valuesPrinted": False} +payload = { + "ok": bridge["ready"] and service_exists and mirror_status_rc == 0 and bridge_logs_rc == 0 and all(row["hookReady"] and row["branchMatchesGitHub"] and row["snapshotMatchesBranch"] for row in rows), + "bridge": bridge, + "repositories": rows, + "url": url, + "mirrorStatus": {"exitCode": mirror_status_rc, "errorTail": text(mirror_status_err_path)}, + "bridgeLogs": {"exitCode": bridge_logs_rc, "events": bridge_events[-12:], "errorTail": text(bridge_log_err_path)}, + "valuesPrinted": False, +} print(json.dumps(payload, ensure_ascii=False, indent=2)) sys.exit(0 if payload["ok"] else 1) PY diff --git a/scripts/src/platform-infra-gitea.ts b/scripts/src/platform-infra-gitea.ts index 07f830b3..cc182163 100644 --- a/scripts/src/platform-infra-gitea.ts +++ b/scripts/src/platform-infra-gitea.ts @@ -34,6 +34,15 @@ interface GiteaTarget { createNamespace: boolean; storageClassName: string; publicExposureEnabled: boolean; + webhookSync: GiteaTargetWebhookSync | null; +} + +interface GiteaTargetWebhookSync { + publicPath: string; + frpc: { + proxyName: string; + remotePort: number; + }; } interface GiteaConfig { @@ -576,6 +585,10 @@ function parseTarget(record: Record, index: number): GiteaTarge if (publicExposureRaw !== undefined && (typeof publicExposureRaw !== "object" || publicExposureRaw === null || Array.isArray(publicExposureRaw))) { throw new Error(`${configLabel}.${path}.publicExposure must be an object`); } + const webhookSyncRaw = record.webhookSync; + if (webhookSyncRaw !== undefined && (typeof webhookSyncRaw !== "object" || webhookSyncRaw === null || Array.isArray(webhookSyncRaw))) { + throw new Error(`${configLabel}.${path}.webhookSync must be an object`); + } const publicExposure = publicExposureRaw as Record | undefined; return { id: y.stringField(record, "id", path), @@ -586,6 +599,18 @@ function parseTarget(record: Record, index: number): GiteaTarge createNamespace: y.booleanField(record, "createNamespace", path), storageClassName: y.stringField(record, "storageClassName", path), publicExposureEnabled: publicExposure === undefined ? true : y.booleanField(publicExposure, "enabled", `${path}.publicExposure`), + webhookSync: webhookSyncRaw === undefined ? null : parseTargetWebhookSync(webhookSyncRaw as Record, `${path}.webhookSync`), + }; +} + +function parseTargetWebhookSync(record: Record, path: string): GiteaTargetWebhookSync { + const frpc = y.objectField(record, "frpc", path); + return { + publicPath: y.apiPathField(record, "publicPath", path), + frpc: { + proxyName: y.stringField(frpc, "proxyName", `${path}.frpc`), + remotePort: y.portField(frpc, "remotePort", `${path}.frpc`), + }, }; } @@ -605,6 +630,15 @@ function validateConfig(gitea: GiteaConfig): void { if (!events.has("push")) throw new Error(`${configLabel}.sourceAuthority.webhookSync.events must include push for GitHub -> Gitea source sync`); if (gitea.sourceAuthority.webhookSync.frpc.remotePort === gitea.app.publicExposure.frpc.remotePort) throw new Error(`${configLabel}.sourceAuthority.webhookSync.frpc.remotePort must differ from app.publicExposure.frpc.remotePort`); } + const webhookPaths = new Set(); + const webhookPorts = new Set(); + for (const route of webhookCaddyRoutes(gitea)) { + if (webhookPaths.has(route.publicPath)) throw new Error(`${configLabel} webhook publicPath must be unique: ${route.publicPath}`); + if (webhookPorts.has(route.remotePort)) throw new Error(`${configLabel} webhook frpc.remotePort must be unique: ${route.remotePort}`); + if (route.remotePort === gitea.app.publicExposure.frpc.remotePort) throw new Error(`${configLabel} webhook frpc.remotePort must differ from app public exposure port: ${route.remotePort}`); + webhookPaths.add(route.publicPath); + webhookPorts.add(route.remotePort); + } const repoKeys = new Set(); for (const repo of gitea.sourceAuthority.repositories) { if (repoKeys.has(repo.key)) throw new Error(`${configLabel}.sourceAuthority.repositories contains duplicate key ${repo.key}`); @@ -653,7 +687,7 @@ async function apply(config: UniDeskConfig, options: ApplyOptions): Promise = { UNIDESK_GITEA_ACTION: action, UNIDESK_GITEA_TARGET_ID: target.id, @@ -1274,16 +1312,16 @@ function remoteScript(action: "apply" | "status" | "validate" | "mirror-bootstra UNIDESK_GITEA_GITHUB_PROXY_ENABLED: gitea.sourceAuthority.githubProxy.enabled ? "1" : "0", UNIDESK_GITEA_GITHUB_PROXY_URL: gitea.sourceAuthority.githubProxy.url, UNIDESK_GITEA_NO_PROXY: gitea.sourceAuthority.githubProxy.noProxy.join(","), - UNIDESK_GITEA_FRPC_ENABLED: publicExposureEnabled(gitea, target) ? "1" : "0", - UNIDESK_GITEA_FRPC_DEPLOYMENT_NAME: gitea.app.publicExposure.frpc.deploymentName, + UNIDESK_GITEA_FRPC_ENABLED: frpcExposure === null ? "0" : "1", + UNIDESK_GITEA_FRPC_DEPLOYMENT_NAME: frpcExposure?.frpc.deploymentName ?? gitea.app.publicExposure.frpc.deploymentName, UNIDESK_GITEA_WEBHOOK_SYNC_ENABLED: gitea.sourceAuthority.webhookSync.enabled ? "1" : "0", - UNIDESK_GITEA_WEBHOOK_PUBLIC_URL: githubWebhookPublicUrl(gitea), - UNIDESK_GITEA_WEBHOOK_PATH: gitea.sourceAuthority.webhookSync.publicPath, - UNIDESK_GITEA_WEBHOOK_EVENTS: gitea.sourceAuthority.webhookSync.events.join(","), - UNIDESK_GITEA_WEBHOOK_DEPLOYMENT: gitea.sourceAuthority.webhookSync.bridge.deploymentName, - UNIDESK_GITEA_WEBHOOK_SERVICE: gitea.sourceAuthority.webhookSync.bridge.serviceName, - UNIDESK_GITEA_WEBHOOK_SERVICE_PORT: String(gitea.sourceAuthority.webhookSync.bridge.httpPort), - UNIDESK_GITEA_WEBHOOK_SECRET_NAME: gitea.sourceAuthority.webhookSync.bridge.secretName, + UNIDESK_GITEA_WEBHOOK_PUBLIC_URL: githubWebhookPublicUrl(gitea, target), + UNIDESK_GITEA_WEBHOOK_PATH: sync.publicPath, + UNIDESK_GITEA_WEBHOOK_EVENTS: sync.events.join(","), + UNIDESK_GITEA_WEBHOOK_DEPLOYMENT: sync.bridge.deploymentName, + UNIDESK_GITEA_WEBHOOK_SERVICE: sync.bridge.serviceName, + UNIDESK_GITEA_WEBHOOK_SERVICE_PORT: String(sync.bridge.httpPort), + UNIDESK_GITEA_WEBHOOK_SECRET_NAME: sync.bridge.secretName, }; if (params.frpcSecret !== undefined) { env.UNIDESK_GITEA_FRPC_SECRET_NAME = params.frpcSecret.secretName; @@ -1347,6 +1385,7 @@ function targetSummary(target: GiteaTarget): Record { createNamespace: target.createNamespace, storageClassName: target.storageClassName, publicExposureEnabled: target.publicExposureEnabled, + webhookSync: target.webhookSync, }; } @@ -1382,21 +1421,23 @@ function publicExposureEnabled(gitea: GiteaConfig, target: GiteaTarget): boolean } function targetPublicExposure(gitea: GiteaConfig, target: GiteaTarget): GiteaConfig["app"]["publicExposure"] { - return publicExposureEnabled(gitea, target) ? gitea.app.publicExposure : { ...gitea.app.publicExposure, enabled: false }; + const frpcExposure = targetFrpcExposure(gitea, target); + return frpcExposure === null ? { ...gitea.app.publicExposure, enabled: false } : frpcExposure; } function prepareGiteaFrpcSecret(gitea: GiteaConfig, target: GiteaTarget): FrpcSecretMaterial | undefined { - if (!publicExposureEnabled(gitea, target)) return undefined; + const exposure = targetFrpcExposure(gitea, target); + if (exposure === null) return undefined; const base = prepareFrpcSecret({ secretRoot: gitea.app.publicExposure.secretRoot, - exposure: gitea.app.publicExposure, + exposure, sourcePathRedactor: redactRepoPath, parseEnvFile, requiredEnvValue, readTextFile, }); - const sync = gitea.sourceAuthority.webhookSync; - if (!sync.enabled) return base; + const sync = targetWebhookSync(gitea, target); + if (!sync.enabled || !publicExposureEnabled(gitea, target)) return base; const extraProxy = [ "[[proxies]]", `name = "${escapeTomlString(sync.frpc.proxyName)}"`, @@ -1414,18 +1455,51 @@ function prepareGiteaFrpcSecret(gitea: GiteaConfig, target: GiteaTarget): FrpcSe }; } +function targetFrpcExposure(gitea: GiteaConfig, target: GiteaTarget): GiteaConfig["app"]["publicExposure"] | null { + if (publicExposureEnabled(gitea, target)) return gitea.app.publicExposure; + if (target.webhookSync === null || !gitea.sourceAuthority.webhookSync.enabled) return null; + const sync = targetWebhookSync(gitea, target); + return { + ...gitea.app.publicExposure, + enabled: true, + frpc: { + ...gitea.app.publicExposure.frpc, + proxyName: sync.frpc.proxyName, + remotePort: sync.frpc.remotePort, + localIP: `${sync.bridge.serviceName}.${target.namespace}.svc.cluster.local`, + localPort: sync.bridge.httpPort, + }, + }; +} + +function targetWebhookSync(gitea: GiteaConfig, target: GiteaTarget): GiteaWebhookSync { + const base = gitea.sourceAuthority.webhookSync; + if (target.webhookSync === null) return base; + return { + ...base, + publicPath: target.webhookSync.publicPath, + frpc: target.webhookSync.frpc, + }; +} + +function caddyExposureNeeded(gitea: GiteaConfig, target: GiteaTarget): boolean { + return publicExposureEnabled(gitea, target) || (gitea.sourceAuthority.webhookSync.enabled && target.webhookSync !== null); +} + async function applyGiteaCaddyBlock(config: UniDeskConfig, gitea: GiteaConfig): Promise> { const exposure = gitea.app.publicExposure; - const sync = gitea.sourceAuthority.webhookSync; - if (!sync.enabled) return await applyPk01CaddyBlock(config, "gitea", exposure); - const siteBlock = `${exposure.dns.hostname} { - handle ${sync.publicPath}* { - reverse_proxy 127.0.0.1:${sync.frpc.remotePort} { + const webhookHandles = webhookCaddyRoutes(gitea) + .sort((left, right) => right.publicPath.length - left.publicPath.length) + .map((route) => ` handle ${route.publicPath}* { + reverse_proxy 127.0.0.1:${route.remotePort} { transport http { response_header_timeout ${exposure.pk01.responseHeaderTimeoutSeconds}s } } - } + }`).join("\n\n"); + if (webhookHandles === "") return await applyPk01CaddyBlock(config, "gitea", exposure); + const siteBlock = `${exposure.dns.hostname} { +${webhookHandles} handle { reverse_proxy 127.0.0.1:${exposure.frpc.remotePort} { @@ -1438,6 +1512,19 @@ async function applyGiteaCaddyBlock(config: UniDeskConfig, gitea: GiteaConfig): return await applyPk01CaddySiteBlock(config, "gitea", exposure, siteBlock, caddyManagedBlockMarkers("gitea")); } +function webhookCaddyRoutes(gitea: GiteaConfig): Array<{ publicPath: string; remotePort: number }> { + const sync = gitea.sourceAuthority.webhookSync; + if (!sync.enabled) return []; + const routes = new Map(); + routes.set(sync.publicPath, { publicPath: sync.publicPath, remotePort: sync.frpc.remotePort }); + for (const target of gitea.targets) { + if (!target.enabled || target.webhookSync === null) continue; + const targetSync = targetWebhookSync(gitea, target); + routes.set(targetSync.publicPath, { publicPath: targetSync.publicPath, remotePort: targetSync.frpc.remotePort }); + } + return [...routes.values()]; +} + function publicExposureSummary(gitea: GiteaConfig): Record { const exposure = gitea.app.publicExposure; return { @@ -1469,11 +1556,11 @@ function frpcSecretSummary(secret: FrpcSecretMaterial | undefined): Record { - const sync = gitea.sourceAuthority.webhookSync; +function webhookSecretSummary(gitea: GiteaConfig, target: GiteaTarget, secrets: MirrorSecrets): Record { + const sync = targetWebhookSync(gitea, target); return { enabled: sync.enabled, - publicUrl: githubWebhookPublicUrl(gitea), + publicUrl: githubWebhookPublicUrl(gitea, target), sourceRef: sync.secret.sourceRef, targetSecret: sync.bridge.secretName, keys: ["github-token", "gitea-username", "gitea-password", "github-webhook-secret"], @@ -1482,9 +1569,9 @@ function webhookSecretSummary(gitea: GiteaConfig, secrets: MirrorSecrets): Recor }; } -function githubWebhookPublicUrl(gitea: GiteaConfig): string { +function githubWebhookPublicUrl(gitea: GiteaConfig, target: GiteaTarget): string { const base = gitea.app.publicExposure.publicBaseUrl.replace(/\/+$/u, ""); - return `${base}${gitea.sourceAuthority.webhookSync.publicPath}`; + return `${base}${targetWebhookSync(gitea, target).publicPath}`; } function statusSummary(payload: Record): Record { @@ -1573,12 +1660,12 @@ function mirrorNextCommands(targetId: string): Record { }; } -function webhookSyncSummary(gitea: GiteaConfig): Record { - const sync = gitea.sourceAuthority.webhookSync; +function webhookSyncSummary(gitea: GiteaConfig, target: GiteaTarget): Record { + const sync = targetWebhookSync(gitea, target); return { enabled: sync.enabled, direction: sync.direction, - publicUrl: githubWebhookPublicUrl(gitea), + publicUrl: githubWebhookPublicUrl(gitea, target), events: sync.events, bridge: sync.bridge, frpc: sync.frpc, @@ -1836,15 +1923,32 @@ function renderMirrorWebhookApply(result: Record): RenderedCliR } function renderMirrorWebhookStatus(result: Record): RenderedCliResult { - const repos = arrayRecords(result.repositories).map((repo) => [stringValue(repo.key), stringValue(repo.repository), boolText(repo.hookReady), stringValue(repo.hookId), stringValue(repo.active), compactTail(stringValue(repo.error))]); + const repos = arrayRecords(result.repositories).map((repo) => { + const latestPush = record(repo.latestPushDelivery); + const latest = record(repo.latestDelivery); + const delivery = latestPush.event !== "" ? latestPush : latest; + const bridge = record(repo.lastBridgeEvent); + return [ + stringValue(repo.key), + boolText(repo.hookReady), + stringValue(repo.githubHead).slice(0, 12), + stringValue(repo.branchCommit).slice(0, 12), + stringValue(repo.snapshotCommit).slice(0, 12), + boolText(repo.branchMatchesGitHub), + `${stringValue(delivery.event)}:${stringValue(delivery.statusCode)}:${stringValue(delivery.status)}`, + `${stringValue(bridge.event)}:${boolText(bridge.ok)}:${stringValue(bridge.sourceCommit).slice(0, 12)}`, + compactTail(stringValue(repo.error)), + ]; + }); const bridge = record(result.bridge); + const bridgeLogs = record(result.bridgeLogs); const next = record(result.next); return rendered(result, "platform-infra gitea mirror webhook status", [ "PLATFORM-INFRA GITEA MIRROR WEBHOOK STATUS", - ...table(["TARGET", "BRIDGE", "READY", "URL"], [[stringValue(record(result.target).id), stringValue(bridge.deployment), boolText(bridge.ready), stringValue(record(result.webhook).publicUrl)]]), + ...table(["TARGET", "BRIDGE", "READY", "URL", "LOGS"], [[stringValue(record(result.target).id), stringValue(bridge.deployment), boolText(bridge.ready), stringValue(record(result.webhook).publicUrl), `rc=${stringValue(bridgeLogs.exitCode)}`]]), "", "GITHUB HOOKS", - ...(repos.length === 0 ? ["-"] : table(["KEY", "REPOSITORY", "READY", "HOOK_ID", "ACTIVE", "ERROR"], repos)), + ...(repos.length === 0 ? ["-"] : table(["KEY", "HOOK", "GH_HEAD", "GITEA", "SNAPSHOT", "MATCH", "DELIVERY", "BRIDGE_EVENT", "ERROR"], repos)), ...remoteErrorLines(result), "", `NEXT ${stringValue(next.webhookApply)}`,