feat: add repository-scoped GitHub authority
This commit is contained in:
@@ -356,6 +356,32 @@ sourceAuthority:
|
||||
disposition: replaced-by-gitea
|
||||
- key: selfmedia-nc01
|
||||
targetId: NC01
|
||||
credentialOverride:
|
||||
github:
|
||||
transport: https-token
|
||||
sourceRef: pikainc-selfmedia-gh-token.txt
|
||||
sourceKey: GH_TOKEN
|
||||
format: raw-token
|
||||
requiredFor:
|
||||
- upstream-mirror
|
||||
- mirror-sync
|
||||
- managed-repository-fetch
|
||||
- github-head-observe
|
||||
- github-hooks-list
|
||||
- github-hooks-reconcile
|
||||
permissions:
|
||||
contents: read
|
||||
metadata: read
|
||||
webhooks: read-write
|
||||
gitFetchCredential:
|
||||
apiVersion: unidesk.ai/v1
|
||||
kind: GitFetchCredential
|
||||
authMode: github-https-token
|
||||
host: github.com
|
||||
secretRef:
|
||||
namespace: devops-infra
|
||||
name: gitea-github-sync-secrets
|
||||
key: github-token-pikainc-selfmedia
|
||||
upstream:
|
||||
repository: pikainc/selfmedia
|
||||
cloneUrl: https://github.com/pikainc/selfmedia.git
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# R1.1 任务报告
|
||||
|
||||
## 任务回链
|
||||
|
||||
- MDTODO:R1.1。
|
||||
- Target:NC01。
|
||||
- targetWorkspace:`/root/.worktrees/unidesk/selfmedia-repo-github-authority`。
|
||||
- repository:`pikasTech/unidesk`。
|
||||
- ref:`feat/selfmedia-repo-github-authority`。
|
||||
|
||||
## Primary Workspace 轻量准备
|
||||
|
||||
- 仅加载 `dad-dev`、`unidesk-trans`、`unidesk-ymalops`、`cli-spec`、`git-spec`、`unidesk-gh` 与 `docs-spec`。
|
||||
- 未在 runner primary workspace 读取、修改或验证目标源码。
|
||||
- 未把 runner 本地结果作为 Target 原入口证据。
|
||||
|
||||
## Target 实现
|
||||
|
||||
- 在 `config/platform-infra/gitea.yaml` 为 `selfmedia-nc01` 声明 repository credential override。
|
||||
- 独立 `sourceRef` 为 `/root/.unidesk/.env` 根下的 `pikainc-selfmedia-gh-token.txt`。
|
||||
- 独立 Secret key 为 `github-token-pikainc-selfmedia`,未修改全局 `gh_token.txt` 权限或内容。
|
||||
- YAML 声明最小权限:Contents Read、Metadata Read、Webhooks Read/Write。
|
||||
- 配置 parser 与 validator 支持通用 repository override,并校验 Secret key 唯一性、环境变量归一化碰撞、目标 namespace、bridge Secret 归属和必需能力。
|
||||
- Secret 渲染支持多个 GitHub token key;token 仅通过环境变量进入受控执行,不进入 YAML、Git、日志或报告。
|
||||
- bridge、candidate、受控 fetch、GitHub head、hooks list/status 和 hook reconciler 均按 repository 选择 credential。
|
||||
- bridge/candidate 只注入所属 Target 仓库 token;hook owner reconciler 额外注入所有声明 `github-hooks-*` 的 override;JD01 不依赖 NC01 selfmedia token。
|
||||
- CLI credential 摘要对 override 仅披露 `sourceRef`、`present`、`requiredKeysPresent`、`fingerprint`、`permissionResult`、有界错误与 `valuesPrinted=false`。
|
||||
|
||||
## Target 原入口证据
|
||||
|
||||
- 已通过 `trans NC01:/root/.worktrees/unidesk/selfmedia-repo-github-authority` 执行全部源码、Git 和验证操作。
|
||||
- `bun test scripts/src/platform-infra-gitea-config.test.ts scripts/src/platform-infra-gitea-disclosure.test.ts`:8 项通过,0 项失败。
|
||||
- `bun scripts/cli.ts platform-infra gitea plan --target NC01`:policy 为 `ok`。
|
||||
- `git diff --check`、Node 语法、Python `py_compile`、Shell `sh -n`:通过。
|
||||
- `mirror bootstrap --target NC01 --repo selfmedia-nc01 --dry-run`:`OK=false`,blocker 为 `github-upstream:selfmedia-nc01`,符合 fail-closed。
|
||||
- `mirror bootstrap --target JD01 --dry-run`:`OK=true`,证明 JD01 不依赖 NC01 override。
|
||||
- 专用凭据只读状态:`sourceRef=pikainc-selfmedia-gh-token.txt`、`presence=false`、`fingerprint=null`、`permissionResult=blocked-missing-credential`。
|
||||
- 因 credential material 缺失,未执行权限 API 探测;不得声称 Contents、Metadata 或 Webhooks 权限已实际授予。
|
||||
|
||||
## 并行保护
|
||||
|
||||
- 已两次按 `stash -u`、`fetch`、`rebase origin/master`、`stash apply` 安全吸收主线。
|
||||
- 最终吸收 `origin/master@3aaa2616`。
|
||||
- R1.2 保持 `[completed]`,`R1.2_Task_Report.md` 保留。
|
||||
|
||||
## 未执行事项
|
||||
|
||||
- 未创建假 token,未复制 SSH 私钥,未修改仓库可见性。
|
||||
- 未手工 apply、Secret sync、PipelineRun、Argo sync/refresh、rollout 或 cutover。
|
||||
- 未触发 PaC 或任何部署,只保留只读运行面证据。
|
||||
|
||||
## 当前结论
|
||||
|
||||
- R1.1 配置、schema、renderer、Secret、bridge、observer 与受控 fetch 的源码能力已落地并通过轻量验证。
|
||||
- 外部 credential material 仍缺失,R1.1 保持 `in_progress`。
|
||||
- 凭据到位后的下一步仅允许通过受控 CLI 观察脱敏 permissionResult、hook reconciler 与自动交付收敛,不得以手工触发替代自动链路。
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
打通 selfmedia WebTerm 在 NC01 的纯自动生产交付:以 config/selfmedia.yaml 与 platform-infra owning YAML 为唯一真相,修复 GitHub→Gitea mirror→PaC/Tekton→GitOps/Argo 的自动触发与可见性,使合并 pikainc/selfmedia/master 后无需人工触发即可构建并发布;保留零 RBAC runtime、双 PVC、YAML-first Secret、NetworkPolicy、公网 IP 152.53.229.148:4317,禁止用手工 PipelineRun、Argo sync/refresh、rollout 或 cutover 代替自动链路,完成任务后将详细报告写入[任务报告](./details/selfmedia-production-delivery/R1_Task_Report.md)。
|
||||
|
||||
### R1.1
|
||||
### R1.1 [in_progress]
|
||||
|
||||
以 YAML-first 为 selfmedia-nc01 增加仓库级 GitHub authority:使用独立 sourceRef 提供 pikainc/selfmedia 私有仓库 Contents Read 与 Webhooks Read/Write 权限,校验权限/Secret presence 与脱敏 fingerprint;不扩大全局 gh_token.txt、不向 Codex 下发、不手工触发 PaC,待凭据到位后仅观察 hook reconciler 与自动交付收敛,并将详细报告写入任务报告,完成任务后将详细报告写入[任务报告](./details/selfmedia-production-delivery/R1.1_Task_Report.md)。
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import {
|
||||
githubTokenEnvNameForSecretKey,
|
||||
readGiteaConfig,
|
||||
resolveTarget,
|
||||
} from "./platform-infra-gitea-config";
|
||||
import { renderManifest } from "./platform-infra-gitea";
|
||||
|
||||
describe("platform-infra Gitea repository GitHub credentials", () => {
|
||||
test("keeps the global credential and selfmedia override on distinct Secret keys", () => {
|
||||
const config = readGiteaConfig();
|
||||
const globalCredential = config.sourceAuthority.credentials.github;
|
||||
const selfmedia = config.sourceAuthority.repositories.find((repo) => repo.key === "selfmedia-nc01");
|
||||
const override = selfmedia?.credentialOverride?.github;
|
||||
|
||||
expect(override?.sourceRef).toBe("pikainc-selfmedia-gh-token.txt");
|
||||
expect(override?.permissions).toEqual({ contents: "read", metadata: "read", webhooks: "read-write" });
|
||||
expect(override?.gitFetchCredential.secretRef.key).not.toBe(globalCredential.gitFetchCredential.secretRef.key);
|
||||
expect(githubTokenEnvNameForSecretKey(override?.gitFetchCredential.secretRef.key ?? ""))
|
||||
.not.toBe(githubTokenEnvNameForSecretKey(globalCredential.gitFetchCredential.secretRef.key));
|
||||
});
|
||||
|
||||
test("normalizes Secret keys to bounded environment variable names", () => {
|
||||
expect(githubTokenEnvNameForSecretKey("github-token-pikainc-selfmedia"))
|
||||
.toBe("UNIDESK_GITEA_GITHUB_TOKEN_GITHUB_TOKEN_PIKAINC_SELFMEDIA");
|
||||
});
|
||||
|
||||
test("keeps repository credential overrides scoped to their configured target", () => {
|
||||
const config = readGiteaConfig();
|
||||
const jd01Overrides = config.sourceAuthority.repositories
|
||||
.filter((repo) => repo.targetId === "JD01")
|
||||
.flatMap((repo) => repo.credentialOverride?.github ?? []);
|
||||
const nc01Overrides = config.sourceAuthority.repositories
|
||||
.filter((repo) => repo.targetId === "NC01")
|
||||
.flatMap((repo) => repo.credentialOverride?.github ?? []);
|
||||
|
||||
expect(jd01Overrides).toHaveLength(0);
|
||||
expect(nc01Overrides.map((credential) => credential.sourceRef)).toContain("pikainc-selfmedia-gh-token.txt");
|
||||
});
|
||||
|
||||
test("does not inject the NC01 selfmedia token into JD01 bridge containers", () => {
|
||||
const config = readGiteaConfig();
|
||||
const tokenEnv = githubTokenEnvNameForSecretKey("github-token-pikainc-selfmedia");
|
||||
const jd01Manifest = renderManifest(config, resolveTarget(config, "JD01"));
|
||||
const nc01Manifest = renderManifest(config, resolveTarget(config, "NC01"));
|
||||
|
||||
expect(jd01Manifest).not.toContain(tokenEnv);
|
||||
expect(nc01Manifest).toContain(tokenEnv);
|
||||
});
|
||||
});
|
||||
@@ -160,6 +160,16 @@ export interface GiteaGithubCredential {
|
||||
};
|
||||
}
|
||||
|
||||
export interface GiteaGithubPermissions {
|
||||
contents: "read";
|
||||
metadata: "read";
|
||||
webhooks: "read-write";
|
||||
}
|
||||
|
||||
export function githubTokenEnvNameForSecretKey(secretKey: string): string {
|
||||
return `UNIDESK_GITEA_GITHUB_TOKEN_${secretKey.replace(/[^A-Za-z0-9_]/gu, "_").toUpperCase()}`;
|
||||
}
|
||||
|
||||
export interface GiteaWebhookSync {
|
||||
enabled: boolean;
|
||||
direction: "github-to-gitea";
|
||||
@@ -273,6 +283,9 @@ export interface GiteaSourceResponsibility {
|
||||
export interface GiteaMirrorRepository {
|
||||
key: string;
|
||||
targetId: string;
|
||||
credentialOverride: {
|
||||
github: GiteaGithubCredential & { permissions: GiteaGithubPermissions };
|
||||
} | null;
|
||||
upstream: {
|
||||
repository: string;
|
||||
cloneUrl: string;
|
||||
@@ -657,9 +670,28 @@ function parseMirrorRepository(record: Record<string, unknown>, index: number):
|
||||
const legacyGitMirror = record.legacyGitMirror === null || record.legacyGitMirror === undefined
|
||||
? null
|
||||
: y.objectField(record, "legacyGitMirror", path);
|
||||
const credentialOverride = record.credentialOverride === undefined
|
||||
? null
|
||||
: y.objectField(record, "credentialOverride", path);
|
||||
const githubOverride = credentialOverride === null
|
||||
? null
|
||||
: y.objectField(credentialOverride, "github", `${path}.credentialOverride`);
|
||||
const githubPermissions = githubOverride === null
|
||||
? null
|
||||
: y.objectField(githubOverride, "permissions", `${path}.credentialOverride.github`);
|
||||
return {
|
||||
key: y.stringField(record, "key", path),
|
||||
targetId: y.stringField(record, "targetId", path),
|
||||
credentialOverride: githubOverride === null || githubPermissions === null ? null : {
|
||||
github: {
|
||||
...parseGithubCredential(githubOverride, `${path}.credentialOverride.github`),
|
||||
permissions: {
|
||||
contents: y.enumField(githubPermissions, "contents", `${path}.credentialOverride.github.permissions`, ["read"] as const),
|
||||
metadata: y.enumField(githubPermissions, "metadata", `${path}.credentialOverride.github.permissions`, ["read"] as const),
|
||||
webhooks: y.enumField(githubPermissions, "webhooks", `${path}.credentialOverride.github.permissions`, ["read-write"] as const),
|
||||
},
|
||||
},
|
||||
},
|
||||
upstream: {
|
||||
repository: repositoryField(upstream, "repository", `${path}.upstream`),
|
||||
cloneUrl: gitCloneUrlField(upstream, "cloneUrl", `${path}.upstream`),
|
||||
@@ -738,6 +770,35 @@ function validateConfig(gitea: GiteaConfig): void {
|
||||
if (gitea.targets.some((target) => target.enabled && target.namespace !== gitFetchCredential.secretRef.namespace)) {
|
||||
throw new Error(`${configLabel}.sourceAuthority.credentials.github.gitFetchCredential.secretRef.namespace must match every enabled Gitea target namespace`);
|
||||
}
|
||||
const credentialKeys = new Set([gitFetchCredential.secretRef.key]);
|
||||
const credentialEnvNames = new Set([githubTokenEnvNameForSecretKey(gitFetchCredential.secretRef.key)]);
|
||||
for (const repo of gitea.sourceAuthority.repositories) {
|
||||
const override = repo.credentialOverride?.github;
|
||||
if (override === undefined) continue;
|
||||
if (!override.requiredFor.includes("managed-repository-fetch")
|
||||
|| !override.requiredFor.includes("github-head-observe")
|
||||
|| !override.requiredFor.includes("github-hooks-list")
|
||||
|| !override.requiredFor.includes("github-hooks-reconcile")) {
|
||||
throw new Error(`${configLabel}.sourceAuthority.repositories.${repo.key}.credentialOverride.github.requiredFor must cover fetch, head observation and hook reconciliation`);
|
||||
}
|
||||
if (override.gitFetchCredential.secretRef.name !== gitea.sourceAuthority.webhookSync.bridge.secretName) {
|
||||
throw new Error(`${configLabel}.sourceAuthority.repositories.${repo.key}.credentialOverride.github.gitFetchCredential.secretRef.name must match sourceAuthority.webhookSync.bridge.secretName`);
|
||||
}
|
||||
const target = resolveTarget(gitea, repo.targetId);
|
||||
if (override.gitFetchCredential.secretRef.namespace !== target.namespace) {
|
||||
throw new Error(`${configLabel}.sourceAuthority.repositories.${repo.key}.credentialOverride.github.gitFetchCredential.secretRef.namespace must match repository target namespace`);
|
||||
}
|
||||
const key = override.gitFetchCredential.secretRef.key;
|
||||
if (!/^[A-Za-z0-9._-]+$/u.test(key) || credentialKeys.has(key)) {
|
||||
throw new Error(`${configLabel}.sourceAuthority.repositories.${repo.key}.credentialOverride.github.gitFetchCredential.secretRef.key must be a unique Kubernetes Secret data key`);
|
||||
}
|
||||
credentialKeys.add(key);
|
||||
const envName = githubTokenEnvNameForSecretKey(key);
|
||||
if (credentialEnvNames.has(envName)) {
|
||||
throw new Error(`${configLabel}.sourceAuthority.repositories.${repo.key}.credentialOverride.github.gitFetchCredential.secretRef.key collides after environment variable normalization`);
|
||||
}
|
||||
credentialEnvNames.add(envName);
|
||||
}
|
||||
if (!gitea.sourceAuthority.enabled) throw new Error(`${configLabel}.sourceAuthority.enabled must be true for GH-1550`);
|
||||
if (gitea.sourceAuthority.repositories.length < 1) throw new Error(`${configLabel}.sourceAuthority.repositories must not be empty`);
|
||||
if (!/^docker\.gitea\.com\/gitea$/u.test(gitea.app.image.repository)) throw new Error(`${configLabel}.app.image.repository must use the official Gitea image registry`);
|
||||
|
||||
@@ -751,13 +751,13 @@ export async function syncRepository(repo, delivery, runtime) {
|
||||
let requestedObject = await executeResult(["git", "-C", work, "cat-file", "-e", `${delivery.requestedCommit}^{commit}`]);
|
||||
if (requestedObject.status !== 0) {
|
||||
const branchFetch = await executeResult([
|
||||
"git", "-C", work, "-c", `http.extraHeader=${runtime.githubAuthHeader}`,
|
||||
"git", "-C", work, "-c", `http.extraHeader=${runtime.githubAuthHeaders[repo.key]}`,
|
||||
"fetch", "--no-tags", repo.upstream.cloneUrl,
|
||||
`+refs/heads/${repo.upstream.branch}:refs/remotes/github/${repo.upstream.branch}`,
|
||||
]);
|
||||
requestedObject = await executeResult(["git", "-C", work, "cat-file", "-e", `${delivery.requestedCommit}^{commit}`]);
|
||||
if (requestedObject.status !== 0) {
|
||||
const exactFetch = await executeResult(["git", "-C", work, "-c", `http.extraHeader=${runtime.githubAuthHeader}`, "fetch", "--no-tags", repo.upstream.cloneUrl, `${delivery.requestedCommit}:refs/remotes/github/delivery`]);
|
||||
const exactFetch = await executeResult(["git", "-C", work, "-c", `http.extraHeader=${runtime.githubAuthHeaders[repo.key]}`, "fetch", "--no-tags", repo.upstream.cloneUrl, `${delivery.requestedCommit}:refs/remotes/github/delivery`]);
|
||||
if (exactFetch.status !== 0) {
|
||||
return {
|
||||
ok: false,
|
||||
@@ -931,14 +931,19 @@ export function retryDelayMs(syncAttempt, initialDelayMs, maxDelayMs) {
|
||||
}
|
||||
|
||||
export function loadRuntimeConfig() {
|
||||
const githubToken = requiredEnv("GITHUB_TOKEN");
|
||||
const giteaUsername = requiredEnv("GITEA_USERNAME");
|
||||
const giteaPassword = requiredEnv("GITEA_PASSWORD");
|
||||
const repos = JSON.parse(readFileSync(process.env.UNIDESK_GITEA_REPOS_PATH || "/etc/gitea-github-sync/repos.json", "utf8"));
|
||||
const defaultTokenEnv = requiredEnv("UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV");
|
||||
const githubAuthHeaders = Object.fromEntries(repos.map((repo) => {
|
||||
const token = requiredEnv(repo.githubCredential?.tokenEnv || defaultTokenEnv);
|
||||
return [repo.key, `Authorization: Basic ${Buffer.from(`x-access-token:${token}`).toString("base64")}`];
|
||||
}));
|
||||
return {
|
||||
port: Number.parseInt(process.env.UNIDESK_GITEA_WEBHOOK_PORT || "8080", 10),
|
||||
path: process.env.UNIDESK_GITEA_WEBHOOK_PATH || "/_unidesk/github-to-gitea",
|
||||
responseBudgetMs: positiveIntegerEnv("UNIDESK_GITEA_WEBHOOK_RESPONSE_BUDGET_MS"),
|
||||
repos: JSON.parse(readFileSync(process.env.UNIDESK_GITEA_REPOS_PATH || "/etc/gitea-github-sync/repos.json", "utf8")),
|
||||
repos,
|
||||
webhookSecret: requiredEnv("GITHUB_WEBHOOK_SECRET"),
|
||||
giteaBaseUrl: requiredEnv("UNIDESK_GITEA_SERVICE_BASE_URL").replace(/\/+$/u, ""),
|
||||
inbox: {
|
||||
@@ -958,7 +963,7 @@ export function loadRuntimeConfig() {
|
||||
},
|
||||
shutdownGraceMs: positiveIntegerEnv("UNIDESK_GITEA_WEBHOOK_SHUTDOWN_GRACE_MS"),
|
||||
maxBodyBytes: positiveIntegerEnv("UNIDESK_GITEA_WEBHOOK_MAX_BODY_BYTES"),
|
||||
githubAuthHeader: `Authorization: Basic ${Buffer.from(`x-access-token:${githubToken}`).toString("base64")}`,
|
||||
githubAuthHeaders,
|
||||
giteaAuthHeader: `Authorization: Basic ${Buffer.from(`${giteaUsername}:${giteaPassword}`).toString("base64")}`,
|
||||
log,
|
||||
run,
|
||||
|
||||
@@ -52,13 +52,33 @@ run_apply() {
|
||||
fi
|
||||
fi
|
||||
if [ "$UNIDESK_GITEA_WEBHOOK_SYNC_ENABLED" = "1" ] && [ "$UNIDESK_GITEA_DRY_RUN" != "1" ]; then
|
||||
github_secret_args=""
|
||||
old_ifs="$IFS"
|
||||
IFS=','
|
||||
for mapping in $UNIDESK_GITEA_GITHUB_SECRET_ENV_MAP; do
|
||||
secret_key=${mapping%%=*}
|
||||
env_name=${mapping#*=}
|
||||
eval "secret_value=\${$env_name-}"
|
||||
if [ -z "$secret_value" ]; then
|
||||
printf '%s\n' "missing GitHub credential environment for Secret key $secret_key" >"$tmp/webhook-secret.err"
|
||||
webhook_secret_rc=1
|
||||
break
|
||||
fi
|
||||
secret_file="$tmp/github-secret-$secret_key"
|
||||
printf '%s' "$secret_value" >"$secret_file"
|
||||
chmod 600 "$secret_file"
|
||||
github_secret_args="$github_secret_args --from-file=$secret_key=$secret_file"
|
||||
done
|
||||
IFS="$old_ifs"
|
||||
if [ "${webhook_secret_rc:-0}" -eq 0 ]; then
|
||||
kubectl -n "$UNIDESK_GITEA_NAMESPACE" create secret generic "$UNIDESK_GITEA_WEBHOOK_SECRET_NAME" \
|
||||
--from-literal="$UNIDESK_GITEA_GITHUB_SECRET_KEY=$UNIDESK_GITEA_GITHUB_TOKEN" \
|
||||
$github_secret_args \
|
||||
--from-literal=gitea-username="$UNIDESK_GITEA_ADMIN_USERNAME" \
|
||||
--from-literal=gitea-password="$UNIDESK_GITEA_ADMIN_PASSWORD" \
|
||||
--from-literal=github-webhook-secret="$UNIDESK_GITEA_GITHUB_WEBHOOK_SECRET" \
|
||||
--dry-run=client -o yaml | kubectl apply --server-side --force-conflicts --field-manager="$UNIDESK_GITEA_FIELD_MANAGER" -f - >"$tmp/webhook-secret.out" 2>"$tmp/webhook-secret.err"
|
||||
webhook_secret_rc=$?
|
||||
fi
|
||||
else
|
||||
webhook_secret_rc=0
|
||||
: >"$tmp/webhook-secret.out"
|
||||
@@ -518,11 +538,12 @@ for i, repo in enumerate(repos):
|
||||
sync_gitops_from_upstream = repo["gitops"].get("flushDisposition") != "gitea-writeback"
|
||||
branches = [source_branch] + ([] if not sync_gitops_from_upstream or gitops_branch == source_branch or gitops_branch == "not-a-gitops-branch" else [gitops_branch])
|
||||
work = f"$tmp/repo-{i}.git"
|
||||
token_env = (repo.get("githubCredential") or {}).get("tokenEnv") or os.environ.get("UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV", "")
|
||||
gitea_write_url = base_url + urllib.parse.urlparse(repo["gitea"]["readUrl"]).path
|
||||
script += [
|
||||
f"mkdir -p {shlex.quote(work)}",
|
||||
f"git init --bare {shlex.quote(work)} >$tmp/{key}.init.out 2>$tmp/{key}.init.err; printf '%s' \"$?\" >$tmp/{key}.init.rc",
|
||||
f"git -C {shlex.quote(work)} -c http.extraHeader=\"$GITHUB_AUTH_HEADER\" fetch --prune {shlex.quote(repo['upstream']['cloneUrl'])} " + " ".join(shlex.quote(f"+refs/heads/{b}:refs/remotes/github/{b}") for b in branches) + f" >$tmp/{key}.fetch.out 2>$tmp/{key}.fetch.err; fetch_rc=$?; printf '%s' \"$fetch_rc\" >$tmp/{key}.fetch.rc",
|
||||
f"repo_token=$(printenv {shlex.quote(token_env)}); repo_basic=$(printf 'x-access-token:%s' \"$repo_token\" | base64 | tr -d '\\n'); git -C {shlex.quote(work)} -c http.extraHeader=\"Authorization: Basic $repo_basic\" fetch --prune {shlex.quote(repo['upstream']['cloneUrl'])} " + " ".join(shlex.quote(f"+refs/heads/{b}:refs/remotes/github/{b}") for b in branches) + f" >$tmp/{key}.fetch.out 2>$tmp/{key}.fetch.err; fetch_rc=$?; unset repo_token repo_basic; printf '%s' \"$fetch_rc\" >$tmp/{key}.fetch.rc",
|
||||
f"if [ \"$fetch_rc\" -eq 0 ]; then sha=$(git -C {shlex.quote(work)} rev-parse refs/remotes/github/{shlex.quote(source_branch)} 2>$tmp/{key}.revparse.err); revparse_rc=$?; printf '%s\\n' \"$sha\" >$tmp/{key}.revparse.out; else sha=''; revparse_rc=1; : >$tmp/{key}.revparse.out; printf '%s\\n' 'fetch failed; revparse skipped' >$tmp/{key}.revparse.err; fi; printf '%s' \"$revparse_rc\" >$tmp/{key}.revparse.rc",
|
||||
f"if [ \"$revparse_rc\" -eq 0 ]; then snapshot_ref={shlex.quote(repo['snapshot']['prefix'])}/$sha; git -C {shlex.quote(work)} -c http.extraHeader=\"$GITEA_AUTH_HEADER\" push --force {shlex.quote(gitea_write_url)} $sha:refs/heads/{shlex.quote(source_branch)} $sha:$snapshot_ref >$tmp/{key}.push.out 2>$tmp/{key}.push.err; push_rc=$?; else snapshot_ref=''; push_rc=1; : >$tmp/{key}.push.out; printf '%s\\n' 'revparse failed; push skipped' >$tmp/{key}.push.err; fi; printf '%s' \"$push_rc\" >$tmp/{key}.push.rc",
|
||||
]
|
||||
@@ -537,15 +558,8 @@ for i, repo in enumerate(repos):
|
||||
open(sys.argv[2], "w", encoding="utf-8").write("\n".join(script) + "\n")
|
||||
json.dump(meta, open(sys.argv[3], "w", encoding="utf-8"), ensure_ascii=False, indent=2)
|
||||
PY
|
||||
github_basic="$(python3 - <<'PY'
|
||||
import base64, os
|
||||
raw = f"x-access-token:{os.environ['UNIDESK_GITEA_GITHUB_TOKEN']}".encode()
|
||||
print(base64.b64encode(raw).decode())
|
||||
PY
|
||||
)"
|
||||
GITHUB_AUTH_HEADER="Authorization: Basic $github_basic"
|
||||
GITEA_AUTH_HEADER="Authorization: Basic $basic"
|
||||
export GITHUB_AUTH_HEADER GITEA_AUTH_HEADER
|
||||
export GITEA_AUTH_HEADER
|
||||
unset GIT_SSH GIT_SSH_COMMAND SSH_AUTH_SOCK
|
||||
export GIT_TERMINAL_PROMPT=0
|
||||
export GIT_CONFIG_NOSYSTEM=1
|
||||
@@ -685,7 +699,6 @@ run_mirror_webhook_apply() {
|
||||
python3 - "$tmp/repos.json" <<'PY'
|
||||
import json, os, sys, urllib.error, urllib.request
|
||||
repos = json.load(open(sys.argv[1], encoding="utf-8"))
|
||||
token = os.environ["UNIDESK_GITEA_GITHUB_TOKEN"]
|
||||
url = os.environ["UNIDESK_GITEA_WEBHOOK_PUBLIC_URL"]
|
||||
secret = os.environ["UNIDESK_GITEA_GITHUB_WEBHOOK_SECRET"]
|
||||
events = [item for item in os.environ.get("UNIDESK_GITEA_WEBHOOK_EVENTS", "push").split(",") if item]
|
||||
@@ -707,6 +720,8 @@ def request(method, api_path, payload=None, expected=(200, 201, 204), tolerate=(
|
||||
rows = []
|
||||
for repo in repos:
|
||||
repository = repo["upstream"]["repository"]
|
||||
token_env = (repo.get("githubCredential") or {}).get("tokenEnv") or os.environ["UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV"]
|
||||
token = os.environ[token_env]
|
||||
list_result = request("GET", f"/repos/{repository}/hooks")
|
||||
hooks = []
|
||||
try:
|
||||
@@ -771,7 +786,6 @@ bridge_log_err_path = sys.argv[10]
|
||||
inbox_status_path = sys.argv[11]
|
||||
inbox_status_err_path = sys.argv[12]
|
||||
hook_topology_path = sys.argv[13]
|
||||
token = os.environ["UNIDESK_GITEA_GITHUB_TOKEN"]
|
||||
url = os.environ["UNIDESK_GITEA_WEBHOOK_PUBLIC_URL"]
|
||||
try:
|
||||
hook_topology = json.load(open(hook_topology_path, encoding="utf-8"))
|
||||
@@ -990,6 +1004,8 @@ for line in text(bridge_log_path, 12000).splitlines():
|
||||
rows = []
|
||||
for repo in repos:
|
||||
repository = repo["upstream"]["repository"]
|
||||
topology_spec = next((item for item in topology_repositories if isinstance(item, dict) and item.get("repository") == repository), {})
|
||||
token = os.environ[topology_spec.get("githubTokenEnv") or os.environ["UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV"]]
|
||||
branch = repo["upstream"]["branch"]
|
||||
result = request(f"/repos/{repository}/hooks")
|
||||
hooks_result = parse_github_hooks_response(result.get("ok"), result.get("status"), result.get("body"))
|
||||
|
||||
@@ -36,6 +36,7 @@ import {
|
||||
} from "./platform-infra-gitea-render";
|
||||
import {
|
||||
GITEA_CONFIG_LABEL,
|
||||
githubTokenEnvNameForSecretKey,
|
||||
readGiteaConfig,
|
||||
resolveTarget,
|
||||
targetWebhookSync,
|
||||
@@ -83,7 +84,7 @@ interface MirrorWebhookStatusOptions extends MirrorOptions {
|
||||
interface MirrorSecrets {
|
||||
adminUsername: string;
|
||||
adminPassword: string;
|
||||
githubToken: string;
|
||||
githubTokens: Record<string, string>;
|
||||
webhookSecret: string;
|
||||
}
|
||||
|
||||
@@ -188,7 +189,10 @@ async function apply(config: UniDeskConfig, options: ApplyOptions): Promise<Reco
|
||||
const policy = policyChecks(gitea, target, manifest);
|
||||
if (!policy.every((check) => check.ok)) return { ok: false, action: "platform-infra-gitea-apply", mode: "policy-blocked", mutation: false, policy };
|
||||
const frpcSecret = prepareGiteaFrpcSecret(gitea, target);
|
||||
const secrets = gitea.sourceAuthority.webhookSync.enabled && !options.dryRun ? ensureMirrorSecrets(gitea, true, true) : undefined;
|
||||
const targetRepos = repositoriesForTarget(gitea, target);
|
||||
const secrets = gitea.sourceAuthority.webhookSync.enabled && !options.dryRun
|
||||
? ensureMirrorSecrets(gitea, target, targetRepos, true, true, true)
|
||||
: undefined;
|
||||
const remote = remoteScript("apply", gitea, target, manifest, options, { frpcSecret, secrets });
|
||||
const result = await capture(config, target.route, ["sh"], remote.script);
|
||||
const parsed = parseJsonOutput(result.stdout);
|
||||
@@ -316,7 +320,7 @@ function mirrorPlan(options: CommonOptions): Record<string, unknown> {
|
||||
sourceAuthority: sourceAuthoritySummary(gitea, target),
|
||||
responsibilities: gitea.sourceAuthority.responsibilities.map(responsibilitySummary),
|
||||
repositories: repositoriesForTarget(gitea, target).map((repo) => repositorySummary(gitea, repo)),
|
||||
credentials: credentialSummaries(gitea),
|
||||
credentials: credentialSummaries(gitea, targetRepos),
|
||||
next: mirrorReadOnlyNextCommands(target.id),
|
||||
};
|
||||
}
|
||||
@@ -324,11 +328,26 @@ function mirrorPlan(options: CommonOptions): Record<string, unknown> {
|
||||
async function mirrorStatus(config: UniDeskConfig, options: CommonOptions & { repoKey?: string | null }): Promise<Record<string, unknown>> {
|
||||
const gitea = readGiteaConfig();
|
||||
const target = resolveCommandTarget(gitea, options.targetId);
|
||||
const selectedRepos = selectedRepositories(gitea, target, options.repoKey ?? null);
|
||||
const credentials = credentialSummaries(gitea, selectedRepos);
|
||||
const blockers = credentialBlockers(credentials);
|
||||
if (blockers.length > 0) {
|
||||
return {
|
||||
ok: false,
|
||||
action: "platform-infra-gitea-mirror-status",
|
||||
mutation: false,
|
||||
target: targetSummary(target),
|
||||
serviceHealth: { ok: false },
|
||||
sourceAuthority: { ...sourceAuthoritySummary(gitea, target), mirrorReady: false, stageReady: false },
|
||||
repositories: selectedRepos.map((repo) => repositorySummary(gitea, repo)),
|
||||
credentials,
|
||||
error: { code: "github-credential-unavailable", blockers, valuesPrinted: false },
|
||||
next: mirrorReadOnlyNextCommands(target.id),
|
||||
};
|
||||
}
|
||||
const health = await validate(config, options);
|
||||
const healthValidation = record(health.validation);
|
||||
const credentials = credentialSummaries(gitea);
|
||||
const secrets = ensureMirrorSecrets(gitea, false, false);
|
||||
const selectedRepos = selectedRepositories(gitea, target, options.repoKey ?? null);
|
||||
const secrets = ensureMirrorSecrets(gitea, target, selectedRepos, false, false, false);
|
||||
const result = await capture(config, target.route, ["sh"], remoteScript("mirror-status", gitea, target, "", { ...options, confirm: false, dryRun: true, wait: false }, { repos: selectedRepos, secrets }).script);
|
||||
const parsed = parseJsonOutput(result.stdout);
|
||||
const repositories = arrayRecords(record(parsed).repositories);
|
||||
@@ -362,10 +381,8 @@ async function mirrorBootstrap(config: UniDeskConfig, options: MirrorOptions): P
|
||||
const target = resolveCommandTarget(gitea, options.targetId);
|
||||
const repos = selectedRepositories(gitea, target, options.repoKey);
|
||||
if (!options.confirm) {
|
||||
const credentials = credentialSummaries(gitea);
|
||||
const blockers = credentials
|
||||
.filter((item) => item.id === "github-upstream" && (item.present !== true || item.requiredKeysPresent !== true))
|
||||
.map((item) => String(item.id));
|
||||
const credentials = credentialSummaries(gitea, repos);
|
||||
const blockers = credentialBlockers(credentials);
|
||||
const repoFlag = options.repoKey === null ? "" : ` --repo ${options.repoKey}`;
|
||||
return {
|
||||
ok: blockers.length === 0,
|
||||
@@ -382,7 +399,7 @@ async function mirrorBootstrap(config: UniDeskConfig, options: MirrorOptions): P
|
||||
},
|
||||
};
|
||||
}
|
||||
const secrets = ensureMirrorSecrets(gitea, true, true);
|
||||
const secrets = ensureMirrorSecrets(gitea, target, repos, false, true, true);
|
||||
const result = await capture(config, target.route, ["sh"], remoteScript("mirror-bootstrap", gitea, target, "", { ...options, dryRun: false, wait: false }, { repos, secrets }).script);
|
||||
const parsed = parseJsonOutput(result.stdout);
|
||||
return {
|
||||
@@ -392,7 +409,7 @@ async function mirrorBootstrap(config: UniDeskConfig, options: MirrorOptions): P
|
||||
mode: "confirmed",
|
||||
target: targetSummary(target),
|
||||
repositories: arrayRecords(record(parsed).repositories),
|
||||
credentials: credentialSummaries(gitea),
|
||||
credentials: credentialSummaries(gitea, repos),
|
||||
remote: parsed ?? compactCapture(result, { full: true }),
|
||||
next: mirrorReadOnlyNextCommands(target.id),
|
||||
};
|
||||
@@ -414,7 +431,7 @@ async function mirrorSync(config: UniDeskConfig, options: MirrorOptions): Promis
|
||||
}
|
||||
if (!options.confirm) return { ok: false, action: "platform-infra-gitea-mirror-sync", mutation: false, mode: "missing-confirm", error: "mirror sync requires --confirm" };
|
||||
const repos = selectedRepositories(gitea, target, options.repoKey);
|
||||
const secrets = ensureMirrorSecrets(gitea, false, false);
|
||||
const secrets = ensureMirrorSecrets(gitea, target, repos, false, false, false);
|
||||
const result = await capture(config, target.route, ["sh"], remoteScript("mirror-sync", gitea, target, "", { ...options, dryRun: false, wait: false }, { repos, secrets }).script);
|
||||
const parsed = parseJsonOutput(result.stdout);
|
||||
return {
|
||||
@@ -435,7 +452,7 @@ async function mirrorWebhookApply(config: UniDeskConfig, options: MirrorOptions)
|
||||
if (!gitea.sourceAuthority.webhookSync.enabled) return { ok: false, action: "platform-infra-gitea-mirror-webhook-apply", mutation: false, error: "sourceAuthority.webhookSync.enabled is false" };
|
||||
if (!options.confirm) return { ok: false, action: "platform-infra-gitea-mirror-webhook-apply", mutation: false, mode: "missing-confirm", error: "mirror webhook apply requires --confirm" };
|
||||
const repos = selectedRepositories(gitea, target, options.repoKey);
|
||||
const secrets = ensureMirrorSecrets(gitea, false, false);
|
||||
const secrets = ensureMirrorSecrets(gitea, target, repos, false, false, false);
|
||||
const result = await capture(config, target.route, ["sh"], remoteScript("mirror-webhook-apply", gitea, target, "", { ...options, dryRun: false, wait: false }, { repos, secrets }).script);
|
||||
const parsed = parseJsonOutput(result.stdout);
|
||||
return {
|
||||
@@ -454,7 +471,7 @@ async function mirrorWebhookStatus(config: UniDeskConfig, options: MirrorWebhook
|
||||
const gitea = readGiteaConfig();
|
||||
const target = resolveCommandTarget(gitea, options.targetId);
|
||||
const repos = selectedRepositories(gitea, target, options.repoKey);
|
||||
const secrets = ensureMirrorSecrets(gitea, false, false);
|
||||
const secrets = ensureMirrorSecrets(gitea, target, repos, false, false, false);
|
||||
const result = await capture(config, target.route, ["sh"], remoteScript("mirror-webhook-status", gitea, target, "", { ...options, confirm: false, dryRun: true, wait: false }, { repos, secrets }).script);
|
||||
const parsed = parseJsonOutput(result.stdout);
|
||||
const repositories = arrayRecords(record(parsed).repositories);
|
||||
@@ -473,7 +490,7 @@ async function mirrorWebhookStatus(config: UniDeskConfig, options: MirrorWebhook
|
||||
};
|
||||
}
|
||||
|
||||
function renderManifest(gitea: GiteaConfig, target: GiteaTarget): string {
|
||||
export function renderManifest(gitea: GiteaConfig, target: GiteaTarget): string {
|
||||
const app = gitea.app;
|
||||
const image = `${app.image.repository}:${app.image.tag}`;
|
||||
const labels = ` app.kubernetes.io/name: ${app.name}
|
||||
@@ -649,6 +666,8 @@ function renderGithubSyncManifest(gitea: GiteaConfig, target: GiteaTarget): stri
|
||||
const ownsHookReconcile = sync.hookReconcile.enabled && sync.hookReconcile.ownerTargetId.toLowerCase() === target.id.toLowerCase();
|
||||
const hookTopologyJson = ownsHookReconcile ? JSON.stringify(githubHookTopology(gitea), null, 2) : "";
|
||||
const hookReconcilerSource = ownsHookReconcile ? readFileSync(githubHookReconcilerFile, "utf8") : "";
|
||||
const bridgeTokenEnv = githubTokenSecretEnv(gitea, target, false);
|
||||
const hookTokenEnv = githubTokenSecretEnv(gitea, target, true);
|
||||
const hookReconcilerConfig = ownsHookReconcile ? ` hook-topology.json: |
|
||||
${indentBlock(hookTopologyJson, 4)}
|
||||
platform_infra_gitea_hook_reconciler.py: |
|
||||
@@ -670,11 +689,9 @@ ${indentBlock(hookReconcilerSource, 4)}
|
||||
env:
|
||||
- name: UNIDESK_GITEA_TARGET_ID
|
||||
value: ${yamlQuote(target.id)}
|
||||
- name: GITHUB_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ${sync.bridge.secretName}
|
||||
key: ${gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key}
|
||||
- name: UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV
|
||||
value: ${yamlQuote(githubTokenEnvName(gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key))}
|
||||
${hookTokenEnv}
|
||||
- name: GITHUB_WEBHOOK_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
@@ -818,11 +835,9 @@ spec:
|
||||
value: ${yamlQuote(String(sync.bridge.inbox.cleanupIntervalMs))}
|
||||
- name: UNIDESK_GITEA_WEBHOOK_SHUTDOWN_GRACE_MS
|
||||
value: ${yamlQuote(String(sync.bridge.shutdownGraceMs))}
|
||||
- name: GITHUB_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ${sync.bridge.secretName}
|
||||
key: ${gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key}
|
||||
- name: UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV
|
||||
value: ${yamlQuote(githubTokenEnvName(gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key))}
|
||||
${bridgeTokenEnv}
|
||||
- name: GITEA_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
@@ -984,11 +999,9 @@ ${hookRecoveryStateEnv} - name: UNIDESK_GITEA_WEBHOOK_INBOX_MAX_BYTES
|
||||
value: ${yamlQuote(String(sync.bridge.inbox.cleanupIntervalMs))}
|
||||
- name: UNIDESK_GITEA_WEBHOOK_SHUTDOWN_GRACE_MS
|
||||
value: ${yamlQuote(String(sync.bridge.shutdownGraceMs))}
|
||||
- name: GITHUB_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ${sync.bridge.secretName}
|
||||
key: ${gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key}
|
||||
- name: UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV
|
||||
value: ${yamlQuote(githubTokenEnvName(gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key))}
|
||||
${bridgeTokenEnv}
|
||||
- name: GITEA_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
@@ -1099,6 +1112,10 @@ function remoteScript(action: "apply" | "status" | "validate" | "mirror-bootstra
|
||||
UNIDESK_GITEA_GITHUB_PROXY_ENABLED: gitea.sourceAuthority.githubProxy.enabled ? "1" : "0",
|
||||
UNIDESK_GITEA_GITHUB_PROXY_URL: gitea.sourceAuthority.githubProxy.url,
|
||||
UNIDESK_GITEA_GITHUB_SECRET_KEY: gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key,
|
||||
UNIDESK_GITEA_GITHUB_DEFAULT_TOKEN_ENV: githubTokenEnvName(gitea.sourceAuthority.credentials.github.gitFetchCredential.secretRef.key),
|
||||
UNIDESK_GITEA_GITHUB_SECRET_ENV_MAP: githubCredentialsForConsumers(gitea, target, repositoriesForTarget(gitea, target), true)
|
||||
.map((credential) => `${credential.gitFetchCredential.secretRef.key}=${githubTokenEnvName(credential.gitFetchCredential.secretRef.key)}`)
|
||||
.join(","),
|
||||
UNIDESK_GITEA_NO_PROXY: gitea.sourceAuthority.githubProxy.noProxy.join(","),
|
||||
UNIDESK_GITEA_FRPC_ENABLED: frpcExposure === null ? "0" : "1",
|
||||
UNIDESK_GITEA_FRPC_DEPLOYMENT_NAME: frpcExposure?.frpc.deploymentName ?? gitea.app.publicExposure.frpc.deploymentName,
|
||||
@@ -1120,7 +1137,7 @@ function remoteScript(action: "apply" | "status" | "validate" | "mirror-bootstra
|
||||
if (params.secrets !== undefined) {
|
||||
env.UNIDESK_GITEA_ADMIN_USERNAME = params.secrets.adminUsername;
|
||||
env.UNIDESK_GITEA_ADMIN_PASSWORD = params.secrets.adminPassword;
|
||||
env.UNIDESK_GITEA_GITHUB_TOKEN = params.secrets.githubToken;
|
||||
for (const [key, token] of Object.entries(params.secrets.githubTokens)) env[githubTokenEnvName(key)] = token;
|
||||
env.UNIDESK_GITEA_GITHUB_WEBHOOK_SECRET = params.secrets.webhookSecret;
|
||||
}
|
||||
const exports = Object.entries(env).map(([key, value]) => `export ${key}=${shQuote(value)}`).join("\n");
|
||||
@@ -1445,7 +1462,14 @@ function githubHookTopology(gitea: GiteaConfig): Record<string, unknown> {
|
||||
repoKeys: [...item.repoKeys].sort(),
|
||||
branches: [...item.branches].sort(),
|
||||
}));
|
||||
return { repository, desiredUrls: desiredHooks.map((item) => item.url).sort(), desiredHooks };
|
||||
const repo = gitea.sourceAuthority.repositories.find((item) => item.upstream.repository === repository);
|
||||
const credential = repo === undefined ? gitea.sourceAuthority.credentials.github : githubCredentialForRepo(gitea, repo);
|
||||
return {
|
||||
repository,
|
||||
githubTokenEnv: githubTokenEnvName(credential.gitFetchCredential.secretRef.key),
|
||||
desiredUrls: desiredHooks.map((item) => item.url).sort(),
|
||||
desiredHooks,
|
||||
};
|
||||
}),
|
||||
};
|
||||
}
|
||||
@@ -1521,6 +1545,7 @@ function repositorySummary(gitea: GiteaConfig, repo: GiteaMirrorRepository): Rec
|
||||
}
|
||||
|
||||
function remoteRepoSpec(repo: GiteaMirrorRepository): Record<string, unknown> {
|
||||
const github = repo.credentialOverride?.github;
|
||||
return {
|
||||
key: repo.key,
|
||||
upstream: repo.upstream,
|
||||
@@ -1528,6 +1553,10 @@ function remoteRepoSpec(repo: GiteaMirrorRepository): Record<string, unknown> {
|
||||
gitops: repo.gitops,
|
||||
snapshot: repo.snapshot,
|
||||
legacyGitMirror: repo.legacyGitMirror,
|
||||
githubCredential: github === undefined ? null : {
|
||||
secretKey: github.gitFetchCredential.secretRef.key,
|
||||
tokenEnv: githubTokenEnvName(github.gitFetchCredential.secretRef.key),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1577,41 +1606,67 @@ function webhookSyncSummary(gitea: GiteaConfig, target: GiteaTarget): Record<str
|
||||
};
|
||||
}
|
||||
|
||||
function credentialSummaries(gitea: GiteaConfig): Array<Record<string, unknown>> {
|
||||
function credentialSummaries(gitea: GiteaConfig, repositories = gitea.sourceAuthority.repositories): Array<Record<string, unknown>> {
|
||||
const adminPath = credentialPath(gitea, gitea.sourceAuthority.credentials.admin.sourceRef);
|
||||
const github = gitea.sourceAuthority.credentials.github;
|
||||
const githubPath = credentialPath(gitea, github.sourceRef);
|
||||
const adminLinePair = existsSync(adminPath) ? parseLinePairCredential(adminPath, gitea.sourceAuthority.credentials.admin) : { username: null, password: null };
|
||||
const githubEnv = existsSync(githubPath) ? parseGithubCredentialFile(githubPath, github) : {};
|
||||
return [
|
||||
const summaries: Array<Record<string, unknown>> = [
|
||||
{
|
||||
id: "gitea-admin",
|
||||
format: gitea.sourceAuthority.credentials.admin.format,
|
||||
sourceRef: gitea.sourceAuthority.credentials.admin.sourceRef,
|
||||
sourcePath: adminPath,
|
||||
present: existsSync(adminPath),
|
||||
requiredKeysPresent: Boolean(adminLinePair.username) && Boolean(adminLinePair.password),
|
||||
fingerprint: fingerprintSecretParts([adminLinePair.username, adminLinePair.password]),
|
||||
permissionResult: { status: "not-applicable", error: null },
|
||||
valuesPrinted: false,
|
||||
},
|
||||
{
|
||||
id: "github-upstream",
|
||||
transport: github.transport,
|
||||
sourceRef: github.sourceRef,
|
||||
sourcePath: githubPath,
|
||||
present: existsSync(githubPath),
|
||||
requiredKeysPresent: Boolean(githubEnv[github.sourceKey]),
|
||||
fingerprint: fingerprintKeys(githubEnv, [github.sourceKey]),
|
||||
permissionResult: existsSync(githubPath) && Boolean(githubEnv[github.sourceKey])
|
||||
? { status: "not-probed", error: null }
|
||||
: { status: "blocked-missing-credential", error: "credential material is absent" },
|
||||
valuesPrinted: false,
|
||||
},
|
||||
];
|
||||
for (const repo of repositories) {
|
||||
const githubOverride = repo.credentialOverride?.github;
|
||||
if (githubOverride === undefined) continue;
|
||||
const sourcePath = credentialPath(gitea, githubOverride.sourceRef);
|
||||
const values = existsSync(sourcePath) ? parseGithubCredentialFile(sourcePath, githubOverride) : {};
|
||||
summaries.push({
|
||||
id: `github-upstream:${repo.key}`,
|
||||
repository: repo.upstream.repository,
|
||||
sourceRef: githubOverride.sourceRef,
|
||||
present: existsSync(sourcePath),
|
||||
requiredKeysPresent: Boolean(values[githubOverride.sourceKey]),
|
||||
fingerprint: fingerprintKeys(values, [githubOverride.sourceKey]),
|
||||
permissionResult: existsSync(sourcePath) && Boolean(values[githubOverride.sourceKey])
|
||||
? { status: "not-probed", permissions: githubOverride.permissions, error: null }
|
||||
: { status: "blocked-missing-credential", permissions: githubOverride.permissions, error: "credential material is absent" },
|
||||
valuesPrinted: false,
|
||||
});
|
||||
}
|
||||
return summaries;
|
||||
}
|
||||
|
||||
function credentialBlockers(credentials: Array<Record<string, unknown>>): string[] {
|
||||
return credentials.filter((item) => item.present !== true || item.requiredKeysPresent !== true).map((item) => String(item.id));
|
||||
}
|
||||
|
||||
function ensureMirrorSecrets(gitea: GiteaConfig, createAdmin: boolean, createWebhookSecret: boolean): MirrorSecrets {
|
||||
function ensureMirrorSecrets(
|
||||
gitea: GiteaConfig,
|
||||
target: GiteaTarget,
|
||||
repositories: GiteaMirrorRepository[],
|
||||
includeHookOwnerCredentials: boolean,
|
||||
createAdmin: boolean,
|
||||
createWebhookSecret: boolean,
|
||||
): MirrorSecrets {
|
||||
const admin = gitea.sourceAuthority.credentials.admin;
|
||||
const adminPath = credentialPath(gitea, admin.sourceRef);
|
||||
if (!existsSync(adminPath)) {
|
||||
@@ -1623,17 +1678,59 @@ function ensureMirrorSecrets(gitea: GiteaConfig, createAdmin: boolean, createWeb
|
||||
chmodSync(adminPath, 0o600);
|
||||
}
|
||||
const adminLinePair = parseLinePairCredential(adminPath, admin);
|
||||
const github = gitea.sourceAuthority.credentials.github;
|
||||
const githubPath = credentialPath(gitea, github.sourceRef);
|
||||
if (!existsSync(githubPath)) throw new Error(`${github.sourceRef} is missing; cannot sync GitHub upstream`);
|
||||
const githubEnv = parseGithubCredentialFile(githubPath, github);
|
||||
const githubTokens: Record<string, string> = {};
|
||||
for (const github of githubCredentialsForConsumers(gitea, target, repositories, includeHookOwnerCredentials)) {
|
||||
const githubPath = credentialPath(gitea, github.sourceRef);
|
||||
if (!existsSync(githubPath)) throw new Error(`${github.sourceRef} is missing; cannot sync GitHub upstream`);
|
||||
const githubEnv = parseGithubCredentialFile(githubPath, github);
|
||||
const githubToken = githubEnv[github.sourceKey];
|
||||
if (!githubToken) throw new Error(`${github.sourceRef} must contain ${github.sourceKey}`);
|
||||
githubTokens[github.gitFetchCredential.secretRef.key] = githubToken;
|
||||
}
|
||||
const webhookSecret = ensureWebhookSecret(gitea, createWebhookSecret);
|
||||
const adminUsername = adminLinePair.username;
|
||||
const adminPassword = adminLinePair.password;
|
||||
const githubToken = githubEnv[github.sourceKey];
|
||||
if (!adminUsername || !adminPassword) throw new Error(`${admin.sourceRef} must contain username on line ${admin.usernameLine} and password on line ${admin.passwordLine}`);
|
||||
if (!githubToken) throw new Error(`${github.sourceRef} must contain ${github.sourceKey}`);
|
||||
return { adminUsername, adminPassword, githubToken, webhookSecret };
|
||||
return { adminUsername, adminPassword, githubTokens, webhookSecret };
|
||||
}
|
||||
|
||||
function githubCredentialForRepo(gitea: GiteaConfig, repo: GiteaMirrorRepository): GiteaGithubCredential {
|
||||
return repo.credentialOverride?.github ?? gitea.sourceAuthority.credentials.github;
|
||||
}
|
||||
|
||||
function githubCredentialsForTarget(gitea: GiteaConfig, target: GiteaTarget): GiteaGithubCredential[] {
|
||||
const credentials = [gitea.sourceAuthority.credentials.github, ...repositoriesForTarget(gitea, target).flatMap((repo) => repo.credentialOverride?.github ?? [])];
|
||||
return [...new Map(credentials.map((credential) => [credential.gitFetchCredential.secretRef.key, credential])).values()];
|
||||
}
|
||||
|
||||
function githubCredentialsForConsumers(
|
||||
gitea: GiteaConfig,
|
||||
target: GiteaTarget,
|
||||
repositories: GiteaMirrorRepository[],
|
||||
includeHookOwnerCredentials: boolean,
|
||||
): GiteaGithubCredential[] {
|
||||
const credentials = [gitea.sourceAuthority.credentials.github, ...repositories.flatMap((repo) => repo.credentialOverride?.github ?? [])];
|
||||
if (includeHookOwnerCredentials
|
||||
&& gitea.sourceAuthority.webhookSync.hookReconcile.ownerTargetId.toLowerCase() === target.id.toLowerCase()) {
|
||||
credentials.push(...gitea.sourceAuthority.repositories.flatMap((repo) => {
|
||||
const github = repo.credentialOverride?.github;
|
||||
return github !== undefined && github.requiredFor.some((item) => item.startsWith("github-hooks-")) ? [github] : [];
|
||||
}));
|
||||
}
|
||||
return [...new Map(credentials.map((credential) => [credential.gitFetchCredential.secretRef.key, credential])).values()];
|
||||
}
|
||||
|
||||
function githubTokenEnvName(secretKey: string): string {
|
||||
return githubTokenEnvNameForSecretKey(secretKey);
|
||||
}
|
||||
|
||||
function githubTokenSecretEnv(gitea: GiteaConfig, target: GiteaTarget, includeHookOwnerCredentials: boolean): string {
|
||||
const secretName = targetWebhookSync(gitea, target).bridge.secretName;
|
||||
return githubCredentialsForConsumers(gitea, target, repositoriesForTarget(gitea, target), includeHookOwnerCredentials).map((credential) => ` - name: ${githubTokenEnvName(credential.gitFetchCredential.secretRef.key)}
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ${secretName}
|
||||
key: ${credential.gitFetchCredential.secretRef.key}`).join("\n");
|
||||
}
|
||||
|
||||
function ensureWebhookSecret(gitea: GiteaConfig, createIfMissing: boolean): string {
|
||||
|
||||
@@ -602,7 +602,7 @@ def reconcile_repository(topology, repository_spec, token, secret, force_patch):
|
||||
return observation, observed
|
||||
|
||||
|
||||
def reconcile_once(topology, token, secret, force_patch):
|
||||
def reconcile_once(topology, secret, force_patch):
|
||||
started = time.monotonic()
|
||||
topology_ready = True
|
||||
recovery_ready = True
|
||||
@@ -624,6 +624,7 @@ def reconcile_once(topology, token, secret, force_patch):
|
||||
emit("github-hook-delivery-recovery-ledger-unavailable", ok=False, errorType=type(error).__name__, error=str(error)[:160])
|
||||
for repository_spec in topology["repositories"]:
|
||||
try:
|
||||
token = os.environ[repository_spec["githubTokenEnv"]]
|
||||
observation, observed_hooks = reconcile_repository(topology, repository_spec, token, secret, force_patch)
|
||||
if not observation["ready"]:
|
||||
topology_ready = False
|
||||
@@ -675,13 +676,12 @@ def main():
|
||||
topology = load_topology(sys.argv[1])
|
||||
if topology.get("ownerTargetId") != os.environ.get("UNIDESK_GITEA_TARGET_ID"):
|
||||
raise SystemExit("hook reconciler target is not the YAML owner")
|
||||
token = os.environ["GITHUB_TOKEN"]
|
||||
secret = os.environ["GITHUB_WEBHOOK_SECRET"]
|
||||
signal.signal(signal.SIGTERM, stop)
|
||||
signal.signal(signal.SIGINT, stop)
|
||||
force_patch = True
|
||||
while not STOP.is_set():
|
||||
if reconcile_once(topology, token, secret, force_patch):
|
||||
if reconcile_once(topology, secret, force_patch):
|
||||
force_patch = False
|
||||
STOP.wait(topology["reconcile"]["intervalMs"] / 1000)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user