mirrors/pikasTech-unidesk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add agentrun git mirror controls

Browse Source
This commit is contained in:
Codex
2026-06-01 17:36:55 +00:00
parent 79933f3c58
commit 9027a310d3
4 changed files with 515 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/cli.md
+3 -2
View File
@@ -43,7 +43,8 @@ CI/CD、GitOps、rollout、artifact 发布、PR 合并后的 DEV/PROD 滚动、P
- `artifact-registry plan|render|status|health|install|deploy-backend-core|deploy-service` 管理 D601 host-managed CNCF Distribution registry 的声明、安装、只读检查和 pull-only artifact CD。该 registry 固定为 D601 loopback `127.0.0.1:5000`，由 systemd + Docker Compose 管理，位于 native k3s 故障域外；`deploy-service` 只拉取 CI 已发布的 commit-pinned 镜像、retag/recreate 或导入 native k3s，并做 live commit 验证，不构建 runtime source。`deploy-backend-core` 是 deprecated 兼容名，标准 backend-core prod CD 入口是 `deploy apply --env prod --service backend-core`。长期规则见 `docs/reference/artifact-registry.md`。
- `commander contract|plan --dry-run|smoke --dry-run|approval request --dry-run|prompt-lint --kind gpt55-pr` 是 host Codex 指挥官直管微服务 skeleton 入口。当前命令返回 `phase=source-contract`、service/API/state/bridge/prompt/trace/#20/#46/ClaudeQQ 审批边界、.state/commander/ 状态模型、dev 无 daemon smoke contract、dry-run 计划和 GPT-5.5 PR prompt 边界辅助 lint，不接 live bridge、不注入 prompt、不发送 ClaudeQQ。`approval request --dry-run` 会生成 200 字以内中文纯文本 ClaudeQQ 审批草案、`notification-path-unavailable` blocker 和授权后唯一可用的 `bun scripts/cli.ts microservice proxy claudeqq /api/push/text --method POST --body-json '<payload>' --raw` 命令；不得提示使用本机 ClaudeQQ skill、powershell 或本地 server。`prompt-lint` 支持 `--prompt-file` 与 `--stdin`，输出 `ok`、`missingClauses`、`riskLevel`、`suggestedPatchSnippet` 且不回显完整 prompt；它是 commander 辅助检查，不是业务 PR 门禁，也不改变 `codex submit` 默认行为。`plan`、`smoke` 与 `approval request` 必须带 `--dry-run`；缺少时返回 `error=dry-run-required`。长期规则见 `docs/reference/host-codex-commander.md`。
- `hwlab g14 monitor-prs [--once] [--dry-run] [--interval-seconds N] [--max-cycles N] [--timeout-seconds N]` 是当前 HWLAB G14 PR -> CI/CD -> DEV rollout 的一行式入口。普通调用创建 `.state/jobs/` 异步 job 并立刻返回 `job.id`、`statusCommand` 和 stdout/stderr 路径；后台 worker 每轮通过 UniDesk `gh pr list/preflight/merge` 监控 `pikasTech/HWLAB` base=`G14` 的 open PR，ready 时合并，然后通过 UniDesk `ssh G14:k3s` 观察 `hwlab-g14-ci-poll-<short>`、Argo `hwlab-g14-dev` 和 DEV `/health/live`，直到 DEV `Synced/Healthy` 且 Deployment/StatefulSet ready；历史 `Completed` smoke/debug pod 不作为 rollout blocker。每次成功 DEV rollout 后，worker 会定位或创建 #7“指挥简报索引”中的北京日期每日简报 issue，并追加 CI/CD 耗时、CI/CD 关键指标、语义化上线 changelog、自动 diff 摘要、PipelineRun、GitOps revision 和 DEV 验证摘要；关键指标来自 G14 Tekton TaskRun results，固定包含 `lazy build reused: x/y`、reused services、rebuild services 和每个 service 的独立耗时/状态/backend，用于观察 lazy build 机制效果。语义化 changelog 优先从 PR body 的 `## 修改`/`## 变更`/`## Changelog` 等段落提取，diff 摘要只作为文件和统计证据保留，不替代 changelog。也可用 `hwlab g14 record-rollout --pr <number> --source-commit <sha>` 手动补记，手动补记同样会按 PipelineRun 采集 TaskRun 指标。状态指针按用途分离：长期监控只写 `.state/hwlab-g14/latest-monitor-job.json`，`--once` 写 `latest-once-job.json`，`--dry-run` 写 `latest-dry-run-job.json`，`--once --dry-run` 写 `latest-once-dry-run-job.json`，避免一次性收口覆盖持续监控入口。`--once --dry-run` 只做单轮监控和 merge plan，不写 GitHub、不等待 rollout。该命令禁止使用原生 `gh` 或手拼 GitHub 请求；如果 UniDesk `gh` 子命令字段或行为不够，必须先改进 `scripts/src/gh.ts` 后再使用。
- `agentrun v01 control-plane status|trigger-current|refresh [--dry-run|--confirm]` 是 AgentRun `v0.1` 在 G14 k3s 的受控 Tekton/Argo 入口。`status` 只读汇总固定 source worktree commit、对应 commit-pinned PipelineRun、GitOps latest、Argo Application 和 `agentrun-v01` workload，并报告 Argo revision 是否对齐 `v0.1-gitops` latest；`trigger-current` 先快进 `G14:/root/agentrun-v01` 到 `origin/v0.1`，再创建 `agentrun-v01-ci-<short12>` PipelineRun。confirmed trigger 只提交 CI/CD 工作并返回后续 `status` 命令，不等待完整 PipelineRun；同名 PipelineRun 运行中或已成功时拒绝重复触发，只允许失败态重建或首次创建。`refresh` 只对 `argocd/agentrun-g14-v01` 执行 hard refresh，用于 GitOps promotion 已完成但 Argo 仍停留旧 revision 时的受控同步入口；它不直接 patch runtime workload。AgentRun 运行时和 SPEC 事实来源仍在 AgentRun 仓库，UniDesk 只维护受控运维入口。
- `agentrun v01 control-plane status|trigger-current|refresh [--dry-run|--confirm]` 是 AgentRun `v0.1` 在 G14 k3s 的受控 Tekton/Argo 入口。`status` 只读汇总固定 source worktree commit、对应 commit-pinned PipelineRun、GitOps latest、Argo Application、`agentrun-v01` workload、`planArtifacts.summary`、env image result 和 git mirror 摘要，并报告 Argo revision 是否对齐 `v0.1-gitops` latest；`trigger-current` 先快进 `G14:/root/agentrun-v01` 到 `origin/v0.1`，检查 `devops-infra` mirror 的 `localV01` 是否等于目标 source commit，必要时先执行受控 mirror sync，再创建 `agentrun-v01-ci-<short12>` PipelineRun。confirmed trigger 只提交 CI/CD 工作并返回后续 `status` 命令，不等待完整 PipelineRun；同名 PipelineRun 运行中或已成功时拒绝重复触发，只允许失败态重建或首次创建。`refresh` 只对 `argocd/agentrun-g14-v01` 执行 hard refresh，用于 GitOps promotion 已完成但 Argo 仍停留旧 revision 时的受控同步入口；它不直接 patch runtime workload。AgentRun 运行时和 SPEC 事实来源仍在 AgentRun 仓库，UniDesk 只维护受控运维入口。
- `agentrun v01 git-mirror status|sync|flush [--dry-run|--confirm]` 是 AgentRun `v0.1` 使用 `devops-infra` git mirror/relay 的受控维护入口。`status` 返回 read/write URL、`localV01`、`githubV01`、`localGitops`、`githubGitops`、`pendingFlush`、`githubInSync` 和 exact full-SHA shallow fetch 结果；`sync` 创建 manual Job，把 GitHub `v0.1` 和 `v0.1-gitops` refs 拉入 `/cache/pikasTech/agentrun.git`；`flush` 把本地 `v0.1-gitops` 快进推回 GitHub。confirmed `sync`/`flush` 默认创建 `.state/jobs/` 异步 job 并立刻返回 `job.id`、`statusCommand` 和日志路径；只有现场同步调试才显式加 `--wait`。该入口与 HWLAB v0.2 mirror 共用 `devops-infra` 服务和 cache PVC，但 repo path、refs、status 文件和 CLI 命令彼此独立。
- `hwlab g14 control-plane status|apply --lane v02 [--dry-run|--confirm]` 是 HWLAB `v0.2` 加法 lane 的受控 Tekton/Argo 控制面维护入口，source commit 只来自 G14 专用 bare repo `/root/hwlab-v02-cicd.git` 的 `refs/remotes/origin/v0.2`；`/root/hwlab-v02` 只作为人工开发和短连接源码工具 workspace 被观测，dirty/stale 状态必须输出为 isolated warning 而不能阻塞 CI/CD。该入口面向 branch `v0.2`、namespace `hwlab-ci` 和 Argo application `hwlab-g14-v02`；默认 `status` 只读汇总最新 source head 的 pipeline、RBAC/ServiceAccount、Argo、当前 commit PipelineRun、当前 PipelineRun 的 TaskRun 条件摘要、最近 PipelineRun 摘要、活跃 PipelineRun、遗留 v02 CronJob 清理状态、commit alignment，以及 19666/19667 的 Cloud Web 静态资源和 API live 探针。分支被后续提交推进后，要复查已完成 run 时使用 `status --lane v02 --pipeline-run hwlab-v02-ci-poll-<short-sha>`；已知完整 source SHA 但不想依赖最新 head 时使用 `status --lane v02 --source-commit <full-sha>`。定点 `status` 输出 `statusTarget.mode`，只检查指定 PipelineRun/source commit 的证据，不因为 `origin/v0.2` 后续推进而把历史 run 判为失败；默认不带定点参数时仍严格判定最新 source head alignment。TaskRun 摘要的 `performance` 字段会把超过 120s 的 build TaskRun 标为慢任务、超过 180s 标为 critical warning，用于暴露 env reuse/git mirror 命中率回归，但不作为阻断门禁；CI/CD 性能验收应同时看 `planArtifacts.summary`、`taskRuns.performance.warningCount` 和 PipelineRun duration，纯 CLI/文档或无 runtime 重建需求的后续提交应稳定表现为 `build=0 reuse=<service-count>` 且无 build TaskRun warning，首次引入或切换 env image 时允许只构建必要 env image 一次。`webAssets` 必须直接给出 `readonly-rpc` 删除、sidebar/workspace/event panel 关键 CSS、`/app.js` 是否可读取和字节数、`/health/live` 与 API revision；`apiRevision` 是 cloud-api 服务自身 revision，Cloud Web 静态资源变更时允许它与 source commit 不同，不能把这种差异误判成 Cloud Web 未发布。默认只读取必要字段，禁止把完整 PipelineRun spec、Tekton 内联脚本、历史大对象或整份 CSS/HTML/JS 展开到默认输出；`apply` 先自动 fetch `/root/hwlab-v02-cicd.git` 并从 commit-pinned detached worktree 执行 render check，再经 `G14:k3s` server-side apply `tekton-v02/rbac.yaml`、`pipeline.yaml`、`argocd/project.yaml` 和 `argocd/application-v02.yaml`，confirmed apply 会删除遗留 v02 CronJob，但不会应用 runtime-v02 workload、Secret 或数据迁移。
- `hwlab g14 control-plane trigger-current --lane v02 [--dry-run|--confirm]` 是 v02 标准手动触发入口：先自动 fetch `/root/hwlab-v02-cicd.git`，解析当前 `origin/v0.2` full SHA，创建 commit-pinned `hwlab-v02-ci-poll-<short12>` PipelineRun；读 Git 走 `git-mirror-http.devops-infra.svc.cluster.local`，GitOps promotion 写 `git-mirror-write.devops-infra.svc.cluster.local`；confirmed trigger 在删除/创建 PipelineRun 前会先按当前 source commit 在 G14 临时 detached worktree 中 render，再 server-side apply v02 Tekton RBAC、Pipeline 与 Argo Application，避免 CI/CD 脚本或 runtime-ready 逻辑已合并但集群仍执行旧 Pipeline 定义；该 render 不要求固定 `/root/hwlab-v02` 工作树 clean，也不得因 `.worktree/` 或其他并行未提交修改阻塞；同名 PipelineRun 成功或运行中时拒绝重复触发，失败或不存在时才删除旧对象并重新创建。
创建 PipelineRun 前会读取 `devops-infra` mirror refs，若 `localV02` 未等于当前 source commit，则自动执行一次受控 manual `git-mirror sync` Job 并复核 ref，复核失败时停止触发，避免 Tekton `prepare-source` 已知失败；services 参数只包含 v02 runtime service matrix，`hwlab-cli` 是固定 repo 短连接源码工具，不进入 PipelineRun service build。
@@ -90,7 +91,7 @@ CI/CD、GitOps、rollout、artifact 发布、PR 合并后的 DEV/PROD 滚动、P
- `codex interrupt|cancel <taskId>` 通过 Code Queue 私有代理请求中断；running/judging 任务会请求 D601 当前 agent run 停止，queued/retry_wait 任务的取消也必须保持与 WebUI 相同代理路径，返回有界 task 摘要和后续查询命令。任何需要接触 active run 的动作仍属于 D601 执行面。
- Code Queue 多队列 lane 由 `codex` 命令命名空间管理：`queues [--full|--all] [--limit N] [--page N|--offset N]` 列表、`queue create <queueId>` 创建、`queue merge <sourceQueueId> --into <targetQueueId>` 合并、`move <taskId> --queue <queueId>` 迁移；这些队列管理入口默认由主 server `code-queue-mgr` 直管 PostgreSQL，仍通过稳定 `code-queue` 用户服务代理路径访问。`codex queues` 默认只返回 active/nonempty/unread/runnable queue 摘要、activity、commanderConcurrency、全局 counts 和 execution diagnostics；`--full` 或 `--all` 只切换为完整队列行视图的一页，仍受 `--limit`/`--page`/`--offset` 分页约束，不再默认携带 deprecated full array。summary 和 full 的稳定机读路径都是 `.data.queues.items[]`，全局元数据固定在 `.data.queues.commanderConcurrency`、`.data.queues.activity`、`.data.queues.counts`、`.data.queues.executionDiagnostics`、`.data.queues.activeTaskIds` 和 `.data.queues.queuedTaskIds`；需要完整 upstream 时使用输出中的 raw command。`commanderConcurrency.activeRunnerCount` / `activity.effectiveActiveTaskCount` 是指挥官并发判断的有效活跃数，`schedulerLocalActiveQueueCount`/`activeQueueIds` 只描述本地 scheduler active-run slots，不能覆盖数据库 running 计数或 heartbeat-fresh runner 计数。旧 full 顶层数组语义已作为 deprecated 兼容信息记录，不再作为 `.data.queues` 主形态。同一个 queue 内部串行执行，不同 queue 之间并行执行。迁移只允许尚未被 scheduler claim 的 `queued`/`retry_wait` 任务，必须满足 `startedAt=null`、`currentAttempt=0` 且没有 active thread/turn；已进入 `running`/`judging` 或已有 claim 标记的任务返回 409，不得被 move/merge 回写成 queued。合并会移动可迁移任务归属并自动删除源 queue 记录，只保留合并后的目标 queue；若 source 或 target queue 存在 active/claimed 任务，合并整体返回 409。合并后的目标 queue 按任务原 `queueEnteredAt`/`createdAt` 时间顺序串行，成功迁移 queued/retry_wait 任务后由 D601 scheduler 轮询推进。
- 所有 `codex` 查询和管理命令必须走与 WebUI 相同的 backend-core 私有代理路径 `/api/microservices/code-queue/proxy/...`；CLI 不得为了提交、移动、中断、取消或队列管理直接调用 D601 内部 Service、数据库、pod curl 或 k3sctl scheduler 子服务。若该路径失败，应先修复 CLI/backend/provider tunnel 链路，而不是绕过控制面。
- `job list [--limit N] [--include-command]` 与 `job status <jobId|latest> [--tail-bytes N]` 查询 `.state/jobs/` 文件系统状态，是异步命令的可观测入口。`job list` 默认只返回最新 50 条摘要，并为已知异步工作流返回轻量 `progress.summary` 与后续查询命令；`job status` 默认返回结构化 `progress`、stdout/stderr 末尾 12000 字节、`tailPolicy` 与完整日志路径。已知工作流应从有界日志尾部抽取阶段、关键对象名和下一步命令，避免为了判断当前阶段而手工打开完整 stdout/stderr。`hwlab_g14_v02_trigger_current` 的 progress 必须暴露 trigger 阶段、source commit 和 PipelineRun；`hwlab_g14_git_mirror_sync|flush` 的 progress 必须暴露 sync/flush 状态、Job 名、pendingFlush 与 fetch/push/total/SSH timing。
- `job list [--limit N] [--include-command]` 与 `job status <jobId|latest> [--tail-bytes N]` 查询 `.state/jobs/` 文件系统状态，是异步命令的可观测入口。`job list` 默认只返回最新 50 条摘要，并为已知异步工作流返回轻量 `progress.summary` 与后续查询命令；`job status` 默认返回结构化 `progress`、stdout/stderr 末尾 12000 字节、`tailPolicy` 与完整日志路径。已知工作流应从有界日志尾部抽取阶段、关键对象名和下一步命令，避免为了判断当前阶段而手工打开完整 stdout/stderr。`hwlab_g14_v02_trigger_current` 的 progress 必须暴露 trigger 阶段、source commit 和 PipelineRun；`hwlab_g14_git_mirror_sync|flush` 与 `agentrun_v01_git_mirror_sync|flush` 的 progress 必须暴露 sync/flush 状态、Job 名、pendingFlush 与 fetch/push/total/SSH timing，并给出对应 repo 的 mirror status 命令。
- `debug health`、`debug dispatch` 与 `debug task` 走真实内部 core、WebSocket、数据库、provider、系统指标、Docker 状态和 Host SSH 维护桥流程，只用于开发调试，不写入 `TEST.md` 的正式验收步骤。
- `e2e run [--only pattern[,pattern...]] [--skip pattern[,pattern...]]` 使用 publicHost 派生的公开 production frontend/dev frontend/provider ingress URL，并通过 Docker 内网验证 core API、PostgreSQL、provider self-connection、系统指标曲线、Docker 状态快照、provider.upgrade 预检和 Playwright 前端页面，是交付前的自动化 E2E 门禁；CLI 默认输出 check 状态摘要，完整诊断写入 `resultPath`，日常迭代应优先用 `--only` / `--skip` 跑最小必要集合。
scripts/src/agentrun.ts
+500 -8
View File
@@ -1,5 +1,6 @@
import type { UniDeskConfig } from "./config";
import { runSshCommandCapture, type SshCaptureResult } from "./ssh";
import { startJob } from "./jobs";
const g14SourceRoute = "G14:/root/agentrun-v01";
const g14K3sRoute = "G14:k3s";
@@ -10,10 +11,18 @@ const pipelineName = "agentrun-v01-ci-image-publish";
const argoNamespace = "argocd";
const argoApplication = "agentrun-g14-v01";
const gitopsBranch = "v0.1-gitops";
const gitMirrorNamespace = "devops-infra";
const gitMirrorReadUrl = "http://git-mirror-http.devops-infra.svc.cluster.local/pikasTech/agentrun.git";
const gitMirrorWriteUrl = "http://git-mirror-write.devops-infra.svc.cluster.local/pikasTech/agentrun.git";
const gitMirrorRepoPath = "/cache/pikasTech/agentrun.git";
const gitMirrorSyncJobPrefix = "git-mirror-agentrun-sync-manual";
const gitMirrorFlushJobPrefix = "git-mirror-agentrun-flush-manual";
const githubSshUrl = "ssh://git@ssh.github.com:443/pikasTech/agentrun.git";
const mirrorToolsImage = "127.0.0.1:5000/hwlab/hwlab-ci-node-tools:node22-alpine-bun-v1";
export function agentRunHelp(): unknown {
return {
command: "agentrun v01 control-plane status|trigger-current|refresh",
command: "agentrun v01 control-plane status|trigger-current|refresh | git-mirror status|sync|flush",
output: "json",
usage: [
"bun scripts/cli.ts agentrun v01 control-plane status",
@@ -21,17 +30,30 @@ export function agentRunHelp(): unknown {
"bun scripts/cli.ts agentrun v01 control-plane trigger-current --confirm",
"bun scripts/cli.ts agentrun v01 control-plane refresh --dry-run",
"bun scripts/cli.ts agentrun v01 control-plane refresh --confirm",
"bun scripts/cli.ts agentrun v01 git-mirror status",
"bun scripts/cli.ts agentrun v01 git-mirror sync --confirm",
"bun scripts/cli.ts agentrun v01 git-mirror flush --confirm",
],
description: "Operate AgentRun v0.1 Tekton/Argo control plane through G14 routes; trigger-current is short-return and status is read-only.",
description: "Operate AgentRun v0.1 Tekton/Argo control plane and devops-infra git mirror through G14 routes; status is read-only and trigger-current pre-syncs mirror refs before creating the PipelineRun.",
};
}
export async function runAgentRunCommand(config: UniDeskConfig, args: string[]): Promise<Record<string, unknown>> {
const [lane, group, action] = args;
if (lane !== "v01" || group !== "control-plane") return unsupported(args);
if (action === "status") return await status(config);
if (action === "trigger-current") return await triggerCurrent(config, parseTriggerOptions(args.slice(3)));
if (action === "refresh") return await refresh(config, parseConfirmOptions(args.slice(3)));
if (lane !== "v01") return unsupported(args);
if (group === "control-plane") {
if (action === "status") return await status(config);
if (action === "trigger-current") return await triggerCurrent(config, parseTriggerOptions(args.slice(3)));
if (action === "refresh") return await refresh(config, parseConfirmOptions(args.slice(3)));
}
if (group === "git-mirror") {
if (action === "status") return await gitMirrorStatus(config);
if (action === "sync" || action === "flush") {
const options = parseGitMirrorOptions(args.slice(3));
if (options.confirm && !options.wait) return startAsyncAgentRunJob(`agentrun_v01_git_mirror_${action}`, ["bun", "scripts/cli.ts", "agentrun", "v01", "git-mirror", action, "--confirm", "--wait", "--timeout-seconds", String(options.timeoutSeconds)], `Run AgentRun v0.1 git mirror ${action} on G14`);
return await runGitMirrorJob(config, action, options);
}
}
return unsupported(args);
}
@@ -45,6 +67,11 @@ interface ConfirmOptions {
dryRun: boolean;
}
interface GitMirrorOptions extends ConfirmOptions {
timeoutSeconds: number;
wait: boolean;
}
function parseTriggerOptions(args: string[]): TriggerOptions {
return parseConfirmOptions(args);
}
@@ -57,6 +84,14 @@ function parseConfirmOptions(args: string[]): ConfirmOptions {
};
}
function parseGitMirrorOptions(args: string[]): GitMirrorOptions {
const base = parseConfirmOptions(args);
const timeoutIndex = args.indexOf("--timeout-seconds");
const timeoutSeconds = timeoutIndex >= 0 ? Number(args[timeoutIndex + 1]) : 300;
if (!Number.isFinite(timeoutSeconds) || timeoutSeconds < 30) throw new Error("--timeout-seconds must be a number >= 30");
return { ...base, timeoutSeconds, wait: args.includes("--wait") };
}
async function status(config: UniDeskConfig): Promise<Record<string, unknown>> {
const source = await capture(config, g14SourceRoute, ["script", "--", [
"cd /root/agentrun-v01",
@@ -76,8 +111,10 @@ async function status(config: UniDeskConfig): Promise<Record<string, unknown>> {
const pipelineRun = sourceCommit ? pipelineRunName(sourceCommit) : null;
const k3s = await capture(config, g14K3sRoute, ["script", "--", statusScript(pipelineRun)]);
const argo = parseArgoStatus(k3s.stdout);
const mirror = await gitMirrorStatus(config);
const ciSummary = labeledJson(k3s.stdout, "ciSummary");
return {
ok: source.exitCode === 0 && k3s.exitCode === 0,
ok: source.exitCode === 0 && k3s.exitCode === 0 && mirror.ok === true,
command: "agentrun v01 control-plane status",
lane: "v0.1",
sourceCommit,
@@ -88,6 +125,13 @@ async function status(config: UniDeskConfig): Promise<Record<string, unknown>> {
expectedPipelineRun: pipelineRun,
source: compactCapture(source),
runtime: compactCapture(k3s),
ciSummary,
gitMirror: {
ok: mirror.ok,
readUrl: mirror.readUrl,
writeUrl: mirror.writeUrl,
summary: mirror.summary,
},
runtimeAlignment: {
localHeadMatchesOrigin: Boolean(localSourceCommit && originSourceCommit && localSourceCommit === originSourceCommit),
argoRevision: argo.revision,
@@ -127,22 +171,55 @@ async function triggerCurrent(config: UniDeskConfig, options: TriggerOptions): P
namespace: ciNamespace,
pipeline: pipelineName,
runtimeNamespace,
gitMirror: {
readUrl: gitMirrorReadUrl,
writeUrl: gitMirrorWriteUrl,
},
};
const mirrorBefore = await gitMirrorStatus(config);
const mirrorRequirement = gitMirrorSyncRequirement(sourceCommit, String(mirrorBefore.raw ?? ""));
if (options.dryRun || !options.confirm) {
return {
ok: true,
command: "agentrun v01 control-plane trigger-current",
dryRun: true,
plan,
gitMirrorPreSync: {
required: mirrorRequirement.required,
reason: mirrorRequirement.reason,
before: mirrorBefore.summary,
},
next: { confirm: "bun scripts/cli.ts agentrun v01 control-plane trigger-current --confirm" },
};
}
let gitMirrorPreSync: Record<string, unknown> = {
required: mirrorRequirement.required,
reason: mirrorRequirement.reason,
before: mirrorBefore.summary,
};
if (mirrorRequirement.required) {
const synced = await runGitMirrorJob(config, "sync", { confirm: true, dryRun: false, timeoutSeconds: 300, wait: true });
const after = await gitMirrorStatus(config);
const afterRequirement = gitMirrorSyncRequirement(sourceCommit, String(after.raw ?? ""));
gitMirrorPreSync = { ...gitMirrorPreSync, sync: synced, after: after.summary, ok: synced.ok === true && afterRequirement.required === false };
if (synced.ok !== true || afterRequirement.required !== false) {
return {
ok: false,
command: "agentrun v01 control-plane trigger-current",
dryRun: false,
degradedReason: "git-mirror-local-v01-not-current-after-sync",
plan,
gitMirrorPreSync,
};
}
}
const created = await capture(config, g14K3sRoute, ["script", "--", triggerScript(sourceCommit, pipelineRun)]);
return {
ok: created.exitCode === 0,
command: "agentrun v01 control-plane trigger-current",
dryRun: false,
plan,
gitMirrorPreSync,
created: compactCapture(created),
next: {
status: "bun scripts/cli.ts agentrun v01 control-plane status",
@@ -204,6 +281,31 @@ function statusScript(pipelineRun: string | null): string {
: "true",
"printf 'recentPipelineRuns\\n'",
`kubectl -n ${ciNamespace} get pipelinerun --sort-by=.metadata.creationTimestamp -o 'custom-columns=NAME:.metadata.name,STATUS:.status.conditions[0].status,REASON:.status.conditions[0].reason,CREATED:.metadata.creationTimestamp' --no-headers 2>/dev/null | tail -n 5 || true`,
"printf 'ciSummary='",
pr.length > 0
? [
`if kubectl -n ${ciNamespace} get taskrun -l tekton.dev/pipelineRun=${shQuote(pr)} -o json >/tmp/agentrun-v01-taskruns.json 2>/dev/null; then`,
"node <<'NODE'",
"const fs = require('node:fs');",
"const data = JSON.parse(fs.readFileSync('/tmp/agentrun-v01-taskruns.json', 'utf8'));",
"const items = Array.isArray(data.items) ? data.items : [];",
"function task(item) { return item?.metadata?.labels?.['tekton.dev/pipelineTask'] || item?.metadata?.name || null; }",
"function result(item, name) { return (item?.status?.results || item?.status?.taskResults || []).find((entry) => entry.name === name)?.value ?? null; }",
"function seconds(start, end) { const s = Date.parse(start || ''); const e = Date.parse(end || ''); return Number.isFinite(s) && Number.isFinite(e) ? Math.round((e - s) / 1000) : null; }",
"const taskRuns = items.map((item) => ({ name: item?.metadata?.name || null, task: task(item), status: item?.status?.conditions?.[0]?.status || null, reason: item?.status?.conditions?.[0]?.reason || null, seconds: seconds(item?.status?.startTime, item?.status?.completionTime) }));",
"const plan = items.find((item) => task(item) === 'plan-artifacts');",
"const image = items.find((item) => task(item) === 'image-publish');",
"const imageSeconds = taskRuns.find((item) => item.task === 'image-publish')?.seconds ?? null;",
"console.log(JSON.stringify({",
" planArtifacts: { summary: plan ? result(plan, 'summary') : null, envIdentity: plan ? result(plan, 'env-identity') : null, buildCount: plan ? Number(result(plan, 'build-count') || 0) : null, reuseCount: plan ? Number(result(plan, 'reuse-count') || 0) : null },",
" envImage: { status: image ? result(image, 'status') : null, envIdentity: image ? result(image, 'env-identity') : null, image: image ? result(image, 'image') : null, digest: image ? result(image, 'digest') : null, repositoryDigest: image ? result(image, 'repository-digest') : null },",
" taskRuns,",
" performance: { imagePublishSeconds: imageSeconds, buildWarning: typeof imageSeconds === 'number' && imageSeconds > 120, criticalWarning: typeof imageSeconds === 'number' && imageSeconds > 180 }",
"}));",
"NODE",
"else printf '{}\\n'; fi",
].join("\n")
: "printf '{}\\n'",
"printf 'argo\\n'",
`kubectl -n ${argoNamespace} get application ${argoApplication} -o 'jsonpath={.status.sync.revision}{"\\t"}{.status.sync.status}{"\\t"}{.status.health.status}{"\\n"}' 2>/dev/null || true`,
"printf 'workloads\\n'",
@@ -266,6 +368,10 @@ function triggerScript(sourceCommit: string, pipelineRun: string): string {
" params:",
" - name: revision",
` value: ${sourceCommit}`,
" - name: git-read-url",
` value: ${gitMirrorReadUrl}`,
" - name: git-write-url",
` value: ${gitMirrorWriteUrl}`,
" workspaces:",
" - name: source",
" volumeClaimTemplate:",
@@ -282,6 +388,365 @@ function triggerScript(sourceCommit: string, pipelineRun: string): string {
].join("\n");
}
async function gitMirrorStatus(config: UniDeskConfig): Promise<Record<string, unknown>> {
const script = [
"set +e",
"printf 'resources\\n'",
`kubectl -n ${gitMirrorNamespace} get deploy,svc,pvc,cm -l app.kubernetes.io/name=git-mirror -o name 2>/dev/null || true`,
"printf 'jobs\\n'",
`kubectl -n ${gitMirrorNamespace} get job -l app.kubernetes.io/name=git-mirror --sort-by=.metadata.creationTimestamp -o 'custom-columns=NAME:.metadata.name,SUCCEEDED:.status.succeeded,FAILED:.status.failed,START:.status.startTime,COMPLETION:.status.completionTime' --no-headers 2>/dev/null | tail -n 8 || true`,
"printf 'cache\\n'",
`kubectl exec -n ${gitMirrorNamespace} deploy/git-mirror-http -- sh -lc ${shQuote(gitMirrorCacheProbeScript())}`,
].join("\n");
const result = await capture(config, g14K3sRoute, ["script", "--", script]);
const raw = result.stdout;
const summary = gitMirrorStatusSummary(raw);
return {
ok: result.exitCode === 0 && summary.localV01 !== null,
command: "agentrun v01 git-mirror status",
namespace: gitMirrorNamespace,
readUrl: gitMirrorReadUrl,
writeUrl: gitMirrorWriteUrl,
raw,
summary,
probe: compactCapture(result),
next: {
sync: "bun scripts/cli.ts agentrun v01 git-mirror sync --confirm",
flush: summary.pendingFlush === true ? "bun scripts/cli.ts agentrun v01 git-mirror flush --confirm" : null,
},
};
}
async function runGitMirrorJob(config: UniDeskConfig, action: "sync" | "flush", options: GitMirrorOptions): Promise<Record<string, unknown>> {
const jobName = `${action === "sync" ? gitMirrorSyncJobPrefix : gitMirrorFlushJobPrefix}-${Date.now().toString(36)}`.slice(0, 63);
const manifest = gitMirrorJobManifest(action, jobName);
const manifestB64 = Buffer.from(JSON.stringify(manifest), "utf8").toString("base64");
const command = `agentrun v01 git-mirror ${action}`;
if (options.dryRun || !options.confirm) {
return {
ok: true,
command,
dryRun: true,
namespace: gitMirrorNamespace,
jobName,
manifest,
next: { confirm: `bun scripts/cli.ts ${command} --confirm` },
};
}
const created = await createGitMirrorJob(config, jobName, manifestB64);
if (created.exitCode !== 0 || !options.wait) {
const status = await gitMirrorStatus(config);
return {
ok: created.exitCode === 0,
command,
dryRun: false,
mode: options.wait ? "create-failed" : "submitted",
namespace: gitMirrorNamespace,
jobName,
result: compactCapture(created),
status,
next: {
status: "bun scripts/cli.ts agentrun v01 git-mirror status",
wait: `bun scripts/cli.ts agentrun v01 git-mirror ${action} --confirm --wait`,
},
};
}
const wait = await waitForGitMirrorJob(config, action, jobName, options.timeoutSeconds);
const status = await gitMirrorStatus(config);
return {
ok: created.exitCode === 0 && wait.ok === true,
command,
dryRun: false,
mode: "waited",
namespace: gitMirrorNamespace,
jobName,
result: compactCapture(created),
wait,
status,
next: {
status: "bun scripts/cli.ts agentrun v01 git-mirror status",
flush: action === "sync" && status.summary && record(status.summary).pendingFlush === true ? "bun scripts/cli.ts agentrun v01 git-mirror flush --confirm" : null,
},
};
}
async function createGitMirrorJob(config: UniDeskConfig, jobName: string, manifestB64: string): Promise<SshCaptureResult> {
const script = [
"set -eu",
`job=${shQuote(jobName)}`,
`manifest_b64=${shQuote(manifestB64)}`,
"manifest_path=\"/tmp/$job.json\"",
"printf '%s' \"$manifest_b64\" | base64 -d > \"$manifest_path\"",
`kubectl delete job -n ${gitMirrorNamespace} "$job" --ignore-not-found=true >/dev/null`,
"kubectl create -f \"$manifest_path\"",
`kubectl get job -n ${gitMirrorNamespace} "$job" -o 'jsonpath=created={.metadata.name}{\"\\n\"}'`,
].join("\n");
return await capture(config, g14K3sRoute, ["script", "--", script]);
}
async function waitForGitMirrorJob(config: UniDeskConfig, action: "sync" | "flush", jobName: string, timeoutSeconds: number): Promise<Record<string, unknown>> {
const startedAtMs = Date.now();
let lastProbe: SshCaptureResult | null = null;
let polls = 0;
while (Date.now() - startedAtMs <= timeoutSeconds * 1000) {
polls += 1;
lastProbe = await capture(config, g14K3sRoute, ["script", "--", gitMirrorJobProbeScript(jobName)]);
const summary = gitMirrorJobProbeSummary(lastProbe.stdout);
process.stderr.write(`${JSON.stringify({
event: `agentrun.git-mirror.${action}.progress`,
at: new Date().toISOString(),
stage: "k8s-job",
status: summary.succeeded ? "succeeded" : summary.failed ? "failed" : "running",
jobName,
polls,
elapsedMs: Date.now() - startedAtMs,
})}\n`);
if (summary.succeeded) {
return {
ok: true,
action,
jobName,
polls,
elapsedMs: Date.now() - startedAtMs,
summary,
probe: compactCapture(lastProbe),
};
}
if (summary.failed) {
return {
ok: false,
action,
jobName,
polls,
elapsedMs: Date.now() - startedAtMs,
degradedReason: "git-mirror-job-failed",
summary,
probe: compactCapture(lastProbe),
};
}
await sleep(5_000);
}
return {
ok: false,
action,
jobName,
polls,
elapsedMs: Date.now() - startedAtMs,
degradedReason: "git-mirror-job-timeout",
probe: lastProbe ? compactCapture(lastProbe) : null,
};
}
function gitMirrorJobProbeScript(jobName: string): string {
return [
"set +e",
`job=${shQuote(jobName)}`,
"printf 'jobStatus='",
`kubectl get job -n ${gitMirrorNamespace} "$job" -o jsonpath='succeeded={.status.succeeded} failed={.status.failed} active={.status.active} start={.status.startTime} completion={.status.completionTime}' 2>/dev/null || true`,
"printf '\\n'",
"printf 'pods\\n'",
`kubectl get pod -n ${gitMirrorNamespace} -l job-name="$job" -o 'custom-columns=NAME:.metadata.name,READY:.status.containerStatuses[*].ready,STATUS:.status.phase,RESTARTS:.status.containerStatuses[*].restartCount,START:.status.startTime' --no-headers 2>/dev/null || true`,
"printf 'logs\\n'",
`kubectl logs -n ${gitMirrorNamespace} "job/$job" --tail=120 2>/dev/null || true`,
].join("\n");
}
function gitMirrorJobProbeSummary(stdout: string): Record<string, unknown> {
const line = matchLine(stdout, "jobStatus=") ?? "";
const succeeded = /(?:^|\s)succeeded=1(?:\s|$)/u.test(line);
const failedRaw = firstMatch(line, /(?:^|\s)failed=([0-9]+)/u);
const failed = failedRaw !== null && Number(failedRaw) > 0;
const activeRaw = firstMatch(line, /(?:^|\s)active=([0-9]+)/u);
return {
statusLine: line,
succeeded,
failed,
active: activeRaw === null ? null : Number(activeRaw),
};
}
function gitMirrorJobManifest(action: "sync" | "flush", name: string): Record<string, unknown> {
return {
apiVersion: "batch/v1",
kind: "Job",
metadata: {
name,
namespace: gitMirrorNamespace,
labels: {
"app.kubernetes.io/name": "git-mirror",
"app.kubernetes.io/part-of": "devops-infra",
"app.kubernetes.io/component": action === "sync" ? "sync-controller" : "flush-controller",
"agentrun.pikastech.local/trigger": "manual-cli",
},
},
spec: {
backoffLimit: 0,
activeDeadlineSeconds: 300,
ttlSecondsAfterFinished: 3600,
template: {
metadata: {
labels: {
"app.kubernetes.io/name": "git-mirror",
"app.kubernetes.io/part-of": "devops-infra",
"app.kubernetes.io/component": action === "sync" ? "sync-controller" : "flush-controller",
"agentrun.pikastech.local/trigger": "manual-cli",
},
},
spec: {
restartPolicy: "Never",
hostNetwork: true,
dnsPolicy: "ClusterFirstWithHostNet",
volumes: [
{ name: "cache", persistentVolumeClaim: { claimName: "git-mirror-cache" } },
{ name: "git-ssh", secret: { secretName: "git-mirror-github-ssh", defaultMode: 0o400 } },
],
containers: [{
name: action,
image: mirrorToolsImage,
imagePullPolicy: "IfNotPresent",
command: ["/bin/sh", "-ec", action === "sync" ? gitMirrorSyncShellScript() : gitMirrorFlushShellScript()],
volumeMounts: [
{ name: "cache", mountPath: "/cache" },
{ name: "git-ssh", mountPath: "/git-ssh", readOnly: true },
],
}],
},
},
},
};
}
function gitMirrorSyncShellScript(): string {
return [
"set -eu",
"started_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"mkdir -p /cache/pikasTech",
"export GIT_SSH_COMMAND='ssh -i /git-ssh/ssh-privatekey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/agentrun-known-hosts'",
`repo=${shQuote(gitMirrorRepoPath)}`,
`remote=${shQuote(githubSshUrl)}`,
"if [ ! -d \"$repo\" ]; then git clone --mirror \"$remote\" \"$repo\"; else git --git-dir=\"$repo\" remote set-url origin \"$remote\"; fi",
"git --git-dir=\"$repo\" config uploadpack.allowReachableSHA1InWant true",
"git --git-dir=\"$repo\" config http.receivepack true",
"git --git-dir=\"$repo\" fetch origin +refs/heads/v0.1:refs/heads/v0.1 +refs/heads/v0.1:refs/mirror-stage/heads/v0.1",
"if git --git-dir=\"$repo\" fetch origin +refs/heads/v0.1-gitops:refs/mirror-stage/heads/v0.1-gitops; then",
" if ! git --git-dir=\"$repo\" rev-parse --verify 'refs/heads/v0.1-gitops^{commit}' >/dev/null 2>&1; then",
" git --git-dir=\"$repo\" update-ref refs/heads/v0.1-gitops \"$(git --git-dir=\"$repo\" rev-parse --verify 'refs/mirror-stage/heads/v0.1-gitops^{commit}')\"",
" fi",
"fi",
"local_v01=$(git --git-dir=\"$repo\" rev-parse --verify 'refs/heads/v0.1^{commit}')",
"github_v01=$(git --git-dir=\"$repo\" rev-parse --verify 'refs/mirror-stage/heads/v0.1^{commit}')",
"local_gitops=$(git --git-dir=\"$repo\" rev-parse --verify 'refs/heads/v0.1-gitops^{commit}' 2>/dev/null || true)",
"github_gitops=$(git --git-dir=\"$repo\" rev-parse --verify 'refs/mirror-stage/heads/v0.1-gitops^{commit}' 2>/dev/null || true)",
"pending=false; if [ -n \"$local_gitops\" ] && { [ -z \"$github_gitops\" ] || [ \"$local_gitops\" != \"$github_gitops\" ]; }; then pending=true; fi",
"json_ref() { if [ -n \"$1\" ]; then printf '\"%s\"' \"$1\"; else printf null; fi; }",
"cat > /cache/agentrun.last-sync.json <<EOF",
"{\"event\":\"git-mirror-sync\",\"repo\":\"pikasTech/agentrun\",\"status\":\"succeeded\",\"startedAt\":\"$started_at\",\"syncedAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"localV01\":\"$local_v01\",\"githubV01\":\"$github_v01\",\"localGitops\":$(json_ref \"$local_gitops\"),\"githubGitops\":$(json_ref \"$github_gitops\"),\"pendingFlush\":$pending}",
"EOF",
"cat /cache/agentrun.last-sync.json",
].join("\n");
}
function gitMirrorFlushShellScript(): string {
return [
"set -eu",
"started_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"export GIT_SSH_COMMAND='ssh -i /git-ssh/ssh-privatekey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/agentrun-known-hosts'",
`repo=${shQuote(gitMirrorRepoPath)}`,
`remote=${shQuote(githubSshUrl)}`,
"test -d \"$repo\"",
"git --git-dir=\"$repo\" remote set-url origin \"$remote\"",
"local_gitops=$(git --git-dir=\"$repo\" rev-parse --verify 'refs/heads/v0.1-gitops^{commit}' 2>/dev/null || true)",
"if [ -n \"$local_gitops\" ]; then",
" git --git-dir=\"$repo\" -c remote.origin.mirror=false push origin refs/heads/v0.1-gitops:refs/heads/v0.1-gitops",
" git --git-dir=\"$repo\" fetch origin +refs/heads/v0.1-gitops:refs/mirror-stage/heads/v0.1-gitops",
"fi",
"github_gitops=$(git --git-dir=\"$repo\" rev-parse --verify 'refs/mirror-stage/heads/v0.1-gitops^{commit}' 2>/dev/null || true)",
"pending=false; if [ -n \"$local_gitops\" ] && { [ -z \"$github_gitops\" ] || [ \"$local_gitops\" != \"$github_gitops\" ]; }; then pending=true; fi",
"json_ref() { if [ -n \"$1\" ]; then printf '\"%s\"' \"$1\"; else printf null; fi; }",
"cat > /cache/agentrun.last-flush.json <<EOF",
"{\"event\":\"git-mirror-flush\",\"repo\":\"pikasTech/agentrun\",\"status\":\"succeeded\",\"startedAt\":\"$started_at\",\"flushedAt\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"localGitops\":$(json_ref \"$local_gitops\"),\"githubGitops\":$(json_ref \"$github_gitops\"),\"pendingFlush\":$pending}",
"EOF",
"cat /cache/agentrun.last-flush.json",
].join("\n");
}
function gitMirrorCacheProbeScript(): string {
return [
"printf 'lastSync='; cat /cache/agentrun.last-sync.json 2>/dev/null || true; printf '\\n'",
"printf 'lastFlush='; cat /cache/agentrun.last-flush.json 2>/dev/null || true; printf '\\n'",
`repo=${shQuote(gitMirrorRepoPath)}`,
"rev() { git --git-dir=\"$repo\" rev-parse --verify \"$1^{commit}\" 2>/dev/null || true; }",
"local_v01=$(rev refs/heads/v0.1)",
"github_v01=$(rev refs/mirror-stage/heads/v0.1)",
"local_gitops=$(rev refs/heads/v0.1-gitops)",
"github_gitops=$(rev refs/mirror-stage/heads/v0.1-gitops)",
"exact=false",
"exact_commit=\"\"",
"if [ -n \"$local_v01\" ]; then",
" tmp=$(mktemp -d)",
" if git init -q \"$tmp\" && git -C \"$tmp\" remote add origin http://127.0.0.1:8080/pikasTech/agentrun.git && git -C \"$tmp\" fetch --depth=1 origin \"$local_v01\" >/tmp/agentrun-exact-fetch.log 2>&1; then",
" exact_commit=$(git -C \"$tmp\" rev-parse FETCH_HEAD 2>/dev/null || true)",
" if [ \"$exact_commit\" = \"$local_v01\" ]; then exact=true; fi",
" fi",
" rm -rf \"$tmp\"",
"fi",
"node -e 'const [localV01,githubV01,localGitops,githubGitops,exact,exactCommit]=process.argv.slice(1); const n=v=>v&&v.length>0?v:null; const pending=Boolean(n(localGitops)&&(!n(githubGitops)||localGitops!==githubGitops)); console.log(\"refs=\"+JSON.stringify({refs:{localV01:n(localV01),githubV01:n(githubV01),localGitops:n(localGitops),githubGitops:n(githubGitops)},pendingFlush:pending,exactFetch:{localV01:exact===\"true\",commit:n(exactCommit)}}));' \"$local_v01\" \"$github_v01\" \"$local_gitops\" \"$github_gitops\" \"$exact\" \"$exact_commit\"",
].join("\n");
}
function gitMirrorStatusSummary(raw: string): Record<string, unknown> {
const refs = labeledJson(raw, "refs");
const lastSync = labeledJson(raw, "lastSync");
const lastFlush = labeledJson(raw, "lastFlush");
const refsRecord = record(record(refs).refs);
const localV01 = stringOrNull(refsRecord.localV01);
const githubV01 = stringOrNull(refsRecord.githubV01);
const localGitops = stringOrNull(refsRecord.localGitops);
const githubGitops = stringOrNull(refsRecord.githubGitops);
const pendingFlush = typeof record(refs).pendingFlush === "boolean" ? Boolean(record(refs).pendingFlush) : null;
return {
localV01,
githubV01,
localGitops,
githubGitops,
pendingFlush,
sourceInSync: Boolean(localV01 && githubV01 && localV01 === githubV01),
gitopsInSync: Boolean(localGitops && githubGitops && localGitops === githubGitops),
githubInSync: Boolean(localV01 && githubV01 && localV01 === githubV01 && (!localGitops || localGitops === githubGitops)),
flushNeeded: pendingFlush === true,
flushCommand: pendingFlush === true ? "bun scripts/cli.ts agentrun v01 git-mirror flush --confirm" : null,
exactFetch: record(refs).exactFetch ?? null,
lastSync,
lastFlush,
};
}
function gitMirrorSyncRequirement(sourceCommit: string, rawStatus: string): { required: boolean; reason: string; localV01: string | null } {
const summary = gitMirrorStatusSummary(rawStatus);
const localV01 = stringOrNull(summary.localV01);
return {
required: localV01 !== sourceCommit,
reason: localV01 === sourceCommit ? "local-v01-current" : localV01 === null ? "local-v01-unresolved" : "local-v01-stale",
localV01,
};
}
function startAsyncAgentRunJob(name: string, command: string[], note: string): Record<string, unknown> {
const job = startJob(name, command, note);
const statusCommand = `bun scripts/cli.ts job status ${job.id} --tail-bytes 12000`;
return {
ok: true,
mode: "async-job",
job,
statusCommand,
tailCommand: `tail -f ${job.stdoutFile}`,
next: {
status: statusCommand,
tail: `tail -f ${job.stdoutFile}`,
},
};
}
async function capture(config: UniDeskConfig, target: string, args: string[]): Promise<SshCaptureResult> {
return await runSshCommandCapture(config, target, args);
}
@@ -303,6 +768,29 @@ function matchLine(text: string, prefix: string): string | null {
return line ? line.slice(prefix.length).trim() || null : null;
}
function firstMatch(text: string, pattern: RegExp): string | null {
return pattern.exec(text)?.[1] ?? null;
}
function labeledJson(text: string, label: string): Record<string, unknown> {
const prefix = `${label}=`;
const line = text.split(/\r?\n/u).find((item) => item.startsWith(prefix));
if (!line) return {};
try {
return record(JSON.parse(line.slice(prefix.length)));
} catch {
return {};
}
}
function record(value: unknown): Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value) ? value as Record<string, unknown> : {};
}
function stringOrNull(value: unknown): string | null {
return typeof value === "string" && value.length > 0 ? value : null;
}
function parseArgoStatus(text: string): { revision: string | null; syncStatus: string | null; healthStatus: string | null } {
const lines = text.split(/\r?\n/u);
const index = lines.findIndex((line) => line.trim() === "argo");
@@ -323,6 +811,10 @@ function tail(text: string, maxChars: number): string {
return text.length > maxChars ? text.slice(-maxChars) : text;
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function shQuote(value: string): string {
return `'${value.replace(/'/gu, "'\\''")}'`;
}
@@ -332,6 +824,6 @@ function unsupported(args: string[]): Record<string, unknown> {
ok: false,
command: `agentrun ${args.join(" ")}`.trim(),
degradedReason: "unsupported-command",
message: "supported commands: agentrun v01 control-plane status|trigger-current|refresh",
message: "supported commands: agentrun v01 control-plane status|trigger-current|refresh; agentrun v01 git-mirror status|sync|flush",
};
}
scripts/src/hwlab-g14.ts
+6 -6
View File
@@ -2652,9 +2652,9 @@ function runGitMirrorSync(options: G14GitMirrorOptions): Record<string, unknown>
"kubectl create -f \"$manifest_path\"",
`deadline=$(( $(date +%s) + ${options.timeoutSeconds} ))`,
"while :; do",
` status=$(kubectl get job -n ${shellQuote(GIT_MIRROR_NAMESPACE)} "$job" -o jsonpath='{.status.succeeded} {.status.failed}' 2>/dev/null || true)`,
" succeeded=$(printf '%s\\n' \"$status\" | awk '{print $1}')",
" failed=$(printf '%s\\n' \"$status\" | awk '{print $2}')",
` status=$(kubectl get job -n ${shellQuote(GIT_MIRROR_NAMESPACE)} "$job" -o jsonpath='succeeded={.status.succeeded} failed={.status.failed}' 2>/dev/null || true)`,
" succeeded=$(printf '%s\\n' \"$status\" | awk '{for (i = 1; i <= NF; i++) { split($i, a, \"=\"); if (a[1] == \"succeeded\") print a[2]; }}')",
" failed=$(printf '%s\\n' \"$status\" | awk '{for (i = 1; i <= NF; i++) { split($i, a, \"=\"); if (a[1] == \"failed\") print a[2]; }}')",
" if [ \"${succeeded:-0}\" = \"1\" ]; then break; fi",
" if [ \"${failed:-0}\" != \"\" ] && [ \"${failed:-0}\" != \"0\" ]; then",
` kubectl logs -n ${shellQuote(GIT_MIRROR_NAMESPACE)} "job/$job" --tail=200 || true`,
@@ -2702,9 +2702,9 @@ function runGitMirrorFlush(options: G14GitMirrorOptions): Record<string, unknown
"kubectl create -f \"$manifest_path\"",
`deadline=$(( $(date +%s) + ${options.timeoutSeconds} ))`,
"while :; do",
` status=$(kubectl get job -n ${shellQuote(GIT_MIRROR_NAMESPACE)} "$job" -o jsonpath='{.status.succeeded} {.status.failed}' 2>/dev/null || true)`,
" succeeded=$(printf '%s\n' \"$status\" | awk '{print $1}')",
" failed=$(printf '%s\n' \"$status\" | awk '{print $2}')",
` status=$(kubectl get job -n ${shellQuote(GIT_MIRROR_NAMESPACE)} "$job" -o jsonpath='succeeded={.status.succeeded} failed={.status.failed}' 2>/dev/null || true)`,
" succeeded=$(printf '%s\n' \"$status\" | awk '{for (i = 1; i <= NF; i++) { split($i, a, \"=\"); if (a[1] == \"succeeded\") print a[2]; }}')",
" failed=$(printf '%s\n' \"$status\" | awk '{for (i = 1; i <= NF; i++) { split($i, a, \"=\"); if (a[1] == \"failed\") print a[2]; }}')",
" if [ \"${succeeded:-0}\" = \"1\" ]; then break; fi",
" if [ \"${failed:-0}\" != \"\" ] && [ \"${failed:-0}\" != \"0\" ]; then",
` kubectl logs -n ${shellQuote(GIT_MIRROR_NAMESPACE)} "job/$job" --tail=200 || true`,
scripts/src/jobs.ts
+6 -4
View File
@@ -25,7 +25,7 @@ export interface JobRecord {
}
export interface JobProgressSummary {
kind: "hwlab-v02-trigger" | "hwlab-git-mirror" | "generic";
kind: "hwlab-v02-trigger" | "git-mirror" | "generic";
stage: string | null;
stageStatus: string | null;
sourceCommit: string | null;
@@ -196,7 +196,7 @@ export function jobWithTail(job: JobRecord, maxBytes = 12000): JobRecord & {
function summarizeJobProgress(job: JobRecord, maxBytes = 96_000, tails?: { stdoutTail: string; stderrTail: string }): JobProgressSummary {
const knownWorkflow = job.name === "hwlab_g14_v02_trigger_current";
const gitMirrorWorkflow = job.name === "hwlab_g14_git_mirror_sync" || job.name === "hwlab_g14_git_mirror_flush";
const gitMirrorWorkflow = job.name === "hwlab_g14_git_mirror_sync" || job.name === "hwlab_g14_git_mirror_flush" || job.name === "agentrun_v01_git_mirror_sync" || job.name === "agentrun_v01_git_mirror_flush";
if (!knownWorkflow && !gitMirrorWorkflow && tails === undefined) return genericJobProgress(job);
const nowMs = Date.now();
const progressTailBytes = Math.max(4096, Math.floor(maxBytes));
@@ -290,7 +290,7 @@ function summarizeGitMirrorJobProgress(job: JobRecord, stdoutTail: string, stder
.filter(([key]) => key.startsWith(`gitMirror.${action}.`) || key === "sshRuntimeMaxMs")
.map(([key, value]) => `${key}=${value}ms`);
return {
kind: "hwlab-git-mirror",
kind: "git-mirror",
stage: action,
stageStatus: status ?? (job.status === "running" ? "running" : null),
sourceCommit: null,
@@ -315,7 +315,9 @@ function summarizeGitMirrorJobProgress(job: JobRecord, stdoutTail: string, stder
].filter(Boolean).join(" "),
nextCommand: job.status === "running"
? `bun scripts/cli.ts job status ${job.id} --tail-bytes 12000`
: "bun scripts/cli.ts hwlab g14 git-mirror status",
: job.name.startsWith("agentrun_")
? "bun scripts/cli.ts agentrun v01 git-mirror status"
: "bun scripts/cli.ts hwlab g14 git-mirror status",
};
}