Merge pull request #2118 from pikasTech/feat/pikaoa-production-backup
Pipelines as Code CI / hwlab-web-probe-sentinel-nc01- Success
Pipelines as Code CI / platform-infra-gitea-nc01- Success
Pipelines as Code CI / unidesk-host- Success

feat: 正规化 PK01 多数据库逻辑备份
This commit is contained in:
Lyon
2026-07-15 12:25:01 +08:00
committed by GitHub
6 changed files with 456 additions and 76 deletions
+36 -11
View File
@@ -11,6 +11,7 @@ metadata:
- 297
- 300
- 2078
- 2115
cluster:
role: primary
@@ -822,17 +823,41 @@ exports:
backup:
phase: minimum-restoreable
logicalDump:
enabled: true
schedule: "17 3 * * *"
database: sub2api
retentionDays: 14
destination:
type: pk01-local
path: /var/backups/unidesk/platform-db/pk01/sub2api
encryption:
enabled: false
futureKeyRef: platform-db/backup-age-recipient.txt
logicalDumps:
- enabled: true
database: sub2api
scriptPath: /usr/local/sbin/unidesk-pk01-sub2api-pgdump.sh
serviceName: unidesk-pk01-sub2api-pgdump.service
timerName: unidesk-pk01-sub2api-pgdump.timer
schedule: "*-*-* 03:17:00"
retentionDays: 14
warningAfterHours: 36
destination:
type: pk01-local
path: /var/backups/unidesk/platform-db/pk01/sub2api
pgDump:
host: 127.0.0.1
format: custom
encryption:
enabled: false
futureKeyRef: platform-db/backup-age-recipient.txt
- enabled: true
database: pikaoa
scriptPath: /usr/local/sbin/unidesk-pk01-pikaoa-pgdump.sh
serviceName: unidesk-pk01-pikaoa-pgdump.service
timerName: unidesk-pk01-pikaoa-pgdump.timer
schedule: "*-*-* 03:37:00"
retentionDays: 14
warningAfterHours: 36
destination:
type: pk01-local
path: /var/backups/unidesk/platform-db/pk01/pikaoa
pgDump:
host: 127.0.0.1
format: custom
encryption:
enabled: false
futureKeyRef: platform-db/backup-age-recipient.txt
physicalWalArchive:
enabled: false
futureTool: pgbackrest
@@ -0,0 +1,49 @@
# R1.6.1.1 任务报告
## 结论
已实现 pikasTech/unidesk#2115,并提交 PR #2118。PK01 PostgreSQL 逻辑备份已从单个 Sub2API 特例正规化为 YAML-first `logicalDumps[]`;保留 Sub2API 既有 03:17 schedule、14 天 retention、目录与 systemd 资源语义,新增 PikaOA 独立 03:37 schedule、14 天 retention、脚本、service、timer 和目录。
任务保持 `in_progress`。本次按派单边界未执行真实 `apply` 或手动 dump,等待主代理审核后完成运行面 mutation 与验收。
## 变更
- `config/platform-db/postgres-pk01.yaml`:每项完整声明 database、scriptPath、serviceName、timerName、目录、schedule、retention、warningAfterHours、pgDump 参数和 encryption 状态。
- `scripts/src/platform-db.ts`:按 YAML 解析、校验、生成远端 payload,动态渲染全部 dump 脚本与 systemd unit;删除 Sub2API 专用路径、unit、timer schedule 硬编码。
- `scripts/src/platform-db-postgres-backups.ts`:集中生成每项备份的声明、timer、最近成功、最新非空 dump、可恢复性和 warning 投影。
- `scripts/src/platform-db-postgres-backups.test.ts`:覆盖配置解析、renderer 去硬编码和 warning 非阻塞语义。
## Primary Workspace 轻量准备
Primary workspace 仅用于读取 `dad-dev``unidesk-ops``unidesk-ymalops``cli-spec``git-spec``unidesk-trans``unidesk-gh``mdtodo-edit``post-task` 技能;未在 runner 本地读取、编辑或验证目标源码,未把 runner 容器结果作为 Target 证据。
## Target 原入口证据
全部源码、Git、测试和运行面命令均通过 `trans NC01:/root/unidesk/.worktree/pikaoa-production-backup ...` 执行。
- `bun --check scripts/src/platform-db-postgres-backups.ts`:通过。
- `bun --check scripts/src/platform-db.ts`:通过。
- `bun test scripts/src/platform-db-postgres-backups.test.ts`3 pass0 fail。
- 生成的远端 apply runner 经 `sh -n`:通过。
- `git diff --check`:通过。
- 只读 `platform-db postgres plan``ok=true``mutation=false`Sub2API timer active/enabled,最近成功为 2026-07-15 03:17:01 CST,最新非空 dump 为 90,798,991 bytes,可恢复性为 truePikaOA 未 apply,脚本/timer/最近成功/latest dump 缺失均只形成 warning。
- 只读 `platform-db postgres status``ok=true``mutation=false``failure=null`;备份 warning 不阻塞 PikaOA MVP。
## Git 与 PR
- 分支:`feat/pikaoa-production-backup`
- 最新 master 已语义合入:merge commit `3086ea8d`
- 功能提交:`c483d6d508f5a9501f211d2b7bffd16a04494483`
- PRpikasTech/unidesk#2118,关联 pikasTech/unidesk#2115,未合并。
## 风险与下一步
- PikaOA 的 systemd 资源与 dump 文件在 apply 前不存在,当前 warning 与预期一致。
- 主代理审核后执行:
1. `bun scripts/cli.ts platform-db postgres plan --config config/platform-db/postgres-pk01.yaml`
2. `bun scripts/cli.ts platform-db postgres apply --config config/platform-db/postgres-pk01.yaml --confirm`
3. 按异步 job 返回的 `statusCommand` 短轮询至完成。
4. `bun scripts/cli.ts platform-db postgres status --config config/platform-db/postgres-pk01.yaml`
5. `sudo systemctl start unidesk-pk01-pikaoa-pgdump.service`
6. 再次执行 status,确认 PikaOA timer active/enabled、最近成功存在、最新非空 dump 大于 0 bytes、recoverable=true 且 warnings 为空。
- 上述 mutation 与手动 dump 验收完成后,再将 MDTODO R1.6.1.1 标记 complete。
@@ -139,6 +139,9 @@
#### R1.6.1 [in_progress]
建立仅跟随产品 release 的 PikaOA 生产 PaC/Tekton/GitOps/Argo lane,部署 NC01 pikaoa namespace,接入 PK01 host PostgreSQL、附件持久化、OTel/Prometheus、备份与 oa.pikapython.com TLS 公网入口(pikasTech/unidesk#2110),完成任务后将详细报告写入[任务报告](./details/pikaoa-enterprise-platform/R1.6.1_Task_Report.md)。
##### R1.6.1.1 [in_progress]
将 PK01 PostgreSQL 逻辑备份正规化为 YAML-first 多数据库 logicalDumps,保留 Sub2API 并新增 PikaOA 独立 dump、timer、retention 与受控状态/可恢复性摘要(pikasTech/unidesk#2115),完成任务后将详细报告写入[任务报告](./details/pikaoa-enterprise-platform/R1.6.1.1_Task_Report.md)。
### R1.7
通过 CLI、OTel、Prometheus 和 https://oa.pikapython.com 桌面/移动原入口验证完整主路径,依赖 R1.6,完成任务后将详细报告写入[任务报告](./details/pikaoa-enterprise-platform/R1.7_Task_Report.md)。
@@ -0,0 +1,57 @@
import { describe, expect, test } from "bun:test";
import { projectLogicalDumps } from "./platform-db-postgres-backups";
import { readPostgresLogicalDumps, renderPostgresRemoteRunnerForTest } from "./platform-db";
describe("PK01 PostgreSQL logical dumps", () => {
test("parses independent Sub2API and PikaOA declarations", () => {
const logicalDumps = readPostgresLogicalDumps("config/platform-db/postgres-pk01.yaml");
expect(logicalDumps).toHaveLength(2);
expect(logicalDumps.map((item) => item.database)).toEqual(["sub2api", "pikaoa"]);
expect(logicalDumps[0]).toMatchObject({
scriptPath: "/usr/local/sbin/unidesk-pk01-sub2api-pgdump.sh",
serviceName: "unidesk-pk01-sub2api-pgdump.service",
timerName: "unidesk-pk01-sub2api-pgdump.timer",
schedule: "*-*-* 03:17:00",
retentionDays: 14,
});
expect(logicalDumps[1]).toMatchObject({
destination: { path: "/var/backups/unidesk/platform-db/pk01/pikaoa" },
schedule: "*-*-* 03:37:00",
retentionDays: 14,
});
});
test("renders YAML-driven backup resources without Sub2API constants", () => {
const runner = renderPostgresRemoteRunnerForTest();
expect(runner).toContain('data["backup"]["logicalDumps"][index]');
expect(runner).toContain("backup.logicalDumps.$backup_index.serviceName");
expect(runner).toContain("backup.logicalDumps.$backup_index.timerName");
expect(runner).not.toContain("unidesk-pk01-sub2api-pgdump.service");
expect(runner).not.toContain("OnCalendar=*-*-* 03:17:00");
});
test("reports stale or missing evidence as warnings", () => {
const [declaration] = readPostgresLogicalDumps("config/platform-db/postgres-pk01.yaml");
const [projection] = projectLogicalDumps([declaration], [{
database: declaration.database,
databaseExists: true,
scriptPath: declaration.scriptPath,
scriptExists: true,
serviceName: declaration.serviceName,
serviceResult: "success",
serviceExitStatus: 0,
lastSuccessAt: null,
timerName: declaration.timerName,
timerActive: true,
timerEnabled: true,
nextRunAt: null,
latestNonEmptyDump: null,
}]);
expect(projection.recoverability.recoverable).toBeFalse();
expect(projection.warnings).toContain("backup-recent-success-missing");
expect(projection.warnings).toContain("backup-latest-non-empty-dump-missing");
});
});
@@ -0,0 +1,90 @@
export interface LogicalDumpConfig {
enabled: boolean;
database: string;
scriptPath: string;
serviceName: string;
timerName: string;
schedule: string;
retentionDays: number;
warningAfterHours: number;
destination: { type: string; path: string };
pgDump: { host: string; format: "custom" };
encryption: { enabled: boolean };
}
export interface LogicalDumpRuntimeFact {
database: string;
databaseExists: boolean;
scriptPath: string;
scriptExists: boolean;
serviceName: string;
serviceResult: string | null;
serviceExitStatus: number | null;
lastSuccessAt: string | null;
timerName: string;
timerActive: boolean;
timerEnabled: boolean;
nextRunAt: string | null;
latestNonEmptyDump: {
path: string;
bytes: number;
modifiedAt: string;
} | null;
}
export interface LogicalDumpProjection extends LogicalDumpConfig {
observed: LogicalDumpRuntimeFact | null;
recoverability: {
recoverable: boolean;
latestNonEmptyDump: LogicalDumpRuntimeFact["latestNonEmptyDump"];
};
warnings: string[];
}
export function projectLogicalDumps(
declarations: LogicalDumpConfig[],
facts: LogicalDumpRuntimeFact[] | null,
observedAt = Date.now(),
): LogicalDumpProjection[] {
return declarations.map((declaration) => {
const observed = facts?.find((item) => item.database === declaration.database && item.timerName === declaration.timerName) ?? null;
const warnings: string[] = [];
if (!declaration.enabled) return projection(declaration, observed, warnings);
if (observed === null) {
warnings.push("backup-observation-unavailable");
return projection(declaration, observed, warnings);
}
if (!observed.databaseExists) warnings.push("backup-database-missing");
if (!observed.scriptExists) warnings.push("backup-script-missing");
if (!observed.timerEnabled) warnings.push("backup-timer-disabled");
if (!observed.timerActive) warnings.push("backup-timer-inactive");
if (observed.lastSuccessAt === null) warnings.push("backup-recent-success-missing");
if (observed.latestNonEmptyDump === null) {
warnings.push("backup-latest-non-empty-dump-missing");
} else {
const modifiedAt = Date.parse(observed.latestNonEmptyDump.modifiedAt);
if (!Number.isFinite(modifiedAt) || observedAt - modifiedAt > declaration.warningAfterHours * 60 * 60 * 1000) {
warnings.push("backup-latest-dump-stale");
}
}
if (observed.serviceResult !== null && observed.serviceResult !== "success") warnings.push("backup-last-run-failed");
if (observed.serviceExitStatus !== null && observed.serviceExitStatus !== 0) warnings.push("backup-last-run-nonzero");
return projection(declaration, observed, warnings);
});
}
function projection(
declaration: LogicalDumpConfig,
observed: LogicalDumpRuntimeFact | null,
warnings: string[],
): LogicalDumpProjection {
return {
...declaration,
observed,
recoverability: {
recoverable: observed?.databaseExists === true && observed.latestNonEmptyDump !== null,
latestNonEmptyDump: observed?.latestNonEmptyDump ?? null,
},
warnings,
};
}
+221 -65
View File
@@ -8,6 +8,11 @@ import { startJob } from "./jobs";
import { MONITOR_CENTRAL_MIGRATIONS, MONITOR_CENTRAL_SCHEMA_VERSION } from "./monitor-central-persistence";
import { runSshCommandCapture, type SshCaptureResult } from "./ssh";
import { compactText, fingerprintEnvValues as fingerprintValues, parseEnvFile, parseJsonOutput } from "./platform-infra-ops-library";
import {
projectLogicalDumps,
type LogicalDumpConfig,
type LogicalDumpRuntimeFact,
} from "./platform-db-postgres-backups";
const defaultConfigPath = "config/platform-db/postgres-pk01.yaml";
const managedHbaStart = "# BEGIN unidesk managed pk01-platform-postgres";
@@ -138,16 +143,7 @@ interface PostgresHostConfig {
exports: {
connectionStrings: ConnectionStringExportConfig[];
};
backup: {
logicalDump: {
enabled: boolean;
schedule: string;
database: string;
retentionDays: number;
destination: { type: string; path: string };
encryption: { enabled: boolean };
};
};
backup: { logicalDumps: LogicalDumpConfig[] };
managedOperations?: { monitorReset: MonitorResetConfig };
}
@@ -272,6 +268,7 @@ interface RemoteFacts {
error: string | null;
}>;
};
backup: { logicalDumps: LogicalDumpRuntimeFact[] };
raw?: Record<string, unknown>;
}
@@ -442,6 +439,7 @@ async function plan(config: UniDeskConfig, options: PlatformDbOptions): Promise<
const facts = remote.parsed;
const checks = facts === null ? [] : desiredChecks(pg, facts, secrets);
const desired = desiredSummary(pg, secrets, facts);
const logicalDumps = projectLogicalDumps(pg.backup.logicalDumps, facts?.backup.logicalDumps ?? null);
const result = {
ok: remote.capture.exitCode === 0 && facts !== null && secrets.ok,
action: "platform-db-postgres-plan",
@@ -450,6 +448,8 @@ async function plan(config: UniDeskConfig, options: PlatformDbOptions): Promise<
secrets: secretSummary(secrets),
remoteFacts: facts,
desired,
logicalDumps,
warnings: logicalDumps.flatMap((item) => item.warnings.map((warning) => ({ database: item.database, warning }))),
checks,
next: {
apply: `bun scripts/cli.ts platform-db postgres apply --config ${pg.configPath} --confirm`,
@@ -491,6 +491,7 @@ async function status(config: UniDeskConfig, options: PlatformDbOptions): Promis
...(endpointHealthy ? [] : [controllerConnection.attempted ? "connection-host-probe-failed" : "connection-host-probe-unavailable"]),
];
const exportTargets = inspectExportTargets(pg);
const logicalDumps = projectLogicalDumps(pg.backup.logicalDumps, facts?.backup.logicalDumps ?? null);
const result = {
ok: remote.capture.exitCode === 0 && facts !== null && deploymentHealthy && secrets.ok,
action: "platform-db-postgres-status",
@@ -526,6 +527,8 @@ async function status(config: UniDeskConfig, options: PlatformDbOptions): Promis
dnsRequiredBeforeSub2ApiCutover: false,
dnsDisposition: facts.network.dns.resolves ? "optional-alias-resolves" : "optional-alias-unresolved",
},
logicalDumps,
warnings: logicalDumps.flatMap((item) => item.warnings.map((warning) => ({ database: item.database, warning }))),
remoteFacts: facts,
secrets: secretSummary(secrets),
exports: exportTargets,
@@ -668,9 +671,6 @@ function readPostgresHostConfig(pathArg: string): PostgresHostConfig {
const exportsConfig = objectField(parsed, "exports", configPath);
const backup = objectField(parsed, "backup", configPath);
const managedOperations = parsed.managedOperations === undefined ? null : objectField(parsed, "managedOperations", configPath);
const logicalDump = objectField(backup, "logicalDump", `${configPath}.backup`);
const destination = objectField(logicalDump, "destination", `${configPath}.backup.logicalDump`);
const encryption = objectField(logicalDump, "encryption", `${configPath}.backup.logicalDump`);
const manager = stringField(postgresPackage, "manager", `${configPath}.postgres.package`);
if (manager !== "apt") throw new Error(`${configPath}.postgres.package.manager must be apt`);
const packageVersion = stringField(postgresPackage, "version", `${configPath}.postgres.package`);
@@ -697,6 +697,23 @@ function readPostgresHostConfig(pathArg: string): PostgresHostConfig {
databaseNames.add(database.name);
}
const connectionStrings = arrayOfRecords(exportsConfig.connectionStrings, `${configPath}.exports.connectionStrings`).map((item, index) => connectionStringExport(item, `${configPath}.exports.connectionStrings[${index}]`));
const logicalDumps = arrayOfRecords(backup.logicalDumps, `${configPath}.backup.logicalDumps`).map((item, index) => logicalDumpConfig(item, `${configPath}.backup.logicalDumps[${index}]`));
if (logicalDumps.length === 0) throw new Error(`${configPath}.backup.logicalDumps must declare at least one logical dump`);
const logicalDumpKeys = new Set<string>();
for (const logicalDump of logicalDumps) {
if (!databaseNames.has(logicalDump.database)) throw new Error(`${configPath}.backup.logicalDumps.${logicalDump.database}.database must reference a declared database`);
for (const [kind, value] of [
["database", logicalDump.database],
["scriptPath", logicalDump.scriptPath],
["serviceName", logicalDump.serviceName],
["timerName", logicalDump.timerName],
["destination.path", logicalDump.destination.path],
] as const) {
const key = `${kind}:${value}`;
if (logicalDumpKeys.has(key)) throw new Error(`${configPath}.backup.logicalDumps contains duplicate ${kind} ${value}`);
logicalDumpKeys.add(key);
}
}
const monitorReset = managedOperations === null ? null : monitorResetConfig(objectField(managedOperations, "monitorReset", `${configPath}.managedOperations`), `${configPath}.managedOperations.monitorReset`);
if (monitorReset !== null) {
const database = databases.find((item) => item.name === monitorReset.database);
@@ -800,25 +817,53 @@ function readPostgresHostConfig(pathArg: string): PostgresHostConfig {
},
objects: { roles, databases },
exports: { connectionStrings },
backup: {
logicalDump: {
enabled: booleanField(logicalDump, "enabled", `${configPath}.backup.logicalDump`),
schedule: stringField(logicalDump, "schedule", `${configPath}.backup.logicalDump`),
database: stringField(logicalDump, "database", `${configPath}.backup.logicalDump`),
retentionDays: integerField(logicalDump, "retentionDays", `${configPath}.backup.logicalDump`),
destination: {
type: stringField(destination, "type", `${configPath}.backup.logicalDump.destination`),
path: absolutePathField(destination, "path", `${configPath}.backup.logicalDump.destination`),
},
encryption: {
enabled: booleanField(encryption, "enabled", `${configPath}.backup.logicalDump.encryption`),
},
},
},
backup: { logicalDumps },
...(monitorReset === null ? {} : { managedOperations: { monitorReset } }),
};
}
export function readPostgresLogicalDumps(pathArg: string): LogicalDumpConfig[] {
return readPostgresHostConfig(pathArg).backup.logicalDumps;
}
function logicalDumpConfig(item: Record<string, unknown>, path: string): LogicalDumpConfig {
const destination = objectField(item, "destination", path);
const pgDump = objectField(item, "pgDump", path);
const encryption = objectField(item, "encryption", path);
const serviceName = systemdUnitName(stringField(item, "serviceName", path), `${path}.serviceName`, ".service");
const timerName = systemdUnitName(stringField(item, "timerName", path), `${path}.timerName`, ".timer");
const format = stringField(pgDump, "format", `${path}.pgDump`);
if (format !== "custom") throw new Error(`${path}.pgDump.format must be custom`);
const retentionDays = integerField(item, "retentionDays", path);
const warningAfterHours = integerField(item, "warningAfterHours", path);
if (retentionDays < 1) throw new Error(`${path}.retentionDays must be positive`);
if (warningAfterHours < 1) throw new Error(`${path}.warningAfterHours must be positive`);
return {
enabled: booleanField(item, "enabled", path),
database: pgIdentifier(stringField(item, "database", path), `${path}.database`),
scriptPath: absolutePathField(item, "scriptPath", path),
serviceName,
timerName,
schedule: stringField(item, "schedule", path),
retentionDays,
warningAfterHours,
destination: {
type: stringField(destination, "type", `${path}.destination`),
path: absolutePathField(destination, "path", `${path}.destination`),
},
pgDump: {
host: stringField(pgDump, "host", `${path}.pgDump`),
format,
},
encryption: { enabled: booleanField(encryption, "enabled", `${path}.encryption`) },
};
}
function systemdUnitName(value: string, path: string, suffix: ".service" | ".timer"): string {
if (!/^[A-Za-z0-9_.@-]+$/u.test(value) || !value.endsWith(suffix)) throw new Error(`${path} must be a safe ${suffix} unit name`);
return value;
}
function monitorResetConfig(item: Record<string, unknown>, path: string): MonitorResetConfig {
return {
database: pgIdentifier(stringField(item, "database", path), `${path}.database`),
@@ -1080,6 +1125,7 @@ function configSummary(pg: PostgresHostConfig): Record<string, unknown> {
roles: pg.objects.roles.map((role) => ({ name: role.name, login: role.login })),
databases: pg.objects.databases.map((database) => ({ name: database.name, owner: database.owner })),
},
logicalDumps: pg.backup.logicalDumps,
noK3s: true,
};
}
@@ -1142,6 +1188,7 @@ function compactPlan(
key: item.writeToSecretSource.key,
consumerScopes: item.consumers.map((consumer) => consumer.scope),
})),
logicalDumps: compactLogicalDumps(pg, facts),
checks: {
ok: checks.every((check) => check.ok === true),
passed: checks.filter((check) => check.ok === true).length,
@@ -1276,6 +1323,7 @@ function compactStatus(
fingerprint: item.fingerprint,
}),
}),
logicalDumps: compactLogicalDumps(pg, facts),
remote: compactPlanCapture(capture),
next: {
plan: `bun scripts/cli.ts platform-db postgres plan --config ${pg.configPath}`,
@@ -1292,6 +1340,28 @@ function compactStatus(
};
}
function compactLogicalDumps(pg: PostgresHostConfig, facts: RemoteFacts | null): Array<Record<string, unknown>> {
return projectLogicalDumps(pg.backup.logicalDumps, facts?.backup.logicalDumps ?? null).map((item) => ({
database: item.database,
enabled: item.enabled,
scriptPath: item.scriptPath,
serviceName: item.serviceName,
timerName: item.timerName,
directory: item.destination.path,
schedule: item.schedule,
retentionDays: item.retentionDays,
timer: item.observed === null ? null : {
active: item.observed.timerActive,
enabled: item.observed.timerEnabled,
nextRunAt: item.observed.nextRunAt,
},
lastSuccessAt: item.observed?.lastSuccessAt ?? null,
latestNonEmptyDump: item.recoverability.latestNonEmptyDump,
recoverable: item.recoverability.recoverable,
warnings: item.warnings,
}));
}
function compactExportSecrets(
pg: PostgresHostConfig,
localState: { inspection: SecretInspection; summary: Record<string, unknown>; exports: Array<Record<string, unknown>> },
@@ -1773,7 +1843,7 @@ function desiredSummary(pg: PostgresHostConfig, secrets: SecretInspection, facts
consumers: item.consumers,
valuesPrinted: false,
})),
backup: pg.backup.logicalDump,
backup: { logicalDumps: projectLogicalDumps(pg.backup.logicalDumps, facts?.backup.logicalDumps ?? null) },
};
}
@@ -1886,6 +1956,20 @@ function factsScript(pg: PostgresHostConfig, probes: Array<{ user: string; passw
`printf '%s' ${shellQuote(appHost)} >"$tmp/appConnHost.${index}"`,
].join("\n");
}).join("\n");
const logicalDumpCommands = pg.backup.logicalDumps.map((logicalDump, index) => {
const encoded = Buffer.from(JSON.stringify(logicalDump), "utf8").toString("base64");
return [
`printf '%s' '${encoded}' | base64 -d >"$tmp/logicalDump.${index}.json"`,
`[ -f ${shellQuote(logicalDump.scriptPath)} ]; printf '%s' "$?" >"$tmp/logicalDumpScriptRc.${index}"`,
`systemctl is-active ${shellQuote(logicalDump.timerName)} >"$tmp/logicalDumpTimerActive.${index}" 2>/dev/null`,
`systemctl is-enabled ${shellQuote(logicalDump.timerName)} >"$tmp/logicalDumpTimerEnabled.${index}" 2>/dev/null`,
`systemctl show ${shellQuote(logicalDump.timerName)} -p NextElapseUSecRealtime --value >"$tmp/logicalDumpNextRun.${index}" 2>/dev/null`,
`systemctl show ${shellQuote(logicalDump.serviceName)} -p Result --value >"$tmp/logicalDumpServiceResult.${index}" 2>/dev/null`,
`systemctl show ${shellQuote(logicalDump.serviceName)} -p ExecMainStatus --value >"$tmp/logicalDumpServiceExitStatus.${index}" 2>/dev/null`,
`systemctl show ${shellQuote(logicalDump.serviceName)} -p InactiveExitTimestamp --value >"$tmp/logicalDumpLastSuccess.${index}" 2>/dev/null`,
`find ${shellQuote(logicalDump.destination.path)} -type f -name ${shellQuote(`${logicalDump.database}-*.dump`)} -size +0c -printf '%T@\t%s\t%p\n' 2>/dev/null | sort -nr | head -n 1 >"$tmp/logicalDumpLatest.${index}"`,
].join("\n");
}).join("\n");
return `
set +e
tmp="$(mktemp -d)"
@@ -1939,8 +2023,9 @@ if command -v psql >/dev/null 2>&1; then
:
${probeCommands}
fi
${logicalDumpCommands}
python3 - "$tmp" "${release}" <<'PY'
import json, os, sys
import datetime, json, os, sys
tmp, release = sys.argv[1], sys.argv[2]
def text(name):
try:
@@ -1968,6 +2053,44 @@ existing_roles = set_from_lines("rolesExisting")
existing_databases = set_from_lines("databasesExisting")
roles = [{"name": name, "exists": name in existing_roles} for name in expected_roles]
databases = [{"name": item["name"], "owner": item["owner"], "exists": item["name"] in existing_databases} for item in expected_databases]
logical_dumps = []
index = 0
while True:
declaration = load_json(f"logicalDump.{index}.json", None)
if declaration is None:
break
latest_fields = text(f"logicalDumpLatest.{index}").split("\t", 2)
latest = None
if len(latest_fields) == 3:
try:
latest = {
"path": latest_fields[2],
"bytes": int(latest_fields[1]),
"modifiedAt": datetime.datetime.fromtimestamp(float(latest_fields[0]), datetime.timezone.utc).isoformat(),
}
except (TypeError, ValueError):
latest = None
service_result = text(f"logicalDumpServiceResult.{index}") or None
exit_status = int_or_none(f"logicalDumpServiceExitStatus.{index}")
last_success = text(f"logicalDumpLastSuccess.{index}") or None
if service_result != "success" or exit_status not in {None, 0}:
last_success = None
logical_dumps.append({
"database": declaration["database"],
"databaseExists": declaration["database"] in existing_databases,
"scriptPath": declaration["scriptPath"],
"scriptExists": yes_file(f"logicalDumpScriptRc.{index}"),
"serviceName": declaration["serviceName"],
"serviceResult": service_result,
"serviceExitStatus": exit_status,
"lastSuccessAt": last_success,
"timerName": declaration["timerName"],
"timerActive": text(f"logicalDumpTimerActive.{index}") == "active",
"timerEnabled": text(f"logicalDumpTimerEnabled.{index}") == "enabled",
"nextRunAt": text(f"logicalDumpNextRun.{index}") or None,
"latestNonEmptyDump": latest,
})
index += 1
app_connections = []
index = 0
while True:
@@ -2045,6 +2168,7 @@ payload = {
"appConnectionError": primary_conn.get("error"),
"appConnections": app_connections,
},
"backup": {"logicalDumps": logical_dumps},
}
print(json.dumps(payload, ensure_ascii=False, indent=2))
PY
@@ -2099,7 +2223,7 @@ function remoteApplyPayload(pg: PostgresHostConfig, secrets: SecretInspection):
databases: pg.objects.databases,
role: roles[0],
database: pg.objects.databases[0],
backup: pg.backup.logicalDump,
backup: pg.backup,
hbaMarkers: { start: managedHbaStart, end: managedHbaEnd },
};
}
@@ -2222,7 +2346,7 @@ import json, sys
data = json.load(open(sys.argv[1], encoding="utf-8"))
value = data
for part in sys.argv[2].split("."):
value = value[part]
value = value[int(part)] if isinstance(value, list) else value[part]
if isinstance(value, bool):
print("true" if value else "false")
else:
@@ -2232,10 +2356,15 @@ PY
render_file() {
name="$1"
python3 - "$payload" "$name" <<'PY'
index=""
if [ "$#" -ge 2 ]; then
index="$2"
fi
python3 - "$payload" "$name" "$index" <<'PY'
import json, sys
data = json.load(open(sys.argv[1], encoding="utf-8"))
name = sys.argv[2]
index = int(sys.argv[3]) if sys.argv[3] else None
if name == "pg_hba":
markers = data["hbaMarkers"]
print(markers["start"])
@@ -2268,7 +2397,7 @@ elif name == "databases_tsv":
for database in data["databases"]:
print("\t".join([database["name"], database["owner"], database["encoding"], database["locale"]]))
elif name == "backup_script":
backup = data["backup"]
backup = data["backup"]["logicalDumps"][index]
db = backup["database"]
database = next((item for item in data["databases"] if item["name"] == db), None)
if database is None:
@@ -2277,6 +2406,7 @@ elif name == "backup_script":
if role is None:
raise SystemExit(f"backup owner {database['owner']} has no declared role")
role_name = role["name"]
host = backup["pgDump"]["host"]
port = data["network"]["port"]
dest = backup["destination"]["path"]
retention = backup["retentionDays"]
@@ -2286,8 +2416,27 @@ elif name == "backup_script":
print(f"dest='{dest}'")
print("mkdir -p \"$dest\"")
print("ts=$(date +%Y%m%dT%H%M%S)")
print(f"PGPASSWORD='{password}' /usr/lib/postgresql/{data['pgVersion']}/bin/pg_dump -h 127.0.0.1 -p {port} -U {role_name} -d {db} --format=custom --file \"$dest/{db}-$ts.dump\"")
print(f"PGPASSWORD='{password}' /usr/lib/postgresql/{data['pgVersion']}/bin/pg_dump -h {host} -p {port} -U {role_name} -d {db} --format={backup['pgDump']['format']} --file \"$dest/{db}-$ts.dump\"")
print(f"find \"$dest\" -type f -name '{db}-*.dump' -mtime +{retention} -delete")
elif name == "backup_service":
backup = data["backup"]["logicalDumps"][index]
print("[Unit]")
print(f"Description=UniDesk PK01 {backup['database']} PostgreSQL logical backup")
print("")
print("[Service]")
print("Type=oneshot")
print(f"ExecStart={backup['scriptPath']}")
elif name == "backup_timer":
backup = data["backup"]["logicalDumps"][index]
print("[Unit]")
print(f"Description=Scheduled UniDesk PK01 {backup['database']} PostgreSQL logical backup")
print("")
print("[Timer]")
print(f"OnCalendar={backup['schedule']}")
print("Persistent=true")
print("")
print("[Install]")
print("WantedBy=timers.target")
PY
}
@@ -2307,8 +2456,6 @@ SWAP_PATH="$(json_get node.swap.path)"
SWAP_SIZE="$(json_get node.swap.size)"
CONFIG_DIR="$(json_get paths.configDir)"
DATA_DIR="$(json_get paths.dataDir)"
BACKUP_ENABLED="$(json_get backup.enabled)"
BACKUP_PATH="$(json_get backup.destination.path)"
TLS_ENABLED="$(json_get tls.enabled)"
TLS_COMMON_NAME="$(json_get tls.commonName)"
TLS_CERT_FILE="$(json_get tls.certFile)"
@@ -2408,34 +2555,39 @@ while IFS="$(printf '\t')" read -r DB_NAME DB_OWNER DB_ENCODING DB_LOCALE; do
fi
done < "$databases_tsv"
if [ "$BACKUP_ENABLED" = "true" ]; then
write_state running configure-backup
mkdir -p "$BACKUP_PATH"
backup_script="/usr/local/sbin/unidesk-pk01-sub2api-pgdump.sh"
render_file backup_script > "$backup_script"
chmod 700 "$backup_script"
cat > /etc/systemd/system/unidesk-pk01-sub2api-pgdump.service <<SERVICE
[Unit]
Description=UniDesk PK01 Sub2API PostgreSQL logical backup
[Service]
Type=oneshot
ExecStart=$backup_script
SERVICE
cat > /etc/systemd/system/unidesk-pk01-sub2api-pgdump.timer <<TIMER
[Unit]
Description=Daily UniDesk PK01 Sub2API PostgreSQL logical backup
[Timer]
OnCalendar=*-*-* 03:17:00
Persistent=true
[Install]
WantedBy=timers.target
TIMER
systemctl daemon-reload
systemctl enable --now unidesk-pk01-sub2api-pgdump.timer
fi
write_state running configure-backups
BACKUP_COUNT="$(python3 - "$payload" <<'PY'
import json, sys
data = json.load(open(sys.argv[1], encoding="utf-8"))
print(len(data["backup"]["logicalDumps"]))
PY
)"
backup_index=0
while [ "$backup_index" -lt "$BACKUP_COUNT" ]; do
enabled="$(json_get backup.logicalDumps.$backup_index.enabled)"
if [ "$enabled" = "true" ]; then
backup_path="$(json_get backup.logicalDumps.$backup_index.destination.path)"
backup_script="$(json_get backup.logicalDumps.$backup_index.scriptPath)"
backup_service="$(json_get backup.logicalDumps.$backup_index.serviceName)"
backup_timer="$(json_get backup.logicalDumps.$backup_index.timerName)"
mkdir -p "$backup_path"
render_file backup_script "$backup_index" > "$backup_script"
chmod 700 "$backup_script"
render_file backup_service "$backup_index" > "/etc/systemd/system/$backup_service"
render_file backup_timer "$backup_index" > "/etc/systemd/system/$backup_timer"
fi
backup_index=$((backup_index + 1))
done
systemctl daemon-reload
backup_index=0
while [ "$backup_index" -lt "$BACKUP_COUNT" ]; do
enabled="$(json_get backup.logicalDumps.$backup_index.enabled)"
if [ "$enabled" = "true" ]; then
backup_timer="$(json_get backup.logicalDumps.$backup_index.timerName)"
systemctl enable --now "$backup_timer"
fi
backup_index=$((backup_index + 1))
done
write_state running final-check
runuser -u postgres -- psql -Atqc "SHOW server_version;" >/dev/null
@@ -2450,6 +2602,10 @@ exit 0
`;
}
export function renderPostgresRemoteRunnerForTest(): string {
return remoteRunnerScript();
}
async function waitRemoteApplyJob(config: UniDeskConfig, pg: PostgresHostConfig, remoteJobId: string): Promise<Record<string, unknown>> {
const observations: Array<Record<string, unknown>> = [];
const maxPolls = 240;