fix(ci): reuse provider egress for backend-core artifacts
This commit is contained in:
@@ -60,7 +60,7 @@ bun scripts/cli.ts artifact-registry deploy-backend-core --commit <full-sha>
|
||||
|
||||
真实 `install` 必须是幂等动作:创建远端目录,写入 CLI 渲染出的 config/compose/unit,执行 `systemctl daemon-reload`,启用并启动 `unidesk-artifact-registry.service`,然后运行与 `health` 相同的验收检查。若远端文件 hash 与期望不一致,install 可以覆盖由本 CLI 管理的 unit/config/compose,但不得删除 registry storage。
|
||||
|
||||
`deploy-backend-core` 是 production backend-core 的 CD 入口。它必须先确认 D601 registry 中已经存在 `unidesk/backend-core:<commit>`,随后只执行短生命周期 relay、`docker pull`、retag、Compose `--no-build` recreate 和 live commit 验证;如果镜像不存在,应失败并要求先运行 CI artifact publication。
|
||||
`deploy-backend-core` 是 production backend-core 的 CD 入口。它必须先通过 CNCF Distribution HTTP API 确认 D601 registry 中已经存在 `unidesk/backend-core:<commit>`,随后通过 provider-gateway Host SSH 流式执行 `docker save | gzip`,在 master server 侧 `docker load`、retag、Compose `--no-build` recreate 和 live commit 验证;如果镜像不存在,应失败并要求先运行 CI artifact publication。
|
||||
|
||||
`status` 和 `health` 通过:
|
||||
|
||||
@@ -100,7 +100,7 @@ docker compose -p unidesk-artifact-registry -f /home/ubuntu/.unidesk/artifact-re
|
||||
2. D601 CI 从 pushed Git checkout 构建 `unidesk/backend-core:<commit>`。
|
||||
3. CI 将镜像 push 到 `127.0.0.1:5000/unidesk/backend-core:<commit>`,并记录 image ref 与 digest。
|
||||
4. CD 在 master server 上确认目标 commit/tag/digest 存在。
|
||||
5. master server 通过受控、短生命周期的 localhost relay 从 D601 registry 拉取 commit-pinned 镜像。
|
||||
5. master server 通过 provider-gateway Host SSH 从 D601 registry 流式读取 commit-pinned 镜像 tar,不开放 registry 端口,也不使用第三方镜像托管。
|
||||
6. master server retag 为 Compose 使用的 backend-core 镜像名,并执行 `docker compose up -d --no-build --no-deps --force-recreate backend-core`。
|
||||
7. 部署后通过 image label、runtime env、health payload 验证 live commit。
|
||||
|
||||
@@ -108,7 +108,7 @@ docker compose -p unidesk-artifact-registry -f /home/ubuntu/.unidesk/artifact-re
|
||||
|
||||
- source commit 来自 pushed Git,不来自 dirty worktree。
|
||||
- 镜像 tag 必须 commit-pinned,不能用 mutable latest 作为部署真相。
|
||||
- relay 是临时控制动作,不开放长期公网 registry。
|
||||
- provider-gateway SSH image stream 是临时控制动作,不开放长期公网 registry。
|
||||
- CI 可以有较多依赖;CD 只做拉取、retag、recreate 和 live commit 验证。
|
||||
- CD 不执行 `cargo build`、`docker build`、`docker compose build backend-core` 或任何等价的 Rust 构建动作。
|
||||
- `server rebuild backend-core` 仍不得作为 master server Rust 编译路径。
|
||||
|
||||
@@ -26,7 +26,7 @@ Each commit CI run performs:
|
||||
|
||||
`ci install` also prewarms the D601 k3s containerd runtime with the Tekton entrypoint/workingdir helper images, `oven/bun:1-debian`, `alpine/git:2.45.2` and `unidesk-code-queue:dev`. Missing images are pulled through the node-local provider-gateway WS egress proxy and then imported into native k3s containerd with digests preserved, so PipelineRun pods do not hang on external registry pulls. Sustained pull throughput below 1 MB/s is treated as a provider/main-server network or proxy degradation first, not as a Dockerfile or application failure.
|
||||
|
||||
Git clone and dependency downloads inside the repo check task use `d601-provider-egress-proxy.unidesk.svc.cluster.local:18789`; the NO_PROXY list keeps the in-cluster read service, D601 TCP egress gateway and any in-cluster CI Git mirror on the cluster network.
|
||||
Git clone and dependency downloads inside the repo check task use `d601-provider-egress-proxy.unidesk.svc.cluster.local:18789`; the NO_PROXY list keeps the in-cluster read service and D601 TCP egress gateway on the cluster network.
|
||||
|
||||
Private repository source authentication is part of the CI contract and follows `docs/reference/devops-hygiene.md`. If the repo-check task fails at `git clone` because credentials are unavailable, treat it as a CI infrastructure/auth gap, not as an application test result.
|
||||
|
||||
@@ -53,6 +53,7 @@ backend-core production image creation belongs to D601 CI, not to master server
|
||||
The CI artifact task must follow these rules:
|
||||
|
||||
- Input revision comes from pushed Git and is resolved to a full 40-character commit. A dirty worktree or unpushed local tree must never be used as the image source.
|
||||
- Source fetch for this artifact uses the existing D601 GitHub SSH deploy identity and the node-local provider-gateway WS egress proxy at `http://127.0.0.1:18789`. D601 prepares a commit-pinned source export under `/home/ubuntu/.unidesk/ci/backend-core-artifacts/<commit>` before creating the PipelineRun; Tekton consumes that prepared source through a read-only hostPath and must not clone GitHub itself, mount GitHub credentials, use an in-cluster Git mirror, or accept an operator-uploaded source tree.
|
||||
- The source checkout, Rust build and Docker build run on D601 CI infrastructure. The master server must not run `cargo build`, `docker compose build backend-core` or `server rebuild backend-core` as part of production backend-core deployment.
|
||||
- The image is tagged with the source commit, for example `unidesk/backend-core:<commit>`, and pushed to the D601 artifact registry as `127.0.0.1:5000/unidesk/backend-core:<commit>`.
|
||||
- The image must carry at least `unidesk.ai/service-id=backend-core`, `unidesk.ai/source-repo`, `unidesk.ai/source-commit` and `unidesk.ai/dockerfile=src/components/backend-core/Dockerfile`.
|
||||
|
||||
@@ -44,6 +44,8 @@ Acceptable source access implementations are:
|
||||
- an in-cluster Git mirror managed by UniDesk; or
|
||||
- the same commit-pinned host-fetch boundary used by `ci run-dev-e2e`, where D601 fetches the manifest commit and passes only verified inputs into Tekton.
|
||||
|
||||
For backend-core artifact publication, the required implementation is the host-fetch boundary: D601 uses the existing GitHub SSH deploy identity plus the node-local provider-gateway WS egress proxy, exports the requested commit to `/home/ubuntu/.unidesk/ci/backend-core-artifacts/<commit>`, and passes only that verified source directory into Tekton. This path must not be replaced by an in-cluster Git mirror, a third-party source mirror, an operator local checkout, or Tekton-mounted GitHub credentials.
|
||||
|
||||
If a CI repo-check task fails at `git clone` because credentials are unavailable, classify it as a CI infrastructure/auth gap, not as an application test failure.
|
||||
|
||||
## Verification Priority
|
||||
|
||||
+143
-209
@@ -1,6 +1,4 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";
|
||||
import { createServer, type AddressInfo, type Server, type Socket } from "node:net";
|
||||
import { readFileSync, writeFileSync } from "node:fs";
|
||||
import { runCommand, type CommandResult } from "./command";
|
||||
import { readConfig, repoRoot, rootPath } from "./config";
|
||||
@@ -24,7 +22,6 @@ interface ArtifactRegistryOptions {
|
||||
dryRun: boolean;
|
||||
runNow: boolean;
|
||||
commit: string | null;
|
||||
localPullPort: number | null;
|
||||
targetImage: string;
|
||||
}
|
||||
|
||||
@@ -61,7 +58,6 @@ const defaultOptions: ArtifactRegistryOptions = {
|
||||
dryRun: false,
|
||||
runNow: false,
|
||||
commit: null,
|
||||
localPullPort: null,
|
||||
targetImage: "unidesk-backend-core",
|
||||
};
|
||||
|
||||
@@ -81,12 +77,6 @@ function positiveInt(value: string, option: string): number {
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function optionalPort(value: string, option: string): number {
|
||||
const parsed = positiveInt(value, option);
|
||||
if (parsed > 65535) throw new Error(`${option} must be <= 65535`);
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function absolutePath(value: string, option: string): string {
|
||||
if (!value.startsWith("/")) throw new Error(`${option} must be an absolute path`);
|
||||
if (value.includes("\n") || value.includes("\0")) throw new Error(`${option} must not contain control characters`);
|
||||
@@ -131,9 +121,6 @@ function parseOptions(args: string[]): ArtifactRegistryOptions {
|
||||
} else if (arg === "--commit") {
|
||||
options.commit = commitValue(requireValue(args, index, arg), arg);
|
||||
index += 1;
|
||||
} else if (arg === "--local-pull-port") {
|
||||
options.localPullPort = optionalPort(requireValue(args, index, arg), arg);
|
||||
index += 1;
|
||||
} else if (arg === "--target-image") {
|
||||
options.targetImage = requireValue(args, index, arg);
|
||||
index += 1;
|
||||
@@ -154,6 +141,18 @@ function shellQuote(value: string): string {
|
||||
return `'${value.replace(/'/g, `'\\''`)}'`;
|
||||
}
|
||||
|
||||
function rootExecPrelude(): string {
|
||||
return [
|
||||
"root_exec() {",
|
||||
" if [ \"$(id -u)\" = \"0\" ]; then \"$@\"; return; fi",
|
||||
" if sudo -n true >/dev/null 2>&1; then sudo -n \"$@\"; return; fi",
|
||||
" if [ -x /mnt/c/Windows/System32/wsl.exe ]; then /mnt/c/Windows/System32/wsl.exe -u root -- \"$@\"; return; fi",
|
||||
" echo 'artifact_registry_root_access=missing' >&2",
|
||||
" return 1",
|
||||
"}",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function file(path: string, content: string, mode = "0644"): RenderedFile {
|
||||
return { path, mode, sha256: sha256(content), content };
|
||||
}
|
||||
@@ -203,15 +202,16 @@ function systemdUnit(options: ArtifactRegistryOptions): string {
|
||||
return `[Unit]
|
||||
Description=UniDesk D601 Artifact Registry (CNCF Distribution)
|
||||
Documentation=https://github.com/pikasTech/unidesk/blob/master/docs/reference/artifact-registry.md
|
||||
Requires=docker.service
|
||||
After=docker.service network-online.target
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
ConditionPathExists=/var/run/docker.sock
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=yes
|
||||
WorkingDirectory=${options.baseDir}
|
||||
ExecStartPre=/bin/mkdir -p ${options.baseDir} ${options.storageDir}
|
||||
ExecStartPre=/bin/sh -lc 'docker info >/dev/null'
|
||||
ExecStart=/bin/sh -lc 'docker compose -p ${options.composeProject} -f ${options.baseDir}/compose.yml up -d --remove-orphans'
|
||||
ExecStop=/bin/sh -lc 'docker compose -p ${options.composeProject} -f ${options.baseDir}/compose.yml down'
|
||||
TimeoutStartSec=120
|
||||
@@ -490,17 +490,18 @@ function runReadonlyStatus(options: ArtifactRegistryOptions, healthMode: boolean
|
||||
|
||||
function remoteWriteFileCommand(item: RenderedFile): string {
|
||||
const encoded = Buffer.from(item.content, "utf8").toString("base64");
|
||||
const rootOwned = item.path.startsWith("/etc/");
|
||||
return [
|
||||
`target=${shellQuote(item.path)}`,
|
||||
"tmp=$(mktemp /tmp/unidesk-artifact-registry.XXXXXX)",
|
||||
"trap 'rm -f \"$tmp\"' EXIT",
|
||||
`printf %s ${shellQuote(encoded)} | base64 -d > "$tmp"`,
|
||||
"if [ \"$(id -u)\" = \"0\" ]; then",
|
||||
`if [ ${rootOwned ? "1" : "0"} = 1 ]; then`,
|
||||
" root_exec mkdir -p \"$(dirname \"$target\")\"",
|
||||
` root_exec install -m ${shellQuote(item.mode)} "$tmp" "$target"`,
|
||||
"else",
|
||||
" mkdir -p \"$(dirname \"$target\")\"",
|
||||
` install -m ${shellQuote(item.mode)} "$tmp" "$target"`,
|
||||
"else",
|
||||
" sudo mkdir -p \"$(dirname \"$target\")\"",
|
||||
` sudo install -m ${shellQuote(item.mode)} "$tmp" "$target"`,
|
||||
"fi",
|
||||
`echo ${shellQuote(`artifact_registry_file_written path=${item.path} sha256=${item.sha256}`)}`,
|
||||
].join("\n");
|
||||
@@ -513,10 +514,11 @@ function install(options: ArtifactRegistryOptions): Record<string, unknown> {
|
||||
"command -v docker >/dev/null",
|
||||
"docker compose version >/dev/null",
|
||||
"command -v systemctl >/dev/null",
|
||||
rootExecPrelude(),
|
||||
`mkdir -p ${shellQuote(bundle.paths.baseDir)} ${shellQuote(bundle.paths.storage)}`,
|
||||
...bundle.files.map(remoteWriteFileCommand),
|
||||
"if [ \"$(id -u)\" = \"0\" ]; then systemctl daemon-reload; else sudo systemctl daemon-reload; fi",
|
||||
`if [ "$(id -u)" = "0" ]; then systemctl enable --now ${shellQuote(options.unitName)}; else sudo systemctl enable --now ${shellQuote(options.unitName)}; fi`,
|
||||
"root_exec systemctl daemon-reload",
|
||||
`root_exec systemctl enable --now ${shellQuote(options.unitName)}`,
|
||||
"sleep 2",
|
||||
`curl -fsS http://${options.host}:${options.port}/v2/ >/dev/null`,
|
||||
].join("\n");
|
||||
@@ -559,91 +561,6 @@ function installDryRun(options: ArtifactRegistryOptions): Record<string, unknown
|
||||
};
|
||||
}
|
||||
|
||||
async function startRegistryRelay(options: ArtifactRegistryOptions): Promise<{
|
||||
server: Server;
|
||||
port: number;
|
||||
close: () => Promise<void>;
|
||||
}> {
|
||||
const children = new Set<ChildProcessWithoutNullStreams>();
|
||||
const sockets = new Set<Socket>();
|
||||
const remoteProxyScript = String.raw`
|
||||
import select
|
||||
import socket
|
||||
import sys
|
||||
|
||||
target = socket.create_connection(("127.0.0.1", 5000), timeout=10)
|
||||
target.setblocking(False)
|
||||
sys.stdin.buffer.raw.fileno()
|
||||
sys.stdout.buffer.raw.fileno()
|
||||
stdin_fd = sys.stdin.fileno()
|
||||
stdout_fd = sys.stdout.fileno()
|
||||
target_fd = target.fileno()
|
||||
|
||||
while True:
|
||||
readable, _, _ = select.select([stdin_fd, target_fd], [], [])
|
||||
if stdin_fd in readable:
|
||||
data = sys.stdin.buffer.read1(65536)
|
||||
if not data:
|
||||
try:
|
||||
target.shutdown(socket.SHUT_WR)
|
||||
except OSError:
|
||||
pass
|
||||
else:
|
||||
target.sendall(data)
|
||||
if target_fd in readable:
|
||||
try:
|
||||
data = target.recv(65536)
|
||||
except BlockingIOError:
|
||||
data = b""
|
||||
if not data:
|
||||
break
|
||||
sys.stdout.buffer.write(data)
|
||||
sys.stdout.buffer.flush()
|
||||
`;
|
||||
const server = createServer((socket) => {
|
||||
sockets.add(socket);
|
||||
socket.on("close", () => sockets.delete(socket));
|
||||
const child = spawn(process.execPath, [
|
||||
"scripts/cli.ts",
|
||||
"ssh",
|
||||
options.providerId,
|
||||
"argv",
|
||||
"python3",
|
||||
"-u",
|
||||
"-c",
|
||||
remoteProxyScript,
|
||||
], {
|
||||
cwd: repoRoot,
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
children.add(child);
|
||||
child.on("close", () => children.delete(child));
|
||||
child.on("error", () => socket.destroy());
|
||||
socket.pipe(child.stdin);
|
||||
child.stdout.pipe(socket);
|
||||
child.stderr.on("data", (chunk) => process.stderr.write(`[artifact-registry-relay] ${chunk.toString("utf8")}`));
|
||||
socket.on("close", () => child.kill());
|
||||
});
|
||||
const listen = new Promise<number>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(options.localPullPort ?? 0, "127.0.0.1", () => {
|
||||
server.off("error", reject);
|
||||
const address = server.address() as AddressInfo;
|
||||
resolve(address.port);
|
||||
});
|
||||
});
|
||||
const port = await listen;
|
||||
return {
|
||||
server,
|
||||
port,
|
||||
close: async () => {
|
||||
for (const socket of sockets) socket.destroy();
|
||||
for (const child of children) child.kill();
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function composeLockScript(innerScript: string): string {
|
||||
const lockDir = rootPath(".state", "locks");
|
||||
const lockPath = rootPath(".state", "locks", "server-compose.lock");
|
||||
@@ -670,6 +587,31 @@ function upsertEnvFileValues(path: string, values: Record<string, string>): void
|
||||
writeFileSync(path, `${lines.join("\n")}\n`, "utf8");
|
||||
}
|
||||
|
||||
function pullBackendCoreArtifactFromD601(options: ArtifactRegistryOptions, sourceImage: string): CommandResult {
|
||||
const remoteScript = [
|
||||
"set -euo pipefail",
|
||||
`image=${shellQuote(sourceImage)}`,
|
||||
"docker pull -q \"$image\" >/dev/null",
|
||||
"docker image inspect \"$image\" --format 'remote_source={{ index .Config.Labels \"unidesk.ai/source-commit\" }} remote_service={{ index .Config.Labels \"unidesk.ai/service-id\" }}' >&2",
|
||||
"docker save \"$image\" | gzip -1",
|
||||
].join("\n");
|
||||
const sshCommand = [
|
||||
process.execPath,
|
||||
rootPath("scripts", "cli.ts"),
|
||||
"ssh",
|
||||
options.providerId,
|
||||
"argv",
|
||||
"bash",
|
||||
"-lc",
|
||||
remoteScript,
|
||||
].map(shellQuote).join(" ");
|
||||
const pipeline = [
|
||||
"set -euo pipefail",
|
||||
`${sshCommand} | gzip -dc | docker load`,
|
||||
].join("\n");
|
||||
return runCommand(["bash", "-lc", pipeline], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 900_000) });
|
||||
}
|
||||
|
||||
async function deployBackendCoreNow(options: ArtifactRegistryOptions): Promise<Record<string, unknown>> {
|
||||
const commit = options.commit;
|
||||
if (commit === null) throw new Error("artifact-registry deploy-backend-core requires --commit <full-sha>");
|
||||
@@ -682,11 +624,13 @@ async function deployBackendCoreNow(options: ArtifactRegistryOptions): Promise<R
|
||||
const commitImage = `${options.targetImage}:${commit}`;
|
||||
const registryProbeScript = [
|
||||
"set -euo pipefail",
|
||||
`image=${shellQuote(`unidesk/backend-core:${commit}`)}`,
|
||||
`registry_image=${shellQuote(sourceImage)}`,
|
||||
"docker manifest inspect \"$registry_image\" >/tmp/unidesk-backend-core-manifest.json",
|
||||
"manifest_digest=$(docker manifest inspect --verbose \"$registry_image\" 2>/dev/null | awk -F'\"' '/Digest/ {print $4; exit}' || true)",
|
||||
"printf 'registry_image=%s\\nmanifest_digest=%s\\n' \"$registry_image\" \"$manifest_digest\"",
|
||||
`manifest_url=${shellQuote(`http://127.0.0.1:${options.port}/v2/unidesk/backend-core/manifests/${commit}`)}`,
|
||||
"headers=$(mktemp /tmp/unidesk-backend-core-manifest.XXXXXX.headers)",
|
||||
"trap 'rm -f \"$headers\"' EXIT",
|
||||
"curl -fsSI -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -D \"$headers\" -o /dev/null \"$manifest_url\"",
|
||||
"manifest_digest=$(awk 'BEGIN{IGNORECASE=1} /^Docker-Content-Digest:/ {gsub(/\\r/, \"\", $2); print $2; exit}' \"$headers\")",
|
||||
"printf 'registry_image=%s\\nmanifest_url=%s\\nmanifest_digest=%s\\n' \"$registry_image\" \"$manifest_url\" \"$manifest_digest\"",
|
||||
].join("\n");
|
||||
const registryProbe = runRemoteScript(options, registryProbeScript, Math.max(options.timeoutMs, 120_000));
|
||||
if (registryProbe.exitCode !== 0 || registryProbe.timedOut) {
|
||||
@@ -698,110 +642,100 @@ async function deployBackendCoreNow(options: ArtifactRegistryOptions): Promise<R
|
||||
registryProbe: commandTail(registryProbe),
|
||||
};
|
||||
}
|
||||
const relay = await startRegistryRelay(options);
|
||||
const localSourceImage = `127.0.0.1:${relay.port}/unidesk/backend-core:${commit}`;
|
||||
try {
|
||||
const pull = runCommand(["docker", "pull", localSourceImage], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 900_000) });
|
||||
if (pull.exitCode !== 0 || pull.timedOut) {
|
||||
return {
|
||||
ok: false,
|
||||
step: "docker-pull",
|
||||
sourceImage,
|
||||
localRelayImage: localSourceImage,
|
||||
registryProbe: commandTail(registryProbe),
|
||||
pull: commandTail(pull),
|
||||
};
|
||||
}
|
||||
const inspectPulled = runCommand(["docker", "image", "inspect", localSourceImage, "--format", "{{json .Config.Labels}}"], repoRoot);
|
||||
const labelCommit = runCommand(["docker", "image", "inspect", localSourceImage, "--format", "{{ index .Config.Labels \"unidesk.ai/source-commit\" }}"], repoRoot);
|
||||
const labelService = runCommand(["docker", "image", "inspect", localSourceImage, "--format", "{{ index .Config.Labels \"unidesk.ai/service-id\" }}"], repoRoot);
|
||||
if (labelCommit.stdout.trim() !== commit || labelService.stdout.trim() !== "backend-core") {
|
||||
return {
|
||||
ok: false,
|
||||
step: "image-label-verify",
|
||||
expected: { commit, serviceId: "backend-core" },
|
||||
observed: { commit: labelCommit.stdout.trim(), serviceId: labelService.stdout.trim(), labels: inspectPulled.stdout.trim() },
|
||||
registryProbe: commandTail(registryProbe),
|
||||
};
|
||||
}
|
||||
const tag = runCommand(["docker", "tag", localSourceImage, composeImage], repoRoot);
|
||||
if (tag.exitCode !== 0 || tag.timedOut) {
|
||||
return { ok: false, step: "docker-tag", targetImage: composeImage, tag: commandTail(tag), registryProbe: commandTail(registryProbe) };
|
||||
}
|
||||
const tagCommit = runCommand(["docker", "tag", localSourceImage, commitImage], repoRoot);
|
||||
if (tagCommit.exitCode !== 0 || tagCommit.timedOut) {
|
||||
return { ok: false, step: "docker-tag-commit", targetImage: commitImage, tag: commandTail(tagCommit), registryProbe: commandTail(registryProbe) };
|
||||
}
|
||||
const config = readConfig();
|
||||
const runtimeEnv = writeComposeEnv(config, false);
|
||||
upsertEnvFileValues(runtimeEnv.envFile, {
|
||||
UNIDESK_DEPLOY_REF: "deploy.json#environments.prod.services.backend-core",
|
||||
UNIDESK_DEPLOY_SERVICE_ID: "backend-core",
|
||||
UNIDESK_DEPLOY_REPO: "https://github.com/pikasTech/unidesk",
|
||||
UNIDESK_DEPLOY_COMMIT: commit,
|
||||
UNIDESK_DEPLOY_REQUESTED_COMMIT: commit,
|
||||
});
|
||||
const compose = resolveComposeCommand(config, runtimeEnv.envFile);
|
||||
const upScript = [
|
||||
"set -euo pipefail",
|
||||
`echo ${shellQuote(`backend_core_artifact_cd source=${sourceImage} target=${composeImage}`)}`,
|
||||
`${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`,
|
||||
"ready=0",
|
||||
"for attempt in $(seq 1 60); do",
|
||||
" cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1 || true)",
|
||||
" if [ -n \"$cid\" ]; then",
|
||||
" health=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' \"$cid\" 2>/dev/null || true)",
|
||||
" echo \"backend_core_container_probe attempt=$attempt cid=$cid health=$health\"",
|
||||
" if [ \"$health\" = \"healthy\" ] || [ \"$health\" = \"running\" ]; then ready=1; break; fi",
|
||||
" else",
|
||||
" echo \"backend_core_container_probe attempt=$attempt cid=missing\"",
|
||||
" fi",
|
||||
" sleep 1",
|
||||
"done",
|
||||
"test \"$ready\" = \"1\"",
|
||||
"cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1)",
|
||||
"image_id=$(docker inspect -f '{{.Image}}' \"$cid\")",
|
||||
"actual_commit=$(docker image inspect -f '{{ index .Config.Labels \"unidesk.ai/source-commit\" }}' \"$image_id\")",
|
||||
`test "$actual_commit" = ${shellQuote(commit)}`,
|
||||
"docker exec \"$cid\" bun -e \"fetch('http://127.0.0.1:8080/health').then(async r=>{const text=await r.text(); console.log(text); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\" >/tmp/unidesk-backend-core-health.json",
|
||||
"cat /tmp/unidesk-backend-core-health.json",
|
||||
].join("\n");
|
||||
const deploy = runCommand(["bash", "-lc", composeLockScript(upScript)], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 300_000) });
|
||||
if (deploy.exitCode !== 0 || deploy.timedOut) {
|
||||
return {
|
||||
ok: false,
|
||||
step: "compose-recreate",
|
||||
sourceImage,
|
||||
targetImage: composeImage,
|
||||
registryProbe: commandTail(registryProbe),
|
||||
deploy: commandTail(deploy),
|
||||
};
|
||||
}
|
||||
const running = runCommand(["docker", "inspect", "unidesk-backend-core", "--format", "image={{.Config.Image}} imageId={{.Image}} status={{.State.Status}} health={{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}"], repoRoot);
|
||||
const pull = pullBackendCoreArtifactFromD601(options, sourceImage);
|
||||
if (pull.exitCode !== 0 || pull.timedOut) {
|
||||
return {
|
||||
ok: true,
|
||||
commit,
|
||||
providerId: options.providerId,
|
||||
ok: false,
|
||||
step: "docker-load",
|
||||
sourceImage,
|
||||
registryProbe: commandTail(registryProbe),
|
||||
localRelayImage: localSourceImage,
|
||||
targetImage: composeImage,
|
||||
targetCommitImage: commitImage,
|
||||
relay: {
|
||||
bind: `127.0.0.1:${relay.port}`,
|
||||
persistent: false,
|
||||
},
|
||||
pull: commandTail(pull),
|
||||
deploy: commandTail(deploy),
|
||||
running: running.stdout.trim(),
|
||||
rollback: {
|
||||
previousImageHint: "Use docker image ls and docker inspect logs for the previous backend-core image id; Compose named volumes were not changed.",
|
||||
commandShape: `${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`,
|
||||
},
|
||||
};
|
||||
} finally {
|
||||
await relay.close();
|
||||
}
|
||||
const localLoadedImage = sourceImage;
|
||||
const inspectPulled = runCommand(["docker", "image", "inspect", localLoadedImage, "--format", "{{json .Config.Labels}}"], repoRoot);
|
||||
const labelCommit = runCommand(["docker", "image", "inspect", localLoadedImage, "--format", "{{ index .Config.Labels \"unidesk.ai/source-commit\" }}"], repoRoot);
|
||||
const labelService = runCommand(["docker", "image", "inspect", localLoadedImage, "--format", "{{ index .Config.Labels \"unidesk.ai/service-id\" }}"], repoRoot);
|
||||
if (labelCommit.stdout.trim() !== commit || labelService.stdout.trim() !== "backend-core") {
|
||||
return {
|
||||
ok: false,
|
||||
step: "image-label-verify",
|
||||
expected: { commit, serviceId: "backend-core" },
|
||||
observed: { commit: labelCommit.stdout.trim(), serviceId: labelService.stdout.trim(), labels: inspectPulled.stdout.trim() },
|
||||
registryProbe: commandTail(registryProbe),
|
||||
};
|
||||
}
|
||||
const tag = runCommand(["docker", "tag", localLoadedImage, composeImage], repoRoot);
|
||||
if (tag.exitCode !== 0 || tag.timedOut) {
|
||||
return { ok: false, step: "docker-tag", targetImage: composeImage, tag: commandTail(tag), registryProbe: commandTail(registryProbe) };
|
||||
}
|
||||
const tagCommit = runCommand(["docker", "tag", localLoadedImage, commitImage], repoRoot);
|
||||
if (tagCommit.exitCode !== 0 || tagCommit.timedOut) {
|
||||
return { ok: false, step: "docker-tag-commit", targetImage: commitImage, tag: commandTail(tagCommit), registryProbe: commandTail(registryProbe) };
|
||||
}
|
||||
const config = readConfig();
|
||||
const runtimeEnv = writeComposeEnv(config, false);
|
||||
upsertEnvFileValues(runtimeEnv.envFile, {
|
||||
UNIDESK_DEPLOY_REF: "deploy.json#environments.prod.services.backend-core",
|
||||
UNIDESK_DEPLOY_SERVICE_ID: "backend-core",
|
||||
UNIDESK_DEPLOY_REPO: "https://github.com/pikasTech/unidesk",
|
||||
UNIDESK_DEPLOY_COMMIT: commit,
|
||||
UNIDESK_DEPLOY_REQUESTED_COMMIT: commit,
|
||||
});
|
||||
const compose = resolveComposeCommand(config, runtimeEnv.envFile);
|
||||
const upScript = [
|
||||
"set -euo pipefail",
|
||||
`echo ${shellQuote(`backend_core_artifact_cd source=${sourceImage} target=${composeImage}`)}`,
|
||||
`${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`,
|
||||
"ready=0",
|
||||
"for attempt in $(seq 1 60); do",
|
||||
" cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1 || true)",
|
||||
" if [ -n \"$cid\" ]; then",
|
||||
" health=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' \"$cid\" 2>/dev/null || true)",
|
||||
" echo \"backend_core_container_probe attempt=$attempt cid=$cid health=$health\"",
|
||||
" if [ \"$health\" = \"healthy\" ] || [ \"$health\" = \"running\" ]; then ready=1; break; fi",
|
||||
" else",
|
||||
" echo \"backend_core_container_probe attempt=$attempt cid=missing\"",
|
||||
" fi",
|
||||
" sleep 1",
|
||||
"done",
|
||||
"test \"$ready\" = \"1\"",
|
||||
"cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1)",
|
||||
"image_id=$(docker inspect -f '{{.Image}}' \"$cid\")",
|
||||
"actual_commit=$(docker image inspect -f '{{ index .Config.Labels \"unidesk.ai/source-commit\" }}' \"$image_id\")",
|
||||
`test "$actual_commit" = ${shellQuote(commit)}`,
|
||||
"docker exec \"$cid\" bun -e \"fetch('http://127.0.0.1:8080/health').then(async r=>{const text=await r.text(); console.log(text); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\" >/tmp/unidesk-backend-core-health.json",
|
||||
"cat /tmp/unidesk-backend-core-health.json",
|
||||
].join("\n");
|
||||
const deploy = runCommand(["bash", "-lc", composeLockScript(upScript)], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 300_000) });
|
||||
if (deploy.exitCode !== 0 || deploy.timedOut) {
|
||||
return {
|
||||
ok: false,
|
||||
step: "compose-recreate",
|
||||
sourceImage,
|
||||
targetImage: composeImage,
|
||||
registryProbe: commandTail(registryProbe),
|
||||
deploy: commandTail(deploy),
|
||||
};
|
||||
}
|
||||
const running = runCommand(["docker", "inspect", "unidesk-backend-core", "--format", "image={{.Config.Image}} imageId={{.Image}} status={{.State.Status}} health={{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}"], repoRoot);
|
||||
return {
|
||||
ok: true,
|
||||
commit,
|
||||
providerId: options.providerId,
|
||||
sourceImage,
|
||||
registryProbe: commandTail(registryProbe),
|
||||
localLoadedImage,
|
||||
targetImage: composeImage,
|
||||
targetCommitImage: commitImage,
|
||||
pull: commandTail(pull),
|
||||
deploy: commandTail(deploy),
|
||||
running: running.stdout.trim(),
|
||||
rollback: {
|
||||
previousImageHint: "Use docker image ls and docker inspect logs for the previous backend-core image id; Compose named volumes were not changed.",
|
||||
commandShape: `${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function deployBackendCoreJob(args: string[], options: ArtifactRegistryOptions): Record<string, unknown> {
|
||||
@@ -815,7 +749,7 @@ function deployBackendCoreJob(args: string[], options: ArtifactRegistryOptions):
|
||||
job,
|
||||
statusCommand: `bun scripts/cli.ts job status ${job.id}`,
|
||||
tailCommand: `bun scripts/cli.ts job status ${job.id} --tail-bytes 30000`,
|
||||
note: "Backend-core CD continues in the background: D601 registry health, short-lived local relay, docker pull, retag, no-build recreate, live commit verification.",
|
||||
note: "Backend-core CD continues in the background: D601 registry health, provider-gateway SSH image stream, docker load, retag, no-build recreate, live commit verification.",
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
+152
-9
@@ -2,6 +2,7 @@ import { randomUUID } from "node:crypto";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { runCommand } from "./command";
|
||||
import { type UniDeskConfig, repoRoot, rootPath } from "./config";
|
||||
import { ensureGithubSshIdentityForProvider, gitSshHttpConnectProxySource } from "./deploy-ssh-identity";
|
||||
import { startJob } from "./jobs";
|
||||
import { coreInternalFetch } from "./microservices";
|
||||
|
||||
@@ -38,6 +39,7 @@ interface CiPublishBackendCoreOptions {
|
||||
repoUrl: string;
|
||||
commit: string;
|
||||
waitMs: number;
|
||||
sourceHostPath: string;
|
||||
}
|
||||
|
||||
interface CiDevE2EOptions {
|
||||
@@ -64,6 +66,20 @@ interface DispatchResult {
|
||||
raw: unknown;
|
||||
}
|
||||
|
||||
interface PipelineRunCondition {
|
||||
ok: boolean | null;
|
||||
status: string;
|
||||
reason: string;
|
||||
message: string;
|
||||
query: {
|
||||
ok: boolean;
|
||||
status: string | null;
|
||||
exitCode: number | null;
|
||||
stdoutTail: string;
|
||||
stderrTail: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface DeployDevManifestSummary {
|
||||
deployCommit: string;
|
||||
desiredRef: string;
|
||||
@@ -124,6 +140,15 @@ function shellQuote(value: string): string {
|
||||
return `'${value.replace(/'/gu, "'\\''")}'`;
|
||||
}
|
||||
|
||||
function repoSshUrl(repoUrl: string): string {
|
||||
if (repoUrl.startsWith("git@")) return repoUrl;
|
||||
if (repoUrl.startsWith("https://github.com/")) {
|
||||
const repoPath = repoUrl.slice("https://github.com/".length).replace(/\.git$/u, "");
|
||||
return `git@github.com:${repoPath}.git`;
|
||||
}
|
||||
return repoUrl;
|
||||
}
|
||||
|
||||
function chunks(value: string, size: number): string[] {
|
||||
const result: string[] = [];
|
||||
for (let index = 0; index < value.length; index += size) {
|
||||
@@ -508,6 +533,8 @@ spec:
|
||||
value: ${JSON.stringify(options.repoUrl)}
|
||||
- name: revision
|
||||
value: ${JSON.stringify(options.commit)}
|
||||
- name: source-host-path
|
||||
value: ${JSON.stringify(options.sourceHostPath)}
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
persistentVolumeClaim:
|
||||
@@ -515,6 +542,77 @@ spec:
|
||||
`;
|
||||
}
|
||||
|
||||
function backendCoreArtifactSourceHostPath(commit: string): string {
|
||||
return `/home/ubuntu/.unidesk/ci/backend-core-artifacts/${commit}`;
|
||||
}
|
||||
|
||||
async function prepareBackendCoreArtifactSource(config: UniDeskConfig, options: CiPublishBackendCoreOptions): Promise<Record<string, unknown>> {
|
||||
const sshIdentity = await ensureGithubSshIdentityForProvider(config, d601ProviderId);
|
||||
if (!sshIdentity.ok) throw new Error(sshIdentity.detail);
|
||||
const proxyPython = gitSshHttpConnectProxySource();
|
||||
const sourceRoot = "/home/ubuntu/.unidesk/ci/backend-core-artifacts";
|
||||
const sourceHostPath = options.sourceHostPath;
|
||||
const repoCache = "/home/ubuntu/.unidesk/ci/git/unidesk.git";
|
||||
const repoFetchUrl = repoSshUrl(options.repoUrl);
|
||||
const script = [
|
||||
"set -euo pipefail",
|
||||
`commit=${shellQuote(options.commit)}`,
|
||||
`repo_url=${shellQuote(options.repoUrl)}`,
|
||||
`repo_fetch_url=${shellQuote(repoFetchUrl)}`,
|
||||
`source_root=${shellQuote(sourceRoot)}`,
|
||||
`source_dir=${shellQuote(sourceHostPath)}`,
|
||||
`repo_cache=${shellQuote(repoCache)}`,
|
||||
`proxy_url=${shellQuote(providerGatewayWsEgressProxyUrl)}`,
|
||||
"mkdir -p \"$(dirname \"$repo_cache\")\" \"$source_root\"",
|
||||
"export HTTP_PROXY=\"$proxy_url\" HTTPS_PROXY=\"$proxy_url\" ALL_PROXY=\"$proxy_url\"",
|
||||
"export NO_PROXY=\"localhost,127.0.0.1,::1,host.docker.internal,.svc,.cluster.local,kubernetes.default.svc\"",
|
||||
"curl -fsSI --max-time 20 -x \"$proxy_url\" https://github.com >/dev/null",
|
||||
"git_ssh_proxy=/tmp/unidesk-git-ssh-http-connect.py",
|
||||
"cat > \"$git_ssh_proxy\" <<'UNIDESK_GIT_SSH_PROXY'",
|
||||
proxyPython,
|
||||
"UNIDESK_GIT_SSH_PROXY",
|
||||
"chmod 700 \"$git_ssh_proxy\"",
|
||||
"export UNIDESK_GIT_SSH_HTTP_PROXY=\"$proxy_url\"",
|
||||
"export GIT_SSH_COMMAND=\"ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts -i $HOME/.ssh/id_ed25519 -o 'ProxyCommand=$git_ssh_proxy %h %p'\"",
|
||||
"echo backend_core_artifact_source_proxy=provider-gateway-ws-egress:$proxy_url",
|
||||
"echo backend_core_artifact_repo_fetch_url=$repo_fetch_url",
|
||||
"if [ ! -d \"$repo_cache\" ]; then git clone --mirror \"$repo_fetch_url\" \"$repo_cache\"; fi",
|
||||
"git -C \"$repo_cache\" remote set-url origin \"$repo_fetch_url\"",
|
||||
"git -C \"$repo_cache\" fetch --no-tags origin \"$commit\" || git -C \"$repo_cache\" fetch --no-tags origin '+refs/heads/*:refs/remotes/origin/*'",
|
||||
"resolved=$(git -C \"$repo_cache\" rev-parse --verify \"$commit^{commit}\")",
|
||||
"test \"$resolved\" = \"$commit\" || { echo \"backend_core_artifact_resolved_commit_mismatch=$resolved expected=$commit\" >&2; exit 1; }",
|
||||
"git -C \"$repo_cache\" cat-file -e \"$commit:src/components/backend-core/Dockerfile\"",
|
||||
"git -C \"$repo_cache\" cat-file -e \"$commit:src/components/backend-core/src\"",
|
||||
"tmp_dir=\"$source_root/.tmp-$commit-$$\"",
|
||||
"rm -rf \"$tmp_dir\"",
|
||||
"mkdir -p \"$tmp_dir\"",
|
||||
"git -C \"$repo_cache\" archive \"$commit\" | tar -x -C \"$tmp_dir\"",
|
||||
"printf '%s\\n' \"$commit\" > \"$tmp_dir/.unidesk-source-commit\"",
|
||||
"printf '%s\\n' \"$repo_url\" > \"$tmp_dir/.unidesk-source-repo\"",
|
||||
"rm -rf \"$source_dir\"",
|
||||
"mv \"$tmp_dir\" \"$source_dir\"",
|
||||
"test -f \"$source_dir/src/components/backend-core/Dockerfile\"",
|
||||
"test -d \"$source_dir/src/components/backend-core/src\"",
|
||||
"echo backend_core_artifact_source_host_path=$source_dir",
|
||||
].join("\n");
|
||||
const result = await runRemoteBackground("prepare-backend-core-source", script, 300_000);
|
||||
if (!result.ok) throw new Error(`failed to prepare backend-core source on D601: ${result.stderr || result.stdout || JSON.stringify(result.raw)}`);
|
||||
return {
|
||||
ok: true,
|
||||
mode: "d601-host-github-ssh-export",
|
||||
providerId: d601ProviderId,
|
||||
repoUrl: options.repoUrl,
|
||||
repoFetchUrl,
|
||||
commit: options.commit,
|
||||
sourceHostPath,
|
||||
identity: {
|
||||
fingerprint: sshIdentity.fingerprint,
|
||||
seededFromLocal: sshIdentity.seededFromLocal,
|
||||
},
|
||||
stdoutTail: result.stdout.slice(-4000),
|
||||
};
|
||||
}
|
||||
|
||||
async function remoteCreatePipelineRun(manifest: string): Promise<string> {
|
||||
const encoded = Buffer.from(manifest, "utf8").toString("base64");
|
||||
const token = randomUUID().replace(/-/gu, "").slice(0, 12);
|
||||
@@ -536,18 +634,19 @@ async function waitForPipelineRun(name: string, waitMs: number): Promise<Dispatc
|
||||
const command = [
|
||||
"set -euo pipefail",
|
||||
`export KUBECONFIG=${shellQuote(d601Kubeconfig)}`,
|
||||
`printf 'waiting_pipelinerun=%s\\n' ${shellQuote(name)}`,
|
||||
`deadline=$((SECONDS + ${Math.ceil(waitMs / 1000)}))`,
|
||||
"while [ \"$SECONDS\" -lt \"$deadline\" ]; do",
|
||||
` condition="$(kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o jsonpath='{range .status.conditions[?(@.type==\"Succeeded\")]}{.status}{\"\\t\"}{.reason}{\"\\t\"}{.message}{end}' 2>/dev/null || true)"`,
|
||||
" case \"$condition\" in",
|
||||
" True*)",
|
||||
" echo \"$condition\"",
|
||||
` kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o json`,
|
||||
" printf 'condition=%s\\n' \"$condition\"",
|
||||
` kubectl get taskrun -n unidesk-ci -l tekton.dev/pipelineRun=${shellQuote(name)} --no-headers 2>/dev/null || true`,
|
||||
" exit 0",
|
||||
" ;;",
|
||||
" False*)",
|
||||
" echo \"$condition\"",
|
||||
` kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o json`,
|
||||
" printf 'condition=%s\\n' \"$condition\"",
|
||||
` kubectl get taskrun,pod -n unidesk-ci -l tekton.dev/pipelineRun=${shellQuote(name)} 2>/dev/null || true`,
|
||||
" exit 1",
|
||||
" ;;",
|
||||
" esac",
|
||||
@@ -560,10 +659,41 @@ async function waitForPipelineRun(name: string, waitMs: number): Promise<Dispatc
|
||||
return dispatchSsh(command, waitMs + 30_000, waitMs + 20_000);
|
||||
}
|
||||
|
||||
async function readPipelineRunCondition(name: string): Promise<PipelineRunCondition> {
|
||||
const result = await runRemoteKubectlRaw([
|
||||
"set -euo pipefail",
|
||||
`condition="$(kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o jsonpath='{range .status.conditions[?(@.type==\"Succeeded\")]}{.status}{\"\\t\"}{.reason}{\"\\t\"}{.message}{end}' 2>/dev/null || true)"`,
|
||||
"printf '%s\\n' \"$condition\"",
|
||||
].join("\n"), 30_000, 15_000);
|
||||
const [status = "", reason = "", ...messageParts] = result.stdout.trim().split("\t");
|
||||
const message = messageParts.join("\t");
|
||||
return {
|
||||
ok: status === "True" ? true : status === "False" ? false : null,
|
||||
status,
|
||||
reason,
|
||||
message,
|
||||
query: {
|
||||
ok: result.ok,
|
||||
status: result.status,
|
||||
exitCode: result.exitCode,
|
||||
stdoutTail: result.stdout.slice(-1200),
|
||||
stderrTail: result.stderr.slice(-1200),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function pipelineRunWaitSucceeded(wait: DispatchResult | null, condition: PipelineRunCondition | null): boolean {
|
||||
if (wait === null) return true;
|
||||
if (condition?.ok === true) return true;
|
||||
if (condition?.ok === false) return false;
|
||||
return wait.ok || wait.exitCode === 0 || wait.stdout.includes("condition=True\tSucceeded\t");
|
||||
}
|
||||
|
||||
async function run(options: CiOptions): Promise<Record<string, unknown>> {
|
||||
const name = await remoteCreatePipelineRun(pipelineRunManifest(options));
|
||||
const wait = await waitForPipelineRun(name, options.waitMs);
|
||||
const waitSucceeded = wait === null || wait.exitCode === 0 || wait.stdout.trimStart().startsWith("True\tSucceeded\t");
|
||||
const condition = wait === null ? null : await readPipelineRunCondition(name);
|
||||
const waitSucceeded = pipelineRunWaitSucceeded(wait, condition);
|
||||
return {
|
||||
ok: waitSucceeded,
|
||||
pipelineRun: name,
|
||||
@@ -571,9 +701,14 @@ async function run(options: CiOptions): Promise<Record<string, unknown>> {
|
||||
repoUrl: options.repoUrl,
|
||||
revision: options.revision,
|
||||
wait: wait === null ? null : {
|
||||
ok: waitSucceeded,
|
||||
dispatchOk: wait.ok,
|
||||
dispatchStatus: wait.status,
|
||||
dispatchExitCode: wait.exitCode,
|
||||
stdoutTail: wait.stdout.slice(-6000),
|
||||
stderrTail: wait.stderr.slice(-6000),
|
||||
},
|
||||
condition,
|
||||
next: [
|
||||
`bun scripts/cli.ts ci logs ${name}`,
|
||||
"bun scripts/cli.ts ci status",
|
||||
@@ -581,22 +716,30 @@ async function run(options: CiOptions): Promise<Record<string, unknown>> {
|
||||
};
|
||||
}
|
||||
|
||||
async function publishBackendCoreArtifact(options: CiPublishBackendCoreOptions): Promise<Record<string, unknown>> {
|
||||
async function publishBackendCoreArtifact(config: UniDeskConfig, options: CiPublishBackendCoreOptions): Promise<Record<string, unknown>> {
|
||||
const source = await prepareBackendCoreArtifactSource(config, options);
|
||||
const name = await remoteCreatePipelineRun(backendCoreArtifactPipelineRunManifest(options));
|
||||
const wait = await waitForPipelineRun(name, options.waitMs);
|
||||
const waitSucceeded = wait === null || wait.exitCode === 0 || wait.stdout.trimStart().startsWith("True\tSucceeded\t");
|
||||
const condition = wait === null ? null : await readPipelineRunCondition(name);
|
||||
const waitSucceeded = pipelineRunWaitSucceeded(wait, condition);
|
||||
return {
|
||||
ok: waitSucceeded,
|
||||
pipelineRun: name,
|
||||
namespace: "unidesk-ci",
|
||||
repoUrl: options.repoUrl,
|
||||
commit: options.commit,
|
||||
source,
|
||||
artifact: `127.0.0.1:5000/unidesk/backend-core:${options.commit}`,
|
||||
boundary: "CI publishes the image to D601 registry; CD must pull it and must not build backend-core",
|
||||
wait: wait === null ? null : {
|
||||
ok: waitSucceeded,
|
||||
dispatchOk: wait.ok,
|
||||
dispatchStatus: wait.status,
|
||||
dispatchExitCode: wait.exitCode,
|
||||
stdoutTail: wait.stdout.slice(-6000),
|
||||
stderrTail: wait.stderr.slice(-6000),
|
||||
},
|
||||
condition,
|
||||
next: [
|
||||
`bun scripts/cli.ts ci logs ${name}`,
|
||||
`bun scripts/cli.ts artifact-registry deploy-backend-core --commit ${options.commit}`,
|
||||
@@ -890,7 +1033,7 @@ function requireRunId(value: string): string {
|
||||
return value;
|
||||
}
|
||||
|
||||
export async function runCiCommand(_config: UniDeskConfig, args: string[]): Promise<Record<string, unknown>> {
|
||||
export async function runCiCommand(config: UniDeskConfig, args: string[]): Promise<Record<string, unknown>> {
|
||||
const [action = "status", nameArg] = args;
|
||||
if (isHelpArg(action) || args.slice(1).some(isHelpArg)) return ciHelp();
|
||||
if (action === "install") return install();
|
||||
@@ -905,7 +1048,7 @@ export async function runCiCommand(_config: UniDeskConfig, args: string[]): Prom
|
||||
const repoUrl = stringOption(args, "--repo") ?? stringOption(args, "--repo-url") ?? "https://github.com/pikasTech/unidesk";
|
||||
const commit = requireFullCommit(stringOption(args, "--commit") ?? stringOption(args, "--revision"));
|
||||
const waitMs = numberOption(args, "--wait-ms", 0);
|
||||
return publishBackendCoreArtifact({ repoUrl, commit, waitMs });
|
||||
return publishBackendCoreArtifact(config, { repoUrl, commit, waitMs, sourceHostPath: backendCoreArtifactSourceHostPath(commit) });
|
||||
}
|
||||
if (action === "run-dev-e2e") {
|
||||
const repoUrl = stringOption(args, "--repo") ?? stringOption(args, "--repo-url") ?? "https://github.com/pikasTech/unidesk";
|
||||
|
||||
@@ -21,6 +21,7 @@ interface GithubSshIdentity {
|
||||
const identityId = "github.com";
|
||||
const defaultPrivateKeyPath = "/root/.ssh/id_ed25519";
|
||||
const defaultKnownHostsPath = "/root/.ssh/known_hosts";
|
||||
const providerGatewayWsEgressProxyUrl = "http://127.0.0.1:18789";
|
||||
|
||||
function pgLiteral(value: string): string {
|
||||
return `'${value.replace(/'/gu, "''")}'`;
|
||||
@@ -255,6 +256,66 @@ function shellQuote(value: string): string {
|
||||
return `'${value.replace(/'/gu, `'\\''`)}'`;
|
||||
}
|
||||
|
||||
export function gitSshHttpConnectProxySource(): string {
|
||||
return String.raw`#!/usr/bin/env python3
|
||||
import os
|
||||
import select
|
||||
import socket
|
||||
import sys
|
||||
from urllib.parse import urlparse
|
||||
|
||||
if len(sys.argv) != 3:
|
||||
raise SystemExit("usage: unidesk-git-ssh-http-connect.py host port")
|
||||
|
||||
target_host = sys.argv[1]
|
||||
target_port = int(sys.argv[2])
|
||||
proxy = urlparse(os.environ.get("UNIDESK_GIT_SSH_HTTP_PROXY", "http://127.0.0.1:18789"))
|
||||
proxy_host = proxy.hostname or "127.0.0.1"
|
||||
proxy_port = proxy.port or 80
|
||||
|
||||
sock = socket.create_connection((proxy_host, proxy_port), timeout=20)
|
||||
sock.sendall(f"CONNECT {target_host}:{target_port} HTTP/1.1\r\nHost: {target_host}:{target_port}\r\n\r\n".encode("ascii"))
|
||||
header = b""
|
||||
while b"\r\n\r\n" not in header:
|
||||
chunk = sock.recv(4096)
|
||||
if not chunk:
|
||||
raise SystemExit("proxy closed before CONNECT response")
|
||||
header += chunk
|
||||
head, rest = header.split(b"\r\n\r\n", 1)
|
||||
if not (head.startswith(b"HTTP/1.1 200") or head.startswith(b"HTTP/1.0 200")):
|
||||
sys.stderr.write(head.decode("latin1", "replace") + "\n")
|
||||
raise SystemExit(1)
|
||||
if rest:
|
||||
os.write(1, rest)
|
||||
|
||||
stdin_open = True
|
||||
sock.setblocking(False)
|
||||
while True:
|
||||
readers = [sock]
|
||||
if stdin_open:
|
||||
readers.append(sys.stdin.buffer)
|
||||
ready, _, _ = select.select(readers, [], [])
|
||||
if sock in ready:
|
||||
try:
|
||||
data = sock.recv(65536)
|
||||
except BlockingIOError:
|
||||
data = b""
|
||||
if not data:
|
||||
break
|
||||
os.write(1, data)
|
||||
if stdin_open and sys.stdin.buffer in ready:
|
||||
data = os.read(0, 65536)
|
||||
if data:
|
||||
sock.sendall(data)
|
||||
else:
|
||||
stdin_open = False
|
||||
try:
|
||||
sock.shutdown(socket.SHUT_WR)
|
||||
except OSError:
|
||||
pass
|
||||
`;
|
||||
}
|
||||
|
||||
function distributeGithubIdentity(providerId: string, identity: GithubSshIdentity): { ok: boolean; detail: string; raw: unknown } {
|
||||
const payload = JSON.stringify({
|
||||
privateKey: identity.privateKey,
|
||||
@@ -262,12 +323,23 @@ function distributeGithubIdentity(providerId: string, identity: GithubSshIdentit
|
||||
knownHosts: identity.knownHosts,
|
||||
});
|
||||
const remotePython = remoteInstallPythonSource();
|
||||
const proxyPython = gitSshHttpConnectProxySource();
|
||||
const remoteCommand = [
|
||||
"set -euo pipefail",
|
||||
`python3 -c ${shellQuote(remotePython)}`,
|
||||
"auth_output=$(ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -T git@github.com 2>&1 || true)",
|
||||
`proxy_url=${shellQuote(providerGatewayWsEgressProxyUrl)}`,
|
||||
"curl -fsSI --max-time 20 -x \"$proxy_url\" https://github.com >/dev/null",
|
||||
"git_ssh_proxy=/tmp/unidesk-git-ssh-http-connect.py",
|
||||
"cat > \"$git_ssh_proxy\" <<'UNIDESK_GIT_SSH_PROXY'",
|
||||
proxyPython,
|
||||
"UNIDESK_GIT_SSH_PROXY",
|
||||
"chmod 700 \"$git_ssh_proxy\"",
|
||||
"export UNIDESK_GIT_SSH_HTTP_PROXY=\"$proxy_url\"",
|
||||
"export GIT_SSH_COMMAND=\"ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts -i $HOME/.ssh/id_ed25519 -o 'ProxyCommand=$git_ssh_proxy %h %p'\"",
|
||||
"auth_output=$(ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts -i $HOME/.ssh/id_ed25519 -o \"ProxyCommand=$git_ssh_proxy %h %p\" -T git@github.com 2>&1 || true)",
|
||||
"printf '%s\\n' \"$auth_output\"",
|
||||
"printf '%s\\n' \"$auth_output\" | grep -q 'successfully authenticated'",
|
||||
].join(" && ");
|
||||
].join("\n");
|
||||
const result = spawnSync(process.execPath, [
|
||||
rootPath("scripts", "cli.ts"),
|
||||
"ssh",
|
||||
|
||||
@@ -276,6 +276,8 @@ spec:
|
||||
- name: registry
|
||||
type: string
|
||||
default: 127.0.0.1:5000
|
||||
- name: source-host-path
|
||||
type: string
|
||||
workspaces:
|
||||
- name: source
|
||||
volumes:
|
||||
@@ -283,26 +285,17 @@ spec:
|
||||
hostPath:
|
||||
path: /var/run/docker.sock
|
||||
type: Socket
|
||||
- name: prepared-source
|
||||
hostPath:
|
||||
path: /home/ubuntu/.unidesk/ci/backend-core-artifacts
|
||||
type: Directory
|
||||
steps:
|
||||
- name: clone
|
||||
- name: prepare-source
|
||||
image: alpine/git:2.45.2
|
||||
env:
|
||||
- name: HTTP_PROXY
|
||||
value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789"
|
||||
- name: HTTPS_PROXY
|
||||
value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789"
|
||||
- name: ALL_PROXY
|
||||
value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789"
|
||||
- name: NO_PROXY
|
||||
value: "localhost,127.0.0.1,::1,host.docker.internal,d601-provider-egress-proxy,d601-provider-egress-proxy.unidesk,d601-provider-egress-proxy.unidesk.svc,d601-provider-egress-proxy.unidesk.svc.cluster.local"
|
||||
- name: http_proxy
|
||||
value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789"
|
||||
- name: https_proxy
|
||||
value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789"
|
||||
- name: all_proxy
|
||||
value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789"
|
||||
- name: no_proxy
|
||||
value: "localhost,127.0.0.1,::1,host.docker.internal,d601-provider-egress-proxy,d601-provider-egress-proxy.unidesk,d601-provider-egress-proxy.unidesk.svc,d601-provider-egress-proxy.unidesk.svc.cluster.local"
|
||||
volumeMounts:
|
||||
- name: prepared-source
|
||||
mountPath: /prepared-source
|
||||
readOnly: true
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
@@ -310,16 +303,22 @@ spec:
|
||||
[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]) ;;
|
||||
*) echo "backend_core_artifact_revision_must_be_full_sha=$(params.revision)" >&2; exit 2 ;;
|
||||
esac
|
||||
rm -rf "$(workspaces.source.path)/backend-core-artifact-repo"
|
||||
git clone --filter=blob:none "$(params.repo-url)" "$(workspaces.source.path)/backend-core-artifact-repo"
|
||||
mkdir -p "$(workspaces.source.path)/backend-core-artifact-repo"
|
||||
find "$(workspaces.source.path)/backend-core-artifact-repo" -mindepth 1 -maxdepth 1 -exec rm -rf {} +
|
||||
case "$(params.source-host-path)" in
|
||||
/home/ubuntu/.unidesk/ci/backend-core-artifacts/*) ;;
|
||||
*) echo "backend_core_artifact_source_host_path_invalid=$(params.source-host-path)" >&2; exit 2 ;;
|
||||
esac
|
||||
source_name="$(basename "$(params.source-host-path)")"
|
||||
source_dir="/prepared-source/$source_name"
|
||||
test -f "$source_dir/.unidesk-source-commit"
|
||||
prepared_commit="$(cat "$source_dir/.unidesk-source-commit")"
|
||||
test "$prepared_commit" = "$(params.revision)"
|
||||
cp -a "$source_dir/." "$(workspaces.source.path)/backend-core-artifact-repo/"
|
||||
cd "$(workspaces.source.path)/backend-core-artifact-repo"
|
||||
git fetch --depth=1 origin "$(params.revision)"
|
||||
git checkout --detach FETCH_HEAD
|
||||
resolved="$(git rev-parse HEAD)"
|
||||
test "$resolved" = "$(params.revision)"
|
||||
git cat-file -e "$resolved:src/components/backend-core/Dockerfile"
|
||||
git cat-file -e "$resolved:src/components/backend-core/src"
|
||||
git rev-parse HEAD | tee "$(workspaces.source.path)/backend-core-artifact-commit.txt"
|
||||
test -f src/components/backend-core/Dockerfile
|
||||
test -d src/components/backend-core/src
|
||||
printf '%s\n' "$prepared_commit" | tee "$(workspaces.source.path)/backend-core-artifact-commit.txt"
|
||||
- name: build-and-push
|
||||
image: "$(params.app-image)"
|
||||
imagePullPolicy: Never
|
||||
@@ -354,13 +353,10 @@ spec:
|
||||
local_image="unidesk/backend-core:$commit"
|
||||
registry_image="$registry/unidesk/backend-core:$commit"
|
||||
command -v docker
|
||||
docker buildx version >/dev/null 2>&1 || docker buildx inspect default >/dev/null 2>&1
|
||||
docker version >/dev/null
|
||||
docker run --rm --network host rancher/mirrored-library-busybox:1.36.1 wget -q -O- "http://$registry/v2/" >/dev/null
|
||||
docker buildx build \
|
||||
--builder default \
|
||||
--load \
|
||||
DOCKER_BUILDKIT=0 docker build \
|
||||
--network host \
|
||||
--progress=plain \
|
||||
--build-arg HTTP_PROXY=http://127.0.0.1:18789 \
|
||||
--build-arg HTTPS_PROXY=http://127.0.0.1:18789 \
|
||||
--build-arg ALL_PROXY=http://127.0.0.1:18789 \
|
||||
@@ -401,6 +397,8 @@ spec:
|
||||
- name: registry
|
||||
type: string
|
||||
default: 127.0.0.1:5000
|
||||
- name: source-host-path
|
||||
type: string
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
tasks:
|
||||
@@ -416,6 +414,8 @@ spec:
|
||||
value: "$(params.app-image)"
|
||||
- name: registry
|
||||
value: "$(params.registry)"
|
||||
- name: source-host-path
|
||||
value: "$(params.source-host-path)"
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
|
||||
Reference in New Issue
Block a user