diff --git a/docs/reference/artifact-registry.md b/docs/reference/artifact-registry.md index 2c91385e..d474c88f 100644 --- a/docs/reference/artifact-registry.md +++ b/docs/reference/artifact-registry.md @@ -60,7 +60,7 @@ bun scripts/cli.ts artifact-registry deploy-backend-core --commit 真实 `install` 必须是幂等动作:创建远端目录,写入 CLI 渲染出的 config/compose/unit,执行 `systemctl daemon-reload`,启用并启动 `unidesk-artifact-registry.service`,然后运行与 `health` 相同的验收检查。若远端文件 hash 与期望不一致,install 可以覆盖由本 CLI 管理的 unit/config/compose,但不得删除 registry storage。 -`deploy-backend-core` 是 production backend-core 的 CD 入口。它必须先确认 D601 registry 中已经存在 `unidesk/backend-core:`,随后只执行短生命周期 relay、`docker pull`、retag、Compose `--no-build` recreate 和 live commit 验证;如果镜像不存在,应失败并要求先运行 CI artifact publication。 +`deploy-backend-core` 是 production backend-core 的 CD 入口。它必须先通过 CNCF Distribution HTTP API 确认 D601 registry 中已经存在 `unidesk/backend-core:`,随后通过 provider-gateway Host SSH 流式执行 `docker save | gzip`,在 master server 侧 `docker load`、retag、Compose `--no-build` recreate 和 live commit 验证;如果镜像不存在,应失败并要求先运行 CI artifact publication。 `status` 和 `health` 通过: @@ -100,7 +100,7 @@ docker compose -p unidesk-artifact-registry -f /home/ubuntu/.unidesk/artifact-re 2. D601 CI 从 pushed Git checkout 构建 `unidesk/backend-core:`。 3. CI 将镜像 push 到 `127.0.0.1:5000/unidesk/backend-core:`,并记录 image ref 与 digest。 4. CD 在 master server 上确认目标 commit/tag/digest 存在。 -5. master server 通过受控、短生命周期的 localhost relay 从 D601 registry 拉取 commit-pinned 镜像。 +5. master server 通过 provider-gateway Host SSH 从 D601 registry 流式读取 commit-pinned 镜像 tar,不开放 registry 端口,也不使用第三方镜像托管。 6. master server retag 为 Compose 使用的 backend-core 镜像名,并执行 `docker compose up -d --no-build --no-deps --force-recreate backend-core`。 7. 部署后通过 image label、runtime env、health payload 验证 live commit。 @@ -108,7 +108,7 @@ docker compose -p unidesk-artifact-registry -f /home/ubuntu/.unidesk/artifact-re - source commit 来自 pushed Git,不来自 dirty worktree。 - 镜像 tag 必须 commit-pinned,不能用 mutable latest 作为部署真相。 -- relay 是临时控制动作,不开放长期公网 registry。 +- provider-gateway SSH image stream 是临时控制动作,不开放长期公网 registry。 - CI 可以有较多依赖;CD 只做拉取、retag、recreate 和 live commit 验证。 - CD 不执行 `cargo build`、`docker build`、`docker compose build backend-core` 或任何等价的 Rust 构建动作。 - `server rebuild backend-core` 仍不得作为 master server Rust 编译路径。 diff --git a/docs/reference/ci.md b/docs/reference/ci.md index e9fd3164..b5accea1 100644 --- a/docs/reference/ci.md +++ b/docs/reference/ci.md @@ -26,7 +26,7 @@ Each commit CI run performs: `ci install` also prewarms the D601 k3s containerd runtime with the Tekton entrypoint/workingdir helper images, `oven/bun:1-debian`, `alpine/git:2.45.2` and `unidesk-code-queue:dev`. Missing images are pulled through the node-local provider-gateway WS egress proxy and then imported into native k3s containerd with digests preserved, so PipelineRun pods do not hang on external registry pulls. Sustained pull throughput below 1 MB/s is treated as a provider/main-server network or proxy degradation first, not as a Dockerfile or application failure. -Git clone and dependency downloads inside the repo check task use `d601-provider-egress-proxy.unidesk.svc.cluster.local:18789`; the NO_PROXY list keeps the in-cluster read service, D601 TCP egress gateway and any in-cluster CI Git mirror on the cluster network. +Git clone and dependency downloads inside the repo check task use `d601-provider-egress-proxy.unidesk.svc.cluster.local:18789`; the NO_PROXY list keeps the in-cluster read service and D601 TCP egress gateway on the cluster network. Private repository source authentication is part of the CI contract and follows `docs/reference/devops-hygiene.md`. If the repo-check task fails at `git clone` because credentials are unavailable, treat it as a CI infrastructure/auth gap, not as an application test result. @@ -53,6 +53,7 @@ backend-core production image creation belongs to D601 CI, not to master server The CI artifact task must follow these rules: - Input revision comes from pushed Git and is resolved to a full 40-character commit. A dirty worktree or unpushed local tree must never be used as the image source. +- Source fetch for this artifact uses the existing D601 GitHub SSH deploy identity and the node-local provider-gateway WS egress proxy at `http://127.0.0.1:18789`. D601 prepares a commit-pinned source export under `/home/ubuntu/.unidesk/ci/backend-core-artifacts/` before creating the PipelineRun; Tekton consumes that prepared source through a read-only hostPath and must not clone GitHub itself, mount GitHub credentials, use an in-cluster Git mirror, or accept an operator-uploaded source tree. - The source checkout, Rust build and Docker build run on D601 CI infrastructure. The master server must not run `cargo build`, `docker compose build backend-core` or `server rebuild backend-core` as part of production backend-core deployment. - The image is tagged with the source commit, for example `unidesk/backend-core:`, and pushed to the D601 artifact registry as `127.0.0.1:5000/unidesk/backend-core:`. - The image must carry at least `unidesk.ai/service-id=backend-core`, `unidesk.ai/source-repo`, `unidesk.ai/source-commit` and `unidesk.ai/dockerfile=src/components/backend-core/Dockerfile`. diff --git a/docs/reference/devops-hygiene.md b/docs/reference/devops-hygiene.md index 33bd35ab..389884af 100644 --- a/docs/reference/devops-hygiene.md +++ b/docs/reference/devops-hygiene.md @@ -44,6 +44,8 @@ Acceptable source access implementations are: - an in-cluster Git mirror managed by UniDesk; or - the same commit-pinned host-fetch boundary used by `ci run-dev-e2e`, where D601 fetches the manifest commit and passes only verified inputs into Tekton. +For backend-core artifact publication, the required implementation is the host-fetch boundary: D601 uses the existing GitHub SSH deploy identity plus the node-local provider-gateway WS egress proxy, exports the requested commit to `/home/ubuntu/.unidesk/ci/backend-core-artifacts/`, and passes only that verified source directory into Tekton. This path must not be replaced by an in-cluster Git mirror, a third-party source mirror, an operator local checkout, or Tekton-mounted GitHub credentials. + If a CI repo-check task fails at `git clone` because credentials are unavailable, classify it as a CI infrastructure/auth gap, not as an application test failure. ## Verification Priority diff --git a/scripts/src/artifact-registry.ts b/scripts/src/artifact-registry.ts index 24414ec8..33a02b8b 100644 --- a/scripts/src/artifact-registry.ts +++ b/scripts/src/artifact-registry.ts @@ -1,6 +1,4 @@ import { createHash } from "node:crypto"; -import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process"; -import { createServer, type AddressInfo, type Server, type Socket } from "node:net"; import { readFileSync, writeFileSync } from "node:fs"; import { runCommand, type CommandResult } from "./command"; import { readConfig, repoRoot, rootPath } from "./config"; @@ -24,7 +22,6 @@ interface ArtifactRegistryOptions { dryRun: boolean; runNow: boolean; commit: string | null; - localPullPort: number | null; targetImage: string; } @@ -61,7 +58,6 @@ const defaultOptions: ArtifactRegistryOptions = { dryRun: false, runNow: false, commit: null, - localPullPort: null, targetImage: "unidesk-backend-core", }; @@ -81,12 +77,6 @@ function positiveInt(value: string, option: string): number { return parsed; } -function optionalPort(value: string, option: string): number { - const parsed = positiveInt(value, option); - if (parsed > 65535) throw new Error(`${option} must be <= 65535`); - return parsed; -} - function absolutePath(value: string, option: string): string { if (!value.startsWith("/")) throw new Error(`${option} must be an absolute path`); if (value.includes("\n") || value.includes("\0")) throw new Error(`${option} must not contain control characters`); @@ -131,9 +121,6 @@ function parseOptions(args: string[]): ArtifactRegistryOptions { } else if (arg === "--commit") { options.commit = commitValue(requireValue(args, index, arg), arg); index += 1; - } else if (arg === "--local-pull-port") { - options.localPullPort = optionalPort(requireValue(args, index, arg), arg); - index += 1; } else if (arg === "--target-image") { options.targetImage = requireValue(args, index, arg); index += 1; @@ -154,6 +141,18 @@ function shellQuote(value: string): string { return `'${value.replace(/'/g, `'\\''`)}'`; } +function rootExecPrelude(): string { + return [ + "root_exec() {", + " if [ \"$(id -u)\" = \"0\" ]; then \"$@\"; return; fi", + " if sudo -n true >/dev/null 2>&1; then sudo -n \"$@\"; return; fi", + " if [ -x /mnt/c/Windows/System32/wsl.exe ]; then /mnt/c/Windows/System32/wsl.exe -u root -- \"$@\"; return; fi", + " echo 'artifact_registry_root_access=missing' >&2", + " return 1", + "}", + ].join("\n"); +} + function file(path: string, content: string, mode = "0644"): RenderedFile { return { path, mode, sha256: sha256(content), content }; } @@ -203,15 +202,16 @@ function systemdUnit(options: ArtifactRegistryOptions): string { return `[Unit] Description=UniDesk D601 Artifact Registry (CNCF Distribution) Documentation=https://github.com/pikasTech/unidesk/blob/master/docs/reference/artifact-registry.md -Requires=docker.service -After=docker.service network-online.target +After=network-online.target Wants=network-online.target +ConditionPathExists=/var/run/docker.sock [Service] Type=oneshot RemainAfterExit=yes WorkingDirectory=${options.baseDir} ExecStartPre=/bin/mkdir -p ${options.baseDir} ${options.storageDir} +ExecStartPre=/bin/sh -lc 'docker info >/dev/null' ExecStart=/bin/sh -lc 'docker compose -p ${options.composeProject} -f ${options.baseDir}/compose.yml up -d --remove-orphans' ExecStop=/bin/sh -lc 'docker compose -p ${options.composeProject} -f ${options.baseDir}/compose.yml down' TimeoutStartSec=120 @@ -490,17 +490,18 @@ function runReadonlyStatus(options: ArtifactRegistryOptions, healthMode: boolean function remoteWriteFileCommand(item: RenderedFile): string { const encoded = Buffer.from(item.content, "utf8").toString("base64"); + const rootOwned = item.path.startsWith("/etc/"); return [ `target=${shellQuote(item.path)}`, "tmp=$(mktemp /tmp/unidesk-artifact-registry.XXXXXX)", "trap 'rm -f \"$tmp\"' EXIT", `printf %s ${shellQuote(encoded)} | base64 -d > "$tmp"`, - "if [ \"$(id -u)\" = \"0\" ]; then", + `if [ ${rootOwned ? "1" : "0"} = 1 ]; then`, + " root_exec mkdir -p \"$(dirname \"$target\")\"", + ` root_exec install -m ${shellQuote(item.mode)} "$tmp" "$target"`, + "else", " mkdir -p \"$(dirname \"$target\")\"", ` install -m ${shellQuote(item.mode)} "$tmp" "$target"`, - "else", - " sudo mkdir -p \"$(dirname \"$target\")\"", - ` sudo install -m ${shellQuote(item.mode)} "$tmp" "$target"`, "fi", `echo ${shellQuote(`artifact_registry_file_written path=${item.path} sha256=${item.sha256}`)}`, ].join("\n"); @@ -513,10 +514,11 @@ function install(options: ArtifactRegistryOptions): Record { "command -v docker >/dev/null", "docker compose version >/dev/null", "command -v systemctl >/dev/null", + rootExecPrelude(), `mkdir -p ${shellQuote(bundle.paths.baseDir)} ${shellQuote(bundle.paths.storage)}`, ...bundle.files.map(remoteWriteFileCommand), - "if [ \"$(id -u)\" = \"0\" ]; then systemctl daemon-reload; else sudo systemctl daemon-reload; fi", - `if [ "$(id -u)" = "0" ]; then systemctl enable --now ${shellQuote(options.unitName)}; else sudo systemctl enable --now ${shellQuote(options.unitName)}; fi`, + "root_exec systemctl daemon-reload", + `root_exec systemctl enable --now ${shellQuote(options.unitName)}`, "sleep 2", `curl -fsS http://${options.host}:${options.port}/v2/ >/dev/null`, ].join("\n"); @@ -559,91 +561,6 @@ function installDryRun(options: ArtifactRegistryOptions): Record Promise; -}> { - const children = new Set(); - const sockets = new Set(); - const remoteProxyScript = String.raw` -import select -import socket -import sys - -target = socket.create_connection(("127.0.0.1", 5000), timeout=10) -target.setblocking(False) -sys.stdin.buffer.raw.fileno() -sys.stdout.buffer.raw.fileno() -stdin_fd = sys.stdin.fileno() -stdout_fd = sys.stdout.fileno() -target_fd = target.fileno() - -while True: - readable, _, _ = select.select([stdin_fd, target_fd], [], []) - if stdin_fd in readable: - data = sys.stdin.buffer.read1(65536) - if not data: - try: - target.shutdown(socket.SHUT_WR) - except OSError: - pass - else: - target.sendall(data) - if target_fd in readable: - try: - data = target.recv(65536) - except BlockingIOError: - data = b"" - if not data: - break - sys.stdout.buffer.write(data) - sys.stdout.buffer.flush() -`; - const server = createServer((socket) => { - sockets.add(socket); - socket.on("close", () => sockets.delete(socket)); - const child = spawn(process.execPath, [ - "scripts/cli.ts", - "ssh", - options.providerId, - "argv", - "python3", - "-u", - "-c", - remoteProxyScript, - ], { - cwd: repoRoot, - stdio: ["pipe", "pipe", "pipe"], - }); - children.add(child); - child.on("close", () => children.delete(child)); - child.on("error", () => socket.destroy()); - socket.pipe(child.stdin); - child.stdout.pipe(socket); - child.stderr.on("data", (chunk) => process.stderr.write(`[artifact-registry-relay] ${chunk.toString("utf8")}`)); - socket.on("close", () => child.kill()); - }); - const listen = new Promise((resolve, reject) => { - server.once("error", reject); - server.listen(options.localPullPort ?? 0, "127.0.0.1", () => { - server.off("error", reject); - const address = server.address() as AddressInfo; - resolve(address.port); - }); - }); - const port = await listen; - return { - server, - port, - close: async () => { - for (const socket of sockets) socket.destroy(); - for (const child of children) child.kill(); - await new Promise((resolve) => server.close(() => resolve())); - }, - }; -} - function composeLockScript(innerScript: string): string { const lockDir = rootPath(".state", "locks"); const lockPath = rootPath(".state", "locks", "server-compose.lock"); @@ -670,6 +587,31 @@ function upsertEnvFileValues(path: string, values: Record): void writeFileSync(path, `${lines.join("\n")}\n`, "utf8"); } +function pullBackendCoreArtifactFromD601(options: ArtifactRegistryOptions, sourceImage: string): CommandResult { + const remoteScript = [ + "set -euo pipefail", + `image=${shellQuote(sourceImage)}`, + "docker pull -q \"$image\" >/dev/null", + "docker image inspect \"$image\" --format 'remote_source={{ index .Config.Labels \"unidesk.ai/source-commit\" }} remote_service={{ index .Config.Labels \"unidesk.ai/service-id\" }}' >&2", + "docker save \"$image\" | gzip -1", + ].join("\n"); + const sshCommand = [ + process.execPath, + rootPath("scripts", "cli.ts"), + "ssh", + options.providerId, + "argv", + "bash", + "-lc", + remoteScript, + ].map(shellQuote).join(" "); + const pipeline = [ + "set -euo pipefail", + `${sshCommand} | gzip -dc | docker load`, + ].join("\n"); + return runCommand(["bash", "-lc", pipeline], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 900_000) }); +} + async function deployBackendCoreNow(options: ArtifactRegistryOptions): Promise> { const commit = options.commit; if (commit === null) throw new Error("artifact-registry deploy-backend-core requires --commit "); @@ -682,11 +624,13 @@ async function deployBackendCoreNow(options: ArtifactRegistryOptions): Promise/tmp/unidesk-backend-core-manifest.json", - "manifest_digest=$(docker manifest inspect --verbose \"$registry_image\" 2>/dev/null | awk -F'\"' '/Digest/ {print $4; exit}' || true)", - "printf 'registry_image=%s\\nmanifest_digest=%s\\n' \"$registry_image\" \"$manifest_digest\"", + `manifest_url=${shellQuote(`http://127.0.0.1:${options.port}/v2/unidesk/backend-core/manifests/${commit}`)}`, + "headers=$(mktemp /tmp/unidesk-backend-core-manifest.XXXXXX.headers)", + "trap 'rm -f \"$headers\"' EXIT", + "curl -fsSI -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -D \"$headers\" -o /dev/null \"$manifest_url\"", + "manifest_digest=$(awk 'BEGIN{IGNORECASE=1} /^Docker-Content-Digest:/ {gsub(/\\r/, \"\", $2); print $2; exit}' \"$headers\")", + "printf 'registry_image=%s\\nmanifest_url=%s\\nmanifest_digest=%s\\n' \"$registry_image\" \"$manifest_url\" \"$manifest_digest\"", ].join("\n"); const registryProbe = runRemoteScript(options, registryProbeScript, Math.max(options.timeoutMs, 120_000)); if (registryProbe.exitCode !== 0 || registryProbe.timedOut) { @@ -698,110 +642,100 @@ async function deployBackendCoreNow(options: ArtifactRegistryOptions): Promise/dev/null || true)", - " echo \"backend_core_container_probe attempt=$attempt cid=$cid health=$health\"", - " if [ \"$health\" = \"healthy\" ] || [ \"$health\" = \"running\" ]; then ready=1; break; fi", - " else", - " echo \"backend_core_container_probe attempt=$attempt cid=missing\"", - " fi", - " sleep 1", - "done", - "test \"$ready\" = \"1\"", - "cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1)", - "image_id=$(docker inspect -f '{{.Image}}' \"$cid\")", - "actual_commit=$(docker image inspect -f '{{ index .Config.Labels \"unidesk.ai/source-commit\" }}' \"$image_id\")", - `test "$actual_commit" = ${shellQuote(commit)}`, - "docker exec \"$cid\" bun -e \"fetch('http://127.0.0.1:8080/health').then(async r=>{const text=await r.text(); console.log(text); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\" >/tmp/unidesk-backend-core-health.json", - "cat /tmp/unidesk-backend-core-health.json", - ].join("\n"); - const deploy = runCommand(["bash", "-lc", composeLockScript(upScript)], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 300_000) }); - if (deploy.exitCode !== 0 || deploy.timedOut) { - return { - ok: false, - step: "compose-recreate", - sourceImage, - targetImage: composeImage, - registryProbe: commandTail(registryProbe), - deploy: commandTail(deploy), - }; - } - const running = runCommand(["docker", "inspect", "unidesk-backend-core", "--format", "image={{.Config.Image}} imageId={{.Image}} status={{.State.Status}} health={{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}"], repoRoot); + const pull = pullBackendCoreArtifactFromD601(options, sourceImage); + if (pull.exitCode !== 0 || pull.timedOut) { return { - ok: true, - commit, - providerId: options.providerId, + ok: false, + step: "docker-load", sourceImage, registryProbe: commandTail(registryProbe), - localRelayImage: localSourceImage, - targetImage: composeImage, - targetCommitImage: commitImage, - relay: { - bind: `127.0.0.1:${relay.port}`, - persistent: false, - }, pull: commandTail(pull), - deploy: commandTail(deploy), - running: running.stdout.trim(), - rollback: { - previousImageHint: "Use docker image ls and docker inspect logs for the previous backend-core image id; Compose named volumes were not changed.", - commandShape: `${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`, - }, }; - } finally { - await relay.close(); } + const localLoadedImage = sourceImage; + const inspectPulled = runCommand(["docker", "image", "inspect", localLoadedImage, "--format", "{{json .Config.Labels}}"], repoRoot); + const labelCommit = runCommand(["docker", "image", "inspect", localLoadedImage, "--format", "{{ index .Config.Labels \"unidesk.ai/source-commit\" }}"], repoRoot); + const labelService = runCommand(["docker", "image", "inspect", localLoadedImage, "--format", "{{ index .Config.Labels \"unidesk.ai/service-id\" }}"], repoRoot); + if (labelCommit.stdout.trim() !== commit || labelService.stdout.trim() !== "backend-core") { + return { + ok: false, + step: "image-label-verify", + expected: { commit, serviceId: "backend-core" }, + observed: { commit: labelCommit.stdout.trim(), serviceId: labelService.stdout.trim(), labels: inspectPulled.stdout.trim() }, + registryProbe: commandTail(registryProbe), + }; + } + const tag = runCommand(["docker", "tag", localLoadedImage, composeImage], repoRoot); + if (tag.exitCode !== 0 || tag.timedOut) { + return { ok: false, step: "docker-tag", targetImage: composeImage, tag: commandTail(tag), registryProbe: commandTail(registryProbe) }; + } + const tagCommit = runCommand(["docker", "tag", localLoadedImage, commitImage], repoRoot); + if (tagCommit.exitCode !== 0 || tagCommit.timedOut) { + return { ok: false, step: "docker-tag-commit", targetImage: commitImage, tag: commandTail(tagCommit), registryProbe: commandTail(registryProbe) }; + } + const config = readConfig(); + const runtimeEnv = writeComposeEnv(config, false); + upsertEnvFileValues(runtimeEnv.envFile, { + UNIDESK_DEPLOY_REF: "deploy.json#environments.prod.services.backend-core", + UNIDESK_DEPLOY_SERVICE_ID: "backend-core", + UNIDESK_DEPLOY_REPO: "https://github.com/pikasTech/unidesk", + UNIDESK_DEPLOY_COMMIT: commit, + UNIDESK_DEPLOY_REQUESTED_COMMIT: commit, + }); + const compose = resolveComposeCommand(config, runtimeEnv.envFile); + const upScript = [ + "set -euo pipefail", + `echo ${shellQuote(`backend_core_artifact_cd source=${sourceImage} target=${composeImage}`)}`, + `${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`, + "ready=0", + "for attempt in $(seq 1 60); do", + " cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1 || true)", + " if [ -n \"$cid\" ]; then", + " health=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' \"$cid\" 2>/dev/null || true)", + " echo \"backend_core_container_probe attempt=$attempt cid=$cid health=$health\"", + " if [ \"$health\" = \"healthy\" ] || [ \"$health\" = \"running\" ]; then ready=1; break; fi", + " else", + " echo \"backend_core_container_probe attempt=$attempt cid=missing\"", + " fi", + " sleep 1", + "done", + "test \"$ready\" = \"1\"", + "cid=$(docker ps -q --filter label=com.docker.compose.project=unidesk --filter label=com.docker.compose.service=backend-core --filter label=com.docker.compose.oneoff=False | head -1)", + "image_id=$(docker inspect -f '{{.Image}}' \"$cid\")", + "actual_commit=$(docker image inspect -f '{{ index .Config.Labels \"unidesk.ai/source-commit\" }}' \"$image_id\")", + `test "$actual_commit" = ${shellQuote(commit)}`, + "docker exec \"$cid\" bun -e \"fetch('http://127.0.0.1:8080/health').then(async r=>{const text=await r.text(); console.log(text); process.exit(r.ok?0:1)}).catch(e=>{console.error(e); process.exit(1)})\" >/tmp/unidesk-backend-core-health.json", + "cat /tmp/unidesk-backend-core-health.json", + ].join("\n"); + const deploy = runCommand(["bash", "-lc", composeLockScript(upScript)], repoRoot, { timeoutMs: Math.max(options.timeoutMs, 300_000) }); + if (deploy.exitCode !== 0 || deploy.timedOut) { + return { + ok: false, + step: "compose-recreate", + sourceImage, + targetImage: composeImage, + registryProbe: commandTail(registryProbe), + deploy: commandTail(deploy), + }; + } + const running = runCommand(["docker", "inspect", "unidesk-backend-core", "--format", "image={{.Config.Image}} imageId={{.Image}} status={{.State.Status}} health={{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}"], repoRoot); + return { + ok: true, + commit, + providerId: options.providerId, + sourceImage, + registryProbe: commandTail(registryProbe), + localLoadedImage, + targetImage: composeImage, + targetCommitImage: commitImage, + pull: commandTail(pull), + deploy: commandTail(deploy), + running: running.stdout.trim(), + rollback: { + previousImageHint: "Use docker image ls and docker inspect logs for the previous backend-core image id; Compose named volumes were not changed.", + commandShape: `${compose.map(shellQuote).join(" ")} up -d --no-build --no-deps --force-recreate backend-core`, + }, + }; } function deployBackendCoreJob(args: string[], options: ArtifactRegistryOptions): Record { @@ -815,7 +749,7 @@ function deployBackendCoreJob(args: string[], options: ArtifactRegistryOptions): job, statusCommand: `bun scripts/cli.ts job status ${job.id}`, tailCommand: `bun scripts/cli.ts job status ${job.id} --tail-bytes 30000`, - note: "Backend-core CD continues in the background: D601 registry health, short-lived local relay, docker pull, retag, no-build recreate, live commit verification.", + note: "Backend-core CD continues in the background: D601 registry health, provider-gateway SSH image stream, docker load, retag, no-build recreate, live commit verification.", }; } diff --git a/scripts/src/ci.ts b/scripts/src/ci.ts index c38e894e..cfa36b7c 100644 --- a/scripts/src/ci.ts +++ b/scripts/src/ci.ts @@ -2,6 +2,7 @@ import { randomUUID } from "node:crypto"; import { existsSync, readFileSync } from "node:fs"; import { runCommand } from "./command"; import { type UniDeskConfig, repoRoot, rootPath } from "./config"; +import { ensureGithubSshIdentityForProvider, gitSshHttpConnectProxySource } from "./deploy-ssh-identity"; import { startJob } from "./jobs"; import { coreInternalFetch } from "./microservices"; @@ -38,6 +39,7 @@ interface CiPublishBackendCoreOptions { repoUrl: string; commit: string; waitMs: number; + sourceHostPath: string; } interface CiDevE2EOptions { @@ -64,6 +66,20 @@ interface DispatchResult { raw: unknown; } +interface PipelineRunCondition { + ok: boolean | null; + status: string; + reason: string; + message: string; + query: { + ok: boolean; + status: string | null; + exitCode: number | null; + stdoutTail: string; + stderrTail: string; + }; +} + interface DeployDevManifestSummary { deployCommit: string; desiredRef: string; @@ -124,6 +140,15 @@ function shellQuote(value: string): string { return `'${value.replace(/'/gu, "'\\''")}'`; } +function repoSshUrl(repoUrl: string): string { + if (repoUrl.startsWith("git@")) return repoUrl; + if (repoUrl.startsWith("https://github.com/")) { + const repoPath = repoUrl.slice("https://github.com/".length).replace(/\.git$/u, ""); + return `git@github.com:${repoPath}.git`; + } + return repoUrl; +} + function chunks(value: string, size: number): string[] { const result: string[] = []; for (let index = 0; index < value.length; index += size) { @@ -508,6 +533,8 @@ spec: value: ${JSON.stringify(options.repoUrl)} - name: revision value: ${JSON.stringify(options.commit)} + - name: source-host-path + value: ${JSON.stringify(options.sourceHostPath)} workspaces: - name: shared-workspace persistentVolumeClaim: @@ -515,6 +542,77 @@ spec: `; } +function backendCoreArtifactSourceHostPath(commit: string): string { + return `/home/ubuntu/.unidesk/ci/backend-core-artifacts/${commit}`; +} + +async function prepareBackendCoreArtifactSource(config: UniDeskConfig, options: CiPublishBackendCoreOptions): Promise> { + const sshIdentity = await ensureGithubSshIdentityForProvider(config, d601ProviderId); + if (!sshIdentity.ok) throw new Error(sshIdentity.detail); + const proxyPython = gitSshHttpConnectProxySource(); + const sourceRoot = "/home/ubuntu/.unidesk/ci/backend-core-artifacts"; + const sourceHostPath = options.sourceHostPath; + const repoCache = "/home/ubuntu/.unidesk/ci/git/unidesk.git"; + const repoFetchUrl = repoSshUrl(options.repoUrl); + const script = [ + "set -euo pipefail", + `commit=${shellQuote(options.commit)}`, + `repo_url=${shellQuote(options.repoUrl)}`, + `repo_fetch_url=${shellQuote(repoFetchUrl)}`, + `source_root=${shellQuote(sourceRoot)}`, + `source_dir=${shellQuote(sourceHostPath)}`, + `repo_cache=${shellQuote(repoCache)}`, + `proxy_url=${shellQuote(providerGatewayWsEgressProxyUrl)}`, + "mkdir -p \"$(dirname \"$repo_cache\")\" \"$source_root\"", + "export HTTP_PROXY=\"$proxy_url\" HTTPS_PROXY=\"$proxy_url\" ALL_PROXY=\"$proxy_url\"", + "export NO_PROXY=\"localhost,127.0.0.1,::1,host.docker.internal,.svc,.cluster.local,kubernetes.default.svc\"", + "curl -fsSI --max-time 20 -x \"$proxy_url\" https://github.com >/dev/null", + "git_ssh_proxy=/tmp/unidesk-git-ssh-http-connect.py", + "cat > \"$git_ssh_proxy\" <<'UNIDESK_GIT_SSH_PROXY'", + proxyPython, + "UNIDESK_GIT_SSH_PROXY", + "chmod 700 \"$git_ssh_proxy\"", + "export UNIDESK_GIT_SSH_HTTP_PROXY=\"$proxy_url\"", + "export GIT_SSH_COMMAND=\"ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts -i $HOME/.ssh/id_ed25519 -o 'ProxyCommand=$git_ssh_proxy %h %p'\"", + "echo backend_core_artifact_source_proxy=provider-gateway-ws-egress:$proxy_url", + "echo backend_core_artifact_repo_fetch_url=$repo_fetch_url", + "if [ ! -d \"$repo_cache\" ]; then git clone --mirror \"$repo_fetch_url\" \"$repo_cache\"; fi", + "git -C \"$repo_cache\" remote set-url origin \"$repo_fetch_url\"", + "git -C \"$repo_cache\" fetch --no-tags origin \"$commit\" || git -C \"$repo_cache\" fetch --no-tags origin '+refs/heads/*:refs/remotes/origin/*'", + "resolved=$(git -C \"$repo_cache\" rev-parse --verify \"$commit^{commit}\")", + "test \"$resolved\" = \"$commit\" || { echo \"backend_core_artifact_resolved_commit_mismatch=$resolved expected=$commit\" >&2; exit 1; }", + "git -C \"$repo_cache\" cat-file -e \"$commit:src/components/backend-core/Dockerfile\"", + "git -C \"$repo_cache\" cat-file -e \"$commit:src/components/backend-core/src\"", + "tmp_dir=\"$source_root/.tmp-$commit-$$\"", + "rm -rf \"$tmp_dir\"", + "mkdir -p \"$tmp_dir\"", + "git -C \"$repo_cache\" archive \"$commit\" | tar -x -C \"$tmp_dir\"", + "printf '%s\\n' \"$commit\" > \"$tmp_dir/.unidesk-source-commit\"", + "printf '%s\\n' \"$repo_url\" > \"$tmp_dir/.unidesk-source-repo\"", + "rm -rf \"$source_dir\"", + "mv \"$tmp_dir\" \"$source_dir\"", + "test -f \"$source_dir/src/components/backend-core/Dockerfile\"", + "test -d \"$source_dir/src/components/backend-core/src\"", + "echo backend_core_artifact_source_host_path=$source_dir", + ].join("\n"); + const result = await runRemoteBackground("prepare-backend-core-source", script, 300_000); + if (!result.ok) throw new Error(`failed to prepare backend-core source on D601: ${result.stderr || result.stdout || JSON.stringify(result.raw)}`); + return { + ok: true, + mode: "d601-host-github-ssh-export", + providerId: d601ProviderId, + repoUrl: options.repoUrl, + repoFetchUrl, + commit: options.commit, + sourceHostPath, + identity: { + fingerprint: sshIdentity.fingerprint, + seededFromLocal: sshIdentity.seededFromLocal, + }, + stdoutTail: result.stdout.slice(-4000), + }; +} + async function remoteCreatePipelineRun(manifest: string): Promise { const encoded = Buffer.from(manifest, "utf8").toString("base64"); const token = randomUUID().replace(/-/gu, "").slice(0, 12); @@ -536,18 +634,19 @@ async function waitForPipelineRun(name: string, waitMs: number): Promise/dev/null || true)"`, " case \"$condition\" in", " True*)", - " echo \"$condition\"", - ` kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o json`, + " printf 'condition=%s\\n' \"$condition\"", + ` kubectl get taskrun -n unidesk-ci -l tekton.dev/pipelineRun=${shellQuote(name)} --no-headers 2>/dev/null || true`, " exit 0", " ;;", " False*)", - " echo \"$condition\"", - ` kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o json`, + " printf 'condition=%s\\n' \"$condition\"", + ` kubectl get taskrun,pod -n unidesk-ci -l tekton.dev/pipelineRun=${shellQuote(name)} 2>/dev/null || true`, " exit 1", " ;;", " esac", @@ -560,10 +659,41 @@ async function waitForPipelineRun(name: string, waitMs: number): Promise { + const result = await runRemoteKubectlRaw([ + "set -euo pipefail", + `condition="$(kubectl get pipelinerun/${shellQuote(name)} -n unidesk-ci -o jsonpath='{range .status.conditions[?(@.type==\"Succeeded\")]}{.status}{\"\\t\"}{.reason}{\"\\t\"}{.message}{end}' 2>/dev/null || true)"`, + "printf '%s\\n' \"$condition\"", + ].join("\n"), 30_000, 15_000); + const [status = "", reason = "", ...messageParts] = result.stdout.trim().split("\t"); + const message = messageParts.join("\t"); + return { + ok: status === "True" ? true : status === "False" ? false : null, + status, + reason, + message, + query: { + ok: result.ok, + status: result.status, + exitCode: result.exitCode, + stdoutTail: result.stdout.slice(-1200), + stderrTail: result.stderr.slice(-1200), + }, + }; +} + +function pipelineRunWaitSucceeded(wait: DispatchResult | null, condition: PipelineRunCondition | null): boolean { + if (wait === null) return true; + if (condition?.ok === true) return true; + if (condition?.ok === false) return false; + return wait.ok || wait.exitCode === 0 || wait.stdout.includes("condition=True\tSucceeded\t"); +} + async function run(options: CiOptions): Promise> { const name = await remoteCreatePipelineRun(pipelineRunManifest(options)); const wait = await waitForPipelineRun(name, options.waitMs); - const waitSucceeded = wait === null || wait.exitCode === 0 || wait.stdout.trimStart().startsWith("True\tSucceeded\t"); + const condition = wait === null ? null : await readPipelineRunCondition(name); + const waitSucceeded = pipelineRunWaitSucceeded(wait, condition); return { ok: waitSucceeded, pipelineRun: name, @@ -571,9 +701,14 @@ async function run(options: CiOptions): Promise> { repoUrl: options.repoUrl, revision: options.revision, wait: wait === null ? null : { + ok: waitSucceeded, + dispatchOk: wait.ok, + dispatchStatus: wait.status, + dispatchExitCode: wait.exitCode, stdoutTail: wait.stdout.slice(-6000), stderrTail: wait.stderr.slice(-6000), }, + condition, next: [ `bun scripts/cli.ts ci logs ${name}`, "bun scripts/cli.ts ci status", @@ -581,22 +716,30 @@ async function run(options: CiOptions): Promise> { }; } -async function publishBackendCoreArtifact(options: CiPublishBackendCoreOptions): Promise> { +async function publishBackendCoreArtifact(config: UniDeskConfig, options: CiPublishBackendCoreOptions): Promise> { + const source = await prepareBackendCoreArtifactSource(config, options); const name = await remoteCreatePipelineRun(backendCoreArtifactPipelineRunManifest(options)); const wait = await waitForPipelineRun(name, options.waitMs); - const waitSucceeded = wait === null || wait.exitCode === 0 || wait.stdout.trimStart().startsWith("True\tSucceeded\t"); + const condition = wait === null ? null : await readPipelineRunCondition(name); + const waitSucceeded = pipelineRunWaitSucceeded(wait, condition); return { ok: waitSucceeded, pipelineRun: name, namespace: "unidesk-ci", repoUrl: options.repoUrl, commit: options.commit, + source, artifact: `127.0.0.1:5000/unidesk/backend-core:${options.commit}`, boundary: "CI publishes the image to D601 registry; CD must pull it and must not build backend-core", wait: wait === null ? null : { + ok: waitSucceeded, + dispatchOk: wait.ok, + dispatchStatus: wait.status, + dispatchExitCode: wait.exitCode, stdoutTail: wait.stdout.slice(-6000), stderrTail: wait.stderr.slice(-6000), }, + condition, next: [ `bun scripts/cli.ts ci logs ${name}`, `bun scripts/cli.ts artifact-registry deploy-backend-core --commit ${options.commit}`, @@ -890,7 +1033,7 @@ function requireRunId(value: string): string { return value; } -export async function runCiCommand(_config: UniDeskConfig, args: string[]): Promise> { +export async function runCiCommand(config: UniDeskConfig, args: string[]): Promise> { const [action = "status", nameArg] = args; if (isHelpArg(action) || args.slice(1).some(isHelpArg)) return ciHelp(); if (action === "install") return install(); @@ -905,7 +1048,7 @@ export async function runCiCommand(_config: UniDeskConfig, args: string[]): Prom const repoUrl = stringOption(args, "--repo") ?? stringOption(args, "--repo-url") ?? "https://github.com/pikasTech/unidesk"; const commit = requireFullCommit(stringOption(args, "--commit") ?? stringOption(args, "--revision")); const waitMs = numberOption(args, "--wait-ms", 0); - return publishBackendCoreArtifact({ repoUrl, commit, waitMs }); + return publishBackendCoreArtifact(config, { repoUrl, commit, waitMs, sourceHostPath: backendCoreArtifactSourceHostPath(commit) }); } if (action === "run-dev-e2e") { const repoUrl = stringOption(args, "--repo") ?? stringOption(args, "--repo-url") ?? "https://github.com/pikasTech/unidesk"; diff --git a/scripts/src/deploy-ssh-identity.ts b/scripts/src/deploy-ssh-identity.ts index 031a02ae..c989f1aa 100644 --- a/scripts/src/deploy-ssh-identity.ts +++ b/scripts/src/deploy-ssh-identity.ts @@ -21,6 +21,7 @@ interface GithubSshIdentity { const identityId = "github.com"; const defaultPrivateKeyPath = "/root/.ssh/id_ed25519"; const defaultKnownHostsPath = "/root/.ssh/known_hosts"; +const providerGatewayWsEgressProxyUrl = "http://127.0.0.1:18789"; function pgLiteral(value: string): string { return `'${value.replace(/'/gu, "''")}'`; @@ -255,6 +256,66 @@ function shellQuote(value: string): string { return `'${value.replace(/'/gu, `'\\''`)}'`; } +export function gitSshHttpConnectProxySource(): string { + return String.raw`#!/usr/bin/env python3 +import os +import select +import socket +import sys +from urllib.parse import urlparse + +if len(sys.argv) != 3: + raise SystemExit("usage: unidesk-git-ssh-http-connect.py host port") + +target_host = sys.argv[1] +target_port = int(sys.argv[2]) +proxy = urlparse(os.environ.get("UNIDESK_GIT_SSH_HTTP_PROXY", "http://127.0.0.1:18789")) +proxy_host = proxy.hostname or "127.0.0.1" +proxy_port = proxy.port or 80 + +sock = socket.create_connection((proxy_host, proxy_port), timeout=20) +sock.sendall(f"CONNECT {target_host}:{target_port} HTTP/1.1\r\nHost: {target_host}:{target_port}\r\n\r\n".encode("ascii")) +header = b"" +while b"\r\n\r\n" not in header: + chunk = sock.recv(4096) + if not chunk: + raise SystemExit("proxy closed before CONNECT response") + header += chunk +head, rest = header.split(b"\r\n\r\n", 1) +if not (head.startswith(b"HTTP/1.1 200") or head.startswith(b"HTTP/1.0 200")): + sys.stderr.write(head.decode("latin1", "replace") + "\n") + raise SystemExit(1) +if rest: + os.write(1, rest) + +stdin_open = True +sock.setblocking(False) +while True: + readers = [sock] + if stdin_open: + readers.append(sys.stdin.buffer) + ready, _, _ = select.select(readers, [], []) + if sock in ready: + try: + data = sock.recv(65536) + except BlockingIOError: + data = b"" + if not data: + break + os.write(1, data) + if stdin_open and sys.stdin.buffer in ready: + data = os.read(0, 65536) + if data: + sock.sendall(data) + else: + stdin_open = False + try: + sock.shutdown(socket.SHUT_WR) + except OSError: + pass +`; +} + function distributeGithubIdentity(providerId: string, identity: GithubSshIdentity): { ok: boolean; detail: string; raw: unknown } { const payload = JSON.stringify({ privateKey: identity.privateKey, @@ -262,12 +323,23 @@ function distributeGithubIdentity(providerId: string, identity: GithubSshIdentit knownHosts: identity.knownHosts, }); const remotePython = remoteInstallPythonSource(); + const proxyPython = gitSshHttpConnectProxySource(); const remoteCommand = [ + "set -euo pipefail", `python3 -c ${shellQuote(remotePython)}`, - "auth_output=$(ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -T git@github.com 2>&1 || true)", + `proxy_url=${shellQuote(providerGatewayWsEgressProxyUrl)}`, + "curl -fsSI --max-time 20 -x \"$proxy_url\" https://github.com >/dev/null", + "git_ssh_proxy=/tmp/unidesk-git-ssh-http-connect.py", + "cat > \"$git_ssh_proxy\" <<'UNIDESK_GIT_SSH_PROXY'", + proxyPython, + "UNIDESK_GIT_SSH_PROXY", + "chmod 700 \"$git_ssh_proxy\"", + "export UNIDESK_GIT_SSH_HTTP_PROXY=\"$proxy_url\"", + "export GIT_SSH_COMMAND=\"ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts -i $HOME/.ssh/id_ed25519 -o 'ProxyCommand=$git_ssh_proxy %h %p'\"", + "auth_output=$(ssh -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts -i $HOME/.ssh/id_ed25519 -o \"ProxyCommand=$git_ssh_proxy %h %p\" -T git@github.com 2>&1 || true)", "printf '%s\\n' \"$auth_output\"", "printf '%s\\n' \"$auth_output\" | grep -q 'successfully authenticated'", - ].join(" && "); + ].join("\n"); const result = spawnSync(process.execPath, [ rootPath("scripts", "cli.ts"), "ssh", diff --git a/src/components/microservices/k3sctl-adapter/k3s/ci/unidesk-ci.pipeline.yaml b/src/components/microservices/k3sctl-adapter/k3s/ci/unidesk-ci.pipeline.yaml index f7f78179..8601f9c7 100644 --- a/src/components/microservices/k3sctl-adapter/k3s/ci/unidesk-ci.pipeline.yaml +++ b/src/components/microservices/k3sctl-adapter/k3s/ci/unidesk-ci.pipeline.yaml @@ -276,6 +276,8 @@ spec: - name: registry type: string default: 127.0.0.1:5000 + - name: source-host-path + type: string workspaces: - name: source volumes: @@ -283,26 +285,17 @@ spec: hostPath: path: /var/run/docker.sock type: Socket + - name: prepared-source + hostPath: + path: /home/ubuntu/.unidesk/ci/backend-core-artifacts + type: Directory steps: - - name: clone + - name: prepare-source image: alpine/git:2.45.2 - env: - - name: HTTP_PROXY - value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789" - - name: HTTPS_PROXY - value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789" - - name: ALL_PROXY - value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789" - - name: NO_PROXY - value: "localhost,127.0.0.1,::1,host.docker.internal,d601-provider-egress-proxy,d601-provider-egress-proxy.unidesk,d601-provider-egress-proxy.unidesk.svc,d601-provider-egress-proxy.unidesk.svc.cluster.local" - - name: http_proxy - value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789" - - name: https_proxy - value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789" - - name: all_proxy - value: "http://d601-provider-egress-proxy.unidesk.svc.cluster.local:18789" - - name: no_proxy - value: "localhost,127.0.0.1,::1,host.docker.internal,d601-provider-egress-proxy,d601-provider-egress-proxy.unidesk,d601-provider-egress-proxy.unidesk.svc,d601-provider-egress-proxy.unidesk.svc.cluster.local" + volumeMounts: + - name: prepared-source + mountPath: /prepared-source + readOnly: true script: | #!/bin/sh set -eu @@ -310,16 +303,22 @@ spec: [0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]) ;; *) echo "backend_core_artifact_revision_must_be_full_sha=$(params.revision)" >&2; exit 2 ;; esac - rm -rf "$(workspaces.source.path)/backend-core-artifact-repo" - git clone --filter=blob:none "$(params.repo-url)" "$(workspaces.source.path)/backend-core-artifact-repo" + mkdir -p "$(workspaces.source.path)/backend-core-artifact-repo" + find "$(workspaces.source.path)/backend-core-artifact-repo" -mindepth 1 -maxdepth 1 -exec rm -rf {} + + case "$(params.source-host-path)" in + /home/ubuntu/.unidesk/ci/backend-core-artifacts/*) ;; + *) echo "backend_core_artifact_source_host_path_invalid=$(params.source-host-path)" >&2; exit 2 ;; + esac + source_name="$(basename "$(params.source-host-path)")" + source_dir="/prepared-source/$source_name" + test -f "$source_dir/.unidesk-source-commit" + prepared_commit="$(cat "$source_dir/.unidesk-source-commit")" + test "$prepared_commit" = "$(params.revision)" + cp -a "$source_dir/." "$(workspaces.source.path)/backend-core-artifact-repo/" cd "$(workspaces.source.path)/backend-core-artifact-repo" - git fetch --depth=1 origin "$(params.revision)" - git checkout --detach FETCH_HEAD - resolved="$(git rev-parse HEAD)" - test "$resolved" = "$(params.revision)" - git cat-file -e "$resolved:src/components/backend-core/Dockerfile" - git cat-file -e "$resolved:src/components/backend-core/src" - git rev-parse HEAD | tee "$(workspaces.source.path)/backend-core-artifact-commit.txt" + test -f src/components/backend-core/Dockerfile + test -d src/components/backend-core/src + printf '%s\n' "$prepared_commit" | tee "$(workspaces.source.path)/backend-core-artifact-commit.txt" - name: build-and-push image: "$(params.app-image)" imagePullPolicy: Never @@ -354,13 +353,10 @@ spec: local_image="unidesk/backend-core:$commit" registry_image="$registry/unidesk/backend-core:$commit" command -v docker - docker buildx version >/dev/null 2>&1 || docker buildx inspect default >/dev/null 2>&1 + docker version >/dev/null docker run --rm --network host rancher/mirrored-library-busybox:1.36.1 wget -q -O- "http://$registry/v2/" >/dev/null - docker buildx build \ - --builder default \ - --load \ + DOCKER_BUILDKIT=0 docker build \ --network host \ - --progress=plain \ --build-arg HTTP_PROXY=http://127.0.0.1:18789 \ --build-arg HTTPS_PROXY=http://127.0.0.1:18789 \ --build-arg ALL_PROXY=http://127.0.0.1:18789 \ @@ -401,6 +397,8 @@ spec: - name: registry type: string default: 127.0.0.1:5000 + - name: source-host-path + type: string workspaces: - name: shared-workspace tasks: @@ -416,6 +414,8 @@ spec: value: "$(params.app-image)" - name: registry value: "$(params.registry)" + - name: source-host-path + value: "$(params.source-host-path)" workspaces: - name: source workspace: shared-workspace