chore: pin dev auth proxy rollout

This commit is contained in:
Codex
2026-05-18 13:07:34 +00:00
parent 0c0c979a56
commit 60f991f826
11 changed files with 151 additions and 984 deletions
+1 -2
View File
@@ -21,7 +21,6 @@ Each commit CI run performs:
- `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust`, so Rust backend-core is checked only inside the D601 CI execution boundary. - `UNIDESK_D601_RUST_CHECK=1 bun scripts/cli.ts check --full --rust`, so Rust backend-core is checked only inside the D601 CI execution boundary.
- Temporary `code-queue-ci-read` Deployment and ClusterIP Service in `unidesk-ci`. - Temporary `code-queue-ci-read` Deployment and ClusterIP Service in `unidesk-ci`.
- Code Queue read performance checks against the production PostgreSQL through `d601-tcp-egress-gateway`. - Code Queue read performance checks against the production PostgreSQL through `d601-tcp-egress-gateway`.
- Manual dev desired-state smoke for Code Queue via `ci run-dev-e2e`, using the Git-pinned `code-queue` service commit from `origin/master:deploy.json#environments.dev`.
`ci install` also prewarms the D601 k3s containerd runtime with the Tekton entrypoint/workingdir helper images, `oven/bun:1-debian`, `alpine/git:2.45.2` and `unidesk-code-queue:dev`. Missing images are pulled through the node-local provider-gateway WS egress proxy and then imported into native k3s containerd with digests preserved, so PipelineRun pods do not hang on external registry pulls. Sustained pull throughput below 1 MB/s is treated as a provider/main-server network or proxy degradation first, not as a Dockerfile or application failure. `ci install` also prewarms the D601 k3s containerd runtime with the Tekton entrypoint/workingdir helper images, `oven/bun:1-debian`, `alpine/git:2.45.2` and `unidesk-code-queue:dev`. Missing images are pulled through the node-local provider-gateway WS egress proxy and then imported into native k3s containerd with digests preserved, so PipelineRun pods do not hang on external registry pulls. Sustained pull throughput below 1 MB/s is treated as a provider/main-server network or proxy degradation first, not as a Dockerfile or application failure.
@@ -49,7 +48,7 @@ This means the CI service can read existing tasks, Trace summaries, Trace steps
`ci run-dev-e2e` is the manual dev desired-state smoke flow. The single authoritative reference for its Git-controlled runner script, short launcher, result directory and no-CD boundary is `docs/reference/dev-ci-runner.md`. `ci run-dev-e2e` is the manual dev desired-state smoke flow. The single authoritative reference for its Git-controlled runner script, short launcher, result directory and no-CD boundary is `docs/reference/dev-ci-runner.md`.
The current dev namespace e2e is a harness and smoke gate, not a full frontend/backend stack rollout. It does include a controlled Code Queue slice: D601 builds or reuses the `environments.dev.services[].id=code-queue` commit, imports the image into native k3s containerd, starts temporary PostgreSQL plus Code Queue scheduler/read/write Services in `unidesk-ci-e2e-<runId>`, and verifies the HTTP API through the Kubernetes API service proxy. This remains CI-only and must not deploy persistent `unidesk-dev` or production resources. The current dev namespace e2e is a harness and smoke gate, not a full frontend/backend/code-queue stack rollout. Full-stack temporary namespace deployment can be added behind the same command only after image build/import and per-run database bootstrap are promoted into a controlled deployment design.
## Performance Gate ## Performance Gate
+2 -10
View File
@@ -15,7 +15,6 @@ The runner exists to prove the dev desired state without interrupting production
- D601 execution: Git fetch, Tekton PipelineRun creation, Kubernetes polling and e2e log collection happen on D601, not on the main master. - D601 execution: Git fetch, Tekton PipelineRun creation, Kubernetes polling and e2e log collection happen on D601, not on the main master.
- CLI observability: the submit command returns a `runId`, result directory and next commands; `ci logs <runId>` can recover status after the local CLI exits. - CLI observability: the submit command returns a `runId`, result directory and next commands; `ci logs <runId>` can recover status after the local CLI exits.
- CI only: the flow may create CI-owned temporary resources, but it must not deploy backend-core, frontend, Code Queue, Decision Center, k3sctl-adapter or any other direct/managed service. - CI only: the flow may create CI-owned temporary resources, but it must not deploy backend-core, frontend, Code Queue, Decision Center, k3sctl-adapter or any other direct/managed service.
- Code Queue reproducibility: the runner must use the `code-queue` commit from `environments.dev.services`, build or reuse a labeled image from that Git commit on D601, import it into native k3s containerd, and validate the HTTP API inside a temporary namespace.
## Design Boundary ## Design Boundary
@@ -55,11 +54,6 @@ Do not add a long-lived DevOps service, run broker, webhook listener or second d
"id": "frontend", "id": "frontend",
"repo": "https://github.com/pikasTech/unidesk", "repo": "https://github.com/pikasTech/unidesk",
"commitId": "<pushed-commit>" "commitId": "<pushed-commit>"
},
{
"id": "code-queue",
"repo": "https://github.com/pikasTech/unidesk",
"commitId": "<pushed-commit>"
} }
] ]
} }
@@ -67,7 +61,7 @@ Do not add a long-lived DevOps service, run broker, webhook listener or second d
} }
``` ```
`scriptPath` must be a repo-relative `scripts/ci/*.sh` path. Inline shell bodies, arbitrary script paths, local dirty scripts and separate `develop.json` or CI manifest files are forbidden. The script is fetched from the same full 40-character manifest commit that supplied `deploy.json`, so the runner logic is auditable and rollbackable with the desired state. Persistent dev rollout service scope is owned by `docs/reference/dev-environment.md`; this runner only consumes the dev service list for smoke verification and must not deploy it. `code-queue` is required in the dev service list for this smoke runner, but that does not enable `deploy apply --env dev --service code-queue`. `scriptPath` must be a repo-relative `scripts/ci/*.sh` path. Inline shell bodies, arbitrary script paths, local dirty scripts and separate `develop.json` or CI manifest files are forbidden. The script is fetched from the same full 40-character manifest commit that supplied `deploy.json`, so the runner logic is auditable and rollbackable with the desired state. Persistent dev rollout service scope is owned by `docs/reference/dev-environment.md`; this runner only consumes the dev service list for smoke verification and must not deploy it.
## Execution Path ## Execution Path
@@ -79,7 +73,7 @@ The automatic path is intentionally single and narrow:
4. D601 creates `/tmp/unidesk-ci/<runId>` and `/home/ubuntu/.unidesk/runs/<runId>`. 4. D601 creates `/tmp/unidesk-ci/<runId>` and `/home/ubuntu/.unidesk/runs/<runId>`.
5. D601 fetches the manifest commit from GitHub through the node-local provider-gateway WS egress proxy at `http://127.0.0.1:18789`. 5. D601 fetches the manifest commit from GitHub through the node-local provider-gateway WS egress proxy at `http://127.0.0.1:18789`.
6. D601 extracts the runner with `git show <commit>:<scriptPath> > /tmp/unidesk-ci/<runId>/runner.sh` and the desired-state blob with `git show <commit>:deploy.json > /tmp/unidesk-ci/<runId>/deploy.json`. 6. D601 extracts the runner with `git show <commit>:<scriptPath> > /tmp/unidesk-ci/<runId>/runner.sh` and the desired-state blob with `git show <commit>:deploy.json > /tmp/unidesk-ci/<runId>/deploy.json`.
7. The runner parses the host-fetched `deploy.json`, requires a full-SHA `code-queue` service commit, builds or reuses a D601 Docker image for that commit, imports the image and `postgres:16-alpine` into native k3s containerd, creates the Tekton PipelineRun in `unidesk-ci`, passes the required dev service commits and Code Queue image tag as PipelineRun params, waits for completion when requested, and writes `result.json`, `launcher.log`, `runner.log`, PipelineRun JSON and pod logs under `/home/ubuntu/.unidesk/runs/<runId>/`. 7. The runner parses the host-fetched `deploy.json`, creates the Tekton PipelineRun in `unidesk-ci`, passes the required dev service commits as PipelineRun params, waits for completion when requested, and writes `result.json`, `launcher.log`, `runner.log`, PipelineRun JSON and pod logs under `/home/ubuntu/.unidesk/runs/<runId>/`.
The CLI must not upload the runner script body. Tekton dev e2e must not clone the private UniDesk repo itself; repo access and desired-state extraction happen once in the D601 host launcher under the manifest commit. The submitted launcher may contain only repo, full commit, script path, run id, environment, timeout, keep-namespace and fixed workspace path settings plus the fixed fetch/execute wrapper. If k3s, Tekton or the provider egress proxy is unavailable, the run fails with visible logs; it must not fall back to an alternate deployment path. The CLI must not upload the runner script body. Tekton dev e2e must not clone the private UniDesk repo itself; repo access and desired-state extraction happen once in the D601 host launcher under the manifest commit. The submitted launcher may contain only repo, full commit, script path, run id, environment, timeout, keep-namespace and fixed workspace path settings plus the fixed fetch/execute wrapper. If k3s, Tekton or the provider egress proxy is unavailable, the run fails with visible logs; it must not fall back to an alternate deployment path.
@@ -102,8 +96,6 @@ scripts/ci/dev-e2e.sh \
The current script creates a Tekton `PipelineRun` for `pipeline/unidesk-dev-namespace-e2e`, stores the generated PipelineRun name in `pipelinerun.txt`, and writes a final `result.json` with `ok`, `status`, `runId`, `manifestCommit`, `pipelineRun`, `temporaryNamespace` and `finishedAt`. The current script creates a Tekton `PipelineRun` for `pipeline/unidesk-dev-namespace-e2e`, stores the generated PipelineRun name in `pipelinerun.txt`, and writes a final `result.json` with `ok`, `status`, `runId`, `manifestCommit`, `pipelineRun`, `temporaryNamespace` and `finishedAt`.
The Tekton task creates a temporary namespace `unidesk-ci-e2e-<runId>` and may create only CI-owned smoke resources there: `postgres-dev`, `code-queue-scheduler-dev`, `code-queue-read-dev`, `code-queue-write-dev`, their ClusterIP Services and a per-run Secret/ConfigMap. It must not mutate `unidesk` or persistent `unidesk-dev`. Code Queue API validation must use ClusterIP Services and the Kubernetes API `services/.../proxy` subresource; NodePort, D601 host ports and direct public service exposure are forbidden. The smoke currently proves `/health`, `/live` and `/api/workdirs` GET/POST/DELETE on read/write/scheduler roles, giving follow-up Code Queue API fixes a reproducible CI target before production rollout.
## Commands ## Commands
Start a run and return after dispatch: Start a run and return after dispatch:
+3 -5
View File
@@ -30,14 +30,14 @@ The unrestricted public network entries are therefore production frontend, dev f
## Desired State ## Desired State
`deploy.json` remains the only version intent file. Dev entries live under `environments.dev` and are read from `origin/master:deploy.json`, never from a dirty local file, when using `--env dev` or `ci run-dev-e2e`. `deploy.json` remains the only version intent file. Dev entries live under `environments.dev` and are read from `origin/master:deploy.json`, never from a dirty local file, when using `--env dev`.
The persistent dev rollout currently supports only: The persistent dev rollout currently supports only:
- `backend-core` - `backend-core`
- `frontend` - `frontend`
`code-queue` is present in `environments.dev.services` only so `ci run-dev-e2e` can build a Git-pinned Code Queue image and run a temporary namespace smoke. It is not part of persistent dev apply: `deploy apply --env dev --service code-queue` must still be rejected. Decision Center, k3sctl-adapter and other D601 services are not part of persistent dev apply yet. Their smoke validation stays under `ci run-dev-e2e` or service-specific future designs. The `environments.dev.ci` declaration and short launcher runner are owned by `docs/reference/dev-ci-runner.md`. `code-queue`, Decision Center, k3sctl-adapter and other D601 services are not part of persistent dev apply yet. Their smoke validation stays under `ci run-dev-e2e` or service-specific future designs. The `environments.dev.ci` declaration and short launcher runner are owned by `docs/reference/dev-ci-runner.md`.
## Rust Backend-Core Boundary ## Rust Backend-Core Boundary
@@ -94,7 +94,7 @@ Use this sequence for backend-core Rust and frontend dev work:
7. If the dev service catalog changes, deploy the pushed `k3sctl-adapter` commit through the controlled local manifest exception, then verify `/api/control-plane` lists `k3s/dev/unidesk-dev-core.k3s.json`. 7. If the dev service catalog changes, deploy the pushed `k3sctl-adapter` commit through the controlled local manifest exception, then verify `/api/control-plane` lists `k3s/dev/unidesk-dev-core.k3s.json`.
8. Rebuild or verify `dev-frontend-proxy` on the main server with `bun scripts/cli.ts server rebuild dev-frontend-proxy` when the proxy config or port changes. 8. Rebuild or verify `dev-frontend-proxy` on the main server with `bun scripts/cli.ts server rebuild dev-frontend-proxy` when the proxy config or port changes.
9. Manually test `http://74.48.78.17:18083/` and the dev health endpoints. 9. Manually test `http://74.48.78.17:18083/` and the dev health endpoints.
10. Run D601 CI for the commit and the dev smoke runner: `bun scripts/cli.ts ci run --revision <commit> --wait-ms <ms>` and `bun scripts/cli.ts ci run-dev-e2e --wait-ms <ms>`. When Code Queue behavior changes, update the `code-queue` entry in `environments.dev.services` to the pushed commit before running the dev smoke; do not use `deploy apply --env dev --service code-queue`. 10. Run D601 CI for the commit and the dev smoke runner: `bun scripts/cli.ts ci run --revision <commit> --wait-ms <ms>` and `bun scripts/cli.ts ci run-dev-e2e --wait-ms <ms>`.
## Validation Commands ## Validation Commands
@@ -105,8 +105,6 @@ bun scripts/cli.ts server status
bun scripts/cli.ts deploy plan --env dev bun scripts/cli.ts deploy plan --env dev
bun scripts/cli.ts deploy plan --env dev --service backend-core bun scripts/cli.ts deploy plan --env dev --service backend-core
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-core.k8s.yaml
bun scripts/cli.ts dev-env validate --manifest src/components/microservices/k3sctl-adapter/k3s/dev/unidesk-dev-code-queue.k8s.yaml
bun scripts/cli.ts ci run-dev-e2e --wait-ms 600000
bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/backend-core-dev/proxy/health --raw --full bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/backend-core-dev/proxy/health --raw --full
bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/frontend-dev/proxy/health --raw --full bun scripts/cli.ts microservice proxy k3sctl-adapter /api/services/frontend-dev/proxy/health --raw --full
curl -fsS http://127.0.0.1:18083/health curl -fsS http://127.0.0.1:18083/health
+2 -104
View File
@@ -116,12 +116,12 @@ write_result() {
local ok="$1" local ok="$1"
local status="$2" local status="$2"
local detail="$3" local detail="$3"
python3 - "$result_json" "$ok" "$status" "$detail" "$run_id" "$repo_url" "$desired_ref" "$manifest_commit" "$environment" "$pipeline_run" "$temporary_namespace" "$code_queue_image" <<'PY' python3 - "$result_json" "$ok" "$status" "$detail" "$run_id" "$repo_url" "$desired_ref" "$manifest_commit" "$environment" "$pipeline_run" "$temporary_namespace" <<'PY'
import json import json
import sys import sys
from datetime import datetime, timezone from datetime import datetime, timezone
path, ok, status, detail, run_id, repo, desired_ref, commit, environment, pipeline_run, temporary_namespace, code_queue_image = sys.argv[1:] path, ok, status, detail, run_id, repo, desired_ref, commit, environment, pipeline_run, temporary_namespace = sys.argv[1:]
record = { record = {
"ok": ok == "true", "ok": ok == "true",
"status": status, "status": status,
@@ -133,7 +133,6 @@ record = {
"environment": environment, "environment": environment,
"pipelineRun": pipeline_run or None, "pipelineRun": pipeline_run or None,
"temporaryNamespace": temporary_namespace or None, "temporaryNamespace": temporary_namespace or None,
"codeQueueImage": code_queue_image or None,
"finishedAt": datetime.now(timezone.utc).isoformat(), "finishedAt": datetime.now(timezone.utc).isoformat(),
} }
with open(path, "w", encoding="utf-8") as handle: with open(path, "w", encoding="utf-8") as handle:
@@ -145,7 +144,6 @@ PY
pipeline_run="" pipeline_run=""
temporary_namespace="unidesk-ci-e2e-$run_id" temporary_namespace="unidesk-ci-e2e-$run_id"
code_queue_image=""
trap 'code=$?; if [ "$code" -ne 0 ] && [ ! -f "$result_json" ]; then write_result false failed "runner exited with code $code" || true; fi' EXIT trap 'code=$?; if [ "$code" -ne 0 ] && [ ! -f "$result_json" ]; then write_result false failed "runner exited with code $code" || true; fi' EXIT
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
@@ -190,106 +188,8 @@ source "$service_env"
backend_commit="${BACKEND_CORE_COMMIT:-unknown}" backend_commit="${BACKEND_CORE_COMMIT:-unknown}"
frontend_commit="${FRONTEND_COMMIT:-unknown}" frontend_commit="${FRONTEND_COMMIT:-unknown}"
code_queue_commit="${CODE_QUEUE_COMMIT:-unknown}" code_queue_commit="${CODE_QUEUE_COMMIT:-unknown}"
if ! [[ "$code_queue_commit" =~ ^[0-9a-f]{40}$ ]]; then
echo "deploy.json environments.$environment.services must include code-queue with a full 40 character commitId; got: $code_queue_commit" >&2
exit 2
fi
deploy_json_b64="$(base64 -w0 "$manifest_file")" deploy_json_b64="$(base64 -w0 "$manifest_file")"
work_dir="$(dirname "$manifest_file")"
repo_dir="$work_dir/repo"
if [ ! -d "$repo_dir/.git" ]; then
echo "launcher repo checkout is missing: $repo_dir" >&2
exit 2
fi
root_exec() {
if [ "$(id -u)" = "0" ]; then
"$@"
return
fi
if sudo -n true >/dev/null 2>&1; then
sudo -n "$@"
return
fi
if [ -x /mnt/c/Windows/System32/wsl.exe ]; then
/mnt/c/Windows/System32/wsl.exe -u root -- "$@"
return
fi
echo "dev_e2e_native_k3s_root_access=missing" >&2
return 1
}
import_image_to_k3s() {
local image="$1"
local archive="/tmp/unidesk-ci-image-${run_id}-${image//[^A-Za-z0-9_.-]/-}.tar"
rm -f "$archive"
docker save "$image" -o "$archive"
root_exec ctr --address /run/k3s/containerd/containerd.sock -n k8s.io images import --digests --all-platforms "$archive"
rm -f "$archive"
echo "dev_e2e_image_imported=$image"
}
ensure_runtime_image() {
local image="$1"
if ! docker image inspect "$image" >/dev/null 2>&1; then
echo "dev_e2e_runtime_image_pull=$image"
docker pull --platform linux/amd64 "$image"
else
echo "dev_e2e_runtime_image_cached=$image"
fi
import_image_to_k3s "$image"
}
build_code_queue_image() {
local short="${code_queue_commit:0:12}"
local commit_image="unidesk-code-queue:ci-$short"
local run_image="unidesk-code-queue:ci-dev-e2e-$run_id"
local source_dir="$work_dir/code-queue-src-$short"
local resolved
git -C "$repo_dir" fetch --no-tags origin "$code_queue_commit" || git -C "$repo_dir" fetch --no-tags origin '+refs/heads/*:refs/remotes/origin/*'
resolved="$(git -C "$repo_dir" rev-parse --verify "$code_queue_commit^{commit}")"
if [ "$resolved" != "$code_queue_commit" ]; then
echo "code_queue_commit_mismatch resolved=$resolved expected=$code_queue_commit" >&2
exit 1
fi
local existing_rev
existing_rev="$(docker image inspect "$commit_image" --format '{{ index .Config.Labels "org.opencontainers.image.revision" }}' 2>/dev/null || true)"
if [ "$existing_rev" != "$resolved" ]; then
rm -rf "$source_dir"
mkdir -p "$source_dir"
git -C "$repo_dir" archive "$resolved" | tar -x -C "$source_dir"
local base_args=()
if docker image inspect unidesk-code-queue:d601-build-base >/dev/null 2>&1; then
base_args=(--build-arg CODE_QUEUE_BASE_IMAGE=unidesk-code-queue:d601-build-base)
elif docker image inspect unidesk-code-queue:d601 >/dev/null 2>&1; then
base_args=(--build-arg CODE_QUEUE_BASE_IMAGE=unidesk-code-queue:d601)
fi
echo "dev_e2e_code_queue_image_build=$commit_image commit=$resolved"
docker build \
"${base_args[@]}" \
--build-arg HTTP_PROXY="${HTTP_PROXY:-}" \
--build-arg HTTPS_PROXY="${HTTPS_PROXY:-}" \
--build-arg ALL_PROXY="${ALL_PROXY:-}" \
--build-arg NO_PROXY="${NO_PROXY:-}" \
--label "org.opencontainers.image.source=$repo_url" \
--label "org.opencontainers.image.revision=$resolved" \
--label "unidesk.ai/ci-run-id=$run_id" \
--label "unidesk.ai/ci-kind=dev-e2e" \
-t "$commit_image" \
-f "$source_dir/src/components/microservices/code-queue/Dockerfile" \
"$source_dir"
else
echo "dev_e2e_code_queue_image_cached=$commit_image commit=$resolved"
fi
docker tag "$commit_image" "$run_image"
import_image_to_k3s "$run_image"
code_queue_image="$run_image"
}
ensure_runtime_image "postgres:16-alpine"
build_code_queue_image
pipeline_manifest="$result_dir/pipelinerun.yaml" pipeline_manifest="$result_dir/pipelinerun.yaml"
cat >"$pipeline_manifest" <<YAML cat >"$pipeline_manifest" <<YAML
apiVersion: tekton.dev/v1 apiVersion: tekton.dev/v1
@@ -327,8 +227,6 @@ spec:
value: "$frontend_commit" value: "$frontend_commit"
- name: code-queue-commit - name: code-queue-commit
value: "$code_queue_commit" value: "$code_queue_commit"
- name: app-image
value: "$code_queue_image"
- name: deploy-json-b64 - name: deploy-json-b64
value: "$deploy_json_b64" value: "$deploy_json_b64"
workspaces: workspaces:
-7
View File
@@ -683,13 +683,6 @@ function resolveDeployDevManifest(desiredRef: string): DeployDevManifestSummary
}; };
}).filter((service) => service.id.length > 0 && service.commitId.length > 0); }).filter((service) => service.id.length > 0 && service.commitId.length > 0);
if (services.length === 0) throw new Error(`origin/${desiredRef}:deploy.json has no environments.dev services with commitId`); if (services.length === 0) throw new Error(`origin/${desiredRef}:deploy.json has no environments.dev services with commitId`);
const codeQueueService = services.find((service) => service.id === "code-queue");
if (codeQueueService === undefined) {
throw new Error(`origin/${desiredRef}:deploy.json environments.dev.services must include code-queue for ci run-dev-e2e`);
}
if (!/^[0-9a-f]{40}$/u.test(codeQueueService.commitId)) {
throw new Error(`origin/${desiredRef}:deploy.json environments.dev.services code-queue commitId must be a full 40-character SHA`);
}
return { return {
deployCommit: deployCommitResult.stdout.trim(), deployCommit: deployCommitResult.stdout.trim(),
desiredRef, desiredRef,
+3 -7
View File
@@ -228,8 +228,6 @@ function compactSummary(summary: unknown, options: CodexTaskOptions, taskId: str
const transcriptCount = asNumber(record.transcriptCount, 0); const transcriptCount = asNumber(record.transcriptCount, 0);
const transcriptMaxSeq = transcriptCount > 0 ? record.transcriptMaxSeq ?? null : null; const transcriptMaxSeq = transcriptCount > 0 ? record.transcriptMaxSeq ?? null : null;
const initialPrompt = asString(record.initialPrompt ?? record.prompt); const initialPrompt = asString(record.initialPrompt ?? record.prompt);
const initialPromptView = textView(initialPrompt, options.full, 3000);
const basePromptView = textView(asString(record.basePrompt), options.full, 2000);
return { return {
id: record.id ?? taskId, id: record.id ?? taskId,
queueId: record.queueId ?? null, queueId: record.queueId ?? null,
@@ -258,9 +256,8 @@ function compactSummary(summary: unknown, options: CodexTaskOptions, taskId: str
startedAt: record.startedAt ?? null, startedAt: record.startedAt ?? null,
updatedAt: record.updatedAt ?? null, updatedAt: record.updatedAt ?? null,
finishedAt: record.finishedAt ?? null, finishedAt: record.finishedAt ?? null,
...(options.full initialPrompt: textView(initialPrompt, options.full, 3000),
? { initialPrompt: initialPromptView, basePrompt: basePromptView } basePrompt: textView(asString(record.basePrompt), options.full, 2000),
: { initialPromptPreview: initialPromptView, basePromptPreview: basePromptView }),
referenceTaskIds: record.referenceTaskIds ?? [], referenceTaskIds: record.referenceTaskIds ?? [],
referenceInjection: record.referenceInjection ?? null, referenceInjection: record.referenceInjection ?? null,
lastAssistantMessage: compactLastAssistant(record.lastAssistantMessage, options.full), lastAssistantMessage: compactLastAssistant(record.lastAssistantMessage, options.full),
@@ -811,7 +808,6 @@ function compactTaskMutationResponse(task: unknown, options: CompactTaskMutation
const record = asRecord(task) ?? {}; const record = asRecord(task) ?? {};
const taskId = asString(record.id); const taskId = asString(record.id);
const prompt = asString(record.displayPrompt ?? record.basePrompt ?? record.prompt); const prompt = asString(record.displayPrompt ?? record.basePrompt ?? record.prompt);
const promptView = textView(prompt, options.fullPrompt === true, 1200);
return { return {
id: taskId || null, id: taskId || null,
queueId: record.queueId ?? null, queueId: record.queueId ?? null,
@@ -829,7 +825,7 @@ function compactTaskMutationResponse(task: unknown, options: CompactTaskMutation
startedAt: record.startedAt ?? null, startedAt: record.startedAt ?? null,
updatedAt: record.updatedAt ?? null, updatedAt: record.updatedAt ?? null,
finishedAt: record.finishedAt ?? null, finishedAt: record.finishedAt ?? null,
...(options.fullPrompt === true ? { prompt: promptView } : { promptPreview: promptView }), prompt: textView(prompt, options.fullPrompt === true, 1200),
commands: taskId.length === 0 ? null : { commands: taskId.length === 0 ? null : {
show: `bun scripts/cli.ts codex task ${taskId}`, show: `bun scripts/cli.ts codex task ${taskId}`,
trace: `bun scripts/cli.ts codex task ${taskId} --trace --tail --limit ${defaultTraceLimit}`, trace: `bun scripts/cli.ts codex task ${taskId} --trace --tail --limit ${defaultTraceLimit}`,
+3 -171
View File
@@ -139,7 +139,6 @@ const FRONTEND_CHECK_NAMES = [
"frontend:code-queue-integrated-visible", "frontend:code-queue-integrated-visible",
"frontend:code-queue-enqueue-await-smoke", "frontend:code-queue-enqueue-await-smoke",
"frontend:code-queue-summary-mobile-wrap", "frontend:code-queue-summary-mobile-wrap",
"frontend:code-queue-long-prompt-observation",
"frontend:code-queue-initial-prompt-full-expand", "frontend:code-queue-initial-prompt-full-expand",
"frontend:code-queue-trace-full-load", "frontend:code-queue-trace-full-load",
"frontend:code-queue-judge-wrap", "frontend:code-queue-judge-wrap",
@@ -487,10 +486,8 @@ function wantsPrefix(options: E2ERunOptions, prefix: string): boolean {
} }
function publicUrls(config: UniDeskConfig): PublicUrls { function publicUrls(config: UniDeskConfig): PublicUrls {
const frontendTarget = String(process.env.UNIDESK_E2E_FRONTEND || "prod").trim().toLowerCase();
const frontendPort = frontendTarget === "dev" ? config.network.devFrontend.port : config.network.frontend.port;
return { return {
frontendUrl: `http://${config.network.publicHost}:${frontendPort}`, frontendUrl: `http://${config.network.publicHost}:${config.network.frontend.port}`,
providerIngressHealthUrl: `http://${config.network.publicHost}:${config.network.providerIngress.port}/health`, providerIngressHealthUrl: `http://${config.network.publicHost}:${config.network.providerIngress.port}/health`,
providerIngressWsUrl: `ws://${config.network.publicHost}:${config.network.providerIngress.port}/ws/provider`, providerIngressWsUrl: `ws://${config.network.publicHost}:${config.network.providerIngress.port}/ws/provider`,
blockedCoreUrl: `http://${config.network.publicHost}:${config.network.core.port}`, blockedCoreUrl: `http://${config.network.publicHost}:${config.network.core.port}`,
@@ -753,149 +750,6 @@ async function runCodeQueueEnqueueAwaitSmoke(page: Page): Promise<any> {
} }
} }
async function runCodeQueueLongPromptObservation(page: Page): Promise<any> {
const marker = `E2E_LONG_PROMPT_TAIL_${Date.now()}_${Math.random().toString(16).slice(2, 8)}`;
const submitQueueId = "e2e-long-prompt";
const filler = Array.from({ length: 76 }, (_, index) => `long-prompt-line-${String(index + 1).padStart(2, "0")} ${"abcdefghijklmnopqrstuvwxyz0123456789".repeat(2)}`).join("\n");
const prompt = [
"Code Queue long prompt observation regression",
"",
filler,
"",
"验收标准:",
`必须在所有原始 prompt 观察层保留这个唯一 tail marker: ${marker}`,
].join("\n");
const promptChars = prompt.length;
if (promptChars <= 2500) throw new Error(`long prompt fixture too short: ${promptChars}`);
const apiFetch = async (path: string, init: RequestInit = {}): Promise<any> => {
const response = await fetch(path, {
credentials: "same-origin",
headers: { "content-type": "application/json", ...(init.headers || {}) },
...init,
});
const text = await response.text();
let body: any = null;
try { body = text.length > 0 ? JSON.parse(text) : null; } catch { body = { text }; }
return { ok: response.ok, status: response.status, body };
};
await page.getByTestId("code-queue-filter-select").selectOption("__all__").catch(() => undefined);
await page.getByTestId("code-queue-id-select").selectOption(submitQueueId).catch(async () => {
page.once("dialog", (dialog) => { void dialog.accept(submitQueueId); });
await page.getByTestId("codex-create-queue-button").click();
await page.waitForFunction((queueId) => {
const select = document.querySelector('[data-testid="code-queue-id-select"]') as HTMLSelectElement | null;
return Array.from(select?.options || []).some((option) => option.value === queueId);
}, submitQueueId, { timeout: 10000 });
await page.getByTestId("code-queue-id-select").selectOption(submitQueueId);
});
await page.getByTestId("codex-max-attempts-input").fill("1");
await page.getByTestId("codex-repeat-count-input").fill("1");
await page.locator('[data-testid="code-queue-task-form"] textarea').fill(prompt);
await page.waitForFunction(() => {
const button = document.querySelector('[data-testid="codex-enqueue-button"]') as HTMLButtonElement | null;
return button !== null && !button.disabled;
}, undefined, { timeout: 5000 });
const responsePromise = page.waitForResponse((response) => isCodeQueueTaskEnqueueRequest(response.url(), response.request().method()), { timeout: 30000 });
await page.getByTestId("codex-enqueue-button").click();
const response = await responsePromise;
const createBody = await response.json().catch((error: unknown) => ({ parseError: error instanceof Error ? error.message : String(error) }));
const createdTask = Array.isArray(createBody?.tasks) ? createBody.tasks[0] : null;
const taskId = String(createdTask?.id || "");
if (!taskId) return { checked: true, marker, promptChars, createStatus: response.status(), taskId: "", error: "missing task id", createBody };
await page.waitForSelector(`[data-testid="codex-task-${taskId}"]`, { timeout: 30000 }).catch(() => undefined);
const overview = await page.evaluate(async (id) => {
const response = await fetch(`/api/microservices/code-queue/proxy/api/tasks/overview?limit=120&transcriptLimit=0&compact=1&selected=1&includeActive=1&stats=0&afterSeq=0&preferId=${encodeURIComponent(String(id))}`, { credentials: "same-origin" });
const body = await response.json().catch(() => null);
return { ok: response.ok, status: response.status, body };
}, taskId);
const meta = await page.evaluate(async (id) => {
const response = await fetch(`/api/microservices/code-queue/proxy/api/tasks/${encodeURIComponent(String(id))}?meta=1`, { credentials: "same-origin" });
const body = await response.json().catch(() => null);
return { ok: response.ok, status: response.status, body };
}, taskId);
const initialPrompt = await page.evaluate(async (id) => {
const response = await fetch(`/api/microservices/code-queue/proxy/api/tasks/${encodeURIComponent(String(id))}/prompt?part=initial`, { credentials: "same-origin" });
const body = await response.json().catch(() => null);
return { ok: response.ok, status: response.status, body };
}, taskId);
await page.getByTestId(`codex-task-${taskId}`).click().catch(() => undefined);
await page.waitForSelector('[data-testid="codex-initial-prompt-base"]', { timeout: 15000 }).catch(() => undefined);
const ui = await page.evaluate(() => {
const base = document.querySelector('[data-testid="codex-initial-prompt-base"]') as HTMLElement | null;
return {
baseText: base?.innerText || "",
baseChars: (base?.innerText || "").length,
traceText: document.querySelector('.codex-output-panel')?.textContent || "",
};
});
const overviewTasks = Array.isArray(overview.body?.tasks) ? overview.body.tasks : [];
const overviewTask = overviewTasks.find((task: any) => String(task?.id || "") === taskId) || null;
const selectedTask = overview.body?.selected?.task || null;
const metaTask = meta.body?.task || null;
const createPrompt = String(createdTask?.prompt || "");
const createDisplayPrompt = String(createdTask?.displayPrompt || "");
const createPreview = createdTask?.displayPromptPreview || createdTask?.promptPreview || null;
const overviewPrompt = String(overviewTask?.prompt || "");
const overviewDisplayPrompt = String(overviewTask?.displayPrompt || "");
const overviewPreview = overviewTask?.displayPromptPreview || overviewTask?.promptPreview || null;
const selectedDisplayPrompt = String(selectedTask?.displayPrompt || "");
const selectedPreview = selectedTask?.displayPromptPreview || selectedTask?.promptPreview || null;
const metaPrompt = String(metaTask?.prompt || "");
const metaBasePrompt = String(metaTask?.basePrompt || "");
const metaDisplayPrompt = String(metaTask?.displayPrompt || "");
const promptText = String(initialPrompt.body?.text || "");
const previewHasTruncationMarker = [createPreview?.text, overviewPreview?.text, selectedPreview?.text].some((value) => String(value || "").includes("...<truncated>"));
const originalValues = [createPrompt, createDisplayPrompt, overviewPrompt, overviewDisplayPrompt, selectedDisplayPrompt, metaPrompt, metaBasePrompt, metaDisplayPrompt, promptText];
await apiFetch(`/api/microservices/code-queue/proxy/api/tasks/${encodeURIComponent(taskId)}/interrupt`, { method: "POST", body: "{}" }).catch(() => null);
return {
checked: true,
marker,
taskId,
promptChars,
createStatus: response.status(),
createOk: createBody?.ok === true,
createPromptChars: createPrompt.length,
createDisplayPromptChars: createDisplayPrompt.length,
overviewStatus: overview.status,
overviewTaskFound: overviewTask !== null,
overviewPromptChars: overviewPrompt.length,
overviewDisplayPromptChars: overviewDisplayPrompt.length,
selectedTaskFound: selectedTask !== null && String(selectedTask?.id || "") === taskId,
selectedDisplayPromptChars: selectedDisplayPrompt.length,
metaStatus: meta.status,
metaPromptChars: metaPrompt.length,
metaBasePromptChars: metaBasePrompt.length,
metaDisplayPromptChars: metaDisplayPrompt.length,
initialPromptStatus: initialPrompt.status,
initialPromptChars: Number(initialPrompt.body?.chars || promptText.length),
uiBaseChars: ui.baseChars,
preview: {
create: createPreview,
overview: overviewPreview,
selected: selectedPreview,
previewHasTruncationMarker,
createPromptPreviewFieldPresent: createdTask?.promptPreview !== undefined,
overviewPromptPreviewFieldPresent: overviewTask?.promptPreview !== undefined,
selectedPromptPreviewFieldPresent: selectedTask?.promptPreview !== undefined,
},
checks: {
createOriginalHasTail: createPrompt.includes(marker) && createDisplayPrompt.includes(marker),
overviewOriginalHasTail: overviewPrompt.includes(marker) && overviewDisplayPrompt.includes(marker),
selectedOriginalHasTail: selectedDisplayPrompt.includes(marker),
metaOriginalHasTail: metaPrompt.includes(marker) && metaBasePrompt.includes(marker) && metaDisplayPrompt.includes(marker),
initialPromptHasTail: promptText.includes(marker),
uiFullPromptHasTail: String(ui.baseText || "").includes(marker),
originalsHaveNoTruncationMarker: originalValues.every((value) => !value.includes("...<truncated>")),
previewFieldsExplicit: createdTask?.promptPreview !== undefined && overviewTask?.promptPreview !== undefined && selectedTask?.promptPreview !== undefined,
previewCanTruncate: Boolean(createPreview?.truncated || overviewPreview?.truncated || selectedPreview?.truncated),
},
};
}
function runPsql(config: UniDeskConfig, sql: string): { ok: boolean; stdout: string; stderr: string; exitCode: number | null } { function runPsql(config: UniDeskConfig, sql: string): { ok: boolean; stdout: string; stderr: string; exitCode: number | null } {
const result = runCommand([ const result = runCommand([
"docker", "docker",
@@ -1596,7 +1450,6 @@ async function frontendCheck(config: UniDeskConfig, urls: PublicUrls, checks: E2
"frontend:code-queue-integrated-visible", "frontend:code-queue-integrated-visible",
"frontend:code-queue-enqueue-await-smoke", "frontend:code-queue-enqueue-await-smoke",
"frontend:code-queue-summary-mobile-wrap", "frontend:code-queue-summary-mobile-wrap",
"frontend:code-queue-long-prompt-observation",
"frontend:code-queue-initial-prompt-full-expand", "frontend:code-queue-initial-prompt-full-expand",
"frontend:code-queue-trace-full-load", "frontend:code-queue-trace-full-load",
"frontend:code-queue-judge-wrap", "frontend:code-queue-judge-wrap",
@@ -1633,10 +1486,8 @@ async function frontendCheck(config: UniDeskConfig, urls: PublicUrls, checks: E2
page.on("dialog", (dialog) => dialog.accept()); page.on("dialog", (dialog) => dialog.accept());
await page.goto(urls.frontendUrl, { waitUntil: "domcontentloaded", timeout: 15000 }); await page.goto(urls.frontendUrl, { waitUntil: "domcontentloaded", timeout: 15000 });
await page.waitForSelector('[data-testid="login-screen"]', { timeout: 10000 }); await page.waitForSelector('[data-testid="login-screen"]', { timeout: 10000 });
const frontendAuthUsername = process.env.UNIDESK_E2E_AUTH_USERNAME || config.auth.username; await page.fill('input[name="username"]', config.auth.username);
const frontendAuthPassword = process.env.UNIDESK_E2E_AUTH_PASSWORD || config.auth.password; await page.fill('input[name="password"]', config.auth.password);
await page.fill('input[name="username"]', frontendAuthUsername);
await page.fill('input[name="password"]', frontendAuthPassword);
await page.click('button[type="submit"]'); await page.click('button[type="submit"]');
await page.waitForSelector('[data-testid="app-shell"]', { timeout: 10000 }); await page.waitForSelector('[data-testid="app-shell"]', { timeout: 10000 });
await page.waitForFunction(() => document.querySelector('[data-testid="conn-text"]')?.textContent?.includes("核心在线"), undefined, { timeout: 15000 }); await page.waitForFunction(() => document.querySelector('[data-testid="conn-text"]')?.textContent?.includes("核心在线"), undefined, { timeout: 15000 });
@@ -1696,7 +1547,6 @@ async function frontendCheck(config: UniDeskConfig, urls: PublicUrls, checks: E2
let codeQueueSidebarUpdateMetrics: any = { cardCount: 0, labels: [], hasRecentUpdateLabel: false }; let codeQueueSidebarUpdateMetrics: any = { cardCount: 0, labels: [], hasRecentUpdateLabel: false };
let codeQueueHtmlGuard: any = { rootAttrMissing: false, sourceAttrMissing: false, sourceNoBasePrompt: false }; let codeQueueHtmlGuard: any = { rootAttrMissing: false, sourceAttrMissing: false, sourceNoBasePrompt: false };
let codeQueueSummaryMobileMetrics: any = { checked: false, summaryCount: 0, ok: false }; let codeQueueSummaryMobileMetrics: any = { checked: false, summaryCount: 0, ok: false };
let codeQueueLongPromptMetrics: any = { checked: false };
let codeQueuePromptDefaultEmpty = false; let codeQueuePromptDefaultEmpty = false;
let codeQueueSubmitGuard: any = { batchRowVisible: false, disabledBeforeConfirm: false, enabledAfterConfirm: false, waitElementMissingBeforeSubmit: false }; let codeQueueSubmitGuard: any = { batchRowVisible: false, disabledBeforeConfirm: false, enabledAfterConfirm: false, waitElementMissingBeforeSubmit: false };
let codeQueueEnqueueAwaitSmoke: any = { checked: false }; let codeQueueEnqueueAwaitSmoke: any = { checked: false };
@@ -2145,9 +1995,6 @@ async function frontendCheck(config: UniDeskConfig, urls: PublicUrls, checks: E2
if (wants("frontend:code-queue-enqueue-await-smoke")) { if (wants("frontend:code-queue-enqueue-await-smoke")) {
codeQueueEnqueueAwaitSmoke = await runCodeQueueEnqueueAwaitSmoke(page); codeQueueEnqueueAwaitSmoke = await runCodeQueueEnqueueAwaitSmoke(page);
} }
if (wants("frontend:code-queue-long-prompt-observation")) {
codeQueueLongPromptMetrics = await runCodeQueueLongPromptObservation(page);
}
codeQueueOptions = await page.locator('[data-testid="code-queue-filter-select"] option').evaluateAll((options) => options.map((option) => (option as HTMLOptionElement).textContent || "")); codeQueueOptions = await page.locator('[data-testid="code-queue-filter-select"] option').evaluateAll((options) => options.map((option) => (option as HTMLOptionElement).textContent || ""));
codeQueueSwitchMetrics = await page.locator('[data-testid="code-queue-filter-select"] option').evaluateAll((options) => ({ codeQueueSwitchMetrics = await page.locator('[data-testid="code-queue-filter-select"] option').evaluateAll((options) => ({
optionCount: options.length, optionCount: options.length,
@@ -3202,21 +3049,6 @@ async function frontendCheck(config: UniDeskConfig, urls: PublicUrls, checks: E2
&& (codeQueueEnqueueAwaitSmoke.interrupt?.ok === true || codeQueueEnqueueAwaitSmoke.interrupt?.status === 409), && (codeQueueEnqueueAwaitSmoke.interrupt?.ok === true || codeQueueEnqueueAwaitSmoke.interrupt?.status === 409),
{ codeQueueEnqueueAwaitSmoke }); { codeQueueEnqueueAwaitSmoke });
addSelectedCheck(checks, options, "frontend:code-queue-summary-mobile-wrap", codeQueueSummaryMobileMetrics.checked === true && (codeQueueSummaryMobileMetrics.summaryCount === 0 || codeQueueSummaryMobileMetrics.ok === true), { codeQueueSummaryMobileMetrics }); addSelectedCheck(checks, options, "frontend:code-queue-summary-mobile-wrap", codeQueueSummaryMobileMetrics.checked === true && (codeQueueSummaryMobileMetrics.summaryCount === 0 || codeQueueSummaryMobileMetrics.ok === true), { codeQueueSummaryMobileMetrics });
addSelectedCheck(checks, options, "frontend:code-queue-long-prompt-observation",
codeQueueLongPromptMetrics.checked === true
&& codeQueueLongPromptMetrics.createOk === true
&& /^codex_\d+_[A-Za-z0-9_-]+$/u.test(String(codeQueueLongPromptMetrics.taskId || ""))
&& Number(codeQueueLongPromptMetrics.promptChars || 0) > 2500
&& codeQueueLongPromptMetrics.checks?.createOriginalHasTail === true
&& codeQueueLongPromptMetrics.checks?.overviewOriginalHasTail === true
&& codeQueueLongPromptMetrics.checks?.selectedOriginalHasTail === true
&& codeQueueLongPromptMetrics.checks?.metaOriginalHasTail === true
&& codeQueueLongPromptMetrics.checks?.initialPromptHasTail === true
&& codeQueueLongPromptMetrics.checks?.uiFullPromptHasTail === true
&& codeQueueLongPromptMetrics.checks?.originalsHaveNoTruncationMarker === true
&& codeQueueLongPromptMetrics.checks?.previewFieldsExplicit === true
&& codeQueueLongPromptMetrics.checks?.previewCanTruncate === true,
{ codeQueueLongPromptMetrics });
addSelectedCheck(checks, options, "frontend:code-queue-error-red-markers", codeQueueErrorHighlightMetrics.checked === true && codeQueueErrorHighlightMetrics.candidateFound === true && codeQueueErrorHighlightMetrics.ok === true, { codeQueueErrorHighlightMetrics }); addSelectedCheck(checks, options, "frontend:code-queue-error-red-markers", codeQueueErrorHighlightMetrics.checked === true && codeQueueErrorHighlightMetrics.candidateFound === true && codeQueueErrorHighlightMetrics.ok === true, { codeQueueErrorHighlightMetrics });
addSelectedCheck(checks, options, "frontend:code-queue-initial-prompt-full-expand", addSelectedCheck(checks, options, "frontend:code-queue-initial-prompt-full-expand",
codexInitialPromptFullMetrics.candidateFound === false codexInitialPromptFullMetrics.candidateFound === false
+3 -18
View File
@@ -535,17 +535,7 @@ function promptLineCount(text: string): number {
return text.length > 0 ? text.split(/\r\n|\r|\n/u).length : 0; return text.length > 0 ? text.split(/\r\n|\r|\n/u).length : 0;
} }
function previewTextField(value: any): string { function taskDisplayPrompt(task: any): string {
if (typeof value === "string") return value;
if (value && typeof value === "object" && !Array.isArray(value) && typeof value.text === "string") return value.text;
return "";
}
function taskDisplayPrompt(task: any, options: AnyRecord = {}): string {
if (options.preview === true) {
const preview = previewTextField(task?.displayPromptPreview || task?.basePromptPreview || task?.promptPreview);
if (preview.length > 0) return preview;
}
const explicit = String(task?.displayPrompt || ""); const explicit = String(task?.displayPrompt || "");
if (explicit.length > 0) return explicit; if (explicit.length > 0) return explicit;
const prompt = String(task?.prompt || ""); const prompt = String(task?.prompt || "");
@@ -650,9 +640,7 @@ function traceSummaryIsCurrent(task: any): boolean {
function taskBasePromptText(task: any): string { function taskBasePromptText(task: any): string {
const summaryPrompt = taskPromptSummary(task); const summaryPrompt = taskPromptSummary(task);
const basePrompt = String(summaryPrompt.basePrompt || ""); const basePrompt = String(summaryPrompt.basePrompt || "");
if (basePrompt.length > 0) return basePrompt; return basePrompt.length > 0 ? basePrompt : taskDisplayPrompt(task);
const direct = String(task?.basePrompt || "");
return direct.length > 0 ? direct : taskDisplayPrompt(task);
} }
function taskFinalResponseText(task: any): string { function taskFinalResponseText(task: any): string {
@@ -1485,7 +1473,7 @@ function TaskCard({ task, selected, onSelect, onCopy, onReference, onMarkRead, c
}, markingRead ? "标记中" : "标为已读") : null, }, markingRead ? "标记中" : "标为已读") : null,
), ),
), ),
h("strong", null, shortText(taskDisplayPrompt(task, { preview: true }), 120) || "空任务"), h("strong", null, shortText(taskDisplayPrompt(task), 120) || "空任务"),
h("div", { className: "codex-task-meta" }, h("div", { className: "codex-task-meta" },
h("span", null, `queue=${taskQueueLabel(task)}`), h("span", null, `queue=${taskQueueLabel(task)}`),
h("span", null, `provider=${task?.providerId || "D601"}`), h("span", null, `provider=${task?.providerId || "D601"}`),
@@ -2241,9 +2229,6 @@ export function CodeQueuePage({ microservices, onRaw, apiBaseUrl = "/api", initi
task?.providerId, task?.providerId,
task?.model, task?.model,
task?.cwd, task?.cwd,
previewTextField(task?.displayPromptPreview),
previewTextField(task?.basePromptPreview),
previewTextField(task?.promptPreview),
task?.displayPrompt, task?.displayPrompt,
task?.basePrompt, task?.basePrompt,
task?.prompt, task?.prompt,
@@ -497,6 +497,16 @@ fn preview(text: &str, max_chars: usize) -> String {
result result
} }
fn prefix_preview(text: &str, max_chars: usize) -> String {
if text.chars().count() <= max_chars {
return text.to_string();
}
let take = max_chars.saturating_sub(1);
let mut result = text.chars().take(take).collect::<String>();
result.push('…');
result
}
fn strip_after_marker(text: &str, marker: &str) -> Option<String> { fn strip_after_marker(text: &str, marker: &str) -> Option<String> {
text.find(marker).map(|index| text[index + marker.len()..].trim_start().to_string()) text.find(marker).map(|index| text[index + marker.len()..].trim_start().to_string())
} }
@@ -601,17 +611,6 @@ fn final_response(task: &TaskMeta) -> String {
.to_string() .to_string()
} }
fn text_preview(value: &str, max_chars: usize) -> Value {
let chars = value.chars().count();
let truncated = chars > max_chars;
json!({
"text": if truncated { preview(value, max_chars) } else { value.to_string() },
"chars": chars,
"truncated": truncated,
"omittedChars": if truncated { chars.saturating_sub(max_chars.saturating_sub(20)) } else { 0 }
})
}
fn text_from_output(item: &Value) -> String { fn text_from_output(item: &Value) -> String {
item.get("text").and_then(Value::as_str).unwrap_or("").to_string() item.get("text").and_then(Value::as_str).unwrap_or("").to_string()
} }
@@ -665,17 +664,13 @@ fn task_list_response(_state: &AppState, task: &TaskMeta, lite: bool) -> Value {
}; };
let final_text = final_response(task); let final_text = final_response(task);
let agent_port = code_agent_port(&task.model); let agent_port = code_agent_port(&task.model);
let preview_limit = if lite { 360 } else { 2000 };
json!({ json!({
"id": task.id, "id": task.id,
"queueId": task.queue_id, "queueId": task.queue_id,
"queueEnteredAt": task.task_json.as_ref().and_then(|value| value.get("queueEnteredAt")).and_then(Value::as_str).unwrap_or(&task.created_at), "queueEnteredAt": task.task_json.as_ref().and_then(|value| value.get("queueEnteredAt")).and_then(Value::as_str).unwrap_or(&task.created_at),
"prompt": task.prompt, "prompt": if lite { prefix_preview(&display_prompt, 360) } else { preview(&task.prompt, 2000) },
"basePrompt": task.base_prompt, "basePrompt": if lite { prefix_preview(&task.base_prompt, 360) } else { preview(&task.base_prompt, 2000) },
"displayPrompt": display_prompt, "displayPrompt": if lite { prefix_preview(&display_prompt, 360) } else { preview(&display_prompt, 2000) },
"promptPreview": text_preview(&task.prompt, preview_limit),
"basePromptPreview": text_preview(&task.base_prompt, preview_limit),
"displayPromptPreview": text_preview(&display_prompt, preview_limit),
"promptChars": task.prompt.chars().count(), "promptChars": task.prompt.chars().count(),
"basePromptChars": task.base_prompt.chars().count(), "basePromptChars": task.base_prompt.chars().count(),
"displayPromptChars": display_prompt.chars().count(), "displayPromptChars": display_prompt.chars().count(),
@@ -734,15 +729,8 @@ fn task_meta_response(state: &AppState, task: &TaskMeta) -> Value {
let map = base.as_object_mut().expect("task response object"); let map = base.as_object_mut().expect("task response object");
let output_count = json_array_len(task.task_json.as_ref(), "output", task.output_count); let output_count = json_array_len(task.task_json.as_ref(), "output", task.output_count);
let prompt_history_count = json_array_len(task.task_json.as_ref(), "promptHistory", 0); let prompt_history_count = json_array_len(task.task_json.as_ref(), "promptHistory", 0);
let display_prompt = if task.base_prompt.is_empty() {
user_prompt_for_display(&task.prompt)
} else {
task.base_prompt.clone()
};
map.insert("prompt".to_string(), json!(task.prompt)); map.insert("prompt".to_string(), json!(task.prompt));
map.insert("basePrompt".to_string(), json!(task.base_prompt)); map.insert("basePrompt".to_string(), json!(task.base_prompt));
map.insert("displayPrompt".to_string(), json!(display_prompt));
map.insert("initialPrompt".to_string(), json!(task.prompt));
map.insert("finalResponse".to_string(), json!(final_response(task))); map.insert("finalResponse".to_string(), json!(final_response(task)));
map.insert( map.insert(
"promptHistory".to_string(), "promptHistory".to_string(),
@@ -1,6 +1,5 @@
import postgres from "postgres"; import postgres from "postgres";
import { createHourlyJsonlWriter, logRetentionBytesForService } from "../../../shared/src/rotating-jsonl"; import { createHourlyJsonlWriter, logRetentionBytesForService } from "../../../shared/src/rotating-jsonl";
import { codeQueuePromptResponseFields, safePreview } from "./prompt-observation";
type JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }; type JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue };
type JsonRecord = Record<string, JsonValue>; type JsonRecord = Record<string, JsonValue>;
@@ -435,6 +434,12 @@ function timestampMs(value: string | Date | null | undefined): number | null {
return Number.isFinite(ms) ? ms : null; return Number.isFinite(ms) ? ms : null;
} }
function safePreview(value: unknown, maxChars: number): string {
const text = String(value ?? "");
if (text.length <= maxChars) return text;
return `${text.slice(0, Math.max(0, maxChars - 20))}\n...<truncated>`;
}
function prefixPreview(value: unknown, maxChars: number): string { function prefixPreview(value: unknown, maxChars: number): string {
const text = String(value ?? ""); const text = String(value ?? "");
return text.length <= maxChars ? text : `${text.slice(0, Math.max(0, maxChars - 1))}`; return text.length <= maxChars ? text : `${text.slice(0, Math.max(0, maxChars - 1))}`;
@@ -1630,12 +1635,17 @@ function taskStatisticsSummary(tasks: QueueTask[], days = 14): JsonRecord {
} }
function taskListResponse(task: QueueTask, lite = true): JsonRecord { function taskListResponse(task: QueueTask, lite = true): JsonRecord {
const promptFields = codeQueuePromptResponseFields(task, { lite, userPromptForDisplay }); const displayPrompt = task.basePrompt || userPromptForDisplay(task.prompt);
return { return {
id: task.id, id: task.id,
queueId: queueIdOf(task), queueId: queueIdOf(task),
queueEnteredAt: task.queueEnteredAt, queueEnteredAt: task.queueEnteredAt,
...promptFields, prompt: lite ? prefixPreview(displayPrompt, 360) : safePreview(displayPrompt, 2000),
basePrompt: lite ? prefixPreview(task.basePrompt, 360) : safePreview(task.basePrompt, 2000),
displayPrompt: lite ? prefixPreview(displayPrompt, 360) : safePreview(displayPrompt, 2000),
promptChars: task.prompt.length,
basePromptChars: task.basePrompt.length,
displayPromptChars: displayPrompt.length,
promptEditable: queuedTaskPromptEditable(task), promptEditable: queuedTaskPromptEditable(task),
finalResponseChars: task.finalResponse.length, finalResponseChars: task.finalResponse.length,
stepCount: numberField(task.stepCount ?? task.llmStepCount, 0), stepCount: numberField(task.stepCount ?? task.llmStepCount, 0),
@@ -1685,8 +1695,6 @@ function taskMetaResponse(task: QueueTask): JsonRecord {
...taskListResponse(task, false), ...taskListResponse(task, false),
prompt: task.prompt, prompt: task.prompt,
basePrompt: task.basePrompt, basePrompt: task.basePrompt,
displayPrompt: task.basePrompt || userPromptForDisplay(task.prompt),
initialPrompt: task.prompt,
finalResponse: task.finalResponse, finalResponse: task.finalResponse,
promptHistory: toJsonValue(task.promptHistory), promptHistory: toJsonValue(task.promptHistory),
attempts: toJsonValue(task.attempts), attempts: toJsonValue(task.attempts),
@@ -88,7 +88,7 @@ rules:
resources: ["namespaces"] resources: ["namespaces"]
verbs: ["get", "list", "watch", "create", "delete", "patch"] verbs: ["get", "list", "watch", "create", "delete", "patch"]
- apiGroups: [""] - apiGroups: [""]
resources: ["configmaps", "secrets", "services", "services/proxy", "pods", "pods/log"] resources: ["configmaps", "services", "pods", "pods/log"]
verbs: ["get", "list", "watch", "create", "delete", "patch"] verbs: ["get", "list", "watch", "create", "delete", "patch"]
- apiGroups: ["apps"] - apiGroups: ["apps"]
resources: ["deployments"] resources: ["deployments"]
@@ -701,8 +701,6 @@ spec:
backend_commit="$(params.backend-core-commit)" backend_commit="$(params.backend-core-commit)"
frontend_commit="$(params.frontend-commit)" frontend_commit="$(params.frontend-commit)"
code_queue_commit="$(params.code-queue-commit)" code_queue_commit="$(params.code-queue-commit)"
app_image="$(params.app-image)"
database_url="postgres://unidesk_ci:unidesk_ci_password@postgres-dev.$ns.svc.cluster.local:5432/unidesk_ci"
kube_api="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT_HTTPS}" kube_api="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT_HTTPS}"
kube_token="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" kube_token="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
kube_ca="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" kube_ca="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
@@ -721,48 +719,6 @@ spec:
cat /tmp/unidesk-dev-e2e-delete-response >&2 cat /tmp/unidesk-dev-e2e-delete-response >&2
return 1 return 1
} }
proxy_get() {
local service_name="$1"
local path="$2"
curl -fsS --cacert "$kube_ca" -H "Authorization: Bearer $kube_token" \
"$kube_api/api/v1/namespaces/$ns/services/${service_name}:4222/proxy$path"
}
proxy_json() {
local method="$1"
local service_name="$2"
local path="$3"
local body="${4:-}"
if [ -n "$body" ]; then
curl -fsS --cacert "$kube_ca" -H "Authorization: Bearer $kube_token" -H "Content-Type: application/json" \
-X "$method" --data "$body" \
"$kube_api/api/v1/namespaces/$ns/services/${service_name}:4222/proxy$path"
else
curl -fsS --cacert "$kube_ca" -H "Authorization: Bearer $kube_token" \
-X "$method" \
"$kube_api/api/v1/namespaces/$ns/services/${service_name}:4222/proxy$path"
fi
}
wait_deployment_available() {
local deployment="$1"
local timeout_seconds="$2"
local deadline=$((SECONDS + timeout_seconds))
while [ "$SECONDS" -lt "$deadline" ]; do
status="$(kube GET "$kube_api/apis/apps/v1/namespaces/$ns/deployments/$deployment")"
replicas="$(printf '%s' "$status" | jq -r '.spec.replicas // 1')"
available="$(printf '%s' "$status" | jq -r '.status.availableReplicas // 0')"
updated="$(printf '%s' "$status" | jq -r '.status.updatedReplicas // 0')"
observed="$(printf '%s' "$status" | jq -r '.status.observedGeneration // 0')"
generation="$(printf '%s' "$status" | jq -r '.metadata.generation // 0')"
if [ "$available" -ge "$replicas" ] && [ "$updated" -ge "$replicas" ] && [ "$observed" -ge "$generation" ]; then
echo "dev_e2e_rollout=available deployment=$deployment namespace=$ns replicas=$available generation=$generation"
return 0
fi
sleep 2
done
echo "dev_e2e_rollout=timeout deployment=$deployment namespace=$ns" >&2
kube GET "$kube_api/apis/apps/v1/namespaces/$ns/deployments/$deployment" >&2
return 1
}
cleanup() { cleanup() {
if [ "$keep" = "true" ]; then if [ "$keep" = "true" ]; then
echo "dev_e2e_namespace_retained=$ns" echo "dev_e2e_namespace_retained=$ns"
@@ -811,640 +767,162 @@ spec:
--data-binary @/tmp/dev-e2e-configmap.yaml \ --data-binary @/tmp/dev-e2e-configmap.yaml \
"$kube_api/api/v1/namespaces/$ns/configmaps/desired-manifest?fieldManager=unidesk-ci&force=true" >/dev/null "$kube_api/api/v1/namespaces/$ns/configmaps/desired-manifest?fieldManager=unidesk-ci&force=true" >/dev/null
database_url_b64="$(printf '%s' "$database_url" | base64 -w0)" cat >/tmp/dev-e2e-target.yaml <<YAML
cat >/tmp/dev-e2e-secret.yaml <<YAML
apiVersion: v1
kind: Secret
metadata:
name: code-queue-dev-e2e-env
namespace: $ns
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: ci-dev-e2e
app.kubernetes.io/part-of: unidesk
type: Opaque
data:
DATABASE_URL: "$database_url_b64"
YAML
kube PATCH \
-H "Content-Type: application/apply-patch+yaml" \
--data-binary @/tmp/dev-e2e-secret.yaml \
"$kube_api/api/v1/namespaces/$ns/secrets/code-queue-dev-e2e-env?fieldManager=unidesk-ci&force=true" >/dev/null
cat >/tmp/dev-e2e-postgres.yaml <<YAML
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
metadata: metadata:
name: postgres-dev name: dev-e2e-target
namespace: $ns namespace: $ns
labels: labels:
app.kubernetes.io/name: postgres-dev app.kubernetes.io/name: unidesk-dev-namespace-e2e
app.kubernetes.io/component: database app.kubernetes.io/component: smoke-target
app.kubernetes.io/part-of: unidesk app.kubernetes.io/part-of: unidesk
spec: spec:
replicas: 1 replicas: 1
selector: selector:
matchLabels: matchLabels:
app.kubernetes.io/name: postgres-dev app.kubernetes.io/name: unidesk-dev-namespace-e2e
app.kubernetes.io/component: database app.kubernetes.io/component: smoke-target
template: template:
metadata: metadata:
labels: labels:
app.kubernetes.io/name: postgres-dev app.kubernetes.io/name: unidesk-dev-namespace-e2e
app.kubernetes.io/component: database app.kubernetes.io/component: smoke-target
app.kubernetes.io/part-of: unidesk app.kubernetes.io/part-of: unidesk
spec: spec:
nodeSelector: nodeSelector:
unidesk.ai/node-id: D601 unidesk.ai/node-id: D601
terminationGracePeriodSeconds: 5 terminationGracePeriodSeconds: 5
containers: containers:
- name: postgres - name: smoke-target
image: postgres:16-alpine image: "$(params.app-image)"
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
ports: command:
- name: postgres - bun
containerPort: 5432 - -e
env: - |
- name: POSTGRES_USER const port = Number(process.env.PORT || 8080);
value: unidesk_ci const payload = {
- name: POSTGRES_PASSWORD ok: true,
value: unidesk_ci_password environment: "dev",
- name: POSTGRES_DB namespace: process.env.CI_E2E_NAMESPACE,
value: unidesk_ci deployCommit: process.env.CI_E2E_DEPLOY_COMMIT,
readinessProbe: backendCoreCommit: process.env.BACKEND_CORE_COMMIT,
exec: frontendCommit: process.env.FRONTEND_COMMIT,
command: codeQueueCommit: process.env.CODE_QUEUE_COMMIT
- pg_isready };
- -U Bun.serve({
- unidesk_ci hostname: "0.0.0.0",
- -d port,
- unidesk_ci fetch(req) {
periodSeconds: 5 const url = new URL(req.url);
timeoutSeconds: 3 if (url.pathname === "/health" || url.pathname === "/") {
failureThreshold: 24 return Response.json(payload);
resources: }
requests: return new Response("not found", { status: 404 });
cpu: 50m }
memory: 128Mi });
limits: console.log(JSON.stringify({ listening: port, ...payload }));
cpu: 250m await new Promise(() => {});
memory: 512Mi
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: postgres-dev
namespace: $ns
labels:
app.kubernetes.io/name: postgres-dev
app.kubernetes.io/component: database
app.kubernetes.io/part-of: unidesk
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: postgres-dev
app.kubernetes.io/component: database
ports:
- name: postgres
port: 5432
targetPort: postgres
YAML
csplit -s -f /tmp/dev-e2e-postgres- /tmp/dev-e2e-postgres.yaml '/^---$/' '{*}'
kube PATCH \
-H "Content-Type: application/apply-patch+yaml" \
--data-binary @/tmp/dev-e2e-postgres-00 \
"$kube_api/apis/apps/v1/namespaces/$ns/deployments/postgres-dev?fieldManager=unidesk-ci&force=true" >/dev/null
kube PATCH \
-H "Content-Type: application/apply-patch+yaml" \
--data-binary @/tmp/dev-e2e-postgres-01 \
"$kube_api/api/v1/namespaces/$ns/services/postgres-dev?fieldManager=unidesk-ci&force=true" >/dev/null
wait_deployment_available postgres-dev 180
cat >/tmp/dev-e2e-code-queue.yaml <<YAML
apiVersion: apps/v1
kind: Deployment
metadata:
name: code-queue-scheduler-dev
namespace: $ns
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: scheduler
app.kubernetes.io/part-of: unidesk
unidesk.ai/ci-run-id: "$(params.run-id)"
unidesk.ai/deploy-service-id: code-queue
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: scheduler
unidesk.ai/ci-run-id: "$(params.run-id)"
template:
metadata:
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: scheduler
app.kubernetes.io/part-of: unidesk
unidesk.ai/ci-run-id: "$(params.run-id)"
unidesk.ai/node-id: D601
unidesk.ai/deploy-service-id: code-queue
spec:
nodeSelector:
unidesk.ai/node-id: D601
terminationGracePeriodSeconds: 10
containers:
- name: code-queue
image: "$app_image"
imagePullPolicy: Never
ports: ports:
- name: http - name: http
containerPort: 4222 containerPort: 8080
envFrom:
- secretRef:
name: code-queue-dev-e2e-env
env: env:
- name: HOST
value: "0.0.0.0"
- name: PORT - name: PORT
value: "4222" value: "8080"
- name: UNIDESK_ENV - name: CI_E2E_NAMESPACE
value: dev
- name: UNIDESK_NAMESPACE
value: "$ns" value: "$ns"
- name: UNIDESK_DATABASE_NAME - name: CI_E2E_DEPLOY_COMMIT
value: unidesk_ci value: "$(params.deploy-commit)"
- name: UNIDESK_DEPLOY_REF - name: BACKEND_CORE_COMMIT
value: origin/master:deploy.json#environments.dev value: "$backend_commit"
- name: UNIDESK_DEPLOY_SERVICE_ID - name: FRONTEND_COMMIT
value: code-queue value: "$frontend_commit"
- name: CODE_QUEUE_DEPLOY_COMMIT - name: CODE_QUEUE_COMMIT
value: "$code_queue_commit" value: "$code_queue_commit"
- name: CODE_QUEUE_DEPLOY_REQUESTED_COMMIT
value: "$code_queue_commit"
- name: CODE_QUEUE_INSTANCE_ID
value: D601-dev-ci-scheduler
- name: CODE_QUEUE_SERVICE_ROLE
value: scheduler
- name: CODE_QUEUE_SCHEDULER_ENABLED
value: "true"
- name: CODE_QUEUE_MAX_ACTIVE_QUEUES
value: "0"
- name: CODE_QUEUE_DATABASE_POOL_MAX
value: "2"
- name: CODE_QUEUE_MAIN_PROVIDER_ID
value: D601-dev
- name: CODE_QUEUE_EXECUTION_PROVIDER_IDS
value: D601-dev
- name: CODE_QUEUE_WORKDIR
value: /workspace-dev
- name: CODE_QUEUE_REMOTE_WORKDIR
value: /home/ubuntu/unidesk-dev-workspace
- name: CODE_QUEUE_DATA_DIR
value: /var/lib/unidesk-dev-e2e/code-queue
- name: CODE_QUEUE_CODEX_HOME
value: /var/lib/unidesk-dev-e2e/code-queue/codex-home
- name: CODE_QUEUE_OPENCODE_XDG_DIR
value: /var/lib/unidesk-dev-e2e/code-queue/opencode-xdg
- name: CODE_QUEUE_EGRESS_PROXY_ENABLED
value: "false"
- name: CODE_QUEUE_NOTIFY_CLAUDEQQ_ENABLED
value: "false"
- name: CODE_QUEUE_STARTUP_OA_BACKFILL_ENABLED
value: "false"
- name: CODE_QUEUE_CODEX_SQLITE_LOG_EXPORT_ENABLED
value: "false"
- name: OA_EVENT_FLOW_BASE_URL
value: http://127.0.0.1:9
- name: LOG_FILE
value: /var/log/unidesk/code-queue-scheduler-dev-e2e.jsonl
- name: NODE_OPTIONS
value: --max-old-space-size=512
readinessProbe: readinessProbe:
httpGet: httpGet:
path: /health path: /health
port: http port: http
periodSeconds: 5 periodSeconds: 3
timeoutSeconds: 3 timeoutSeconds: 2
failureThreshold: 24 failureThreshold: 20
startupProbe:
httpGet:
path: /live
port: http
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 60
resources: resources:
requests: requests:
cpu: 100m cpu: 20m
memory: 64Mi
limits:
memory: 256Mi memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
volumeMounts:
- name: state
mountPath: /var/lib/unidesk-dev-e2e/code-queue
- name: logs
mountPath: /var/log/unidesk
volumes:
- name: state
emptyDir: {}
- name: logs
emptyDir: {}
--- ---
apiVersion: v1 apiVersion: v1
kind: Service kind: Service
metadata: metadata:
name: code-queue-scheduler-dev name: dev-e2e-target
namespace: $ns namespace: $ns
labels: labels:
app.kubernetes.io/name: code-queue app.kubernetes.io/name: unidesk-dev-namespace-e2e
app.kubernetes.io/component: scheduler app.kubernetes.io/component: smoke-target
app.kubernetes.io/part-of: unidesk app.kubernetes.io/part-of: unidesk
spec: spec:
type: ClusterIP type: ClusterIP
selector: selector:
app.kubernetes.io/name: code-queue app.kubernetes.io/name: unidesk-dev-namespace-e2e
app.kubernetes.io/component: scheduler app.kubernetes.io/component: smoke-target
unidesk.ai/ci-run-id: "$(params.run-id)"
ports: ports:
- name: http - name: http
port: 4222 port: 8080
targetPort: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: code-queue-read-dev
namespace: $ns
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: read
app.kubernetes.io/part-of: unidesk
unidesk.ai/ci-run-id: "$(params.run-id)"
unidesk.ai/deploy-service-id: code-queue
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: read
unidesk.ai/ci-run-id: "$(params.run-id)"
template:
metadata:
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: read
app.kubernetes.io/part-of: unidesk
unidesk.ai/ci-run-id: "$(params.run-id)"
unidesk.ai/node-id: D601
unidesk.ai/deploy-service-id: code-queue
spec:
nodeSelector:
unidesk.ai/node-id: D601
terminationGracePeriodSeconds: 10
containers:
- name: code-queue
image: "$app_image"
imagePullPolicy: Never
ports:
- name: http
containerPort: 4222
envFrom:
- secretRef:
name: code-queue-dev-e2e-env
env:
- name: HOST
value: "0.0.0.0"
- name: PORT
value: "4222"
- name: UNIDESK_ENV
value: dev
- name: UNIDESK_NAMESPACE
value: "$ns"
- name: UNIDESK_DATABASE_NAME
value: unidesk_ci
- name: UNIDESK_DEPLOY_REF
value: origin/master:deploy.json#environments.dev
- name: UNIDESK_DEPLOY_SERVICE_ID
value: code-queue
- name: CODE_QUEUE_DEPLOY_COMMIT
value: "$code_queue_commit"
- name: CODE_QUEUE_DEPLOY_REQUESTED_COMMIT
value: "$code_queue_commit"
- name: CODE_QUEUE_INSTANCE_ID
value: D601-dev-ci-read
- name: CODE_QUEUE_SERVICE_ROLE
value: read
- name: CODE_QUEUE_SCHEDULER_ENABLED
value: "false"
- name: CODE_QUEUE_DATABASE_POOL_MAX
value: "2"
- name: CODE_QUEUE_MAIN_PROVIDER_ID
value: D601-dev
- name: CODE_QUEUE_EXECUTION_PROVIDER_IDS
value: D601-dev
- name: CODE_QUEUE_WORKDIR
value: /workspace-dev
- name: CODE_QUEUE_REMOTE_WORKDIR
value: /home/ubuntu/unidesk-dev-workspace
- name: CODE_QUEUE_DATA_DIR
value: /var/lib/unidesk-dev-e2e/code-queue
- name: CODE_QUEUE_CODEX_HOME
value: /var/lib/unidesk-dev-e2e/code-queue/codex-home
- name: CODE_QUEUE_OPENCODE_XDG_DIR
value: /var/lib/unidesk-dev-e2e/code-queue/opencode-xdg
- name: CODE_QUEUE_EGRESS_PROXY_ENABLED
value: "false"
- name: CODE_QUEUE_NOTIFY_CLAUDEQQ_ENABLED
value: "false"
- name: CODE_QUEUE_STARTUP_OA_BACKFILL_ENABLED
value: "false"
- name: CODE_QUEUE_CODEX_SQLITE_LOG_EXPORT_ENABLED
value: "false"
- name: OA_EVENT_FLOW_BASE_URL
value: http://127.0.0.1:9
- name: LOG_FILE
value: /var/log/unidesk/code-queue-read-dev-e2e.jsonl
- name: NODE_OPTIONS
value: --max-old-space-size=512
readinessProbe:
httpGet:
path: /live
port: http
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 24
startupProbe:
httpGet:
path: /live
port: http
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 60
resources:
requests:
cpu: 80m
memory: 192Mi
limits:
cpu: 300m
memory: 768Mi
volumeMounts:
- name: state
mountPath: /var/lib/unidesk-dev-e2e/code-queue
- name: logs
mountPath: /var/log/unidesk
volumes:
- name: state
emptyDir: {}
- name: logs
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: code-queue-read-dev
namespace: $ns
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: read
app.kubernetes.io/part-of: unidesk
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: read
unidesk.ai/ci-run-id: "$(params.run-id)"
ports:
- name: http
port: 4222
targetPort: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: code-queue-write-dev
namespace: $ns
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: write
app.kubernetes.io/part-of: unidesk
unidesk.ai/ci-run-id: "$(params.run-id)"
unidesk.ai/deploy-service-id: code-queue
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: write
unidesk.ai/ci-run-id: "$(params.run-id)"
template:
metadata:
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: write
app.kubernetes.io/part-of: unidesk
unidesk.ai/ci-run-id: "$(params.run-id)"
unidesk.ai/node-id: D601
unidesk.ai/deploy-service-id: code-queue
spec:
nodeSelector:
unidesk.ai/node-id: D601
terminationGracePeriodSeconds: 10
containers:
- name: code-queue
image: "$app_image"
imagePullPolicy: Never
ports:
- name: http
containerPort: 4222
envFrom:
- secretRef:
name: code-queue-dev-e2e-env
env:
- name: HOST
value: "0.0.0.0"
- name: PORT
value: "4222"
- name: UNIDESK_ENV
value: dev
- name: UNIDESK_NAMESPACE
value: "$ns"
- name: UNIDESK_DATABASE_NAME
value: unidesk_ci
- name: UNIDESK_DEPLOY_REF
value: origin/master:deploy.json#environments.dev
- name: UNIDESK_DEPLOY_SERVICE_ID
value: code-queue
- name: CODE_QUEUE_DEPLOY_COMMIT
value: "$code_queue_commit"
- name: CODE_QUEUE_DEPLOY_REQUESTED_COMMIT
value: "$code_queue_commit"
- name: CODE_QUEUE_INSTANCE_ID
value: D601-dev-ci-write
- name: CODE_QUEUE_SERVICE_ROLE
value: write
- name: CODE_QUEUE_SCHEDULER_ENABLED
value: "false"
- name: CODE_QUEUE_DATABASE_POOL_MAX
value: "2"
- name: CODE_QUEUE_MAIN_PROVIDER_ID
value: D601-dev
- name: CODE_QUEUE_EXECUTION_PROVIDER_IDS
value: D601-dev
- name: CODE_QUEUE_WORKDIR
value: /workspace-dev
- name: CODE_QUEUE_REMOTE_WORKDIR
value: /home/ubuntu/unidesk-dev-workspace
- name: CODE_QUEUE_DATA_DIR
value: /var/lib/unidesk-dev-e2e/code-queue
- name: CODE_QUEUE_CODEX_HOME
value: /var/lib/unidesk-dev-e2e/code-queue/codex-home
- name: CODE_QUEUE_OPENCODE_XDG_DIR
value: /var/lib/unidesk-dev-e2e/code-queue/opencode-xdg
- name: CODE_QUEUE_EGRESS_PROXY_ENABLED
value: "false"
- name: CODE_QUEUE_NOTIFY_CLAUDEQQ_ENABLED
value: "false"
- name: CODE_QUEUE_STARTUP_OA_BACKFILL_ENABLED
value: "false"
- name: CODE_QUEUE_CODEX_SQLITE_LOG_EXPORT_ENABLED
value: "false"
- name: OA_EVENT_FLOW_BASE_URL
value: http://127.0.0.1:9
- name: LOG_FILE
value: /var/log/unidesk/code-queue-write-dev-e2e.jsonl
- name: NODE_OPTIONS
value: --max-old-space-size=512
readinessProbe:
httpGet:
path: /health
port: http
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 24
startupProbe:
httpGet:
path: /live
port: http
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 60
resources:
requests:
cpu: 80m
memory: 192Mi
limits:
cpu: 300m
memory: 768Mi
volumeMounts:
- name: state
mountPath: /var/lib/unidesk-dev-e2e/code-queue
- name: logs
mountPath: /var/log/unidesk
volumes:
- name: state
emptyDir: {}
- name: logs
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: code-queue-write-dev
namespace: $ns
labels:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: write
app.kubernetes.io/part-of: unidesk
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: code-queue
app.kubernetes.io/component: write
unidesk.ai/ci-run-id: "$(params.run-id)"
ports:
- name: http
port: 4222
targetPort: http targetPort: http
YAML YAML
csplit -s -f /tmp/dev-e2e-code-queue- /tmp/dev-e2e-code-queue.yaml '/^---$/' '{*}' csplit -s -f /tmp/dev-e2e-target- /tmp/dev-e2e-target.yaml '/^---$/' '{*}'
kube PATCH -H "Content-Type: application/apply-patch+yaml" --data-binary @/tmp/dev-e2e-code-queue-00 "$kube_api/apis/apps/v1/namespaces/$ns/deployments/code-queue-scheduler-dev?fieldManager=unidesk-ci&force=true" >/dev/null kube PATCH \
kube PATCH -H "Content-Type: application/apply-patch+yaml" --data-binary @/tmp/dev-e2e-code-queue-01 "$kube_api/api/v1/namespaces/$ns/services/code-queue-scheduler-dev?fieldManager=unidesk-ci&force=true" >/dev/null -H "Content-Type: application/apply-patch+yaml" \
kube PATCH -H "Content-Type: application/apply-patch+yaml" --data-binary @/tmp/dev-e2e-code-queue-02 "$kube_api/apis/apps/v1/namespaces/$ns/deployments/code-queue-read-dev?fieldManager=unidesk-ci&force=true" >/dev/null --data-binary @/tmp/dev-e2e-target-00 \
kube PATCH -H "Content-Type: application/apply-patch+yaml" --data-binary @/tmp/dev-e2e-code-queue-03 "$kube_api/api/v1/namespaces/$ns/services/code-queue-read-dev?fieldManager=unidesk-ci&force=true" >/dev/null "$kube_api/apis/apps/v1/namespaces/$ns/deployments/dev-e2e-target?fieldManager=unidesk-ci&force=true" >/dev/null
kube PATCH -H "Content-Type: application/apply-patch+yaml" --data-binary @/tmp/dev-e2e-code-queue-04 "$kube_api/apis/apps/v1/namespaces/$ns/deployments/code-queue-write-dev?fieldManager=unidesk-ci&force=true" >/dev/null kube PATCH \
kube PATCH -H "Content-Type: application/apply-patch+yaml" --data-binary @/tmp/dev-e2e-code-queue-05 "$kube_api/api/v1/namespaces/$ns/services/code-queue-write-dev?fieldManager=unidesk-ci&force=true" >/dev/null -H "Content-Type: application/apply-patch+yaml" \
wait_deployment_available code-queue-scheduler-dev 420 --data-binary @/tmp/dev-e2e-target-01 \
wait_deployment_available code-queue-read-dev 420 "$kube_api/api/v1/namespaces/$ns/services/dev-e2e-target?fieldManager=unidesk-ci&force=true" >/dev/null
wait_deployment_available code-queue-write-dev 420
proxy_get code-queue-write-dev /health >"$run_dir/code-queue-write-health.json" deadline=$((SECONDS + 180))
proxy_get code-queue-scheduler-dev /health >"$run_dir/code-queue-scheduler-health.json" while [ "$SECONDS" -lt "$deadline" ]; do
proxy_get code-queue-read-dev /live >"$run_dir/code-queue-read-live.json" status="$(kube GET "$kube_api/apis/apps/v1/namespaces/$ns/deployments/dev-e2e-target")"
proxy_get code-queue-read-dev /api/workdirs >"$run_dir/code-queue-read-workdirs.json" available="$(printf '%s' "$status" | jq -r '.status.availableReplicas // 0')"
proxy_json POST code-queue-write-dev /api/workdirs '{"providerId":"D601-dev","executionMode":"default","path":"/home/ubuntu/unidesk-dev-workspace/ci-workdirs-smoke"}' >"$run_dir/code-queue-workdir-created.json" observed="$(printf '%s' "$status" | jq -r '.status.observedGeneration // 0')"
proxy_get code-queue-write-dev '/api/workdirs?providerId=D601-dev&executionMode=default' >"$run_dir/code-queue-write-workdirs.json" generation="$(printf '%s' "$status" | jq -r '.metadata.generation // 0')"
proxy_json DELETE code-queue-write-dev /api/workdirs/D601-dev/default/%2Fhome%2Fubuntu%2Funidesk-dev-workspace%2Fci-workdirs-smoke >"$run_dir/code-queue-workdir-deleted.json" if [ "$available" -ge 1 ] && [ "$observed" -ge "$generation" ]; then
echo "dev_e2e_target_rollout=available namespace=$ns"
break
fi
sleep 2
done
if [ "$SECONDS" -ge "$deadline" ]; then
echo "dev_e2e_target_rollout=timeout namespace=$ns" >&2
kube GET "$kube_api/apis/apps/v1/namespaces/$ns/deployments/dev-e2e-target" >&2
exit 1
fi
python3 - "$ns" "$(params.deploy-commit)" "$backend_commit" "$frontend_commit" "$code_queue_commit" "$app_image" "$result_json" "$run_dir" <<'PY' bun - "$ns" "$(params.deploy-commit)" "$backend_commit" "$frontend_commit" "$code_queue_commit" "$result_json" <<'BUN'
import json const [ns, deployCommit, backendCommit, frontendCommit, codeQueueCommit, resultPath] = process.argv.slice(2);
import sys const url = `http://dev-e2e-target.${ns}.svc.cluster.local:8080/health`;
from pathlib import Path const started = performance.now();
const response = await fetch(url);
ns, deploy_commit, backend_commit, frontend_commit, code_queue_commit, app_image, result_path, run_dir = sys.argv[1:] const elapsedMs = Math.round(performance.now() - started);
run_path = Path(run_dir) const body = await response.json();
const checks = [
def read_json(name): response.ok,
with (run_path / name).open("r", encoding="utf-8") as handle: body.ok === true,
return json.load(handle) body.environment === "dev",
body.namespace === ns,
health = read_json("code-queue-write-health.json") body.deployCommit === deployCommit,
scheduler = read_json("code-queue-scheduler-health.json") body.backendCoreCommit === backendCommit,
read_live = read_json("code-queue-read-live.json") body.frontendCommit === frontendCommit,
initial_workdirs = read_json("code-queue-read-workdirs.json") body.codeQueueCommit === codeQueueCommit
created = read_json("code-queue-workdir-created.json") ];
listed = read_json("code-queue-write-workdirs.json") const result = { ok: checks.every(Boolean), elapsedMs, url, body };
deleted = read_json("code-queue-workdir-deleted.json") await Bun.write(resultPath, JSON.stringify(result, null, 2) + "\n");
checks = [ console.log(JSON.stringify(result));
health.get("ok") is True and health.get("role") == "write" and health.get("deploy", {}).get("commit") == code_queue_commit, if (!result.ok) process.exit(1);
scheduler.get("ok") is True and scheduler.get("role") == "scheduler" and scheduler.get("schedulerEnabled") is True, BUN
read_live.get("ok") is True and read_live.get("role") == "read",
initial_workdirs.get("ok") is True and isinstance(initial_workdirs.get("workdirs"), list),
created.get("ok") is True and created.get("workdir", {}).get("providerId") == "D601-dev" and created.get("workdir", {}).get("path") == "/home/ubuntu/unidesk-dev-workspace/ci-workdirs-smoke",
listed.get("ok") is True and any(item.get("path") == "/home/ubuntu/unidesk-dev-workspace/ci-workdirs-smoke" for item in listed.get("workdirs", [])),
deleted.get("ok") is True,
]
result = {
"ok": all(checks),
"namespace": ns,
"deployCommit": deploy_commit,
"backendCoreCommit": backend_commit,
"frontendCommit": frontend_commit,
"codeQueueCommit": code_queue_commit,
"codeQueueImage": app_image,
"accessPath": "kubernetes-api-service-proxy",
"services": ["code-queue-scheduler-dev", "code-queue-read-dev", "code-queue-write-dev"],
"health": health,
"scheduler": scheduler,
"readLive": read_live,
"initialWorkdirs": initial_workdirs,
"created": created,
"listed": listed,
"deleted": deleted,
}
Path(result_path).write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(json.dumps(result, ensure_ascii=False))
if not result["ok"]:
raise SystemExit(1)
PY
--- ---
apiVersion: tekton.dev/v1 apiVersion: tekton.dev/v1
kind: Pipeline kind: Pipeline