115 lines
14 KiB
Markdown
115 lines
14 KiB
Markdown
# G14 Provider Node
|
|
|
|
G14 is the current HWLAB DEV/PROD source and k3s/GitOps runtime truth, and it remains a UniDesk provider node for staging other infrastructure workloads. Its UniDesk provider id is `G14`; the local UniDesk worktree is `/root/unidesk`, and the native k3s kubeconfig is `/etc/rancher/k3s/k3s.yaml`.
|
|
|
|
G14's long-lived k3s control bridge is `k3sctl-adapter-g14`, a UniDesk direct service outside the k3s fault domain. It listens on the G14 host loopback port `127.0.0.1:4266` and is registered separately from the D601 `k3sctl-adapter`, so G14 infrastructure services can be built and tested without taking over user services that still run on D601.
|
|
|
|
For Code Queue and non-HWLAB CI/CD migration preparation, G14 uses native k3s labels `unidesk.ai/node-id=G14` and `unidesk.ai/provider-id=G14`. The G14 Code Queue manifests `src/components/microservices/k3sctl-adapter/k3s/code-queue.g14.k8s.yaml` and `src/components/microservices/k3sctl-adapter/k3s/code-queue.g14.k3s.json` are candidate staging artifacts only until an explicit production cutover is approved. Non-HWLAB production Code Queue, CI/CD and user-service execution must remain on D601 while D601 is carrying those services.
|
|
|
|
## HWLAB DEV/PROD Runtime
|
|
|
|
G14 hosts the current HWLAB DEV runtime in native k3s namespace `hwlab-dev`, and the same node/GitOps line is the HWLAB PROD target unless a newer HWLAB repo rule says otherwise. The canonical G14 HWLAB source workspace is `/root/hwlab` on branch `G14`, with `origin` set to `git@github.com:pikasTech/HWLAB.git`; before any G14 HWLAB work, this fixed workspace must be immediately fast-forwarded to the latest `origin/G14`. HWLAB source edits, CI/CD/GitOps script changes, render work, manual polling and runtime validation must be based on that updated workspace through UniDesk SSH passthrough. Do not use `/root/HWLAB`, `/home/ubuntu/hwlab`, `/workspace/hwlab`, D601 workspace or a master-server checkout as persistent HWLAB source truth. G14-local details are mirrored on the node in `/root/docs/hwlab-g14-workspace.md`.
|
|
|
|
The standard entry forms are:
|
|
|
|
```bash
|
|
tran G14:/root/hwlab script -- 'git fetch origin G14 && git pull --ff-only origin G14 && git status --short --branch && git remote -v'
|
|
tran G14 apply-patch < patch.diff
|
|
tran G14:k3s kubectl get pods -n hwlab-dev
|
|
```
|
|
|
|
`G14:k3s` is the only supported k3s route form. Do not use `ssh G14 k3s ...`; the first token must locate the distributed target, and the following tokens must be the operation.
|
|
|
|
If `/root/hwlab` has unrelated local changes when this sync starts, first determine whether they can be quickly merged with the latest `origin/G14`. Merge them immediately when they are mergeable; do not default to stash, discard or a behind worktree. Only when the changes cannot be automatically merged should they be isolated and the operation stopped for human decision. A behind fixed workspace is not a valid basis for precheck, new worktree creation, render, polling, deployment or runtime validation.
|
|
|
|
The G14 HWLAB runtime boundary is:
|
|
|
|
- Current DEV public endpoints are `http://74.48.78.17:17666/` and `http://74.48.78.17:17667/health/live`. D601 `16666/16667` is legacy/migration evidence only.
|
|
- Keep HWLAB Services as `ClusterIP` unless a repo-owned G14 GitOps rule explicitly exposes them. Public exposure should stay in the approved G14 edge/proxy path, not ad hoc NodePort or local port-forward.
|
|
- Use a G14-local PostgreSQL instance such as `hwlab-g14-postgres` and a G14-local `hwlab-cloud-api-dev-db` Secret for cloud-api durable runtime tests. Do not copy D601 database credentials.
|
|
- Use only G14-local Codex auth material and k8s Secrets authorized for HWLAB on G14; do not copy D601 or production auth material by hand.
|
|
- Set `HWLAB_CLOUD_API_PORT=6667` explicitly in the G14 cloud-api Deployment. Kubernetes otherwise injects a `HWLAB_CLOUD_API_PORT=tcp://...` Service environment variable that breaks the Node port parser.
|
|
- `HWLAB_PUBLIC_ENDPOINT` and health/live evidence must describe the G14 endpoint, not the old D601 production endpoint.
|
|
- Do not run HWLAB repository `check`, Playwright/browser smoke, image builds or other heavy validation on the master server. Run those through G14 `/root/hwlab`, G14 k3s/Tekton, or another explicitly approved external execution plane.
|
|
- Manual device-agent experiments for real hardware must be standalone resources in `hwlab-dev` such as `device-agent-71-freq` and must not patch existing HWLAB Deployments, Services, ArgoCD Applications, FRP, CD desired-state or public frontend routing unless a separate HWLAB change authorizes it.
|
|
- A D601 Windows `hwlab-gateway` may connect outbound to G14 DEV cloud-api as an external host bridge for Keil/serial/workspace access. That bridge does not make D601 the HWLAB runtime truth; it is only a hardware access provider behind the G14 device-agent/cloud-api path.
|
|
|
|
After the G14-local database is provisioned, run the HWLAB migration CLI only against the G14 DEV database with explicit non-production confirmations:
|
|
|
|
```bash
|
|
kubectl -n hwlab-dev exec deploy/hwlab-cloud-api -- \
|
|
node /app/cmd/hwlab-cloud-api/migrate.mjs \
|
|
--apply --confirm-dev --confirmed-non-production
|
|
```
|
|
|
|
Healthy G14 HWLAB runtime means the main Deployments and StatefulSets are Ready, `cloud-api` and `edge-proxy` return `/health/live` with `status=ok`, durable runtime checks pass, and the public G14 DEV endpoints report the expected revision. For a device-agent smoke, health also requires the standalone device-agent Service to answer in-cluster and the D601 Windows gateway session/resource/capability to be visible through G14 cloud-api.
|
|
|
|
## HWLAB v0.2 Expansion Line
|
|
|
|
HWLAB `v0.2` is an additive G14 expansion line. It must not rename, delete, repurpose, or mutate the existing `G14` source branch, `G14-gitops` branch, `hwlab-dev` namespace, `hwlab-prod` namespace, DEV public ports `17666/17667`, or PROD public ports `18666/18667`. Existing DEV/PROD CI/CD remains the stability baseline while `v0.2` is introduced beside it.
|
|
|
|
The fixed `v0.2` source branch is `v0.2`, forked from the current `G14` branch after the G14 long-term reference docs record this decision. The fixed G14 development workspace for that branch is:
|
|
|
|
```bash
|
|
tran G14:/root/hwlab-v02 script -- 'git status --short --branch && git remote -v'
|
|
```
|
|
|
|
`/root/hwlab-v02` is the long-lived `v0.2` development workspace, not a scratch clone or CI/CD source selector. It must track `origin/v0.2` with `origin git@github.com:pikasTech/HWLAB.git`; local dirty state, stale `HEAD`, and untracked `.worktree/` only affect human development. Existing `G14` work continues to use `/root/hwlab`; do not reuse `/root/hwlab` or `/root/hwlab/.worktree/*` as the `v0.2` fixed workspace.
|
|
|
|
`v0.2` CI/CD source selection is isolated in the dedicated bare repo `G14:/root/hwlab-v02-cicd.git`. UniDesk control-plane commands must fetch `origin/v0.2` into that repo and render from a commit-pinned detached worktree; they must not read the source commit from `/root/hwlab-v02` checkout state.
|
|
|
|
The fixed `v0.2` runtime namespace is `hwlab-v02`. The intended public FRP allocation is:
|
|
|
|
- Cloud Web browser entry: `http://74.48.78.17:19666/`.
|
|
- API/edge entry and live health: `http://74.48.78.17:19667/health/live`.
|
|
|
|
Master-side FRP server maintenance for HWLAB public ports is documented in `docs/reference/hwlab.md#hwlab-frp-维护`; keep the detailed allowlist, restart boundary and verification sequence there instead of duplicating another runbook in this G14 node reference.
|
|
|
|
The `v0.2` CI/CD integration must be additive: add a manual UniDesk trigger, dedicated CI/CD source repo, `devops-infra` git mirror/relay, GitOps desired-state lane, Argo CD Application, namespace resources, artifact catalog, and `deploy.json` environment only when they target `v0.2`/`hwlab-v02` explicitly. Do not add a `v0.2` branch poller or retarget the existing `G14` poller, DEV/PROD Argo Applications, DEV/PROD runtime paths, or existing namespace resources to bootstrap `v0.2`.
|
|
|
|
The `devops-infra` git mirror/relay remains manual and CLI-controlled, not CronJob-driven. The standard `v0.2` CI/CD trigger is `bun scripts/cli.ts hwlab g14 control-plane trigger-current --lane v02 --confirm`; this command must fetch `/root/hwlab-v02-cicd.git`, resolve the current `origin/v0.2` source commit, check the mirror's `localV02` ref before creating the PipelineRun, run one bounded manual `git-mirror sync` Job when the mirror is stale, and only continue after the mirror ref matches the current source commit. Use `hwlab g14 git-mirror sync --confirm` directly only for explicit mirror maintenance or diagnosis.
|
|
|
|
After a `v0.2` PipelineRun completes, treat runtime rollout and remote GitOps persistence as two separate checks. `hwlab g14 control-plane status --lane v02` is the runtime check: it must show the expected source commit, PipelineRun completed, Argo `Synced/Healthy`, public 19666/19667 probes passing, and Cloud Web asset probes such as `/app.js` readable. `hwlab g14 git-mirror status` is the persistence check: `cache.summary.pendingFlush` must be false and `cache.summary.githubInSync` true before declaring GitOps fully flushed back to GitHub. If runtime is healthy but `pendingFlush=true`, run `bun scripts/cli.ts hwlab g14 git-mirror flush --confirm` and poll the returned job with `bun scripts/cli.ts job status <jobId> --tail-bytes 12000`; do not replace this with raw `kubectl`, native `git push`, or a long SSH wait.
|
|
|
|
`/health/live` revision is owned by `hwlab-cloud-api`; it can legitimately differ from the source commit for a Cloud Web-only change. Do not call that difference a failed Cloud Web rollout when `webAssets.checks.htmlOk`, `webAssets.checks.appJsOk`, CSS probes, Argo health, and `hwlab-cloud-web` Deployment readiness have passed. For Cloud Web behavior changes, the public JS asset probe or a bounded browser/DOM check is stronger evidence than cloud-api `apiRevision`.
|
|
|
|
Do not turn `v0.2` expansion governance into a stack of broad compatibility gates. The stable control points are branch, dedicated CI/CD source repo, git mirror/relay refs, GitOps branch, namespace, runtime path, Argo Application, FRP ports and generated-output ownership. Legacy DEV/D601/main preflights that block the `v0.2` lane should be removed from that lane, not patched with fallback or legacy modes. Naming, RBAC scope, cleanup policy, resource quota and rollback order are design decisions or runbook entries unless they protect a concrete high-value risk that cannot be enforced by the fixed boundaries above.
|
|
|
|
## Node-Local VPN Proxy
|
|
|
|
G14 has a node-local VPN/proxy stack for infrastructure bootstrap and recovery downloads:
|
|
|
|
- Primary mixed HTTP/SOCKS proxy: `127.0.0.1:10808`.
|
|
- Backup Hysteria2 HTTP proxy: `127.0.0.1:11809`.
|
|
- Backup Hysteria2 SOCKS5 proxy: `127.0.0.1:11808`.
|
|
- Operator-only local details remain on G14 under `/root/docs/vpn-proxy-ops.md`; subscription URLs, node credentials and GUI database contents must not be copied into the UniDesk repository.
|
|
|
|
The G14 host persists this proxy configuration in these local files:
|
|
|
|
- `/etc/profile.d/unidesk-g14-proxy.sh` exports `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, lowercase aliases and `NO_PROXY` for new login shells. Set `UNIDESK_G14_DISABLE_PROXY=1` before shell startup to opt out.
|
|
- `/root/.npmrc` pins npm `proxy`, `https-proxy`, `noproxy` and retry settings for root-side bootstrap commands.
|
|
- `/root/.gitconfig` pins root Git HTTP/HTTPS proxy settings.
|
|
- `/root/.docker/config.json` pins Docker client proxy settings for commands and build contexts that honor Docker client proxy configuration.
|
|
- `/etc/systemd/system/docker.service.d/proxy.conf` pins Docker daemon pull proxy settings. Updating this drop-in requires `systemctl daemon-reload` and a Docker restart before the active daemon sees the new `NO_PROXY`; do not restart Docker while G14 provider-gateway, k3s bootstrap or image builds are in flight unless that interruption is intentional.
|
|
|
|
The `NO_PROXY` list must include localhost, the main server, private LAN ranges, k3s pod/service CIDRs, Kubernetes service domains and the loopback registry so that k3s, `127.0.0.1:5000`, Kubernetes API access and UniDesk control paths do not route through the VPN proxy.
|
|
|
|
The primary proxy can be used for G14 target-side image bootstrap when Docker Hub, npm, GitHub or Playwright downloads are unreliable through direct network or provider-gateway WS egress. For Docker build steps that use `127.0.0.1`, build with host networking so the build container reaches the host proxy:
|
|
|
|
```bash
|
|
docker build --network host \
|
|
--build-arg HTTP_PROXY=http://127.0.0.1:10808 \
|
|
--build-arg HTTPS_PROXY=http://127.0.0.1:10808 \
|
|
--build-arg ALL_PROXY=socks5h://127.0.0.1:10808 \
|
|
--build-arg http_proxy=http://127.0.0.1:10808 \
|
|
--build-arg https_proxy=http://127.0.0.1:10808 \
|
|
--build-arg all_proxy=socks5h://127.0.0.1:10808 \
|
|
...
|
|
```
|
|
|
|
`127.0.0.1:10808` is a G14 host loopback endpoint. Inside an ordinary k3s Pod, `127.0.0.1` is the Pod network namespace, not the node proxy. Do not set long-lived workload proxy env to `http://127.0.0.1:10808` unless that workload is intentionally `hostNetwork` and the port conflict/DEV-PROD blast radius has been reviewed. Temporary hostNetwork debug Pods may use the node-local proxy only for bounded bootstrap proof or cache prewarm; they must not become GitOps desired state just to make external downloads work.
|
|
|
|
The backup proxy uses `HTTP_PROXY=http://127.0.0.1:11809`, `HTTPS_PROXY=http://127.0.0.1:11809` and `ALL_PROXY=socks5h://127.0.0.1:11808`.
|
|
|
|
This proxy is not a replacement for UniDesk runtime egress. k3s workloads such as Code Queue must still use the cataloged `g14-provider-egress-proxy` Kubernetes Service and `g14-tcp-egress-gateway` for normal runtime access to PostgreSQL, OA Event Flow and external APIs. The node-local VPN proxy is allowed only for G14 host-side bootstrap, image build, cache prewarm or recovery steps, and those steps should record the proxy choice in issue or deployment evidence.
|