Image Mirroring (Air-Gapped)

Definite on-prem publishes its container images to the public GitHub Container Registry (ghcr.io/definite-app). Many regulated environments — banks, government, healthcare — will not let Kubernetes workloads pull from a public registry. They deploy only from an internal registry (Amazon ECR, Google Artifact Registry, Azure Container Registry, Harbor, Nexus, Quay, …) that runs image scanning and policy enforcement.

This page is the supported path for those environments: copy the Definite image set into your private registry, then point the Helm chart at it.

The image set

A Definite on-prem release ships these images. All container images are linux/amd64 only.

The default mirror set is the four runtime images the chart can pull: api, frontend, fi-sandbox, and job-runner. The cli image is published with the same release tag for build provenance, but the runtime chart does not pull it directly.

Source references are ghcr.io/definite-app/<image>:<tag>. The <tag> is the bare release version such as 0.0.4 (the Git tag is v0.0.4), or a git-sha-<short> tag. Use the exact tag Definite gave you for your release.

Step 1 — Mirror the images

scripts/mirror-images.sh pulls each image from ghcr.io/definite-app and pushes it into your registry under the same tag. It is idempotent: it compares source and destination manifest digests and skips anything already in sync, so re-running after a partial failure only moves what is missing.

Prerequisites

• docker running locally, or skopeo (set USE_SKOPEO=1 — skopeo needs no daemon and does not stage layers on local disk, which is nicer for CI runners and air-gap bastion hosts).
• You are logged in to your destination registry (see per-registry commands below).
• ghcr.io is public; no source login is needed unless your network proxies it behind auth, in which case run docker login ghcr.io first.

Run it

# Amazon ECR — also create the destination repos (ECR will not auto-create):
aws ecr get-login-password --region us-east-1 \
  | docker login --username AWS --password-stdin \
      123456789012.dkr.ecr.us-east-1.amazonaws.com

./scripts/mirror-images.sh \
  --dest 123456789012.dkr.ecr.us-east-1.amazonaws.com/definite \
  --tag 0.0.4 \
  --create-ecr-repos

# Google Artifact Registry:
gcloud auth configure-docker us-central1-docker.pkg.dev

./scripts/mirror-images.sh \
  --dest us-central1-docker.pkg.dev/acme-prod/definite \
  --tag 0.0.4

# Azure Container Registry:
az acr login --name acmecorp

./scripts/mirror-images.sh \
  --dest acmecorp.azurecr.io/definite \
  --tag 0.0.4

# Harbor (or any registry) via skopeo — daemonless, good for CI:
docker login harbor.acme.internal     # or: skopeo login harbor.acme.internal
USE_SKOPEO=1 ./scripts/mirror-images.sh \
  --dest harbor.acme.internal/definite \
  --tag 0.0.4

Use --dry-run to print exactly what would be copied without copying anything, and --force to re-push even images already in sync. --help prints the full option reference.

--dest is the registry host plus the namespace Definite images should live under, with no trailing slash and no image name. The script appends /<image>:<tag> to it. So --dest .../definite --tag 0.0.4 produces .../definite/api:0.0.4, .../definite/frontend:0.0.4, and so on.

ECR note

ECR does not create repositories on push. Either pass --create-ecr-repos (the script then creates each missing repo with scanOnPush=true and IMMUTABLE tags via the aws CLI), or pre-create the repos yourself:

for img in api frontend fi-sandbox job-runner; do
  aws ecr create-repository --region us-east-1 \
    --repository-name "definite/${img}" \
    --image-scanning-configuration scanOnPush=true \
    --image-tag-mutability IMMUTABLE
done

GAR, ACR, and Harbor create repositories implicitly on first push, so --create-ecr-repos is ECR-only.

Step 2 — Create an image pull Secret

Unless your nodes have ambient access to the registry (for example, EKS nodes with an instance role that grants ECR pull, or GKE nodes with the Artifact Registry scope), Kubernetes needs a docker-registry Secret to authenticate the pull. Create it in the deployment namespace:

kubectl create namespace definite   # if it does not exist yet

# Amazon ECR — the password is a 12-hour token, so for long-lived clusters
# prefer node IAM (no Secret needed) or a controller that refreshes it.
kubectl create secret docker-registry registry-pull \
  --namespace definite \
  --docker-server=123456789012.dkr.ecr.us-east-1.amazonaws.com \
  --docker-username=AWS \
  --docker-password="$(aws ecr get-login-password --region us-east-1)"

# Google Artifact Registry with a service-account key file:
kubectl create secret docker-registry registry-pull \
  --namespace definite \
  --docker-server=us-central1-docker.pkg.dev \
  --docker-username=_json_key \
  --docker-password="$(cat gar-pull-sa.json)"

# Azure Container Registry with a token / service principal:
kubectl create secret docker-registry registry-pull \
  --namespace definite \
  --docker-server=acmecorp.azurecr.io \
  --docker-username=<sp-app-id> \
  --docker-password=<sp-password>

# Harbor / Nexus / Quay with a robot or service account:
kubectl create secret docker-registry registry-pull \
  --namespace definite \
  --docker-server=harbor.acme.internal \
  --docker-username='robot$definite' \
  --docker-password=<robot-token>

If your nodes already have pull access to the registry you can skip this step and leave pull_secrets empty in the next step.

Step 3 — Point Definite at your registry

Definite's config.yaml (consumed by definite init, or fed straight to Helm as values) has an image: block that overrides the registry, tag, pull policy, and pull Secrets for the api, frontend, and job-runner images. The examples/*.yaml files ship it commented out.

# config.yaml
image:
  # Registry + namespace you mirrored into — the SAME value you passed to
  # mirror-images.sh as --dest. The chart appends /<image>:<tag>.
  registry: 123456789012.dkr.ecr.us-east-1.amazonaws.com/definite
  tag: 0.0.4
  pull_policy: IfNotPresent
  # Name(s) of the docker-registry Secret(s) from Step 2. Omit / leave
  # empty if your nodes have ambient registry access.
  pull_secrets:
    - registry-pull

fi-sandbox is configured separately

The Fi sandbox pod is launched through an Agent Sandbox template, not the shared image: block, so its image is set under fi.sandbox.image as a full image reference (registry + name + tag). Point it at the mirrored fi-sandbox image. The sandbox pod still reuses image.pull_policy and image.pull_secrets from the block above, so the same Secret covers it.

# config.yaml (continued)
fi:
  sandbox:
    image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/definite/fi-sandbox:0.0.4

Keep fi.sandbox.image's tag in lockstep with image.tag. They are set in two places only because the sandbox launches via a different mechanism; they should always reference the same release.

mirror-images.sh prints both of these snippets, pre-filled with your --dest and --tag, when it finishes.

Using Helm directly

If you drive Helm yourself instead of the definite CLI, the same values in values.yaml shape are:

image:
  registry: 123456789012.dkr.ecr.us-east-1.amazonaws.com/definite
  tag: "0.0.4"
  pullPolicy: IfNotPresent
  pullSecrets:
    - registry-pull
fi:
  sandbox:
    image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/definite/fi-sandbox:0.0.4

Step 4 — Deploy and verify

Run definite init --config config.yaml (or helm upgrade --install) as normal. Confirm every pod is pulling from your registry, not ghcr.io:

kubectl get pods -n definite \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[*].image}{"\n"}{end}'

Every image should be prefixed with your private registry host. If any pod is stuck in ImagePullBackOff, kubectl describe pod <name> -n definite shows the pull error — usually a missing pull Secret (Step 2), a Secret in the wrong namespace, or a tag that was never mirrored.

Upgrades

For each new Definite release, re-run Step 1 with the new --tag, then bump image.tag (and fi.sandbox.image's tag) in config.yaml and redeploy. Because mirror-images.sh is idempotent, re-running it only transfers images that changed.

Toward a fully air-gapped install

Mirroring covers the runtime images. A deployment with no egress to the public internet also needs:

1. The definite CLI binary. The quick-start curl | sh installer reaches storage.googleapis.com. For an air-gapped host, obtain the release binary out of band and place it on PATH directly — it is a single static binary with no runtime dependencies.

2. The Helm chart. Either run from a checkout of this repository inside the network, or helm pull the chart into a local chart repository / OCI registry.

3. A reachable Kubernetes registry mirror. Steps 1–3 above already point every workload at your internal registry. Make sure pull_policy resolves without contacting ghcr.io — IfNotPresent is correct once the images are mirrored.

4. LLM connectivity. Fi and automations need an LLM endpoint. In an air-gapped environment this is typically AWS Bedrock / Azure OpenAI / Vertex over a private link, or the self-hosted LiteLLM proxy pointed at an in-network model. There is no Definite-hosted LLM fallback.

5. Published images. A mirrored install uses the published release images. You do not need to build images from source to deploy from a private registry.