Support

How to get help with a Definite On-Prem deployment, what to try first, and what to attach when you open a ticket. A first-version runbook — adjust the contacts and SLAs to match your support agreement.

Contacts

Always include your deployment identifier (deployment.name from config.yaml) and the CLI / chart version (definite version) in the first message.

Severity & response

Set real SLAs with your account contact; these are defaults.

For a suspected security issue, email hello@definite.app directly and do not post details in shared channels.

Triage before you escalate

Most issues are diagnosable from the cluster. Run these first and attach the output — it shortens every ticket.

1. Health snapshot

definite version
definite status --config config.yaml

status is kubectl get pods,svc,ingress for the deployment namespace. Look for pods not Running/Ready, restart counts, and CrashLoopBackOff.

2. Re-run preflight

definite doctor --config config.yaml

doctor re-checks Postgres, Kubernetes, object store, and the LLM provider. A failure here usually means an external dependency changed — see prerequisites.md and network-requirements.md — not a Definite bug.

3. Component logs

definite logs api --tail 500
definite logs job-runner --tail 500
definite logs frontend --tail 200

Add --follow to watch live. Components: api, frontend, job-runner. (Lakehouse queries run as embedded DuckDB inside the API and job-runner pods — there is no separate lakehouse pod or log stream.)

4. Common causes

Usage and telemetry

Settings → Usage reads the deployment's local Postgres records for the last 30 days. It shows successful query and data-app compute plus fresh-input, cache-read, cache-write, and output LLM tokens. The local AI-credit figure is a raw-token estimate for visibility, not an invoice: managed hosted billing weights those four provider-priced classes centrally. The daily telemetry CronJob sends the raw 24-hour classes to Definite when telemetry is enabled.

To verify that the daily job is completing:

kubectl get cronjob,job -n definite -l app.kubernetes.io/component=telemetry
kubectl get pods -n definite -l app.kubernetes.io/component=telemetry
kubectl logs -n definite -l app.kubernetes.io/component=telemetry --tail=200

The daily job uses only the application Postgres database. It does not attach DuckLake or read object-storage credentials; DuckLake data-scale metadata is collected separately by the already-running API process.

5. Restart safely

The deployment is stateless; restarting a component does not lose data.

kubectl rollout restart deploy/definite-api -n definite
kubectl rollout restart deploy/definite-job-runner -n definite

If a bad config or upgrade is the cause, re-render and re-apply:

definite upgrade --config config.yaml --dry-run   # inspect first
definite upgrade --config config.yaml

API-down recovery channel

The chart runs a small support-recovery pod separately from the API. If the API is crash-looping or no longer serving HTTP, Definite support can queue one short-lived, audited command: restart this release's API Deployment. The watchdog cannot run shell commands, read logs, or mutate any other workload; its namespace Role is restricted to get and patch on the exact API Deployment name.

The existing Settings → Support access toggle is authoritative. The watchdog reads it directly from the deployment's Postgres database before polling and again before executing. Turning support access off therefore shuts the channel even when the API itself is unavailable; a database read failure also fails closed. Operators that require a chart-level hard disable can set:

supportRecovery:
  enabled: false

External uptime checks should use /readyz. The ingress routes that path to the real API readiness handler rather than the frontend fallback.

What to attach to a ticket

Bundle these into the ticket so the first reply can be a fix, not a request for more data:

1. definite version — CLI and chart version.
2. definite status --config config.yaml output.
3. definite doctor --config config.yaml output.
4. Logs from the affected component(s) — definite logs <component> --tail 500. Redact secrets and any sensitive data.
5. What changed — recent definite upgrade, a config edit, a cluster or Postgres change, a credential rotation.
6. Severity and business impact.
7. config.yaml with all secrets removed (postgres.url password, object store keys, OIDC secret, LLM keys). The CLI substitutes ${VAR} env vars — never send their values.

Collecting logs without the CLI

If the definite CLI is unavailable, raw kubectl works:

kubectl get pods,svc,ingress,pvc -n definite
kubectl describe pod <pod> -n definite
kubectl logs <pod> -n definite --tail 500 --previous   # --previous for a crashed pod
kubectl get events -n definite --sort-by=.lastTimestamp

Security & privacy when sharing diagnostics

• Never send Postgres passwords, object-store keys, OIDC client secrets, or LLM API keys. Redact them from configs and logs before attaching.
• Logs can contain query text and table/column names. Redact anything sensitive under your data policy before sharing externally.
• For a confirmed or suspected breach, email hello@definite.app and treat it as SEV-1.

Escalation path

1. Triage — run the steps above; check prerequisites.md, network-requirements.md, and architecture.md.
2. Open a ticket — email hello@definite.app with the attachments above and a severity.
3. SEV-1 — flag the email subject [SEV-1] and, if you have one, post in your shared channel for fastest pickup.
4. Data-loss / restore — follow backup-restore.md; loop in support before destructive restore steps if the data is irreplaceable.
5. Feature requests / non-blocking bugs — file via your support contact; they are triaged into the next release window.

Related docs

• cli.md — full CLI reference.
• prerequisites.md — dependency checklist.
• network-requirements.md — data-source connectivity troubleshooting.
• backup-restore.md — backup and disaster recovery.
• architecture.md — component map.