Skip to main content
Version: 1.13.0-dev

Cloud Provider Deployment

Deploy DVARA on AWS (EKS or ECS), Google Cloud, Azure, or DigitalOcean with provider-specific configurations and best practices.

Most sections below target managed Kubernetes and build on the Kubernetes / Helm chart — read that first. The AWS (ECS) section is the exception: it covers a Fargate-based deployment that skips Kubernetes entirely.

From 1.8.2 the gateway has no database

Only DVARA Flightdeck connects to PostgreSQL, and only Flightdeck runs schema migrations. Each gateway:

  • enrols with Flightdeck on first start, using a shared enrolment secret, and gets a certificate for Flightdeck's internal channel on port 8443;
  • reads its configuration as a signed bundle from Flightdeck. A change saved in Flightdeck reaches a gateway in about 10 seconds at the default settings, with no restart;
  • writes audit, usage and cost records to a local spool on its own data volume and ships them to Flightdeck, which stores them (about 30 seconds at the defaults).

So every managed database below is for Flightdeck only. Give the gateway no SPRING_DATASOURCE_* settings: a gateway given SPRING_DATASOURCE_URL refuses to start. The chart refuses a helm install or helm upgrade whose llmGatewayServer.extraEnv sets any of SPRING_DATASOURCE_URL, SPRING_DATASOURCE_USERNAME, SPRING_DATASOURCE_PASSWORD, DVARA_DB_MIGRATE, SPRING_FLYWAY_ENABLED, DVARA_BOOTSTRAP_FILE or GATEWAY_BOOTSTRAP_FILE. DVARA_DB_POOL_SIZE and DVARA_DB_SOCKET_TIMEOUT_SECONDS are no longer read on the gateway.

The Helm chart wires the rest for you. It generates the enrolment secret on install (secrets.enrolmentSharedSecret) and keeps it across upgrades, serves Flightdeck's internal channel through a <flightdeck>-internal Service, and gives each gateway pod a data volume. Two cases need you to act:

  • With secrets.existingSecret, that Secret must carry the key enrolment-shared-secret (at least 32 characters, for example openssl rand -base64 32).
  • When you render with helm template or Argo CD, set secrets.enrolmentSharedSecret explicitly. Those tools cannot read the existing value back and would make a new one on every render.

The gateway's data volume is an emptyDir by default. It survives a container restart but not the pod's deletion, so undelivered records are lost only if a pod dies without its shutdown flush. For production, set llmGatewayServer.persistence.enabled: true: each pod then gets its own volume at /var/lib/dvara, and the gateway runs as a StatefulSet. Also set llmGatewayServer.rollingUpdate.maxUnavailable: 1: the default 0 is refused for a StatefulSet on clusters that enable the MaxUnavailableStatefulSet feature gate.

AWS (EKS)​

Prerequisites​

  • eksctl or an existing EKS cluster (Kubernetes 1.28+)
  • AWS CLI configured with appropriate permissions
  • Helm 3.8+ installed

Create an EKS Cluster​

eksctl create cluster \
  --name dvara \
  --region us-east-1 \
  --nodegroup-name standard \
  --node-type m6i.large \
  --nodes 3 \
  --nodes-min 2 \
  --nodes-max 5 \
  --managed

Install the AWS Load Balancer Controller​

Required for ALB-based Ingress:

# Install the controller (see AWS docs for full IAM setup)
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=dvara \
  --set serviceAccount.create=true

Store Secrets in AWS Secrets Manager​

# Create the secret
aws secretsmanager create-secret \
  --name dvara/provider-credentials \
  --secret-string '{
    "openai-api-key": "sk-...",
    "anthropic-api-key": "sk-ant-...",
    "enterprise-license-key": "DVARA-...",
    "enrolment-shared-secret": "<openssl rand -base64 32>",
    "audit-hmac-secret": "<openssl rand -base64 32>",
    "llm-gateway-server-api-key": "<openssl rand -base64 32>",
    "gateway-metrics-api-key": "<a different openssl rand -base64 32>",
    "gateway-encryption-master-password": "<openssl rand -base64 32>"
  }'

With secrets.existingSecret, the chart creates none of the platform keys, so this Secret must carry them too. (The chart still makes the gateways' cluster certificate, <fullname>-server-cluster-tls, where <fullname> is the release name when it contains dvara, and <release>-dvara otherwise. With helm template or Argo CD, set llmGatewayServer.cluster.tls.secretName to a Secret you keep, or it changes on every render.)

  • enrolment-shared-secret (at least 32 characters). Both workloads read it with no optional flag, so without it the pods fail with CreateContainerConfigError.
  • audit-hmac-secret (at least 32 characters). On the chart's default production profile both pods refuse to start without it.
  • llm-gateway-server-api-key and gateway-metrics-api-key, two different values, for the actuator endpoints.
  • gateway-encryption-master-password, for ENCRYPTED-mode provider credentials.

See the table of platform keys in the GKE section.

Use the External Secrets Operator to sync into Kubernetes:

# external-secret.yaml
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: dvara-provider-keys
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: dvara-external
  # Copies every property of the JSON secret into a key of the same name.
  dataFrom:
    - extract:
        key: dvara/provider-credentials
kubectl apply -f external-secret.yaml

Deploy with ALB Ingress​

# aws-values.yaml
llmGatewayServer:
  replicaCount: 3
  javaOpts: "-Xms512m -Xmx768m"

  autoscaling:
    enabled: true
    minReplicas: 3
    maxReplicas: 10
    targetCPUUtilizationPercentage: 70

  pdb:
    enabled: true
    minAvailable: 2

  serviceMonitor:
    enabled: true
    additionalLabels:
      release: kube-prometheus-stack

  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app.kubernetes.io/component: gateway-server

flightdeck:
  enabled: true

secrets:
  create: false
  existingSecret: dvara-external

ingress:
  enabled: true
  className: alb
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789012:certificate/abc-123
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS13-1-2-2021-06
    alb.ingress.kubernetes.io/healthcheck-path: /actuator/health/readiness
  llmGatewayServer:
    hosts:
      - host: gateway.mycompany.com
        paths:
          - path: /
            pathType: Prefix
  flightdeck:
    hosts:
      - host: admin.mycompany.com
        paths:
          - path: /
            pathType: Prefix
helm install dvara oci://ghcr.io/dvarahq/charts/dvara --version 1.12.0 \
  -f aws-values.yaml -f db-values.yaml

db-values.yaml holds Flightdeck's database settings; see Database for Flightdeck below.

AWS Bedrock with explicit credentials​

DVARA does not use IRSA or the AWS SDK default credential chain for Bedrock. Give the DVARA LLM Gateway an explicit access-key and secret-key pair whose IAM principal can invoke the Bedrock models on your routes.

# bedrock-values.yaml
secrets:
  auditHmacSecret: "<openssl rand -base64 32>"
  bedrockEnabled: "true"
  providerKeys:
    awsAccessKeyId: <aws-access-key-id>
    awsSecretAccessKey: <aws-secret-access-key>
helm install dvara oci://ghcr.io/dvarahq/charts/dvara --version 1.12.0 \
  -f bedrock-values.yaml -f db-values.yaml

The chart's default profile is production, so it refuses to render when secrets.auditHmacSecret is empty.

Bedrock calls go to us-east-1 unless AWS_REGION is set on the gateway. The chart has no value for it, so for another region add it to llmGatewayServer.extraEnv:

llmGatewayServer:
  extraEnv:
    - name: AWS_REGION
      value: eu-west-1

For production, put these values in an externally managed Kubernetes Secret and set secrets.create: false with secrets.existingSecret. The Secret keys are aws-access-key-id, aws-secret-access-key, and bedrock-enabled.

Database for Flightdeck​

DVARA Flightdeck requires PostgreSQL. From 1.8.2 the gateway has no database, so only Flightdeck gets the connection settings. Rate limiting and API-key caching run in the gateway process — no external cache infrastructure is required.

RDS PostgreSQL:

aws rds create-db-instance \
  --db-instance-identifier dvara-db \
  --db-instance-class db.r6g.large \
  --engine postgres --engine-version 14 \
  --master-username dvara --master-user-password "${DB_PASSWORD}" \
  --allocated-storage 100 --storage-encrypted \
  --vpc-security-group-ids sg-... --db-subnet-group-name dvara-subnet

Helm values for the database (Flightdeck only):

# db-values.yaml
flightdeck:
  extraEnv:
    - name: SPRING_DATASOURCE_URL
      value: "jdbc:postgresql://dvara-db.xxx.us-east-1.rds.amazonaws.com:5432/dvara"
    - name: SPRING_DATASOURCE_USERNAME
      value: "dvara"
    - name: SPRING_DATASOURCE_PASSWORD
      valueFrom:
        secretKeyRef:
          name: dvara-db-credentials
          key: password

Multi-Region on AWS​

Run one release per region. Each release has its own gateways and its own Flightdeck, with the same region.id. Flightdeck publishes configuration only for its own region, and a gateway asks for its own region's. A gateway pointed at another region's Flightdeck never gets a bundle and never becomes ready.

All the Flightdecks share one PostgreSQL database, so every region sees the same workspaces, keys, routes and policies. Only Flightdeck crosses regions to reach it; gateway requests do not. Use the same auditHmacSecret and gatewayEncryptionMasterPassword in every region. Each release generates its own enrolment secret, which never leaves its region.

Primary region (us-east-1):

# primary-values.yaml
llmGatewayServer:
  replicaCount: 3
  region:
    id: us-east-1
    name: US East
  autoscaling:
    enabled: true
    minReplicas: 3
    maxReplicas: 10

flightdeck:
  enabled: true

secrets:
  providerKeys:
    openai: sk-...
  # The same two values in every region.
  auditHmacSecret: "<openssl rand -base64 32>"
  gatewayEncryptionMasterPassword: "<openssl rand -base64 32>"
  enterpriseLicenseKey: "DVARA-..."

Secondary region (eu-west-1), with its own Flightdeck on the same database:

# secondary-eu-values.yaml
llmGatewayServer:
  replicaCount: 2
  region:
    id: eu-west-1
    name: EU West
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10

flightdeck:
  enabled: true

secrets:
  providerKeys:
    openai: sk-...
  auditHmacSecret: "<the same value as the primary region>"
  gatewayEncryptionMasterPassword: "<the same value as the primary region>"

Both releases take the same db-values.yaml, pointing at the one database.

# Deploy primary in us-east-1
kubectl config use-context us-east-1
helm install dvara oci://ghcr.io/dvarahq/charts/dvara --version 1.12.0 \
  -f primary-values.yaml -f db-values.yaml

# Deploy secondary in eu-west-1
kubectl config use-context eu-west-1
helm install dvara-eu oci://ghcr.io/dvarahq/charts/dvara --version 1.12.0 \
  -f secondary-eu-values.yaml -f db-values.yaml

A runnable version is in kubernetes/multi-region/ in dvara-examples.

AWS (ECS)​

AWS ECS on Fargate is a lower-operational-overhead alternative to EKS. If you don't want to run a Kubernetes control plane, ECS runs the same DVARA container images (ghcr.io/dvarahq/dvara-*) as serverless tasks behind an Application Load Balancer.

From 1.8.2 only DVARA Flightdeck connects to RDS PostgreSQL. The gateway has no database: it enrols with Flightdeck, reads its configuration as a signed bundle from Flightdeck's internal port (8443), and keeps a local spool of audit, usage and cost records that it ships to Flightdeck. That spool is state. A gateway task flushes it when ECS stops the task, so give the task enough stop time (below).

Prerequisites​

  • AWS CLI configured with an account that can create ECS clusters, task definitions, Secrets Manager secrets, and ALBs
  • An existing VPC with at least two private subnets (for Fargate tasks) and two public subnets (for the ALB)
  • A running RDS PostgreSQL instance reachable from the VPC's private subnets (see Database Requirements)
  • At least one provider API key

Store secrets in AWS Secrets Manager​

Reference the secrets directly at task-definition time; ECS injects them as environment variables.

aws secretsmanager create-secret \
  --name dvara/license \
  --secret-string "DVARA-...your-license-key"

aws secretsmanager create-secret \
  --name dvara/openai \
  --secret-string "sk-..."

aws secretsmanager create-secret \
  --name dvara/db \
  --secret-string '{"username":"dvara","password":"..."}'

# The secret every gateway enrols with Flightdeck by. At least 32 characters.
aws secretsmanager create-secret \
  --name dvara/enrolment \
  --secret-string "$(openssl rand -base64 32)"

# Signs the audit chain. The gateway signs and Flightdeck verifies, so both get it.
aws secretsmanager create-secret \
  --name dvara/audit-hmac \
  --secret-string "$(openssl rand -base64 32)"

# Encrypts ENCRYPTED-mode provider credentials. Both tasks get it.
aws secretsmanager create-secret \
  --name dvara/encryption \
  --secret-string "$(openssl rand -base64 32)"

# Bearer keys for the actuator endpoints: one for /actuator/gateway-status and the
# other authenticated paths, and a different one for /actuator/prometheus. Both tasks get both.
aws secretsmanager create-secret \
  --name dvara/actuator \
  --secret-string "$(openssl rand -base64 32)"

aws secretsmanager create-secret \
  --name dvara/actuator-metrics \
  --secret-string "$(openssl rand -base64 32)"

dvara/db goes to the Flightdeck task only. dvara/enrolment, dvara/audit-hmac, dvara/encryption, dvara/actuator and dvara/actuator-metrics go to both tasks.

Task execution role​

The ECS task execution role needs permission to read the secrets you just created and write to CloudWatch Logs. The DVARA images on GHCR are public, so no registry credential is needed:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "kms:Decrypt"
      ],
      "Resource": [
        "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

Task definition​

Register one task definition with both the gateway and Flightdeck (dvara-flightdeck) as containers in the same task (simple), or split them across two task definitions to scale independently (recommended for production). This example shows the split:

{
  "family": "dvara-gateway",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024",
  "memory": "2048",
  "executionRoleArn": "arn:aws:iam::<account>:role/dvara-ecs-task-exec",
  "taskRoleArn": "arn:aws:iam::<account>:role/dvara-ecs-task",
  "containerDefinitions": [
    {
      "name": "gateway-server",
      "image": "ghcr.io/dvarahq/dvara-gateway:1.12.0",
      "essential": true,
      "stopTimeout": 65,
      "portMappings": [
        { "containerPort": 8080, "protocol": "tcp" }
      ],
      "secrets": [
        { "name": "OPENAI_API_KEY", "valueFrom": "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/openai" },
        { "name": "DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_SHARED_SECRET", "valueFrom": "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/enrolment" },
        { "name": "DVARA_AUDIT_HMAC_SECRET", "valueFrom": "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/audit-hmac" },
        { "name": "DVARA_ENCRYPTION_MASTER_PASSWORD", "valueFrom": "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/encryption" },
        { "name": "DVARA_ACTUATOR_API_KEY", "valueFrom": "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/actuator" },
        { "name": "DVARA_ACTUATOR_METRICS_API_KEY", "valueFrom": "arn:aws:secretsmanager:us-east-1:<account>:secret:dvara/actuator-metrics" }
      ],
      "environment": [
        { "name": "DVARA_DATA_PLANE_CONTROL_PLANE_BASE_URL", "value": "https://flightdeck.dvara.local:8443" },
        { "name": "DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_DIRECTORY", "value": "/var/lib/dvara/enrolment" },
        { "name": "DVARA_DATA_PLANE_CONTROL_PLANE_SNAPSHOT_PATH", "value": "/var/lib/dvara/bundle-snapshot.json" },
        { "name": "DVARA_LLM_GATEWAY_LOCAL_STORE_PATH", "value": "/var/lib/dvara/spool.db" }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/dvara/gateway-server",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "healthCheck": {
        "command": ["CMD", "curl", "-sf", "http://localhost:8080/actuator/health"],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 60
      }
    }
  ]
}

Register it and create the service:

aws ecs register-task-definition --cli-input-json file://dvara-gateway-taskdef.json

aws ecs create-service \
  --cluster dvara \
  --service-name dvara-gateway \
  --task-definition dvara-gateway \
  --desired-count 3 \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet-aaa,subnet-bbb],securityGroups=[sg-dvara],assignPublicIp=DISABLED}" \
  --load-balancers "targetGroupArn=arn:aws:elasticloadbalancing:...:targetgroup/dvara-gateway/xxx,containerName=gateway-server,containerPort=8080"

The gateway task has no database settings. From 1.8.2 a gateway given SPRING_DATASOURCE_URL refuses to start. It also refuses to start without DVARA_DATA_PLANE_CONTROL_PLANE_BASE_URL (Flightdeck) or DVARA_LLM_GATEWAY_LOCAL_STORE_PATH (the spool).

The spool, the bundle snapshot and the enrolment files live under /var/lib/dvara, on the task's own storage. stopTimeout: 65 gives the gateway time to finish in-flight calls and then flush its spool (up to 20 seconds) when ECS stops the task. A task that is killed without that flush loses the records it had not yet shipped. Do not point several tasks at one shared directory (for example one EFS path): each gateway needs its own spool. A task that starts on empty storage enrols again with the shared secret.

Create a similar task definition and service for dvara-flightdeck (port 8090, image ghcr.io/dvarahq/dvara-flightdeck:1.12.0). Flightdeck is the only task that gets the database settings. It needs:

  • SPRING_DATASOURCE_URL, and SPRING_DATASOURCE_USERNAME and SPRING_DATASOURCE_PASSWORD from dvara/db;
  • DVARA_INTERNAL_ENROLMENT_ENABLED=true, DVARA_INTERNAL_ENROLMENT_SERVER_NAMES=flightdeck.dvara.local (the name gateways use to reach it), and DVARA_INTERNAL_ENROLMENT_SHARED_SECRET from dvara/enrolment;
  • DVARA_FLIGHTDECK_GATEWAY_URL set to the gateway's own URL: its host name on the ALB, or its internal address. Flightdeck calls the gateway's /actuator endpoints there;
  • DVARA_AUDIT_HMAC_SECRET from dvara/audit-hmac, the same value the gateway tasks get. A gateway without it signs with a built-in development value, and Flightdeck then cannot verify the chain;
  • DVARA_ENCRYPTION_MASTER_PASSWORD from dvara/encryption, the same value the gateway tasks get, so Flightdeck can store BYOK provider credentials;
  • DVARA_ACTUATOR_API_KEY from dvara/actuator and DVARA_ACTUATOR_METRICS_API_KEY from dvara/actuator-metrics, the same values the gateway tasks get. Flightdeck uses them to reach the gateway's status and metrics endpoints, for its connection status and Dashboard metrics. Without them, those calls return 401;
  • optionally DVARA_PII_TOKEN_ENCRYPTION_MASTER_PASSWORD, which Flightdeck seals each workspace's PII token key with. Only Flightdeck needs it. Without it, pii.action=TOKENIZE is unavailable and each workspace's pii.degraded-action applies;
  • port 8443 open to the gateway tasks' security group, and the service registered in Cloud Map as flightdeck.dvara.local.

Start Flightdeck first. A gateway enrols with it when it starts, and ECS restarts a gateway task that gave up waiting.

The Gateway task needs no licence secret. From 1.8.2 the licence is stored in Flightdeck and reaches every Gateway pod in the configuration bundle. Paste it on Flightdeck's License page, or give the Flightdeck task DVARA_LICENSE_KEY from dvara/license, which Flightdeck copies into its store once, when none is stored.

Application Load Balancer​

Create an ALB with two target groups — one for the gateway (port 8080) and one for Flightdeck (port 8090). Give each service its own host name, with one listener rule per host, or use two ALBs. Do not split one host by path: Flightdeck's own pages also live under /mcp/ and /a2a/, and its config API under /v1/admin/, so path rules for the gateway's /v1/*, /mcp/* and /a2a/* would send those Flightdeck paths to the gateway.

Health check paths on the ALB target groups:

  • dvara-gateway target group: GET /actuator/health (port 8080) — anonymous, no Bearer needed
  • dvara-flightdeck target group: GET /actuator/health/readiness (port 8090)

Clustering on Fargate​

DVARA Gateway instances share rate-limit counters, spend counters and API key lookups across the fleet through an embedded cluster. On Fargate, the tasks need to discover each other. Fargate doesn't support multicast, so use DNS discovery through AWS ECS service discovery (Cloud Map):

  1. Create a Cloud Map namespace (e.g., dvara.local) attached to your VPC.
  2. Register the dvara-gateway service with Cloud Map — this gives every task an A record at gateway-server.dvara.local.
  3. Set CACHE_SERVICE_DNS=gateway-server.dvara.local on every gateway task, so each task finds the others by DNS. A fixed list in DVARA_CLUSTER_MEMBERS (host:7800,host:7800) works too.
  4. Open TCP port 7800 between gateway tasks in the security group. The members talk, and send heartbeats, on that one port.
  5. Give each gateway task a cluster TLS certificate from one CA: DVARA_CLUSTER_TLS_SSL_BUNDLE plus the SPRING_SSL_BUNDLE_PEM_<NAME>_* settings shown in Who can join the cluster. On a production profile a gateway given other members refuses to start without it. On the default profile it joins without TLS and logs a warning.

See Which rate-limit properties are supported? for the rate-limit settings.

Auto scaling​

Use ECS service auto scaling with target tracking on ECSServiceAverageCPUUtilization or a custom CloudWatch metric (e.g., gateway_requests_total rate scraped from Prometheus via the AWS Distro for OpenTelemetry collector).

aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/dvara/dvara-gateway \
  --min-capacity 3 \
  --max-capacity 20

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/dvara/dvara-gateway \
  --policy-name cpu-tracking \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration file://cpu-tracking.json

Observability​

  • Logs — DVARA emits structured JSON by default. Point the awslogs log driver at CloudWatch Logs, then ship to your SIEM via subscription filters (Kinesis Firehose → S3 / OpenSearch / Splunk).
  • Metrics — use AWS Distro for OpenTelemetry (ADOT) as a sidecar container to scrape /actuator/prometheus on each task and forward to Amazon Managed Prometheus or CloudWatch. The scrape requires Authorization: Bearer $DVARA_ACTUATOR_METRICS_API_KEY — mount the metrics key as a file (for example via Secrets Manager) and configure the ADOT receiver's bearer_token_file. The metrics secret is intentionally distinct from DVARA_ACTUATOR_API_KEY so a leaked scrape token can't unlock the rich gateway status surface.
  • Tracing — set OTEL_EXPORTER_OTLP_ENDPOINT to the ADOT sidecar's traces URL (http://localhost:4318/v1/traces) to ship traces to AWS X-Ray. The gateway uses the value as written and adds no path, so include /v1/traces.

When to choose ECS over EKS​

ECS (Fargate)EKS
Cluster opsNone (serverless)Control plane + node groups
Helm / kubectlNoYes
Right forSmall-to-medium deployments, teams without Kubernetes expertiseLarge fleets, teams already running Kubernetes
DVARA featuresAllAll
Cost modelPay per taskPay per node + control plane

ECS is the simpler path if you're new to AWS or don't already have Kubernetes in production. EKS wins once you need advanced networking (Istio, Linkerd), custom autoscaling, or already have a fleet of charts to reuse.

Google Cloud (GKE)​

Run DVARA on GKE with Cloud SQL (managed PostgreSQL for Flightdeck, private IP), Workload Identity for secrets, and a GCE Ingress fronted by a Google-managed TLS certificate. This builds on the Kubernetes / Helm chart — read that first.

Scripted recipe

A complete, runnable version of everything below — gcloud provisioning + helm install + verification in one script — lives in kubernetes/gke/ in dvara-examples. This page is the explained walkthrough; the recipe is the copy-paste path.

Prerequisites​

  • gcloud CLI authenticated against a billing-enabled project
  • kubectl and Helm 3.8+
  • A DVARA licence envelope (DVARA-…) only when you need Enterprise Production posture; see Licensing

Create a VPC-native GKE cluster​

Workload Identity (--workload-pool) and VPC-native networking (--enable-ip-alias) are both required — the first lets pods authenticate to Google APIs without static keys; the second lets the cluster reach Cloud SQL over a private IP.

PROJECT_ID=my-project
gcloud container clusters create dvara \
  --region us-central1 \
  --num-nodes 1 \
  --release-channel regular \
  --enable-ip-alias \
  --workload-pool="${PROJECT_ID}.svc.id.goog"

gcloud container clusters get-credentials dvara --region us-central1
kubectl create namespace dvara

Cloud SQL (private IP)​

Cloud SQL is Flightdeck's database. From 1.8.2 the gateway has no database: it reads route and policy changes as a signed bundle from Flightdeck, about 10 seconds after you save them at the default settings, with no restart.

# One-time: private services access so Cloud SQL gets a private IP
gcloud compute addresses create google-managed-services-default \
  --global --purpose=VPC_PEERING --prefix-length=16 --network=default
gcloud services vpc-peerings connect \
  --service=servicenetworking.googleapis.com \
  --ranges=google-managed-services-default --network=default

# Private-IP-only Postgres 16 instance
# --edition=ENTERPRISE is required: new Cloud SQL instances default to
# ENTERPRISE_PLUS, which rejects db-custom-* tiers (it wants db-perf-optimized-*).
gcloud sql instances create dvara-pg \
  --database-version=POSTGRES_16 --tier=db-custom-1-3840 --edition=ENTERPRISE \
  --region=us-central1 \
  --network="projects/${PROJECT_ID}/global/networks/default" \
  --no-assign-ip
gcloud sql databases create dvara --instance=dvara-pg
gcloud sql users create dvara --instance=dvara-pg --password="${DB_PASSWORD}"

# Capture the private IP for the JDBC URL below
gcloud sql instances describe dvara-pg \
  --format='value(ipAddresses.filter("type=PRIVATE").extract("ipAddress").flatten())'
Why private IP, not the Cloud SQL Auth Proxy

The Auth Proxy normally runs as a per-pod sidecar, and the DVARA chart has no extraContainers knob to inject one. Private IP needs no sidecar — Flightdeck connects with a plain JDBC URL.

Workload Identity for secrets​

Make Google Secret Manager the source of truth and sync into Kubernetes with the External Secrets Operator, authenticated through Workload Identity — no secret ever rides a helm --set command line. The DVARA pods never call Secret Manager themselves. The External Secrets Operator reads it, here with the token of the chart's gateway service account, which the commands below bind to a Google service account.

DVARA needs six operational secrets (each openssl rand -base64 32 except the license):

Secret (k8s key)Purpose
enterprise-license-keyOptional signed DVARA-… envelope. Flightdeck copies it into its licence store once, when none is stored; Gateway pods do not read it. You can paste the licence on Flightdeck's License page instead.
gateway-encryption-master-passwordAES-256-GCM key for ENCRYPTED-mode provider credentials — loss is unrecoverable, escrow offline
llm-gateway-server-api-keyOperator bearer for /actuator/gateway-status and Enterprise /actuator/license
gateway-metrics-api-keyBearer for /actuator/prometheus — must differ from the above
audit-hmac-secretHMAC-SHA256 key signing the audit chain
enrolment-shared-secretThe secret gateways enrol with Flightdeck by (from 1.8.2). At least 32 characters. With secrets.existingSecret the chart does not generate it, so this Secret must carry it
# Push to Secret Manager
for s in license-key encryption-master-password actuator-api-key \
         actuator-metrics-api-key audit-hmac-secret enrolment-shared-secret; do
  gcloud secrets create "dvara-${s}" --replication-policy=automatic
done
# (add a version to each with `gcloud secrets versions add … --data-file=-`)

# GCP service account + Workload Identity binding to the chart's two KSAs
gcloud iam service-accounts create dvara-gke
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
  --member="serviceAccount:dvara-gke@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"
for ksa in dvara-gateway-server dvara-flightdeck; do
  gcloud iam service-accounts add-iam-policy-binding \
    "dvara-gke@${PROJECT_ID}.iam.gserviceaccount.com" \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:${PROJECT_ID}.svc.id.goog[dvara/${ksa}]"
done
# cluster-secret-store.yaml — how the External Secrets Operator reaches Secret Manager
apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata: { name: gcp-secret-manager }
spec:
  provider:
    gcpsm:
      projectID: my-project
      auth:
        workloadIdentity:
          clusterLocation: us-central1
          clusterName: dvara
          serviceAccountRef: { name: dvara-gateway-server, namespace: dvara }

The service account dvara-gateway-server exists once the chart is installed. Until then the ExternalSecret reports an error and retries, and the pods wait for the Secret. To avoid that, bind the Google service account to the External Secrets Operator's own service account instead, and leave out auth.

# external-secret.yaml — syncs Secret Manager → the k8s Secret the chart consumes
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: dvara-platform
  namespace: dvara
spec:
  refreshInterval: 1h
  secretStoreRef: { name: gcp-secret-manager, kind: ClusterSecretStore }
  target: { name: dvara-platform }
  data:
    - { secretKey: enterprise-license-key,             remoteRef: { key: dvara-license-key } }
    - { secretKey: gateway-encryption-master-password, remoteRef: { key: dvara-encryption-master-password } }
    - { secretKey: llm-gateway-server-api-key,         remoteRef: { key: dvara-actuator-api-key } }
    - { secretKey: gateway-metrics-api-key,            remoteRef: { key: dvara-actuator-metrics-api-key } }
    - { secretKey: audit-hmac-secret,                  remoteRef: { key: dvara-audit-hmac-secret } }
    - { secretKey: enrolment-shared-secret,            remoteRef: { key: dvara-enrolment-shared-secret } }
Provider keys aren't here

DVARA uses the BYOK model — workspace provider keys live AES-encrypted in Flightdeck's database and are added via the Flightdeck after install, not as env vars. The chart's provider-key references are all optional: true, so the synced Secret only needs the six platform keys above.

Deploy with GCE Ingress + managed TLS​

# gke-values.yaml
llmGatewayServer:
  replicaCount: 2
  javaOpts: "-Xms256m -Xmx768m"
  serviceAccount:
    create: true
    name: dvara-gateway-server          # pinned → matches the WI binding above
    annotations:
      iam.gke.io/gcp-service-account: dvara-gke@my-project.iam.gserviceaccount.com
  # No database settings: from 1.8.2 the gateway has none. Give each pod its
  # own volume for the spool, the bundle snapshot and the enrolment files.
  persistence: { enabled: true }
  # With persistence the gateway is a StatefulSet; its update needs maxUnavailable of 1 or more.
  rollingUpdate: { maxUnavailable: 1 }
  pdb: { enabled: true, minAvailable: 1 }

flightdeck:
  enabled: true
  serviceAccount:
    create: true
    name: dvara-flightdeck
    annotations:
      iam.gke.io/gcp-service-account: dvara-gke@my-project.iam.gserviceaccount.com
  extraEnv:
    - { name: SPRING_DATASOURCE_URL, value: "jdbc:postgresql://CLOUDSQL_PRIVATE_IP:5432/dvara?sslmode=require" }
    - { name: SPRING_DATASOURCE_USERNAME, value: "dvara" }
    - name: SPRING_DATASOURCE_PASSWORD
      valueFrom: { secretKeyRef: { name: dvara-db, key: password } }

secrets:
  create: false
  existingSecret: dvara-platform        # synced by External Secrets above

# GCE Ingress: explicit hosts + empty tls so NO spec.tls renders — Google
# managed certs are driven by the annotation + the ManagedCertificate CRD.
ingress:
  enabled: true
  className: gce
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "dvara-ip"
    networking.gke.io/managed-certificates: "dvara-cert"
    kubernetes.io/ingress.allow-http: "false"
  llmGatewayServer:
    hosts: [{ host: api.mycompany.com, paths: [{ path: /, pathType: Prefix }] }]
    tls: []
  flightdeck:
    hosts: [{ host: flightdeck.mycompany.com, paths: [{ path: /, pathType: Prefix }] }]
    tls: []
# managed-certificate.yaml
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata: { name: dvara-cert, namespace: dvara }
spec:
  domains: [api.mycompany.com, flightdeck.mycompany.com]
# Reserve the static IP the Ingress annotation references
gcloud compute addresses create dvara-ip --global

kubectl -n dvara create secret generic dvara-db --from-literal=password="${DB_PASSWORD}"
kubectl -n dvara apply -f external-secret.yaml
kubectl -n dvara apply -f managed-certificate.yaml

helm install dvara oci://ghcr.io/dvarahq/charts/dvara \
  --version 1.12.0 \
  --namespace dvara \
  --values gke-values.yaml \
  --wait
Pin the image tag — never :latest

The chart version above is pinned to 1.12.0, a published release on GHCR. The chart runs the images of its own version (1.12.0) unless you set llmGatewayServer.image.tag and flightdeck.image.tag. Pin a real, published tag so the deploy is reproducible and rollbacks are clean — :latest is a moving pointer and can change under you. When a newer release is published, bump these to that version. Don't pin a tag that hasn't shipped yet, or pods fail with ImagePullBackOff.

After DNS A-records for both hosts point at the static IP, the Google-managed certificate provisions in 15–60 minutes (kubectl -n dvara describe managedcertificate dvara-cert → status Active). The Ingress serves HTTP 503 until then.

Verify (including config hot-reload)​

kubectl -n dvara get pods                                    # all Ready
kubectl -n dvara port-forward svc/dvara-flightdeck 8090:8090 &
open http://localhost:8090/                                  # walk /setup → workspace + API key

The chart-specific acceptance check is config hot-reload: create or edit a route in the Flightdeck, fire a request through the gateway, and confirm the new routing takes effect without restarting any pod. Allow about 10 seconds: Flightdeck republishes the signed bundle and each gateway fetches it. That proves the gateway enrolled with Flightdeck and is receiving its configuration.

Gemini (optional)​

Pass a Gemini API key as a BYOK credential in the Flightdeck. The Gemini provider calls the Gemini API with an API key on every call. It has no Vertex AI or Workload Identity path, so a service account role does not give it access.

Azure (AKS)​

Prerequisites​

  • Azure CLI configured
  • An existing AKS cluster or permissions to create one
  • Helm 3.8+ installed

Create an AKS Cluster​

az group create --name dvara-rg --location eastus

az aks create \
  --resource-group dvara-rg \
  --name dvara \
  --node-count 3 \
  --node-vm-size Standard_D4s_v3 \
  --enable-cluster-autoscaler \
  --min-count 2 \
  --max-count 6 \
  --enable-managed-identity \
  --enable-workload-identity \
  --enable-oidc-issuer \
  --enable-addons azure-keyvault-secrets-provider

az aks get-credentials --resource-group dvara-rg --name dvara

Store Secrets in Azure Key Vault​

# Create key vault
az keyvault create --name dvara-vault --resource-group dvara-rg --location eastus

# Store secrets
az keyvault secret set --vault-name dvara-vault --name openai-api-key --value "sk-..."
az keyvault secret set --vault-name dvara-vault --name anthropic-api-key --value "sk-ant-..."
az keyvault secret set --vault-name dvara-vault --name enterprise-license-key --value "DVARA-..."
# Platform keys: with secrets.existingSecret the chart generates none of them
az keyvault secret set --vault-name dvara-vault --name enrolment-shared-secret --value "$(openssl rand -base64 32)"
az keyvault secret set --vault-name dvara-vault --name audit-hmac-secret --value "$(openssl rand -base64 32)"
az keyvault secret set --vault-name dvara-vault --name llm-gateway-server-api-key --value "$(openssl rand -base64 32)"
az keyvault secret set --vault-name dvara-vault --name gateway-metrics-api-key --value "$(openssl rand -base64 32)"
az keyvault secret set --vault-name dvara-vault --name gateway-encryption-master-password --value "$(openssl rand -base64 32)"

The pods need the platform keys as well as the provider keys. Without enrolment-shared-secret they fail with CreateContainerConfigError, and on the chart's default production profile they refuse to start without audit-hmac-secret.

Use the Azure Key Vault Provider for Secrets Store CSI Driver. The --enable-addons azure-keyvault-secrets-provider flag above installs it. (On an existing cluster, run az aks enable-addons --addons azure-keyvault-secrets-provider -g dvara-rg -n dvara.) Let the add-on's identity read the vault, and note its client ID for the SecretProviderClass:

CLIENT_ID=$(az aks show -g dvara-rg -n dvara \
  --query addonProfiles.azureKeyvaultSecretsProvider.identity.clientId -o tsv)
az keyvault set-policy --name dvara-vault --spn "$CLIENT_ID" --secret-permissions get

On a vault that uses Azure RBAC instead of access policies, give that identity the Key Vault Secrets User role on the vault.

# secret-provider-class.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: dvara-keyvault
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "<CLIENT_ID from above>"
    keyvaultName: dvara-vault
    tenantId: "<azure-tenant-id>"
    objects: |
      array:
        - |
          objectName: openai-api-key
          objectType: secret
        - |
          objectName: anthropic-api-key
          objectType: secret
        - |
          objectName: enterprise-license-key
          objectType: secret
        - |
          objectName: enrolment-shared-secret
          objectType: secret
        - |
          objectName: audit-hmac-secret
          objectType: secret
        - |
          objectName: llm-gateway-server-api-key
          objectType: secret
        - |
          objectName: gateway-metrics-api-key
          objectType: secret
        - |
          objectName: gateway-encryption-master-password
          objectType: secret
  secretObjects:
    - secretName: dvara-external
      type: Opaque
      data:
        - objectName: openai-api-key
          key: openai-api-key
        - objectName: anthropic-api-key
          key: anthropic-api-key
        - objectName: enterprise-license-key
          key: enterprise-license-key
        - objectName: enrolment-shared-secret
          key: enrolment-shared-secret
        - objectName: audit-hmac-secret
          key: audit-hmac-secret
        - objectName: llm-gateway-server-api-key
          key: llm-gateway-server-api-key
        - objectName: gateway-metrics-api-key
          key: gateway-metrics-api-key
        - objectName: gateway-encryption-master-password
          key: gateway-encryption-master-password
kubectl apply -f secret-provider-class.yaml

Apply the SecretProviderClass in the namespace you install the chart into.

Deploy with NGINX Ingress Controller​

note

ingress-nginx reached end of life in March 2026. For a new cluster, use Traefik (see the DigitalOcean section) or Application Gateway for Containers. The values below work with any Ingress class: change className.

# Install NGINX Ingress Controller
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz
# azure-values.yaml
llmGatewayServer:
  replicaCount: 3
  javaOpts: "-Xms512m -Xmx768m"

  autoscaling:
    enabled: true
    minReplicas: 3
    maxReplicas: 10

  pdb:
    enabled: true
    minAvailable: 2

  serviceMonitor:
    enabled: true

  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app.kubernetes.io/component: gateway-server

flightdeck:
  enabled: true

secrets:
  create: false
  existingSecret: dvara-external

ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
  llmGatewayServer:
    hosts:
      - host: gateway.mycompany.com
        paths:
          - path: /
            pathType: Prefix
    tls:
      - secretName: gateway-tls
        hosts:
          - gateway.mycompany.com
  flightdeck:
    hosts:
      - host: admin.mycompany.com
        paths:
          - path: /
            pathType: Prefix
    tls:
      - secretName: admin-tls
        hosts:
          - admin.mycompany.com

The CSI driver creates dvara-external only while a pod mounts the volume, and removes it when none does. So mount it in both workloads:

# csi-values.yaml
# Mount the Key Vault CSI volume in both pods. The driver creates the
# dvara-external Secret (secretObjects) only while a pod mounts the volume.
llmGatewayServer:
  extraVolumes:
    - name: secrets-store
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: dvara-keyvault
  extraVolumeMounts:
    - name: secrets-store
      mountPath: /mnt/secrets-store
      readOnly: true
flightdeck:
  extraVolumes:
    - name: secrets-store
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: dvara-keyvault
  extraVolumeMounts:
    - name: secrets-store
      mountPath: /mnt/secrets-store
      readOnly: true
helm install dvara oci://ghcr.io/dvarahq/charts/dvara --version 1.12.0 \
  -f azure-values.yaml -f csi-values.yaml -f db-values.yaml

Database for Flightdeck​

DVARA Flightdeck requires PostgreSQL. From 1.8.2 the gateway has no database. Rate limiting and API-key caching run in the gateway process — no external cache infrastructure is required.

Azure Database for PostgreSQL:

az postgres flexible-server create \
  --resource-group dvara-rg --name dvara-db \
  --version 14 --sku-name Standard_D2s_v3 --storage-size 128 \
  --admin-user dvara --admin-password "${DB_PASSWORD}" \
  --tier GeneralPurpose --high-availability ZoneRedundant

Helm values: use the same SPRING_DATASOURCE_* environment variables shown in the AWS section, under flightdeck.extraEnv only, substituting the Azure Database for PostgreSQL endpoint.

Azure OpenAI Service​

To use Azure OpenAI instead of the public OpenAI API, configure the Azure OpenAI provider:

# azure-openai-values.yaml
llmGatewayServer:
  extraEnv:
    - name: AZURE_OPENAI_API_KEY
      value: "<azure-openai-api-key>"
    - name: AZURE_OPENAI_BASE_URL
      value: "https://my-resource.openai.azure.com/openai"

DigitalOcean (DOKS — Managed Kubernetes)​

Run DVARA on DigitalOcean Kubernetes (DOKS) with DO Managed PostgreSQL for Flightdeck and Traefik v3 ingress with built-in Let's Encrypt. This builds on the Kubernetes / Helm chart. For the no-Kubernetes path, see DigitalOcean (App Platform) below.

Scripted recipe

A runnable version — doctl provisioning + Traefik + helm install + verification in one script — lives in kubernetes/doks/ in dvara-examples. This page is the explained walkthrough.

Prerequisites​

  • doctl authenticated (doctl auth init), plus kubectl and Helm 3.8+
  • A domain for Let's Encrypt. Add a DVARA licence envelope (DVARA-…) when you need Enterprise Production posture; Development posture boots without one and serves every feature, including MCP and A2A, within 3 workspaces and 100,000 governed calls a month.

Create a DOKS cluster + Managed PostgreSQL​

doctl kubernetes cluster create dvara \
  --region nyc1 \
  --node-pool "name=default;size=s-2vcpu-4gb;count=2" --wait
doctl kubernetes cluster kubeconfig save dvara
CLUSTER_ID=$(doctl kubernetes cluster get dvara --format ID --no-header)

doctl databases create dvara-pg --engine pg --version 16 \
  --region nyc1 --size db-s-1vcpu-2gb --num-nodes 1 --wait
DB_ID=$(doctl databases list --format ID,Name --no-header | awk '$2=="dvara-pg"{print $1}')

# Lock the DB to the cluster, then create the app database + user
doctl databases firewalls append "$DB_ID" --rule "k8s:${CLUSTER_ID}"
doctl databases db create   "$DB_ID" dvara
doctl databases user create "$DB_ID" dvara

# Private connection details (port 25060, sslmode=require)
doctl databases connection "$DB_ID" --private --format Host --no-header
doctl databases user get "$DB_ID" dvara --format Password --no-header
Only Flightdeck uses this database

From 1.8.2 the gateway has no database. Flightdeck publishes routes and policies to the gateways as a signed bundle, so only Flightdeck needs the connection details above.

Secrets​

DigitalOcean has no Workload-Identity / Secret-Manager equivalent, so the six platform secrets are a plain Kubernetes Secret (or point secrets.existingSecret at one synced by your own Vault / Doppler). Each is openssl rand -base64 32 except the license; the two actuator Bearers must differ.

kubectl create namespace dvara
kubectl -n dvara create secret generic dvara-db \
  --from-literal=password="$DB_PASSWORD"
kubectl -n dvara create secret generic dvara-platform \
  --from-literal=enterprise-license-key="$DVARA_LICENSE_KEY" \
  --from-literal=gateway-encryption-master-password="$(openssl rand -base64 32)" \
  --from-literal=llm-gateway-server-api-key="$(openssl rand -base64 32)" \
  --from-literal=gateway-metrics-api-key="$(openssl rand -base64 32)" \
  --from-literal=audit-hmac-secret="$(openssl rand -base64 32)" \
  --from-literal=enrolment-shared-secret="$(openssl rand -base64 32)"

enrolment-shared-secret is the secret gateways enrol with Flightdeck by (from 1.8.2). With secrets.existingSecret the chart does not generate it, so this Secret must carry it.

Provider keys aren't here — DVARA uses BYOK (workspace keys live AES-encrypted in Flightdeck's database, added via the Flightdeck after install). The chart's provider-key references are optional: true, so this Secret only needs the six platform keys.

Ingress — Traefik v3 (not ingress-nginx)​

ingress-nginx reached end of life in March 2026, so this path uses Traefik v3, which has a built-in ACME client (no separate cert-manager):

helm repo add traefik https://traefik.github.io/charts && helm repo update
helm upgrade --install traefik traefik/traefik \
  --namespace traefik --create-namespace \
  --set "certificatesResolvers.letsencrypt.acme.email=ops@example.com" \
  --set "certificatesResolvers.letsencrypt.acme.storage=/data/acme.json" \
  --set "certificatesResolvers.letsencrypt.acme.tlsChallenge=true" \
  --set "persistence.enabled=true" --wait

(Traefik Helm value paths are version-sensitive — confirm with helm show values traefik/traefik.)

Deploy​

# doks-values.yaml
llmGatewayServer:
  replicaCount: 2
  serviceAccount: { create: true }   # no cloud identity needed on DO
  # No database settings: from 1.8.2 the gateway has none. Give each pod its
  # own volume for the spool, the bundle snapshot and the enrolment files.
  persistence: { enabled: true }
  # With persistence the gateway is a StatefulSet; its update needs maxUnavailable of 1 or more.
  rollingUpdate: { maxUnavailable: 1 }
  pdb: { enabled: true, minAvailable: 1 }

flightdeck:
  enabled: true
  extraEnv:
    - { name: SPRING_DATASOURCE_URL, value: "jdbc:postgresql://DB_PRIVATE_HOST:25060/dvara?sslmode=require" }
    - { name: SPRING_DATASOURCE_USERNAME, value: "dvara" }
    - name: SPRING_DATASOURCE_PASSWORD
      valueFrom: { secretKeyRef: { name: dvara-db, key: password } }

secrets:
  create: false
  existingSecret: dvara-platform

# Traefik issues Let's Encrypt certs for the baseDomain-derived hosts
# (api.<domain> + flightdeck.<domain>).
ingress:
  enabled: true
  className: traefik
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls.certresolver: letsencrypt
  baseDomain: dvara.example.com
helm install dvara oci://ghcr.io/dvarahq/charts/dvara \
  --version 1.12.0 --namespace dvara \
  --values doks-values.yaml --wait
Pin the image tag — never :latest

The chart version is pinned to 1.12.0, a published release on GHCR, and the chart runs the images of its own version. Pin a real, published tag so deploys are reproducible and rollbacks clean. When a newer release is published, bump to that version — don't pin a tag that hasn't shipped, or pods fail with ImagePullBackOff.

Point api.dvara.example.com and flightdeck.dvara.example.com at the DO Load Balancer IP (kubectl -n traefik get svc traefik); Traefik issues the certs within a few minutes of DNS resolving.

Verify (including config hot-reload)​

kubectl -n dvara get pods                                    # all Ready
kubectl -n dvara port-forward svc/dvara-flightdeck 8090:8090 &
open http://localhost:8090/                                  # /setup → workspace + API key

The chart-specific acceptance check is config hot-reload: create or edit a route in the Flightdeck, fire a request through the gateway, and confirm the new routing takes effect without restarting any pod. Allow about 10 seconds: Flightdeck republishes the signed bundle and each gateway fetches it. That proves the gateway enrolled with Flightdeck and is receiving its configuration.

DigitalOcean (App Platform)​

Prerequisites​

  • doctl CLI configured
  • A DigitalOcean account

One-Click Deploy​

Use the CLI:

doctl apps create --spec .do/app.yaml

From 1.8.2 the gateway cannot run on its own: it needs a DVARA Flightdeck to enrol with and get its configuration from. Use the multi-service spec below. Set your provider API keys in the App Platform dashboard under Settings → Environment Variables.

Custom App Spec​

For a multi-service deployment with the DVARA Flightdeck:

# .do/app.yaml
spec:
  name: dvara
  region: nyc
  # The managed database the flight-deck service's ${dvara-db.*} variables refer to.
  databases:
    - name: dvara-db
      engine: PG
      cluster_name: dvara-db
      production: true
  services:
    - name: gateway-server
      image:
        registry_type: GHCR
        registry: dvarahq
        repository: dvara-gateway
        tag: 1.12.0
      http_port: 8080
      instance_count: 1
      # 2 GB: the JVM holds about 1 GiB at rest, and a 1 GB instance is killed on start.
      instance_size_slug: professional-s
      health_check:
        http_path: /actuator/health
        initial_delay_seconds: 30
        period_seconds: 15
      envs:
        # The image's java launcher reads JDK_JAVA_OPTIONS; JAVA_OPTS reaches no JVM.
        - key: JDK_JAVA_OPTIONS
          scope: RUN_TIME
          value: "-Xmx768m"
        - key: OPENAI_API_KEY
          scope: RUN_TIME
          type: SECRET
        - key: ANTHROPIC_API_KEY
          scope: RUN_TIME
          type: SECRET
        # The same value on both services, openssl rand -base64 32.
        - key: DVARA_AUDIT_HMAC_SECRET
          scope: RUN_TIME
          type: SECRET
        - key: DVARA_ENCRYPTION_MASTER_PASSWORD
          scope: RUN_TIME
          type: SECRET
        # Actuator bearer keys, two different values, the same on both services.
        - key: DVARA_ACTUATOR_API_KEY
          scope: RUN_TIME
          type: SECRET
        - key: DVARA_ACTUATOR_METRICS_API_KEY
          scope: RUN_TIME
          type: SECRET
        # No database: the gateway enrols with Flightdeck and reads its
        # configuration from it (from 1.8.2).
        - key: DVARA_DATA_PLANE_CONTROL_PLANE_BASE_URL
          scope: RUN_TIME
          value: "https://flight-deck:8443"
        - key: DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_SHARED_SECRET
          scope: RUN_TIME
          type: SECRET
        - key: DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_DIRECTORY
          scope: RUN_TIME
          value: /var/lib/dvara/enrolment
        - key: DVARA_DATA_PLANE_CONTROL_PLANE_SNAPSHOT_PATH
          scope: RUN_TIME
          value: /var/lib/dvara/bundle-snapshot.json
        - key: DVARA_LLM_GATEWAY_LOCAL_STORE_PATH
          scope: RUN_TIME
          value: /var/lib/dvara/spool.db

    - name: flight-deck
      image:
        registry_type: GHCR
        registry: dvarahq
        repository: dvara-flightdeck
        tag: 1.12.0
      http_port: 8090
      internal_ports: [8443]
      instance_count: 1
      instance_size_slug: basic-s
      health_check:
        http_path: /actuator/health/liveness
        initial_delay_seconds: 30
      envs:
        - key: DVARA_FLIGHTDECK_GATEWAY_URL
          scope: RUN_TIME
          value: "${gateway-server.PRIVATE_URL}"
        - key: DVARA_INTERNAL_ENROLMENT_ENABLED
          scope: RUN_TIME
          value: "true"
        - key: DVARA_INTERNAL_ENROLMENT_SERVER_NAMES
          scope: RUN_TIME
          value: flight-deck
        # The same value as the gateway's enrolment secret, at least 32 characters.
        - key: DVARA_INTERNAL_ENROLMENT_SHARED_SECRET
          scope: RUN_TIME
          type: SECRET
        # The same values as on gateway-server.
        - key: DVARA_AUDIT_HMAC_SECRET
          scope: RUN_TIME
          type: SECRET
        - key: DVARA_ENCRYPTION_MASTER_PASSWORD
          scope: RUN_TIME
          type: SECRET
        - key: DVARA_ACTUATOR_API_KEY
          scope: RUN_TIME
          type: SECRET
        - key: DVARA_ACTUATOR_METRICS_API_KEY
          scope: RUN_TIME
          type: SECRET

The images set no Spring profile, so both services run on the default profile. Without DVARA_AUDIT_HMAC_SECRET they sign the audit chain with a published development value, and without DVARA_ENCRYPTION_MASTER_PASSWORD Flightdeck refuses to store BYOK credentials.

The gateway's spool is not durable on App Platform

App Platform gives a service no volume that survives a redeploy. The gateway keeps its spool of undelivered audit, usage and cost records under /var/lib/dvara on the instance's own disk. A normal stop flushes the spool to Flightdeck first, but an instance that dies without that flush loses what it had not yet shipped. For production, run on DOKS with llmGatewayServer.persistence.enabled: true.

Database for Flightdeck​

DVARA Flightdeck requires PostgreSQL. From 1.8.2 the gateway has no database. Rate limiting and API-key caching run in the gateway process — no external cache infrastructure is required.

Managed PostgreSQL:

doctl databases create dvara-db \
  --engine pg --version 14 \
  --size db-s-2vcpu-4gb \
  --region nyc1 --num-nodes 2

Add the database connection details to the flight-deck service only, as environment variables in the App Platform dashboard or in the app spec. The ${dvara-db.*} variables resolve only when the spec has the databases: entry named dvara-db shown above; without it the literal text reaches Flightdeck and the JDBC URL is invalid:

envs:
  - key: SPRING_DATASOURCE_URL
    scope: RUN_TIME
    value: "jdbc:postgresql://${dvara-db.HOSTNAME}:${dvara-db.PORT}/${dvara-db.DATABASE}?sslmode=require"
  - key: SPRING_DATASOURCE_USERNAME
    scope: RUN_TIME
    value: "${dvara-db.USERNAME}"
  - key: SPRING_DATASOURCE_PASSWORD
    scope: RUN_TIME
    type: SECRET
    value: "${dvara-db.PASSWORD}"

Flightdeck takes a JDBC URL. It does not read DigitalOcean's DATABASE_URL, which is a postgresql://user:password@host URI, so build the JDBC URL from the parts and pass the user and password separately.

Scaling​

App Platform supports horizontal scaling via the dashboard or CLI:

# Scale to 3 instances
doctl apps update <app-id> --spec <updated-spec-with-instance_count-3>

For production, use professional-s (2 GB) or larger for the gateway.

Monitoring Stack​

All cloud providers can use the same monitoring setup. Install the kube-prometheus-stack for Prometheus + Grafana:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring --create-namespace \
  --set grafana.adminPassword=admin

Enable ServiceMonitor in DVARA:

llmGatewayServer:
  serviceMonitor:
    enabled: true
    interval: 15s
    additionalLabels:
      release: monitoring

# The MCP and A2A planes run inside the gateway, so the job above scrapes all
# three planes — there is no second workload to add a ServiceMonitor for.

Import the dashboards from grafana/dashboards/ in dvara-examples for pre-built panels covering request rates, latencies, token usage, provider health, cost metrics, and agent session stats.

Key Prometheus Metrics to Alert On​

# Example alerting rules
groups:
  - name: dvara
    rules:
      - alert: HighErrorRate
        expr: sum(rate(gateway_requests_total{status=~"5.."}[5m])) / sum(rate(gateway_requests_total[5m])) > 0.05
        for: 5m
        labels:
          severity: critical
      - alert: HighLatency
        expr: max(gateway_latency_seconds{quantile="0.95"}) > 5
        for: 5m
        labels:
          severity: warning
      - alert: BudgetCapBreached
        expr: increase(gateway_budget_blocked_total[1h]) > 0
        labels:
          severity: warning

status on gateway_requests_total is the HTTP status code, such as 200 or 503. Sum both sides of the error ratio: without sum, each 5xx series is divided by itself and the ratio is always 1. gateway_latency_seconds publishes the 0.5, 0.95 and 0.99 quantiles on each series (per workspace, model, provider, status and region). Quantiles cannot be averaged across series; max alerts when any one series is slow. It also publishes ten _bucket series, from 50 ms to 60 s, so you can compute a P95 across pods instead: histogram_quantile(0.95, sum by (le) (rate(gateway_latency_seconds_bucket[5m]))) > 5.

Enterprise Features on Cloud​

Vault Integration​

DVARA supports HashiCorp Vault, AWS Secrets Manager, and Azure Key Vault as credential backends for LLM provider keys. This is separate from Kubernetes secret management — it provides runtime credential resolution for the gateway itself.

Known issue in 1.12.0

The AWS Secrets Manager and Azure Key Vault backends cannot start in the published 1.12.0 gateway image. Their client libraries are not in the image, so a gateway with DVARA_VAULT_BACKEND set to aws-secrets-manager or azure-key-vault fails at startup. HashiCorp Vault works. As a workaround, use HashiCorp Vault, or sync the provider keys into a Kubernetes Secret (with the External Secrets Operator or the Secrets Store CSI driver) and pass them to the gateway as environment variables.

HashiCorp Vault (any cloud):

llmGatewayServer:
  extraEnv:
    - name: DVARA_VAULT_BACKEND
      value: hashicorp
    - name: VAULT_ADDR
      value: "https://vault.internal:8200"
    - name: VAULT_AUTH_METHOD
      value: approle
    - name: VAULT_ROLE_ID
      valueFrom:
        secretKeyRef:
          name: vault-credentials
          key: role-id
    - name: VAULT_SECRET_ID
      valueFrom:
        secretKeyRef:
          name: vault-credentials
          key: secret-id

AWS Secrets Manager:

llmGatewayServer:
  extraEnv:
    - name: DVARA_VAULT_BACKEND
      value: aws-secrets-manager
    - name: AWS_VAULT_REGION
      value: us-east-1
    - name: AWS_SECRET_NAME
      value: dvara/provider-credentials
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/dvara-secrets

Azure Key Vault:

llmGatewayServer:
  extraEnv:
    - name: DVARA_VAULT_BACKEND
      value: azure-key-vault
    - name: AZURE_VAULT_URL
      value: "https://dvara-vault.vault.azure.net"
    - name: AZURE_TENANT_ID
      value: "<azure-tenant-id>"
    - name: AZURE_CLIENT_ID
      value: "<service-principal-client-id>"
    - name: AZURE_CLIENT_SECRET
      valueFrom:
        secretKeyRef:
          name: azure-keyvault-credentials
          key: client-secret

The Key Vault backend signs in as a service principal, with its client ID, client secret and tenant ID. Managed identity and workload identity are not supported for it.

OIDC / SSO Setup​

Enable OIDC-based authentication for DVARA Flightdeck. These settings belong on Flightdeck, not the gateway:

flightdeck:
  extraEnv:
    - name: DVARA_FLIGHTDECK_SECURITY_ENABLED
      value: "true"
    - name: DVARA_FLIGHTDECK_SECURITY_OIDC_ISSUER_URI
      value: "https://login.microsoftonline.com/<azure-tenant-id>/v2.0"  # Azure AD (Entra tenant ID)
      # value: "https://accounts.google.com"                        # Google
      # value: "https://cognito-idp.us-east-1.amazonaws.com/<pool>" # AWS Cognito
    - name: DVARA_FLIGHTDECK_SECURITY_OIDC_AUDIENCE
      value: "dvara-gateway"
    - name: DVARA_FLIGHTDECK_SECURITY_OIDC_ROLE_CLAIM
      value: "roles"  # or "realm_access.roles" for Keycloak
    # Browser sign-in: Flightdeck shows the SSO button only when a client ID is set
    - name: DVARA_FLIGHTDECK_SECURITY_OIDC_CLIENT_ID
      value: "<oidc-client-id>"
    - name: DVARA_FLIGHTDECK_SECURITY_OIDC_CLIENT_SECRET
      valueFrom:
        secretKeyRef:
          name: dvara-oidc
          key: client-secret

Database Requirements​

PostgreSQL is required by DVARA Flightdeck. From 1.8.2 the gateway has no database: it reads its configuration as a signed bundle from Flightdeck and ships its audit, usage and cost records to Flightdeck, which stores them. A gateway given SPRING_DATASOURCE_URL refuses to start. Rate limiting and the distributed cache for API-key lookups both run in the gateway process — no external cache infrastructure is needed.

DependencyMinimum VersionPurpose
PostgreSQL14+Required by Flightdeck. Audit events, policies, cost records, token usage, workspaces, API keys, routes, etc.

Flightdeck runs the database migrations, and is the only process that does. They run automatically on first startup, so no manual schema setup is required. Subsequent upgrades apply incremental migrations without data loss.

Connection pool defaults: Flightdeck and bulk-load jobs default to ten connections. Increase pool sizes only after measuring database saturation and leave enough headroom for every Flightdeck replica. DVARA_DB_POOL_SIZE and DVARA_DB_SOCKET_TIMEOUT_SECONDS are no longer read on the gateway. The Helm chart refuses DVARA_DB_MIGRATE in llmGatewayServer.extraEnv.

Quick Reference: Cloud-Specific Values​

SettingAWSGCPAzureDigitalOcean
Ingress classalbgcenginxApp Platform (built-in)
TLS terminationACM certificate ARN annotationGKE ManagedCertificatecert-manager + Let's EncryptAutomatic (App Platform)
Pod identity for your secrets tooling (DVARA itself does not use it)IRSA (eks.amazonaws.com/role-arn)Workload Identity (iam.gke.io/gcp-service-account)Managed IdentityN/A
Secret managementSecrets Manager + External SecretsSecret Manager + External SecretsKey Vault + CSI DriverApp Platform env vars (encrypted)
Node type (recommended)m6i.large (2 vCPU, 8 GB)e2-standard-4 (4 vCPU, 16 GB)Standard_D4s_v3 (4 vCPU, 16 GB)professional-s (1 vCPU, 2 GB)
AutoscalerCluster Autoscaler or KarpenterGKE Autopilot or Cluster AutoscalerAKS Cluster AutoscalerManual instance count
Managed PostgreSQL (required, Flightdeck only)RDS for PostgreSQLCloud SQL for PostgreSQLAzure Database for PostgreSQLDigitalOcean Managed Databases

Production Checklist​

Before going to production on any cloud provider:

  • Enable HPA with appropriate min/max replicas
  • Enable PDB with minAvailable >= 2
  • Configure topology spread constraints for cross-zone scheduling
  • Use external secret management (not Helm --set for API keys)
  • Enable Ingress with TLS termination
  • Enable ServiceMonitor for Prometheus scraping
  • Set JVM options: -Xms512m -Xmx768m (adjust for workload)
  • Keep terminationGracePeriodSeconds at the default 65 and preStopSleepSeconds: 5. It must be at least preStop + shutdown phase + spool drain + 5 s (60 at the defaults); the chart refuses less
  • For a Deployment (persistence off), keep maxSurge: 1 and maxUnavailable: 0 for zero-downtime deploys. With llmGatewayServer.persistence.enabled: true the gateway is a StatefulSet, which has no maxSurge: set llmGatewayServer.rollingUpdate.maxUnavailable: 1
  • Provision PostgreSQL 14+ for Flightdeck (required) — the gateway has no database, and no other external cache infrastructure is needed
  • Give each gateway pod its own data volume for its spool (llmGatewayServer.persistence.enabled: true)
  • Keep the enrolment secret (secrets.enrolmentSharedSecret, or enrolment-shared-secret in your existing Secret) the same across upgrades
  • Add the licence on Flightdeck's License page for production rights and no workspace or call limits
  • Set up alerting on error rate, latency, and budget metrics
  • Configure OIDC/SSO for Flightdeck access
  • Enable audit HMAC signing with a production secret
  • Review and set budget caps for cost governance