Skip to content

Supabase JWT Token Management

This document explains how to properly manage JWT tokens in the Kubernetes Supabase deployment and troubleshoot common authentication issues.

Overview

Supabase uses JWT (JSON Web Tokens) for authentication. There are two critical token types:

  1. ANON_KEY - Public token for unauthenticated requests
  2. SERVICE_ROLE_KEY - Admin token with full database access (bypasses RLS)

IMPORTANT: As of November 2024, our Kong configuration uses environment variable injection from supabase-secrets, eliminating the need for manual synchronization. Tokens are now automatically synchronized when supabase-secrets is updated and services are restarted.

Token Storage and Injection

Single Source of Truth: Kubernetes Secret (supabase-secrets)

All Supabase services read tokens from the centralized secret:

Services using tokens: - GoTrue - Validates JWT signatures - PostgREST - Validates JWT for database access - Storage - Validates JWT for file uploads - Studio - Uses SERVICE_ROLE_KEY for admin operations - Kong - Authenticates incoming requests via environment variable injection

# View current tokens in secret
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.ANON_KEY}' | base64 -d
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.SERVICE_ROLE_KEY}' | base64 -d

Kong Init Container Pattern

IMPORTANT: Kong's declarative configuration does NOT natively support environment variable substitution. We use an init container to solve this (matching the official Supabase Docker approach with envsubst).

Location: /root/k8s/manifests/supabase/kong.yaml

Architecture: 

supabase-secrets → Init Container → Kong Config → Kong Pod
                   (sed substitution)  (runtime)

How it works:

  1. ConfigMap contains template with $VARIABLES: 

    # kong-config ConfigMap
consumers:
  - username: anon
    keyauth_credentials:
      - key: $SUPABASE_ANON_KEY  # Template variable
  - username: service_role
    keyauth_credentials:
      - key: $SUPABASE_SERVICE_KEY  # Template variable

  2. Init Container substitutes variables before Kong starts: 

    initContainers:
  - name: kong-config-init
    image: busybox:1.36
    command: ['sh', '-c']
    args:
      - |
        # Read template, substitute vars, write to runtime volume
        cat /kong-template/kong.yml | \
        sed "s|\$SUPABASE_ANON_KEY|$SUPABASE_ANON_KEY|g" | \
        sed "s|\$SUPABASE_SERVICE_KEY|$SUPABASE_SERVICE_KEY|g" \
        > /kong-config/kong.yml
    env:
      - name: SUPABASE_ANON_KEY
        valueFrom:
          secretKeyRef:
            name: supabase-secrets
            key: ANON_KEY
      - name: SUPABASE_SERVICE_KEY
        valueFrom:
          secretKeyRef:
            name: supabase-secrets
            key: SERVICE_ROLE_KEY
    volumeMounts:
      - name: kong-config-template
        mountPath: /kong-template
      - name: kong-config-runtime
        mountPath: /kong-config

  3. Kong container reads from the runtime volume: 

    containers:
  - name: kong
    env:
      - name: KONG_DECLARATIVE_CONFIG
        value: /usr/local/kong/declarative/kong.yml
    volumeMounts:
      - name: kong-config-runtime
        mountPath: /usr/local/kong/declarative

  4. Volumes: 

    volumes:
  - name: kong-config-template
    configMap:
      name: kong-config
  - name: kong-config-runtime
    emptyDir: {}  # Runtime storage for substituted config

Why this approach: - Kong doesn't support $VARIABLE substitution in declarative configs - Official Supabase Docker uses envsubst for the same reason - Init containers run before the main container, perfect for preprocessing - Template stays in git without secrets, actual tokens from Kubernetes secrets

How Token Validation Works

Kong acts as the API gateway with init container-based token synchronization:

Pod Start:
  1. Init container reads template ConfigMap
  2. Substitutes $VARIABLES with secrets
  3. Writes to emptyDir volume
  4. Kong starts and reads from emptyDir

Request Flow:
  Browser/App → Kong (validates with substituted token) → GoTrue/PostgREST/Storage
                ↑                                          ↑
            Reads from emptyDir                    Reads from supabase-secrets
            (substituted by init)                  (same source)

Benefits: - Single source of truth (supabase-secrets) - Automatic synchronization on pod restart (init container re-runs) - No manual ConfigMap updates required - Template stays in git without secrets - Matches official Supabase Docker architecture

Updating JWT Tokens

When to Update

  • Security rotation (recommended every 6-12 months)
  • JWT_SECRET changed
  • Tokens expired (check exp claim)
  • Initial setup

The easiest way to update tokens is using the management script which handles everything automatically:

# Generate new tokens and update all services automatically
/root/k8s/scripts/manage-supabase-jwt.sh generate

What this script does: 1. Backs up current secrets 2. Generates new ANON_KEY and SERVICE_ROLE_KEY using current JWT_SECRET 3. Updates supabase-secrets 4. Kong automatically picks up new tokens via env var injection 5. Restarts all affected services (gotrue, rest, storage, kong) 6. Validates new tokens

If you need to manually update tokens:

Generating New JWT Tokens

When to Generate New Tokens

  • Initial deployment setup
  • Token rotation for security
  • Tokens expired (check exp claim in JWT)
  • JWT secret changed

Using jwt.io or JWT CLI

Using jwt.io website:

  1. Go to https://jwt.io
  2. Select algorithm: HS256
  3. Set payload: 
    {
  "iss": "supabase",
  "ref": "localhost",
  "role": "anon",
  "iat": 1762817111,
  "exp": 2078177111
}
  4. Set secret: your-super-secret-jwt-token-with-at-least-32-characters-long
  5. Copy encoded token

Using Docker:

# Generate ANON_KEY
docker run --rm supabase/gotrue:latest \
  jwt generate anon \
  --secret "your-super-secret-jwt-token-with-at-least-32-characters-long" \
  --expiry 10y

# Generate SERVICE_ROLE_KEY
docker run --rm supabase/gotrue:latest \
  jwt generate service_role \
  --secret "your-super-secret-jwt-token-with-at-least-32-characters-long" \
  --expiry 10y

CRITICAL: The --secret must match the JWT_SECRET environment variable in your Supabase deployment.

Complete Token Update Procedure

Follow this checklist when updating JWT tokens:

Pre-Update Checks

# 1. Verify current JWT secret
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.JWT_SECRET}' | base64 -d

# 2. Check current tokens
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.ANON_KEY}' | base64 -d
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.SERVICE_ROLE_KEY}' | base64 -d

# 3. Verify Kong's current tokens
kubectl exec -n supabase -l app=kong -- cat /usr/local/kong/declarative/kong.yml | grep -A 3 "consumers:"

Update Process

# 1. Generate new tokens (using jwt.io or Docker command from "Generating New JWT Tokens" section)
NEW_ANON_KEY="<paste-your-generated-anon-token-here>"
NEW_SERVICE_KEY="<paste-your-generated-service-role-token-here>"

# 2. Update Kong manifest file
# Edit /root/k8s/manifests/supabase/kong.yaml
# Replace tokens in consumers section (see example below)

# 3. Apply Kong config
kubectl apply -f /root/k8s/manifests/supabase/kong.yaml

# 4. Update Kubernetes secret
kubectl patch secret -n supabase supabase-secrets --type='json' -p='[
  {"op": "replace", "path": "/data/ANON_KEY", "value": "'"$(echo -n "$NEW_ANON_KEY" | base64 -w0)"'"},
  {"op": "replace", "path": "/data/SERVICE_ROLE_KEY", "value": "'"$(echo -n "$NEW_SERVICE_KEY" | base64 -w0)"'"}
]'

# 5. Restart Kong (must restart to reload config)
kubectl rollout restart deployment -n supabase kong

# 6. Restart services using the secret
kubectl rollout restart deployment -n supabase gotrue
kubectl rollout restart deployment -n supabase rest
kubectl rollout restart deployment -n supabase studio

# 7. Wait for all pods to be ready
kubectl get pods -n supabase -w

Verification

# 1. Check Kong has new tokens
kubectl exec -n supabase -l app=kong -- cat /usr/local/kong/declarative/kong.yml | grep -A 2 "service_role"

# 2. Check secret updated
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.SERVICE_ROLE_KEY}' | base64 -d

# 3. Check Kong logs (should show no 401 errors)
kubectl logs -n supabase -l app=kong --tail=20

# 4. Test user creation in Studio
# Navigate to http://10.89.97.215:3000 and try creating a test user

Kong ConfigMap Example

File: /root/k8s/manifests/supabase/kong.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: kong-config
  namespace: supabase
data:
  kong.yml: |
    _format_version: "2.1"
    _transform: true

    services:
      - name: auth-v1
        url: http://gotrue:9999
        routes:
          - name: auth-v1-all
            strip_path: true
            paths:
              - /auth/v1/
        plugins:
          - name: cors
          - name: key-auth
            config:
              hide_credentials: false

      # ... other services ...

    consumers:
      - username: anon
        keyauth_credentials:
          - key: <YOUR_ANON_KEY_HERE>
      - username: service_role
        keyauth_credentials:
          - key: <YOUR_SERVICE_ROLE_KEY_HERE>

    plugins:
      - name: cors
        config:
          origins:
            - "*"
          credentials: true

Key Points: - consumers section defines allowed API keys - Each consumer has a keyauth_credentials list - These must match the tokens in supabase-secrets - Kong only accepts requests with these exact token values

Updating Application Configs

After updating tokens with the management script, update all development .env.local files:

Local Development (Host-based)

Get the new ANON_KEY and update each project's .env.local:

# Get new ANON_KEY (use supabase-sandbox for dev, supabase for prod)
NEW_ANON_KEY=$(kubectl get secret -n supabase-sandbox supabase-secrets -o jsonpath='{.data.ANON_KEY}' | base64 -d)

# Update all development projects on host
sed -i "s/^NEXT_PUBLIC_SUPABASE_ANON_KEY=.*/NEXT_PUBLIC_SUPABASE_ANON_KEY=$NEW_ANON_KEY/" /root/projects/subtitleai/.env.local
sed -i "s/^NEXT_PUBLIC_SUPABASE_ANON_KEY=.*/NEXT_PUBLIC_SUPABASE_ANON_KEY=$NEW_ANON_KEY/" /root/projects/home-portal/.env.local
sed -i "s/^NEXT_PUBLIC_SUPABASE_ANON_KEY=.*/NEXT_PUBLIC_SUPABASE_ANON_KEY=$NEW_ANON_KEY/" /root/projects/money-tracker/.env.local
sed -i "s/^NEXT_PUBLIC_SUPABASE_ANON_KEY=.*/NEXT_PUBLIC_SUPABASE_ANON_KEY=$NEW_ANON_KEY/" /root/projects/trip-planner/.env.local

# Restart dev servers for changes to take effect (in their tmux sessions)

Production Deployments (Kubernetes)

For production apps deployed to Kubernetes, update their ConfigMaps or Secrets and restart:

# Example for home-portal
kubectl patch configmap -n home-portal home-portal-config --type='json' -p='[
  {"op": "replace", "path": "/data/NEXT_PUBLIC_SUPABASE_ANON_KEY", "value": "'"$NEW_ANON_KEY"'"}
]'
kubectl rollout restart deployment -n home-portal home-portal

Debugging JWT Issues

Check Token Signature

# Decode JWT to view claims (does not verify signature)
echo "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." | cut -d. -f2 | base64 -d | jq

# Should show:
{
  "iss": "supabase",
  "ref": "localhost",
  "role": "anon",
  "iat": 1762817111,
  "exp": 2078177111
}

Verify Token is Valid

# Test ANON_KEY against Kong
curl -H "apikey: $NEW_ANON_KEY" http://10.89.97.214:8000/rest/v1/

# Should return 404 (schema not found) not 401 (unauthorized)
# 404 = Kong accepted token and forwarded to PostgREST
# 401 = Kong rejected token

Check Kong Authentication

# Watch Kong logs in real-time
kubectl logs -n supabase -l app=kong -f

# Try creating user in Studio and watch for:
# ✅ Success: "POST /auth/v1/admin/users 201"
# ❌ Failure: "POST /auth/v1/admin/users 401"

Common Error Messages

Error: invalid JWT: unable to parse or verify signature - Cause: Token signed with wrong JWT_SECRET - Fix: Regenerate tokens with correct secret

Error: 401 Unauthorized from Kong - Cause: Token not in Kong's consumer list - Fix: Update Kong ConfigMap and restart Kong

Error: relation "identities" does not exist - Cause: Database search_path doesn't include auth schema - Fix: Update DATABASE_URL to include ?search_path=auth

Security Best Practices

Token Rotation

Rotate JWT tokens every 6-12 months:

# 1. Generate new tokens with far-future expiry
# 2. Update Kong and secrets simultaneously
# 3. Update all applications
# 4. Monitor logs for authentication failures

Secret Management

Never commit tokens to Git: - Keep tokens in Kubernetes secrets only - Use .env.local for local development (gitignored) - Document token rotation procedures, not token values

Restrict SERVICE_ROLE_KEY: - SERVICE_ROLE_KEY bypasses all Row Level Security (RLS) - Only use in trusted backend services - Never expose in client-side code - Studio uses it for admin operations only

Reference

Token Claims Explained

{
  "iss": "supabase",        // Issuer - must be "supabase"
  "ref": "localhost",       // Project reference - "localhost" for self-hosted
  "role": "anon",           // Role - "anon" or "service_role"
  "iat": 1762817111,        // Issued at timestamp (Unix epoch)
  "exp": 2078177111         // Expiration timestamp (Unix epoch)
}

JWT Secret Location

The JWT_SECRET used to sign tokens is stored in:

# View JWT secret
kubectl get secret -n supabase supabase-secrets -o jsonpath='{.data.JWT_SECRET}' | base64 -d

# Default value (CHANGE IN PRODUCTION):
your-super-secret-jwt-token-with-at-least-32-characters-long

CRITICAL: All tokens must be signed with this secret. If you change JWT_SECRET, you must regenerate ALL tokens.