Skip to content

Migrating Existing Apps to Kubernetes

This guide covers migrating an existing Next.js app to Kubernetes with Authentik SSO.

When to Use This Guide

  • User says "migrate [app] to k8s"
  • User says "deploy [app] to production"
  • User says "add auth to [app]"

Prerequisites

  • Existing Next.js app in /root/projects/<app>/
  • App has basic functionality working locally
  • Supabase migrations exist in supabase/migrations/

Critical Order of Operations

Complete steps 1-2 BEFORE proceeding to infrastructure (steps 3-7)

Why this order matters: - Auth is baked into the Docker image at build time - Deploying without auth = publicly accessible app - Retrofitting auth after deployment requires full rebuild anyway

Step 1: Study Reference Implementation

Before writing any code, examine home-portal's auth implementation:

# Check auth file structure
ls /root/projects/home-portal/src/lib/auth/

# Check environment variables
cat /root/projects/home-portal/.env.local | grep -E "AUTH|AUTHENTIK"

# Check ingress hostname (should be *.bogocat.com, NOT *.internal)
kubectl get ingress -n home-portal -o jsonpath='{.items[0].spec.rules[0].host}'

Step 2: Add Authentik Authentication (BLOCKING)

  • [ ] Install deps: npm install next-auth@beta jsonwebtoken
  • [ ] Copy auth files from home-portal (NOT from templates - they may be stale):
  • lib/auth/config.ts
  • lib/auth/index.ts
  • lib/auth/middleware.ts
  • lib/auth/providers/authentik.ts
  • types/next-auth.d.ts
  • app/login/page.tsx
  • app/api/auth/[...nextauth]/route.ts
  • app/api/auth/mode/route.ts
  • [ ] Update root middleware.ts to use auth middleware
  • [ ] Create Authentik application in admin UI (https://auth.bogocat.com)
  • Provider name: {app}-provider
  • Redirect URIs:
    • http://localhost:3000/api/auth/callback/authentik
    • https://{app}.bogocat.com/api/auth/callback/authentik
  • Scopes: openid profile email groups
  • [ ] Configure .env.local with Authentik credentials
  • IMPORTANT: Set AUTH_MODE=authentik (not none) to allow production builds
  • [ ] Test locally with AUTH_MODE=authentik
  • [ ] Add SessionProvider to root layout
  • [ ] Add user avatar/menu component:
  • Install: npx shadcn@latest add avatar dropdown-menu
  • Copy UserMenu from scaffold: cp /root/tower-fleet/scaffolds/nextjs/src/components/layout/user-menu.tsx src/components/layout/
  • Copy SessionProvider: cp /root/tower-fleet/scaffolds/nextjs/src/components/layout/session-provider.tsx src/components/layout/
  • Add to header or sidebar

Guide: /root/tower-fleet/docs/reference/authentik-auth-pattern.md

Step 3: Verify Database Schema

  • [ ] Ensure migrations exist in supabase/migrations/
  • [ ] Schema should be isolated (e.g., tcg schema, not public)

Guide: /root/tower-fleet/docs/workflows/database-migrations.md

Step 4: Create Dockerfile

  • [ ] Copy from home-portal (reference implementation)
  • [ ] Add output: "standalone" to next.config.ts
  • [ ] Create .dockerignore

Location: /root/projects/{app}/Dockerfile

Example Dockerfile: 

FROM node:22-alpine AS base

FROM base AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci

FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build

FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 3000
CMD ["node", "server.js"]

Step 5: Create K8s Manifests

  • [ ] Namespace, Deployment, Service, Ingress, SealedSecret
  • [ ] IMPORTANT: Ingress host must be {app}.bogocat.com (NOT {app}.internal)
  • [ ] Add proxy buffer size annotations for OAuth headers (required for Authentik):
annotations:
  cert-manager.io/cluster-issuer: letsencrypt-prod
  nginx.ingress.kubernetes.io/proxy-body-size: "10m"
  nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"      # Required for OAuth
  nginx.ingress.kubernetes.io/proxy-buffers-number: "4"     # Required for OAuth

Without buffer annotations, OAuth callback will fail with 502 (header too big).

Copy from: /root/tower-fleet/manifests/apps/home-portal/

Required files: - namespace.yaml - deployment.yaml - service.yaml - ingress.yaml - sealed-secret.yaml - deploy.sh

Step 6: Create Deploy Script

  • [ ] Copy from /root/tower-fleet/scripts/deploy-home-portal.sh
  • [ ] Update app name, namespace, registry path

Location: /root/tower-fleet/scripts/deploy-{app}.sh

Step 7: Configure External Access

Domain naming: All production apps use {app}.bogocat.com (NOT {app}.internal)

  • [ ] Add entry to VPS Caddyfile (/root/tower-fleet/manifests/vps/Caddyfile.production)
  • [ ] Deploy to VPS: scp + caddy reload
  • [ ] (Optional) Add OPNsense DNS override for internal direct access

Guide: /root/tower-fleet/docs/infrastructure/vps-reverse-proxy.md

Step 8: Create Sealed Secret

  • [ ] Create secret with auth credentials:
  • AUTH_SECRET
  • AUTHENTIK_CLIENT_ID
  • AUTHENTIK_CLIENT_SECRET
  • [ ] Seal with kubeseal
  • [ ] Apply to cluster
# Create secret
kubectl create secret generic {app}-secrets \
  --from-literal=AUTH_SECRET=$(openssl rand -base64 32) \
  --from-literal=AUTHENTIK_CLIENT_ID=<from-authentik> \
  --from-literal=AUTHENTIK_CLIENT_SECRET=<from-authentik> \
  --dry-run=client -o yaml > secret.yaml

# Seal it
kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# Apply
kubectl apply -f sealed-secret.yaml

Step 9: Build, Deploy, and Verify

  • [ ] Build Docker image with .env.production containing AUTH_MODE=authentik
  • [ ] Push to registry
  • [ ] Deploy to cluster
  • [ ] Test auth flow end-to-end at https://{app}.bogocat.com
# Build and push
docker build -t 10.89.97.201:30500/{app}:v1.0.0 .
docker push 10.89.97.201:30500/{app}:v1.0.0

# Deploy
./scripts/deploy-{app}.sh

# Verify
kubectl get pods -n {app}
kubectl logs -n {app} -l app={app}

Reference Implementation

home-portal is the canonical reference: - Code: /root/projects/home-portal/src/lib/auth/ - Manifests: /root/tower-fleet/manifests/apps/home-portal/ - Deploy script: /root/tower-fleet/scripts/deploy-home-portal.sh - Ingress: portal.bogocat.com (via Caddy on VPS)

Troubleshooting

Auth not working after deploy: - Check AUTH_MODE in deployment env vars - Verify sealed secret was applied - Check Authentik redirect URIs include production domain

502 Bad Gateway: - Pod may not be ready - check kubectl get pods -n {app} - Check ingress annotations for buffer sizes

Can't reach app externally: - Verify Caddyfile entry on VPS - Check DNS resolution - Verify ingress host matches Caddy config