Skip to content

OtterWiki Bidirectional Sync Design

Status: Future Enhancement (Roadmap) Priority: Low Date: 2025-12-10

Current Workflow: Edit docs in tower-fleet/docs/, push to GitHub, changes sync to OtterWiki automatically. This works well for now.

Problem

Currently, OtterWiki is read-only from a user perspective: - git-sync pulls from tower-fleet-docs → OtterWiki - Edits made in OtterWiki UI are overwritten on next sync (60 seconds)

We want bidirectional sync so users can edit in the wiki UI.

Current Architecture

tower-fleet/docs → (GitHub Action) → tower-fleet-docs → (git-sync) → OtterWiki
                                                                         │
                                                                    OVERWRITES
                                                                    wiki edits

Proposed Architecture

                    ┌────────────────────────────────────────┐
                    │            GitHub                       │
                    │                                         │
                    │   tower-fleet/docs ◀──── GitHub Action  │
                    │         │                    │          │
                    │         │              tower-fleet-docs │
                    │         │                               │
                    └─────────┼───────────────────────────────┘
                              │
                         push │ pull
                              │
                    ┌─────────▼───────────────────────────────┐
                    │         OtterWiki Pod                    │
                    │                                          │
                    │  ┌──────────┐     ┌─────────────────┐   │
                    │  │OtterWiki │────▶│  /app-data/     │   │
                    │  │  (UI)    │     │  (git repo)     │   │
                    │  └──────────┘     │                 │   │
                    │                   │  remote: origin │   │
                    │                   │  → tower-fleet  │   │
                    │                   └────────┬────────┘   │
                    │                            │            │
                    │                   post-receive hook     │
                    │                   (auto push)           │
                    └────────────────────────────────────────┘

Implementation Options

Configure OtterWiki's internal git repo with a remote and hooks.

Steps:

  1. Remove git-sync sidecar (no longer needed)

  2. Initialize remote in OtterWiki's repo: 

    kubectl exec -n otterwiki deployment/otterwiki -c otterwiki -- \
  git -C /app-data remote add origin git@github.com:jakecelentano/tower-fleet.git

  3. Add post-commit hook to push on every save: 

    # /app-data/.git/hooks/post-commit
#!/bin/sh
git push origin main 2>&1 | logger -t otterwiki-sync

  4. Add cron/timer for pull (catch remote changes): 

    # Every 5 minutes
*/5 * * * * cd /app-data && git pull --rebase origin main

  5. Mount SSH key for GitHub auth

Tradeoffs: | Pros | Cons | |------|------| | Simple, native git | Conflicts need manual resolution | | Real-time push on save | Need SSH key in pod | | No extra sidecars | Pull is polling-based |

Option 2: Bidirectional Sync Sidecar

Keep git-sync for pulling, add push sidecar.

Architecture: 

containers:
- name: otterwiki
  # ... wiki container

- name: git-sync
  # Pulls from GitHub every 60s

- name: git-push
  # Watches for new commits, pushes to GitHub
  image: alpine/git
  command: ["/scripts/push-watcher.sh"]

push-watcher.sh: 

#!/bin/sh
cd /app-data
LAST_COMMIT=""

while true; do
  CURRENT=$(git rev-parse HEAD 2>/dev/null)
  if [ "$CURRENT" != "$LAST_COMMIT" ] && [ -n "$LAST_COMMIT" ]; then
    echo "New commit detected, pushing..."
    git push origin main
  fi
  LAST_COMMIT="$CURRENT"
  sleep 5
done

Tradeoffs: | Pros | Cons | |------|------| | Separation of concerns | Two sidecars = more complexity | | Can add custom logic | Potential race conditions | | Easy to disable one direction | More resource usage |

Option 3: GitHub Webhook + Pull

Real-time bidirectional like Wiki.js 3.

Architecture: 

GitHub ──webhook──▶ OtterWiki webhook receiver ──▶ git pull
OtterWiki save ──▶ git push ──▶ GitHub

Requirements: - Expose webhook endpoint (e.g., /api/git-webhook) - Configure GitHub webhook to POST on push - OtterWiki or sidecar handles webhook

Tradeoffs: | Pros | Cons | |------|------| | Instant sync both ways | Need webhook endpoint exposed | | Most responsive | More complex setup | | Like Wiki.js 3 | OtterWiki may not support webhooks |

Conflict Resolution

All options face the same conflict challenge:

Scenario: You edit page A in OtterWiki. Someone else pushes changes to page A in tower-fleet.

Resolution strategies:

  1. Last write wins - Whoever pushes last overwrites

  2. Simple but can lose changes

  3. Git merge - Let git handle it

  4. Works for different files

  5. Same file = merge conflict (needs manual resolution)

  6. Wiki-first - Wiki changes always win

  7. Never lose wiki edits

  8. May overwrite repo changes

  9. Repo-first - Repo changes always win

  10. Current behavior (git-sync)
  11. Wiki edits get overwritten

Recommendation: Git merge with notification on conflict.

Path Mapping

Challenge: OtterWiki stores at /app-data/, tower-fleet has docs at /docs/

Options:

  1. Sync to tower-fleet-docs (flat structure, matches wiki)
  2. Sync to tower-fleet/docs (need path prefix)
  3. Sparse checkout (only checkout /docs subdirectory)

Recommendation: Continue using tower-fleet-docs as intermediate, or configure sparse checkout of tower-fleet.

Start with Option 1 (Native Git Remote):

  1. It's simplest and uses standard git
  2. OtterWiki already has git internally
  3. Easy to understand and debug
  4. Can upgrade to webhooks later if needed

Implementation steps:

# 1. Stop current git-sync
kubectl scale deployment otterwiki -n otterwiki --replicas=0

# 2. Exec into pod and configure
kubectl exec -it -n otterwiki deployment/otterwiki -- sh

# Inside pod:
cd /app-data
git remote add origin git@github.com:jakecelentano/tower-fleet-docs.git
git fetch origin
git branch --set-upstream-to=origin/main main

# 3. Create post-commit hook
cat > .git/hooks/post-commit << 'EOF'
#!/bin/sh
git push origin main 2>&1
EOF
chmod +x .git/hooks/post-commit

# 4. Create pull script (run via cron or sidecar)
# Pull every 5 minutes to catch remote changes

Migration Plan

  1. Backup current wiki content
  2. Test in dev environment first
  3. Configure SSH key secret with write access
  4. Remove git-sync sidecar
  5. Add remote and hooks
  6. Monitor for conflicts
  7. Document conflict resolution process

Security Considerations

  • SSH key needs write access to repo
  • Key stored as K8s secret
  • Consider deploy key with limited scope
  • Audit trail via git commits

References