Specification: Canonical JSON and Hash-Chained Audit¶

Spec ID: tower-fleet/audit/v1 Status: Normative Version: 1.0.0 Created: 2025-12-18

Abstract¶

This specification defines the audit logging format for tower-fleet intent execution. It establishes canonical JSON serialization, hash computation, chain verification, and event schemas for tamper-evident, replay-supporting audit trails.

Conformance¶

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

1. Goals¶

Append-only - Events are written sequentially, never modified
Tamper-evident - Hash chains detect unauthorized modifications
Replay-supporting - Sufficient context to reproduce or explain execution
Unambiguous hashing - Canonical serialization ensures consistent hashes

2. File Format and Location¶

2.1 File Structure¶

Audit logs use JSON Lines (JSONL) format: - One JSON object per line - Lines separated by \n (LF, 0x0A) - No trailing comma between objects - UTF-8 encoding

2.2 File Location¶

/root/tower-fleet/logs/intents/<YYYY-MM-DD>/<request_id>.jsonl

Example:

/root/tower-fleet/logs/intents/2025-12-18/req_abc123def456.jsonl

2.3 Directory Structure¶

/root/tower-fleet/logs/
├── intents/
│   ├── 2025-12-18/
│   │   ├── req_abc123def456.jsonl
│   │   └── req_xyz789ghi012.jsonl
│   └── 2025-12-17/
│       └── ...
└── locks/
    └── ...

2.4 File Permissions¶

Audit files MUST: - Be created with mode 0644 or more restrictive - Be append-only after creation (where filesystem supports it) - NOT be modified after completion

3. Canonical JSON Serialization¶

3.1 Definition¶

Canonical JSON is a deterministic serialization that produces identical byte sequences for equivalent objects.

3.2 Rules¶

Encoding: UTF-8, no BOM
Key ordering: Object keys MUST be sorted lexicographically (Unicode code point order)
Whitespace: No insignificant whitespace (compact form)
Numbers: Serialize as JSON numbers, not strings (unless originally strings)
Strings: Standard JSON escaping, no unnecessary escapes
Booleans: Lowercase true and false
Null: Lowercase null
Arrays: Preserve order (arrays are ordered)

3.3 Implementation¶

Using jq (recommended):

canonical_json() {
  jq -cS '.'
}

echo '{"b":2,"a":1}' | canonical_json
# Output: {"a":1,"b":2}

Using Python:

import json

def canonical_json(obj):
    return json.dumps(obj, sort_keys=True, separators=(',', ':'))

3.4 Examples¶

Input	Canonical Output
`{"b":2,"a":1}`	`{"a":1,"b":2}`
`{ "x" : 1 }`	`{"x":1}`
`{"n":1.0}`	`{"n":1.0}`
`{"s":"hello\nworld"}`	`{"s":"hello\nworld"}`

4. Hash Computation¶

4.1 Algorithm¶

All hashes MUST use SHA-256.

4.2 Event Hash Computation¶

For each event:

Create a copy of the event object WITHOUT the event_hash field
Serialize to canonical JSON
Compute SHA-256 of the UTF-8 bytes
Encode as lowercase hexadecimal with sha256: prefix

compute_event_hash() {
  local event_json="$1"

  # Remove event_hash field, canonicalize, hash
  echo "$event_json" \
    | jq -cS 'del(.event_hash)' \
    | sha256sum \
    | cut -d' ' -f1 \
    | sed 's/^/sha256:/'
}

4.3 Chain Hash (`prev_hash`)¶

First event in a request: prev_hash MUST be null
All subsequent events: prev_hash MUST equal the event_hash of the immediately preceding event

4.4 Hash Format¶

sha256:<64 lowercase hex characters>

Example: sha256:a1b2c3d4e5f6... (64 hex chars total)

5. Event Structure¶

5.1 Required Fields¶

Every event MUST include these fields:

Field	Type	Description
`audit_version`	string	Always `"v1"`
`event`	string	Event type identifier
`request_id`	string	Unique execution identifier
`timestamp`	string	ISO-8601 UTC with `Z` suffix
`prev_hash`	string\|null	Hash of previous event, or null for first
`event_hash`	string	Hash of this event (computed)

5.2 Timestamp Format¶

Timestamps MUST be: - ISO-8601 format - UTC timezone - Z suffix (not +00:00) - Second precision minimum

2025-12-18T10:30:45Z

Millisecond precision is OPTIONAL:

2025-12-18T10:30:45.123Z

5.3 Request ID Format¶

Request IDs MUST: - Start with req_ - Contain only [a-z0-9_] - Be unique across all executions

RECOMMENDED format: req_<timestamp>_<random>

generate_request_id() {
  echo "req_$(date +%s)_$(openssl rand -hex 8)"
}
# Example: req_1734517845_a1b2c3d4e5f6g7h8

6. Event Types¶

6.1 Lifecycle Events¶

`intent_received`¶

First event, logged when intent execution begins.

{
  "audit_version": "v1",
  "event": "intent_received",
  "request_id": "req_1734517845_abc123",
  "timestamp": "2025-12-18T10:30:45Z",
  "prev_hash": null,
  "event_hash": "sha256:...",

  "intent": "deploy-app",
  "intent_version": "1.2.0",
  "intent_hash": "sha256:...",
  "policy_hash": "sha256:...",
  "executor_version": "0.1.0",
  "host_id": "tower-01",
  "actor": "claude-code",
  "source": "slash_command",
  "raw_input": "/deploy:app money-tracker --env=production"
}

Required fields: - intent: Intent name - intent_version: Version from intent metadata - intent_hash: SHA-256 of intent definition file bytes - policy_hash: SHA-256 of policy rules file bytes - executor_version: Version of tower-fleet executor - host_id: Executing host identifier - actor: Who initiated (user, automation name) - source: slash_command | llm_proposal | api | scheduled - raw_input: Original input string

`intent_completed`¶

Final event, logged when execution finishes.

{
  "audit_version": "v1",
  "event": "intent_completed",
  "request_id": "req_1734517845_abc123",
  "timestamp": "2025-12-18T10:45:30Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "result": "success",
  "duration_ms": 885000,
  "steps_completed": 4,
  "steps_failed": 0,
  "verifications_passed": 2,
  "verifications_failed": 0,
  "outputs": {
    "url": "https://money-tracker.bogocat.com",
    "image_digest": "sha256:..."
  }
}

Result values: success | failure | aborted | rejected

6.2 Resolution Events¶

`params_resolved`¶

{
  "audit_version": "v1",
  "event": "params_resolved",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:46Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "params": {
    "app": "money-tracker",
    "environment": "production",
    "skip_build_check": false
  },
  "params_hash": "sha256:..."
}

`context_captured`¶

{
  "audit_version": "v1",
  "event": "context_captured",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:47Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "context": {
    "git_sha": "abc123def456",
    "git_dirty": false,
    "node_version": "v22.0.0",
    "docker_version": "24.0.0",
    "kubectl_version": "v1.28.0",
    "kube_context": "k3s-cluster"
  },
  "context_hash": "sha256:..."
}

6.3 Policy Events¶

`policy_evaluated`¶

{
  "audit_version": "v1",
  "event": "policy_evaluated",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:48Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "decision": "require_confirmation",
  "confirmation_kind": "type_to_confirm",
  "confirm_value": "money-tracker",
  "rules_matched": [
    "high_risk_confirmation",
    "destructive_requires_typing"
  ],
  "reasons": [
    "risk=high requires confirmation",
    "risk=destructive requires typing resource name"
  ]
}

Decision values: allow | deny | require_confirmation | require_approval

Confirmation kinds (when decision is require_confirmation): - interactive - Simple yes/no confirmation - type_to_confirm - Must type confirm_value exactly - flag - Must pass --confirm flag

6.4 Prerequisite Events¶

`prereq_checked`¶

{
  "audit_version": "v1",
  "event": "prereq_checked",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:50Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "prereq": "auth_configured",
  "result": "pass",
  "check_type": "file_exists",
  "check_target": "/root/projects/money-tracker/lib/auth/config.ts",
  "duration_ms": 5
}

For failed prerequisites:

{
  "prereq": "build_passes",
  "result": "fail",
  "check_type": "exec",
  "check_target": "npm run build",
  "error": "Build failed with exit code 1",
  "output_hash": "sha256:..."
}

6.5 Lock Events¶

`lock_acquired`¶

{
  "audit_version": "v1",
  "event": "lock_acquired",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:52Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "lock_name": "money-tracker-production",
  "lock_path": "/root/tower-fleet/logs/locks/money-tracker-production.lock",
  "ttl_seconds": 900
}

`lock_stolen`¶

{
  "audit_version": "v1",
  "event": "lock_stolen",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:52Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "lock_name": "money-tracker-production",
  "previous_lock_hash": "sha256:...",
  "previous_holder": {
    "request_id": "req_old123",
    "actor": "claude-code",
    "created_at": "2025-12-18T10:00:00Z"
  },
  "reason": "stale_lock_forced"
}

`lock_released`¶

{
  "audit_version": "v1",
  "event": "lock_released",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:45:28Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "lock_name": "money-tracker-production",
  "held_duration_seconds": 876,
  "result": "success"
}

6.6 Execution Events¶

`plan_rendered`¶

{
  "audit_version": "v1",
  "event": "plan_rendered",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:55Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "steps": [
    {"name": "build_image", "description": "Build Docker image"},
    {"name": "push_image", "description": "Push image to registry"},
    {"name": "deploy_manifests", "description": "Apply Kubernetes manifests"},
    {"name": "wait_rollout", "description": "Wait for rollout to complete"}
  ],
  "plan_hash": "sha256:..."
}

`confirmation_received`¶

{
  "audit_version": "v1",
  "event": "confirmation_received",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:31:10Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "confirmed": true,
  "confirmation_kind": "type_to_confirm",
  "typed_value": "money-tracker",
  "expected_value": "money-tracker",
  "plan_hash": "sha256:plan123..."
}

Fields: - confirmed: Whether confirmation was received - confirmation_kind: How confirmation was provided (interactive | type_to_confirm | flag | api) - typed_value: What user typed (for type_to_confirm) - expected_value: What was expected (for type_to_confirm) - plan_hash: Hash of plan that was confirmed (ties confirmation to specific plan)

`step_started`¶

{
  "audit_version": "v1",
  "event": "step_started",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:31:15Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "step": "build_image",
  "step_index": 0,
  "command_hash": "sha256:..."
}

`step_finished`¶

{
  "audit_version": "v1",
  "event": "step_finished",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:35:45Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "step": "build_image",
  "step_index": 0,
  "result": "success",
  "exit_code": 0,
  "duration_ms": 270000,
  "stdout_hash": "sha256:...",
  "stderr_hash": "sha256:...",
  "outputs": {
    "image_tag": "registry.internal/money-tracker:abc123"
  }
}

Result values: success | failure | skipped | timeout

6.7 Verification Events¶

`verify_finished`¶

{
  "audit_version": "v1",
  "event": "verify_finished",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:44:30Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "verification": "pods_ready",
  "result": "pass",
  "attempts": 3,
  "check_type": "k8s_condition",
  "check_target": "deployment/money-tracker"
}

6.8 Rollback Events¶

`rollback_started`¶

{
  "audit_version": "v1",
  "event": "rollback_started",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:40:00Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "reason": "step_failed",
  "failed_step": "deploy_manifests",
  "rollback_from_step": "deploy_manifests",
  "rollback_to_step": "push_image"
}

`rollback_finished`¶

{
  "audit_version": "v1",
  "event": "rollback_finished",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:42:00Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "result": "success",
  "steps_rolled_back": ["deploy_manifests"],
  "duration_ms": 120000
}

6.9 Error Events¶

`template_error`¶

{
  "audit_version": "v1",
  "event": "template_error",
  "request_id": "req_...",
  "timestamp": "2025-12-18T10:30:46Z",
  "prev_hash": "sha256:...",
  "event_hash": "sha256:...",

  "error_type": "template_missing_variable",
  "expression": "${params.environment}",
  "scope": "params",
  "available_vars": ["app", "skip_build_check"]
}

7. Redaction Rules¶

7.1 Never Log Raw¶

The following MUST NEVER appear in audit logs: - Passwords, tokens, API keys - Private keys, certificates - Authorization headers - .npmrc, .netrc contents - kubeconfig credentials - Environment variables containing secrets

7.2 Hash-Only Fields¶

For command outputs that may contain sensitive data: - Store stdout_hash and stderr_hash, not raw content - OPTIONALLY store a bounded, redacted excerpt (first N lines, patterns removed)

7.3 Redaction Patterns¶

If storing excerpts, redact patterns matching:

(?i)(authorization|bearer|token|password|secret|key|credential)[:\s]*\S+

8. Verification¶

8.1 Verification Tool Requirements¶

audit-viewer --verify <file> MUST:

Parse each line as JSON
Validate required fields present
Verify prev_hash chain (each event's prev_hash equals previous event_hash)
Recompute event_hash and compare to stored value
Report first failure with line number and reason
Exit 0 if valid, non-zero if invalid

8.2 Verification Output¶

$ audit-viewer --verify /root/tower-fleet/logs/intents/2025-12-18/req_abc123.jsonl

Verifying: req_abc123.jsonl
  Lines: 15
  Events: 15
  Chain: VALID
  Hashes: VALID

Result: PASS

On failure:

$ audit-viewer --verify corrupted.jsonl

Verifying: corrupted.jsonl
  Lines: 15
  Events: 15
  Chain: INVALID at line 8
    Expected prev_hash: sha256:abc123...
    Found prev_hash: sha256:def456...

Result: FAIL

9. Implementation Reference¶

9.1 Audit Logger¶

#!/bin/bash
# /root/tower-fleet/scripts/audit-log.sh

set -euo pipefail

LOG_DIR="/root/tower-fleet/logs/intents"

# Global state for chain
declare CURRENT_REQUEST_ID=""
declare PREV_HASH="null"

init_audit() {
  local request_id="$1"
  CURRENT_REQUEST_ID="$request_id"
  PREV_HASH="null"

  local date_dir
  date_dir=$(date -u +%Y-%m-%d)
  mkdir -p "${LOG_DIR}/${date_dir}"
}

emit_event() {
  local event_type="$1"
  local event_data="$2"

  local timestamp
  timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)

  local date_dir
  date_dir=$(date -u +%Y-%m-%d)

  local log_file="${LOG_DIR}/${date_dir}/${CURRENT_REQUEST_ID}.jsonl"

  # Build event object
  local event_json
  event_json=$(jq -n \
    --arg audit_version "v1" \
    --arg event "$event_type" \
    --arg request_id "$CURRENT_REQUEST_ID" \
    --arg timestamp "$timestamp" \
    --arg prev_hash "$PREV_HASH" \
    --argjson data "$event_data" \
    '{
      audit_version: $audit_version,
      event: $event,
      request_id: $request_id,
      timestamp: $timestamp,
      prev_hash: (if $prev_hash == "null" then null else $prev_hash end)
    } + $data'
  )

  # Compute event hash
  local event_hash
  event_hash=$(echo "$event_json" | jq -cS '.' | sha256sum | cut -d' ' -f1)
  event_hash="sha256:${event_hash}"

  # Add event_hash to event
  local final_event
  final_event=$(echo "$event_json" | jq -cS --arg eh "$event_hash" '. + {event_hash: $eh}')

  # Append to log
  echo "$final_event" >> "$log_file"

  # Update chain state
  PREV_HASH="$event_hash"

  # Return for caller
  echo "$final_event"
}

# Example usage:
# init_audit "req_123"
# emit_event "intent_received" '{"intent":"deploy-app","actor":"claude-code"}'

9.2 Audit Viewer¶

#!/bin/bash
# /root/tower-fleet/scripts/audit-viewer.sh

set -euo pipefail

verify_audit() {
  local file="$1"
  local line_num=0
  local prev_hash="null"
  local errors=0

  echo "Verifying: $(basename "$file")"

  while IFS= read -r line; do
    ((line_num++))

    # Parse JSON
    if ! echo "$line" | jq -e '.' >/dev/null 2>&1; then
      echo "  ERROR line $line_num: Invalid JSON"
      ((errors++))
      continue
    fi

    # Extract hashes
    local stored_prev_hash stored_event_hash
    stored_prev_hash=$(echo "$line" | jq -r '.prev_hash // "null"')
    stored_event_hash=$(echo "$line" | jq -r '.event_hash')

    # Verify chain
    if [[ "$stored_prev_hash" != "$prev_hash" ]]; then
      echo "  ERROR line $line_num: Chain broken"
      echo "    Expected prev_hash: $prev_hash"
      echo "    Found prev_hash: $stored_prev_hash"
      ((errors++))
    fi

    # Verify event hash
    local computed_hash
    computed_hash=$(echo "$line" | jq -cS 'del(.event_hash)' | sha256sum | cut -d' ' -f1)
    computed_hash="sha256:${computed_hash}"

    if [[ "$stored_event_hash" != "$computed_hash" ]]; then
      echo "  ERROR line $line_num: Hash mismatch"
      echo "    Stored: $stored_event_hash"
      echo "    Computed: $computed_hash"
      ((errors++))
    fi

    prev_hash="$stored_event_hash"

  done < "$file"

  echo "  Lines: $line_num"
  echo "  Errors: $errors"

  if [[ $errors -eq 0 ]]; then
    echo "Result: PASS"
    return 0
  else
    echo "Result: FAIL"
    return 1
  fi
}

# Main
case "${1:-}" in
  --verify)
    verify_audit "${2:?Missing file argument}"
    ;;
  --show)
    jq -C '.' "${2:?Missing file argument}"
    ;;
  *)
    echo "Usage: audit-viewer.sh --verify <file>"
    echo "       audit-viewer.sh --show <file>"
    exit 1
    ;;
esac

10. Retention and Archival¶

10.1 Retention Policy¶

Audit logs SHOULD be retained for: - Minimum: 90 days - Recommended: 365 days - Compliance-driven: As required by policy

10.2 Archival¶

Old logs MAY be: - Compressed (gzip, preserving JSONL structure) - Moved to cold storage - Integrity-checked before archival

10.3 Deletion¶

Before deletion: - Verify hash chain integrity - Create summary record (request_id, intent, result, timestamps) - Store summary separately for long-term reference