Quantum/docs/websocket-sync.md

# Real-time WebSocket Synchronization

## Overview

The Caddy-RS file synchronization system now supports real-time synchronization using WebSocket connections. This eliminates the 30-second polling delay and provides instant file synchronization across all connected clients.

## Architecture

```
┌─────────────────┐    WebSocket    ┌─────────────────┐
│   Client A      │ ◄─────────────► │   Server        │
│                 │                 │                 │
└─────────────────┘                 └─────────────────┘
                                             ▲
                                             │ WebSocket
                                             ▼
                                    ┌─────────────────┐
                                    │   Client B      │
                                    │                 │
                                    └─────────────────┘
```

### Message Flow

1. **Client connects** → WebSocket handshake → Subscribe to updates
2. **File changes** → Local watcher detects → Broadcast to all clients
3. **All clients** → Receive operation → Apply changes immediately

## WebSocket Protocol

### Connection Endpoint

```
WS:  ws://localhost:8080/ws
WSS: wss://localhost:8080/ws (when HTTPS is enabled)
```

### Message Types

All messages are JSON-formatted with a `type` field:

#### Client → Server Messages

**Subscribe**
```json
{
  "type": "Subscribe",
  "client_id": "uuid-string"
}
```

**File Operation**
```json
{
  "type": "FileOperation",
  "operation": {
    "Create": {
      "metadata": {
        "path": "documents/new-file.txt",
        "size": 1024,
        "modified": "2024-01-21T12:00:00Z",
        "hash": "sha256hash",
        "is_directory": false
      }
    }
  },
  "source_client": "uuid-string"
}
```

**Heartbeat**
```json
{
  "type": "Ping"
}
```

#### Server → Client Messages

**Acknowledgment**
```json
{
  "type": "Ack",
  "operation_id": "subscribe"
}
```

**File Operation Broadcast**
```json
{
  "type": "FileOperation",
  "operation": {
    "Update": {
      "metadata": {
        "path": "documents/updated-file.txt",
        "size": 2048,
        "modified": "2024-01-21T12:05:00Z",
        "hash": "newhash",
        "is_directory": false
      }
    }
  },
  "source_client": "other-client-uuid"
}
```

**Error**
```json
{
  "type": "Error",
  "message": "Error description"
}
```

## Usage

### Enhanced Sync Client

Use the new real-time sync client:

```bash
cargo run --bin realtime-sync-client -- \
  --server http://localhost:8080 \
  --local-path ./my-realtime-sync \
  --realtime \
  --initial-sync
```

### Command Line Options

- `--realtime` / `-r`: Enable WebSocket real-time sync
- `--client-id <ID>`: Specify client identifier (auto-generated if omitted)
- `--server <URL>`: Server URL (same as regular client)
- `--local-path <PATH>`: Local sync directory
- `--initial-sync`: Download all files on startup

### Fallback Mode

If WebSocket connection fails, the client automatically falls back to periodic HTTP sync:

```bash
# Without --realtime flag, uses periodic sync
cargo run --bin realtime-sync-client -- \
  --server http://localhost:8080 \
  --local-path ./my-sync
```

## Implementation Details

### Server-side WebSocket Support

The `WsManager` handles:
- Client connection lifecycle
- Message broadcasting to all connected clients
- Heartbeat/keepalive management
- Stale connection cleanup

### Client-side WebSocket Support

The `WsClient` provides:
- Automatic reconnection (future enhancement)
- Message serialization/deserialization
- Heartbeat transmission
- Operation broadcasting

### Integration with File System Watcher

The enhanced client combines:
1. **File system watcher** → Detects local changes
2. **WebSocket client** → Broadcasts changes to server
3. **WebSocket receiver** → Applies remote changes locally
4. **HTTP client** → Fallback for large file transfers

## Performance Benefits

### Real-time vs Periodic Sync

| Aspect | Periodic (30s) | WebSocket Real-time |
|--------|----------------|-------------------|
| Latency | Up to 30 seconds | < 1 second |
| Network Usage | Periodic polling | Event-driven |
| Server Load | Regular load spikes | Distributed load |
| Conflict Window | 30-second window | Minimal window |

### Bandwidth Optimization

- **Metadata only**: WebSocket sends file metadata, not content
- **HTTP for content**: Large files still use HTTP upload/download
- **Event-driven**: No unnecessary polling traffic

## Security Considerations

### Connection Security

- **Authentication**: Client ID-based identification
- **Transport**: WSS (WebSocket Secure) for encrypted connections
- **Validation**: All paths and operations validated server-side

### Message Integrity

- **JSON Schema**: All messages validated against expected format
- **Source Filtering**: Prevents clients from receiving their own operations
- **Error Handling**: Graceful degradation on invalid messages

## Configuration

### Server Configuration

WebSocket endpoint is automatically available when file sync is enabled:

```json
{
  "routes": [
    {
      "match": [{"matcher": "path", "paths": ["/api/*", "/ws"]}],
      "handle": [
        {
          "handler": "file_sync",
          "root": "./sync-data",
          "enable_upload": true
        }
      ]
    }
  ]
}
```

### Client Configuration

Set environment variables for debugging:

```bash
# Enable debug logging
RUST_LOG=debug cargo run --bin realtime-sync-client -- \
  --server ws://localhost:8080 \
  --local-path ./test-sync \
  --realtime
```

## Error Handling

### Connection Failures

1. **Initial connection failure** → Fall back to periodic sync
2. **Connection drops** → Attempt reconnection (future)
3. **Message parsing errors** → Log and continue

### Operation Conflicts

WebSocket reduces conflict probability but doesn't eliminate it:
- **Simultaneous edits** → Still handled by existing conflict resolution
- **Network partitions** → Resolved when connection restored
- **Client crashes** → Other clients continue normally

## Monitoring and Debugging

### Server-side Logs

```
INFO WebSocket client abc-123 connected and subscribed
DEBUG Broadcasting file operation: Create { metadata: ... }
INFO Client def-456 disconnected
DEBUG Cleaned up stale WebSocket client: ghi-789
```

### Client-side Logs

```
INFO Connecting to WebSocket server: ws://localhost:8080/ws
INFO WebSocket client abc-123 connected and subscribed
INFO Received real-time operation: Update { metadata: ... }
DEBUG Received pong from server
```

### Health Checks

Monitor WebSocket health:
- **Heartbeat interval**: 30 seconds
- **Connection timeout**: 5 minutes
- **Stale cleanup**: Automatic every 5 minutes

## Future Enhancements

### Planned Features

1. **Automatic Reconnection**
   - Exponential backoff strategy
   - Connection state persistence
   - Offline queue for operations

2. **Message Compression**
   - Gzip compression for large operations
   - Binary protocol option
   - Batch operation support

3. **Advanced Authentication**
   - JWT token-based auth
   - Client certificate validation
   - Role-based permissions

4. **Scaling Support**
   - Redis pub/sub for multi-server
   - Client clustering
   - Load balancing

### Integration Points

- **Web Interface**: Real-time file browser updates
- **Metrics**: WebSocket connection and message metrics
- **Admin API**: WebSocket management and monitoring

## Troubleshooting

### Common Issues

**WebSocket connection refused:**
```
Error: Connection refused (os error 61)
```
- Ensure server is running with file sync enabled
- Check firewall settings for WebSocket port

**Messages not received:**
```
DEBUG No local listeners for WebSocket file operations
```
- Verify client is subscribed with correct client ID
- Check message filtering logic

**High memory usage:**
```
WARN WebSocket message queue growing
```
- Implement message queue limits
- Add backpressure handling

### Debug Commands

```bash
# Test WebSocket connection manually
wscat -c ws://localhost:8080/ws

# Monitor network traffic
tcpdump -i lo0 port 8080

# Check client connections
curl http://localhost:8080/api/list
```

---

**Status: WebSocket real-time sync implemented and ready for testing!**