Quantum/docs/architecture.md

Caddy-RS Architecture Documentation

Overview

Caddy-RS is built as a modular, async-first reverse proxy server using Rust's powerful type system and memory safety guarantees. The architecture is designed for high performance, maintainability, and extensibility.

Core Design Principles

1. Memory Safety

• Zero unsafe code in the core application logic
• Ownership-based resource management prevents memory leaks
• No garbage collection overhead unlike Go-based Caddy

2. Async-First Architecture

• Tokio runtime for high-performance async I/O
• Non-blocking operations throughout the request pipeline
• Efficient connection handling with async/await patterns

3. Modular Design

• Separation of concerns with distinct modules
• Pluggable components for easy extension
• Clean interfaces between modules

4. Type Safety

• Compile-time guarantees for configuration validity
• Serde-based serialization with validation
• Strong typing prevents runtime errors

Module Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│      main       │───▶│     config      │───▶│     server      │
│   (Entry Point) │    │  (Configuration)│    │  (HTTP Server)  │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                                       │
                                                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│    metrics      │◀───│   middleware    │◀───│      proxy      │
│   (Monitoring)  │    │   (Pipeline)    │    │ (Load Balancer) │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                                       │
                                                       ▼
                               ┌─────────────────┐
                               │       tls       │
                               │ (Certificates)  │
                               └─────────────────┘

Module Details

Main Module (src/main.rs)

Responsibilities:

• Application bootstrapping
• Command-line argument parsing
• Configuration loading
• Server startup and shutdown

Key Components:

• main() function with Tokio runtime setup
• CLI argument handling with clap
• Configuration file loading
• Error handling and logging initialization

Flow:

main() -> parse_args() -> load_config() -> create_server() -> run_server()

Config Module (src/config/mod.rs)

Responsibilities:

• JSON configuration parsing
• Configuration validation
• Default value management
• Type-safe configuration structures

Key Structures:

pub struct Config {
    pub admin: AdminConfig,
    pub apps: Apps,
}

pub struct Server {
    pub listen: Vec<String>,
    pub routes: Vec<Route>,
    pub automatic_https: AutomaticHttps,
    pub tls: Option<TlsConfig>,
}

pub struct Route {
    pub handle: Vec<Handler>,
    pub match_rules: Option<Vec<Matcher>>,
}

Features:

• Serde-based deserialization with validation
• Caddy v2 JSON format compatibility
• Flexible default value handling
• Configuration file watching (planned)

Server Module (src/server/mod.rs)

Responsibilities:

• HTTP/HTTPS/HTTP3 server management
• Connection handling across all protocols
• Multi-port listening
• Request routing to proxy service

Architecture:

Server::new(config) -> Server::run() -> spawn_listeners() -> handle_connections()
├── HTTP/1.1 & HTTP/2 (TCP + TLS)
└── HTTP/3 (QUIC + TLS)

Key Features:

• Async TCP and QUIC listener management
• Per-server configuration handling
• Connection-level error handling
• Unified certificate management across protocols
• Graceful shutdown (planned)

HTTP/3 Server Module (src/server/http3.rs)

Responsibilities:

• QUIC protocol implementation
• HTTP/3 request/response handling
• Connection pooling and management
• H3 ↔ HTTP/1.1 protocol translation

Architecture:

Http3Server::new() -> serve() -> handle_connection() -> handle_request()
├── ConnectionManager (pooling, limits, cleanup)
├── QuicCertificateResolver (SNI support)
└── Protocol Translation (H3 ↔ HTTP/1.1)

Key Features:

• Quinn-based QUIC implementation
• Connection limits (1000 concurrent connections)
• Automatic idle connection cleanup (5-minute timeout)
• Real-time connection metrics and monitoring
• Seamless integration with existing proxy infrastructure

HTTP/1.1 & HTTP/2 Connection Flow:

1. Accept incoming TCP connection
2. Wrap in Tokio I/O abstraction
3. Create HTTP service handler
4. Route to ProxyService
5. Handle request/response lifecycle

HTTP/3 Connection Flow:

1. Accept incoming QUIC connection
2. Register with ConnectionManager
3. Handle H3 request streams
4. Translate H3 ↔ HTTP/1.1 protocol
5. Route to ProxyService
6. Send H3 response

Proxy Module (src/proxy/mod.rs)

Responsibilities:

• HTTP request/response proxying
• Route matching and handler dispatch
• Load balancing and upstream selection
• Request/response transformation

Core Components:

ProxyService

pub struct ProxyService {
    config: Arc<Config>,
    client: HttpClient,
    middleware: Arc<MiddlewareChain>,
    load_balancer: LoadBalancer,
}

Request Processing Pipeline

Request → Middleware → Route Matching → Handler Selection → Response
   ↓           ↓             ↓              ↓            ↑
Preprocess → Match → Select Handler → Execute → Postprocess

Handler Types

1. ReverseProxy: Proxies requests to upstream servers
2. StaticResponse: Returns configured static content
3. FileServer: Serves files from disk

Load Balancer

pub struct LoadBalancer;

impl LoadBalancer {
    pub fn select_upstream<'a>(
        &self,
        upstreams: &'a [Upstream],
        policy: &LoadBalancing,
    ) -> Result<&'a Upstream>;
}

Algorithms:

• Round Robin: Cyclical upstream selection
• Random: Randomly selected upstream
• Least Connections: Choose least loaded upstream (planned)
• IP Hash: Consistent upstream based on client IP (planned)

Middleware Module (src/middleware/mod.rs)

Responsibilities:

• Request preprocessing
• Response postprocessing
• Cross-cutting concerns (logging, CORS, etc.)
• Extensible middleware pipeline

Architecture:

pub trait Middleware {
    async fn preprocess_request(
        &self,
        req: Request<Incoming>,
        remote_addr: SocketAddr,
    ) -> Result<Request<Incoming>>;

    async fn postprocess_response(
        &self,
        resp: Response<BoxBody>,
        remote_addr: SocketAddr,
    ) -> Result<Response<BoxBody>>;
}

Built-in Middleware:

• LoggingMiddleware: Request/response logging
• CorsMiddleware: Cross-Origin Resource Sharing headers

Execution Order:

Request  → [Middleware 1] → [Middleware 2] → ... → Handler
Response ← [Middleware 1] ← [Middleware 2] ← ... ← Handler

TLS Module (src/tls/mod.rs) - Complete

Responsibilities:

• Unified certificate management for HTTP/2 and HTTP/3
• ACME/Let's Encrypt integration
• TLS termination and QUIC certificate resolution
• Certificate renewal and caching

Key Components:

pub struct TlsManager {
    config: Option<TlsConfig>,
    pub cert_resolver: Arc<CertificateResolver>,
    tls_acceptor: Option<TlsAcceptor>,
    pub acme_manager: Option<AcmeManager>,
}

pub struct CertificateResolver {
    certificates: RwLock<HashMap<String, Arc<CertifiedKey>>>,
    default_cert: RwLock<Option<Arc<CertifiedKey>>>,
}

pub struct AcmeManager {
    domains: Vec<String>,
    cache_dir: PathBuf,
    cert_resolver: Arc<CertificateResolver>,
}

Key Features:

• SNI (Server Name Indication) support for both protocols
• Wildcard certificate matching
• Thread-safe certificate storage
• Automatic certificate renewal
• Unified certificate resolver for HTTP/2 and HTTP/3

Completed Features:

• Automatic certificate acquisition via ACME
• Certificate validation and renewal
• Background renewal task with daily checking
• HTTP-01 challenge handling
• Certificate persistence and caching
• SNI (Server Name Indication) support
• OCSP stapling

Metrics Module (src/metrics/mod.rs) - Planned

Responsibilities:

• Performance metrics collection
• Prometheus endpoint
• Health monitoring
• Statistics aggregation

Planned Metrics:

• Request rate and latency
• Upstream health status
• Connection counts
• Error rates
• Memory and CPU usage

Data Flow

Request Processing Flow

1. Client Request → TCP Socket
2. TCP Socket → HTTP Parser
3. HTTP Parser → ProxyService.handle_request()
4. Middleware.preprocess_request()
5. Route matching against configured rules
6. Handler selection and execution
7. Upstream request (for reverse proxy)
8. Response processing
9. Middleware.postprocess_response()
10. Client Response

Configuration Loading Flow

1. Parse CLI arguments
2. Locate configuration file
3. Read and parse JSON
4. Deserialize into Config structures
5. Validate configuration
6. Apply defaults
7. Create server instances

Load Balancing Flow

1. Route matches reverse proxy handler
2. LoadBalancer.select_upstream() called
3. Algorithm selection based on config
4. Upstream health check (planned)
5. Return selected upstream
6. Proxy request to upstream

Performance Considerations

Memory Management

• Zero-copy operations where possible
• Efficient buffer management with Bytes crate
• Connection pooling for upstream requests
• Request/response streaming for large payloads

Concurrency

• Per-connection tasks for isolation
• Shared state minimization with Arc<> for read-only data
• Lock-free operations where possible
• Async I/O throughout the pipeline

Network Optimization

• HTTP keep-alive for upstream connections
• Connection reuse with hyper client
• Efficient header processing
• Streaming responses for large files

Error Handling Strategy

Error Types

// Using anyhow for application errors
use anyhow::{Result, Error};

// Custom error types for specific domains
#[derive(thiserror::Error, Debug)]
pub enum ProxyError {
    #[error("Upstream unavailable: {0}")]
    UpstreamUnavailable(String),
    
    #[error("Configuration invalid: {0}")]
    ConfigurationError(String),
}

Error Propagation

• Result types throughout the codebase
• Context-aware errors with anyhow
• Graceful degradation where possible
• Client-friendly error responses

Error Recovery

• Upstream failover for proxy requests
• Circuit breaker pattern (planned)
• Graceful shutdown on critical errors
• Configuration reload on config errors (planned)

Security Architecture

Input Validation

• Configuration validation at load time
• Request header validation
• Path traversal prevention for file server
• Size limits on requests and responses

Memory Safety

• Rust ownership model prevents common vulnerabilities
• No buffer overflows by design
• Safe string handling with UTF-8 validation
• Resource cleanup guaranteed by RAII

Network Security

• TLS termination (planned)
• Secure defaults in configuration
• Header sanitization in middleware
• Rate limiting (planned)

Testing Strategy

Unit Tests

• Module-level testing for each component
• Mock dependencies for isolated testing
• Property-based testing for critical algorithms
• Error condition testing

Integration Tests

• End-to-end request processing
• Configuration loading and validation
• Multi-server scenarios
• Load balancing behavior

Performance Tests

• Load testing with realistic traffic patterns
• Memory usage profiling
• Latency measurement under various conditions
• Scalability testing with multiple upstreams

Future Architecture Enhancements

Plugin System

pub trait Plugin {
    fn name(&self) -> &str;
    fn init(&mut self, config: &PluginConfig) -> Result<()>;
    fn handle_request(&self, req: &mut Request) -> Result<()>;
}

Configuration Hot Reload

• File system watching with notify crate
• Graceful configuration updates
• Zero-downtime reloads
• Configuration validation before applying

Advanced Load Balancing

• Consistent hashing for session affinity
• Weighted round-robin
• Geographic load balancing
• Custom load balancing algorithms

Observability

• Distributed tracing with OpenTelemetry
• Structured logging with JSON output
• Real-time metrics dashboard
• Health check endpoints

This architecture provides a solid foundation for building a high-performance, reliable reverse proxy server while maintaining the flexibility to add advanced features as the project evolves.