Quantum/docs/development.md

Development Guide

This guide covers everything you need to know to contribute to Caddy-RS development.

Development Setup

Prerequisites

• Rust 1.75+ with 2024 edition support
• Cargo package manager
• Git for version control
• Optional: Docker for testing

Environment Setup

1. Install Rust via rustup:

   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
   source ~/.cargo/env
   

2. Install useful development tools:

   cargo install cargo-watch    # Auto-reload during development
   cargo install cargo-edit     # Add/remove dependencies easily
   cargo install cargo-audit    # Security vulnerability scanning
   cargo install cargo-flamegraph  # Performance profiling
   

3. Clone and setup the project:

   git clone <repository-url>
   cd caddy-rs
   cargo build
   cargo test
   

Project Structure

quantum/
├── Cargo.toml              # Dependencies and project metadata
├── README.md               # Main documentation
├── SIMPLE-CONFIG.md        # Simple configuration guide
├── QUICKSTART.md           # Quick start scenarios
├── example-config.json     # Example full configuration
├── examples/               # Simple configuration examples
│   ├── proxy-simple.json
│   ├── static-files.json
│   └── full-stack.json
├── public/                 # Test files for file server
│   └── index.html
├── src/                    # Source code
│   ├── main.rs            # Application entry point
│   ├── config/            # Configuration parsing
│   │   ├── mod.rs         # Full Caddy configuration format
│   │   └── simple.rs      # Simple configuration format
│   ├── server/            # HTTP server implementation
│   │   └── mod.rs
│   ├── proxy/             # Reverse proxy and load balancing
│   │   └── mod.rs
│   ├── middleware/        # Request/response middleware
│   │   └── mod.rs
│   ├── tls/               # TLS and certificate management
│   │   └── mod.rs
│   ├── metrics/           # Metrics and monitoring
│   │   └── mod.rs
│   └── file_sync/         # File synchronization system
├── docs/                   # Documentation
│   ├── architecture.md    # Architecture documentation
│   ├── api.md             # API and configuration reference
│   └── development.md     # This file
└── tests/                  # Integration tests (planned)

Development Workflow

Daily Development

1. Start with tests:

   cargo test
   

2. Run with auto-reload during development:

   cargo watch -x 'run -- --config example-config.json'
   

3. Check code quality:

   cargo clippy -- -D warnings  # Linting
   cargo fmt                     # Code formatting
   

4. Test with different configurations:

   cargo run -- --port 3000
   cargo run -- --config custom-config.json
   

Configuration System

Quantum supports two configuration formats:

Simple Configuration (src/config/simple.rs)

The simple configuration format is designed for ease of use:

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SimpleConfig {
    #[serde(default)]
    pub proxy: HashMap<String, String>,
    #[serde(default)]
    pub static_files: HashMap<String, String>,
    #[serde(default)]
    pub file_sync: HashMap<String, String>,
    #[serde(default = "default_tls")]
    pub tls: String,
    pub admin_port: Option<String>,
}

Key features:

• Auto-validation: Comprehensive validation with helpful error messages
• Auto-conversion: Converts to full Caddy format internally
• Port normalization: Handles various port formats automatically
• Error messages: User-friendly validation with emojis and examples

Full Configuration (src/config/mod.rs)

Full Caddy v2 compatibility for advanced features:

• Complex route matching
• Advanced load balancing
• Health checks
• Custom middleware
• Complex TLS automation

Configuration Detection

The system automatically detects format in Config::from_file():

// Try simple config first
match serde_json::from_str::<simple::SimpleConfig>(&content) {
    Ok(simple_config) => {
        println!("✅ Detected simple configuration format");
        return simple_config.to_caddy_config();
    }
    Err(simple_err) => {
        // Fall back to full format
        match serde_json::from_str::<Config>(&content) {
            Ok(config) => {
                println!("✅ Detected full Caddy configuration format");
                Ok(config)
            }
            // Provide helpful error message for both formats
        }
    }
}

Adding New Features

1. Plan the feature:

   • Update documentation first (README, API docs)
   • Add configuration structures if needed
   • Plan the module interfaces
   • Consider if simple config support is needed

2. Implement incrementally:

   • Start with configuration parsing
   • Add simple config support if applicable
   • Add core logic
   • Implement tests
   • Add integration with existing modules

3. Example: Adding a new handler type

   // 1. Add to config/mod.rs
   #[derive(Debug, Clone, Serialize, Deserialize)]
   #[serde(tag = "handler")]
   pub enum Handler {
       // ... existing handlers
   
       #[serde(rename = "my_handler")]
       MyHandler {
           setting1: String,
           setting2: Option<u32>,
       },
   }
   
   // 2. Implement in proxy/mod.rs
   async fn handle_route(&self, req: Request<Incoming>, handler: &Handler) -> Result<Response<BoxBody>> {
       match handler {
           // ... existing handlers
   
           Handler::MyHandler { setting1, setting2 } => {
               self.handle_my_handler(req, setting1, setting2).await
           }
       }
   }
   
   // 3. Add the handler implementation
   async fn handle_my_handler(&self, req: Request<Incoming>, setting1: &str, setting2: &Option<u32>) -> Result<Response<BoxBody>> {
       // Implementation here
   }
   

Code Style Guidelines

1. Follow Rust conventions:

   • Use snake_case for functions and variables
   • Use PascalCase for types and traits
   • Use SCREAMING_SNAKE_CASE for constants

2. Error handling:

   // Use Result types throughout
   pub async fn my_function() -> Result<String> {
       let value = some_operation().await?;
       Ok(value)
   }
   
   // Use anyhow for application errors
   use anyhow::{Result, Context};
   
   let config = load_config().context("Failed to load configuration")?;
   

3. Async patterns:

   // Use async/await consistently
   pub async fn handle_request(&self, req: Request) -> Result<Response> {
       let processed = self.middleware.process(req).await?;
       let response = self.upstream_client.request(processed).await?;
       Ok(response)
   }
   

4. Documentation:

   /// Handles reverse proxy requests to upstream servers.
   /// 
   /// This function selects an upstream server using the configured
   /// load balancing algorithm and proxies the request.
   /// 
   /// # Arguments
   /// 
   /// * `req` - The incoming HTTP request
   /// * `upstreams` - List of available upstream servers
   /// 
   /// # Returns
   /// 
   /// Returns the response from the upstream server or an error
   /// if all upstreams are unavailable.
   pub async fn proxy_request(
       &self,
       req: Request<Incoming>,
       upstreams: &[Upstream],
   ) -> Result<Response<BoxBody>> {
       // Implementation
   }
   

Testing Strategy

Quantum includes comprehensive test coverage with 41 tests across all modules.

Current Test Coverage

Core Tests (35 tests):

• Config module: 17 tests covering configuration parsing, serialization, handlers, matchers
• Proxy module: 8 tests covering load balancing, upstream selection, content-type detection
• Server module: 8 tests covering address parsing, TLS detection, edge cases
• Middleware module: 4 tests covering CORS headers, middleware chain

Simple Config Tests (6 tests):

• Configuration validation and conversion
• Port normalization and error handling
• JSON serialization/deserialization
• Empty config handling with defaults

Running Tests

# Run all tests
cargo test

# Run specific module tests
cargo test config
cargo test simple
cargo test proxy

# Run with output
cargo test -- --nocapture

# Run with detailed logging
RUST_LOG=debug cargo test

Test Quality Standards

Real Business Logic Testing:

• ✅ No stub tests - All tests validate actual functionality
• ✅ Genuine validation - Tests parse real JSON, validate algorithms, check error paths
• ✅ Edge case coverage - IPv6 addresses, port ranges, empty configurations
• ✅ Error path testing - All validation errors have corresponding tests

Example Real Test:

#[tokio::test]
async fn test_config_serialization_deserialization() {
    let config_json = r#"{
        "admin": {"listen": ":2019"},
        "apps": {
            "http": {
                "servers": {
                    "test_server": {
                        "listen": [":8080"],
                        "routes": [{
                            "match": [{"matcher": "host", "hosts": ["example.com"]}],
                            "handle": [{
                                "handler": "reverse_proxy",
                                "upstreams": [{"dial": "backend:8080"}]
                            }]
                        }]
                    }
                }
            }
        }
    }"#;

    let config: Config = serde_json::from_str(config_json).unwrap();
    
    assert_eq!(config.admin.listen, Some(":2019".to_string()));
    assert!(config.apps.http.servers.contains_key("test_server"));
    
    let server = &config.apps.http.servers["test_server"];
    assert_eq!(server.listen, vec![":8080"]);
    
    // Validates complete JSON parsing pipeline
    if let Handler::ReverseProxy { upstreams, .. } = &server.routes[0].handle[0] {
        assert_eq!(upstreams[0].dial, "backend:8080");
    }
}

Unit Tests

Place unit tests in the same file as the code they test:

#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_load_balancer_no_upstreams() {
        let lb = LoadBalancer::new();
        let upstreams: Vec<Upstream> = vec![];
        let load_balancing = LoadBalancing {
            selection_policy: SelectionPolicy::RoundRobin,
        };

        let result = lb.select_upstream(&upstreams, &load_balancing);
        assert!(result.is_err());
        assert!(result.unwrap_err().to_string().contains("No upstreams available"));
    }
    
    #[test]
    fn test_simple_config_validation() {
        let mut proxy = HashMap::new();
        proxy.insert("localhost".to_string(), ":8080".to_string()); // Missing port
        
        let config = SimpleConfig {
            proxy,
            static_files: HashMap::new(),
            file_sync: HashMap::new(),
            tls: "auto".to_string(),
            admin_port: None,
        };

        let result = config.validate();
        assert!(result.is_err());
        assert!(result.unwrap_err().to_string().contains("must include port"));
    }
}

Integration Tests

Create integration tests in the tests/ directory:

// tests/integration_test.rs
use caddy_rs::config::Config;
use std::time::Duration;
use tokio::time::timeout;

#[tokio::test]
async fn test_server_starts_and_responds() {
    let config = Config::default_with_ports(8090, 8091);
    let server = caddy_rs::server::Server::new(config).await.unwrap();
    
    // Start server in background
    let server_handle = tokio::spawn(async move {
        server.run().await
    });
    
    // Give server time to start
    tokio::time::sleep(Duration::from_millis(100)).await;
    
    // Test request
    let response = reqwest::get("http://localhost:8090/").await.unwrap();
    assert!(response.status().is_success());
    
    // Cleanup
    server_handle.abort();
}

Manual Testing

Create test configurations for different scenarios:

# Basic functionality test
cargo run -- --config example-config.json

# Test in another terminal
curl http://localhost:8080/
curl http://localhost:8081/

# Load testing
wrk -t12 -c400 -d30s http://localhost:8080/

Debugging

Logging

Use different log levels for debugging:

# Basic logging
RUST_LOG=info cargo run

# Detailed debugging
RUST_LOG=debug cargo run

# Very detailed (including dependencies)
RUST_LOG=trace cargo run

# Module-specific logging
RUST_LOG=caddy_rs::proxy=debug cargo run

Debugging with LLDB/GDB

# Build with debug symbols
cargo build

# Run with debugger
lldb target/debug/caddy-rs
(lldb) run -- --config example-config.json

Performance Profiling

# Install profiling tools
cargo install cargo-flamegraph

# Profile the application
cargo flamegraph --bin caddy-rs -- --config example-config.json

# This generates a flamegraph.svg file showing performance hotspots

Common Development Tasks

Adding a New Configuration Option

1. Update the config structures:

   // In src/config/mod.rs
   #[derive(Debug, Clone, Serialize, Deserialize)]
   pub struct Server {
       pub listen: Vec<String>,
       pub routes: Vec<Route>,
       #[serde(default)]
       pub my_new_option: bool,  // Add your option
   }
   

2. Handle the option in the relevant module:

   // In src/server/mod.rs or wherever appropriate
   if server_config.my_new_option {
       // Handle the new feature
   }
   

3. Add tests:

   #[test]
   fn test_my_new_option_parsing() {
       let config_json = r#"
       {
         "listen": [":8080"],
         "routes": [],
         "my_new_option": true
       }
       "#;
   
       let config: ServerConfig = serde_json::from_str(config_json).unwrap();
       assert_eq!(config.my_new_option, true);
   }
   

4. Update documentation:

   • Add to API documentation
   • Update README with examples
   • Add to example configurations

Adding a New Middleware

1. Implement the Middleware trait:

   // In src/middleware/mod.rs
   pub struct MyMiddleware {
       config: MyMiddlewareConfig,
   }
   
   #[async_trait]
   impl Middleware for MyMiddleware {
       async fn preprocess_request(
           &self,
           mut req: Request<Incoming>,
           remote_addr: SocketAddr,
       ) -> Result<Request<Incoming>> {
           // Modify request here
           Ok(req)
       }
   
       async fn postprocess_response(
           &self,
           mut resp: Response<BoxBody>,
           remote_addr: SocketAddr,
       ) -> Result<Response<BoxBody>> {
           // Modify response here
           Ok(resp)
       }
   }
   

2. Add to middleware chain:

   // In MiddlewareChain::new()
   Self {
       middlewares: vec![
           Box::new(LoggingMiddleware::new()),
           Box::new(CorsMiddleware::new()),
           Box::new(MyMiddleware::new(config)),  // Add here
       ],
   }
   

Adding a New Load Balancing Algorithm

1. Add to SelectionPolicy enum:

   // In src/config/mod.rs
   #[derive(Debug, Clone, Serialize, Deserialize)]
   #[serde(tag = "policy")]
   pub enum SelectionPolicy {
       // ... existing policies
   
       #[serde(rename = "my_algorithm")]
       MyAlgorithm { param1: u32 },
   }
   

2. Implement the algorithm:

   // In src/proxy/mod.rs LoadBalancer implementation
   match load_balancing.selection_policy {
       // ... existing algorithms
   
       SelectionPolicy::MyAlgorithm { param1 } => {
           let index = self.my_algorithm_selection(upstreams, *param1);
           Ok(&upstreams[index])
       }
   }
   

Release Process

Version Management

1. Update version in Cargo.toml:

   [package]
   name = "caddy-rs"
   version = "0.2.0"  # Update this
   

2. Update version in main.rs if displayed:

   let matches = Command::new("caddy-rs")
       .version("0.2.0")  # Update this
   

3. Tag the release:

   git tag v0.2.0
   git push origin v0.2.0
   

Pre-release Checklist

• All tests pass: cargo test
• Code is properly formatted: cargo fmt
• No clippy warnings: cargo clippy -- -D warnings
• Documentation is updated
• Example configurations work
• Performance hasn't regressed
• Security audit passes: cargo audit

Troubleshooting Common Issues

Compilation Errors

Error: Cannot move out of borrowed reference

// Problem:
let body = req.into_body(); // req is &Request

// Solution:
let (parts, body) = req.into_parts(); // Take ownership first

Error: Async trait object lifetime issues

// Problem:
Box<dyn Middleware>

// Solution:
Box<dyn Middleware + Send + Sync>

Runtime Issues

Server doesn't start:

• Check if port is already in use: lsof -i :8080
• Verify configuration file syntax: cargo run -- --config invalid.json
• Check log output for specific errors

High memory usage:

• Profile with: cargo build --release && valgrind ./target/release/caddy-rs
• Check for connection leaks in proxy module
• Monitor with: ps aux | grep caddy-rs

Poor performance:

• Enable release mode: cargo run --release
• Profile with flamegraph: cargo flamegraph
• Check async task spawning patterns
• Monitor with system tools: htop, iotop

Contributing Guidelines

Pull Request Process

1. Fork and create feature branch:

   git checkout -b feature/my-new-feature
   

2. Make changes with tests:

   • Add unit tests for new functionality
   • Add integration tests if needed
   • Update documentation

3. Ensure code quality:

   cargo test
   cargo clippy -- -D warnings
   cargo fmt
   

4. Submit pull request:

   • Clear description of changes
   • Reference any related issues
   • Include testing instructions

Code Review Criteria

• Functionality: Does the code work as intended?
• Performance: Is the implementation efficient?
• Safety: Does it follow Rust safety principles?
• Style: Does it follow project conventions?
• Documentation: Is new functionality documented?
• Tests: Are there appropriate tests?

This development guide should help you get started contributing to Caddy-RS. For questions or clarifications, please open an issue in the project repository.