Quantum/docs/development.md

# Development Guide

This guide covers everything you need to know to contribute to Caddy-RS development.

## Development Setup

### Prerequisites

- **Rust 1.75+** with 2024 edition support
- **Cargo** package manager
- **Git** for version control
- **Optional**: Docker for testing

### Environment Setup

1. **Install Rust via rustup:**
   ```bash
   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
   source ~/.cargo/env
   ```

2. **Install useful development tools:**
   ```bash
   cargo install cargo-watch    # Auto-reload during development
   cargo install cargo-edit     # Add/remove dependencies easily
   cargo install cargo-audit    # Security vulnerability scanning
   cargo install cargo-flamegraph  # Performance profiling
   ```

3. **Clone and setup the project:**
   ```bash
   git clone <repository-url>
   cd caddy-rs
   cargo build
   cargo test
   ```

## Project Structure

```
quantum/
├── Cargo.toml              # Dependencies and project metadata
├── README.md               # Main documentation
├── SIMPLE-CONFIG.md        # Simple configuration guide
├── QUICKSTART.md           # Quick start scenarios
├── example-config.json     # Example full configuration
├── examples/               # Simple configuration examples
│   ├── proxy-simple.json
│   ├── static-files.json
│   └── full-stack.json
├── public/                 # Test files for file server
│   └── index.html
├── src/                    # Source code
│   ├── main.rs            # Application entry point
│   ├── config/            # Configuration parsing
│   │   ├── mod.rs         # Full Caddy configuration format
│   │   └── simple.rs      # Simple configuration format
│   ├── server/            # HTTP server implementation
│   │   └── mod.rs
│   ├── proxy/             # Reverse proxy and load balancing
│   │   └── mod.rs
│   ├── middleware/        # Request/response middleware
│   │   └── mod.rs
│   ├── tls/               # TLS and certificate management
│   │   └── mod.rs
│   ├── metrics/           # Metrics and monitoring
│   │   └── mod.rs
│   └── file_sync/         # File synchronization system
├── docs/                   # Documentation
│   ├── architecture.md    # Architecture documentation
│   ├── api.md             # API and configuration reference
│   └── development.md     # This file
└── tests/                  # Integration tests (planned)
```

## Development Workflow

### Daily Development

1. **Start with tests:**
   ```bash
   cargo test
   ```

2. **Run with auto-reload during development:**
   ```bash
   cargo watch -x 'run -- --config example-config.json'
   ```

3. **Check code quality:**
   ```bash
   cargo clippy -- -D warnings  # Linting
   cargo fmt                     # Code formatting
   ```

4. **Test with different configurations:**
   ```bash
   cargo run -- --port 3000
   cargo run -- --config custom-config.json
   ```

## Configuration System

Quantum supports two configuration formats:

### Simple Configuration (`src/config/simple.rs`)

The simple configuration format is designed for ease of use:

```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SimpleConfig {
    #[serde(default)]
    pub proxy: HashMap<String, String>,
    #[serde(default)]
    pub static_files: HashMap<String, String>,
    #[serde(default)]
    pub file_sync: HashMap<String, String>,
    #[serde(default = "default_tls")]
    pub tls: String,
    pub admin_port: Option<String>,
}
```

**Key features:**
- **Auto-validation**: Comprehensive validation with helpful error messages
- **Auto-conversion**: Converts to full Caddy format internally
- **Port normalization**: Handles various port formats automatically
- **Error messages**: User-friendly validation with emojis and examples

### Full Configuration (`src/config/mod.rs`)

Full Caddy v2 compatibility for advanced features:
- Complex route matching
- Advanced load balancing
- Health checks
- Custom middleware
- Complex TLS automation

### Configuration Detection

The system automatically detects format in `Config::from_file()`:

```rust
// Try simple config first
match serde_json::from_str::<simple::SimpleConfig>(&content) {
    Ok(simple_config) => {
        println!("✅ Detected simple configuration format");
        return simple_config.to_caddy_config();
    }
    Err(simple_err) => {
        // Fall back to full format
        match serde_json::from_str::<Config>(&content) {
            Ok(config) => {
                println!("✅ Detected full Caddy configuration format");
                Ok(config)
            }
            // Provide helpful error message for both formats
        }
    }
}
```

### Adding New Features

1. **Plan the feature:**
   - Update documentation first (README, API docs)
   - Add configuration structures if needed
   - Plan the module interfaces
   - Consider if simple config support is needed

2. **Implement incrementally:**
   - Start with configuration parsing
   - Add simple config support if applicable
   - Add core logic
   - Implement tests
   - Add integration with existing modules

3. **Example: Adding a new handler type**
   ```rust
   // 1. Add to config/mod.rs
   #[derive(Debug, Clone, Serialize, Deserialize)]
   #[serde(tag = "handler")]
   pub enum Handler {
       // ... existing handlers
       
       #[serde(rename = "my_handler")]
       MyHandler {
           setting1: String,
           setting2: Option<u32>,
       },
   }
   
   // 2. Implement in proxy/mod.rs
   async fn handle_route(&self, req: Request<Incoming>, handler: &Handler) -> Result<Response<BoxBody>> {
       match handler {
           // ... existing handlers
           
           Handler::MyHandler { setting1, setting2 } => {
               self.handle_my_handler(req, setting1, setting2).await
           }
       }
   }
   
   // 3. Add the handler implementation
   async fn handle_my_handler(&self, req: Request<Incoming>, setting1: &str, setting2: &Option<u32>) -> Result<Response<BoxBody>> {
       // Implementation here
   }
   ```

### Code Style Guidelines

1. **Follow Rust conventions:**
   - Use `snake_case` for functions and variables
   - Use `PascalCase` for types and traits
   - Use `SCREAMING_SNAKE_CASE` for constants

2. **Error handling:**
   ```rust
   // Use Result types throughout
   pub async fn my_function() -> Result<String> {
       let value = some_operation().await?;
       Ok(value)
   }
   
   // Use anyhow for application errors
   use anyhow::{Result, Context};
   
   let config = load_config().context("Failed to load configuration")?;
   ```

3. **Async patterns:**
   ```rust
   // Use async/await consistently
   pub async fn handle_request(&self, req: Request) -> Result<Response> {
       let processed = self.middleware.process(req).await?;
       let response = self.upstream_client.request(processed).await?;
       Ok(response)
   }
   ```

4. **Documentation:**
   ```rust
   /// Handles reverse proxy requests to upstream servers.
   /// 
   /// This function selects an upstream server using the configured
   /// load balancing algorithm and proxies the request.
   /// 
   /// # Arguments
   /// 
   /// * `req` - The incoming HTTP request
   /// * `upstreams` - List of available upstream servers
   /// 
   /// # Returns
   /// 
   /// Returns the response from the upstream server or an error
   /// if all upstreams are unavailable.
   pub async fn proxy_request(
       &self,
       req: Request<Incoming>,
       upstreams: &[Upstream],
   ) -> Result<Response<BoxBody>> {
       // Implementation
   }
   ```

## Testing Strategy

Quantum includes comprehensive test coverage with **41 tests** across all modules.

### Current Test Coverage

**Core Tests (35 tests):**
- **Config module**: 17 tests covering configuration parsing, serialization, handlers, matchers
- **Proxy module**: 8 tests covering load balancing, upstream selection, content-type detection
- **Server module**: 8 tests covering address parsing, TLS detection, edge cases
- **Middleware module**: 4 tests covering CORS headers, middleware chain

**Simple Config Tests (6 tests):**
- Configuration validation and conversion
- Port normalization and error handling
- JSON serialization/deserialization
- Empty config handling with defaults

### Running Tests

```bash
# Run all tests
cargo test

# Run specific module tests
cargo test config
cargo test simple
cargo test proxy

# Run with output
cargo test -- --nocapture

# Run with detailed logging
RUST_LOG=debug cargo test
```

### Test Quality Standards

**Real Business Logic Testing:**
- ✅ **No stub tests** - All tests validate actual functionality
- ✅ **Genuine validation** - Tests parse real JSON, validate algorithms, check error paths
- ✅ **Edge case coverage** - IPv6 addresses, port ranges, empty configurations
- ✅ **Error path testing** - All validation errors have corresponding tests

**Example Real Test:**
```rust
#[tokio::test]
async fn test_config_serialization_deserialization() {
    let config_json = r#"{
        "admin": {"listen": ":2019"},
        "apps": {
            "http": {
                "servers": {
                    "test_server": {
                        "listen": [":8080"],
                        "routes": [{
                            "match": [{"matcher": "host", "hosts": ["example.com"]}],
                            "handle": [{
                                "handler": "reverse_proxy",
                                "upstreams": [{"dial": "backend:8080"}]
                            }]
                        }]
                    }
                }
            }
        }
    }"#;

    let config: Config = serde_json::from_str(config_json).unwrap();
    
    assert_eq!(config.admin.listen, Some(":2019".to_string()));
    assert!(config.apps.http.servers.contains_key("test_server"));
    
    let server = &config.apps.http.servers["test_server"];
    assert_eq!(server.listen, vec![":8080"]);
    
    // Validates complete JSON parsing pipeline
    if let Handler::ReverseProxy { upstreams, .. } = &server.routes[0].handle[0] {
        assert_eq!(upstreams[0].dial, "backend:8080");
    }
}
```

### Unit Tests

Place unit tests in the same file as the code they test:

```rust
#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_load_balancer_no_upstreams() {
        let lb = LoadBalancer::new();
        let upstreams: Vec<Upstream> = vec![];
        let load_balancing = LoadBalancing {
            selection_policy: SelectionPolicy::RoundRobin,
        };

        let result = lb.select_upstream(&upstreams, &load_balancing);
        assert!(result.is_err());
        assert!(result.unwrap_err().to_string().contains("No upstreams available"));
    }
    
    #[test]
    fn test_simple_config_validation() {
        let mut proxy = HashMap::new();
        proxy.insert("localhost".to_string(), ":8080".to_string()); // Missing port
        
        let config = SimpleConfig {
            proxy,
            static_files: HashMap::new(),
            file_sync: HashMap::new(),
            tls: "auto".to_string(),
            admin_port: None,
        };

        let result = config.validate();
        assert!(result.is_err());
        assert!(result.unwrap_err().to_string().contains("must include port"));
    }
}
```

### Integration Tests

Create integration tests in the `tests/` directory:

```rust
// tests/integration_test.rs
use caddy_rs::config::Config;
use std::time::Duration;
use tokio::time::timeout;

#[tokio::test]
async fn test_server_starts_and_responds() {
    let config = Config::default_with_ports(8090, 8091);
    let server = caddy_rs::server::Server::new(config).await.unwrap();
    
    // Start server in background
    let server_handle = tokio::spawn(async move {
        server.run().await
    });
    
    // Give server time to start
    tokio::time::sleep(Duration::from_millis(100)).await;
    
    // Test request
    let response = reqwest::get("http://localhost:8090/").await.unwrap();
    assert!(response.status().is_success());
    
    // Cleanup
    server_handle.abort();
}
```

### Manual Testing

Create test configurations for different scenarios:

```bash
# Basic functionality test
cargo run -- --config example-config.json

# Test in another terminal
curl http://localhost:8080/
curl http://localhost:8081/

# Load testing
wrk -t12 -c400 -d30s http://localhost:8080/
```

## Debugging

### Logging

Use different log levels for debugging:

```bash
# Basic logging
RUST_LOG=info cargo run

# Detailed debugging
RUST_LOG=debug cargo run

# Very detailed (including dependencies)
RUST_LOG=trace cargo run

# Module-specific logging
RUST_LOG=caddy_rs::proxy=debug cargo run
```

### Debugging with LLDB/GDB

```bash
# Build with debug symbols
cargo build

# Run with debugger
lldb target/debug/caddy-rs
(lldb) run -- --config example-config.json
```

### Performance Profiling

```bash
# Install profiling tools
cargo install cargo-flamegraph

# Profile the application
cargo flamegraph --bin caddy-rs -- --config example-config.json

# This generates a flamegraph.svg file showing performance hotspots
```

## Common Development Tasks

### Adding a New Configuration Option

1. **Update the config structures:**
   ```rust
   // In src/config/mod.rs
   #[derive(Debug, Clone, Serialize, Deserialize)]
   pub struct Server {
       pub listen: Vec<String>,
       pub routes: Vec<Route>,
       #[serde(default)]
       pub my_new_option: bool,  // Add your option
   }
   ```

2. **Handle the option in the relevant module:**
   ```rust
   // In src/server/mod.rs or wherever appropriate
   if server_config.my_new_option {
       // Handle the new feature
   }
   ```

3. **Add tests:**
   ```rust
   #[test]
   fn test_my_new_option_parsing() {
       let config_json = r#"
       {
         "listen": [":8080"],
         "routes": [],
         "my_new_option": true
       }
       "#;
       
       let config: ServerConfig = serde_json::from_str(config_json).unwrap();
       assert_eq!(config.my_new_option, true);
   }
   ```

4. **Update documentation:**
   - Add to API documentation
   - Update README with examples
   - Add to example configurations

### Adding a New Middleware

1. **Implement the Middleware trait:**
   ```rust
   // In src/middleware/mod.rs
   pub struct MyMiddleware {
       config: MyMiddlewareConfig,
   }
   
   #[async_trait]
   impl Middleware for MyMiddleware {
       async fn preprocess_request(
           &self,
           mut req: Request<Incoming>,
           remote_addr: SocketAddr,
       ) -> Result<Request<Incoming>> {
           // Modify request here
           Ok(req)
       }
   
       async fn postprocess_response(
           &self,
           mut resp: Response<BoxBody>,
           remote_addr: SocketAddr,
       ) -> Result<Response<BoxBody>> {
           // Modify response here
           Ok(resp)
       }
   }
   ```

2. **Add to middleware chain:**
   ```rust
   // In MiddlewareChain::new()
   Self {
       middlewares: vec![
           Box::new(LoggingMiddleware::new()),
           Box::new(CorsMiddleware::new()),
           Box::new(MyMiddleware::new(config)),  // Add here
       ],
   }
   ```

### Adding a New Load Balancing Algorithm

1. **Add to SelectionPolicy enum:**
   ```rust
   // In src/config/mod.rs
   #[derive(Debug, Clone, Serialize, Deserialize)]
   #[serde(tag = "policy")]
   pub enum SelectionPolicy {
       // ... existing policies
       
       #[serde(rename = "my_algorithm")]
       MyAlgorithm { param1: u32 },
   }
   ```

2. **Implement the algorithm:**
   ```rust
   // In src/proxy/mod.rs LoadBalancer implementation
   match load_balancing.selection_policy {
       // ... existing algorithms
       
       SelectionPolicy::MyAlgorithm { param1 } => {
           let index = self.my_algorithm_selection(upstreams, *param1);
           Ok(&upstreams[index])
       }
   }
   ```

## Release Process

### Version Management

1. **Update version in Cargo.toml:**
   ```toml
   [package]
   name = "caddy-rs"
   version = "0.2.0"  # Update this
   ```

2. **Update version in main.rs if displayed:**
   ```rust
   let matches = Command::new("caddy-rs")
       .version("0.2.0")  # Update this
   ```

3. **Tag the release:**
   ```bash
   git tag v0.2.0
   git push origin v0.2.0
   ```

### Pre-release Checklist

- [ ] All tests pass: `cargo test`
- [ ] Code is properly formatted: `cargo fmt`
- [ ] No clippy warnings: `cargo clippy -- -D warnings`
- [ ] Documentation is updated
- [ ] Example configurations work
- [ ] Performance hasn't regressed
- [ ] Security audit passes: `cargo audit`

## Troubleshooting Common Issues

### Compilation Errors

**Error: Cannot move out of borrowed reference**
```rust
// Problem:
let body = req.into_body(); // req is &Request

// Solution:
let (parts, body) = req.into_parts(); // Take ownership first
```

**Error: Async trait object lifetime issues**
```rust
// Problem:
Box<dyn Middleware>

// Solution:
Box<dyn Middleware + Send + Sync>
```

### Runtime Issues

**Server doesn't start:**
- Check if port is already in use: `lsof -i :8080`
- Verify configuration file syntax: `cargo run -- --config invalid.json`
- Check log output for specific errors

**High memory usage:**
- Profile with: `cargo build --release && valgrind ./target/release/caddy-rs`
- Check for connection leaks in proxy module
- Monitor with: `ps aux | grep caddy-rs`

**Poor performance:**
- Enable release mode: `cargo run --release`
- Profile with flamegraph: `cargo flamegraph`
- Check async task spawning patterns
- Monitor with system tools: `htop`, `iotop`

## Contributing Guidelines

### Pull Request Process

1. **Fork and create feature branch:**
   ```bash
   git checkout -b feature/my-new-feature
   ```

2. **Make changes with tests:**
   - Add unit tests for new functionality
   - Add integration tests if needed
   - Update documentation

3. **Ensure code quality:**
   ```bash
   cargo test
   cargo clippy -- -D warnings
   cargo fmt
   ```

4. **Submit pull request:**
   - Clear description of changes
   - Reference any related issues
   - Include testing instructions

### Code Review Criteria

- **Functionality**: Does the code work as intended?
- **Performance**: Is the implementation efficient?
- **Safety**: Does it follow Rust safety principles?
- **Style**: Does it follow project conventions?
- **Documentation**: Is new functionality documented?
- **Tests**: Are there appropriate tests?

This development guide should help you get started contributing to Caddy-RS. For questions or clarifications, please open an issue in the project repository.