Add comprehensive documentation with event-driven architecture details

README.md

@@ -1,10 +1,11 @@
 # Livestream Archiver
 
-A Rust application that monitors a directory for livestream recordings, processes them with hardware-accelerated transcoding, and syncs them to a remote server.
+A Rust application that monitors a directory for livestream recordings, processes them with hardware-accelerated transcoding, and syncs them to a remote server using an event-driven architecture.
 
 ## Features
 
-- **Automatic Detection**: Monitors a configured directory for new MP4 files
+- **Event-Driven Architecture**: Uses filesystem events for instant file detection
+- **Automatic Detection**: Monitors a configured directory for new MP4 files  
 - **Smart Processing**: Waits for files to be completely written before processing
 - **Remote Sync**: Uses rsync to sync files to a remote server immediately after detection
 - **Hardware Acceleration**: Converts files to AV1 using Intel QSV hardware acceleration
@@ -12,6 +13,7 @@ A Rust application that monitors a directory for livestream recordings, processe
 - **Metadata Generation**: Creates NFO files for media servers
 - **Retry Logic**: Caches and retries failed sync operations
 - **Service Integration**: Runs as a systemd service with automatic restart
+- **Environment Configuration**: Uses .env file for easy configuration management
 
 ## Prerequisites
 
@@ -39,9 +41,10 @@ cargo build --release
 
 1. Copy and customize the example files:
 ```bash
+cp .env.example .env
 cp livestream-archiver.service.example livestream-archiver.service
 cp deploy.sh.example deploy.sh
-# Edit both files with your specific configuration
+# Edit all files with your specific configuration
 ```
 
 2. Run the deployment:
@@ -51,12 +54,21 @@ cp deploy.sh.example deploy.sh
 
 ## Configuration
 
-### Environment Variables
+### Environment Variables (.env file)
 
-- `INPUT_DIR`: Directory to monitor for new MP4 files (default: `/home/user/livestreams`)
+Create a `.env` file in the project root with the following variables:
-- `OUTPUT_DIR`: Local output directory for processed files (default: `/media/archive/livestreams`)
+
+- `INPUT_DIR`: Directory to monitor for new MP4 files
+- `OUTPUT_DIR`: Local output directory for processed files
 - `PC_SYNC_TARGET`: Remote sync destination (e.g., `user@server:/path/to/destination/`)
 
+Example `.env` file:
+```bash
+INPUT_DIR=/home/user/livestreams
+OUTPUT_DIR=/media/archive/livestreams  
+PC_SYNC_TARGET=user@server:/remote/path/to/livestreams/
+```
+
 ### Service Configuration
 
 1. Copy the example service file:
@@ -67,7 +79,7 @@ cp livestream-archiver.service.example livestream-archiver.service
 2. Edit the service file with your configuration:
 - Update `User` and `Group`
 - Set correct paths for `WorkingDirectory` and `ExecStart`
-- Configure the `PC_SYNC_TARGET` environment variable
+- Ensure the service can access your `.env` file
 
 3. Install and start the service:
 ```bash
@@ -82,7 +94,7 @@ sudo systemctl start livestream-archiver
 ### Running Manually
 
 ```bash
-# Run directly
+# Run directly (uses .env file automatically)
 cargo run
 
 # Or run the compiled binary
@@ -104,17 +116,19 @@ sudo systemctl restart livestream-archiver
 
 ## How It Works
 
-1. **Monitoring**: Continuously watches the input directory for new MP4 files
+The application uses an event-driven architecture for efficient file monitoring:
-2. **Validation**: Waits for files to be completely written (stable size)
+
-3. **Sync**: Immediately syncs new files to the remote server via rsync
+1. **Event Monitoring**: Uses filesystem events to detect new files instantly (no polling)
-4. **Processing**: Converts to AV1 format using hardware acceleration
+2. **File Validation**: Waits for files to be completely written before processing
-5. **Organization**: Moves processed files to date-based directories
+3. **Immediate Sync**: Syncs new files to remote server via rsync as soon as they're detected
-6. **Metadata**: Creates NFO files for media server compatibility
+4. **Hardware Processing**: Converts to AV1 format using Intel QSV hardware acceleration
+5. **Smart Organization**: Moves processed files to date-based directories
+6. **Metadata Generation**: Creates NFO files for media server compatibility
 7. **Cleanup**: Optionally removes original files after successful processing
 
 ## Directory Structure
 
-Output files are organized by date:
+Output files are organized by date for easy browsing:
 ```
 <output-directory>/
 ├── 2024/
@@ -132,18 +146,26 @@ Output files are organized by date:
 
 ### Service not starting
 - Check logs: `journalctl -u livestream-archiver -f`
-- Verify all paths exist and have proper permissions
+- Verify `.env` file exists and has correct permissions
-- Ensure ffmpeg has QSV support: `ffmpeg -hwaccels | grep qsv`
+- Ensure all paths exist and have proper permissions
+- Verify ffmpeg has QSV support: `ffmpeg -hwaccels | grep qsv`
 
 ### Sync failures
 - Verify SSH key authentication to remote server
 - Check network connectivity
+- Test rsync manually: `rsync -av /local/path/ user@server:/remote/path/`
 - Review cached sync attempts in `~/.cache/livestream-archiver/`
 
 ### Processing issues
 - Ensure Intel QSV drivers are installed
-- Check available disk space
+- Check available disk space in both input and output directories
 - Verify file permissions in input/output directories
+- Test hardware acceleration: `ffmpeg -hwaccel qsv -i input.mp4 -c:v av1_qsv output.mp4`
+
+### Files not detected
+- Ensure the input directory exists and is accessible
+- Check that the application has read permissions on the input directory
+- Verify filesystem events are working (may not work on some network filesystems)
 
 ## Development
 
@@ -157,6 +179,23 @@ cargo test
 cargo build
 ```
 
+### Development Environment
+```bash
+# Copy example env for development
+cp .env.example .env
+# Edit .env with your development paths
+
+# Run with debug output
+RUST_LOG=debug cargo run
+```
+
+## Performance Notes
+
+- Uses filesystem events instead of polling for better performance
+- Hardware acceleration significantly reduces CPU usage during transcoding
+- Processes files as they arrive rather than batch processing
+- Minimal memory footprint due to streaming processing approach
+
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.