livestream-archiver/README.md

# Livestream Archiver

A Rust application that monitors a directory for livestream recordings, processes them with hardware-accelerated transcoding, and syncs them to a remote server using an event-driven architecture.

## Features

- **Event-Driven Architecture**: Uses filesystem events for instant file detection
- **Automatic Detection**: Monitors a configured directory for new MP4 files  
- **Smart Processing**: Waits for files to be completely written before processing
- **Remote Sync**: Uses rsync to sync files to a remote server immediately after detection
- **Hardware Acceleration**: Converts files to AV1 using Intel QSV hardware acceleration
- **Organized Storage**: Creates date-based directory structure (Jellyfin-compatible)
- **Metadata Generation**: Creates NFO files for media servers
- **Retry Logic**: Caches and retries failed sync operations
- **Service Integration**: Runs as a systemd service with automatic restart
- **Environment Configuration**: Uses .env file for easy configuration management

## Prerequisites

- Rust toolchain (1.70 or later)
- `ffmpeg` with Intel QSV support
- `rsync` for file synchronization
- SSH key authentication for passwordless rsync (if using remote sync)

## Installation

### Building from Source

```bash
# Clone the repository
git clone <repository-url>
cd livestream-archiver

# Build release binary
cargo build --release

# Binary will be at target/release/livestream_archiver
```

### Quick Deploy

1. Copy and customize the example files:
```bash
cp .env.example .env
cp livestream-archiver.service.example livestream-archiver.service
cp deploy.sh.example deploy.sh
# Edit all files with your specific configuration
```

2. Run the deployment:
```bash
./deploy.sh
```

## Configuration

### Environment Variables (.env file)

Create a `.env` file in the project root with the following variables:

- `INPUT_DIR`: Directory to monitor for new MP4 files
- `OUTPUT_DIR`: Local output directory for processed files
- `PC_SYNC_TARGET`: Remote sync destination (e.g., `user@server:/path/to/destination/`)

Example `.env` file:
```bash
INPUT_DIR=/home/user/livestreams
OUTPUT_DIR=/media/archive/livestreams  
PC_SYNC_TARGET=user@server:/remote/path/to/livestreams/
```

### Service Configuration

1. Copy the example service file:
```bash
cp livestream-archiver.service.example livestream-archiver.service
```

2. Edit the service file with your configuration:
- Update `User` and `Group`
- Set correct paths for `WorkingDirectory` and `ExecStart`
- Ensure the service can access your `.env` file

3. Install and start the service:
```bash
sudo cp livestream-archiver.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable livestream-archiver
sudo systemctl start livestream-archiver
```

## Usage

### Running Manually

```bash
# Run directly (uses .env file automatically)
cargo run

# Or run the compiled binary
./target/release/livestream_archiver
```

### Running as a Service

```bash
# Check service status
sudo systemctl status livestream-archiver

# View logs
sudo journalctl -u livestream-archiver -f

# Restart service
sudo systemctl restart livestream-archiver
```

## How It Works

The application uses an event-driven architecture for efficient file monitoring:

1. **Event Monitoring**: Uses filesystem events to detect new files instantly (no polling)
2. **File Validation**: Waits for files to be completely written before processing
3. **Immediate Sync**: Syncs new files to remote server via rsync as soon as they're detected
4. **Hardware Processing**: Converts to AV1 format using Intel QSV hardware acceleration
5. **Smart Organization**: Moves processed files to date-based directories
6. **Metadata Generation**: Creates NFO files for media server compatibility
7. **Cleanup**: Optionally removes original files after successful processing

## Directory Structure

Output files are organized by date for easy browsing:
```
<output-directory>/
├── 2024/
│   ├── 01-January/
│   │   ├── Recording - January 01 2024.mp4
│   │   ├── Recording - January 01 2024.nfo
│   │   └── ...
│   └── 02-February/
│       └── ...
└── 2025/
    └── ...
```

## Troubleshooting

### Service not starting
- Check logs: `journalctl -u livestream-archiver -f`
- Verify `.env` file exists and has correct permissions
- Ensure all paths exist and have proper permissions
- Verify ffmpeg has QSV support: `ffmpeg -hwaccels | grep qsv`

### Sync failures
- Verify SSH key authentication to remote server
- Check network connectivity
- Test rsync manually: `rsync -av /local/path/ user@server:/remote/path/`
- Review cached sync attempts in `~/.cache/livestream-archiver/`

### Processing issues
- Ensure Intel QSV drivers are installed
- Check available disk space in both input and output directories
- Verify file permissions in input/output directories
- Test hardware acceleration: `ffmpeg -hwaccel qsv -i input.mp4 -c:v av1_qsv output.mp4`

### Files not detected
- Ensure the input directory exists and is accessible
- Check that the application has read permissions on the input directory
- Verify filesystem events are working (may not work on some network filesystems)

## Development

### Running Tests
```bash
cargo test
```

### Building for Debug
```bash
cargo build
```

### Development Environment
```bash
# Copy example env for development
cp .env.example .env
# Edit .env with your development paths

# Run with debug output
RUST_LOG=debug cargo run
```

## Performance Notes

- Uses filesystem events instead of polling for better performance
- Hardware acceleration significantly reduces CPU usage during transcoding
- Processes files as they arrive rather than batch processing
- Minimal memory footprint due to streaming processing approach

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.