livestream-archiver/README.md

Livestream Archiver

A Rust application that monitors a directory for livestream recordings, processes them with hardware-accelerated transcoding, and syncs them to a remote server.

Features

• Automatic Detection: Monitors a configured directory for new MP4 files
• Smart Processing: Waits for files to be completely written before processing
• Remote Sync: Uses rsync to sync files to a remote server immediately after detection
• Hardware Acceleration: Converts files to AV1 using Intel QSV hardware acceleration
• Organized Storage: Creates date-based directory structure (Jellyfin-compatible)
• Metadata Generation: Creates NFO files for media servers
• Retry Logic: Caches and retries failed sync operations
• Service Integration: Runs as a systemd service with automatic restart

Prerequisites

• Rust toolchain (1.70 or later)
• ffmpeg with Intel QSV support
• rsync for file synchronization
• SSH key authentication for passwordless rsync (if using remote sync)

Installation

Building from Source

# Clone the repository
git clone <repository-url>
cd livestream-archiver

# Build release binary
cargo build --release

# Binary will be at target/release/livestream_archiver

Quick Deploy

1. Copy and customize the example files:

cp livestream-archiver.service.example livestream-archiver.service
cp deploy.sh.example deploy.sh
# Edit both files with your specific configuration

2. Run the deployment:

./deploy.sh

Configuration

Environment Variables

• PC_SYNC_TARGET: Remote sync destination (e.g., user@server:/path/to/destination/)
• INPUT_DIR: Directory to monitor for new files
• OUTPUT_DIR: Local output directory for processed files

Service Configuration

1. Copy the example service file:

cp livestream-archiver.service.example livestream-archiver.service

2. Edit the service file with your configuration:

• Update User and Group
• Set correct paths for WorkingDirectory and ExecStart
• Configure the PC_SYNC_TARGET environment variable

3. Install and start the service:

sudo cp livestream-archiver.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable livestream-archiver
sudo systemctl start livestream-archiver

Usage

Running Manually

# Run directly
cargo run

# Or run the compiled binary
./target/release/livestream_archiver

Running as a Service

# Check service status
sudo systemctl status livestream-archiver

# View logs
sudo journalctl -u livestream-archiver -f

# Restart service
sudo systemctl restart livestream-archiver

How It Works

1. Monitoring: Continuously watches the input directory for new MP4 files
2. Validation: Waits for files to be completely written (stable size)
3. Sync: Immediately syncs new files to the remote server via rsync
4. Processing: Converts to AV1 format using hardware acceleration
5. Organization: Moves processed files to date-based directories
6. Metadata: Creates NFO files for media server compatibility
7. Cleanup: Optionally removes original files after successful processing

Directory Structure

Output files are organized by date:

<output-directory>/
├── 2024/
│   ├── 01-January/
│   │   ├── Recording - January 01 2024.mp4
│   │   ├── Recording - January 01 2024.nfo
│   │   └── ...
│   └── 02-February/
│       └── ...
└── 2025/
    └── ...

Troubleshooting

Service not starting

• Check logs: journalctl -u livestream-archiver -f
• Verify all paths exist and have proper permissions
• Ensure ffmpeg has QSV support: ffmpeg -hwaccels | grep qsv

Sync failures

• Verify SSH key authentication to remote server
• Check network connectivity
• Review cached sync attempts in ~/.cache/livestream-archiver/

Processing issues

• Ensure Intel QSV drivers are installed
• Check available disk space
• Verify file permissions in input/output directories

Development

Running Tests

cargo test

Building for Debug

cargo build

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.